Re: [Help-tar] help request on tar archiving to lto5 tape

[Jakob Bohm]

On 17/09/2015 08:55, Balazs Balint wrote:
> Dear Help-Tar list members,
>
> I am asking for your advices about a tape archiving issue we 
> encountered when using tar with a Dell PV-124T autoloader (enclosed 
> tape drive is a Quantum Ultrium 5).
> We usually archive up to 8 Terabytes of data with mixed-size files ( 
> from several bytes to 30 GB/file) using the command:
>
> tar --create --multi-volume 
> --listed-incremental=/data/\!Archive/26Aug2015-raids.copy2.txt 
> --file=/dev/st0 4Archive/
>
> When doing so, writing to tape is always as fast as expected (80-120 
> MBytes/s)
>
> However, upon reading back the archive we get two different options
>
> 1., some of the tapes are very fast to extract tar from: extraction 
> write speeds top around 120 MBytes/s as expected, no problem
>
> 2., for other tapes, tar gives the following warning: "tar: Record 
> size 1 block". In addition, reading from the tar or extracting from 
> the tar is amazingly slow (~8MBytes/s) for these tapes
>
> What I could read so far in this subject, the situation has to do with 
> the --blocking-factor which is supposed to be 20 by default. 
> Unfortunately, I did not explicitly set such a value when creating 
> tars in the first place and it seems that the automatically selected 
> value is not always OK. From now on every new tar archiving shall be 
> carried out with the explicit call of --blocking-factor-200 that seems 
> to work fine.
>
> However, we have a decent amount of tapes that turned out to be only 
> slow-readable. Is there a way to speed up data read from those tapes 
> where tar was created with an automatically chosen blocking factor of 1?
>
> From the tar manual I tried command
>
> tar --verbose --extract --read-full-records --blocking-factor=300 
> --file /dev/nst0
>
> but yet the readout speed tops at 8MB/s.
> when trying
> dd if=/dev/nst0 of=./tape_image the throughput is very slow as well.
>
> Software environment and version data:
>
> System: Ubuntu 14.04.1 LTS
> mt-st: 1.1-5ubuntu1
> tar:     1.27.1-1
>
> Can you please advice how could I read back data fast back from tar 
> written in 1-block records to LTO5 tapes?
Do what I do:

Use the buffer program (from the buffer package) to do
high-speed tape reads to a large circular buffer (I
usually use about 1GB), then pipe the result to tar
as a stdin archive.  Remember to specify the "buffer"
option to only start reading from tap when at least
80% (or more) of the memory buffer is free.  The same
technique is good at tape write time, but with the
"only start writing when 80% full" option.

(Exact option names depend on the exact buffer release,
I even have my own private fork).

This has the following benefits:

- The tape drive and driver doesn't have to wait for
 the tar program to process each tape record.

- If (as is often the case), tar cannot keep up with
 the tape speed, the tape only has to pause, rewind
 and pick itself up about once per gigabyte rather
 than all the time.  This saves a lot of wear and
 tear ("shoeshining") and also avoids the slowdown
 waiting for this wasteful process.

- If the backup is being (de)compressed (bzip2, lzop
 etc.), the buffering will even out the wait for the
 variable speed of the (de)compression.

Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S.  https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark.  Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded