[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Help-tar] help request on tar archiving to lto5 tape

From: Jakob Bohm
Subject: Re: [Help-tar] help request on tar archiving to lto5 tape
Date: Thu, 17 Sep 2015 23:42:42 +0200
User-agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0

On 17/09/2015 08:55, Balazs Balint wrote:
Dear Help-Tar list members,

I am asking for your advices about a tape archiving issue we encountered when using tar with a Dell PV-124T autoloader (enclosed tape drive is a Quantum Ultrium 5). We usually archive up to 8 Terabytes of data with mixed-size files ( from several bytes to 30 GB/file) using the command:

tar --create --multi-volume --listed-incremental=/data/\!Archive/26Aug2015-raids.copy2.txt --file=/dev/st0 4Archive/

When doing so, writing to tape is always as fast as expected (80-120 MBytes/s)

However, upon reading back the archive we get two different options

1., some of the tapes are very fast to extract tar from: extraction write speeds top around 120 MBytes/s as expected, no problem

2., for other tapes, tar gives the following warning: "tar: Record size 1 block". In addition, reading from the tar or extracting from the tar is amazingly slow (~8MBytes/s) for these tapes

What I could read so far in this subject, the situation has to do with the --blocking-factor which is supposed to be 20 by default. Unfortunately, I did not explicitly set such a value when creating tars in the first place and it seems that the automatically selected value is not always OK. From now on every new tar archiving shall be carried out with the explicit call of --blocking-factor-200 that seems to work fine.

However, we have a decent amount of tapes that turned out to be only slow-readable. Is there a way to speed up data read from those tapes where tar was created with an automatically chosen blocking factor of 1?

From the tar manual I tried command

tar --verbose --extract --read-full-records --blocking-factor=300 --file /dev/nst0

but yet the readout speed tops at 8MB/s.
when trying
dd if=/dev/nst0 of=./tape_image the throughput is very slow as well.

Software environment and version data:

System: Ubuntu 14.04.1 LTS
mt-st: 1.1-5ubuntu1
tar:     1.27.1-1

Can you please advice how could I read back data fast back from tar written in 1-block records to LTO5 tapes?
Do what I do:

Use the buffer program (from the buffer package) to do
high-speed tape reads to a large circular buffer (I
usually use about 1GB), then pipe the result to tar
as a stdin archive.  Remember to specify the "buffer"
option to only start reading from tap when at least
80% (or more) of the memory buffer is free.  The same
technique is good at tape write time, but with the
"only start writing when 80% full" option.

(Exact option names depend on the exact buffer release,
I even have my own private fork).

This has the following benefits:

- The tape drive and driver doesn't have to wait for
 the tar program to process each tape record.

- If (as is often the case), tar cannot keep up with
 the tape speed, the tape only has to pause, rewind
 and pick itself up about once per gigabyte rather
 than all the time.  This saves a lot of wear and
 tear ("shoeshining") and also avoids the slowdown
 waiting for this wasteful process.

- If the backup is being (de)compressed (bzip2, lzop
 etc.), the buffering will even out the wait for the
 variable speed of the (de)compression.


Jakob Bohm, CIO, Partner, WiseMo A/S.
Transformervej 29, 2860 Søborg, Denmark.  Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded

reply via email to

[Prev in Thread] Current Thread [Next in Thread]