[Dvdrtools-users] Re: Backup of a partition with compression

[Bryan J. Smith <address@hidden>]

From: Ekkehard Kraemer <address@hidden>
> Hello,
> I would like to backup a partition (/dev/hdaX, not the file system) to a DVD.
> The partition is too large to fit on a single DVD, but with compression (gzip)
> it fits. It is then still larger than 2GB (close to 4GB).

First off, I would recommend you use "[e2fs]dump" (or "xfsdump" if you have XFS)
instead of "dd" to backup a partition.  Either off-line the partition (which 
you have
to do with "dd" anyway), or use snapshots to make a safe dump.

Secondly, I don't recommend compression of the whole archive.  Doing so puts
the _entire_ backup at risk.  Consider using afio to do per-file compression 
inside
the archive.  afio is cpio (USTAR 5KB blocks, GNU Tar/Star are USTAR 10KB 
blocks,
Star is fully POSIX 1003-2001 compliant, GNU Tar is working on that in beta) 
with
per-file compression.  Afio uses LZ77 aka gzip by default.  I perfer to use LZO 
for
real-time compression.

For a comparison of LZ77 (Gzip), BWT (Bzip2) and LZO (Lzop), see:  
http://www.samag.com/documents/s=7033/sam0204c/sam0204c_s1.htm  

If you want an ISO image, alternatively you can use my back2cd script.  It
was specifically designed for CD/DVD backup -- including the fact that making
an ISO file is actually an archive in itself (so any use of another archiving
program is like a "double archive" and not only redundant, but causes a lot
of access/recoverability issues).  For more, see 2002 April Sys Admin:  
  http://www.samag.com/documents/sam0204c/  

Although the article is a bit dated (it was written over 3.5 years ago, 
published
over 3 years ago), it is still great for single DVD backup as well as CD.  This 
is
assuming, of course, that you want to make an ISO file for CD-R or DVD-R.

> I cannot use temporary disk space to store the partition before burning it.

You could use an external hard drive for this, or a server mount.
But I understand your delima, so my "back2cd" script is not optimal in your 
case.
You want to directly write to the disk.

I've been doing such in Linux with DVD-RAM since 1998.
Prior to kernel 2.4, it was mounting a SCSI disk.
As of kernel 2.4, DVD-RAM drives are mounted read/write just like they were
a DVD-ROM drive, directly with the block device.

I typically just "find|cpio" files over directly, or use [xfs]dump/star.
But you mentioned you need compression, so consider "afio."
Although the stable version of afio has some 2GiB limitations, you can
tell afio to make multiple files -- i.e., file.afio.001, file.afio.002, etc...
They are standalone files that do not need the other files when you are
recovering (which makes it ideal for multiple volume tapes, CDs, etc...).

> I would like to use DVD+RW media on a Plextor PX-716A.
> It is perfectly fine if I can get at the data from Linux only;
> compatibility to some other OS is not an issue.

In using DVD-RAM for so many years included with the stock kernel, after
initial frustration trying to get the DVD+RW patches to work with the kernel,
I have little experience there.  From my understanding, I thought you didn't
need to use growisofs just to directly access the DVD+RW disk _if_ you
use Ext2 or UDF as the filesystem (?).

I.e., it's more of a filesystem consideration than a drive one.  You only need
growisofs if you are going to dynamically grow/shrink ISO9660 filesystems as
it was never designed to be a random-write block filesystem.  UDF is designed
to be both a pre-mastered (like ISO9660) _and_ random-write block filesystem
(like any common OS filesystem).  Hence why it is recommended.

As such, can you not directly write to a DVD+RW disk by merely mounting
the device rw as long as the filesystem is UDF, Ext2, etc...?  E.g.,  
  mount -t rw /dev/hdc /mnt/dvdrw  

> Now I'm at a loss. I tried using the streaming mode of growisofs/mkisofs;
> that works insofar as I don't get error messages, but the file on the DVD
> cannot be read later (as it is > 2 GB, I assume).

The >2GiB issue for POSIX32 is a continuing issue, even though POSIX 1003-2001
defines Large File Support (LFS) for such systems as Linux/x86 (POSIX32).
That is a complex issue and I'd have to see your issues/error messages first
hand to determine what the issue issue.

As of Linux 2.4 / GLibC 2.2, Linux is fully POSIX32+LFS compliant.  Of course,
individual programs and code might not take advantage of it, and have to be
recoded even if they support POSIX64 ("64-bit clean") to understand to test
for POSIX32+LFS support.

But I'm not sure this is your issue.

> I played around with dvdrecord, but didn't get far (piping stuff into 
> dvdrecord,
> it didn't seem able to work with media that had already been written to).

_Never_ use MO media (CD-RW, DVD-RAM, DVD-RW, DVD+RW) with cd/dvdrecord.
It is not recommended, even though cd/dvdrecord may be capable.
dvdrecord typically doesn't support DVD+RW drive firmware/media anyway.

Unlike the kernel block drivers that control the drive for rewriting, 
cdrecord/dvdrecord
are character streaming/command programs that directly control the drive.
Hence why compatibility is so limited.

> Can I use growisofs to supply my own stream of data without an ISO FS being
> created? I.e., can I somehow access the "builtin_dd" of growisofs, which seems
> perfectly capable to work with my combination of drive and media? Or is there
> some other way?

Do you absolute need an ISO9660 filesystem?
I would consider using UDF (if multi-OS compatibility is needed) or Ext2 (if 
not) instead.
If you use UDF, Ext2, etc..., you should be able to directly access the rw 
media in
the device without having to use another program.

At least that's how we do it with DVD-RAM.
My experience with DVD+RW has been more limited, but I've seen the patches in 
action
and I was under the impression that it works the same, you can use the "mount"
command to mount the media and use it just like a regular disk.
The only difference between DVD-RAM and DVD+RW is that DVD-RAM comes 
pre-formatted
(and has a physical pre-format) as UDF, and DVD+RW disks need to be formatted.

-- Bryan

P.S.  If anyone is wondering, the majority of my experience for the last 5+ 
years has been:  
- DVD-R with ISO9660 for Record via character device c/o 
cdrecord-ProDVD/cdrecord+DVDpatches
- DVD-RAM with UDF for Rewrite via direct kernel block access


--
Bryan J. Smith   mailto:address@hidden