Re: [rdiff-backup-users] Resource Forks and metadata file

[Daniel Hazelbaker]

    After 9 hours of it still trying to parse the metadata file and still
getting to the actual backup stage I finally killed it.

    A quick look at resource fork data size shows 1.2GB of actual RF data.
Doing some greps on that about 200MB of that is old OS 9 stuff. Doing some
sorting the top 500 files of 93,000 comprise about another 300MB of RF data
(most of these are things like Carbon games, apps, etc. not actual "data"
files). My rough average, figuring 800MB of useful RF data, divided by the
number of files on the server gives me approx 8K of RF data per file. Which
is probably fine for the average person trying to back up their 2GB home
directory. Which is possibly why I never noticed it in my testing.
Certainly, though, when you multiply that by 250 times, it slows down a tad.
:) What might take 2 minutes of parsing metadata data would now take over 8
hours.

    Most Microsoft files (Word, Excel, Powerpoint) seem to have about a 236
byte resource fork each. Powerpoint files have about a 7K RF fork. Pagemaker
has none, font files that do not come with OS X (OS 9 or third party fonts)
store all the font data in the resource fork, so that can be anywhere from
20K to 200K. We have a few thousand fonts so I am sure that is a four chunk
of the data too.

    Ben, unless you can/want to whip out a patch to change the RF storing
style to another method (you mentioned some ways that ACL's are done), could
you point me in the direction of a few functions I can start looking at for
doing that, as well as your thoughts on the best method to store the RF
data?

    I'll see if I can resource a little bit about the HFS+ file system this
weekend to see what other relevant data it stores. I know creator/type are
only 4 bytes each, but I am not sure what other data might be in there. If
it is all small it would be fine to put in the metadata, otherwise it might
be better to incorporate it with the RF data somehow.

    I wonder, would there be another option to use, like bdb or something
like that which might make more sense? Thinking out loud...

Daniel Hazelbaker
High Desert Church