Re: undeletion at filesystem level or in extra filesystem?

[Bryan Wagstaff]

I'm confused about the 'saving the differences' issue.  Are we
considering delayed delete, automatic versioning, both, or something
else entirely?

On one side I hear that we are saving diferences (versioning) and on
the other I hear that we are only saving files opened with O_TRUNC,
which are completely erased.  I think the issue isn't as hard as it is
being made out to be.

-------------------
>> >closes, it would be nice if the old contents were cleanly saved as
a
>> >"version".
>> That could get really nasty when it comes to large files that are
>> opened/closed frequently. 
>"cleanly saved" might well include saving only differences.  
>I should have specified opening O_WRONLY|O_CREAT|O_TRUNC; 
>this is the normal modes for "writing a file", not writing log files

-------------------
>> My main concern is that the file update pattern open, write, close,
>> should be atomic when seen by other processes. I.e if some other
>> process opens the file for reading in the middle of the update, it
>> should see the previous version, independently of what the updating
>> process is doing.
>
>I would like this too, but it's contrary to Posix, which *guarantees*
>that other process *will* see the file in intermediate states, unless
>you use one single write call.  (And even then, they must be able to
>see the file in its truncated state if they check between the open
and
>the write.) 

[ I seem to remember that if a file was already opened, it couldn't be
opened again with O_TRUNC | O_CREAT, open would return EACCES because
the file is already being written to.  The excption of course was
shared access and file locking, but that's another issue. ]


So one the one hand we WANT to save differences, as one would do with
a versioning, but on the other hand we NEED to process the file in
Posix fasion, which allows for an undelete.  I thought this thread
decided earlier that delayed delete was the goal, and we were
examining cases for applications not written to use an undelete
library or API.  

As mentioned earlier by others, why not just have the file system move
files to some trash bin when a file is opened with O_TRUNC to get
undelete?  Insead of watching O_TRUNC it could do it for O_WRONLY or
O_RDRW to get versioning.  Someone earlier mentioned moving them to
/tmp, but it would have to be somewhere on the same device, perhaps
somewhere like lost+found.  If the file already exists in the trash
bin, make a second version or append a version number.  With that
minor difference in configuration, the filesystem could enable
undelete and/or file versioning with no problem just by checking the
open flags, as well as unlink and remove.

While it would always be nice to merge duplicate data blocks in files,
or to modify the inodes in a copy-on-write style, I think such tasks
are better suited for a disk cleaning daemon. Anyone interested in 
'save the differences' could use diff to find unchanged sections of
the file and merge inodes where appropriate, which would be good for a
background task since the system should obviously be busy with the app
that is changing the files in the first place.  But that 'save the
differences' daemon would be better off running filesystem wide rather
than trash bin wide, since the requirements would be the same and both
would result in improved usage (and more fragmentation...).

IMHO, A filesystem with undelete capability would be responsible for
making a 'version' of the file for undelete or versioning purposes by
simply moving the file to a trash bin when opening a file (as an
atomic operation within the file system, of course).  I believe that
would satisfy the Posix requirement because as far as the app is
concerned, the old file really is gone (it is elsewhere) and writes to
the file would be processed correctly.   If the filesystem needed to
reclaim space in the delayed delete area, then we would check the
config to see how to reclaim space.  Additional rules in the config
could specify a cap on file size to process, when to expire old files
(or to delete them on backup) and so on.

Am I totally off base?

Bryan Wagstaff
bryanw@xmission.com