Re: [Vrs-development] Distributed Filesystem

[Ian Fung]

> * Guaranteed consistency, synchronization, etc. -- in short it
> must
>
>       guarantee standard Unix filesystem semantics
> * Duplication of data -- each block of data must be duplicated on
>       multiple hosts, in case one or more hosts goes offline.
> * Splitting up of data -- it should be possible to require that a
>       file is never completely stored on a single host (or, more
>       generally, that no more than XX% of a file is ever stored
>       on a single host)
> * Encryption -- all data must be encrypted
> * Dynamically balanced -- hosts should be able to go on and
> offline at will; when hosts go offline the remain hosts should
> re- balance the distribution of the data blocks; when hosts come
> online the data must be synchronized.
> * Efficiency (a nicety) -- hosts should try to store the data
> they personally need, to cut down on network traffic, etc.

ok i hear you on the point that it is theoretically possible. the
point about the unreliability being extended from the hardware to
also include the network is a valid point. there is a difference
though. in the case with hardware, failures are not expected too
often and usually do not happen in mass quantities. for example in
a RAID, its fine if one harddrive fails, but you wouldnt expect
half the harddrives to. another thing is that LDS's are allowed and
expected to come and leave as they see fit. this makes things way
more complicated when trying to meet the requirements.

if you want to try to design some of the parts or even just
identify some of the issues to talk about when designing a
distributed file system, then i think i can help a lot. i was
recently involved in a project very similiar and have thought about
a lot of the issues surrounding the design of a distributed file
system. for example, you need to provide a way for a safe exit
where a LDS will push the info it has to other nodes for
redistribution. also there needs to be schemes for recovery when a
LDS fails. etcetc. i dunno where chris + bill stands in this
aspect, but maybe we should do an irc meeting sometime.

ian