Re: [Vrs-development] Distributed Filesystem

[Eric Altendorf]

On Wednesday 10 July 2002 20:47, Ian Fung wrote:
> first question is whether the vrs is going to house a distributed
> file *system*? or a distributed file *repository*?

What they want is a distributed file *system*.

> distributed file repository doesnt have the same properties as a
> dfs. a file system needs to be consistent and sychronized.

The goals are as follows:

* Guaranteed consistency, synchronization, etc. -- in short it must
        guarantee standard Unix filesystem semantics
* Duplication of data -- each block of data must be duplicated on
        multiple hosts, in case one or more hosts goes offline.
* Splitting up of data -- it should be possible to require that a
        file is never completely stored on a single host (or, more
        generally, that no more than XX% of a file is ever stored
        on a single host)
* Encryption -- all data must be encrypted
* Dynamically balanced -- hosts should be able to go on and offline
        at will; when hosts go offline the remain hosts should re-
        balance the distribution of the data blocks; when hosts come
        online the data must be synchronized.
* Efficiency (a nicety) -- hosts should try to store the data they
        personally need, to cut down on network traffic, etc.

> it is impossible to create a file system by the
> traditional definition in a distributed environment. what we can do
> is try to maintain consistency and sychronization as best we can.

Not exactly.  It is possible to guarantee consistency and
synchronization.  It is possible to guarantee standard Unix
filesystem semantics despite the highly dynamic and unreliable
hardware base.  It's just very difficult (nobody else has ever done
this, to my knowledge -- the closest I think that currently exists
would be distributed database servers), and may result in very poor
performance in some situations.

All hardware is intrinsically unreliable; the science of writing
reliable software on unreliable hardware is the science of
transaction processing.  The proposed distributed filesystem is
possible; but I think it is probably too ambitious at this stage of
the project.

That's my take on it....

Eric

--
"First they ignore you.  Then they laugh at you.
 Then they fight you.  And then you win."             -Gandhi