[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Vrs-development] Distributed Filesystem
From: |
Chris Smith |
Subject: |
Re: [Vrs-development] Distributed Filesystem |
Date: |
Fri, 12 Jul 2002 15:23:33 +0100 |
On Friday 12 July 2002 15:28, Ian Fung wrote:
> ok i hear you on the point that it is theoretically possible. the
> point about the unreliability being extended from the hardware to
> also include the network is a valid point. there is a difference
> though. in the case with hardware, failures are not expected too
> often and usually do not happen in mass quantities. for example in
> a RAID, its fine if one harddrive fails, but you wouldnt expect
> half the harddrives to. another thing is that LDS's are allowed and
> expected to come and leave as they see fit. this makes things way
> more complicated when trying to meet the requirements.
Exactly.
I really haven't got my head around how the propogation of data segments is
going to work. However I do feel strongly that it should be transparent to
the application layer (the LDS itself) and handled by the RM layer directly.
[Which brings in a previous email of mine which stated that there is a little
crossover between the RM and the CM as the CM needs to keep the RM updated as
to what LDS's are currently active in the cluster (VRS). This is just
messaging in the end, the CM dispatches an event to the RM to say that
something has changed and supplies it with enough info to sort out the
Resource image.
I'm tackling everything from a messaging and abstraction point of view as I
see it as the key to breaking the problem up into managable sub-projects].
> if you want to try to design some of the parts or even
> identify some of the issues to talk about when designing a
> distributed file system, then i think i can help a lot. i was
> recently involved in a project very similiar and have thought about
> a lot of the issues surrounding the design of a distributed file
> system. for example, you need to provide a way for a safe exit
> where a LDS will push the info it has to other nodes for
> redistribution.
This should happen when an LDS joins the cluster so that it's data is
immediately (or quickly) distributed across the cluster. Any LDS in the
cluster can then access the data. (Some data/resources served by an LDS may
be private and thus will not be shared - and so if that LDS goes away
(failure/network outage etc) then those resources will be unavailable).
> ... also there needs to be schemes for recovery when a
> LDS fails. etcetc. i dunno where chris + bill stands in this
> aspect, but maybe we should do an irc meeting sometime.
If an LDS's (public) data/resourcdes are shared with the cluster on joining,
then a failure will not be fatal as far as the VRS is concerned as there will
be copies of the resoureces elsewhere. The resource manager will take care
of locating and retrieving the data. The Cluster manager keeps the RM up to
date as to what LDS's are available.
As the retrieval of data is done through messaging, Goldwater simplifies the
whole thing by performing namespace and data dependant routing of request
messages to a suitable LDS.
IRC at some point would be good.
--
Chris Smith
Technical Architect - netFluid Technology Ltd.
"Internet Technologies, Distributed Systems and Tuxedo Consultancy"
E: address@hidden W: http://www.nfluid.co.uk
- [Vrs-development] Distributed Filesystem, Ian Fung, 2002/07/10
- Re: [Vrs-development] Distributed Filesystem, Eric Altendorf, 2002/07/11
- Re: [Vrs-development] Distributed Filesystem, Ian Fung, 2002/07/12
- Re: [Vrs-development] Distributed Filesystem,
Chris Smith <=
- Re: [Vrs-development] Distributed Filesystem, Eric Altendorf, 2002/07/12
- Re: [Vrs-development] Distributed Filesystem, Ian Fung, 2002/07/12
- Re: [Vrs-development] Distributed Filesystem, Eric Altendorf, 2002/07/12
- Re: [Vrs-development] Distributed Filesystem, Ian Fung, 2002/07/12
- Re: [Vrs-development] Distributed Filesystem, Eric Altendorf, 2002/07/12
- [Vrs-development] Re: IRC, Chris Smith, 2002/07/14