Re: Namespace-based translator selection; project details

[olafBuddenhagen]

Hi,

On Fri, May 30, 2008 at 11:40:09PM +0300, Sergiu Ivanov wrote:
> On Fri, May 30, 2008 at 6:32 AM, <olafBuddenhagen@gmx.net> wrote:

> > > However, the matter about ignoring some translators ('-u', for
> > > instance) is a bit foggy for me. Do you mean that these should
> > > also be symbolic links?

Looking at it again, I might have misread your question.

"-u" is more tricky that "-gunzip", as it doesn't skip one specific
translator, but rather filters a whole class. While it is certainly
possible to implement this also by means of a generic filter translator
with some more complex rules, it might be easier to use a hardcoded
implementation, making "-u" a real translator on its own...

"-gunzip" however is trivial, and can be easily implemented as a symlink
to a generic filter translator, or as a simple launcher script invoking
the filter translator.

> > That would be one of the implementation methods I suggested: Links
> > to a common filter translator. The script variant might be more
> > elegant, though...
> 
> 
> I'm very sorry, but I feel completely lost :-( As far as I can get,
> the '-u' filter should show the file/directory on which it is applied
> without any archive translators.

Right.

> Does it mean that, when the node 'file,,-u' is accessed, the proxy
> filesystem should generate a virtual node and stack upon it all
> translators which precede the 'u' translator?

Well, that's more or less the effect from the client's point of view.

Technically, it's a bit different: It should actually create a virtual
node that mirrors the original "file" node, and attach the "-u"
translator to this proxy node. "-u" will in turn present the client with
another virtual node, which corresponds to the original "file",
translated by all static translators present on the original node,
except for those that need to be skipped according to the filter rules.

As "-u" must be able to filter possibly present "gunzip" or "bunzip2" or
similar static translators attached to "file", "-u" probably needs to be
attached to a mirror of the *untranslated* "file" node. "-u" then will,
when handling client requests, have to explicitely query for any static
translators attached to "file", and follow them, unless it finds some
that needs to be skipped.

> I'm trying to say the following thing: suppose there are the following
> translators on 'file': x,u,y,z. When I ask for 'file,,-u' should I get
> the file 'file' with only 'x' set upon it?

Yes, probably. It makes most sense I think. (Keeping "y" and "z" while
skipping "u" is in fact impossible technically I believe, as "y" is
attached to the root node of "u". Skipping "u" actually means skipping
"u" and anything on top of it...)

Note that what you in fact get is a proxy node mirroring "file", with
only "-u" attached to it; when you open that however, "-u" will see that
the original node has static translators attached to it, and gives you
the "file" translated by "x" (while skipping the "u" etc.). But I
already wrote that above... :-)

> By the way, could you please suggest an idea where I can find out how
> to list the entries of a stack of translators?

Check the implementation of open() in glibc. This function normally
follows all translators, or none (when using O_NOTRANS); but it should
give you an idea how translator traversal works, and allow you to come
up with a more selective implementation...

> Is showtrans with its single file_get_translator call relevant in this
> case?..

"showtrans" only lists passive translators. "fsysopts" is used to query
active ones. Checking its implementation should also be helpful.

> Actually, I should start with the question: should the proxy create
> its own stacks of translators or just stack active translators?..

The dynamic translators specified through the special file name suffixes
should be attached (as active translators) to the proxy node.

The proxy node itself, as explained in the provious mail, probably
sometimes needs to mirror the original node ignoring any (static)
translators attached to it, so things like "-gunzip" can explicitely
check the static translators, and follow them selectively; while in
other cases (like "gunzip"), it needs to mirror the node already
translated by any static translators present. ("gunzip" knows nothing
about the proxying, and thus unlike the "-gunzip" won't follow the
static translators explicitely.)

> I think fetching the list of translators will be needed when the proxy
> will be told to ignore some of the static translators.

Yes, that's my understanding as well.

> When you say scripts, do you mean bash scripts or other scripting
> languages?

I was actually thinking of shell scripts here. Other languages could be
used as well, but probably would be overkill for a simple launcher :-)

Of course, instead of a simple launcher, you could implement some more
complicated scripts, if it seems useful...

> What really leaves me confused is how the interaction between the
> proxy filesystem and the script should take place...

Well, the point is that it doesn't really matter whether "-gunzip" is
implemented as an actual translator, or as a symlink to an actual
translator, or as a shell script that exec-s an actual translator...

Or is your question more general: How the interaction between the main
namespace proxy and the special translators like "-gunzip" or "-u"
works?

> > And if not, when we stack a '-gunzip' translator upon a 'gunzip'
> > > translator, don't we get into redundancy somehow?
> >
> > I don't see any redundancy...
> 
> 
> I didn't suppose that translators could be _stacked_ or _executed_. As
> far as I can get, executing a translator implies that it is not
> stacked anywhere, doesn't it?

No idea what you mean :-(

> I was thinking about the stack containing a '-gunzip' upon 'gunzip'...

Yeah, I understood that. And I don't see how this is redundant.

"gunzip" is a static translator, which would normally present the
underlying node in gunzipped form. "-gunzip" is a special dynamic
translator that skips the static translator, thus presenting the
untranslated contents of the node. Nothing redundant about that...

> > The example shows a real problem though: For "-gunzip" applied this
> > way to take any effect, the "-gunzip" translator must be executed
> > *before* opening the underlying node -- before any static
> > translators that might be present on the node get traversed. When
> > applying a "normal" translator like gunzip on the other hand, it
> > should be simply be stacked on top of anything that might already be
> > present, just like stacking static translators. Not sure about the
> > best behaviour when applying a magic translator like "u" or "-u"
> > recursively -- in this case, I'm not sure it's necessary to apply it
> > before opening the node; but on the other hand it shouldn't hurt to
> > do so...
> 
> 
> Am I right when I infer from these words of yours that there is no way
> to ignore a translator which has been traversed already?

Good question actually...

Every translator knows the node it is attached to. I don't know though
whether there is any standard RPC that would allow to query this
information. If it turns out there is, this would actually simplify
things a lot -- "-gunzip" could just ask for the original (untranslated)
node under the static "gunzip" translator; no need for special handling
to make sure "-gunzip" gets active before following static translators
:-)

> If the proxy filesystem will create its own translator stacks (as I
> have also supposed above), static translator stacks will remain
> unchanged, right?

Yes. The static translators are never touched. They might be ignored
when presenting the virtual nodes, but that doesn't affect their
presence on the real, underlying filesystem.

> And here I completely fail to imagine the mechanism by which the proxy
> will allow ignoring static translators...

That's trivial. To skip *all* translators present on a node, just use
O_NOTRANS. Skipping selectively requires a bit more work, as there is no
standard glibc function allowing for that (AFAIK); but I'm pretty sure
it can be done using the same underlying mechanisms.

> I'm really sorry if my questions look silly, [...]

They don't. In fact, discussing those questions has already greatly
improved my own understanding of these matters :-)

-antrik-