Re: Reliability of RPC services

[Marcus Brinkmann]

Hi,

I am stating it up front: I have no intention of convincing you that
send-once rights are necessary.  One reason is that I do not know if
they are necessary.  Another reason is that I don't have time to make
a formally strong argument in their favor.

This mail is just intended to correct a couple of obvious
misunderstandings, and to ask for a clarification.

At Sat, 22 Apr 2006 10:22:25 -0400,
"Jonathan S. Shapiro" <address@hidden> wrote:
> 1. The behavior that you want can *only* be accomplished with reference
> counting.

This is not true.  A send-once capability does not require reference
counting as only ever one reference is extant (guaranteed by move vs
copy semantics).

> 5. Note that reference counting doesn't really solve the problem either.
> A server could simply store an extra copy of the reply descriptor and
> forget about it for a very long time. The existence of this descriptor
> is sufficient to prevent the "last drop" message from being sent for an
> indefinite time.

It depends on what you perceive as the problem.  I seem to have a much
narrower definition of the problem in mind than you.

It may be that the problem I am trying to solve is a non-problem.  It
may also be that the solutions I am looking for are not sufficient to
construct the systems I am trying to construct.  Both are interesting
issues, and I am looking at them as well.  However, my suspicion is
that without any such mechanisms, no useful, persistent, fault
tolerant, user-extensible multi-server system can be built (for some
definition of useful, which includes running a browser and sharing
files with friends).

> 6. You will soon generalize this to RcvQ send capabilities, but in that
> case the problem is unsolvable because the message will commonly be
> delayed (therefore lost).

I have no intention to do so.
 
> > I think the following
> > condition should be sufficient: The kernel guarantees that a reply
> > message is sent _at the latest_ when the callee process is destroyed.
> > This should hold true independent of what the callee does between
> > being invoked and exiting.  In particular, simply dropping the reply
> > capability should not change this guarantee (which in effect means
> > that the kernel has to invoke the reply capability when it is
> > dropped).
> 
> Several problems:
> 
> 1. This requires dynamic storage allocation in the kernel. Dynamic
> storage allocation in the kernel implies denial of resource
> vulnerabilities and makes any statement of kernel robustness impossible.

Can you elaborate why you think that dynamic storage allocation is
required?

> 2. Your description fails in the case where C calls S which forwards to
> T, because the exit of S will cause an improper reply.

My description (above) does not even include this case.  You snipped
the following:

"Going back to the simple case of one caller and one callee, where the
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 callee does not invoke any other process, I think the following
 condition should be sufficient: The kernel guarantees that a reply..."

If you combine this description with the idea of send-once
capabilities, which are moved, not copied, you get a description which
works with forwarding.
 
> 3. Your proposal seems to have the side effect (I am not certain) of
> dictating a hierarchical calling relationship. This is bad.

I am not sure what that is supposed to mean.

> > In short: I think overwriting a capability and destroying it should
> > behave the same with regards to this issue.
> 
> So let me see if I have this right: you want to sent a death notice on
> overwrite.

I want an RPC mechanism that's "better" than UDP.

> So this implies that every IPC must check the destination slots to see
> if they cause such an overwrite, and must issue death notice calls on
> those capabilities. If the IPC payload contains up to N capabilities,
> and we assume that the death notice itself does not transfer a
> capability, then every IPC has just been multiplied by up to N IPCs.

The common case is that no capability that is overwritten is a valid
reply capability.  The common case is thus N checks on memory that is
going to become hot anyway.

> This just won't work, Marcus. Technically it can be done, but the
> resulting system will perform much worse than Mach.

> > I am happy to defer discussion of implementation details, but I would
> > like to clarify the issue of (accidentially, maliciously) dropped
> > reply capabilities.
> 
> I believe that I have offered a compelling argument for why it should
> not be done: in order to solve a very rare problem, your "highly robust"
> solution imposes a 400% overhead on the common case operation!

In fact, I didn't find the arguments compelling.  What I would find a
complling argument is one of the following: (1) A strategy to
implement a useful multi-server operating system which has an RPC
service that is as reliable as UDP (for definition of useful, see
above ;) (2) A different strategy to make RPC mechanisms more
reliable.
 
> I am inclined to think that disk GC provides a better approach to the
> whole problem, but I haven't really thought about it enough to have a
> sensible opinion about this.

Disk GC may prove to be a good alternative.

Thanks,
Marcus