Re: Getting Started with Hurd-L4

[Sam Mason]

  [ Hope this isn't too much of a mess of thoughts, I've been writing
    and learning at the same time; with the result that this email has
    been rewritten several times as I've learnt more. . . ]

Marcus Brinkmann wrote:
>Both questions are central to the Hurd design decisions.  You should
>find out quickly that your simple idea will neither of them, and
>something more complex is needed.

I'm of the opinion that you can model very complex abstractions on top
of my simple idea - from POSIX strangeness to the sorts of algorithms
that would be needed to route data efficiently between the modules in a
micro-kernel based operating system.

I'm interested to hear how your "Containers" work first though.  It
sounds as though you've spent quite a bit of time thinking about their
design.

>Still, one
>way to catch up and challenge us at the same time is to ask questions
>about the design (like, why do you need to do this thing here and why
>is that thing there necessary at this place).  The more specific your
>questions are, the better.

I've been trying to figure out how these container things work.  For a
basic operation, I think the idea is that the client process (what do
you call these things? I basically mean a running program/server/
module) would ask physmem for a new container of the appropriate size;
it would then give this container to the server module, who would put
the data into it, or get the data out; the server would then give the
container back to the client; the client would then probably have to
dump the container that was used for the transaction (I'm assuming
that there will be some memory pressure here).

Assuming I've got all that right, there will be quite a few trips
through the kernel involved.  For a basic file system operation
there's going to be, at very least, three processes involved in
getting a block to disk - the client, file-system and device driver.
If these are all going to have to have an extra journey into physmem
to move the data around we're looking at least 10 context switches
between processes before we've even done anything interesting.

 context
switches  operation
 elapsed
       2  get a container initialised to the correct pages
       3  send it to file system
       5  get data out of container and reformat it in a suitable
          format for the device driver
       6  send it to the device driver
       8  get data out of container and write it to disk
       9  send response back to file system
      10  send response back to client

If we're expecting any sort of state back (I think this will happen
quite a lot as well) then most of the above process would have to be
repeated - that's a whole lot of cache and TLB thrashing going on!  If
this is a multi-CPU system, I hope that the IPC will only happen
between single CPUs - otherwise you're going to be getting a lot of
contention between the CPUs caches as well.  The reason for limiting
the operation to a single CPU is that the operation is fundamentally
serial in nature, so moving work across to other CPUs is just going
to give them lots of unnecessary work to do.  If things actually end
up blocking, however, like if we start having to wait for a block
to come back from the disk, then we can start taking over work from
other processors.  But if the general course of action is to send
things to the least used processor things are going to get very slow -
witness all the the stuff that's been happening in Linux recently with
trying to get processes to "stick" on one processor.  I've put quite a
lot of thought into moving these sorts of decisions back out of the
kernel, but that's for another discussion!

Anyway, hope that's specific enough!  As I said, I've never actually
written one of these things before, so it's conjecture that any of
these things will actually be a problem.

>>Why have you gone for a micro-kernel based design? :)
>
>Because we believe in the benefits of it,
>and nobody else seems to be doing it.

Sorry, that comment was made in jest (hence the smiley)!  I totally
agree with you that a micro-kernel based design is the way to go.

Cheers,
  Sam