Re: Mach emulation

[Farid Hajji]

g> This is essentially the reason why microkernels were considered a
g> failure in the early 90's. People tried to build u-kernels using
g> evolution from existing kernels. The results were Mach and other slow
g> systems. Liedtke showed that you need to start from scratch, as the
g> evolutionary approach won't converge against a minimal system.
g> 
g> It is my belief that an evolutionary approach to porting will produce
g> the same result.
Agreed. One more reason to consider reimplementing the Hurd from
scratch.

g> What I'm saying is that for the project to succeed, the following
g> approach should be taken:
g> 
g>  0) understand the L4 philosophy and API.
The libmom we're thinking about right now should provide a minimal
set of kernel abstractions. I already had in mind to keep libmom
as close to L4 as possible, without loosing the potential for
portability to other systems/[u]kernels. L4 seems already minimal
enough and the biggest part of libmom[-l4] should reflect the L4 ABI
as closely as possible (BTW, no clans involved here!).

g>  1) understand how the hurd API can be mapped onto the L4 API
g>     _directly_, without any reference to Mach
Sure, that is the key to performance. We'll have to do synchronouus
IPC and thread-paging in a very L4-ish way. This is the reason I'm
suggesting to drop the overloaded libports and use a minimal IPC
communication style between hurd servers. The L4-API should ultimately
be used by libmom-l4 (the l4 version of the planned libmom) rather
directly. If we can limit libmom (the upper half) basically to the
L4 features, implementing them in the bottom half (libmom-l4) would
be mostly a matter of inline code in header files (like what you can
see in L4 libutils implementations). The Hurd components would effectively
make direct L4 syscalls through libmom-l4.

It would still be useful to _not_ use a L4 API (is this really stable
already?) directly in the Hurd sources. Most of it should really be
hidden behind libmom calls, which could (for performance reasons) be
simple macros to L4-API(s) in libmom-l4. This way, we preserve
performance as well as portability to other systems/kernels.

g>  2) start implementing (reusing existing (Mach-based) code wherever
g>     possible, without compromising the design.
That is a difficult one. The current implementation of the Hurd is
intimately tied to mach syscalls and for its threadiness to the
mach-specific cthreads. Even worse is that an important part of
the Hurd (mostly the Unix emulation syscalls and RPC functionality)
is hidden in glibc (Ugh!). The portion of the Hurd code that is
independant of Mach syscalls, cthreads and those glibc sysdeps is
minimal and not worth mentioning.

If we want a small chance to keep some of the current Hurd code,
it is necessary to port either glibc to L4 (anyone with enough
glibc _and_ L4 expertise out there?) _or_ extract the hurd-ish
sysdeps from glibc into an independant library (I've already
suggested to boldly call it libhurd). This second approach
has the advantage of:
 1. not having to port glibc to L4 in the first place,
 2. not having to port glibc to another L4-supported
    architecture whenever L4 is ported,
 3. using libc from OSKit which provides (currently
    embryonic) pthreads support that we need,
 4. being able to port the hurd to libmom-{guest-os-with-posix-c-library}
    directly if possible.
Nothing prevents us from using glibc as well, but I strongly plead
to consider glibc as _one_ possible C library for the Hurd, not as
a _required_ C library.

Considering the small amount of potential code reuse right now, it
seems like a better idea to reimplement the Hurd from scratch as
already suggested.

g> I'd be keen to see it done. I'd be happy to help, mostly by
g> supervising/supporting local L4-literate student(s) working on
g> it. We're producing 1-2 dozen students a year who have written a small
g> OS on top of L4 (see http://www.cse.unsw.edu.au/~cs9242). I've
g> actually offered Hurd/L4 as a honours thesis topic here. So far no
g> bites, but doing it as part of a world-wide open-source project might
g> help attracting a good hacker.
Would you be willing to help us in this current critical stage of
redesign? We need a capable systems architect with L4 background
that is also willing to coordinate our efforts. No-one on l4-hurd
volunteered yet to manage the project (which is no surprise at this
early stage of brainstorming).

BTW, we're currently needing a L4-HOWTO. Your L4 User Manual is great,
but we also need a small collection of root_tasks (code) exercising
the L4-API. This would greatly help L4 novices on this list. Among others,
we need:
  * an intro to sigma0 (e.g. how to read-only map the kernel info page
    (clock!) into the root_task; how to exercise the sigma0 rpc protocol
    in general etc. How to remap memory on demand (vmremap())....),
  * an intro to rmgr (e.g. how to obtain resources from rmgr like interrupts,
    memory pages etc. What is the rmgr protocol and how can it be used?)
  * more explanations about thread scheduling and synchronization (e.g.
    how to suspend/resume a thread from outside [other prio? timeslice?
    external preempter (what that is, code?)?]? how to synchronize threads
    [on ipc_receive(), joining, condition variables?].
  * more code of the possible IPC calls (short, long messages, timeout
    receive etc...).
  * thoughts (and toy code!) on how to implement user-land device driver
    servers/tasks on top of L4.
Most of the above questions are already answered in real code in the L4Ka
and L4-Fiasco implementations (mostly libutils etc.), but Hurd people
new to L4 would have a hard time to sort it out. A small collection of
real root_tasks code (that could e.g. compile with L4Ka or L4-Fiasco)
showing single L4 features (and nothing more) would be much more
instructive here.

L4-code and introductory L4-HOWTO material could be published anywhere.
A Hurd/L4 HTML page is in preparation and a link to such a L4-HOWTO would
have to be added there.

Regards,

-Farid.

-- 
Farid Hajji -- Unix Systems and Network Admin | Phone: +49-2131-67-555
Broicherdorfstr. 83, D-41564 Kaarst, Germany  | address@hidden
- - - - - - - - - - - - - - - - - - - - - - - + - - - - - - - - - - - -
Murphy's Law fails only when you try to demonstrate it, and thus succeeds.