Re: [Chicken-users] faster threading

[F. Wittenberger]

Am Dienstag, den 28.10.2008, 09:33 +0100 schrieb felix winkelmann:
> Thanks for the patch, Jörg. I'm impressed how deeply you hack the
> scheduler.

Thank you for writing a clean code to begin with making this an easy way
to walk on!

> Using a smarter data-structure for the fd-list makes sense,
> even though I'd like to avoid adding yet another core library unit (there
> are already too many), but perhaps a scaled down version could be
> included, exclusively for the scheduler (and thus easier to tune).
> I have to review this change thoroughly, and apologize in advance that
> it'll take some time.

If you agree to use smarter data structures in the scheduler, I'll take
my time to expand my proposal.

My remarks:

* I'd think twice before I strip down some code to tune it without end.
When you take a look at the wttree.scm I posted, you'll find a
cond-expanded section implementing the tree nodes as sloted structures.
Sure I tried it before I switched it off: there was no visible gain.
You might likely expend a lot of effort for very little.  Chicken seems
to do quite a lot for you.  Trust it! ;-)
* I prefer the srfi-9 declaration at least for the development time.
* Just this code is very well tested.  I used it to implement a data
base, which takes XSD-Element declarations as row descriptions and some
additional elements holding Scheme/SXPath expressions to declare indexes
and accumulator functions.  This db has is currently being used to
develop a specialised issue tracking system - i.e., on a daily basis.
Here is the db logic: http://askemos2.tc-mw.de/jfw/bin/xsdbd.xml
* Do you really want to maintain two similar implementations of the same
code?  Besides, it will result in code duplication in those executables,
which need a full version anyway.
* I plan to expand the code anyway.  Using cond-expand and macros, the
idea is to compile a version without any overhead (as what you would get
it with a striped down version), one for a binary balanced tree with the
virtualization included and one which takes a configurable number of
values per node -- because my persistent node store has a fixed overhead
per node, which is high in comparison to reading one more value.
* Maybe we should make it possible to decide on configure time, which
implementation to use.  Simplistic (linear list), minimal (no overhead
and unused code) and complete for those who would need the full module
anyway.
* I would to change the interface the scheduler uses to manipulate the
fd-list to read like fd-list:ref fd-list:ref/default fd-list:delete!
where ref, ref/default etc. have the same signature as in srfi-69.  (And
have those interfaces exported from the smarter data structure.)  Then a
simple  "(import (prefix my-great-structure fd-list:))" will do all the
work.
* At the end of the day, we might take a minute to consider the question
which data structure would be the best.  I took the wttree only because
a) it has approximately the complexity I desire, b) I know how well
tested this file is, so I've been pretty sure that there will be no
surprises I could try endlessly to attribute to the scheduler while the
bug sits in the data structure and c) chickens hash tables are not
atomic enough to be used in the scheduler plus I recall postings about
them being slow.  But if we arrange for a switch at import time, we
shall have an easy route to try a few algorithms without ever touching
the schedulers real code again.

To sum up: may I apply for a svn branch, say "jfw" to ease my
development.  I shall notify you when I'm satisfied with my
modifications.


Best Regards

/Jörg