[Dazuko-devel] 2.2.0-pre1 posted

[John Ogness]

Hi,

Summary:
A pre-release of 2.2.0 (pre1) has been posted. The reason for the sudden
jump from 2.1.1 to 2.2.0 is that an important bugfix in the internal
locking was found. This bug exists in every version of Dazuko until now.
The bug can occur if events are coming in faster than than registered
processes can work them off (thus being put on a waiting queue). In such
a situation, it is possible that a deadlock between two processes can
occur. Depending on the Dazuko Include Paths at that moment, this could
lead to the registered process freezing or in the case of "/", the
entire machine. For this reason, I want to make 2.2.0 the next stable
release as soon as possible and encourage everyone to upgrade.

Background:
As I've mentioned previously, I really wanted to get a pre-release of
DazukoFS posted for Dazuko's birthday. After successfully implementing
the basic components of DazukoFS, I began doing extensive system
testing. This was leading to frequent lock-ups. I ended up tracing the
problem to the Dazuko core (the main, cross-platform part).

The problem was an error in the way that the queues and slots were being
concurrently locked. While digging deeper, it also became clear to me
that the way the wait-queues were being used was very inefficient. I
then went through and did a complete analysis of all the states and
locking relationships within Dazuko. Then I localized and optimized the
wait queues so that they are only being used exactly when they are needed.

These changes have not only eliminated the deadlock problem, but have
improved performance by dramatically reducing unnecessary context
switches within the kernel.

However, there still seem to be an issue with Dazuko not treating all
processes equally at all times. This would be a problem with the
individual operating system implementations of Dazuko. There are no
deadlocks, but under situations where enormous amounts of events are
being generated by many processes (and few registered daemons), a type
of "round-robin batch processing" can be seen. I will need to
investigate this further and determine exactly which processes exhibit
this behavior.

DazukoFS:
So what about DazukoFS? Well I am happy to say that it is looking much
better than I expected. I have seen it work without problems on FreeBSD,
NetBSD, Linux 2.4, and Linux 2.6. I plan to get the code pushed onto the
CVS server soon, but want to get 2.2.0 out as soon as possible first.

Keep in mind that we are still at an early stage. Although it seems to
be running very well, there are still a lot of pieces missing. Once
pre-releases are available, I will need lots of help with testing.

I want to give out a special thanks to Gerhard Sittig, who spent many
hours working together with me to debug the locking problem with Dazuko,
help me learn about the inner-workings of VFS, and help with the
implementation of DazukoFS. Dazuko 3.0 (with DazukoFS) is really
starting to look good and I am excited about moving to VFS (and away
from syscall and LSM).

John Ogness

-- 
Dazuko Maintainer