[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Dazuko-devel] Dazuko driver takes lots of cpu time, bottleneck disc
Re: [Dazuko-devel] Dazuko driver takes lots of cpu time, bottleneck discovered
Mon, 04 Oct 2004 20:42:16 +0200
Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.1) Gecko/20040808
Tikka, Sami wrote:
When I have dazuko driver loaded and our on-access-scanner running, I
frequently see my system go into a state where the system takes 50-90% of all
cpu time (reported by 'top').
This usually happens after accessing a lot of files, like hundreds of
thousands. (Compile XFree86 and glibc)
When I kill the on-access-scanner process, the system cpu usage goes down to
almost nothing, i.e. the normal situation.
During such a system-load situation I ran oprofile and this is what it
samples % app name symbol name
1606 56.0950 dazuko.o xp_compare_file
523 18.2676 dazuko.o dazuko_mark_hash_dirty
93 3.2483 dazuko.o dazuko_get_hash
[deleted the rest]
All of these functions have to do with the temporary file pointers that
are stored at ON_OPEN so that information can be returned at ON_CLOSE.
This is a linked list structure which was slightly optimized. However,
if many files are open simultaneously, it would still be very
inefficient. I would like to change the structure to be an n-ary tree.
These are simple to implement and are very fast.
However, I am more concerned that there is a "file" leak occurring. If
ON_OPEN events occur without a corresponding ON_CLOSE event, the local
file copy will remain allocated forever. This causes a gradual memory
leak as well as damaging the hash-lookup performance.
If you compile Dazuko with debug, Dazuko will print out all currently
allocated "open" files when Dazuko is being removed. If you compile
XFree86, glibc, mozilla, openoffice, etc and then afterwards kill the
scanner and unload Dazuko, do you see a huge list of "open" files?