[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Koha-zebra] Re: Import Speed
From: |
Mike Taylor |
Subject: |
[Koha-zebra] Re: Import Speed |
Date: |
Thu, 2 Mar 2006 16:29:32 +0000 (GMT) |
> Date: Thu, 02 Mar 2006 11:05:44 -0500
> From: Sebastian Hammer <address@hidden>
>
> Importing records one at a time when first building a database, or
> when doing a batch update that is a substantial percentage of the
> size of the database is not a good idea. The software has no way to
> optimize the layout of the index files, so for each record update,
> things get shuffled around, resulting on very sluggish update
> performance and a less-than-ideal layout inside the index files.
Sure, but ...
> It would be highly advisable to do at least the initial import from
> the command-line. I think it would make a lot of sense if this could
> be done well from the protocol, but AFAIK, the extended service
> interface at the moment only allows you to insert one record at a
> time.
But -- ?? What magic does the command-line import have access to that
ZOOM update doesn't? Clearly it's using some kind of in-memory
caching to hugely reduce the frequency of disk-writes, but why
shouldn't that also be used the doing a ZOOM update? Isn't that (part
of) the purpose of delaying the "commit" call? If not, then we need
to add $conn->option("updateCacheSize" => 100*1024*1024);
_/|_ ___________________________________________________________________
/o ) \/ Mike Taylor <address@hidden> http://www.miketaylor.org.uk
)_v__/\ "If I write in C++ I probably don't use even 10% of the language,
and in fact the other 90% I don't think I understand" -- Brian
W. Kernighan.