[Ifile-dev] Re: performance improvements

[Dave Marquardt]

On Thu, 30 Jan 2003 08:30:21 -0800, "Paul Chambers" <address@hidden> said:

Paul> Excuse me for being dense, but why the interest in moving the databases
Paul> between machines?

I'm thinking more of an environment of mixed machines where home
directories are NFS-mounted and you may run on a big-endian machine
one day and a little-endian machine another day.  The file doesn't
move, but ifile does.

Paul> I would rather see performance optimization for the machine it is
Paul> running on, with a tool to export/import the native format to an
Paul> interchange format if and when the database needed to be moved across
Paul> architectures. Don't pay a constant performance penalty for a relatively
Paul> infrequent operation.

That seems like a good idea too.  Currently we have a format that is
machine independent, human readable, and must be parsed, which makes
it slow.  For those who never run on different machine types, it would
be nice to have an option for a machine-dependent file that needn't be
parsed, i.e. just read it or mmap it and manipulate it.  A third
option would be a quickly parsed, machine independent database.

Paul> Paul

>> -----Original Message-----
>> From: address@hidden 
>> [mailto:address@hidden On Behalf 
>> Of Kai Großjohann
>> Sent: Thursday, January 30, 2003 7:49 AM
>> To: address@hidden
>> Subject: [Ifile-dev] Re: performance improvements
>> 
>> Dave Marquardt <address@hidden> writes:
>> 
>> > On Wed, 29 Jan 2003 21:15:32 +0100, 
>> > address@hidden (Kai Großjohann) said:
>> >
>> > Kai> Dave Marquardt <address@hidden> writes:
>> >>> I guess another option would be to invent some sort of 
>> >>> portable binary database format, but that's pretty far
>> >>> down on my list right now.
>> >
>> > Kai> I guess that most machines can convert to/from network 
>> > Kai> byte order.
>> >
>> > Not exactly a high performance operation, though.  We could even use
>> > XDR, but again, not a high performance operation.
>> 
>> I don't know what ifile stores, but if it is anything like inverted
>> files, another idea would be to use the inverted file compression
>> algorithms from the Managing Gigabytes folks.  I think these produce
>> machine-independent files, but if not it could probably fairly easily
>> be achieved.
>> 
>> The idea is that the additional computation spent on compression is
>> compensated for by the saved I/O.
>> -- 
>> Ambibibentists unite!
>> 
>> 
>> 
>> _______________________________________________
>> Ifile-dev mailing list
>> address@hidden
>> http://mail.nongnu.org/mailman/listinfo/ifile-dev
>> 



Paul> _______________________________________________
Paul> Ifile-dev mailing list
Paul> address@hidden
Paul> http://mail.nongnu.org/mailman/listinfo/ifile-dev


-- 
Dave Marquardt
Round Rock, TX