Re: Add option for forcing a 64bit hash on 32bit systems

[Kevin Atkinson]

On Tue, 22 Dec 2020, Érico Nogueira wrote:

> As seen in [1], aspell has a rather unfortunate design limitation in its
> compiled dictionary format. This not only makes it impossible to share
> these files between systems with different endianness, but, if compiled
> with default settings, makes sharing impossible between 32bit and 64bit
> systems as well.

Aspell compiled dictionary formats are not really meant to be portable. 
Compiling a dictionary is now very fast, it is even faster if checks are 
disabled.  One thing I will be open to is the creation of a portable text 
based format which can be compiled very quickly on startup.  This will take 
some refactoring though to make it work.

> The only compatibility option offered is forcing 32bit hashes for all 
> systems, which makes 64bit systems incapable of reading 64bit dictionaries. 
> Since most systems nowadays (especially ones where aspell would be used) are 
> 64bit, it feels kind of backwards to force 32bit dicts. Would it be 
> possible, then, to force 64bit dicts on 32bit systems? Would a contribution 
> that adds this possibility be accepted?

To be clear the only thing this does is change the type the hash function 
used from size_t to u32int in modules/speller/default/readonly_ws.cpp:
  #ifdef USE_32_BIT_HASH_FUN
  typedef u32int hash_int_t;
  #else
  typedef size_t hash_int_t;
  #endif
All integers used in the dictionary are 32 bit as 64 bit integers will be an 
overkill.  The fact that a 64 bit hash function is used on a 32 bit integer is 
an oversight.  I would rather that a 32 bit hash function is used on all 
systems.  The only reason that option exists is to avoid breaking dictionary 
compatibility on 64bit systems.  I am open to enabling 32 bit hashes by 
default on the next major version bump.

Note that on Debian Aspell is compiled with --enable-32-bit-hash-fun.

Kevin