Re: Native compilation: the bird-eye view

[Andrea Corallo]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I've updated how we generate the eln filenames and now they are in the
>> form: filename-path_hash-content_hash.eln
>>
>> where path_hash is computed using the absoulte path of the source file
>> and content_hash using its content.
>
> [ Note: it's not a "path" but a "file name", according the GNU's
>   conventions.  ]
>
> Hmm... this results in too-long file names IMO, and the absolute file
> name can change "gratuitously", making those eln files suddenly unusable.

I shorted also the hash length for this reason, I think should be very
unlikely to get collisions anyway.  This is a file I've here as an example:

advice-505b05f39e8e7db0ef9610bf48b6298e-7c08dcc023e8926affda38e4071cba52.eln

>> This should enable us for keepeng the eln-cache folders clean and solve
>> the issue mentioned in my previous message.
>
> I don't have a clear understanding of the problem this is trying to solve.
> Could you summarize it?

Sure,

we need a rule to go from the source file to the eln filename.

/xxx/foo.el -> foo-something1.eln
/yyy/foo.el -> foo-something2.eln

Given we store all eln "flatted" in a single directory indeed we need to
have "something1" different from "something2".

As a first approach I went for using just the abs filename as input to
the hashing algo (with some tweaking for the eln being installed by make
install).

Unfortunatelly dlopen may return the same handle on two subsequent loads
if the filename is the same, even if the first file has been renamed or
removed.  This make recompiling and reloading a file uneffective.  Here
comes in the requirement of generating a different filename when
recompiling.

Facing that I came up with the idea of using the last modificaiton date
as additional input to the hash, modifying the source then generates a
different eln filename and everything works.  Unfortunatelly make
install is using tar to zip the source file and the last modificaiton
date is not preserved precisely.  Aside from that I perceived the last
modification date is a bit weak for this scope.

I then came-up with the current idea of using the source content as
input of the hash, this should be robust.

At the moment we are using 2 separate hashes to make the cleaning (easy)
possible, this is a separate problem that's correct.  The idea is that
with the two hashes when recompiling is easy to just remove the old file
as you can identify using the first hash the old one.  Additionally to
that, at any point in time, if you have two file sharing the first hash
you know that all but the most recent can be removed.

> If you're worried about how to delete old .eln files, I think there are
> two problems: identifying which .elc files still exist and identifying
> which Emacs binaries still exist.
>
> But the first problem can arguably be solved by scanning the `load-path`,
> collecting the .elc files we find there (and kind of GC algorithm
> where `load-path` acts as the roots).

I think this may be problematic as we could have two different Emacs
with two different load-path that are sharing the eln-cache folders.
This may be a bit of patological case tho.

If we want to solve the cleaning in some other way we could certanly
remove the hash computed from the filename.

Thanks

  Andrea

--
akrl@sdf.org