[Help-tar] Reproducibility of tar archives

[Yann E. MORIN]

Hello All,

Recent versions of tar have slightly changed the format of archives.
Most notably:

  - 1.27 changed gnu long link headers for path elements > 100
    characters

  - 1.30 changed --numeric-owner for filenames > 100 characters

In Buildroot, we are using hashes of archives to ensure reproducibility
of the source code we build. We also generate tarballs for licensing
compliance. In both cases, we use hashes for those archives.

The two changes above mean that we have to restrict the tar versions we
accept to a small subset. All the hashes we have so far have been made
over the years, and they all use the format that was generated by
versions 1.27 to 1.29. As distributions are updated, they all switch to
1.30 or later, we have to then always build our own version of tar.

Currently, we envision three paths:

  - keep the status quo: this is not nice, because we would always have
    to build our own tar going forward, for every builds;

  - switch to an alternate archive format: this is not nice, because
    people are used to tarballs, and the alternatives are not all
    reproducible either; those that are repriducible are much less
    known, or practical to use, than tarballs;

  - bite the bullet, and redo all the hashes with the newer tar format:
    in the future every one will have a newer tar, and so we won't have
    o build our own every time.

That last point is what we would prefer, if we could be sure that there
would be no change in the output format in the foreseeable future.

So, here's my question: starting with tar-1.32 (the latest release as of
today), is the gnu tar format considered stable now, or is there no
guarantee about the gnu tar format stability?

For reference, here's how we generate the archives:

    tar cf - \
        --numeric-owner --owner=0 --group=0 --mtime="${date}" \
        --format=gnu -T "list.sorted" >"${output}.tar"

Can we expect this to be reproducible with future tar releases?

Thank you!

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'