What to do about nar storage for bordeaux.guix.gnu.org?

[Christopher Baines]

Hey!

The bordeaux build farm and substitute server has been operating for
around 2 years now, and during that time the storage for the nars has
changed a few times.

At first they were stored on bayfront, but when it was low on space
there was a switch to storing them on a different machine (lakeside),
and caching some proportion on bayfront. Then lakeside had a hardware
failure, and it was replaced by bishan.

There's now around 10TiB of nars, and bishan only has 11TiB of storage,
so it's running rather low on free space.

The lakeside machine and it's replacement bishan are both machines I
rented from Hetzner. While it would be possible to replace bishan with
another rented machine from Hetzner, I'd like to stop personally renting
machines for this purpose, so this is an option I'd like to avoid.

In terms of what could replace bishan, it just needs more than 11TiB of
storage, and good upload speed to the internet.

Trying to reduce the nars stored could be part of the solution here, I
already want to try removing nars that aren't helpful to store (that
relate to derivations that never made it to master), but more could be
removed. However, one of my aims with the bordeaux build farm was to
improve on the substitute availability, not only for recent revisions
but also older ones. The nars that are removed could be useful to keep
around, it's hard to tell what might be useful to users in the
future. Ideally we'll have more nars for new architectures at some point
as well, so along with the fact that guix is generally growing, it would
be good to grow the available storage, rather than trying to do more
with the same amount of space.

With the last two similar emails I sent out about infrastructure
planning, I just included guix-sysadmin, but I thought I'd cast a wider
net and include guix-devel in this one.

Let me know if you have any thoughts, suggestions or questions!

Thanks,

Chris

signature.asc
Description: PGP signature