Re: booting btrfs

[Michael Chang]

2013/12/21 Andrey Borzenkov <address@hidden>:
> В Fri, 20 Dec 2013 21:38:47 -0700
> Chris Murphy <address@hidden> пишет:
>
>>
>> On Dec 20, 2013, at 7:54 AM, Michael Chang <address@hidden> wrote:
>>
>> >> Every volume has a name, even if you don't know it. Use grub-mkrelpath
>> >> to find out.
>> >
>> > That means we need to modify the grub,cfg in snapshot to make files
>> > used by config refer to new path output by grub-mkrelpath (relative to
>> > real root), right ? That could be difficult to manage especially if
>> > you have a lot of snapshots and the update takes time to finish.
>>
>> This isn't just a problem for core.img looking for the wrong grub.cfg for a 
>> /boot on Btrfs subvolume snapshot. It's a problem for that grub.cfg pointing 
>> to the wrong root snapshot. And it's a problem for the /etc/fstab on that 
>> root snapshot, which is likewise incorrect and will be asking for the wrong 
>> subvolumes to be mounted.
>>
>> I really don't think snapshot management is GRUB's job. I think all of this 
>> snapshot management is a userspace tool, and it's going to have to figure 
>> out a way to deal with this.
>
> Yes I completely agree here. Expecting to be able to boot from pure
> btrfs snapshot is naïve. Michael, here is what openSUSE does by default
> when you tell it to use btrfs:
>
> linux-dwhw:~ # btrfs subvolume list /
> ID 256 gen 47 top level 5 path boot/grub2/i386-pc
> ID 258 gen 18 top level 5 path home
> ID 259 gen 18 top level 5 path opt
> ID 260 gen 18 top level 5 path srv
> ID 261 gen 65 top level 5 path tmp
> ID 262 gen 52 top level 5 path usr/local
> ID 263 gen 18 top level 5 path var/crash
> ID 264 gen 67 top level 5 path var/log
> ID 265 gen 18 top level 5 path var/opt
> ID 266 gen 62 top level 5 path var/spool
> ID 267 gen 57 top level 5 path var/tmp
> ID 269 gen 59 top level 5 path .snapshots
> ID 270 gen 58 top level 5 path .snapshots/1/snapshot
> ID 271 gen 78 top level 5 path test
> linux-dwhw:~ # grep btrfs /proc/self/mountinfo
> 21 1 0:17 / / rw,relatime shared:1 - btrfs /dev/sda2 rw,space_cache
>
> "test" is snapshot of / which I set as default and am currently booted
> with it.
>
> linux-dwhw:~ # btrfs subvolume get-default /
> ID 271 gen 78 top level 5 path test
>
> And if I now try to access any other subvolumes ...
>
> linux-dwhw:~ # ls -l /boot/grub2/i386-pc/
> total 0
> linux-dwhw:~ # touch /boot/grub2/i386-pc/x
> touch: cannot touch ‘/boot/grub2/i386-pc/x’: Permission denied
> linux-dwhw:~ # ls -l /var/spool
> total 0
> linux-dwhw:~ # touch /var/spool/x
> touch: cannot touch ‘/var/spool/x’: Permission denied
> linux-dwhw:~ # ls -l /var/log
> total 0
> linux-dwhw:~ # touch /var/log/x
> touch: cannot touch ‘/var/log/x’: Permission denied
>
> So booting from this snapshot is rather useless.
>
> The point here is - creating of fully functional alternate boot
> environment involves a bit more than single "btrfs subvolume snapshot"
> invocation. Adding "grub-mkconfig" (or even grub-mkimage to record
> correct prefix) is really just the minor part of it.

Well .. yes, I've agreed on that system has to work to support it, not
just bootloader.

>
>> And probably the simplest solution in the short term is for this user
>> space tool to rename the subvolumes. So e.g. subvolumes:
>>
>> boot
>> root
>> home
>>
>> And their read only snapshots:
>>
>> boot_ro.1
>> boot_ro.2
>> root_ro.1
>> root_ro.2
>> home_ro.1
>> home_ro.2
>>
>> The user uses a tool to indicate they now want to boot "the most recent 
>> snapshot", and the tool does:
>>
>> mv boot boot_ro.0
>> mv root root_ro.0
>> mv home home_ro.0
>> btrfs subvol snapshot boot_ro.1 boot
>> btrfs subvol snapshot root_ro.1 root
>> btrfs subvol snapshot home_ro.1 root
>>
>
> Do you need to reinvent the wheel?
>
> /Boot-Environments
>   /Boot_Environment_1
>     /root
>     /boot
>     ...
>   /Boot_Environment_2
>     ...
>
> The only thing you need to do to switch is equivalent of "btrfs
> set-default /Boot-Environments/Boot_Envirnment_2 ... except it is
> not that straightforward in current btrfs because path names are
> resolved relative to current root :)
>
>> The lack of -r makes the snapshots rw, the file system metadata contains 
>> relationship information: each snapshot has a uuid, and a parent uuid. And 
>> the parent contains information about each snapshot made of it. But all of 
>> this is domain of the snapshot tool. That's a lot easier than having to go 
>> find fstab, grub.cfg, and figure out how to get core.img to know what boot 
>> subvolume was intended, etc.
>>
>>
>> > Compare to use path relative to snapshot's fs root, we can leave the
>> > grub.cfg in snapshot unmodified and by setting snapshot id or name in
>> > a master config to switch the snapshot we want to boot. That will make
>> > things a lot easier.
>>
>
> Michael, snapshot of *what*?

Only the toplevel subvolume 5 (os installation's /) and all other
subvolume mount points should be defined in /etc/fstab for those
whatever wont be snapshotted.

> Whatever means you will use (set-default,
> environment variable, mount options) can set only one single property
> - root of filesystem.

Not use set default, but use rootflags=subvol=<SNAPSHOT_NAME> (or
rootflags=suvolid=<SNAPSHOT_NAME>...) kernel command line options to
set root of file system.

> You *STILL* need to describe relationships
> between different (snapshots of) multiple subvolumes. I.e. *which*
> snapshot of /boot/grub2/i386-pc are you going to access?

/boot/grub2/i386-pc shouldn't be snapshotted and is a subvolume
mounted by /etc/fstab. It's for the reason that we have to keep
(embedded) grub2 image and module's version in sync.

>
> Having grub to always use full pathnames makes it unambiguous. Otherwise
> it is unmanageable on grub level (*any* directory you access may
> potentially have multiple versions). This must really be solved on OS
> level by either
>
> - mandating single subvolume for your snapshottable OS, or
> - adding support to grafting individual subvolumes inside of btrfs.
>   Normal mount will not solve this on bootloader level

Single subvolume is already horrible enough. :) We should mandate it.

Thanks for your suggestions and feedback,

Regards,
Michael

>
>
>> That sounds something like the Bootloaderspec, which I like in principle in 
>> that it recognizes how hostile the distributions are at breaking the boot 
>> behavior of the prior OS, in multiboot contexts. But there's some other 
>> things that just don't seem workable, and it's also not even adopted 
>> upstream yet by GRUB and I don't know what the status of this whole idea is.
>>
>> I think the idea of snapshots in the domain of a boot manager/boot loader is 
>> really overly complicated. For another thing, it's not really appropriate to 
>> do a rollback and then immediately start modifying it by booting from it. 
>> What you'd want to do is snapshot the rollback, and then use that "cloned" 
>> copy of the rollback, leaving the original rollback in place. Otherwise the 
>> provenance of that <inserttime> snapshot is obliterated.
>>
>> And with all of these snapshots being created, something to clean up all 
>> these bouquets is necessary. Do we really want GRUB doing that also? I just 
>> this this is out of scope for GRUB.
>>
>> An FHS for Btrfs installation locations and shapshot behaviors is possibly 
>> needed, so the distributions aren't stepping all over each other in an even 
>> worse way that booting already is.
>>
>> Chris Murphy
>> _______________________________________________
>> Grub-devel mailing list
>> address@hidden
>> https://lists.gnu.org/mailman/listinfo/grub-devel
>
>
> _______________________________________________
> Grub-devel mailing list
> address@hidden
> https://lists.gnu.org/mailman/listinfo/grub-devel