Re: strange grub error on ARM64

[Atish Patra]

On Mon, Mar 6, 2023 at 7:21 PM Michael Chang <mchang@suse.com> wrote:
>
> On Mon, Mar 06, 2023 at 04:22:53PM -0800, Atish Patra wrote:
> > On Mon, Mar 6, 2023 at 1:25 PM Fabian Vogt <fvogt@suse.de> wrote:
> > >
> > > Hi,
> > >
> > > Am Montag, 6. März 2023, 21:59:23 CET schrieb Daniel Kiper:
> > > > Adding Gary and Michael...
> > > >
> > > > Gary, Michael, could you help with this? Or at least forward this email
> > > > to folks who may help...
> > > >
> > > > Daniel
> > > >
> > > > On Mon, Mar 06, 2023 at 10:56:52AM -0800, Atish Patra wrote:
> > > > > Hi,
> > > > > I am working on a cross-architecture cleanup series that removes the
> > > > > arch specific headers from RISC-V, ARM, ARM64 and unifies the linux
> > > > > loader on ARM64 & Linux.
> > > > >
> > > > > The v7 version can be found here.
> > > > > https://lists.gnu.org/archive/html/grub-devel/2023-02/msg00046.html
> > > > >
> > > > > It had a few bugs which are fixed and can be found here.
> > > > >
> > > > > Here are the patches
> > > > > https://github.com/atishp04/grub/commits/riscv_uefi_v8
> > > > >
> > > > > Anybody with ARM64 and/or test setup: Can you please do a quick test
> > > > > to see if you see similar or any other failure ?
> > > > >
> > > > > I noticed a very strange error during ARM64 testing. I have probably
> > > > > done something stupid during the grub build
> > > > > as I can reproduce this with upstream grub as well.
> > > > >
> > > > > I was trying to boot an OpenSuse image in qemu virt machine for ARM64 
> > > > > as per
> > > > > the instructions[1]. The stock grub image works fine however my custom
> > > > > build fails to boot
> > > > > and drops to the grub command line. It seems the grub detects the disk
> > > > > but doesn't read the root partition correctly on my cross-compiled 
> > > > > image.
> > > > > It has one additional directory ("@") which I don't see when I mount
> > > > > the disk. it doesn't list any files as well.
> > >
> > > What you see is the expected behaviour with upstream GRUB.
> > > The btrfs volume contains various subvolumes below @ which are mounted 
> > > into the
> > > root file system at various points. The root filesystem itself is at
> > > @/.snapshots/<number>/snapshot/ and set as the default subvolume. That's 
> > > what
> > > is mounted in a running system if you pass default flags. To see the same 
> > > as
> > > GRUB, you can do mount -o subvolume=/ /dev/foo3 /mnt.
> > >
> >
> > Thanks!! Thank god I am not crazy as I thought I was ;)
> >
> > > Downstream there's a patch (and multiple related ones) that makes GRUB 
> > > mount
> > > the default subvolume instead of the root of the filesystem:
> > > https://build.opensuse.org/package/view_file/openSUSE:Factory/grub2/grub2-btrfs-03-follow_default.patch?expand=1&rev=285
> > >
> >
> > Is this the only patch or I need to pull all the patches on that tree
> > to make it work ?
>
> You'll need other patch so that files not in default subvolume can be
> accessed through mount at grub's runtime, also to enable automatic setup
> via the utility grub-(install|mkconfig).
>
> >
> > Is there an easy way to test upstream grub stuff on ARM64 in case I
> > need to pull more dependent patches to make it work?
>
> You can also use rpm from this Open Build Service (OBS) project:
>
> https://build.opensuse.org/package/show/home:michael-chang:grub:devel/grub2
>
> It is for merging upstream git with SUSE patches rebased regularly per
> month convenient to catch early build failure, find regression, try out
> new feature and so on.
>

Thanks for pointing that out. But it seems all the patches are on top
of some older release of grub.
I am trying to test patches on top of the upstream and some of these
patches may not apply cleanly.

I was looking for alternatives with other distros. But every other
distro probably carries a similar number
of out-of-tree patches.
Here is the fedora one.

https://src.fedoraproject.org/rpms/grub2/blob/rawhide/f/grub.patches

I have a small favor to ask. Is it possible for you to test out the
changes[1] on the ARM64 platform ?
You probably have to rebase suse patches on top of upstream first as
well. But you would probably do it much much much faster than me
given my limited knowledge about grub & suse.

[1] https://github.com/atishp04/grub/commits/riscv_uefi_v8

> Thanks,
> Michael
>
> >
> > > The design of the "Boot into Snapshot" functionality relies on this.
> > >
> > > > > ============= Cross compiled grub image =====================
> > > > > grub> ls -l (hd0,gpt3)/
> > > > > DIR          20230302080325 @/
> > > > >
> > > > > grub> ls -l (hd0,gpt3)/@/
> > > > > DIR          20230302080326 boot/
> > > > > DIR         srv/
> > > > > DIR          20230302080326 usr/
> > > > > DIR         opt/
> > > > > DIR          20230302080326 home/
> > > > > DIR         var/
> > > > > DIR         tmp/
> > > > > DIR         root/
> > > > >
> > > > > grub> ls -l (hd0,gpt3)/@/boot/
> > > > > DIR          20230302080326 grub2/
> > > > >
> > > > > grub>
> > > > > ============= Stock grub image from the OpenSuse 
> > > > > image=====================
> > > > > grub> ls (hd0,gpt3)/
> > > > > dev/ run/ boot/ config.partids lib sbin srv/ usr/ opt/ 
> > > > > config.bootoptions lib64
> > > > > .snapshots/ bin home/ etc/ var/ tmp/ sys/ mnt/ proc/ root/
> > > > >
> > > > > The same build steps works fine for RISC-V for OpenSuse image though.
> > > > > However, I couldn't figure out what it was. Any help is appreciated.
> > >
> > > It's possible that it does not use snapshot or not even btrfs.
> > >
> > > Cheers,
> > > Fabian
> > >
> > > > > [1] https://en.opensuse.org/openSUSE:AArch64
> > > > >
> > > > > --
> > > > > Regards,
> > > > > Atish
> > > >
> > > > _______________________________________________
> > > > Grub-devel mailing list
> > > > Grub-devel@gnu.org
> > > > https://lists.gnu.org/mailman/listinfo/grub-devel
> > >
> > >
> >
> >
> > --
> > Regards,
> > Atish



-- 
Regards,
Atish