grub-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: LVM & RAID10


From: Wojciech Pyczak
Subject: Re: LVM & RAID10
Date: Fri, 05 Dec 2008 22:11:50 +0100
User-agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); pl; rv:1.9.1b1pre) Gecko/20081006 Shredder/3.0a3



Hi,

Perhaps you can describe the steps used to create the LVM volume,
along with version number of tools. Let me see if the error can be
recreated in qemu.

OK. I've made some less chaotic tests:

Hardware (probably insignificant):
 - AMD Phenom X3
 - Gigabyte mobo (780G/SB700 AMD chipset)
- 2x WD SE-16 6400AAKS (before I started testing I zeroed with dd a few GB of each drive - just in case..)

Software:
 - System RescueCD (used to manage partitions, RAID/LVM volumes)
- Linux sysresccd 2.6.26.07-std111-amd64 #1 SMP Sun Oct 26 11:28:49 UTC 2008 x86_64 AMD Phenom(tm) 8450 Triple-Core Processor AuthenticAMD GNU/Linux - LVM version: 2.02.39 (2008-06-27), Library version: 1.02.27 (2008-06-25), Driver version: 4.13.0
    - mdadm - v2.6.4 - 19th October 2007

- Gentoo custom ("stable amd64") stage3 - http://www.funtoo.org/linux/amd64/funtoo-amd64-2008.12.05/ (used for chrooting)

- Grub2 rev 1933 (latest), build on my current PC:
- ./configure --prefix=/usr --enable-grub-emu --enable-grub-fstest LDFLAGS=-static
    - gcc version 4.1.2 (Gentoo 4.1.2 p1.1)
    - GNU Make 3.81
    - bison (GNU Bison) 2.3
    - sys-devel/binutils-2.18-r3
* IMO also insignificant, since previously I build grub on my new system which used many "experimental" packages, such as GCC 4.3.2, etc. * * Small note - I renamed grub binaries (in bin/sbin) to *.org and made a wrapper script, so I would know arguments used by grub-install/setup: *
        #!/bin/sh
        echo \#${0} address@hidden 1>&2
        exec ${0}.org address@hidden


I used cfdisk to partition disks. Commands used to create RAID arrays and LVM volumes: mdadm --create /dev/md0 (md1) --raid-devices=3 --level=10 --layout=f2 --chunk=256 /dev/sd{a,b}1 (sd{a,b}2) missing
    pvcreate /dev/md0
    vgcreate test /dev/md0
    lvcreate test -ntest -l1788

    mdadm --assemble /dev/md0 (md1) /dev/sd{a,b}1 (sd{a,b}2)
    pvscan
    vgchange -ay
Nothing fancy...


Test procedure looked something like that:
- booting system with RescueCD
- partitioning, creating RAID/LVM, formatting, reboot
- after reboot:
    - assembling RAID/LVM volumes
    - mounting corresponding device
    - unpacking stage3
    - unpacking package containing grub
- creating missing directories (/boot/grub) and files (device.map, simple grub.cfg)
    - mouting proc, sys, dev
    - grub-install/setup, whatever



===== Single partition, RAID10, reiserfs
Disk /dev/sda: 77825 cylinders, 255 heads, 63 sectors/track
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0

   Device Boot Start     End   #cyls    #blocks   Id  System
   /dev/sda1   *      0+    607     608-   4883728+  da  Non-FS data
   /dev/sda2          0       -       0          0    0  Empty
   /dev/sda3          0       -       0          0    0  Empty
   /dev/sda4          0       -       0          0    0  Empty

Disk /dev/sdb: 77825 cylinders, 255 heads, 63 sectors/track
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0

   Device Boot Start     End   #cyls    #blocks   Id  System
   /dev/sdb1   *      0+    607     608-   4883728+  da  Non-FS data
   /dev/sdb2          0       -       0          0    0  Empty
   /dev/sdb3          0       -       0          0    0  Empty
   /dev/sdb4          0       -       0          0    0  Empty
--------------------------------------------------------------------------------------------------------------------------
 Personalities : [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
 md0 : active raid10 sda1[0] sdb1[1]
       7325184 blocks 256K chunks 2 far-copies [3/2] [UU_]

       unused devices: <none>
--------------------------------------------------------------------------------------------------------------------------
sysresccd grub # grub-install "(md0)"
#/usr/sbin/grub-install (md0)
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=device /boot/grub #/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=fs --device /dev/md0 #/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=abstraction --device /dev/md0 #/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=drive --device /dev/md0 #/usr/bin/grub-mkimage --output=/boot/grub/core.img --prefix=(md0)/boot/grub biosdisk reiserfs pc pc raid mdraid #/usr/sbin/grub-setup --directory=/boot/grub --device-map=/boot/grub/device.map
(md0)
Installation finished. No error reported.
This is the contents of the device map /boot/grub/device.map.
Check if this is correct or not. If any of the lines is incorrect,
fix it and re-run the script `grub-install'.

(hd0)   /dev/sda
(hd1)   /dev/sdb
(hd2)   /dev/sdc <= USB stick
--------------------------------------------------------------------------------------------------------------------------
Everything works (I copied memtest86 from RescueCD to /boot and managed to boot it).





===== Single partion, LVM @ RAID10, reiserfs
Partitions/RAID like above
--------------------------------------------------------------------------------------------------------------------------
address@hidden /root % pvdisplay
--- Physical volume ---
PV Name               /dev/md0
VG Name               test
PV Size               6.99 GB / not usable 1.50 MB
Allocatable           yes (but full)
PE Size (KByte)       4096
Total PE              1788
Free PE               0
Allocated PE          1788
PV UUID               6i6MZH-7Lv7-ZcXb-yr2W-ShFz-2c13-srXd07

address@hidden /root % vgdisplay
--- Volume group ---
VG Name               test
System ID
Format                lvm2
Metadata Areas        2
Metadata Sequence No  2
VG Access             read/write
VG Status             resizable
MAX LV                0
Cur LV                1
Open LV               1
Max PV                0
Cur PV                1
Act PV                1
VG Size               6.98 GB
PE Size               4.00 MB
Total PE              1788
Alloc PE / Size       1788 / 6.98 GB
Free  PE / Size       0 / 0
VG UUID               aCTAWb-xbpL-3oSp-1V2X-Oy3T-E2D7-gJoCZG

address@hidden /root % lvdisplay
--- Logical volume ---
LV Name                /dev/test/test
VG Name                test
LV UUID                IzYAb2-9y4u-HqB2-27bD-ihvH-uCmn-2a1xlH
LV Write Access        read/write
LV Status              available
# open                 2
LV Size                6.98 GB
Current LE             1788
Segments               1
Allocation             inherit
Read ahead sectors     auto
- currently set to     256
Block device           253:0
--------------------------------------------------------------------------------------------------------------------------
sysresccd / # grub-install '(md0)'
#/usr/sbin/grub-install (md0)
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=device /boot/grub
error: We don't support multiple metadata areas
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=fs --device /dev/mapper/test-test
error: We don't support multiple metadata areas
grub-probe: error: no mapping exists for `test-test'
Auto-detection of a filesystem module failed.
Please specify the module with the option `--modules' explicitly.
--------------------------------------------------------------------------------------------------------------------------
sysresccd / # grub-install.org '(md0)' --modules='biosdisk pc raid mdraid lvm reiserfs' #/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=device /boot/grub
error: We don't support multiple metadata areas
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=fs --device /dev/mapper/test-test
error: We don't support multiple metadata areas
grub-probe: error: no mapping exists for `test-test'
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=abstraction --device /dev/mapper/test-test
error: We don't support multiple metadata areas
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=drive --device /dev/mapper/test-test
error: We don't support multiple metadata areas
#/usr/bin/grub-mkimage --output=/boot/grub/core.img --prefix=(test-test)/boot/grub biosdisk pc raid mdraid lvm reiserfs biosdisk lvm #/usr/sbin/grub-setup --directory=/boot/grub --device-map=/boot/grub/device.map
(md0)
error: We don't support multiple metadata areas
grub-setup: error: no mapping exists for `test-test'
--------------------------------------------------------------------------------------------------------------------------
Grub works (after reboot) but I can't access filesystem, LVM module is not present





====== Added second partition
Disk /dev/sda: 77825 cylinders, 255 heads, 63 sectors/track
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0

Device Boot Start     End   #cyls    #blocks   Id  System
/dev/sda1   *      0+    607     608-   4883728+  da  Non-FS data
/dev/sda2        608    1215     608    4883760   da  Non-FS data
/dev/sda3          0       -       0          0    0  Empty
/dev/sda4          0       -       0          0    0  Empty

Disk /dev/sdb: 77825 cylinders, 255 heads, 63 sectors/track
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0

Device Boot Start     End   #cyls    #blocks   Id  System
/dev/sdb1   *      0+    607     608-   4883728+  da  Non-FS data
/dev/sdb2        608    1215     608    4883760   da  Non-FS data
/dev/sdb3          0       -       0          0    0  Empty
/dev/sdb4          0       -       0          0    0  Empty
--------------------------------------------------------------------------------------------------------------------------
RAID/LVM like above
--------------------------------------------------------------------------------------------------------------------------
#/usr/sbin/grub-install (md0)
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=device /boot/grub
error: no such partition
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=fs --device /dev/mapper/test-test
error: no such partition
grub-probe: error: no mapping exists for `test-test'
Auto-detection of a filesystem module failed.
Please specify the module with the option `--modules' explicitly.
--------------------------------------------------------------------------------------------------------------------------
sysresccd / # grub-install.org '(md0)' --modules="biosdisk pc raid mdraid lvm reiserfs" #/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=device /boot/grub
error: no such partition
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=fs --device /dev/mapper/test-test
error: no such partition
grub-probe: error: no mapping exists for `test-test'
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=abstraction --device /dev/mapper/test-test
error: no such partition
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=drive --device /dev/mapper/test-test
error: no such partition
#/usr/bin/grub-mkimage --output=/boot/grub/core.img --prefix=(test-test)/boot/grub biosdisk pc raid mdraid lvm reiserfs biosdisk lvm #/usr/sbin/grub-setup --directory=/boot/grub --device-map=/boot/grub/device.map
(md0)
grub-setup: error: no mapping exists for `test-test'
--------------------------------------------------------------------------------------------------------------------------
Second partition is unsed, however grub stoped complaining about "multiple metadata". Desipte that I still can't boot system.




===== Added RAID10 using second partition pair, I'll try to boot system from it sysresccd / # grub-mkimage -o /boot/grub/core.img biosdisk pc raid mdraid lvm reiserfs #/usr/bin/grub-mkimage -o /boot/grub/core.img biosdisk pc raid mdraid lvm reiserfs
sysresccd / # grub-setup '(md1)'
#/usr/sbin/grub-setup (md1)
grub-setup: error: Core image is too big for embedding, but this is required when
the root device is on a RAID array or LVM volume.
--------------------------------------------------------------------------------------------------------------------------
After seeing that error message I created core.img without lvm module and rebooted computer (I was able to boot system), then I loaded lvm module but I couldn't see any lvm volumes (despite fact that I saw md0 containing it). I tried to rmmod/inmod
again but it didn't help.



OK so here are my thoughts - most ppl use ext2 for /boot, I used reiserfs which has 4k larger module so LVM just don't fit info core.img. Using grub-install/setup on address@hidden always ends up with some errors I just couldn't see it before (those errors were discarted somewhere along the way ?) but it doesn't explain why I couldn't see lvm volumes desipe the fact that loaded lvm module from disk. Perhaps I should try to create LVM on md1 and boot from md0 to see what will happen ?
Anyway there is something wrong with LVM+RAID on single partition ;)


It's caused by size limitation of mbr, which is normally 63 sectors.
When you use the following modules:

biosdisk pc reiserfs lvm mdraid raid6 raid6rec

The size of core.img would exceed 63 sectors, so that it can't be
installed to mbr.

Is it possible to use more sectors by leaving some unused space before fisrt partion ?






reply via email to

[Prev in Thread] Current Thread [Next in Thread]