Re: [bug #48885] zfs_mount fails with `org.zfsonlinux:large_dnode` enabled: Unsupported features in pool

[Brian Behlendorf]

Thomas,

You should be able to simply add 'org.zfsonlinux:large_dnode' to the
list of supported features.  As long as the dataset property dnodesize
is set to legacy there won't be any on-disk format changes.  The dnodes
on disk will still be 512 bytes in size and it will just leverage some
previously unused pad space in the dnode_phys_t.

Since we were concerned about grub support the zfs command now prohibits
users from setting the dnodesize property to a non-legacy value for
datasets with the bootfs property set.

It shouldn't be a huge amount of work to fully support large dnode in
grub but simply allowing the feature flag and checking the property
should be enough for the vast majority of use cases.

-- 
Thanks,
Brian

On 08/24/2016 11:07 AM, Toomas Soome wrote:
> 
> The commit itself is here:
> https://github.com/zfsonlinux/zfs/commit/50c957f702ea6d08a634e42f73e8a49931dd8055
> 
> But I took the liberty of adding “responsible” people to CC: list - since 
> this feature is zol specific, and I don’t have any linux systems to test on, 
> perhaps zol community can find some resources to address this feature 
> request:) Ned & Brian, can you help please?:)
> 
> rgds,
> toomas
> 
> 
>> On 24. aug 2016, at 20:54, Andrei Borzenkov <address@hidden> wrote:
>>
>> Thomas, any idea? Does it need explicit support or we can simply list
>> this feature? Could you help with implementation? Thank you!
>>
>>
>> -------- Перенаправленное сообщение --------
>> Тема: [bug #48885] zfs_mount fails with `org.zfsonlinux:large_dnode`
>> enabled: Unsupported features in pool
>> Дата: Wed, 24 Aug 2016 17:21:55 +0000 (UTC)
>> От: Niklas Haas <address@hidden>
>> Кому: Niklas Haas <address@hidden>, address@hidden
>>
>> URL:
>>  <http://savannah.gnu.org/bugs/?48885>
>>
>>                 Summary: zfs_mount fails with `org.zfsonlinux:large_dnode`
>> enabled: Unsupported features in pool
>>                 Project: GNU GRUB
>>            Submitted by: haasn
>>            Submitted on: Wed 24 Aug 2016 05:21:52 PM GMT
>>                Category: Filesystem
>>                Severity: Major
>>                Priority: 5 - Normal
>>              Item Group: None
>>                  Status: None
>>                 Privacy: Public
>>             Assigned to: None
>>         Originator Name:         Originator Email:
>> Open/Closed: Open
>>         Discussion Lock: Any
>>                 Release:                  Release: Git master
>>         Reproducibility: Every Time
>>         Planned Release: None
>>
>>    _______________________________________________________
>>
>> Details:
>>
>>
>>
>> # zpool get address@hidden
>> NAME         PROPERTY             VALUE                SOURCE
>> zarathustra  address@hidden  active               local
>>
>> # grub-probe -vvvv /
>> ...
>> grub-core/kern/fs.c:56: Detecting zfs...
>> grub-core/osdep/hostdisk.c:416: opening the device `/dev/sdc1' in
>> open_device()
>> grub-core/fs/zfs/zfs.c:1192: label ok 0
>> grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdc1'
>> grub-core/fs/zfs/zfs.c:1007: check 2 passed
>> grub-core/fs/zfs/zfs.c:1018: check 3 passed
>> grub-core/fs/zfs/zfs.c:1025: check 4 passed
>> grub-core/fs/zfs/zfs.c:1035: check 6 passed
>> grub-core/fs/zfs/zfs.c:1043: check 7 passed
>> grub-core/fs/zfs/zfs.c:1054: check 8 passed
>> grub-core/fs/zfs/zfs.c:1064: check 9 passed
>> grub-core/fs/zfs/zfs.c:1086: check 11 passed
>> grub-core/fs/zfs/zfs.c:1112: check 10 passed
>> grub-core/fs/zfs/zfs.c:1128: str=com.delphix:embedded_data
>> grub-core/fs/zfs/zfs.c:1137: check 12 passed (feature flags)
>> grub-core/fs/zfs/zfs.c:1878: zio_read: E 0: size 2048/2048
>> grub-core/fs/zfs/zfs.c:1899: endian = -1
>> grub-core/fs/zfs/zfs.c:595: dva=8, b47f00
>> grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdc1'
>> grub-core/fs/zfs/zfs.c:2681: endian = -1, blkid=0
>> grub-core/fs/zfs/zfs.c:2023: endian = -1
>> grub-core/fs/zfs/zfs.c:2054: endian = -1
>> grub-core/fs/zfs/zfs.c:1878: zio_read: E 0: size 16384/4096
>> grub-core/fs/zfs/zfs.c:1899: endian = -1
>> grub-core/fs/zfs/zfs.c:595: dva=8, b47ef8
>> grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdc1'
>> grub-core/fs/zfs/zfs.c:2023: endian = 1
>> grub-core/fs/zfs/zfs.c:2049: endian = 1
>> grub-core/fs/zfs/zfs.c:1878: zio_read: E 0: size 16384/4096
>> grub-core/fs/zfs/zfs.c:1899: endian = 1
>> grub-core/fs/zfs/zfs.c:595: dva=8, b3fdd8
>> grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdc1'
>> grub-core/fs/zfs/zfs.c:2685: alive
>> grub-core/fs/zfs/zfs.c:2496: looking for 'features_for_read'
>> grub-core/fs/zfs/zfs.c:2023: endian = 1
>> grub-core/fs/zfs/zfs.c:2049: endian = 1
>> grub-core/fs/zfs/zfs.c:1878: zio_read: E 0: size 1024/1024
>> grub-core/fs/zfs/zfs.c:1899: endian = 1
>> grub-core/fs/zfs/zfs.c:595: dva=8, 8afdc8
>> grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdc1'
>> grub-core/fs/zfs/zfs.c:2506: zap read
>> grub-core/fs/zfs/zfs.c:2510: micro zap
>> grub-core/fs/zfs/zfs.c:2513: returned 0
>> grub-core/fs/zfs/zfs.c:2681: endian = -1, blkid=1
>> grub-core/fs/zfs/zfs.c:2023: endian = -1
>> grub-core/fs/zfs/zfs.c:2054: endian = -1
>> grub-core/fs/zfs/zfs.c:1878: zio_read: E 0: size 16384/4096
>> grub-core/fs/zfs/zfs.c:1899: endian = -1
>> grub-core/fs/zfs/zfs.c:595: dva=8, b47ef8
>> grub-core/fs/zfs/zfs.c:2023: endian = 1
>> grub-core/fs/zfs/zfs.c:2049: endian = 1
>> grub-core/fs/zfs/zfs.c:1878: zio_read: E 0: size 16384/4096
>> grub-core/fs/zfs/zfs.c:1899: endian = 1
>> grub-core/fs/zfs/zfs.c:595: dva=8, b47ee8
>> grub-core/fs/zfs/zfs.c:2685: alive
>> grub-core/fs/zfs/zfs.c:2023: endian = 1
>> grub-core/fs/zfs/zfs.c:2049: endian = 1
>> grub-core/fs/zfs/zfs.c:1878: zio_read: E 0: size 512/512
>> grub-core/fs/zfs/zfs.c:1899: endian = 1
>> grub-core/fs/zfs/zfs.c:595: dva=8, 8b00a0
>> grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdc1'
>> grub-core/fs/zfs/zfs.c:2112: zap: name = com.delphix:extensible_dataset,
>> value
>> = 26, cd = 0
>> grub-core/fs/zfs/zfs.c:2112: zap: name = com.delphix:embedded_data,
>> value = 1,
>> cd = 0
>> grub-core/fs/zfs/zfs.c:2112: zap: name = org.open-zfs:large_blocks,
>> value = 0,
>> cd = 0
>> grub-core/fs/zfs/zfs.c:2112: zap: name = org.zfsonlinux:large_dnode, value =
>> 26, cd = 0
>> grub-core/kern/fs.c:78: zfs detection failed.
>> ...
>>
>>
>> Aside: For some reason, the “Unsupported features in pool” message doesn't
>> actually get printed. Don't ask me why, but I have stepped through
>> `zfs_mount`
>> in gdb and confirmed that it hits the following call:
>>
>>
>>      grub_error (GRUB_ERR_BAD_FS, "Unsupported features in pool");
>>
>>
>> This issue is most likely due to the fact that `large_dnode` is not
>> listed in
>> the list of supported features in `static const char *spa_feature_names[]`,
>> and therefore also probably doesn't work.
>>
>>
>> # grub-probe --version
>> grub-probe (GRUB) 2.02~beta3
>>
>> # emerge --info grub
>> Portage 2.3.0 (python 2.7.12-final-0, hardened/linux/amd64, gcc-5.4.0,
>> glibc-2.23-r2, 4.7.1-hardened-gnu x86_64)
>> =================================================================
>>                         System Settings
>> =================================================================
>> System uname:
>> address@hidden
>> KiB Mem:    32907552 total,    786920 free
>> KiB Swap:          0 total,         0 free
>> Timestamp of repository gentoo: Tue, 23 Aug 2016 18:15:01 +0000
>> sh dash 0.5.9
>> ld GNU ld (Gentoo 2.26.1 p1.0) 2.26.1
>> app-shells/bash:          4.3_p46::gentoo
>> dev-lang/perl:            5.24.0-r1::gentoo
>> dev-lang/python:          2.7.12::gentoo, 3.4.5::gentoo, 3.5.2::gentoo
>> dev-util/cmake:           3.6.1::gentoo
>> dev-util/pkgconfig:       0.29.1::gentoo
>> sys-apps/baselayout:      2.2-r1::gentoo
>> sys-apps/openrc:          0.21.3::gentoo
>> sys-apps/sandbox:         2.10-r2::gentoo
>> sys-devel/autoconf:       2.13::gentoo, 2.69-r2::gentoo
>> sys-devel/automake:       1.11.6-r2::gentoo, 1.13.4-r1::gentoo,
>> 1.14.1-r1::gentoo, 1.15-r2::gentoo
>> sys-devel/binutils:       2.26.1::gentoo
>> sys-devel/gcc:            5.4.0::gentoo
>> sys-devel/gcc-config:     1.8-r1::gentoo
>> sys-devel/libtool:        2.4.6-r2::gentoo
>> sys-devel/make:           4.2.1::gentoo
>> sys-kernel/linux-headers: 4.7::gentoo (virtual/os-headers)
>> sys-libs/glibc:           2.23-r2::gentoo
>> Repositories:
>>
>> gentoo
>>    location: /usr/portage
>>    sync-type: rsync
>>    sync-uri: rsync://rsync.de.gentoo.org/gentoo-portage
>>    priority: -1000
>>
>> local_overlay
>>    location: /usr/local/portage
>>    masters: gentoo
>>    priority: 0
>>
>> deter
>>    location: /var/lib/layman/deter
>>    masters: gentoo
>>    priority: 50
>>
>> dotnet
>>    location: /var/lib/layman/dotnet
>>    masters: gentoo
>>    priority: 50
>>
>> emery
>>    location: /var/lib/layman/emery
>>    masters: gentoo
>>    priority: 50
>>
>> haskell
>>    location: /var/lib/layman/haskell
>>    masters: gentoo
>>    priority: 50
>>
>> kde
>>    location: /var/lib/layman/kde
>>    masters: gentoo
>>    priority: 50
>>
>> mva
>>    location: /var/lib/layman/mva
>>    masters: gentoo
>>    priority: 50
>>
>> palemoon
>>    location: /var/lib/layman/palemoon
>>    masters: gentoo
>>    priority: 50
>>
>> qt
>>    location: /var/lib/layman/qt
>>    masters: gentoo
>>    priority: 50
>>
>> sunrise
>>    location: /var/lib/layman/sunrise
>>    masters: gentoo
>>    priority: 50
>>
>> vapoursynth
>>    location: /var/lib/layman/vapoursynth
>>    masters: gentoo
>>    priority: 50
>>
>> ACCEPT_KEYWORDS="amd64 ~amd64"
>> ACCEPT_LICENSE="* address@hidden"
>> CBUILD="x86_64-pc-linux-gnu"
>> CFLAGS="-march=native -O2 -pipe"
>> CHOST="x86_64-pc-linux-gnu"
>> CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt
>> /usr/share/i18n/locales"
>> CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d
>> /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild
>> /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d
>> /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
>> CXXFLAGS="-march=native -O2 -pipe"
>> DISTDIR="/var/cache/distfiles"
>> EMERGE_DEFAULT_OPTS="--ask --ask-enter-invalid --alert --keep-going
>> --quiet-unmerge-warn --quiet-build --quiet-fail
>> --ignore-built-slot-operator-deps=y --jobs=20 --load-average=16 --tree"
>> FCFLAGS="-O2 -pipe"
>> FEATURES="assume-digests binpkg-logs compressdebug
>> config-protect-if-modified
>> distlocks ebuild-locks fixlafiles installsources merge-sync news
>> parallel-fetch preserve-libs protect-owned sandbox sfperms splitdebug strict
>> unknown-features-warn unmerge-logs unmerge-orphans usersandbox usersync"
>> FFLAGS="-O2 -pipe"
>> GENTOO_MIRRORS="http://de-mirror.org/gentoo/ http://gentoo.mneisen.org/
>> http://mirror.netcologne.de/gentoo/
>> http://ftp.halifax.rwth-aachen.de/gentoo/
>> http://ftp.spline.inf.fu-berlin.de/mirrors/gentoo/
>> http://ftp.uni-erlangen.de/pub/mirrors/gentoo
>> http://ftp-stud.hs-esslingen.de/pub/Mirrors/gentoo/";
>> LANG="en_US.UTF-8"
>> LDFLAGS="-Wl,-O1 -Wl,--as-needed"
>> MAKEOPTS="--jobs=33"
>> PKGDIR="/usr/portage/packages"
>> PORTAGE_CONFIGROOT="/"
>> PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times
>> --omit-dir-times --compress --force --whole-file --delete --stats
>> --human-readable --timeout=180 --exclude=/distfiles --exclude=/local
>> --exclude=/packages --exclude=/.git"
>> PORTAGE_TMPDIR="/var/tmp"
>> USE="X a52 aac acl acpi amd64 aspell avx bash-completion bazaar berkdb
>> branding btrfs bzip2 cairo caps cdr cjk clang cli color-management cracklib
>> crypt cuda cvs cxx darcs dri dts dvd dvdr emboss encode exif fam ffmpeg
>> firefox flac fontconfig gdbm gif git glamor gpg gpm gtk guile hardened http2
>> iconv idn ipv6 jbig jpeg jpeg2k justify lcms ldap libnotify littlecms
>> llvm lua
>> lzma lzo mad mem-scramble mercurial mmx mmxext mng modules mp3 mp4 mpeg
>> multilib ncurses nls nptl numa ogg opencl openexr opengl openmp otr pam
>> pango
>> pax_kernel pcre pdf perl pie png polarssl ppds python qt qt3support qt5
>> readline ruby sasl savedconfig sdl seccomp session socks socks5 spell
>> sse sse2
>> sse3 sse4 sse4_1 sse4_2 ssl ssp ssse3 subversion svg systemd tcpd theora
>> threads tiff truetype tta udev unicode urandom usb vim vim-syntax vorbis
>> wavpack wxwidgets x264 xattr xcb xetex xfs xft xml xtpax xvid xz zlib zsh
>> zsh-completion" ABI_X86="64 32" ALSA_CARDS="ali5451 als4000 atiixp
>> atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968
>> fm801
>> hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx
>> via82xx-modem
>> ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions
>> alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file
>> authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user
>> autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env
>> expires ext_filter file_cache filter headers include info log_config logio
>> mem_cache mime mime_magic negotiation rewrite setenvif speling status
>> unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="krita"
>> CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap
>> syslog" CPU_FLAGS_X86="mmx sse sse2 sse3 ssse3 mmxext sse4 sse4_1 sse4_2 avx
>> aes" DRACUT_MODULES="btrfs" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm
>> earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip
>> navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2
>> timing
>> tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="pc" INPUT_DEVICES="keyboard
>> mouse
>> evdev" KERNEL="linux" L10N="en" LCD_DEVICES="bayrad cfontz cfontz633 glk
>> hd44780 lb216 lcdm001 mtxorb ncurses text"
>> LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en"
>> OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6"
>> PYTHON_SINGLE_TARGET="python3_5" PYTHON_TARGETS="python2_7 python3_5"
>> RUBY_TARGETS="ruby20 ruby21 ruby22 ruby23" USERLAND="GNU"
>> XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p
>> iface
>> geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac
>> delude chaos account"
>> Unset:  CC, CPPFLAGS, CTARGET, CXX, INSTALL_MASK, LC_ALL,
>> PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS,
>> PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
>>
>> =================================================================
>>                        Package Settings
>> =================================================================
>>
>> sys-boot/grub-9999::gentoo was built with the following:
>> USE="fonts libzfs multislot nls sdl themes truetype -debug
>> -device-mapper -doc
>> -efiemu -mount -static -test" ABI_X86="64" GRUB_PLATFORMS="pc -coreboot
>> -efi-32 -efi-64 -emu -ieee1275 -loongson -multiboot -qemu -qemu-mips -uboot
>> -xen -xen-32"
>> CFLAGS=""
>> LDFLAGS=""
>>
>>
>>
>>
>>
>>    _______________________________________________________
>>
>> Reply to this item at:
>>
>>  <http://savannah.gnu.org/bugs/?48885>
>>
>> _______________________________________________
>>  Message sent via/by Savannah
>>  http://savannah.gnu.org/
>>
>>
>> _______________________________________________
>> Bug-grub mailing list
>> address@hidden
>> https://lists.gnu.org/mailman/listinfo/bug-grub
>