no ZFS-GRUB for current ubuntu 14 LTS 'trusty tahr'?

[RJVB]

The question is in the subject - is there no zfs-aware grub package for the current Ubuntu LTS release in the ppa? The grub package that comes with the distro (Kubuntu, in my case) clearly does not, at least grub-probe fails on a chrooted zfs pool with the message

/usr/sbin/grub-probe: error: failed to get canonical path of `/dev/usb-XXX-0:0-part1'.

I cheated by installing the version for "Raring", and that allowed me to generate a grub.cfg without errors (still in a chrooted environment with all the special directories mounted via mount --bind). However, the bootloader appears to be dysfunctional:

• I get an early error about an invalid label and/or about sparse files not being supported
• when I force the process to continue, it tries to load rpool instead of my actual pool
• when I mount my pool by hand, the boot continues but freezes just after loading cryptdisks (which completes successfully).
• once I reach that stage, even Ctrl-Alt-Del won't unfreeze the system (it'll start the shutdown but remain frozen somewhere in the process).

Any suggestion on how to get this right?

[dajhorn]

Remember that /boot/grub/i386-pc must be manually replaced with /usr/lib/grub/i386-pc and the loader reinstalled if the GRUB packages are downgraded.

Upstream support for ZFS is greatly improved since the grub-1.99 series, and it seems to have passed the "good enough" threshold for most users. This kind of bug report should now go to distro.

However, in this particular case, you'll probably get a wontfix if the problem involves ZoL and a USB device.

[RJVB]

Somehow my reply-via-email disappeared, so:

On May 15, 2014, at 03:43, Darik Horn wrote:

Remember that /boot/grub/i386-pc must be manually replaced with /usr/lib/grub/i386-pc and the loader reinstalled if the GRUB packages are downgraded.

I did (on both points :) )

Upstream support for ZFS is greatly improved since the grub-1.99 series, and it seems to have passed the "good enough" threshold for most users. This kind of bug report should now go to distro.

However, in this particular case, you'll probably get a wontfix if the problem involves ZoL and a USB device.

Because a USB device isn't worthy of support and shouldn't even be used for testing purposes??

Anyway, it seems that the issue affects also my internal drive with its ZFS pool; upstream grub-probe apparently plucks the device specifier from the pool (non-mirrored in my case), prepends it with /dev (rather than /dev/disk/by-id) and then complains it cannot find /dev/ataXXXX .

R.

[RJVB]

A few things I learned since the reply above (d.d. May 15th):

• The grub package in Ubuntu 14.04 is built without ZFS support, which explains the fact that grub-probe complains about ZFS pools. And which is not surprising, given that the build system missed a zfs dependency in one component (grub-ofpathname). I reported and posted a patch for that: https://savannah.gnu.org/bugs/index.php?42392
• However, the unit tests for zfs fail (and I've not investigated that any further). Worse, even if update-grub will detect a ZFS root, it generates menuentries that lack the load commands for the required modules, and that do not pass all the zfs flags to the kernel.
• And finally, after correcting grub.cfg by hand, I still have to import my specific pool by hand as the boot process attempts to import "rpool".

And I don't have any idea who's fault that last observation is ... grub, or the kernel. Or yet another component because of this:
I get the same issue when I use the zfs-aware grub currently in the ZoL Debian packages (and used with success on my LMDE rig). Conversely, when I install the 3.13 kernel from Kubuntu 14.04 LTS on said LMDE rig, it boots perfectly fine ...

[fajarnugraha]

Workaround for trusty:

• import the pool (possibly from live cd/usb) with "zpool import -d /dev". This is so zfs will use /dev/sd? devices, to circumvent the first issue
• install zfs-initramfs
• edit /usr/share/initramfs-tools/scripts/zfs, replace "zpool import" with "zpool import -d /dev"
• edit /etc/default/grub, add something like this (adjust pool and dataset names appropriately). This is to circumvent the second and third issue.

GRUB_CMDLINE_LINUX="boot=zfs rpool=rpool bootfs=rpool/ROOT/trusty"

• update-grub, update-initramfs -u

[RJVB]

zfs-initramfs is installed, and indeed I was thinking along the lines of editing default/grub to hard-code the bootfs name instead of using a variable.
I'm not sure what the 1st issue you're referring to is, the invalid label error? If the manpage is correct, "-d /dev" is the default, is there any reason to believe that there'd be a regression in that?
I'll agree that trying and seeing could provide that reason, but pool import works fine from the initram (busybox) prompt without having to specify -d /dev.

I'll see though if there's any unexpected differences between the LMDE and Trusty initramfs zfs scripts. Should have occurred to me earlier to do that ...!

[RJVB]

Well, not much luck with your suggestions! There's indeed a considerable difference between the initramfs scripts for zfs in the Debian package vs. the Ubuntu/Trusty package (now why would that be??) I tried with the Debian zfs-initramfs ... no change. I tried adding -d /dev to the script - no change.
I tried replacing $bootfs with the value it should have in grub.cfg ... again no change, as if the parameter isn't even handed off to the kernel.

I didn't make my change in /etc/default/grub just yet, and I notice that I use a slightly different format, "root=ZFS=zPatux/ ro boot=zfs zPatux" - maybe that should have read bootfs=zPatux, with the additional rpool= specifier.

What I don't get is why there would be such big differences between Debian and Ubuntu, especially so early in the boot process...

[FransUrbo]

There's indeed a considerable difference between the initramfs scripts for zfs in the Debian package vs. the Ubuntu/Trusty package

What I don't get is why there would be such big differences between Debian and Ubuntu, especially so early in the boot process...

That's because I've spend a lot of time and work in rewriting the whole thing, it was a quick job to begin with and Darik have been unwilling to merge my patches (no, I don't know why).

These changes is also in openzfs/zfs#2087 and the system startup script (init) changes in openzfs/zfs#2106.

[RJVB]

Strangely enough Darik's package for Ubuntu is more recent :)

Anyway, I think you're implying that there's indeed no fundamental reason for boot-time differences between Debian and Ubuntu? Would you be willing to try your hand at a (K)Ubuntu VM, or else give me some guidance in figuring out what goes wrong where on my set-up?

Darik seemed to frown upon ZFS on an external USB drive, but I can connect it via FW800 and boot a VM off the disk too if that's more politically correct ;)

[FransUrbo]

Strangely enough Darik's package for Ubuntu is more recent :)

More recent than what?
Anyway, I think you're implying that there's indeed no fundamental reason for boot-time differences between Debian and Ubuntu?

On the contrary, I'm saying that there's huge differences between the two.
Would you be willing to try your hand at a (K)Ubuntu VM

I have no resources (nor time really) to invest in more VMs, so that's why I'm currently only concentrating on Debian GNU/Linux at the moment.
Darik seemed to frown upon ZFS on an external USB drive

So do we all... It's slow, prone to disconnections and not designed for something like this - it's temporary storage.
but I can connect it via FW800 and boot a VM off the disk too if that's more politically correct ;)

Why not just rip the HD from the external casing and put it on proper SATA?

[RJVB]

On Jun 05, 2014, at 20:51, Turbo Fredriksson wrote:

Strangely enough Darik's package for Ubuntu is more recent :)

More recent than what?

The current zfs-initramfs package for Debian.

So do we all... It's slow, prone to disconnections and not designed for something like this - it's temporary storage.

As can be used for temporary installs :)
As to disconnects, I've never had one ...

but I can connect it via FW800 and boot a VM off the disk too if that's more politically correct ;)

Why not just rip the HD from the external casing and put it on proper SATA?

Because I have nothing else to put it in, other than replacing the internal disk of my Macbook Pro. In fact that won't work either, it's an ATA disk.

Anyway, IMHO there is no reason whatsoever that the boot process would work better on an internal than on an external disk.

R.

[FransUrbo]

The current zfs-initramfs package for Debian.

The Ubuntu version was pushed to the ppa on 2014-04-09 and my version on 2014-05-23.

But that's beside the point. Darik have not accepted the changes...

As to disconnects, I've never had one ...

You will. As soon as you start loading it extensively and for a longer period of time.

[RJVB]

On Jun 05, 2014, at 21:49, Turbo Fredriksson wrote:

The Ubuntu version was pushed to the ppa on 2014-04-09 and my version on 2014-05-23.

I'm pointing to "deb http://archive.zfsonlinux.org/debian wheezy main", which contains a package from january 2014. Should I change the repo ... and if so, what will that imply for the zfs and spl packages themselves?

As to disconnects, I've never had one ...

You will. As soon as you start loading it extensively and for a longer period of time.

Like doing a kernel build? I've had a netbook booted off a (btrfs) USB external for days, putting it to sleep every evening, never an issue. Anyway, as soon as the booting issue is resolved I'll move the LMDE install on my internal disk to Kubuntu 14.04 .

R.

[RJVB]

Update: I grabbed Turbo's 2014-05-23 zfs-initramfs .deb file via my LMDE install, and installed it onto my ZFS Kubuntu 14.04 install, keeping all the other packages the same. In particular, I've held the grub packages at 2.01-22debian1+zfs3~wheezy (grub-pc, grub-pc-bin, grub-common, grub2-common).

Testing is being done with the ZFS pool exposed to a VM via VirtualBox's raw disk access (to the external mounted via FW800 on a Macbook Pro); to prevent issues, /boot is on a dedicated vdi image. (VirtualBox can use the bootloader on a raw disk exposed fully but in my experience that only works for reading, not for rewriting via grub-install.)

The result is that now I can boot more or less without glitches or manual intervention. I still get the "invalid label" message when I boot, and the grub version I have installed writes out kernel command lines containing root=ZFS=/ (= with trailing slash, not appreciated by the new initrd) and without zfsforce=1. The latter appears to be required as otherwise import is refused even though the pool was last mounted on the same VM. Maybe that's a VM issue?

I almost lost the new zfs-initramfs again as the first apt-get dist-upgrade wanted to downgrade the package (don't recall seeing that before with a package installed manually through dpkg -i!) so I held that package too.

Where should I go from here? I'd prefer to keep to the stable spl/zfs packages but I'd probably have to grab the appropriate grub packages from wheezy-daily? If so, which ones? The drive I'm testing with has a GPT table (with a dedicated GPT Boot Partition) and my current grub packages handle it just fine.

BTW: I tried adding the Debian repos to Kubuntu (using apt-add-repository), but got stuck on a missing key error during apt-get update. Have I simply forgotten something ... or is there another "official" way to download binary packages from a repo server without actually editing something in apt/sources (other than figuring out the http URL through a web browser)?