Bug#1035317: grub-pc: /boot on LVM fails if logical volume consists of multiple physical volumes

[Vagrant Cascadian]

On 2023-05-01, Steve McIntyre wrote:
> On Sun, Apr 30, 2023 at 04:28:01PM -0700, Vagrant Cascadian wrote:
>>On 2023-04-30, Steve McIntyre wrote:
>>> On Sun, Apr 30, 2023 at 12:56:29PM -0700, Vagrant Cascadian wrote:
>>>>When I tried extending the /dev/vvm/root partition to include more
>>>>space from the /dev/vdc1 physical volume, grub fails to load at boot,
>>>>unable to find the lvm volume.
>>...
>>> Can you give us a bit more detail about the system please? With
>>> /dev/vdc, this suggests you're in a VM and this is the third drive
>>> using virtio? How big is /dev/vdc? What's the partition type?
>>
>>The disk is about 2TB and the partition type is GPT.
>>
>>There is also a 1TB disk as /dev/vdb GPT, unused.
>>
>>The main boot disk is ~500GB /dev/vda MBR.
...
>>> If possible, on your system, could you also reboot and call up a grub
>>> command line (hit "c" from the grub menu)?
...
>>> From there, I'd love to see what you get if you run "ls" here...

"ls" from the grub prompt did not show the other disk...

...until I made the second disk bootable from libvirt!

Then grub now sees both disks, and boots fine!

So this is possibly a quirk of the way libvirt exposes boot disks.


> OK, I can reproduce with something like this:
>
> vda - small-ish dos-partitioned disk, fresh bullseye installation made
>       using LVM directly for the rootfs, no /boot
...
> *However*, using pvmove to move the root LV to vdc fails. I'd expect
> similar if you've just extended and grown the rootfs. I get an error:
>
>   error: disk `lvmid/<stuff>' not found

That sounds consistent with the error message I got.


> and this is exactly the kind of thing I was looking for - maybe
> grub-pc can't access the drive due to BIOS limitations. *However*, I
> think I'm wrong and Pascal Hambourg is right here...!

I was able to reproduce with a simpler virtual machine (two disks,
lvm-backed, 10GB and 20GB)...


> I ran d-i in rescue mode to get into the system, simply ran
> dpkg-reconfigure grub-pc (which will run grub-install *and*
> update-grub), and the system now boots again. It looks like what we're
> seeing might be a limit in what's built in to the core image by
> default. grub-pc is deliberately designed to build minimal images
> here, to minimise the chance of the image being too large to embed.

Re-running grub-install /dev/vda from debian-installer rescue mode did
*not* fix the issue for me (though, now I am curious if dpkg-reconfigure
grub-pc would do anything more?)


> Being brutally honest, I think the problem is in your disk
> setup. You've found an edge case that *can* be accomodated and
> supported, but needs some extra effort to make it work.
>
> I'd *strongly* recommend re-thinking your setup here. If you're going
> to set up complicated storage here *for a VM*, then it would be
> trivial to set up a /boot partition which grub will never have
> problems finding and booting from. Make your life easier and just do
> that...

I only got into that situation due to a somewhat weird circumstance
(e.g. having a user on a system with libvirt access, but not root, and
libvirt helpfully re-owns files so that nobody else can mess with them).
Needed more space than originally planned, figured I could just add more
disks to LVM... and here we are.

I had used this sort of setup for at least a release or two of Debian
releases on real hardware without much trouble, so was inclined to go
all-in on LVM...

For the future, I guess I'll just resign myself to a split /boot.

Once I saw this bug I figured it was worth reporting, as it appeared
different enough from others I coudl see in the BTS.


Thanks for poking at this, hopefully someone else who stubs their toe on
this in the future will benefit. :)


live well,
  vagrant
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 227 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/pkg-grub-devel/attachments/20230430/b02cd95d/attachment.sig>