Bug#1035317: grub-pc: /boot on LVM fails if logical volume consists of multiple physical volumes

[Steve McIntyre]

Control: severity -1 minor
Control: tags -1 wontfix
Control: retitle -1 grub-pc: needs reconfiguration with complex storage setup

On Sun, Apr 30, 2023 at 04:28:01PM -0700, Vagrant Cascadian wrote:
>On 2023-04-30, Steve McIntyre wrote:
>> On Sun, Apr 30, 2023 at 12:56:29PM -0700, Vagrant Cascadian wrote:
>>>When I tried extending the /dev/vvm/root partition to include more
>>>space from the /dev/vdc1 physical volume, grub fails to load at boot,
>>>unable to find the lvm volume.
>...
>> Can you give us a bit more detail about the system please? With
>> /dev/vdc, this suggests you're in a VM and this is the third drive
>> using virtio? How big is /dev/vdc? What's the partition type?
>
>The disk is about 2TB and the partition type is GPT.
>
>There is also a 1TB disk as /dev/vdb GPT, unused.
>
>The main boot disk is ~500GB /dev/vda MBR.
>
>> Why am I asking? grub-pc is limited by the platform underneath here
>> when it comes to assembling RAID or LVM volumes. If a complex disk
>> setup depends on a drive that can't be seen/read by grub at boot, it's
>> going to struggle. I'm wondering if this might be the underlying cause
>> of your issue.
>>
>> If possible, on your system, could you also reboot and call up a grub
>> command line (hit "c" from the grub menu)?
>
>At the moment, this is difficult as I am now doing some long-running
>builds on it... but will try when I get the chance or maybe reproduce
>the situation in another VM.
>
>
>> From there, I'd love to see what you get if you run "ls" here...

OK, I can reproduce with something like this:

vda - small-ish dos-partitioned disk, fresh bullseye installation made
      using LVM directly for the rootfs, no /boot

(all works)

Then add:

vdb - 8T GPT drive, qcow2, unused
vdc - 8T GPT drive, qcow2, a partition to add to the LVM VG

Rebooting straight away at this point still boots ok.

*However*, using pvmove to move the root LV to vdc fails. I'd expect
similar if you've just extended and grown the rootfs. I get an error:

  error: disk `lvmid/<stuff>' not found

(not copying all of the UUID-style path by hand!)

and this is exactly the kind of thing I was looking for - maybe
grub-pc can't access the drive due to BIOS limitations. *However*, I
think I'm wrong and Pascal Hambourg is right here...!

I ran d-i in rescue mode to get into the system, simply ran
dpkg-reconfigure grub-pc (which will run grub-install *and*
update-grub), and the system now boots again. It looks like what we're
seeing might be a limit in what's built in to the core image by
default. grub-pc is deliberately designed to build minimal images
here, to minimise the chance of the image being too large to embed.

Being brutally honest, I think the problem is in your disk
setup. You've found an edge case that *can* be accomodated and
supported, but needs some extra effort to make it work.

I'd *strongly* recommend re-thinking your setup here. If you're going
to set up complicated storage here *for a VM*, then it would be
trivial to set up a /boot partition which grub will never have
problems finding and booting from. Make your life easier and just do
that...

-- 
Steve McIntyre, Cambridge, UK.                                steve at einval.com
"Further comment on how I feel about IBM will appear once I've worked out
 whether they're being malicious or incompetent. Capital letters are forecast."
 Matthew Garrett, http://www.livejournal.com/users/mjg59/30675.html