Bug#1035317: grub-pc: /boot on LVM fails if logical volume consists of multiple physical volumes

[Pascal Hambourg]

On Sun, 30 Apr 2023 17:37:13 -0700 Vagrant Cascadian 
<vagrant at debian.org> wrote:
> On 2023-05-01, Steve McIntyre wrote:
>> From there, I'd love to see what you get if you run "ls" here...
> 
> "ls" from the grub prompt did not show the other disk...
> ...until I made the second disk bootable from libvirt!
> Then grub now sees both disks, and boots fine!
> So this is possibly a quirk of the way libvirt exposes boot disks.

Apparently. GRUB can only see disks exposed by the BIOS/UEFI/other 
platform firmware. There are other known situations where a given 
firmware may not expose some disks, including but not limited to:
- disks connected to a SATA controller card without a BIOS expansion ROM
- unsupported media types: USB other than the boot disk, NVMe, SD/MMC...

The UEFI firmware on an old Intel board only had EFI drivers for the 
SATA controller in IDE mode and lacked EFI drivers for AHCI and USB. It 
has been reported that more recent boards had support for NVMe only in 
EFI mode, not in legacy mode.

>> I ran d-i in rescue mode to get into the system, simply ran
>> dpkg-reconfigure grub-pc (which will run grub-install *and*
>> update-grub), and the system now boots again. It looks like what we're
>> seeing might be a limit in what's built in to the core image by
>> default. grub-pc is deliberately designed to build minimal images
>> here, to minimise the chance of the image being too large to embed.

I wonder how much this policy is still relevant for PC platforms. 
Originally the core image was designed to fit in the "post-MBR gap" 
whose typical size was 62 sectors (31 KiB) because the first partition 
used to start at sector 63. But nowadays a 1-MiB post-MBR gap has been 
the standard for many years. I do not remember when this was introduced 
in fdisk and other Linux partition editors, but Windows 7 installer had 
it. Besides, I observe that the size of the core.img built for LVM+ext4 
on my bullseye system is 34 KiB so it would not even fit in a 31-KiB 
post-MBR gap.

The minimal core image policy is even less relevant for EFI images, as 
the EFI partition size is usually several MB so a few more kB won't 
hurt. I cannot tell for other platforms.

I think that when building the i386-pc core image with support for 
storage possibly involving multiple disks (LVM, RAID, btrfs), support 
for at least both MSDOS and GPT partition schemes (other partition 
schemes are unlikely to be used on PC) could be added unconditionally to 
prevent such GRUB failure after adding a disk with a different partition 
scheme to the /boot filesystem. It would add only 2 KB to the core 
image, and it is likely that the minimal size is already above 31-KiB 
anyway when the above storage drivers are embedded. Opinions ?

> Re-running grub-install /dev/vda from debian-installer rescue mode did
> *not* fix the issue for me

Because there were two issues preventing GRUB from seeing the new PV:
- the disk was not exposed by libvirt BIOS
- the disk had an unsupported GPT partition scheme

grub-install fixed only the second issue. Making the disk "bootable" in 
libvirt was required to fix the first issue.

> (though, now I am curious if dpkg-reconfigure
> grub-pc would do anything more?)

As Steve wrote, dpkg-reconfigure also runs update-grub to rebuild 
grub.cfg. AFAIK here the only difference with the old grub.cfg is 
additional "insmod part_gpt" commands to load GPT support, but the 
module must already be embedded in the core image so this addition is 
not required.