Bug#764798: grub2: Grub rescue shell with RAID 6 mdadm over 8 disks

[Steve McIntyre]

Control: severity -1 important
Control: retitle -1 GRUB /boot on large software RAID systems may break due to BIOS limitations

On Sat, Dec 13, 2014 at 04:01:20PM +0100, Jeroen Dekkers wrote:
>Control: tag -1 moreinfo
>
>Hi Mike,
>
>On Thu, 23 Oct 2014 18:20:00 -0500 "Mike" <ctrlbrk at yahoo.com> wrote:
>> Greetings, anything else needed from me?
>
>I don't have VMWare, but tried to reproduce it using KVM (the host
>machine is running wheezy). When I configure 10 SATA disks I see all
>of them in grub, but when I use VirtIO disks I only see 7 of
>them. Turning debugging on and looking further I see that the bios
>simply gives an error when trying to access disk 8 and the boot menu
>also only shows 7 disks - so in this case the problem is in the bios,
>not in grub.
>
>Can you turn on debugging in the grub console using 'set pager=1' and
>'set debug=all' and give the output? The debug information from
>biosdisk.c is the most interesting. If the problem is that the bios
>doesn't support more than 8 disks there is not much we can do except
>for documenting it and give a warning for RAID arrays with more than 8
>disks.

I'm working on debugging this, and I can reproduce issue with 12 disks
in a RAID6 in qemu/KVM. Adding the suggested debug shows that things
are definitely failing due to BIOS limitations (qemu only shows 4 BIOS
disks), not GRUB limitations. If I change my qemu/KVM config to move
most of the disks to be emulated USB storage (slow!), things work just
fine and the system boots correctly.

Based on that, I'm downgrading this bug to important. I'm afraid that
there's nothing we can do directly to *fix* it ourselves, as it's not
caused by something in GRUB. What would be nice is a warning at
grub-install(?) time if lots of devices are needed for /boot: "your
system may not boot like this".

Mike: I'm afraid the only way you're going to get your system back
properly is to re-arrange your filesystems. Maybe re-shuffle the disks
to get a smaller RAID1 for /boot, or add a USB stick/disk for /boot
only?

-- 
Steve McIntyre, Cambridge, UK.                                steve at einval.com
Into the distance, a ribbon of black
Stretched to the point of no turning back