[parted-devel] [parted 1.8.8] msdos label and file system corruption issues with > 2TB disk

[Michael Reed]

(I'm copying parted-devel as this ties into GPT-MBR hybrid discussion in that
there is a need to be able to boot BIOS based systems from disks larger than
2TB and still be able to address all the storage.)

Using an on board hardware raid5 "disk" of 5,851,561,984 512 byte sectors,
or roughly 2.9TB, we installed name_brand_Enterprise_distro.  We created an
800 MB swap partition (1), a 25GB root partition (2), and the rest of the space
to a large scratch xfs file system.

The OS installation went fine and upon reboot the system complained that the
large scratch file system was, essentially, corrupt.  Here's my analysis of
what happened.

Via analysis of the MSDOS partition table, I can state that it can address
a maximum of 4TB of space.  Each partition entry consists of a starting block
and a length.  Both of these fields are 32 bits in length.  No individual
partition can begin beyond 2TB.  No individual partition can be longer than
2TB.  In order to address 4TB, at least two partition table entries must be
used, for example:

	start		length
	0x00000000	0xfffffffe
	0xffffffff	0xffffffff

(It's interesting to note that Linux will handle 4TB of space defined via
the msdos label, and Windows 2003/2008 appears to stop at 2TB.)

Working with parted after the system was up I can draw these conclusions:

	1) parted doesn't complain that a partition length is too long
	   for the msdos label in which it will be stored.

	2) parted "truncates" the partition

	3) parted "lies" to the OS about the size of the partition.

It's item three that's the worst problem as it allows the OS to properly
address the partition, and allows the user to make a file system and then
load it with data.  Following a reboot, the OS sets the partition table
to the "truncated" length and the file system is no longer intact.

Example:

(parted) mkpart p xfs 33GB 2996GB
(parted) p
Disk geometry for /dev/sda: 0kB - 2996GB
Disk label type: msdos
Number  Start   End     Size    Type      File system  Flags
1       1kB     8000MB  8000MB  primary   linux-swap   type=82
2       8000MB  33GB    25GB    primary   xfs          type=83
3       33GB    797GB   764GB   primary   xfs          type=83
(parted) quit
Information: Don't forget to update /etc/fstab, if necessary.

labrat2:~ # cat /proc/partitions
major minor  #blocks  name

   8     0 2925780992 sda
   8     1    7812500 sda1
   8     2   24414062 sda2
   8     3 2893408256 sda3   # (*2 for sectors) = TOO BIG

Notice the discrepancy between the partition size reported by parted
and the size reported by the OS for sda3.

parted does not appear to provide an adequate bounds check on its
input ending address to assure it does not overflow the partition
table.

The cross over ending addresses for fail/pass are 

>	mkpart primary xfs 33Gb 2232GB
>	mkfs and reboot, the file system will mount.
>	but if I use:
>	mkpart primary xfs 33Gb 2233GB
>	it will not mount after a reboot.


This problem has been confirmed to be present in both the parted provided
by two major distros as well as version 1.8.8 downloaded from gnu.org.

I should mention at this point that we no longer have the system available
for debugging.  I can probably improvise a configuration using "md" if
you don't have adequate hardware at your disposal.

Let me know if I can be of help in resolving this issue.

Thanks,
 Mike Reed