[parted-devel] [parted 1.8.8] msdos label and file system corruption issues with > 2TB disk

[Jim Meyering]

Michael Reed <mdr at sgi.com> wrote:
> (I'm copying parted-devel as this ties into GPT-MBR hybrid discussion in that
> there is a need to be able to boot BIOS based systems from disks larger than
> 2TB and still be able to address all the storage.)
>
> Using an on board hardware raid5 "disk" of 5,851,561,984 512 byte sectors,
> or roughly 2.9TB, we installed name_brand_Enterprise_distro.  We created an
> 800 MB swap partition (1), a 25GB root partition (2), and the rest of the space
> to a large scratch xfs file system.
>
> The OS installation went fine and upon reboot the system complained that the
> large scratch file system was, essentially, corrupt.  Here's my analysis of
> what happened.
>
> Via analysis of the MSDOS partition table, I can state that it can address
> a maximum of 4TB of space.  Each partition entry consists of a starting block
> and a length.  Both of these fields are 32 bits in length.  No individual
> partition can begin beyond 2TB.  No individual partition can be longer than
> 2TB.  In order to address 4TB, at least two partition table entries must be
> used, for example:
>
> 	start		length
> 	0x00000000	0xfffffffe
> 	0xffffffff	0xffffffff
>
> (It's interesting to note that Linux will handle 4TB of space defined via
> the msdos label, and Windows 2003/2008 appears to stop at 2TB.)
>
> Working with parted after the system was up I can draw these conclusions:
>
> 	1) parted doesn't complain that a partition length is too long
> 	   for the msdos label in which it will be stored.
>
> 	2) parted "truncates" the partition
>
> 	3) parted "lies" to the OS about the size of the partition.

Thank you for the detailed bug report!
With the patch below, I've fixed Parted so it won't do that any more.

The patch to parted is deceptively simple looking.
Just finding the right place in the call tree to insert the
tests was not easy.  Name/semantic mismatches (and lack of comments)
led me to experiment with two others first.

An early version of the patch did not have this guard condition:

  if (!(part->type & PED_PARTITION_METADATA)

Everything appeared to work until I tried to *print*
a partition table containing the largest (2TB-1-sector) partition.
Without the guard, it'd complain about the starting sector being too large.
The actual sector number may have been 2^32+2k.

Designing the tests was a challenge, too.
Just creating a file that parted can use as a "device" in which to
create such a large partition (>= 2TB) is tricky, since many common
file system types have a maximum file size of just under 2TB.
But thanks to sparse files, the actual space required is just a few MB.

Since I ended up choosing to create and mount an XFS partition, the test
does useful work only when run by root.  You may also need to install
the xfsprogs package to get mkfs.xfs.  You can run the test manually
like this:

  (cd tests && sudo ./t4100-msdos-partition-limits.sh --verbose)

Here's the change-set I expect to push today or tomorrow:
-----------------