[parted-devel] Copyright assignment form for parted

[Al Dunsmuir]

On Wednesday, September 30, 2015, 8:42:28 AM, Phil Susi wrote:
> On 9/25/2015 12:40 AM, Al Dunsmuir wrote:
>> On Thursday, September 24, 2015, 9:15:52 AM, Phil Susi wrote:
>>> < let's keep this on the list >
>> Sorry... my email client defaulted to your direct email.

> Get in the habit of using reply-to-all.  You want all of those on the Cc
> list to stay on the Cc list unless you have a darn good reason for
> excluding them from the conversation going forward, and I didn't see
> this reply until now because I didn't get the direct reply and don't
> check all of my mailing lists every day.

Sorry,  Fighting  a  cold, focused on the contents and missed checking
the reply addresses set by my email client. Again.

>>> How would such a thing make any sense?  It isn't a filesystem.  You
>>> can't have it on a non APM partitioned disk; it only has meaning in
>>> conjunction with the partition table.  Any checking of it should be done
>>> by a partitioning tool accessing the raw disk instead of the partition.
>> 
>> I'm  not  sure  in which area(s) I'm not making sense.  I'm not saying
>> they don't exist 8^), merely that I'm not clear in my understanding.
>> 
>> The  map  partition  is not hidden on OS/X, and deciding to hide it on
>> Linix at this point in time seems like more trouble than it is worth.
>> 
>> I'm  treating  it  as a filesystem that is only valid for mac volumes,
>> and  in  fact  is  mandatory  for  mac  volumes. It gets automatically
>> created  when the mac volume is created, and must be kept in sync with
>> events such as creation, deletion, or movement of any other partitions
>> on the mac volume.

> What I am saying is that it is not a filesystem at all, and can not be
> checked on its own, independent of the main partition table.  That is,
> if you dd'd your map partition to a partition onto a GPT partitioned
> disk, you couldn't fsck it.

There  is  _no_  other  partition  table  on a a mac (APM) volume.

The  APM  volmue  begins with the driver table (Driver Desriptor map),
which is normally 1 sector.

After that, the actual partitions begin.   The first partition must
be    the   Apple_partition_map   partition (I rechecked the docs, and
there is no other metadata that would allow it to be elsewhere).

The map partition is an array of elements which either represent:
- active  partitions:  The  partition  map, and all other conventional
  filesystems such as HFS, HFS+, ext4, etc.
- free  partitions:  Maps  any  ranges  of free space not in an active
  partition.  The  Apple  Disk  Utility  normally  creates  small free
  partitions  between  active partitions. This allows convenient later
  creation of boot partitions for dual booting.
- unused  map  elements:  Used  to map new partitions created when new
  partitions are created by splitting an existing free extent.

The  partition  map  element  that  describes  the Apple_partition_map
partition  does not have to be the first element. It just has to claim
ownership  of  the sectors for the first physical partition. There can
only be one partition map element.

In realizing that the Apple_partition_map partition must be the first
partition, I also now see that you are correct that that partition is
really metadata for the mac volume.

There  are  Linux filesystem-specific fsck utilities for hfs and hfs+,
but  I don't believe there is currently a fsck utility to sanity check
the  actual map partition. I don't know if there is any sort of volume
level  fsck  for  MBR or GUID disks.

A volume-level fsck utility for a mac volume would check the partition
map  elements  to  ensure that they do not overlap (sectors), that all
disk sectors are claimed, and that all elements have a recognized type
string value.

I'm  not  sure if the OS/X 10.5 Disk Utility source is available. Much
of  the  Darwin  base  OS  code  is.  If it was, it would provide some
insight into what problems can exist, and how it repairs them.

>> Swap  isn't  a  much  more  complex filesystem, but libparted supports
>> multiple type (3 Linux variants, and I believe 1 BSD). This goes along
>> with the "swap" flag.

> Yes, but you can choose to have a swap partition or not ( or multiple
> ones ), and swap is still swap no matter what partition type it is in,
> and it can be mounted.  With the map partition, there can be one and
> only one, and only for a mac partitioned disk, and its contents are
> useless to anything but a mac partition table editor.

>>> Speaking of that, doesn't blkid already identify the usage type of this
>>> partition correctly and gparted could use that?
>> 
>> blkid returns PARTLABEL="Apple". meh.

> Even blkid -p /dev/sda?  It should indicate ID_FS_USAGE...

All that is returned for blkid -p /dev/sde is:
/dev/sde: PTTYPE='mac'

I suspect that the support for mac volumes is pretty nominal in many
areas, with plenty of opportunity for improvements.

What did you think would be returned?

>> lsblk  (the  new  utility  the  udev  developers  want  folks  to use)
>> recognizes  it  as  partition  type  "Apple_partition_map",  and label
>> "Apple".

> That's just a user friendly frontend to blkid.  If it sees that
> Apple_partition_map type, so should blkid and so gparted could use that.

It looks like blkid and lsblk don't show that much useful information.
There  is  likely  a long list of things that could be done to improve
support for mac volumes (and hfs/hfs+). I may actually accomplish some
of that 8^).

It  sounds  like I should write off what I've done so far with parted,
gparted and pyparted and approach it from the metadata angle.

I  suspect that the correct approach with gparted is to have libparted
report  the  Apple_partition_map  partition as PED_PARTITION_METADATA,
not  PED_PARTITION_NORMAL.  That  alone  would  prevent  gparted  from
offering the delete and reformat operatons. If gparted doesn't already
suppress  displaying  metadata  partitions, I could change it to do so
for metadata partitions on mac volumes.

>> gparted  initially  uses  libparted to obtain partition and filesystem
>> information.  It  defaults  to  allowing  deletion/reformatting of any
>> partition  with an unknown type - OK from a brute force POV for simple
>> partitions,  but  when the partition controls all the other partitions
>> on the volume it becomes risky.

> Come to think of it... even if gparted doesn't disable the deletion, why
> the hell doesn't libparted refuse to do it and return an error code?  It
> should at least do that.

Agreed.  Marking it as PED_PARTITION_METADATA sounds like a good first
step  in  that  direction - it may trigger existing protection code in
libparted  (or  there  may  be  a  general  need  to  protect metadata
partitions).

>> It  seems  like many tools expect active (and important) partitions to
>> also  have  a  filesystem type.  Maybe it's a bad assumption, but that
>> the way they tools are currently implemented.

> Yes, but it lets other sources of information like blkid supersede that
> provided by libparted.

And libblockdev.

>> Having  libparted  mark  the Apple_partition_map partition with a flag
>> called  "map" gives users useful info. Recognizing a "fake" filesystem
>> type  of  "apple-map"  (with alias "Apple_partition_map") provides the
>> rest.  Adding  a  few  lines  of  code  to  protect  against deletion,
>> reformatting and changing the flags was very simple.
>> 
>> The  blivet  and  blivet-gui  packages  make use of libparted (through
>> pyparted).   That's  my  main  interest,  so  I wanted to use the same
>> source  of information.  blivet now also uses libblkdev, so I may need
>> to add some code (plugin?) there too.

> The problem I have is that this seems to be outside of the intended
> usage for both filesystem type and flags.  The filesystem type is
> supposed to identify which one of many filesystems is present in the
> partition currently, and you can choose to format a different one there
> at any time.  That isn't really so with the map partition.  The flags
> are supposed to be special toggles that the user can flip when they need
> to force a partition table specific behavior, that is irrespective of
> any filesystem.  If the user can't set or clear the flag, then why have it?

My  original  intent  was  to make the already-visible partition entry
useful,  complete,  and  to  protect  it  from  damage. Completing the
sectors used/unused info would have also allowed display of number and
% of partition entries in use.

> Since they can't do anything with it, wouldn't it be much simpler and
> less confusing to just not show the partition to the user in the first
> place?

Yes.

Realizing though our discussion that the Apple_partition_map partition
must  be  the first partition on a mac volume has convinced me that it
is indeed just volume metadata from the end-user's point of view.

Marking  the  map  partition as metadata and hiding it from display in
most disk utilities eliminates the problem. It's overhead, and only at
the start of the mac volume so will not interfere with user conversion
of  various  free  space  partitions (which are in the map, but hidden
from display) into new active filesystems.

The changes I've made to the disktype utility give me something that I
can  use  to  display  the  actual  underlying partition map elements.
That's  all  I  really  need for now to validate parted/libparted/etc.
changes.

> I suppose if it must be one or the other, it should be just the flag.
> Why have both?  It also wouldn't make sense to report an apple partition
> map filesystem found on a GPT disk, or to report a fat filesystem if you
> found one in what is supposed to be the mac partition map partition
> would it?

The  gparted  utility  currently flags any visible partition without a
filesystem  as  incomplete. Making it a filesystem provided a means to
display the rest of the information like the other partitions, even if
it was not very interesting to most folks.

Taking  the  other  approach of marking the map partition as metadata,
and  doing the minimal work in gparted to hide that partition seems in
the  end to work better. As you mentioned earlier, libparted does need
to  prevent  someone  from  trying  to delete or reformat that special
partition type.   I'll focus "take 2" on that.

Al