[Python-modules-team] Bug#975872: Deletes trailing whitespace in e.g. "c: ZCA="

[Trent W. Buck]

Package: python3-ldif3
Version: 3.2.2-1
Severity: normal

I wrote a trivial ldifsort tool (attached).
I noticed it was not idempotent.
This is because python3-ldif3 is not preserving trailing space.

An easy way to see this is:

    diff -u <(printf 'dn: a=b\nc:: ZCAgIA==\n'|ldifsort) \
            <(printf 'dn: a=b\nc:: ZCAgIA==\n'|ldifsort|ldifsort) | cat -A
    --- /dev/fd/63^I2020-11-26 13:48:36.865826580 +1100$
    +++ /dev/fd/62^I2020-11-26 13:48:36.865826580 +1100$
    @@ -1,3 +1,3 @@$
     dn: a=b$
    -c: d   $
    +c: d$
     $

It *is* correctly preserving leading spaces, and leading and trailing newlines.

Please investigate why the ldif3 writer is not using base64 to preserve trailing space.


-- System Information:
Debian Release: 10.6
  APT prefers stable
  APT policy: (990, 'stable'), (500, 'stable-updates'), (500, 'proposed-updates')
Architecture: amd64 (x86_64)

Kernel: Linux 5.6.0-0.bpo.2-amd64 (SMP w/2 CPU cores)
Locale: LANG=en_AU.utf8, LC_CTYPE=en_AU.utf8 (charmap=UTF-8), LANGUAGE=en_AU:en (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages python3-ldif3 depends on:
ii  python3  3.7.3-1

python3-ldif3 recommends no packages.

python3-ldif3 suggests no packages.

-- no debconf information
-------------- next part --------------
#!/usr/bin/python3
import collections

import click
import ldif3

__doc__ == """GOAL: ldifsort.py, but less ugly/annoying/buggy

See also https://ldapwiki.com/wiki/LDIF%20Sort%20Tools

FIXME: ldif3 incorrectly fails to preserve base64 escaping of trailing spaces.
       Leading spaces and leading/trailing newlines ARE preserved.

       This is a simple test case:

           diff -u <(printf 'dn: a=b\nc:: ZCAgIA==\n'|ldifsort) \
                   <(printf 'dn: a=b\nc:: ZCAgIA==\n'|ldifsort|ldifsort)|cat -A
"""


@click.command(
    help="""
    To sort prior BEFORE git committing:

        ldapsearch -LLL -YEXTERNAL '*' + | ldifsort - backup.ldif

    To sort AFTER git committing (at "git log -p" time):

        git config --global diff.ldif.textconv ldifsort
    """)
@click.argument('src', type=click.File('rb'), default='-')
@click.argument('dst', type=click.File('wb'), default='-')
def main(src, dst):
    # Read everything into memory.
    reader = ldif3.LDIFParser(src)
    entities = sorted(          # sort objects (entities)
        (dn, collections.OrderedDict(
            sorted((attribute_name,           # sort attribute names
                    sorted(attribute_values))  # sort attribute values
                   for attribute_name, attribute_values in attributes.items())))
        for dn, attributes in reader.parse())
    # Write everything out again.
    writer = ldif3.LDIFWriter(dst)
    for pair in entities:
        writer.unparse(*pair)


if __name__ == '__main__':
    main()