Bug#891328: python-debian: DebFile.changelog() fails to parse DebianVersion (in some cases)

[Philipp Hahn]

Package: python-debian
Version: 0.1.30
Severity: normal

Dear fellow Maintainer,

   * What led up to the situation?

I wrote a small Python tool to comapre the debian/changelog of two Debian
binary package files. It fails on "apache2-suexec-custom" as currently
available in Debian-Sid.

   * What exactly did you do (or not do) that was effective (or
     ineffective)?

>apt download apache2-suexec-custom
>python -c 'from debian.debfile import DebFile
>from sys import argv
>for block in DebFile(argv[1]).changelog():
>   print(block.version)' apache2-suexec-custom_*.deb

   * What was the outcome of this action?

>2.4.29-2
...
>2.0.37-1
>Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
>  File "/usr/lib/python2.7/dist-packages/debian/changelog.py", line 103, in _get_version
>    return Version(self._raw_version)
>  File "/usr/lib/python2.7/dist-packages/debian/debian_support.py", line 213, in __init__
>    super(AptPkgVersion, self).__init__(version)
>  File "/usr/lib/python2.7/dist-packages/debian/debian_support.py", line 111, in __init__
>    self.full_version = version
>  File "/usr/lib/python2.7/dist-packages/debian/debian_support.py", line 137, in __setattr__
>    self._set_full_version(str(value))
>  File "/usr/lib/python2.7/dist-packages/debian/debian_support.py", line 116, in _set_full_version
>    raise ValueError("Invalid version string %r" % version)
>ValueError: Invalid version string '2.0.37+cvs.JCW_PRE2_2037-1'

The underscore character '_' is not valid in
debian/debian_support.py:105
> 105              "(?P<upstream_version>[A-Za-z0-9.+:~-]+?)"

   * What outcome did you expect instead?

No traceback.


Please travel back in time and fix the version number of
"apache2-suexec-custom" before it gets uploaded to Debian ;-)

Seriously: I see several options

- add a LazyVersion like
	from debian.debian_support import Version
	class NonConformantVersion(Version):
	    re_valid_version = re.compile(
	            r"^((?P<epoch>\d+):)?"
	             "(?P<upstream_version>[A-Z_a-z0-9.+:~-]+?)"
	             "(-(?P<debian_revision>[A-Za-z0-9+.~]+))?$")
	import debian.changelog
	debian.changelog.Version = NonConformantVersion

- add an option to ignore invalid/unparseable changelog blocks

-- System Information:
Debian Release: 9.3
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.9.0-5-amd64 (SMP w/4 CPU cores)
Locale: LANG=de_DE.UTF-8, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8), LANGUAGE=de:en_US (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages python-debian depends on:
ii  python          2.7.13-2
ii  python-chardet  2.3.0-2
ii  python-six      1.10.0-3

Versions of packages python-debian recommends:
ii  python-apt  1.4.0~beta3

Versions of packages python-debian suggests:
ii  gpgv  2.1.18-8~deb9u1

-- no debconf information
-------------- next part --------------
#!/usr/bin/python2.7
"""
Compare Debian package changelogs
"""

from __future__ import print_function
from optparse import OptionParser
from debian.debfile import DebFile, DebError
from debian.debian_support import Version
import debian.changelog
from difflib import unified_diff
import re


# Work-around Debian Bug allowing '_' in version string
# /usr/lib/python2.7/dist-packages/debian/debian_support.py:103
class NonConformantVersion(Version):
	re_valid_version = re.compile(
		r"^((?P<epoch>\d+):)?"
		r"(?P<upstream_version>[A-Z_a-z0-9.+:~-]+?)"
		r"(-(?P<debian_revision>[A-Za-z0-9+.~]+))?$")


debian.changelog.Version = NonConformantVersion


class DiffError(Exception):
	pass


def load(deb):
	try:
		debfile = DebFile(deb)
		changelog = debfile.changelog()
		return changelog
	except (DebError, EnvironmentError):
		raise DiffError(deb)


def render(changelog, common_version):
	for block in changelog:
		yield '{0.version} [{0.date}] {0.author}:'.format(block)
		for change in block.changes():
			yield change
		if block.version == common_version:
			break


def diff(old_deb, new_deb):
	old_changelog = load(old_deb)
	old_versions = set(block.version for block in old_changelog)

	new_changelog = load(new_deb)
	new_versions = set(block.version for block in new_changelog)

	common_version = max(old_versions & new_versions)

	old = list(render(old_changelog, common_version))
	new = list(render(new_changelog, common_version))
	return unified_diff(old, new, old_deb, new_deb, lineterm='')


def main():
	usage = '%prog <1.deb> <2.deb>'
	parser = OptionParser(usage=usage, description=__doc__)
	opt, args = parser.parse_args()
	try:
		old_deb, new_deb = args
	except ValueError:
		parser.error('Expecting exactly 2 Debian binary package files')

	for line in diff(old_deb, new_deb):
		print(line)


if __name__ == '__main__':
	main()