Bug#585393: Please be more robust against bogus data in a deb822 file

[Michael Vogt]

Package: python-debian
Version: 0.1.16
Severity: normal

It appears that the deb822.Deb822.iter_paragraph method gets confused
if there are bogus entries (like a single line) in the file. Below is
a test that shows the behavior. Depending on the policy the excpeted
value is either 2 or 3 (2 if we want to discard invalid stanzas).

It appears that the problem is "while len(x) != 0" in deb822.py, that
will make the parser stop on the first bogus line. Attached is a
possible patch for this that makes the EOF handling explicit. 

Thanks,
 Michael

--- a/lib/debian/deb822.py	2010-03-16 02:06:28.000000000 +0100
+++ b/lib/debian/deb822.py	2010-06-10 10:51:30.000000000 +0200
@@ -241,7 +241,7 @@
             (All values are given back as unicode objects, so an encoding is
             necessary in order to properly interpret the strings.)
         """
-
+        self.eof = False
         if hasattr(sequence, 'items'):
             _dict = sequence
             sequence = None
@@ -249,12 +249,11 @@
             _dict = None
         Deb822Dict.__init__(self, _dict=_dict, _parsed=_parsed, _fields=fields,
                             encoding=encoding)
-
         if sequence is not None:
             try:
                 self._internal_parser(sequence, fields)
             except EOFError:
-                pass
+                self.eof = True
 
         self.gpg_info = None
 
@@ -285,7 +284,7 @@
         else:
             iterable = iter(sequence)
             x = cls(iterable, fields, encoding=encoding)
-            while len(x) != 0:
+            while not x.eof:
                 yield x
                 x = cls(iterable, fields, encoding=encoding)
 


--- a/tests/test_deb822.py	2010-03-16 02:06:28.000000000 +0100
+++ b/tests/test_deb822.py	2010-06-10 09:59:59.000000000 +0200
@@ -235,6 +235,12 @@
 -----END PGP SIGNATURE-----
 '''
 
+BOGUS_DATA = '''
+
+xxx-bogus-entry
+
+'''
+
 class TestDeb822Dict(unittest.TestCase):
     def make_dict(self):
         d = deb822.Deb822Dict()
@@ -705,6 +711,15 @@
         self.assertEqual(utf8_contents, latin1_to_utf8.getvalue())
         self.assertEqual(latin1_contents, utf8_to_latin1.getvalue())
 
+    def test_bogus(self):
+        text = (UNPARSED_PACKAGE + '\n\n\n' + 
+                BOGUS_DATA + '\n\n\n' + 
+                UNPARSED_PACKAGE).splitlines()
+        l = []
+        for d in deb822.Deb822.iter_paragraphs(text, use_apt_pkg=True):
+            l.append(d)
+        self.assertEqual(len(l), 3)
+
 class TestPkgRelations(unittest.TestCase):
 
     def test_packages(self):


-- System Information:
Debian Release: squeeze/sid
  APT prefers maverick
  APT policy: (500, 'maverick'), (500, 'lucid-security'), (500, 'lucid')
Architecture: i386 (i686)

Kernel: Linux 2.6.32-22-generic (SMP w/2 CPU cores)
Locale: LANG=en_DK.UTF-8, LC_CTYPE=en_DK.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages python-debian depends on:
ii  python                 2.6.5-0ubuntu1    An interactive high-level object-o
ii  python-apt             0.7.94.2ubuntu6.1 Python interface to libapt-pkg
ii  python-support         1.0.4ubuntu1      automated rebuilding support for P

python-debian recommends no packages.

Versions of packages python-debian suggests:
ii  gpgv                     1.4.10-2ubuntu1 GNU privacy guard - signature veri

-- no debconf information