Bug#913274: Incorrectly parsing whitespace in Sources.iter_paragraphs

[Stuart Prescott]

Hi Marcus,

Thanks for this interesting report.

> I have come across a case where whitespace is added in
> Packages{.gz,.bz2} and I am not sure how it should be parsed.
[...]
> Should this whitespace be parsed as a paragraph delimiter?

For a Packages file, each paragraph is defined as a set of DEBIAN/control 
paragraphs; the Description field is not allowed to contain lines that are 
whitespace-only.

https://wiki.debian.org/DebianRepository/Format#A.22Packages.22_Indices

https://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-description

So the strict answer is yes, it should be a paragraph delimiter but most 
implementations seem to be more forgiving in what they accept.

Note that for debian/control files in source packages, whitespace-only lines 
are treated as paragraph separators so that whitespace errors in an editor 
don't accidentally make packages disappear from the archive.


> Currently, the whitespace is being treated as a paragraph delimiter,
> in python-debian, but not by apt-get, etc.

Could you expand on this with an example, perhaps?

python-debian actually uses python-apt for dealing with Sources and Packages 
files (i.e. the exact same code as apt) and already does treat whitespace-only 
lines as being part of a paragraph rather than breaking them:


$ ipython3 
Python 3.6.7 (default, Oct 21 2018, 08:08:16)  
Type "copyright", "credits" or "license" for more information. 

In [1]: from debian.deb822 import Packages 

In [2]: with open('Packages') as fh: 
  ...:     for p in Packages.iter_paragraphs(fh): 
  ...:         if p['Version'] == '1.25.0-1529904044': 
  ...:             print(p) 
  ...:              
Package: code-insiders 
Priority: optional 
Section: devel 
Installed-Size: 213952 
Maintainer: Microsoft Corporation <vscode-linux at microsoft.com> 
Architecture: amd64 
Version: 1.25.0-1529904044 
Replaces: visual-studio-code-insiders 
Provides: visual-studio-code-insiders 
Depends: libnotify4, libnss3, gnupg, apt, libxkbfile1, libgconf-2-4, 
libsecret-1-0, libgtk-3-0 (>= 3.10.0) 
Conflicts: visual-studio-code-insiders 
Filename: pool/main/c/code-insiders/code-insiders_1.25.0-1529904044_amd64.deb 
Size: 46342528 
MD5sum: 6cd7bbd2eebcae2ebddf95bba4627681 
SHA1: 793a36306fa6f78b7d20b1a8e99458fcdcd5221c 
SHA256: 7c75fece717778bc3fc3f86efc77c4cd7a717f05e23e3f23d86442d3c95365fc 
SHA512: 
19196ecdedd0306dcb1504d51bbb8ece9bcdac224e34623992e0cfda3fa43ed4a0def10a0200ae1b16652ec772e9265d2024f7ba80f
294e1cec57184f9213432 
Description: Code editing. Redefined. 
Visual Studio Code is a new choice of tool that combines the simplicity of a 
code editor with what developers need
for the core edit-build-debug cycle. See https://code.visualstudio.com/docs/
setup/linux for installation instructi
ons and FAQ. 
 
Homepage: https://code.visualstudio.com/ 

Cheers
Stuart


-- 
Stuart Prescott    http://www.nanonanonano.net/   stuart at nanonanonano.net
Debian Developer   http://www.debian.org/         stuart at debian.org
GPG fingerprint    90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7