[Python-modules-team] Bug#806826: pyyaml does not support literals in unicode over codepoint 0xffff

[John R. Lenton]

Source: pyyaml
Severity: normal
Tags: upstream patch

Dear Maintainer,

the yaml spec says that

   “The allowed character range explicitly excludes the surrogate
    block #xD800-#xDFFF, DEL #x7F, the C0 control block #x0-#x1F
    (except for #x9, #xA, and #xD), the C1 control block #x80-#x9F,
    #xFFFE, and #xFFFF.”

however pyyaml has chosen to negate that check and apply it to only
plane 0. This means that any yaml document that contains unicode
literals in higher planes will fail to parse (and, on output, use the
rather unfriendly \Uxxxxxxxx format).

The attached patch fixes this in a minimally intrusive way, by
extending the checks to cover the additional codepoints where
appropriate. A better fix would be to use the check as the spec
specifies it, but that would be a bigger change.

I tried to file this upstream first, but was unable to find a way to
report bugs there; sorry.

-- System Information:
Debian Release: stretch/sid
  APT prefers xenial-updates
  APT policy: (500, 'xenial-updates'), (500, 'xenial-security'), (500, 'xenial'), (100, 'xenial-backports')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.2.0-19-generic (SMP w/4 CPU cores)
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: wide-unicode.diff
Type: text/x-diff
Size: 3257 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/python-modules-team/attachments/20151201/ab28bc4e/attachment-0001.diff>