[Pkg-xen-devel] Bug#908453: xen-utils-common: README.comet seems to no longer apply to the current Xen packages

[Keith Bare]

Package: xen-utils-common
Version: 4.8.3+xsa267+shim4.10.1+xsa267-1+deb9u9
Severity: important

Dear Maintainer,

   * What led up to the situation?

I needed to build a new Xen DomU.  Since I had seen that Debian had picked
up the 4.8 "comet" changes and pvshim, I wanted to experiment with Xen's PVH
mode and have the DomU run the Linux 4.17 kernel in stretch-backports, which
should be new enough to support PVH.

It seemed like this should work, given instructions in
/usr/share/doc/xen-utils-common/README.comet
-------- 
* Converting a PV config to a PVH config

If you have a kernel capable of booting PVH, then PVH mode is both
faster and more secure than PV or PVH-shim mode.

- Remove any reference to 'builder' (e.g., `builder="generic"`)
- Add the following line:
  type="pvh"
--------


   * What exactly did you do (or not do) that was effective (or
     ineffective)?

I ran xen-create-image (from the xen-tools package) with --dist=stretch to
build the DomU and its configuration.  I also used --pygrub so Xen would
boot the kernel installed in the DomU's filesystem.

I temporarily edited the DomU's Xen/xl configuration to use the pvshim, per
the instructions in README.comet (type = 'pvh' / pvshim = 1), started the
DomU, added the stretch-backports repository, and installed
linux-image-amd64 from backports.  I then shut down the DomU, edited the
DomU's configuration to remove the pvshim = 1 line, and re-started the DomU.


   * What was the outcome of this action?

It appeared the DomU was running in PV mode, despite my having added type =
'pvh' to its Xen/xl configuration.

It was difficult to tell, but I believed the DomU was running in PV mode as
its kernel printed:

 - [    0.000000] Hypervisor detected: Xen PV
 - [    0.000000] Kernel/User page tables isolation: disabled on XEN PV.

Some Internet searching also seemed to indicate that "xl list -l <id>"
should mention "pvh" if the DomU is running in PVH mode.  However, I only
saw:

 - "type": "pv",

There was no mention of "pvh" in the output.


   * What outcome did you expect instead?

I expected the DomU to run in PVH mode.  I wasn't sure what this was
supposed to look like, but I did some experiments (installed the old
4.8.3+comet2+shim4.10.0+comet3-1+deb9u5 packages and worked around
bootloader being broken for PVH) and saw that in fact, with that version of
the Xen packages, the DomU kernel prints:

 - [    0.000000] Hypervisor detected: Xen HVM
 - [    0.000000] Booting paravirtualized kernel on Xen PVH
 - [    0.000000] Kernel/User page tables isolation: enabled

And xl list -l <id> shows:

 - "type": "pvh",


   * More thoughts/discussion.

It looks like the Debian packages lost support for booting DomUs in PVH mode
with version 4.8.3+xsa262+shim4.10.0+comet3-1+deb9u6.  Probably because:

    Update to new upstream version 4.8.3+xsa262+shim4.10.0+comet3.
    (This is the upstream staging-4.8 branch, which is ahead of the
    upstream CI-tested stable-4.8 branch by precisely the three
    most recent XSA fixes.  We are switching away from the special
    upstream 4.8 comet branch.)

And maybe that's fine... if the mitigation comet and the pvshim provided is
also effectively provided by XPTI changes that were present in the
stable-4.8 branch, then I guess it isn't really necessary for anybody to use
(and thus PVH-mode boot) the shim anymore.  However, if that's the case,
then it probably doesn't make sense to include README.comet and continue
packaging the shim in the Debian packages anymore.

The thing that's nefarious (and could be grounds for increasing the bug
severity), is that anybody that followed the README.comet instructions and
configured DomUs to boot a PVH-capable kernel without the shim is now,
probably to their surprise, running their DomU in PV mode.  This means
they've lost Linux's KPTI protections from Meltdown within their DomU.  The
underlying issue is that the xl command seems to silently ignore
configuration directives it doesn't understand--which, without the 4.8
comet2 changes, includes 'type'!

This isn't a huge deal for me and my deployment (at the CMU Computer Club).
Our Xen infrastructure was running jessie/Xen 4.4 at best when the
Meltdown/Spectre news broke.  So our initial mitigation was to switch
everything to run in HVM mode (and we've continued doing so since then).  I
was interested in exploring PVH mode though, since it looked like it was
more similar to PV mode in some ways that would make it work better with
various tooling (e.g., xen-tools).  The fact it didn't work with the Debian
packages the way I thought it would was surprising, and I figured it might
be surprising for other people too.

--Keith Bare


-- System Information:
Debian Release: 9.5
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 4.9.0-7-amd64 (SMP w/16 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages xen-utils-common depends on:
ii  lsb-base        9.20161125
ii  python          2.7.13-2
ii  ucf             3.0036
ii  udev            232-25+deb9u4
ii  xenstore-utils  4.8.3+xsa267+shim4.10.1+xsa267-1+deb9u9

xen-utils-common recommends no packages.

xen-utils-common suggests no packages.

-- Configuration Files:
/etc/default/xendomains changed [not included]

-- no debconf information