[Pkg-xen-devel] Bug#912975: Bug#912975: xen-hypervisor-4.8-amd64: Dom0 crashes randomly without logs on Debian Stretch with Xen 4.8.4

[Hans van Kranenburg]

Hi,

On 11/5/18 12:37 PM, Roalt Zijlstra wrote:
> Package: src:xen
> Version: 4.8.4+xsa273+shim4.10.1+xsa273-1+deb9u10
> Severity: important
> 
> Updating Xen to the latest 4.8 version from the security repo makes servers unstable.

Can you confirm that this is the only change that you made between the
before/after scenario? I mean, if you downgrade the packages, or you
drop the old hypervisor xen-x.y-amd64.gz in /boot again, it's stable again?

> The servers randomly reset without any logs.

Do you have the noreboot option set on the Xen hypervisor command line?

Are you able to configure and capture output from serial console?

First interesting thing to know is if it's the Dom0 that crashes, or if
it's the hypervisor itself, and the logging will tell you that.

> We have serveral Debian Stretch servers running Xen 4.8 and only the ones updated to the 4.8.4+xsa273+shim4.10.1+xsa273-1+deb9u10
> version tend to crash ranging from 'twice a day' to 'once every two weeks'. We have already ruled out if hardware was an 
> issue, since we have 4 individual servers which are different in hardware setup and also were bought at different times. 
> And these servers ran stable with the previsous version 4.8.3+xsa267+shim4.10.1+xsa267-1+deb9u9.
> These servers are acting exactly the same. Every thing works as it should, but without any logs it crashes and resets at 
> a certain point.
> 
> It looks like it could have something to do with DomUs running older (3.16) Linux kernels. As a test we applied 4.9 kernels to 
> all Jessie DomU servers and so far it runs for 13 days (but this server did crash twice on a day). 
> We have seen this behaviour with Xen on CentOS6 and 7 too, but the trouble seems to be fixed after some more updates.

It can be frustrating that there's not much response on the mailing
lists. But, these kinds of problems can be really hard to debug and
solve. Unless there's a clear reproduction scenario and debug output,
there's often noone who can help you remotely.

> As said.. I cannot provide logs since it simply resets without notice.

It's still the best starting point...

Thanks,
Hans