[Qa-jenkins-dev] Bug#875990: Bug#875990: reproducible: i/o issues with profitbricks-build2-i386 since stretch upgrade

[Vagrant Cascadian]

On 2017-09-17, Vagrant Cascadian wrote:
> On 2017-09-17, Holger Levsen wrote:
>> On Sat, Sep 16, 2017 at 07:48:42PM -0700, Vagrant Cascadian wrote:
>>> My *hunch* is that build2 and build12 are running a PAE kernel with more
>>> than 8GB of ram, and affected by this kernel bug (introduced in linux
>>> ~4.2, possibly):
>>>   https://bugzilla.kernel.org/show_bug.cgi?id=196157
>>>   https://bugs.launchpad.net/ubuntu/+source/linux-hwe/+bug/1698118
>>  
>> indeed!
>
> Not sure if it makes sense to also file a bug in the debian bug tracker
> about this...

I did end up filing https://bugs.debian.org/876035 and the response so
far was only to downgrade it as minor, due to the unusual use-case of
running i386 userspace with a PAE instead of amd64 kernel these days.


>>> Reducing the ram of the affected builders to 8GB and having more PAE
>>> builders with lighter workloads might be a workaround that would get
>>> better performance... while still testing 32/64-bit kernel
>>> variation.
>>
>> we lack the diskspace to do so.
>
> Then it might still get better performance to lower the PAE builders to
> only use 8GB of ram, even if that means running fewer jobs in
> parallel...

Again, I think simply lowering the ram to 8GB might actually result in
better performance, as it's a non-linear degredation. Looking at the ram
usage patterns of the i386 builders, as they infrequently use more than
8GB:

  https://jenkins.debian.net/munin/debian.net/profitbricks-build12-i386.debian.net/memory.html

Or even 12GB or 16GB, though that will trigger the i/o wait issue more.


>> another option is to just wait. :/
>
> I suspect that *might* be an infinite wait; I get the impression this is
> a very low-priority issue upstream, and it would take some active
> attempt to fix it upstream...

Haven't seen any progress on issue in Debian or upstream, several months
later...


live well,
  vagrant
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/qa-jenkins-dev/attachments/20180121/01dc180c/attachment.sig>