[Piuparts-devel] Bug#847277: lava-server: fails to upgrade from jessie to stretch

Wed Feb 8 21:25:46 UTC 2017

On Wed, 8 Feb 2017 21:03:20 +0100
Andreas Beckmann <anbe at debian.org> wrote:

> On 2017-02-08 20:16, Neil Williams wrote:
> > retitle 847277 django migrations in jessie require django from  
> jessie-backports to upgrade to stretch
> [...]
> 
> > On Wed, 08 Feb 2017 14:48:21 +0100
> > Andreas Beckmann <anbe at debian.org> wrote:  
> 
> > The flow needs to go via jessie-backports, just like any django
> > project in jessie as the database migrations from jessie cannot be
> > handled by django1.10 without going via the LTS release, django1.8.
> > The bug, if any, is in django1.7 which does not (cannot) know how
> > to handle database migrations the way that django1.8 (and
> > subsequent releases) need to handle migrations. However, that
> > change can't be backported to jessie except by using the new flow
> > from 1.8 which breaks all other packages using django in jessie.
> > 
> > On a real instance, the postgres cluster would also need to be
> > migrated.  
> 
> I've started doing this in piuparts recently ...

Interesting. When tested in piuparts, lava-server checks to see if the
database contains only the migrations and no actual objects. It would
be good for piuparts to support purge leaving a database behind as this
is the expected behaviour when the installation created the empty
database but the user created (valuable) data afterwards. There is
nothing the package can do in an automated manner when a database
migration goes wrong anyway, even repeating the attempt could cause
permanent data loss.

> > If piuparts cannot handle such upgrades, that must be a bug in
> > piuparts.  
> 
> Having uncommon upgrade requirements (beyond
>     sed -i s/jessie/stretch/ sources.list
>     apt-get update
>     apt-get dist-upgrade
> ) will need extra coding ...

.. or a control file in debian/ or debian/migrate/ maybe.

backports: jessie
Depends: python-django, python-django-tables2, $@

?

At least if it is restricted to official backports, it isn't another
open-ended maintainer script doing random things.

The extra steps in our test job are to ensure that the final state is a
fully working package - the a2enmod and other apache steps are not
required to allow the package to setup correctly. All that's actually
needed in a piuparts type test is to have the specified packages
installed on top of jessie before adding the stretch apt source. To me,
that would seem to be a useful option. (Syntax borrowed from autopkgtest.)

> > We test this regularly, albeit to the newer version in our staging
> > repo rather than the version in stretch.   
> 
> OK.
> 
> > https://staging.validation.linaro.org/scheduler/job/162057/definition#defline59
> > 
> > # on jessie, with backports enabled:
> > DEBIAN_FRONTEND=noninteractive apt-get -y install lava-dispatcher
> > lava-server ....
> > apt -q -y -t jessie-backports install python-django-kvstore
> > python-django python-django-tables2 lava-server # now proceed with
> > the upgrade. DEBIAN_FRONTEND=noninteractive apt -y upgrade
> > DEBIAN_FRONTEND=noninteractive apt -y dist-upgrade  
> 
> Given that you are continuously testing your special upgrade path,
> I'll not reimplement that in piuparts for now (although it should be
> quite easy).
> 
> > (alternatively, admins can just install python-django
> > python-django-tables2 from jessie-backports and run `lava-server
> > managedb migrate` manually.)
> > 
> > There's nothing wrong AFAICT with requiring a step via
> > jessie-backports.  
> 
> That's a *very* uncommon requirement (lava-server is the first package
> doing this).

I suspect it would be much more common if services which use Debian
actually had a route to simple packaging within Debian which had this
support. It could provide an alternative to the endless docker hacks by
some upstreams.

> > There is also nothing LAVA can do about this, it's a django issue
> > which cannot be fixed in jessie as there are other packages in
> > jessie which are not compatible with django1.8 LTS (including the
> > version of LAVA in jessie).  
> 
> lava-server could error-out in preinst if an unsupported upgrade path
> is attempted, giving instructions to go via backports (link to
> lava-announce/2016-June/000010.html)

As it happens, I'm looking at something similar for the stretch ->
buster packaging changes. (lava-server has a more complex route then
between jessie and stretch, involving deliberate data loss due to the
removal of deprecated database objects.)

I'll see if I can get some testing done for this. The awkward part
about exiting in a preinst with a message about backports is that it
leaves the package in a broken state which apt will continually try to
fix whenever any other package is installed or upgraded. This makes it
very problematic when a lot of these upgrades will happen remotely using
configuration management tools like salt and puppet. Even debconf
critical notes don't necessarily help when the admin is not on the same
console or tools like salt hide the output or bury it in a lot of
noise. Add in the frequent conffile prompts and it becomes a lot of
noise for the admin. It's a situation which is just not handled
particularly well and with no obvious solution.

It really should be possible to do a better job of providing useful
feedback via management tools without making the problems worse. Maybe
if the preinst exited non-zero, dpkg should clear the installation
attempt as if it never happened, then provide output only at the end of
the run.

> > https://staging.validation.linaro.org/scheduler/job/163429
> > https://staging.validation.linaro.org/scheduler/job/163429/definition#defline59
> > 
> > Users are already aware of this:
> > https://lists.linaro.org/pipermail/lava-announce/2016-June/000010.html  
> 
> Good. But leave the bug open in case someone still runs into this
> problem.

Definitely, it will be useful documentation.

-- 

Neil Williams
=============
http://www.linux.codehelp.co.uk/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.alioth.debian.org/pipermail/piuparts-devel/attachments/20170208/41fe104a/attachment.sig>