[Piuparts-devel] Bug#847277: lava-server: fails to upgrade from jessie to stretch

Wed Feb 8 20:03:20 UTC 2017

On 2017-02-08 20:16, Neil Williams wrote:
> retitle 847277 django migrations in jessie require django from
jessie-backports to upgrade to stretch
[...]

> On Wed, 08 Feb 2017 14:48:21 +0100
> Andreas Beckmann <anbe at debian.org> wrote:

> The flow needs to go via jessie-backports, just like any django project
> in jessie as the database migrations from jessie cannot be handled by
> django1.10 without going via the LTS release, django1.8. The bug, if
> any, is in django1.7 which does not (cannot) know how to handle
> database migrations the way that django1.8 (and subsequent releases)
> need to handle migrations. However, that change can't be backported to
> jessie except by using the new flow from 1.8 which breaks all other
> packages using django in jessie.
> 
> On a real instance, the postgres cluster would also need to be migrated.

I've started doing this in piuparts recently ...

> If piuparts cannot handle such upgrades, that must be a bug in piuparts.

Having uncommon upgrade requirements (beyond
    sed -i s/jessie/stretch/ sources.list
    apt-get update
    apt-get dist-upgrade
) will need extra coding ...

> We test this regularly, albeit to the newer version in our staging repo
> rather than the version in stretch. 

OK.

> https://staging.validation.linaro.org/scheduler/job/162057/definition#defline59
> 
> # on jessie, with backports enabled:
> DEBIAN_FRONTEND=noninteractive apt-get -y install lava-dispatcher lava-server
> ....
> apt -q -y -t jessie-backports install python-django-kvstore python-django python-django-tables2 lava-server
> # now proceed with the upgrade.
> DEBIAN_FRONTEND=noninteractive apt -y upgrade
> DEBIAN_FRONTEND=noninteractive apt -y dist-upgrade

Given that you are continuously testing your special upgrade path, I'll
not reimplement that in piuparts for now (although it should be quite easy).

> (alternatively, admins can just install python-django
> python-django-tables2 from jessie-backports and run `lava-server
> managedb migrate` manually.)
> 
> There's nothing wrong AFAICT with requiring a step via jessie-backports.

That's a *very* uncommon requirement (lava-server is the first package
doing this).

> There is also nothing LAVA can do about this, it's a django issue which
> cannot be fixed in jessie as there are other packages in jessie which
> are not compatible with django1.8 LTS (including the version of LAVA in
> jessie).

lava-server could error-out in preinst if an unsupported upgrade path is
attempted, giving instructions to go via backports (link to
lava-announce/2016-June/000010.html)

> https://staging.validation.linaro.org/scheduler/job/163429
> https://staging.validation.linaro.org/scheduler/job/163429/definition#defline59
> 
> Users are already aware of this:
> https://lists.linaro.org/pipermail/lava-announce/2016-June/000010.html

Good. But leave the bug open in case someone still runs into this problem.

>> Control: retitle -1 lava-server: fails to upgrade from jessie to
>> stretch: InconsistentMigrationHistory: Migration
>> lava_scheduler_app.0001_initial is applied before its dependency
>> linaro_django_xmlrpc.0001_initial on database 'default'
>>
>> The problem is also reproducible on a plain jessie->stretch upgrade:
> 
> The problem would only apply on a jessie-> stretch upgrade without
> upgrading django using jessie-backports.
> 
>>
>> https://piuparts.debian.org/jessie2stretch/fail/lava-server_2016.12-1.log
>>
>> [...]
>>   Setting up lava-server (2016.12-1) ...
>>   Installing new version of config
>> file /etc/apache2/sites-available/lava-server.conf ... Installing new
>> version of config file /etc/init.d/lava-server ... Installing new
>> version of config file /etc/lava-server/settings.conf ... lavaserver
>>    lavaserver
>>    devel
>>    devel
>>   System check identified some issues:
>>   
>>   WARNINGS:
>>   va_scheduler_app.Notification.job_status_trigger: (fields.W901)
>> CommaSeparatedIntegerField has been deprecated. Support for it
>> (except in historical migrations) will be removed in Django 2.0.
>> HINT: Use
>> CharField(validators=[validate_comma_separated_integer_ceback (most
>> recent call last): File "/usr/bin/lava-server", line 78, in <module>
>> main() File "/usr/bin/lava-server", line 74, in main
>> execute_from_command_line(django_options) File
>> "/usr/lib/python2.7/dist-packages/django/core/management/__init__.py",
>> line 367, in execute_from_command_line utility.execute() File
>> "/usr/lib/python2.7/dist-packages/django/core/management/__init__.py",
>> line 359, in execute
>> self.fetch_command(subcommand).run_from_argv(self.argv) File
>> "/usr/lib/python2.7/dist-packages/django/core/management/base.py",
>> line 294, in run_from_argv self.execute(*args, **cmd_options) File
>> "/usr/lib/python2.7/dist-packages/django/core/management/base.py",
>> line 345, in execute output = self.handle(*args, **options) File
>> "/usr/lib/python2.7/dist-packages/django/core/management/commands/migrate.py",
>> line 86, in handle
>> executor.loader.check_consistent_history(connection) File
>> "/usr/lib/python2.7/dist-packages/django/db/migrations/loader.py",
>> line 292, in check_consistent_history connection.alias,
>> django.db.migrations.exceptions.InconsistentMigrationHistory:
>> Migration lava_scheduler_app.0001_initial is applied before its
>> dependency linaro_django_xmlrpc.0001_initial on database 'default'.
>> dpkg: error processing package lava-server (--configure): subprocess
>> installed post-installation script returned error exit status 1
>> Processing triggers for python-support (1.0.15) ... Processing
>> triggers for libc-bin (2.24-8) ... Processing triggers for systemd
>> (232-14) ... Processing triggers for ca-certificates (20161130) ...
>> Updating certificates in /etc/ssl/certs... 0 added, 0 removed; done.
>> Running hooks in /etc/ca-certificates/update.d... done. Errors were
>> encountered while processing: lava-server list]) instead.
>>
>> I can easily provide more debug information from the chroot after the
>> failure, just tell me what you need (and how to acquire it).
> 
> To fix the chroot, downgrade python-django, python-django-common and
> python-django-tables2 to jessie-backports versions. Then the upgrade to
> stretch will work. Or, as above, install these from jessie-backports
> first, then the upgrade proceeds.

Andreas