[Python-modules-team] Bug#863267: Bug#863267: Miscalculates MigrationHistory dependencies between multiple django apps - regression from 1.8

[Neil Williams]

On Fri, 26 May 2017 10:16:41 +0100
Neil Williams <codehelp at debian.org> wrote:

> On Fri, 26 May 2017 10:05:58 +0100
> Neil Williams <codehelp at debian.org> wrote:
> 
> > On Fri, 26 May 2017 18:56:04 +1000
> > Brian May <bam at debian.org> wrote:
> >   
> > > Neil Williams <codehelp at debian.org> writes:
> > >     
> > > > django.db.migrations.exceptions.InconsistentMigrationHistory:
> > > > Migration lava_scheduler_app.0001_initial is applied before its
> > > > dependency linaro_django_xmlrpc.0001_initial on database
> > > > 'default'.      
> > > 
> > > Ok, I see what is going on now. Untested theory at the moment, but
> > > it fits the symptoms.
> > > 
> > > ./lava_scheduler_app/migrations/0001_initial.py contains:
> > > 
> > >     dependencies = [  
> > >         ('auth', '0001_initial'),  
> > >         migrations.swappable_dependency(settings.AUTH_USER_MODEL),  
> > >         ('linaro_django_xmlrpc', '__first__'),  
> > >         ('dashboard_app', '__first__'),  
> > >     ]      
> > 
> > Again, I also spotted this and thought it was the source. However,
> > changing this causes the migration to fail with 1.10 as there are
> > objects in this model which must be applied before
> > lava_scheduler_app/0001_initial will complete. e.g. the AuthToken
> > object is referred to directly in lava_scheduler_app/0001_initial
> > and this is defined by linaro_django_xmlrpc
> > 
> > I tried a few simplistic edits of those migration files on a test
> > instance, the migrations still fail to apply.
> >   
> > > 
> > > My reading of this is that this means this migration depends on
> > > the first migration from 'linaro_django_xmlrpc' to be already
> > > applied. However on Jessie, 'linaro_django_xmlrpc' has no
> > > migrations. Hence I suspect whatever version of Django that
> > > installed this migration to be buggy, because it didn't check the
> > > dependencies could be satisfied. Or maybe this was considered OK
> > > at the time.    
> > 
> > 7/9 Add initial Django migrations
> >     
> >     From a25d49c6c96217011fead69b675e281b6e5b8fc5 Mon Sep 17
> > 00:00:00 2001
> >     From: Raphaël Hertzog <hertzog at debian.org> Date: Tue, 9 Sep
> >     2014 07:20:56 +0000 
> > 
> > https://git.linaro.org/lava/lava-server.git/commit/?id=9e5595adae2b2ea6fc7e19820356ca1bd098110c
> >   
> > > This migration is already applied when we come to do the new
> > > migrations for Django 1.8 or Django 1.10
> > > 
> > > Django 1.8 doesn't care that the first migration for
> > > 'linaro_django_xmlrpc' hasn't been applied yet. As a result, it
> > > can fake the migration and everyone is happy.
> > > 
> > > Django 1.10 does care. It says the database is broken because the
> > > prerequisite for this migration was never applied. It does this
> > > check before applying any migrations.
> > > 
> > > I don't know for sure what is correct behaviour here.    
> > 
> > Changing existing migration files *after* user data has been created
> > is likely to cause data loss.
> >   
> > > However I am
> > > inclined to think maybe this isn't a Django 1.10 fault, because
> > > the migration in Jessie clearly says it depends on a migration
> > > that was never applied - because it doesn't even exist at this
> > > point.    
> > 
> > No. This is django making the wrong decision about problems it has
> > previously supported when trying to include bug fixes for other
> > problems. It is a regression in django 1.10.
> >   
> 
> Operations to perform:
>   Synchronize unmigrated apps: linaro_django_xmlrpc,
> django_openid_auth, lava_projects, lava_scheduler_daemon,
> django_tables2, lava_markitup Apply all migrations: sessions, admin,
> dashboard_app, auth, sites, contenttypes, lava_scheduler_app,
> google_analytics Synchronizing apps without migrations: Creating
> tables... Creating table django_openid_auth_nonce Creating table
> django_openid_auth_association Creating table
> django_openid_auth_useropenid Creating table
> linaro_django_xmlrpc_authtoken
> 
> From the installation of 2014.9-1 from jessie.
> 
> At no point did 1.7 require or support linaro_django_xmlrpc
> migrations. (Creating them caused django to fail to apply them.) It's
> not that is does not exist, it is that django 1.7 in jessie does not
> allow it to exist - yet 1.10 requires it and it's code in 1.8 that
> handles the transition.

Of the changes between 1.8 and 1.10, this is the problematic function,
new in 1.10

https://sources.debian.net/src/python-django/1:1.10.7-1/django/db/migrations/loader.py/#L270

This code has been relocated from 1.7 to 1.10, relating to __first__:

https://sources.debian.net/src/python-django/1:1.10.7-1/django/db/migrations/loader.py/#L166

I'm wondering if once a migration has been seen by
add_internal_dependencies as __first__, that migration should not be
checked again in check_consistent_history.

-- 


Neil Williams
=============
http://www.linux.codehelp.co.uk/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.alioth.debian.org/pipermail/python-modules-team/attachments/20170526/0a9979ad/attachment.sig>