[Python-apps-team] Bug#824347: fails in jessie with "django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet."

[Antoine Beaupré]

Package: planet-venus
Version: 0~git9de2109-3
Severity: grave
Tags: upstream

I get a fatal exception when following the bootstrap procedure for
Debian planet documented here:

https://wiki.debian.org/PlanetDebian#test

Here's the backtrace, which happens after all the feeds are parsed (so
it takes a *loong* time to get there on debian-planet!):

INFO:planet.runner:Loading cached data
Traceback (most recent call last):
  File "/usr/bin/planet", line 158, in <module>
    splice.apply(doc.toxml('utf-8'))
  File "/usr/lib/python2.7/dist-packages/planet/splice.py", line 142, in apply
    output_file = shell.run(template_file, doc)
  File "/usr/lib/python2.7/dist-packages/planet/shell/__init__.py", line 66, in run
    module.run(template_resolved, doc, output_file, options)
  File "/usr/lib/python2.7/dist-packages/planet/shell/dj.py", line 38, in run
    t = get_template(script)
  File "/usr/lib/python2.7/dist-packages/django/template/loader.py", line 144, in get_template
    template, origin = find_template(template_name, dirs)
  File "/usr/lib/python2.7/dist-packages/django/template/loader.py", line 126, in find_template
    loader = find_template_loader(loader_name)
  File "/usr/lib/python2.7/dist-packages/django/template/loader.py", line 98, in find_template_loader
    TemplateLoader = import_string(loader)
  File "/usr/lib/python2.7/dist-packages/django/utils/module_loading.py", line 26, in import_string
    module = import_module(module_path)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/usr/lib/python2.7/dist-packages/django/template/loaders/app_directories.py", line 33, in <module>
    app_template_dirs = calculate_app_template_dirs()
  File "/usr/lib/python2.7/dist-packages/django/template/loaders/app_directories.py", line 21, in calculate_app_template_dirs
    for app_config in apps.get_app_configs():
  File "/usr/lib/python2.7/dist-packages/django/apps/registry.py", line 137, in get_app_configs
    self.check_apps_ready()
  File "/usr/lib/python2.7/dist-packages/django/apps/registry.py", line 124, in check_apps_ready
    raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.

In fact, it actually fails before that, because there are two
dependencies missing (python-django and python-genshi) but that's a
separate issue.

The fix I have found it to patch venus with this:

--- planet-venus-0~git9de2109/planet/shell/dj.py	2016-05-13 15:56:44.000000000 -0400
+++ /usr/lib/python2.7/dist-packages/planet/shell/dj.py	2016-05-14 08:43:55.587328754 -0400
@@ -27,6 +27,9 @@
             )
     except RuntimeError:
         pass
+    import django
+    django.setup()
+
     from django.template import Context
     from django.template.loader import get_template
 
I am not sure it is the correct way of doing this: i am only
marginally familiar with Django. I basically cargo-culted this from:

http://stackoverflow.com/a/27194927/1174784
http://stackoverflow.com/a/26215548/1174784
http://django.readthedocs.io/en/latest/releases/1.7.html#standalone-scripts
https://docs.djangoproject.com/en/1.8/topics/settings/#calling-django-setup-is-required-for-standalone-django-usage

So it looks okay-ish. I still get deprecation warnings, but at least
the feed is generated:

INFO:planet.runner:Loading cached data
/usr/lib/python2.7/dist-packages/django/template/base.py:290: RemovedInDjango18Warning: 'The `firstof` template tag is changing to escape its arguments; the non-autoescaping version is deprecated. Load it from the `future` tag library to start using the new behavior.
  compiled_result = compile_func(self, token)

WARNING:py.warnings:/usr/lib/python2.7/dist-packages/django/template/base.py:290: RemovedInDjango18Warning: 'The `firstof` template tag is changing to escape its arguments; the non-autoescaping version is deprecated. Load it from the `future` tag library to start using the new behavior.
  compiled_result = compile_func(self, token)

Proof it is working: http://paste.anarc.at/planet/www/

-- System Information:
Debian Release: 8.4
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'proposed-updates'), (500, 'stable'), (1, 'testing')
Architecture: amd64 (x86_64)

Kernel: Linux 4.2.0-0.bpo.1-amd64 (SMP w/2 CPU cores)
Locale: LANG=fr_CA.UTF-8, LC_CTYPE=fr_CA.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages planet-venus depends on:
ii  python              2.7.9-1
ii  python-chardet      2.3.0-1
ii  python-feedparser   5.1.3-3
ii  python-html5lib     0.999-3
ii  python-htmltmpl     1.22-10
ii  python-httplib2     0.9+dfsg-2
ii  python-librdf       1.0.17.1+dfsg-1.1+b1
ii  python-libxml2      2.9.1+dfsg1-5+deb8u1
ii  python-portalocker  0.5.1-1
ii  python-utidylib     0.2-9

Versions of packages planet-venus recommends:
ii  python-beautifulsoup  3.2.1-1
ii  python-libxslt1       1.1.28-2+b2

Versions of packages planet-venus suggests:
ii  python-django  1.7.7-1+deb8u4
ii  python-genshi  0.7-3
ii  python-lxml    3.4.0-1

-- no debconf information