[Pkg-mailman-hackers] Bug#1043098: Bug#1043098: python3-django-hyperkitty: tracebacks with update_index_one_list on mailman2 migration

[Pierre-Elliott Bécue]

Hello,

Steve Langasek <vorlon at dodds.net> wrote on 06/08/2023 at 01:10:58+0100:

> Package: python3-django-hyperkitty
> Version: 1.3.4-4
> Severity: important
>
> Dear Maintainer,
>
> Having just upgraded a system to bullseye, I am in the process of migrating
> my mailing lists from the obsolete mailman 2 to mailman3.  Following guides
> such as https://docs.mailman3.org/en/latest/migration.html leads me to run,
> as list:
>
> $ django-admin hyperkitty_import --settings mailman-web \
>     --pythonpath /var/lib/mailman3/hyperkitty/ -l $list_email $mbox_path
> $
>
> which succeeds (/var/lib/mailman3/hyperkitty/mailman-web.py is a forked copy
> of /etc/mailman3/mailman-web.py with appropriate permissions).
>
> I am then supposed to run update_index_one_list, which fails with a 
> confusing error about missing templates:
>
> $ django-admin update_index_one_list --settings mailman-web \
>     --pythonpath /var/lib/mailman3/hyperkitty/ -l $list_email
> Indexing 421 emails
> [ERROR/MainProcess] Failed indexing 1 - 421 (retry 5/5): search/indexes/hyperkitty/email_text.txt (pid 6818): search/indexes/hyperkitty/email_text.txt
> Traceback (most recent call last):
>   File "/usr/lib/python3/dist-packages/haystack/management/commands/update_index.py", line 111, in do_update
>     backend.update(index, current_qs, commit=commit)
>   File "/usr/lib/python3/dist-packages/haystack/backends/whoosh_backend.py", line 269, in update
>     doc = index.full_prepare(obj)
>   File "/usr/lib/python3/dist-packages/haystack/indexes.py", line 236, in full_prepare
>     self.prepared_data = self.prepare(obj)
>   File "/usr/lib/python3/dist-packages/haystack/indexes.py", line 227, in prepare 
>     self.prepared_data[field.index_fieldname] = field.prepare(obj)
>   File "/usr/lib/python3/dist-packages/haystack/fields.py", line 235, in prepare
>     return self.convert(super(CharField, self).prepare(obj))
>   File "/usr/lib/python3/dist-packages/haystack/fields.py", line 99, in prepare
>     return self.prepare_template(obj)
>   File "/usr/lib/python3/dist-packages/haystack/fields.py", line 212, in prepare_template
>     t = loader.select_template(template_names)
>   File "/usr/lib/python3/dist-packages/django/template/loader.py", line 47, in select_template
>     raise TemplateDoesNotExist(', '.join(template_name_list), chain=chain)
> django.template.exceptions.TemplateDoesNotExist: search/indexes/hyperkitty/email_text.txt
> Traceback (most recent call last):
>   File "/usr/bin/django-admin", line 5, in <module>
>     management.execute_from_command_line()
>   File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
>     utility.execute()
>   File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 375, in execute
>     self.fetch_command(subcommand).run_from_argv(self.argv)
>   File "/usr/lib/python3/dist-packages/django/core/management/base.py", line 323, in run_from_argv
>     self.execute(*args, **cmd_options)
>   File "/usr/lib/python3/dist-packages/django/core/management/base.py", line 364, in execute
>     output = self.handle(*args, **options)
>   File "/usr/lib/python3/dist-packages/hyperkitty/management/commands/update_index_one_list.py", line 41, in handle
>     update_index(listname=options.get("listname")[0],
>   File "/usr/lib/python3/dist-packages/hyperkitty/search_indexes.py", line 117, in update_index
>     update_cmd.update_backend("hyperkitty", "default")
>   File "/usr/lib/python3/dist-packages/haystack/management/commands/update_index.py", line 319, in update_backend
>     max_pk = do_update(
>   File "/usr/lib/python3/dist-packages/haystack/management/commands/update_index.py", line 111, in do_update
>     backend.update(index, current_qs, commit=commit)
>   File "/usr/lib/python3/dist-packages/haystack/backends/whoosh_backend.py", line 269, in update
>     doc = index.full_prepare(obj)
>   File "/usr/lib/python3/dist-packages/haystack/indexes.py", line 236, in full_prepare
>     self.prepared_data = self.prepare(obj)
>   File "/usr/lib/python3/dist-packages/haystack/indexes.py", line 227, in prepare   
>     self.prepared_data[field.index_fieldname] = field.prepare(obj)
>   File "/usr/lib/python3/dist-packages/haystack/fields.py", line 235, in prepare
>     return self.convert(super(CharField, self).prepare(obj))
>   File "/usr/lib/python3/dist-packages/haystack/fields.py", line 99, in prepare
>     return self.prepare_template(obj)
>   File "/usr/lib/python3/dist-packages/haystack/fields.py", line 212, in prepare_template
>     t = loader.select_template(template_names)
>   File "/usr/lib/python3/dist-packages/django/template/loader.py", line 47, in select_template
>     raise TemplateDoesNotExist(', '.join(template_name_list), chain=chain)
> django.template.exceptions.TemplateDoesNotExist: search/indexes/hyperkitty/email_text.txt
> $
>
> This is quite confusing, as 
> /usr/lib/python3/dist-packages/hyperkitty/templates/search/indexes/hyperkitty/email_text.txt
> certainly exists.
>
> It appears the only impact is on searchability of the mailing list 
> archive, which isn't terribly important to me, but I'd like to know why 
> it throws a backtrace.

Have you tried to clear mailman3-web's cache?

You can do so via /usr/share/mailman3-web/manage.py clear_cache

Bests,

-- 
PEB
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 853 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/pkg-mailman-hackers/attachments/20240317/4a304234/attachment-0003.sig>