[Pkg-mailman-hackers] Bug#891691: Bug#891691: Bug#891691: mailman3-core: Cannot install mailman3-suite with mysql database

Thu Mar 1 13:57:31 UTC 2018

Control: reassign -1 mailman3-suite
Control: rename -1 MySQL DB backend broken with MariaDB 10.1
Control: block -1 by 886756

Hi there,

Am 28.02.2018 um 19:29 schrieb Pierre-Elliott Bécue:
>> I have attempted to install mailman3-suite, and allowed automatic configuration using mysql. I think this was the default presented choice.

This should not have been the case. Default choice is sqlite, second
option is pgsql and third option is mysql.

>> This configuration failed, with this error transcript.
>>   [...]
>>   File "/usr/lib/python2.7/dist-packages/django/db/backends/mysql/base.py", line 30, in <module>
>>     'Did you install mysqlclient or MySQL-python?' % e
>> django.core.exceptions.ImproperlyConfigured: Error loading MySQLdb module: No module named MySQLdb.
>> Did you install mysqlclient or MySQL-python?
>> dpkg: error processing package mailman3-suite (--configure):
>>  installed mailman3-suite package post-installation script subprocess returned error exit status 1
>> Errors were encountered while processing:
>>  mailman3-suite
>>
>> So I'm guessing there is a missing dependency here. I tried installing python-mysqldb,

Indeed, you ran into a missing dependency on 'python-mysqldb'. We don't
directly depend on it as it's an alternative DB backend. For alternative
solutions, it's common to use alternative dependencies. So for the DB
backends, we have the following dependencies:

dbconfig-sqlite3 | dbconfig-pgsql | dbconfig-mysql | dbconfig-no-thanks
and
python-psycopg2 | python3-pymysql

There's a mistake indeed as the alternative dependency for mysql should
be 'python-mysqldb' (the python2 version, not the python3 one). This is
fixed in the Git repo now and will be fixed with the next upload.

But still, you have to take care of installing the alternative
depenencies yourself if you want to use a different DB backend than the
default one.

For pgsql (the prefered DB backend) this would be dbconfig-pgsql and
python-psycopg2, for mysql this would be dbconfig-mysql and python-mysqldb.

Still, you ran into a valid (and annoying) bug afterwards ...

>> [...] but this provided a different error:
>>
>> Errors were encountered while processing:
>>  mailman3-suite
>> E: Sub-process /usr/bin/dpkg returned an error code (1)
>> Setting up mailman3-suite (0+20170523-10) ...
>> Determining localhost credentials from /etc/mysql/debian.cnf: succeeded.
>> dbconfig-common: writing config to /etc/dbconfig-common/mailman3-suite.conf
>> mailman3suite already exists and has privileges on mailman3suite.
>> creating database mailman3suite: already exists.
>> dbconfig-common: flushing administrative password
>> Traceback (most recent call last):
>>   File "/usr/bin/django-admin", line 21, in <module>
>>     management.execute_from_command_line()
>>   File "/usr/lib/python2.7/dist-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
>>     utility.execute()
>>   File "/usr/lib/python2.7/dist-packages/django/core/management/__init__.py", line 356, in execute
>>     self.fetch_command(subcommand).run_from_argv(self.argv)
>>   File "/usr/lib/python2.7/dist-packages/django/core/management/base.py", line 283, in run_from_argv
>>     self.execute(*args, **cmd_options)
>>   File "/usr/lib/python2.7/dist-packages/django/core/management/base.py", line 330, in execute
>>     output = self.handle(*args, **options)
>>   File "/usr/lib/python2.7/dist-packages/django/core/management/commands/migrate.py", line 204, in handle
>>     fake_initial=fake_initial,
>>   File "/usr/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 115, in migrate
>>     state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
>>   File "/usr/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 145, in _migrate_all_forwards
>>     state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
>>   File "/usr/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 244, in apply_migration
>>     state = migration.apply(state, schema_editor)
>>   File "/usr/lib/python2.7/dist-packages/django/db/migrations/migration.py", line 129, in apply
>>     operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
>>   File "/usr/lib/python2.7/dist-packages/django/db/migrations/operations/fields.py", line 221, in database_forwards
>>     schema_editor.alter_field(from_model, from_field, to_field)
>>   File "/usr/lib/python2.7/dist-packages/django/db/backends/base/schema.py", line 515, in alter_field
>>     old_db_params, new_db_params, strict)
>>   File "/usr/lib/python2.7/dist-packages/django/db/backends/base/schema.py", line 690, in _alter_field
>>     params,
>>   File "/usr/lib/python2.7/dist-packages/django/db/backends/base/schema.py", line 120, in execute
>>     cursor.execute(sql, params)
>>   File "/usr/lib/python2.7/dist-packages/django/db/backends/utils.py", line 64, in execute
>>     return self.cursor.execute(sql, params)
>>   File "/usr/lib/python2.7/dist-packages/django/db/utils.py", line 94, in __exit__
>>     six.reraise(dj_exc_type, dj_exc_value, traceback)
>>   File "/usr/lib/python2.7/dist-packages/django/db/backends/utils.py", line 64, in execute
>>     return self.cursor.execute(sql, params)
>>   File "/usr/lib/python2.7/dist-packages/django/db/backends/mysql/base.py", line 101, in execute
>>     return self.cursor.execute(query, args)
>>   File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 250, in execute
>>     self.errorhandler(self, exc, value)
>>   File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 50, in defaulterrorhandler
>>     raise errorvalue
>> django.db.utils.OperationalError: (1071, 'Specified key was too long; max key length is 767 bytes')
>> dpkg: error processing package mailman3-suite (--configure):
>>  installed mailman3-suite package post-installation script subprocess returned error exit status 1
>> Errors were encountered while processing:
>>  mailman3-suite
> 
> This error is a bummer. It comes from innoDB implementations of SQL servers.
> The main idea is that there is a limited length for the index deployment on
> varchar fields, and one field in django-allauth/mailman is 255 chars, which
> is more than 767 bytes (191 chars).

Yep, correct. See Debian bugreport #886756[1] and upstream bugreport
MDEV-14904[2].

This is an ugly bug that only exists in MariaDB 10.1 due to the
combination of default charset 'utf8mb4', old innodb_file_format and no
inoodb_default_file_format.

This topic is already discussed in section 'MySQL database backend' in
README.Debian (/usr/share/doc/mailman3-suite/README.Debian or [3]).

I don't see an easy solution to that except fixing MDEV-14904 upstream
(which was promised to be done) and backporting it to the mariadb-10.1
packages in Debian.

For now, your options are:

* Switch to mariadb10.2 or mariadb10.3
* Switch back to 'utf8' as default charset at your mariadb 10.1 instance
  (not even sure if that is sufficient)
* Use another DB backend

Cheers
 jonas

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=886756
[2] https://jira.mariadb.org/browse/MDEV-14904
[3]
https://salsa.debian.org/mailman-team/mailman-suite/blob/master/debian/README.Debian#L49

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 866 bytes
Desc: OpenPGP digital signature
URL: <http://lists.alioth.debian.org/pipermail/pkg-mailman-hackers/attachments/20180301/cf8e660c/attachment.sig>