[PKG-Openstack-devel] SQLAlchemy 0.9.X in unstable

[Thomas Goirand]

On 02/24/2014 03:17 AM, Piotr Ożarowski wrote:
> [Thomas Goirand, 2014-02-23]
>> CC-ing the release team, since I think you should have coordinate with
>> them here.
> 
> dropping everyone else. I forgot to set MFT, my bad.
> 
>> On 02/23/2014 11:58 PM, Piotr Ożarowski wrote:
>>> Hi guys,
>>>
>>> I just uploaded sqlalchemy 0.9.3 to unstable
>>
>> Noooooooooooooooooooooo !!!
>>
>> Man, this is a COMPLETE DISASTER for me. Don't do this again without any
>> coordination!
> 
> Jessie will be released with 0.9.X, IIRC I told you that months ago,
> during DebConf.

I have no problem with this, and I think it's good thing to be in sync
with upstream releases. I strongly support this goal. However, again,
the problem is the lack of coordination and upload without checking
anything on the reverse dependencies, just like what happened last July.
Once, I would understand. But since that's the second time, I have to
tell that it's my opinion it shouldn't have happen this way.

> 0.8.X → 0.9.X changes are not that big and I even considered uploading 0.9.1
> directly to unstable. I used experimental where you had more than a
> month to test it.

Things don't go that fast in OpenStack. It usually takes about a month
or so to get a commit reviewed and approved. Plus there was some
problems with the gating system, which slows down things.

Anyway, saying that "I had more than one month to test" is IMO
irrelevant. I strongly believe that you should have checked before
uploading a key library with so many reverse dependencies, or at least
ask the current package maintainers (a short email would have been
enough). And possibly, also ask the release team to manage the
transition. That's why we have transitions managed by the release team
by the way: to make sure nothing breaks too bad just like right now.

> "recovering" from artificial limits that I enforce on all
> python-sqlalchemy's reverse dependencies by generating << X.Y
> dependency.

Recovering from the lack of SQLAlchemy 0.8.x support from upstream.
They've just switched to 0.8.x, and now, everything broke again as soon
as upstream supports SQLA 0.8. I just hope it wont happen again the same
way, cause it just drives me crazy.

> I'm pretty sure that 99% of packages will need simple
> rebuild.

All of OpenStack is already more than 99%. That's more than 25% of the
40 reverse dependencies of python-sqlalchemy, according to apt-rdepends.

> Those who don't will block python-sqlalchemy's migration (they
> already have appropriate Depends line)

Which IMO, not what should have happened. python-sqlalchemy should have
been uploaded only when we had support for all reverse dependencies.
That's what all other distributions are doing: Ubuntu, RedHat, etc. I
just don't understand why you just carelessly rushed for a 2nd time.

>> any coordination, yet you're doing it again. It took about 7 months for
>> upstream to do the switch, and during a full release cycle, I had lots
>> and lots of troubles.
> 
>> I'm letting you know. You have just broke:
>>
>> - ceilometer
>> - cinder
>> - glance
>> - heat
>> - keystone
>> - neutron
>> - nova
>> - trove
>> - taskflow
> 
> will you back this up with a traceback?
> (/me really wanted to say more here but will stop for now)

Sure, I just changed the global-requirements.txt in the OpenStack CI, to
see it crash:

https://review.openstack.org/#/c/75723/

What's happening, is that keystone, which is the authentication system
for OpenStack, just fails to create its database. The relevant log is
just after "keystone-manage db_sync":

+ mysql -uroot -psecret -h127.0.0.1 -e 'CREATE DATABASE keystone
CHARACTER SET utf8;'
+ /opt/stack/new/keystone/bin/keystone-manage db_sync
2014-02-23 22:44:47.985 21700 WARNING
keystone.openstack.common.db.sqlalchemy.session [-] This application has
not enabled MySQL traditional mode, which means silent data corruption
may occur. Please encourage the application developers to enable this mode.
/usr/local/lib/python2.7/dist-packages/migrate/changeset/databases/mysql.py:37:
SADeprecationWarning: Use ``<obj>.name.quote``
  old_col_name = self.preparer.quote(delta.current_name, table.quote)
/usr/local/lib/python2.7/dist-packages/migrate/changeset/databases/mysql.py:37:
SADeprecationWarning: Use ``<obj>.name.quote``
  old_col_name = self.preparer.quote(delta.current_name, table.quote)
/usr/local/lib/python2.7/dist-packages/migrate/changeset/databases/mysql.py:37:
SADeprecationWarning: Use ``<obj>.name.quote``
  old_col_name = self.preparer.quote(delta.current_name, table.quote)
/usr/local/lib/python2.7/dist-packages/migrate/changeset/ansisql.py:278:
SADeprecationWarning: Use ``<obj>.name.quote``
  return self.preparer.quote(ret, cons.quote)
2014-02-23 22:44:48.253 21700 CRITICAL keystone [-] AttributeError:
'unicode' object has no attribute 'quote'

Also, all of the packages of OpenStack are using either Alembic or
SQLAlchemy migrate. I'm not sure for Alembic, but at least
python-migrate doesn't support SQLAlchemy 0.9 (as you can see on the
above log). I'm also convinced that there will be more breakage once
SQLA-Migrate is fixed.

By the way, using apt-rdepends shows that also: buildbot, openlp,
pytrainer and zine depend on python-migrate, and will have troubles.

Help on fixing python-migrate upstream would be appreciated, I believe.

> [...]
>> Please revert this upload immediately (with an EPOC) and until other
>> packages have reasonable upstream support for it.
> 
> I will not do that!

I can understand why you don't want to back up. However, it'd be nice if
you could consider having a bit of coordination *before* uploading any
new major release of SQLAlchemy in the future.

Cheers,

Thomas Goirand (zigo)

P.S: Even if I was really shocked yesterday by this upload, I will keep
no hard feelings this time again, and I will try to deal with this as
much and as fast as possible. But I don't expect OpenStack to be fixed
before a few months at least, and hope that we will manage major SQLA
upgrades better next time.