Zope2 packaging

[Jonas Meurer]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hey,

Am 24.06.2011 23:20, schrieb Jonas Meurer:
>> In my opinion some issues are left before the zope2.12 packages can be
>> uploaded to Debian. I would like to hear your comments about that:
> 
>> 3. It seems like not all zope dependencies are pushed into the tarball
>> by Michaels debian/scripts/fetch.py. If I compare the list of
>> dependencies in versions.cfg and the list of installed dependencies, a
>> lot are missing even though not listed in DEB_SATISFIED. These are:
> 
>> buildout.dumppickedversions, jinja2, lxml2, pygments, python-gettext,
>> roman, sphinx, unittest2, z3c.checkversions, zc.buildout, zc.recipe.egg,
>> zc.recipe.testrunner, zodbcode, zope.app.apidoc,
>> zope.app.applicationcontrol, zope.app.authentication, zope.app.broken,
>> zope.app.cache, zope.app.catalog, zope.app.content, zope.app.dav,
>> zope.app.dtmlpage, zope.app.error, zope.app.exception, zope.app.file,
>> zope.app.folder, zope.app.generations, zope.app.http, zope.app.i18n,
>> zope.app.interface, zope.app.intid, zope.app.locales,
>> zope.app.localpermission, zope.app.principalannotation,
>> zope.app.renderer, zope.app.rotterdam, zope.app.security,
>> zope.app.securitypolicy, zope.app.server, zoper.app.session,
>> zope.app.traversing, zope.app.undo, zope.app.wsgi, zope.app.zapi,
>> zope.app.zcmlfiles, zope.app.zopeappgenerations, zope.app.zptpage,
>> zope.catalog, zope.decorator, zope.documenttemplate, zope.index,
>> zope.keyreference, zope.modulealias, zope.principalannotation,
>> zope.principalregistry, zope.securitypolicy, zope.server, zope.thread
> 
>> Some of them are clearly not required (buildout stuff), and some might
>> be installed as python modules packages (jinja2, python-gettext, ...),
>> but I still wonder why they aren't pushed into the tarball by fetch.py
>> despite not being listed in exclusions. Michael, maybe you have an idea?
> 
> This one is more important. But I have a suspcion why this happens now.
> fetch.py uses pip to fetch "Zope2==2.12.18", which fetches all
> dependencies of Zope2 as well.
> First I thought that all packages that are listed in versions.cfg would
> be fetched, but that seems to be wrong. I'm not sure yet whether I got
> the purpose of versions.cfg, but it seems like get_specs() uses it to
> override versions from the list of Zope2 dependencies. It's unclear to
> me why this is required, but I guess PIP/buildout internals are the reason.
> 
> Michael, maybe you can add a short comment on this?

While I'm still unsure about that point, I tracked other issues in the
meanwhile:

- - python-setuptools is not required by the zope2.12 binary package,
while python-pkg-resources is. I checked that by grepping through sources.

- - python-transaction actually is a Zope module, maintained by Zope devs
and with dependencies on other Zope modules. Thus we should include it
into the zope2.12 binary package instead of depending on the debian
package. Additionally, the Debian-packaged version of python-transaction
is to old for our zope2.12 packages, but see below for that.

- - the python interpreter wrapper written by Michael Mulich has some
issues:

- - First, it appends the directories with Python/Zope eggs to the end of
sys.path using site.addsitedir(). This brings problems whenever
newer/older/different versions of the modules are installed in the
system-wide Python pathes (e.g. by a Debian package). In these cases,
the other installed version is taken by import rather than the one we
ship in the zope2.12 packages.

- - Second, the Python interpreter wrapper introduces shebang-recursion.
And for some reason, this shebang-recursion breaks on older linux
kernels (e.g. the Debian Lenny kernel). Gael Le Mignot spotted this
issue. I still didn't get, what and why exactly breaks, but using a
python wrapper as shebang, which in turns has python2.6 in shebang
breaks. This can easily be reproduced by simple example scripts.

- - For these two reasons I wrote a small python interpreter wrapper in
shell for our purposes: it appends all Zope2.12 egg directories to
$PYTHONPATH, adds ${INSTANCE_HOME}/lib/python as well in case it exists,
and executes python2.6 with all options and arguments afterwards. So far
it seems to work.

- - Now the biggest problem, and the only one I didn't resolve yet: Zope2
2.12.18 lists both "ZODB3 = 2.9.7" and "transaction = 1.0.1" in its
versions.cfg. Unfortunately, ZODB3 2.9.7 itself depends on transaction
>= 1.1.0. But Michaels fetch.py script always fetches the version from
Zope2 versions.cfg in case one is specified. This leads to a conflict.
We need to find a solution here. Either we trust in versions.cfg from
Zope2 index, or we take versions from egg dependencies if they differ.
I'm not sure whether this conflict is a mistake by Zope2 upstream, or
whether it is intended and handled intelligently by buildout/pip.

Michael, maybe you can comment on this?

Greetings,
 jonas
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBAgAGBQJOB6diAAoJEFJi5/9JEEn+/icP/AoqI5h6jtmgqVVtl/dVb6C6
HTBzfC5XXzA19IOZaLZ5ZCyFEn3WpHy4pBFUiZH555WZmBaujEhiG3FTAAyJN7Rq
dVtAWII+H82dmUSJzMCaso+4rj/aLkcKfjOhC7LyeOjhLrYrnzOB7wPxLxoPcHXo
m4D7PJsIgxMkkva0UWnorsNyYs1TmNMc15QVe7Zlmq8FfNglJvs0hMkZUjvxm89z
L8J3M/l8gBeg/3QUqBbOAeed2NF+ugIhDEUHLIicIUtOtglthDKqv17JwNhtpCUy
OIb7+QExx8j9V0Pamy/wMnHr+QXPGpdOcJmuvPAhdl8efLdBGiZoCcOnxUaBn2O9
dLOvf+NnwsoctOOrY/mxpi5bYG4WU4h2pExezw7FhjArtSfO05v4diC3Xlzo60NC
VmjGx38jEO7Fq9yAlmw02/j9dLte35gh035xu+hPRlm+HaxWVt5koDJOBxrFE2K/
2CjDjMhnB05n4pi9c+KyNH3RtFBo7OyfdJQ7gst4yhAIjnMTNYtGmbronfODGFtj
laP9IqmvQAUP832FaNAdNIK0rOz73YJkOHicFweaEscCGVaQ5Ox6FzcV3iFLcDd0
awKoIzglzdKprM02IUWPzJzpkgxjJvr43K3gDzAg7/Vd0ImuVFCJLcZDWh+qwFkw
UN/Iy22MaCmOKQc4cd3j
=S/kF
-----END PGP SIGNATURE-----