r2126 - in zope2.12/branches/with-revived-tarball/debian (16 files)

Sun Nov 14 00:13:47 UTC 2010

Jonas-

On 11/13/10 2:49 PM, Jonas Meurer wrote:
> On 12/11/2010 Michael Mulich wrote:
>> On 11/12/10 2:08 PM, Jonas Meurer wrote:
>>> On 12/11/2010 Michael Mulich wrote:
>>>> On 11/12/10 1:03 PM, Jonas Meurer wrote:
>>>>> First, the changes you commit to svn look very promising!
>>>>
>>>> Thank you. I've been able to install zope2.12 and create instances.
>>>> My latest changes to dzhandle have made it possible to install
>>>> zope2.12-sandbox as well.
>>>
>>> great! your work is much appreciated. looking forward to have zope2.12
>>> packages in debian/unstable ...
>>>
>>
>> I'm looking forward to zope2.13 in unstable, because of the added
>> WSGI support. But first things first, a somewhat stable zope2.12
>> would be nice. I've made the tarball source so that it won't be hard
>> to pull in the latest release. (https://weblion.psu.edu/trac/weblion/browser/weblion/hosting/zope2-tarball/trunk)
>>
>>>>> Today I took a look at the with-revived-tarball branch, and noted that
>>>>> something must be wrong with the build process. dpkg-source adds a patch
>>>>> to debian/patches at build time. I guess this is due to differences
>>>>> between the original tarball at ../tarballs/zope2.12_2.12.0.orig.tar.gz
>>>>> and the build environment you build from.
>>>>> maybe the reason is that you don't update the tarball while you modify
>>>>> the source files in your build directory, from where you start
>>>>> dpkg-buildpackage.
>>>>
>>>> I'm still a little uncertain about what happens between when
>>>> dpkg-source and dpkg-buildpackage are run. So I'm not exactly sure
>>>> what you mean. I'll definitely make it my next priority to fix this
>>>> issue.
>>>>
>>>>> I'm not sure what causes this patch. But this event indicates, that
>>>>> something wents wrong.
>>>>
>>>> This explains why I haven't been able to use svn-buildpackage. I
>>>> thought it was just me. I was planning to deal with it later, but
>>>> apparently it's an issue. I've been unpacking the source and running
>>>> debuild over it up to this point.
>>>
>>> I'm happy to help you with packaging issues. please write down step by
>>> step instructions about how you build the debian package. i'll take a
>>> look at your build process and see whether the problem can be located.
>>>
>>   1) Obtain the current tarball: wget http://weblion.psu.edu/static/zope2-tarballs/Zope2-2.12.10-pkgd_3.tar.gz
>>   2) Unpack the tarball: tar xzf Zope2-2.12.10-pkgd_3.tar.gz
>>   3) Move stuff around: mv Zope2-2.12.10-pkgd_3.tar.gz
>> zope2.12_2.12.10.orig.tar.gz&&  mv Zope2-2.12.10-pkgd_3
>> zope2.12-2.12.10&&  cd zope2.12-2.12.10
>>   4) Insert the Debian directory: svn co
>> svn+ssh://user@svn.debian.org/svn/pkg-zope/zope2.12/trunk/debian
>>   5) Build it: debuild
>
> ok, gave that a try (in a clean pbuilder environment), and spotted some
> issues:
>
> 1) the build process on python-pip, which is not listed in the
> build-dependencies:
>
> # make PYTHON=python DIST_EXCLUSIONS=ClientForm:Docutils:mechanize:pytz:transaction:zc.lockfile:ZConfig:zdaemon:ZODB3:zope.authentication:zope.browser:zope.component:zope.configuration:zope.contenttype:zope.copy:zope.event:zope.exceptions:zope.hookable:zope.i18n:zope.i18nmessageid:zope.location:zope.proxy:zope.publisher:zope.schema:zope.security:zope.testbrowser:zope.testing:zope.traversing
> rm -rf ./build
> Preparing the environment for the build...
> Traceback (most recent call last):
>    File "./scripts/prep.py", line 80, in<module>
>      main()
>    File "./scripts/prep.py", line 38, in main
>      from pip.req import InstallRequirement
> ImportError: No module named pip.req
> make: *** [prep] Error 1
>

Good catch. Also, see below about 'get-orig-source' and pip.

> 2) with python-pip installed another error occurs:
>
> Building Zope2...
> ERROR PLACE HOLDER
> ('install_dir /root/zope2/zope2.12-2.12.10/build/tmp_installs/tmpkoaz_S\nProcessing Record\nRunning setup.py -q bdist_egg --dist-dir /root/zope2/zope2.12-2.12.10/source/Record/egg-dist-tmp-r9o85A\n', "In file included from src/Record/_Record.c:20:\ninclude/ExtensionClass/ExtensionClass.h:83:20: error: Python.h: No such file or directory\ninclude/ExtensionClass/ExtensionClass.h:84:20: error: import.h: No such file or directory\nIn file included from src/Record/_Record.c:20:\ninclude/ExtensionClass/ExtensionClass.h:95: error: expected specifier-qualifier-list before 'PyObject_HEAD'\ninclude/ExtensionClass/ExtensionClass.h:108: error: expected specifier-qualifier-list before 'PyObject'\ninclude/ExtensionClass/ExtensionClass.h:114: error: 'NULL' undeclared here (not in a function)\nsrc/Record/_Record.c:26: error: expected ')' before '*' token\nsrc/Record/_Record.c:27: error: expected '=', ',', ';', 'asm' or '__attribute__' before '*' token\nsrc/Record/_Record.c:28: error: expec
ted '=', ',', ';', 'asm' or '__attribute__' before '*' token\nsrc/Record/_Record.c:29: error: expected ')' before '*' token\nsrc/Record/_Record.c:30: error: expected ')' before '*' token\nsrc/Record/_Record.c:35: error: expected ')' before '*' token\nsrc/Record/_Record.c:41: error: expected '=', ',', ';', 'asm' or '__attribute__' before '*' token\nsrc/Record/_Record.c:46: error: expected specifier-qualifier-list before 'PyObject_HEAD'\nsrc/Record/_Record.c:51: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'PyTypeObject'\nsrc/Record/_Record.c: In function 'Record_init':\nsrc/Record/_Record.c:60: error: 'Record' has no member named 'schema'\nsrc/Record/_Record.c:61: error: 'Record' has no member named 'schema'\nsrc/Record/_Record.c:61: warning: implicit declaration of function 'PyObject_GetAttr'\nsrc/Record/_Record.c:61: error: 'PyObject' undeclared (first use in this function)\nsrc/Record/_Record.c:61: error: (Each undeclared identifier is reported only once\n
src/Record/_Record.c:61: error: for each function it appears in.)\nsrc/Record/_Record.c:61: error: expected expression before ')' token\nsrc/Record/_Record.c:61: error: 'Record' has no member named 'ob_type'\nsrc/Record/_Record.c:61: error: 'py___record_schema__' undeclared (first use in this function)\nsrc/Record/_Record.c:63: warning: implicit declaration of function 'PyObject_Length'\nsrc/Record/_Record.c:63: error: 'Record' has no member named 'schema'\nsrc/Record/_Record.c:64: error: 'Record' has no member named 'data'\nsrc/Record/_Record.c:66: error: 'Record' has no member named 'data'\nsrc/Record/_Record.c:66: warning: implicit declaration of function 'malloc'\nsrc/Record/_Record.c:66: warning: incompatible implicit declaration of built-in function 'malloc'\nsrc/Record/_Record.c:66: error: expected expression before ')' token\nsrc/Record/_Record.c:68: warning: implicit declaration of function 'PyErr_NoMemory'\nsrc/Record/_Record.c:71: warning: implicit declaration of f
unction 'memset'\nsrc/Record/_Record.c:71: warning: incompatible implicit declaration of built-in function 'memset'\nsrc/Record/_Record.c:71: error: 'Record' has no member named 'data'\nsrc/Record/_Record.c:71: error: expected expression before ')' token\nsrc/Record/_Record.c: At top level:\nsrc/Record/_Record.c:77: error: expected '=', ',', ';', 'asm' or '__attribute__' before '*' token\nsrc/Record/_Record.c:112: error: expected '=', ',', ';', 'asm' or '__attribute__' before '*' token\nsrc/Record/_Record.c: In function 'Record_deal':\nsrc/Record/_Record.c:136: error: 'PyObject' undeclared (first use in this function)\nsrc/Record/_Record.c:136: error: 'd' undeclared (first use in this function)\nsrc/Record/_Record.c:138: error: 'Record' has no member named 'schema'\nsrc/Record/_Record.c:140: error: 'Record' has no member named 'schema'\nsrc/Record/_Record.c:141: error: 'Record' has no member named 'data'\nsrc/Record/_Record.c:143: warning: implicit declaration of function 'Py
_XDECREF'\nsrc/Record/_Record.c:145: warning: implicit declaration of function 'Py_DECREF'\nsrc/Record/_Record.c:145: error: 'Record' has no member named 'schema'\nsrc/Record/_Record.c:146: warning: implicit declaration of function 'free'\nsrc/Record/_Record.c:146: warning: incompatible implicit declaration of built-in function 'free'\nsrc/Record/_Record.c:146: error: 'Record' has no member named 'data'\nsrc/Record/_Record.c: At top level:\nsrc/Record/_Record.c:150: error: array type has incomplete element type\nsrc/Record/_Record.c:151: error: 'PyCFunction' undeclared here (not in a function)\nsrc/Record/_Record.c:151: error: expected '}' before 'Record___getstate__'\nsrc/Record/_Record.c:154: error: expected '}' before 'Record___setstate__'\nsrc/Record/_Record.c:157: error: expected '}' before 'Record___setstate__'\nsrc/Record/_Record.c: In function 'Record_dealloc':\nsrc/Record/_Record.c:169: error: 'Record' has no member named 'ob_type'\nsrc/Record/_Record.c:170: error: '
Record' has no member named 'ob_type'\nsrc/Record/_Record.c:170: error: 'Py_TPFLAGS_HAVE_CLASS' undeclared (first use in this function)\nsrc/Record/_Record.c:170: error: 'Record' has no member named 'ob_type'\nsrc/Record/_Record.c:170: error: 'Record' has no member named 'ob_type'\nsrc/Record/_Record.c:170: error: 'PyObject' undeclared (first use in this function)\nsrc/Record/_Record.c:170: error: expected expression before ')' token\nsrc/Record/_Record.c:170: warning: implicit declaration of function 'PyObject_FREE'\nsrc/Record/_Record.c: At top level:\nsrc/Record/_Record.c:173: error: expected '=', ',', ';', 'asm' or '__attribute__' before '*' token\nsrc/Record/_Record.c:209: error: expected declaration specifiers or '...' before 'PyObject'\nsrc/Record/_Record.c:209: error: expected declaration specifiers or '...' before 'PyObject'\nsrc/Record/_Record.c: In function 'Record_setattr':\nsrc/Record/_Record.c:212: error: 'PyObject' undeclared (first use in this function)\nsrc/R
ecord/_Record.c:212: error: 'io' undeclared (first use in this function)\nsrc/Record/_Record.c:215: warning: implicit declaration of function 'PyObject_GetItem'\nsrc/Record/_Record.c:215: error: 'Record' has no member named 'schema'\nsrc/Record/_Record.c:215: error: 'name' undeclared (first use in this function)\nsrc/Record/_Record.c:217: warning: implicit declaration of function 'PyInt_Check'\nsrc/Record/_Record.c:219: warning: implicit declaration of function 'PyErr_SetString'\nsrc/Record/_Record.c:219: error: 'PyExc_TypeError' undeclared (first use in this function)\nsrc/Record/_Record.c:222: warning: implicit declaration of function 'PyInt_AsLong'\nsrc/Record/_Record.c:226: warning: implicit declaration of function 'Py_XINCREF'\nsrc/Record/_Record.c:226: error: 'v' undeclared (first use in this function)\nsrc/Record/_Record.c:227: warning: implicit declaration of function 'PyVar_Assign'\nsrc/Record/_Record.c:227: error: 'Record' has no member named 'data'\nsrc/Record/_Rec
ord.c:232: warning: implicit declaration of function 'PyErr_SetObject'\nsrc/Record/_Record.c:232: error: 'PyExc_AttributeError' undeclared (first use in this function)\nsrc/Record/_Record.c: In function 'Record_compare':\nsrc/Record/_Record.c:241: error: 'PyObject' undeclared (first use in this function)\nsrc/Record/_Record.c:241: error: 'dv' undeclared (first use in this function)\nsrc/Record/_Record.c:241: error: 'dw' undeclared (first use in this function)\nsrc/Record/_Record.c:241: warning: left-hand operand of comma expression has no effect\nsrc/Record/_Record.c:247: error: 'Record' has no member named 'data'\nsrc/Record/_Record.c:247: error: 'Record' has no member named 'data'\nsrc/Record/_Record.c:247: warning: left-hand operand of comma expression has no effect\nsrc/Record/_Record.c:247: warning: left-hand operand of comma expression has no effect\nsrc/Record/_Record.c:252: warning: implicit declaration of function 'PyObject_Compare'\nsrc/Record/_Record.c: At top leve
l:\nsrc/Record/_Record.c:264: error: expected '=', ',', ';', 'asm' or '__attribute__' before '*' token\nsrc/Record/_Record.c:272: error: expected '=', ',', ';', 'asm' or '__attribute__' before '*' token\nsrc/Record/_Record.c:280: error: expected '=', ',', ';', 'asm' or '__attribute__' before '*' token\nsrc/Record/_Record.c:294: error: expected '=', ',', ';', 'asm' or '__attribute__' before '*' token\nsrc/Record/_Record.c:310: error: expected '=', ',', ';', 'asm' or '__attribute__' before '*' token\nsrc/Record/_Record.c:319: error: expected declaration specifiers or '...' before 'PyObject'\nsrc/Record/_Record.c: In function 'Record_ass_item':\nsrc/Record/_Record.c:326: warning: implicit declaration of function 'IndexError'\nsrc/Record/_Record.c:330: error: 'v' undeclared (first use in this function)\nsrc/Record/_Record.c:332: error: 'PyExc_TypeError' undeclared (first use in this function)\nsrc/Record/_Record.c:336: warning: implicit declaration of function 'Py_INCREF'\nsrc/Re
cord/_Record.c:337: error: 'Record' has no member named 'data'\nsrc/Record/_Record.c: At top level:\nsrc/Record/_Record.c:342: error: expected declaration specifiers or '...' before 'PyObject'\nsrc/Record/_Record.c: In function 'Record_ass_slice':\nsrc/Record/_Record.c:344: error: 'PyExc_TypeError' undeclared (first use in this function)\nsrc/Record/_Record.c: At top level:\nsrc/Record/_Record.c:349: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'Record_as_sequence'\nsrc/Record/_Record.c:361: error: expected '=', ',', ';', 'asm' or '__attribute__' before '*' token\nsrc/Record/_Record.c:402: error: expected declaration specifiers or '...' before 'PyObject'\nsrc/Record/_Record.c:402: error: expected declaration specifiers or '...' before 'PyObject'\nsrc/Record/_Record.c: In function 'Record_ass_sub':\nsrc/Record/_Record.c:405: error: 'PyObject' undeclared (first use in this function)\nsrc/Record/_Record.c:405: error: 'io' undeclared (first use in this function)
\nsrc/Record/_Record.c:409: error: 'key' undeclared (first use in this function)\nsrc/Record/_Record.c:413: error: 'v' undeclared (first use in this function)\nsrc/Record/_Record.c:413: error: too many arguments to function 'Record_ass_item'\nsrc/Record/_Record.c:416: error: 'Record' has no member named 'schema'\nsrc/Record/_Record.c:420: error: 'PyExc_TypeError' undeclared (first use in this function)\nsrc/Record/_Record.c:428: error: 'Record' has no member named 'data'\nsrc/Record/_Record.c: At top level:\nsrc/Record/_Record.c:436: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'Record_as_mapping'\nsrc/Record/_Record.c:444: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'RecordType'\nsrc/Record/_Record.c:478: error: array type has incomplete element type\nsrc/Record/_Record.c:480: error: expected '}' before 'NULL'\nsrc/Record/_Record.c: In function 'init_Record':\nsrc/Record/_Record.c:488: error: 'PyObject' undeclared (first use in this funct
ion)\nsrc/Record/_Record.c:488: error: 'm' undeclared (first use in this function)\nsrc/Record/_Record.c:488: error: 'd' undeclared (first use in this function)\nsrc/Record/_Record.c:488: warning: left-hand operand of comma expression has no effect\nsrc/Record/_Record.c:490: error: 'py___record_schema__' undeclared (first use in this function)\nsrc/Record/_Record.c:490: warning: implicit declaration of function 'PyString_FromString'\nsrc/Record/_Record.c:492: warning: implicit declaration of function 'PyCObject_Import'\nsrc/Record/_Record.c:492: warning: assignment makes pointer from integer without a cast\nsrc/Record/_Record.c:495: warning: implicit declaration of function 'Py_InitModule4'\nsrc/Record/_Record.c:497: error: expected expression before ')' token\nsrc/Record/_Record.c:500: warning: implicit declaration of function 'PyModule_GetDict'\nsrc/Record/_Record.c:502: warning: assignment makes pointer from integer without a cast\nsrc/Record/_Record.c:502: error: 'struct 
ExtensionClassCAPIstruct' has no member named 'PyExtensionClass_Export_'\nsrc/Record/_Record.c:502: error: 'RecordType' undeclared (first use in this function)\nerror: Setup script exited with error: command 'gcc' failed with exit status 1\n")
> make[2]: *** [build] Error 1
>

Looks like it can't find Python.h; so python-dev build dependency.

> 3) the debian patch at debian/patches/debian-changes-2.12.10-1 is not
> required, it just contains the Makefile and a test script, the former
> being created by ./configure anyway, and the latter not used in the
> build process to my knowledge.
>
> so you can remove that patch (from debian/series as well).
>
> i guess that patch was created by a dpkg-buildpackage invokation which
> ran dpkg-source, which detected differences between current directory
> and source tarball.
>

I had a feeling that someone might say something about this eventually, 
because it seemed very 'per build' specific.

> in general, the current build directory needs to be clean again after
> invoking 'make clean'. in other words, 'make clean/distclean' needs to
> revoke any changes made by build process.
>
> i guess the problem at your Makefile is, that 'make distclean' doesn't
> remove the created Makefile, thus dpkg-source detects the difference and
> creates the patch.
>

Ah, I see. So is this related to the issue we are having with 
svn-buildpackage?

>>> the best solution would be to automate all preliminary steps in the
>>> debian/rules makefile:
>>>
>>> a target called 'get-orig-source', should automate the process of
>>> building a custom orig.tar.gz from the upstream zope2.12 tarball and all
>>> required zope dependencies.
>>> this target must not be invoked at build process, but instead needs to
>>> be invoked manually.
>>>
>>> many debian packages already provide this 'get-orig-source' target. see
>>> http://www.debian.org/doc/debian-policy/ch-source.html chapter 4.9 for
>>> more information.
>>>
>>> a get-orig-source target also has the advantage, that you don't need to
>>> maintain a custom zope source tarball, but instead have code that
>>> automates the preparation task, which easily can be updated for
>>> subsequent zope releases.
>>>
>>> the custom build scripts (e.g. buildout receipe, configure script)
>>> should be provided as debian patches in debian/patches, and applied to
>>> the source at package build process instead of being part of the
>>> (crafted) source tarball.
>>>
>> The tarball was created out of a request from a few people in the
>> zope/plone community that wanted something more distribution
>> agnostic. So this tarball solution is something that can also be
>> used in other distribution environments. If it's an issue, we can
>> always pull this into the deb, but I'd rather not maintain two
>> versions of the same process.
>>
>> By the way, I'm no longer using zc.buildout in the build process. :)
>>
>> I've also got documentation about the tarball up at
>> http://weblion.psu.edu/static/zope2-tarballs/docs/current/
>
> ok, understood the point. but i'd suggest to implement the
> 'get-orig-source' target anyway. other distributions can use it as well.
>

I definitely agree. I was actually looking at something similar in 
python-markdown. Except, python-markdown's procedure is not part of the 
rules file, it's a separate script; so I like the get-orig-source target 
better.

> at least i would implement a script which crafts your custom tarball
> from upstream sources automaticly in order to make the tarball creation
> process more transparent.
>

Yeah, that sounds like a good idea. I foresee one small issue though. 
We'll need to test it but I think the feature I use to 'fetch' the 
source is only in 0.8.2, but it may be in 0.8.1 which is packaged in 
unstable. We'll need to test it.

I'll also point out another flaw. I've neglected it up to this point, 
but I want to point it out before you notice another mistake. ;)
I'm not pinning to a particular version of python. Since we are building 
binaries I imagine we will likely need to do this.
And, I just noticed that the tarball's make clean doesn't clean the 
source builds (e.g. source/Record/build/*). These do get paved over on 
rebuild, but there is no guarantee that will be the case in the future.

Watch for the next changeset and give it another go. Thanks.

-Michael Mulich (pumazi)