[Python-apps-team] Bug#464751: Description of the problem

[Ondrej Certik]

Forwarding a description of the problem to the Debian bug.

---------- Forwarded message ----------
From: Stefan Behnel <>
Date: Feb 16, 2008 11:54 AM
Subject: Re: [Cython] Cython 0.9.6.12 released


Hi,


Ondrej Certik wrote:
> On Feb 15, 2008 11:35 AM, Ondrej Certik <ondrej at certik.cz> wrote:
>> On Thu, Feb 14, 2008 at 5:42 AM, Robert Bradshaw
>> <robertwb at math.washington.edu> wrote:
>>> Cython 0.9.6.12 released!
>> It's in Debian now. The following problems still remain:
>>
>> * http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=464751
>> * /usr/bin/cython contains #!/usr/bin/python instead of #!/usr/bin/env python
>
> After discussing the second point with Tim Abbott and Robert Kern, we
> think it is actually a correct behavior and all is fine.
> So the only problem that remains is the Debian bug #464751 - there is
> a simple fix suggested by Stefan, but it probably
> wasn't applied yet.

It wasn't applied as it's not a "fix" but a "work-around". It solves this
problem, but it degenerates the generated C code. Currently, Cython will
generate this:

----------------------------------------
  /*
 * #u = unicode  # doesn't work!!
 * z = unicode('test')             # <<<<<<<<<<<<<<
 */
  __pyx_1 = PyObject_Unicode(__pyx_n_test); if (unlikely(!__pyx_1))
{__pyx_filename = __pyx_f[0]; __pyx_lineno = 11; goto __pyx_L1;}
  if (PyObject_SetAttr(__pyx_m, __pyx_n_z, __pyx_1) < 0) {__pyx_filename =
__pyx_f[0]; __pyx_lineno = 11; goto __pyx_L1;}
  Py_DECREF(__pyx_1); __pyx_1 = 0;
----------------------------------------

With the work-around applied, it becomes this:

----------------------------------------
  /*
 * u = unicode  # works now!!            # <<<<<<<<<<<<<<
 * z = unicode('test')
 */
  if (PyObject_SetAttr(__pyx_m, __pyx_n_u, ((PyObject*)&PyUnicode_Type)) < 0)
{__pyx_filename = __pyx_f[0]; __pyx_lineno = 10; goto __pyx_L1;}

  /*
 * u = unicode  # works now!!
 * z = unicode('test')             # <<<<<<<<<<<<<<
 */
  __pyx_1 = PyTuple_New(1); if (unlikely(!__pyx_1)) {__pyx_filename =
__pyx_f[0]; __pyx_lineno = 11; goto __pyx_L1;}
  Py_INCREF(__pyx_n_test);
  PyTuple_SET_ITEM(__pyx_1, 0, __pyx_n_test);
  __pyx_2 = PyObject_Call(((PyObject*)&PyUnicode_Type), __pyx_1, NULL); if
(unlikely(!__pyx_2)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 11; goto
__pyx_L1;}
  Py_DECREF(__pyx_1); __pyx_1 = 0;
  if (PyObject_SetAttr(__pyx_m, __pyx_n_z, __pyx_2) < 0) {__pyx_filename =
__pyx_f[0]; __pyx_lineno = 11; goto __pyx_L1;}
  Py_DECREF(__pyx_2); __pyx_2 = 0;
----------------------------------------

So you get all the Python call overhead, including tuple packing/unpacking.

The right way to solve this would be to fix the dual identity of types as
Python functions and Python C-API functions. Calls to "unicode()" should
become C-API calls, while the name "unicode" should stay the thing that can be
accessed through __builtin__.

BTW, I wouldn't mind applying the work-around (i.e. to remove the mapping to C
API functions completely). Correct is better than fast if you can't have both
(at least for now). We already do this for other builtins like "type" or "str"
in Symtab.py, so it doesn't add much to it.

Stefan