[Python-modules-team] Bug#681270: causes FTBFS of pymvpa2 on s390x: TypeError: Improper input parameters in leastsq call

Yaroslav Halchenko debian at onerussian.com
Thu Jul 12 16:18:43 UTC 2012 

On Thu, 12 Jul 2012, Denis Laxalde wrote:
> tags 681270 +moreinfo

as much as you like/need sir ;)

> Yaroslav Halchenko wrote:
> >Please see attached a somewhat minimalistic code to reproduce the
> >problem.

> >it fails only on s390x -- fine on s390 and any other architecture/port where
> >pymvpa2 was just built.

> >Traceback (most recent call last):
> >   File "scipy-fx-failure2.py", line 65, in<module>
> >     1.47441522,  1.78073152,  2.08704783,  2.39336413,  2.69968044]))
> >   File "scipy-fx-failure2.py", line 52, in least_sq_fit
> >     return leastsq(efx, params)
> >   File "/usr/lib/python2.7/dist-packages/scipy/optimize/minpack.py", line 324, in leastsq
> >     raise errors[info][1](errors[info][0])
> >TypeError: Improper input parameters.


>  I'd help if you
> could attach the full output of your script and also print err in
> efx.

Here is output of a script with print's (attached) -- so it doesn't get to
the next evaluation of efx:

(s390x-sid)yoh at zelenka:~/pymvpa2$ PYTHONPATH=/home/yoh/scipy/python-scipy-0.10.1+dfsg1 /usr/bin/python scipy-fx-failure3.py
EFX:  [  1.00000000e+03   5.00000000e-01   1.00000000e-01   1.00000000e+03
   8.00000000e-01   5.00000000e-02]
[-3.12032937 -2.81401307 -2.50769676 -2.20138046 -1.89506415 -1.58874784
 -1.28243154 -0.97611523 -0.66979893 -0.36348262 -0.05716631  0.24914999
  0.5554663   0.8617826   1.16809891  1.47441522  1.78073152  2.08704783
  2.39336413  2.69968044 -3.12032937 -2.81401307 -2.50769676 -2.20138046
 -1.89506415 -1.58874784 -1.28243154 -0.97611523 -0.66979893 -0.36348262
 -0.05716631  0.24914999  0.5554663   0.8617826   1.16809891  1.47441522
  1.78073152  2.08704783  2.39336413  2.69968044]
ERR:  [  1.00000000e+000  -1.30312768e-235  -1.45985159e-193  -1.37612526e-155
   5.00000000e+000   2.00000000e+000   6.00000000e+000   9.00000000e+000
   6.00000000e+000   1.20000000e+001   1.39992757e+001  -1.62591296e+002
  -3.40866838e+003  -3.71744395e+003   6.99999918e+000   5.00000000e+000
   2.00000000e+000   2.00000000e+000  -5.71959143e-075   1.00000000e+000
   1.00000000e+000  -1.30312768e-235  -1.45985159e-193  -1.37612526e-155
   5.00000000e+000   2.00000000e+000   6.00000000e+000   9.00000000e+000
   6.00000000e+000   1.20000000e+001   1.39992757e+001  -1.62591296e+002
  -3.40866838e+003  -3.71744395e+003   6.99999918e+000   5.00000000e+000
   2.00000000e+000   2.00000000e+000  -5.71959143e-075   1.00000000e+000]
EFX:  [  1.00000000e+03   5.00000000e-01   1.00000000e-01   1.00000000e+03
   8.00000000e-01   5.00000000e-02]
[-3.12032937 -2.81401307 -2.50769676 -2.20138046 -1.89506415 -1.58874784
 -1.28243154 -0.97611523 -0.66979893 -0.36348262 -0.05716631  0.24914999
  0.5554663   0.8617826   1.16809891  1.47441522  1.78073152  2.08704783
  2.39336413  2.69968044 -3.12032937 -2.81401307 -2.50769676 -2.20138046
 -1.89506415 -1.58874784 -1.28243154 -0.97611523 -0.66979893 -0.36348262
 -0.05716631  0.24914999  0.5554663   0.8617826   1.16809891  1.47441522
  1.78073152  2.08704783  2.39336413  2.69968044]
ERR:  [  1.00000000e+000  -1.30312768e-235  -1.45985159e-193  -1.37612526e-155
   5.00000000e+000   2.00000000e+000   6.00000000e+000   9.00000000e+000
   6.00000000e+000   1.20000000e+001   1.39992757e+001  -1.62591296e+002
  -3.40866838e+003  -3.71744395e+003   6.99999918e+000   5.00000000e+000
   2.00000000e+000   2.00000000e+000  -5.71959143e-075   1.00000000e+000
   1.00000000e+000  -1.30312768e-235  -1.45985159e-193  -1.37612526e-155
   5.00000000e+000   2.00000000e+000   6.00000000e+000   9.00000000e+000
   6.00000000e+000   1.20000000e+001   1.39992757e+001  -1.62591296e+002
  -3.40866838e+003  -3.71744395e+003   6.99999918e+000   5.00000000e+000
   2.00000000e+000   2.00000000e+000  -5.71959143e-075   1.00000000e+000]
Traceback (most recent call last):
  File "scipy-fx-failure3.py", line 66, in <module>
    1.47441522,  1.78073152,  2.08704783,  2.39336413,  2.69968044]))
  File "scipy-fx-failure3.py", line 53, in least_sq_fit
    return leastsq(efx, params)
  File "/home/yoh/scipy/python-scipy-0.10.1+dfsg1/scipy/optimize/minpack.py", line 324, in leastsq
    raise errors[info][1](errors[info][0])
TypeError: Improper input parameters.


Here is output of first 3 "blocks" (above there is 2, and then error) on
zelenka in s390 build:

(sid)yoh at zelenka:~/pymvpa2$ python scipy-fx-failure3.py | head -60
EFX:  [  1.00000000e+03   5.00000000e-01   1.00000000e-01   1.00000000e+03
   8.00000000e-01   5.00000000e-02]
[-3.12032937 -2.81401307 -2.50769676 -2.20138046 -1.89506415 -1.58874784
 -1.28243154 -0.97611523 -0.66979893 -0.36348262 -0.05716631  0.24914999
  0.5554663   0.8617826   1.16809891  1.47441522  1.78073152  2.08704783
  2.39336413  2.69968044 -3.12032937 -2.81401307 -2.50769676 -2.20138046
 -1.89506415 -1.58874784 -1.28243154 -0.97611523 -0.66979893 -0.36348262
 -0.05716631  0.24914999  0.5554663   0.8617826   1.16809891  1.47441522
  1.78073152  2.08704783  2.39336413  2.69968044]
ERR:  [  1.00000000e+000  -1.30312768e-235  -1.45985159e-193  -1.37612526e-155
   5.00000000e+000   2.00000000e+000   6.00000000e+000   9.00000000e+000
   6.00000000e+000   1.20000000e+001   1.39992757e+001  -1.62591296e+002
  -3.40866838e+003  -3.71744395e+003   6.99999918e+000   5.00000000e+000
   2.00000000e+000   2.00000000e+000  -5.71959143e-075   1.00000000e+000
   1.00000000e+000  -1.30312768e-235  -1.45985159e-193  -1.37612526e-155
   5.00000000e+000   2.00000000e+000   6.00000000e+000   9.00000000e+000
   6.00000000e+000   1.20000000e+001   1.39992757e+001  -1.62591296e+002
  -3.40866838e+003  -3.71744395e+003   6.99999918e+000   5.00000000e+000
   2.00000000e+000   2.00000000e+000  -5.71959143e-075   1.00000000e+000]
EFX:  [  1.00000000e+03   5.00000000e-01   1.00000000e-01   1.00000000e+03
   8.00000000e-01   5.00000000e-02]
[-3.12032937 -2.81401307 -2.50769676 -2.20138046 -1.89506415 -1.58874784
 -1.28243154 -0.97611523 -0.66979893 -0.36348262 -0.05716631  0.24914999
  0.5554663   0.8617826   1.16809891  1.47441522  1.78073152  2.08704783
  2.39336413  2.69968044 -3.12032937 -2.81401307 -2.50769676 -2.20138046
 -1.89506415 -1.58874784 -1.28243154 -0.97611523 -0.66979893 -0.36348262
 -0.05716631  0.24914999  0.5554663   0.8617826   1.16809891  1.47441522
  1.78073152  2.08704783  2.39336413  2.69968044]
ERR:  [  1.00000000e+000  -1.30312768e-235  -1.45985159e-193  -1.37612526e-155
   5.00000000e+000   2.00000000e+000   6.00000000e+000   9.00000000e+000
   6.00000000e+000   1.20000000e+001   1.39992757e+001  -1.62591296e+002
  -3.40866838e+003  -3.71744395e+003   6.99999918e+000   5.00000000e+000
   2.00000000e+000   2.00000000e+000  -5.71959143e-075   1.00000000e+000
   1.00000000e+000  -1.30312768e-235  -1.45985159e-193  -1.37612526e-155
   5.00000000e+000   2.00000000e+000   6.00000000e+000   9.00000000e+000
   6.00000000e+000   1.20000000e+001   1.39992757e+001  -1.62591296e+002
  -3.40866838e+003  -3.71744395e+003   6.99999918e+000   5.00000000e+000
   2.00000000e+000   2.00000000e+000  -5.71959143e-075   1.00000000e+000]
EFX:  [  1.00000000e+03   5.00000000e-01   1.00000000e-01   1.00000000e+03
   8.00000000e-01   5.00000000e-02]
[-3.12032937 -2.81401307 -2.50769676 -2.20138046 -1.89506415 -1.58874784
 -1.28243154 -0.97611523 -0.66979893 -0.36348262 -0.05716631  0.24914999
  0.5554663   0.8617826   1.16809891  1.47441522  1.78073152  2.08704783
  2.39336413  2.69968044 -3.12032937 -2.81401307 -2.50769676 -2.20138046
 -1.89506415 -1.58874784 -1.28243154 -0.97611523 -0.66979893 -0.36348262
 -0.05716631  0.24914999  0.5554663   0.8617826   1.16809891  1.47441522
  1.78073152  2.08704783  2.39336413  2.69968044]
ERR:  [  1.00000000e+000  -1.30312768e-235  -1.45985159e-193  -1.37612526e-155
   5.00000000e+000   2.00000000e+000   6.00000000e+000   9.00000000e+000
   6.00000000e+000   1.20000000e+001   1.39992757e+001  -1.62591296e+002
  -3.40866838e+003  -3.71744395e+003   6.99999918e+000   5.00000000e+000
   2.00000000e+000   2.00000000e+000  -5.71959143e-075   1.00000000e+000
   1.00000000e+000  -1.30312768e-235  -1.45985159e-193  -1.37612526e-155
   5.00000000e+000   2.00000000e+000   6.00000000e+000   9.00000000e+000
   6.00000000e+000   1.20000000e+001   1.39992757e+001  -1.62591296e+002
  -3.40866838e+003  -3.71744395e+003   6.99999918e+000   5.00000000e+000
   2.00000000e+000   2.00000000e+000  -5.71959143e-075   1.00000000e+000]
...

> Does it occur when dual_gaussian returns np.nan?

not in this case -- but if I always output nan's -- same error with similar output

(s390x-sid)yoh at zelenka:~/pymvpa2$ PYTHONPATH=/home/yoh/scipy/python-scipy-0.10.1+dfsg1 /usr/bin/python scipy-fx-failure3-nan.py 
EFX:  [  1.00000000e+03   5.00000000e-01   1.00000000e-01   1.00000000e+03
   8.00000000e-01   5.00000000e-02]
[-3.12032937 -2.81401307 -2.50769676 -2.20138046 -1.89506415 -1.58874784
 -1.28243154 -0.97611523 -0.66979893 -0.36348262 -0.05716631  0.24914999
  0.5554663   0.8617826   1.16809891  1.47441522  1.78073152  2.08704783
  2.39336413  2.69968044 -3.12032937 -2.81401307 -2.50769676 -2.20138046
 -1.89506415 -1.58874784 -1.28243154 -0.97611523 -0.66979893 -0.36348262
 -0.05716631  0.24914999  0.5554663   0.8617826   1.16809891  1.47441522
  1.78073152  2.08704783  2.39336413  2.69968044]
ERR:  [ nan  nan  nan  nan  nan  nan  nan  nan  nan  nan  nan  nan  nan  nan  nan
  nan  nan  nan  nan  nan  nan  nan  nan  nan  nan  nan  nan  nan  nan  nan
  nan  nan  nan  nan  nan  nan  nan  nan  nan  nan]
EFX:  [  1.00000000e+03   5.00000000e-01   1.00000000e-01   1.00000000e+03
   8.00000000e-01   5.00000000e-02]
[-3.12032937 -2.81401307 -2.50769676 -2.20138046 -1.89506415 -1.58874784
 -1.28243154 -0.97611523 -0.66979893 -0.36348262 -0.05716631  0.24914999
  0.5554663   0.8617826   1.16809891  1.47441522  1.78073152  2.08704783
  2.39336413  2.69968044 -3.12032937 -2.81401307 -2.50769676 -2.20138046
 -1.89506415 -1.58874784 -1.28243154 -0.97611523 -0.66979893 -0.36348262
 -0.05716631  0.24914999  0.5554663   0.8617826   1.16809891  1.47441522
  1.78073152  2.08704783  2.39336413  2.69968044]
ERR:  [ nan  nan  nan  nan  nan  nan  nan  nan  nan  nan  nan  nan  nan  nan  nan
  nan  nan  nan  nan  nan  nan  nan  nan  nan  nan  nan  nan  nan  nan  nan
  nan  nan  nan  nan  nan  nan  nan  nan  nan  nan]
Traceback (most recent call last):
  File "scipy-fx-failure3-nan.py", line 66, in <module>
    1.47441522,  1.78073152,  2.08704783,  2.39336413,  2.69968044]))
  File "scipy-fx-failure3-nan.py", line 53, in least_sq_fit
    return leastsq(efx, params)
  File "/home/yoh/scipy/python-scipy-0.10.1+dfsg1/scipy/optimize/minpack.py", line 324, in leastsq
    raise errors[info][1](errors[info][0])
TypeError: Improper input parameters.


> It is not clear to me why this is a bug in scipy. This exception
> comes from MINPACK's lmdif FORTRAN routine directly. 

hm... and Debian package of scipy falls into the trap of not using system-wide
minpack library but internal scipy's copy:

$> ls **/minpack*
scipy/optimize/minpack.h  scipy/optimize/minpack.py  scipy/optimize/minpack2/minpack2.pyf

scipy/optimize/minpack:
chkder.f  dogleg.f  dpmpar.f  enorm.f  fdjac1.f  fdjac2.f  hybrd.f  hybrd1.f  hybrj.f  hybrj1.f  lmder.f  lmder1.f  lmdif.f  lmdif1.f  lmpar.f  lmstr.f  lmstr1.f  qform.f  qrfac.f  qrsolv.f  r1mpyq.f  r1updt.f  rwupdt.f

scipy/optimize/minpack2:
dcsrch.f  dcstep.f  minpack2.pyf

hence -- bug against scipy ;)

FWIW -- there is no difference in lmdif*.f between scipy's and system-wide
library:

novo:/tmp/python-scipy-0.10.1+dfsg1
$> diff scipy/optimize/minpack/lmdif.f /tmp/minpack-19961126+dfsg1/lmdif.f 
$> diff scipy/optimize/minpack/lmdif1.f /tmp/minpack-19961126+dfsg1/lmdif1.f 
$>

> It is often
> triggered when the objective function returns an array with wrong
> dimensions.

although might be the case due to some casting issue -- it is specific to s390x
-- works just fine on any other architecture, including s390 ;)

BTW -- there is a bulk of possibly related unittests failures on s390x (if only
someone was brave enough to take care about
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=678546), e.g.  (a piece
of diff from output on s390)...

> FAIL: test_datatypes.test_uint64_max
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
>     self.test(*self.arg)
>   File "/usr/lib/python2.7/dist-packages/scipy/ndimage/tests/test_datatypes.py", line 57, in test_uint64_max
>     assert_true(x[1] > (2**63))
> AssertionError: False is not true
> 
> ======================================================================
> FAIL: test_full_output (test_minpack.TestLeastSq)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/dist-packages/scipy/optimize/tests/test_minpack.py", line 160, in test_full_output
>     assert_(ier in (1,2,3,4), 'solution not found: %s'%mesg)
>   File "/usr/lib/pymodules/python2.7/numpy/testing/utils.py", line 34, in assert_
>     raise AssertionError(msg)
> AssertionError: solution not found: Improper input parameters.
> 
> ======================================================================
> FAIL: test_input_untouched (test_minpack.TestLeastSq)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/dist-packages/scipy/optimize/tests/test_minpack.py", line 169, in test_input_untouched
>     assert_(ier in (1,2,3,4), 'solution not found: %s'%mesg)
>   File "/usr/lib/pymodules/python2.7/numpy/testing/utils.py", line 34, in assert_
>     raise AssertionError(msg)
> AssertionError: solution not found: Improper input parameters.
> 
> ======================================================================

full copies of the two are attached now (if get allowed by Debian BTS).
alternatively find them on

http://www.onerussian.com/tmp/nosetests-scipy-s390.log.gz
http://www.onerussian.com/tmp/nosetests-scipy-s390x.log.gz

-- 
Yaroslav O. Halchenko
Postdoctoral Fellow,   Department of Psychological and Brain Sciences
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        
-------------- next part --------------
A non-text attachment was scrubbed...
Name: scipy-fx-failure3.py
Type: text/x-python
Size: 2044 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/python-modules-team/attachments/20120712/1a14d7c8/attachment-0001.py>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: nosetests-scipy-s390.log.gz
Type: application/x-gzip
Size: 40880 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/python-modules-team/attachments/20120712/1a14d7c8/attachment-0002.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: nosetests-scipy-s390x.log.gz
Type: application/x-gzip
Size: 41385 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/python-modules-team/attachments/20120712/1a14d7c8/attachment-0003.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://lists.alioth.debian.org/pipermail/python-modules-team/attachments/20120712/1a14d7c8/attachment-0001.pgp>

More information about the Python-modules-team mailing list