[Python-modules-team] Bug#467095: python-numpy: numpy.average() returns the wrong result with weights.

[Ondrej Certik]

On Sat, Feb 23, 2008 at 1:08 AM, Greg Kochanski <gpk at kochanski.org> wrote:
> Package: python-numpy
>  Version: 1:1.0.4-5
>  Severity: normal
>
>
>  >>> x = numpy.array([[1,2,3,4,5,6,2],[2,2,3,4,5,6,3]]).transpose()
>  >>> wt = numpy.array([1,1,1,1,1,1,2])
>  >>> numpy.average(x, weights=wt, axis=0)
>  array([[[[[[ 1.,  2.],
>            [ 2.,  2.],
>            [ 3.,  3.],
>            [ 4.,  4.],
>            [ 5.,  5.],
>            [ 6.,  6.],
>            [ 2.,  3.]]]]]])
>
>  So, the weighted average of a 2-dimentional array is a 6-dimensional
>  array?
>
>  Or, slightly better:
>  >>> x = numpy.array([range(100),range(100)]).transpose()
>  >>> wt = numpy.array(range(100))
>  >>> numpy.average(x, weights=wt, axis=0)
>  Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
>   File "/usr/local/lib/python2.4/site-packages/numpy-1.0.4-py2.4-linux-i686.egg/numpy/lib/function_base.py", line 379, in average
>     w1 = eval("w["+repr(tuple(r))+"]*ones(ash, float)")
>   File "<string>", line 0, in ?
>  IndexError: too many indices
>  >>>
>
>
>  The problem seems to be in .../numpy/lib/function_base.py
>  right here:
>
>             if wsh == ():
>                 wsh = (1,)
>             if wsh == ash:
>                 n = add.reduce(a*w, axis)
>                 d = add.reduce(w, axis)
>             elif wsh == (ash[axis],):
>                 # ni = len(ash)         #CORRECT!
>                 ni = ash[axis]          # WRONG!
>                 r = [newaxis]*ni
>                 r[axis] = slice(None, None, 1)
>                 w1 = eval("w["+repr(tuple(r))+"]*ones(ash, float)")
>                 n = add.reduce(a*w1, axis)
>                 d = add.reduce(w1, axis)
>             else:
>                 raise ValueError, 'averaging weights have wrong shape'
>
>
>  The existing code generates an array whose dimensionality is
>  equal to shape[axis], so it you have a 2-dimensional array
>  with a shape of (1000,3), it will try to generate a 1000-dimensional
>  array on the output.

Thanks for the bug report and a fix. Reported upstream:

http://projects.scipy.org/pipermail/numpy-discussion/2008-February/031562.html

>  And, Geez!  Calling eval() in code that is supposed to be fast?
>  Yikes!  (But that's a separate issue...)

Please report the issue. :) Even better - offer upstream a fix
directly on their mailinglist.

Thanks a lot and continue bug reporting,
Ondrej