[Python-modules-team] Bug#467095: python-numpy: numpy.average() returns the wrong result with weights.

[Greg Kochanski]

Package: python-numpy
Version: 1:1.0.4-5
Severity: normal


>>> x = numpy.array([[1,2,3,4,5,6,2],[2,2,3,4,5,6,3]]).transpose()
>>> wt = numpy.array([1,1,1,1,1,1,2])
>>> numpy.average(x, weights=wt, axis=0)
array([[[[[[ 1.,  2.],
           [ 2.,  2.],
           [ 3.,  3.],
           [ 4.,  4.],
           [ 5.,  5.],
           [ 6.,  6.],
           [ 2.,  3.]]]]]])

So, the weighted average of a 2-dimentional array is a 6-dimensional
array?

Or, slightly better:
>>> x = numpy.array([range(100),range(100)]).transpose()   
>>> wt = numpy.array(range(100))
>>> numpy.average(x, weights=wt, axis=0)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/local/lib/python2.4/site-packages/numpy-1.0.4-py2.4-linux-i686.egg/numpy/lib/function_base.py", line 379, in average
    w1 = eval("w["+repr(tuple(r))+"]*ones(ash, float)")
  File "<string>", line 0, in ?
IndexError: too many indices
>>> 


The problem seems to be in .../numpy/lib/function_base.py
right here:

            if wsh == ():
                wsh = (1,)
            if wsh == ash:
                n = add.reduce(a*w, axis)
                d = add.reduce(w, axis)
            elif wsh == (ash[axis],):
                # ni = len(ash)		#CORRECT!
                ni = ash[axis]		# WRONG!
                r = [newaxis]*ni
                r[axis] = slice(None, None, 1)
                w1 = eval("w["+repr(tuple(r))+"]*ones(ash, float)")
                n = add.reduce(a*w1, axis)
                d = add.reduce(w1, axis)
            else:
                raise ValueError, 'averaging weights have wrong shape'


The existing code generates an array whose dimensionality is
equal to shape[axis], so it you have a 2-dimensional array
with a shape of (1000,3), it will try to generate a 1000-dimensional
array on the output.


And, Geez!  Calling eval() in code that is supposed to be fast?
Yikes!  (But that's a separate issue...)


-- System Information:
Debian Release: lenny/sid
  APT prefers testing
  APT policy: (500, 'testing')
Architecture: i386 (i686)

Kernel: Linux 2.6.22-3-686 (SMP w/1 CPU core)
Locale: LANG=en_GB, LC_CTYPE=en_GB (charmap=ISO-8859-1)
Shell: /bin/sh linked to /bin/bash

Versions of packages python-numpy depends on:
ii  atlas3-base [liblapack 3.6.0-20.6        Automatically Tuned Linear Algebra
ii  atlas3-sse2 [liblapack 3.6.0-20.6        Automatically Tuned Linear Algebra
ii  lapack3 [liblapack.so. 3.0.20000531a-6.1 library of linear algebra routines
ii  libc6                  2.7-6             GNU C Library: Shared libraries
ii  libg2c0                1:3.4.6-6         Runtime library for GNU Fortran 77
ii  libgcc1                1:4.3-20080202-1  GCC support library
ii  python                 2.4.4-6           An interactive high-level object-o
ii  python-support         0.7.6             automated rebuilding support for p
ii  refblas3 [libblas.so.3 1.2-8             Basic Linear Algebra Subroutines 3

python-numpy recommends no packages.

-- no debconf information