[pymvpa] GPR woes

[Scott Gorlin]

Epsilon is the Tikhonov regularization parameter, no?  (Haven't read 
much about this classifier, but it looks similar to the RLS solution to 
me...).  As such it should be adjustable to enable punishing overfit 
solutions, so maybe it should actually be a formal parameter.

If the default returns a LinAlg error, you could also stick the cholesky 
decomposition in a finite loop where epsilon is doubled each time 
(though not too many iterations...)

On a wild tangent, also check that your kernel matrix (here self._C) is 
double precision, since 1e-20 is close to the precision limit for single 
floats.

Per B. Sederberg wrote:
> Hi Everybody (and Emanuele in particular):
>
> I've tried GPR on about 5 different datasets and I've never actually
> gotten it to run through an entire cross validation process.  I always
> get the following error:
>
> /home/per/lib/python/mvpa/clfs/gpr.pyc in _train(self, data)
>     308             except SLAError:
>     309                 epsilon = 1.0e-20 * N.eye(self._C.shape[0])
> --> 310                 self._L = SLcholesky(self._C + epsilon, lower=True)
>     311                 self._LL = (self._L, True)
>     312                 pass
>
> /usr/lib/python2.5/site-packages/scipy/linalg/decomp.pyc in
> cholesky(a, lower, overwrite_a)
>     552     potrf, = get_lapack_funcs(('potrf',),(a1,))
>     553     c,info = potrf(a1,lower=lower,overwrite_a=overwrite_a,clean=1)
> --> 554     if info>0: raise LinAlgError, "matrix not positive definite"
>     555     if info<0: raise ValueError,\
>     556        'illegal value in %-th argument of internal potrf'%(-info)
>
> LinAlgError: matrix not positive definite
>
> I think Yarik mentioned changing epsilon or something like that, but
> it seems to me that having to change source code to tweak a classifier
> to not crash is not ideal.
>
> Do you have any suggestions for how to fix this?  I've been tantalized
> with some very good transfer errors in the folds that GPR was able to
> complete before crashing.  Is there a permanent change we could make
> to the GPR code to make it more stable across datasets?  Or could we
> turn epsilon into a configurable variable when setting up the
> classifier?  If so, what should I change epsilon to in order to see if
> I can get it to run?
>
> Thanks,
> Per
>
> _______________________________________________
> Pkg-ExpPsy-PyMVPA mailing list
> Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
> http://lists.alioth.debian.org/mailman/listinfo/pkg-exppsy-pymvpa
>