[pymvpa] extracting sample predictions from a searchlight

[Yaroslav Halchenko]

On Wed, 06 Apr 2016, Brian Murphy wrote:

> Hi,

> > > Should I instead be writing a custom function to read out and record
> > > from each instance of the CrossValidation, with its postproc attribute
> > > (e.g. via CrossValidation.stats.sets)?

> > quick one if I got it right -- may be just to provide errorfx=None to
> > CrossValidation so it doesn't estimate any error and just spits out all
> > the estimates and then you could compute errors yourself... ?
> Cool - I've tried that and it kind of works, but throws a "TypeError:
> cannot perform reduce with flexible type" when its wrapping up after
> computing all the searchlights (see the first code snippet and errors
> below).

> Then I also tried putting in a custom function as errorfx just to print
> out, and tag them on a list for later processing, and it did manage to
> read out the set of estimates from the first fold-wise partition of the
> first searchlight... but then it threw this error "ValueError:
> AttrDataset only supports dtypes as samples that have a `dtype`
> attribute that behaves similar to the one of an array-like." ... again
> the full output is below.

> SNIPPET USING errorfx=None:
> >     ...: miniDs = ds.copy(deep=True)
> >     ...: miniDs = miniDs[:,:10]
> >     ...: searchLightSize = 3
> >     ...: clf = PLR();
> >     ...: cv = CrossValidation(clf, 
> >     ...: 	NFoldPartitioner(),
> >     ...: 	errorfx=None,
> >     ...: 	enable_ca=['probabilities', 'samples_error','stats', 'calling_time','confusion', 'estimates', 'predictions', 'repetition_results', 'raw_results', 'null_prob']
> >     ...: )
> >     ...: sl = sphere_searchlight(cv, radius=searchLightSize, postproc=mean_sample(), nproc=1)

those return just samples which would be predicted labels... what could
be the "mean" of them? ;)  just disable postproc for now to see if you
are getting what you wanted first

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience     http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik