[pymvpa] extracting sample predictions from a searchlight

[Yaroslav Halchenko]

On Tue, 05 Apr 2016, Brian Murphy wrote:

> Dear PyMVPAers,

> in a searchlight analysis, I would like to be able to extract the
> individual predictions for each sample, on every cross-validated
> searchlight. 

> Why, I hear you ask? Currently I'm getting very marginal results for my
> classifier, and I would like to boost sensitivity by amalgamating
> results across the 20+ participants that I have access to. The idea
> would be to take a committee classification decision for each test
> sample, across the 20+ participants (e.g. by majority voting of the
> prediction, or a mean of a graded measure like the probabilities that a
> logistic regression gives).

> So far I've looked at the postproc argument of
> mvpa2.measures.searchlight.sphere_searchlight but don't seem to be able
> to dig down deeper than the foldwise error rates. On my reading the
> results_* parameters are not relevant to my question.

> Should I instead be writing a custom function to read out and record
> from each instance of the CrossValidation, with its postproc attribute
> (e.g. via CrossValidation.stats.sets)?

quick one if I got it right -- may be just to provide errorfx=None to
CrossValidation so it doesn't estimate any error and just spits out all
the estimates and then you could compute errors yourself... ?

> My current classifier definition is this:
> > # searchlight classifier
> > searchLightSize = 3
> > clf = PLR(); 

> > cv = CrossValidation(clf, NFoldPartitioner(),
> > enable_ca=['probabilities', 'samples_error','stats',
> > 'calling_time','confusion', 'estimates', 'predictions',
> > 'repetition_results', 'raw_results', 'null_prob'])

> > sl = sphere_searchlight(cv, radius=searchLightSize,
> > postproc=mean_sample())

> Ideas and suggestions welcome!

> best,

> Brian


> _______________________________________________
> Pkg-ExpPsy-PyMVPA mailing list
> Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa


-- 
Yaroslav O. Halchenko
Center for Open Neuroscience     http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik