[pymvpa] Within-subject classification, getting each subject's scores averaged over number of runs

[Yaroslav Halchenko]

On Fri, 22 Apr 2016, Martin Sjøgård wrote:

>    I'm running some within-subject classifications in pyMVPA (same as the
>    example script on the site/in the manual in the hyperalignment section).
>    Everything works fine, and I get the average within-subject classification
>    scores. However, I also want to the classification scores for each
>    subject.

>    The results are organized by the lines:

>    wsc_results = [cv(sd) for sd in sd_all)

>    wsc_results = vstack(wsc_results)

>    If I type in:

>    print wsc_results

>    I get each subject's scores per run (7 runs), but I of course want to
>    average the runs per subject. I've tried:

>    wsc_results = [cv(sd)/nruns for sd in sd_all)

>    But that doesn't work (TypeError: unsupported operand type(s) for /:
>    'Dataset' and 'int').

>    Do you know how to get each subject's classification scores, averaged over
>    number of runs?

even this would work

[np.mean(cv(sd)) for sd in sd_all]

but I would recommend to collect not just "mean scores" but all the
confusion matrices "just in case", e.g.

# enable_ca=['stats'] for your cv definion
cms = []
errors = []
for sd in sd_all:
 errors.append(np.mean(cv(sd)))
 cms.append(cv.ca.stats)

then you can get all kinds of stats, e.g.

[cm.stats['ACC%'] for cm in cms]

would be your mean accuracies as well
-- 
Yaroslav O. Halchenko
Center for Open Neuroscience     http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik