[pymvpa] Train and test on different classes from a dataset

Yaroslav Halchenko debian at onerussian.com
Tue Feb 5 00:37:51 UTC 2013

On Sat, 02 Feb 2013, Michael Hanke wrote:

>      now (if you are already asleep, I will try doing it later) - just extend
>      it into the power (or ROC detection) analysis -- what we are actually
>      after here.  Add SNR=0.00 -- collect both types of samples and
>      describe each plot with its power to detect the true signal using both
>      kinds of permutation.   Ideally your power should grow with SNR ;)

>    I am not asleep, but I am having a good time doing nothing ;-)

I have tortured your script a bit to bootstrap multiple cases and add
plotting ROCs (even cheated and used scikit-learn for that since we
apparently need overhaul of ROCCurve).  As you see, keeping
testing  portion intact results in lower detection power

Yaroslav O. Halchenko
Postdoctoral Fellow,   Department of Psychological and Brain Sciences
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        
-------------- next part --------------
A non-text attachment was scrubbed...
Name: permutation_test_rocs.png
Type: image/png
Size: 100020 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20130204/a36533de/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: permutation_test_rocs.py
Type: text/x-python
Size: 4574 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20130204/a36533de/attachment-0001.py>

More information about the Pkg-ExpPsy-PyMVPA mailing list