[pymvpa] Crossvalidation and permutation scheme on one run only

[J.A. Etzel]

It's sometimes surprising how decent performance can be even with fairly 
few examples; often classifying with say, only 6 highly temporally 
compressed images in the training set will do better than using a few dozen 
less compressed images. Similarly, I suggest not dismissing the possibility 
of across-subjects classification out of hand; it can work quite well.

Correlation-based analyses (which I assume you mean by RSA) may also be 
suitable, depending on your hypotheses.

Jo


On May 30, 2016 4:31:50 PM Richard Dinga <dinga92 at gmail.com> wrote:

>> Do you have to do within-subjects classification, or could you
> cross-validate on the subjects? (I guess that is option 4.) That would also
> simplify the permutation testing.
>
> yes, I am interested in within subject pattern of activities, possibly RSA
>
>> If you need to stick to cross-validation within people, I'd prefer
> splitting the dataset into two halves (first half of the run, second half
> of the run, which I assume is your option 1) over option 2: adding more
> cross-validation folds just compounds the potential confounds, and there's
> nothing special about 10 folds.
>
> 10 folds has almost twice as many training points, therefore I guess it
> will have higher accuracy, and with split half, there still wouldn't be
> much to permute
>
>
>
> ----------
> _______________________________________________
> Pkg-ExpPsy-PyMVPA mailing list
> Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20160530/1e26a403/attachment.html>