[pymvpa] Crossvalidation and permutation scheme on one run only
jetzel at wustl.edu
Tue May 31 01:53:11 UTC 2016
It's sometimes surprising how decent performance can be even with fairly
few examples; often classifying with say, only 6 highly temporally
compressed images in the training set will do better than using a few dozen
less compressed images. Similarly, I suggest not dismissing the possibility
of across-subjects classification out of hand; it can work quite well.
Correlation-based analyses (which I assume you mean by RSA) may also be
suitable, depending on your hypotheses.
On May 30, 2016 4:31:50 PM Richard Dinga <dinga92 at gmail.com> wrote:
>> Do you have to do within-subjects classification, or could you
> cross-validate on the subjects? (I guess that is option 4.) That would also
> simplify the permutation testing.
> yes, I am interested in within subject pattern of activities, possibly RSA
>> If you need to stick to cross-validation within people, I'd prefer
> splitting the dataset into two halves (first half of the run, second half
> of the run, which I assume is your option 1) over option 2: adding more
> cross-validation folds just compounds the potential confounds, and there's
> nothing special about 10 folds.
> 10 folds has almost twice as many training points, therefore I guess it
> will have higher accuracy, and with split half, there still wouldn't be
> much to permute
> Pkg-ExpPsy-PyMVPA mailing list
> Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Pkg-ExpPsy-PyMVPA