[pymvpa] Crossvalidation and permutation scheme on one run only
jetzel at wustl.edu
Mon May 30 20:35:48 UTC 2016
Do you have to do within-subjects classification, or could you
cross-validate on the subjects? (I guess that is option 4.) That would also
simplify the permutation testing.
If you need to stick to cross-validation within people, I'd prefer
splitting the dataset into two halves (first half of the run, second half
of the run, which I assume is your option 1) over option 2: adding more
cross-validation folds just compounds the potential confounds, and there's
nothing special about 10 folds.
On May 30, 2016 4:28:19 AM Richard Dinga <dinga92 at gmail.com> wrote:
> I am running within subject searchlight on fast eventrelated data collected
> only in one run per subject. Therefore no naturally independent blocks to
> do my crossvalidation on. So far I thought about 3 approaches:
> 1. delete 20 seconds of data between training and test set
> 2. don't do anything and perform 10 fold CV without any adjustments
> 3. use some non CV based statistic like correlation distance
> although 2 and 3 doesn’t seems reasonable.
> Also, I would like to do a permutation test to find out if my results are
> significant, but I am not sure how to set up a permutation scheme since it
> seems like everything is dependent and nothing is exchangeable.
> Any suggestions what to do in this situation?
> Pkg-ExpPsy-PyMVPA mailing list
> Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Pkg-ExpPsy-PyMVPA