[pymvpa] Crossvalidation and permutation scheme on one run only

[J.A. Etzel]

Do you have to do within-subjects classification, or could you 
cross-validate on the subjects? (I guess that is option 4.) That would also 
simplify the permutation testing.

If you need to stick to cross-validation within people, I'd prefer 
splitting the dataset into two halves (first half of the run, second half 
of the run, which I assume is your option 1) over option 2: adding more 
cross-validation folds just compounds the potential confounds, and there's 
nothing special about 10 folds.

Jo


On May 30, 2016 4:28:19 AM Richard Dinga <dinga92 at gmail.com> wrote:

> Hi,
> I am running within subject searchlight on fast eventrelated data collected
> only in one run per subject. Therefore no naturally independent blocks to
> do my crossvalidation on. So far I thought about 3 approaches:
>
> 1. delete 20 seconds of data between training and test set
> 2. don't do anything and perform 10 fold CV without any adjustments
> 3. use some non CV based statistic like correlation distance
>
> although 2 and 3 doesn’t seems reasonable.
>
> Also, I would like to do a permutation test to find out if my results are
> significant, but I am not sure how to set up a permutation scheme since it
> seems like everything is dependent and nothing is exchangeable.
>
> Any suggestions what to do in this situation?
>
>
>
> ----------
> _______________________________________________
> Pkg-ExpPsy-PyMVPA mailing list
> Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20160530/2461e4e8/attachment.html>