[pymvpa] Crossvalidation and permutation scheme on one run only

[Yaroslav Halchenko]

On Mon, 30 May 2016, Richard Dinga wrote:

> assuming I didn't screw up loading:

> Counter-balance table for orders up to 2:
> Targets/Order O1                 |  O2               |
>     angry:     0 16  3  1  2  2  |   2  8 5  4 3  2  |
>   baseline:   17  1 16 13 20 12  |   7 37 8 10 4 12  |
>     fear:      2 17  0  1  0  4  |   2  8 2  3 6  3  |
>     happy:     1 17  1  1  2  2  |   4  7 3  2 3  5  |
>    neutral:    3 14  0  4  0  3  |   2 10 4  1 6  1  |
>      sad:      1 15  4  3  0  1  |   7  9 2  3 2  1  |
> Correlations: min=-0.2 max=0.16 mean=-0.005 sum(abs)=13

which is making it "tricky" somewhat:  e.g. in this subject you have
quite a disballance of which trials follow the baseline.  So sad follows
baseline only 12 times, while neutral 20 (so almost twice of the
discrepancy).  Imagine that responses to all conditions (but the
baseline, which I guess you exclude from your analysis?) are the same,
but our assumption about linear additivity of responses is not perfectly
correct (as we actually already know).  Then you will get "signal"
capable of discriminating interesting conditions solely based on their
frequency of following the baseline condition.

with across-subjects classification, if such order effects aren't
replicated across subjects (i.e. design was randomized), it would allow
to mitigate this.

within subject, not sure if it would be easy to balance it out without
heavy sacrifices (e.g. choosing only 12 first trials per each condition
which followed the baseline, but not some other category)

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience     http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik