[pymvpa] Cross validation and permutation test

[Michael Hanke]

On Tue, Mar 08, 2011 at 06:49:59PM +0000, Meng Liang wrote:
> I was wondering if there is an easy implementation for performing
> permutation test for a cross validation classification? I saw in the
> permutation examples provided in the manual permutation can be built
> in 'TransferError' and the p value can be obtained by '.null_prob'. Is
> there a similar implementation for CrossValidatedTransferError?  Also,
> regarding the permutation example in the manual, I ran this example
> several times and each time I got different p value - sometime very
> significant sometime non-significant. The difference across runs still
> exists even when I increase the permutation to 5000. Any explanation?

Just a quick note (and sorry for the dealy). I'll see whether we can
finally come up with a proper example for PyMVPA 0.6. The changes from
0.5 to 0.6 were partly motivated by the limitations that you are
pointing out. In 0.6 it should be smoother -- we just need to document
it. We'll see whether we can post a full example over the weekend.

Hope that helps,

Michael

-- 
Michael Hanke
http://mih.voxindeserto.de