[pymvpa] Cross validation and permutation test

[Meng Liang]

Thanks, Yaroslav, I've had a glance at the tutorial for 0.6 version and it looks like I do have to learn twice... Is there a PDF file of the tutorial? - it would be easier to read and search for specific commands.
For performing permutation test for cross validation on 0.4 version, if there isn't an easy way to do the permutation directly using CrossValidatedTransferError, is the following code (using TransferError) correct?
#===========nd = MCNullDist(permutations=2500,tail='left',enable_states=['dist_samples'])terr = TransferError(clf,null_dist=nd)        cv_index = 0splitter = NFoldSplitter(cvtype=1)for wdata, vdata in splitter(dataset):   err = terr(vdata,wdata)   Error[SubjIndex,cv_index] = err   Pvalue[SubjIndex,cv_index] = terr.null_prob   Distribution_normalizedMeanSD = nd.dist_samples   nd.clean()   cv_index += 1#===========
In this way, I can get the null distribution for each split (i.e. dist_samples). The prediction error of the cross-validation is the average of prediction error of all splits, I suppose I will need to generate another null distribution for the mean predication error by sampling from these null distributions of all split and then calculating the mean of the new samples and repeat the procedure for, e.g., 1000 times? Is it correct?

Best,Meng

> Date: Fri, 11 Mar 2011 16:03:36 -0500
> From: debian at onerussian.com
> To: pkg-exppsy-pymvpa at lists.alioth.debian.org
> Subject: Re: [pymvpa] Cross validation and permutation test
> 
> yes -- 0.6 version diverges from 0.4 quite significantly.  that is why I
> recommended to start learning it instead of 0.4 so you would not need to
> learn twice.
> 
> For 0.6 there is a good tutorial which would guide you through:
> http://www.pymvpa.org/tutorial.html
> 
> 
> On Fri, 11 Mar 2011, Meng Liang wrote:
> 
> >    I removed python-mvpa and installed python-mvpa-snapshot, everything
> >    seemed fine. However, after I installed the 0.6.0~rc2 version, I got
> >    error message when I tried to import 'detrend' and 'NiftiDataset'.
> -- 
> =------------------------------------------------------------------=
> Keep in touch                                     www.onerussian.com
> Yaroslav Halchenko                 www.ohloh.net/accounts/yarikoptic
> 
> _______________________________________________
> Pkg-ExpPsy-PyMVPA mailing list
> Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
> http://lists.alioth.debian.org/mailman/listinfo/pkg-exppsy-pymvpa
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20110311/33dc3173/attachment.htm>