[pymvpa] How to average group-level permutation test in pyMVPA

[Richard Dinga]

You can do it in a way you described, but a better way is to create one big
dataset with all subjects, and run your analysis so that you will get
average accuracy of your within subject classifications. Then you use
attribute_permutator with the limit='subject' or something like that which
will create permuted datasets where permutations are only performed within
subjects. Then you just tepeat your analysis for each permutation to get
the null distrinution of average within subject accuracied. You might also
consider strategy='chunks' to deal with temporal dependencies.

I am sorry i don't have an example code that would run this

On Tue, Apr 20, 2021, 19:43 Xiaobei Zhang <xiaobeiz at usc.edu> wrote:

> Hi all,
> I am trying to do a group-level permutation test in pyMVPA. I have
> obtained individual p-values with permutation test and I think I should
> average the permuted distributions across subjects to get the p-value and
> mean accuracy.
> I am not sure what I should do next and here are the current lines I have:
> sub=[2003,2016,2077,2098,2989,1989......]
> for subject in sub:
> ............ #basic parameters
> clf = LinearCSVMC()
> permutator = AttributePermutator('targets', count=1000)
> distr_est = MCNullDist(permutator, tail='left', enable_ca=['dist_samples'])
> cvte = CrossValidation(clf,splitter,errorfx=mean_mismatch_error,
>                      postproc=mean_sample(),
>                      null_dist=distr_est,enable_ca=['stats'])
>     err=cvte(dataset)
>     cvte.null_dist.append(cvte.null_dist)
>     p = cvte.ca.null_prob
>     assert(p.shape == (1,1))
>     print 'Corresponding p-value:',  np.asscalar(p)
>
> Thanks for your help!
> _______________________________________________
> Pkg-ExpPsy-PyMVPA mailing list
> Pkg-ExpPsy-PyMVPA at alioth-lists.debian.net
> https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/pkg-exppsy-pymvpa/attachments/20210621/78df061e/attachment.htm>