[pymvpa] every permutation returning the same value

[Ben Acland]

Hmm, I'm trying to follow the same pattern as the tuturial at http://www.pymvpa.org/tutorial_significance.html:

>>> distr_est = MCNullDist(repeater, tail='left',
...                        measure=null_cv,
...                        enable_ca=['dist_samples'])

Can you give me a hint as to what I'd need to change?

The cross validation folds should be leave-two-out (one from each subject group), hence the partitioner and sifter. Then, we should do the same analysis PERM_COUNT times, permuting the labels of each training set for each fold.

Here, it looks like dist_est is starting with the same dataset PERM_COUNT times, then building its folds by applying the partitioner, then the sifter, then a single iteration of the permutator to each resulting fold. This sounds like what I want, but isn't coming out that way.

Ben

On Aug 31, 2014, at 10:08 AM, Nick Oosterhof <nikolaas.oosterhof at unitn.it> wrote:

> On Aug 31, 2014, at 4:24 PM, Ben Acland <benacland at gmail.com> wrote:
> 
>> Can anyone see a mistake in the following code that would cause all of my permutation tests to return the same mean error value? 
>> 
>> # cross-validated linear svm with permutation testing (leave out 1 from each sub group)
>> rep = mp.Repeater(count=PERM_COUNT)
> 
> [...]
> 
>> distr_est = mp.MCNullDist(rep,
>>                         tail='left',
>>                         measure=null_cv,
>>                         enable_ca=['dist_samples'])
> 
> [...]
> 
>> ... and at the end of all of this, cvmcc.null_dist.ca.dist_samples contains PERM_COUNT repetitions of a single value.
> 
> You are feeding your Repeater as the permutator argument. It would seem that the MCNullDist has no access to unpermuted datasets. 
> _______________________________________________
> Pkg-ExpPsy-PyMVPA mailing list
> Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20140831/e08e7a78/attachment.html>