[pymvpa] every permutation returning the same value

Ben Acland benacland at gmail.com
Sun Aug 31 14:24:08 UTC 2014


Can anyone see a mistake in the following code that would cause all of my permutation tests to return the same mean error value? 

# cross-validated linear svm with permutation testing (leave out 1 from each sub group)
rep = mp.Repeater(count=PERM_COUNT)
perm = mp.AttributePermutator('sub_group', limit={'partitions': 1}, count=1)
part = mp.NFoldPartitioner(cvtype=2, attr="subject")
sift = mp.Sifter([('partitions',2), 
    ('sub_group', dict(uvalues=np.unique(clf_ds.sa.sub_group),
        balanced=True))])
kernel = mp.CachedKernel(mp.LinearSGKernel(normalizer_cls=False))
clf = svmsg.SVM(svm_impl='libsvm', kernel=kernel)
null_cv = mp.CrossValidation(
    clf,
    mp.ChainNode([part, sift, perm],
                 space=part.get_space()),
    postproc=mp.mean_sample())
distr_est = mp.MCNullDist(rep,
                          tail='left',
                          measure=null_cv,
                          enable_ca=['dist_samples'])
cvmcc = mp.CrossValidation(clf,
                           mp.ChainNode([part, sift],
                               space=part.get_space())
                           postproc=mp.mean_sample(),
                           null_dist=distr_est,
                           enable_ca=['stats'])
kernel.compute(clf_ds)
result = cvmcc(clf_ds)

... and at the end of all of this, cvmcc.null_dist.ca.dist_samples contains PERM_COUNT repetitions of a single value.

Ben


More information about the Pkg-ExpPsy-PyMVPA mailing list