[pymvpa] every permutation returning the same value
Ben Acland
benacland at gmail.com
Sun Aug 31 14:24:08 UTC 2014
Can anyone see a mistake in the following code that would cause all of my permutation tests to return the same mean error value?
# cross-validated linear svm with permutation testing (leave out 1 from each sub group)
rep = mp.Repeater(count=PERM_COUNT)
perm = mp.AttributePermutator('sub_group', limit={'partitions': 1}, count=1)
part = mp.NFoldPartitioner(cvtype=2, attr="subject")
sift = mp.Sifter([('partitions',2),
('sub_group', dict(uvalues=np.unique(clf_ds.sa.sub_group),
balanced=True))])
kernel = mp.CachedKernel(mp.LinearSGKernel(normalizer_cls=False))
clf = svmsg.SVM(svm_impl='libsvm', kernel=kernel)
null_cv = mp.CrossValidation(
clf,
mp.ChainNode([part, sift, perm],
space=part.get_space()),
postproc=mp.mean_sample())
distr_est = mp.MCNullDist(rep,
tail='left',
measure=null_cv,
enable_ca=['dist_samples'])
cvmcc = mp.CrossValidation(clf,
mp.ChainNode([part, sift],
space=part.get_space())
postproc=mp.mean_sample(),
null_dist=distr_est,
enable_ca=['stats'])
kernel.compute(clf_ds)
result = cvmcc(clf_ds)
... and at the end of all of this, cvmcc.null_dist.ca.dist_samples contains PERM_COUNT repetitions of a single value.
Ben
More information about the Pkg-ExpPsy-PyMVPA
mailing list