[pymvpa] high prediction rate in a permutation test

[Yaroslav Halchenko]

sorry for scarce follow-ups

On Thu, 19 May 2011, Jonas Kaplan wrote:

> We've been dealing with this issue as well.  Part of our problem is that permutation tests give a limited amount of precision in estimating p values... to find a p value small enough to satisfy a full Bonferoni correction would require ridiculous amounts of iterations.   For example, if we do 10,000 iterations we can only estimate what value corresponds to p < .0001.  If we do a searchlight on a whole brain with 60K voxels... 

this problem could be addressed using parametrization of the
chance-distribution, e.g. assume that it is normal with unknown variance
-- collect e.g. 100 samples, estimate the std -- DONE, get p-values up
to arbitrary precision.  In PyMVPA we also have 1 evil but possibly
handy function, match_distribution (e.g. run
doc/examples/match_distribution.py), which you could hunt onto your data
samples to get the best matching distributions across variety of
families (or a specific set of those) available in
scipy.stats.distributions.


-- 
=------------------------------------------------------------------=
Keep in touch                                     www.onerussian.com
Yaroslav Halchenko                 www.ohloh.net/accounts/yarikoptic