[pymvpa] high prediction rate in a permutation test
Yaroslav Halchenko
debian at onerussian.com
Thu May 19 18:06:17 UTC 2011
sorry for scarce follow-ups
On Thu, 19 May 2011, Jonas Kaplan wrote:
> We've been dealing with this issue as well. Part of our problem is that permutation tests give a limited amount of precision in estimating p values... to find a p value small enough to satisfy a full Bonferoni correction would require ridiculous amounts of iterations. For example, if we do 10,000 iterations we can only estimate what value corresponds to p < .0001. If we do a searchlight on a whole brain with 60K voxels...
this problem could be addressed using parametrization of the
chance-distribution, e.g. assume that it is normal with unknown variance
-- collect e.g. 100 samples, estimate the std -- DONE, get p-values up
to arbitrary precision. In PyMVPA we also have 1 evil but possibly
handy function, match_distribution (e.g. run
doc/examples/match_distribution.py), which you could hunt onto your data
samples to get the best matching distributions across variety of
families (or a specific set of those) available in
scipy.stats.distributions.
--
=------------------------------------------------------------------=
Keep in touch www.onerussian.com
Yaroslav Halchenko www.ohloh.net/accounts/yarikoptic
More information about the Pkg-ExpPsy-PyMVPA
mailing list