[pymvpa] MCNullDist question
debian at onerussian.com
Wed Jul 10 15:09:23 UTC 2013
quick reply, so might be incomplete
On Tue, 09 Jul 2013, Derek Huffman wrote:
> Hello all,
> I�am using MCNullDist with enable_ca=['dist_samples'] for significance
> testing. I am wondering, for subjects with differing number of trials per
> run (but balanced targets within each run) is the
> .null_dist.ca.dist_samples returning the correct average accuracy (that
> is, number_correct / total_number_of_trials, akin to
> .ca.stats.percent_correct) or is it simply averaging over each
> cross-validation assuming equal number of trials per run (akin to
> np.mean(cv_results))? I am assuming it is the former but would appreciate
> your confirmation.
MCNullDist doesn't average anything -- it just collects samples of
whatever measure you are computing. If it does averaging across splits
(postproc=mean_sample()) then that is what it would "operate on".
but your usecase is also interesting because of the differing number of
trials per functional run, so I expect chance distribution possibly
diverging even more from a "theoretical" (bi|multi)nomial one in
comparison as if you had equal number of samples across runs.
share your observations happen you find them interesting.
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate, Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
More information about the Pkg-ExpPsy-PyMVPA