[pymvpa] MCNullDist question

[Yaroslav Halchenko]

quick reply, so might be incomplete


On Tue, 09 Jul 2013, Derek Huffman wrote:

>    Hello all,
>    I�am using MCNullDist with enable_ca=['dist_samples'] for significance
>    testing. I am wondering, for subjects with differing number of trials per
>    run (but balanced targets within each run) is the
>    .null_dist.ca.dist_samples returning the correct average accuracy (that
>    is, number_correct / total_number_of_trials, akin to
>    .ca.stats.percent_correct) or is it simply averaging over each
>    cross-validation assuming equal number of trials per run (akin to
>    np.mean(cv_results))? I am assuming it is the former but would appreciate
>    your confirmation.

MCNullDist doesn't average anything -- it just collects samples of
whatever measure you are computing.  If it does averaging across splits
(postproc=mean_sample()) then that is what it would "operate on".

but your usecase is also interesting because of the differing number of
trials per functional run, so I expect chance distribution possibly
diverging even more from a "theoretical" (bi|multi)nomial one in
comparison as if you had equal number of samples across runs.

share your observations happen you find them interesting.

Cheers,
-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate,     Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik