[pymvpa] effect size (in lieu of zscore)

[Francisco Pereira]

On Wed, Jan 4, 2012 at 6:04 PM, J.A. Etzel <jetzel at artsci.wustl.edu> wrote:
>
> Non-independence is definitely a big issue (not the only one, unfortunately).

It's most definitely an issue. Michael Hanke helped me think about
this and we tried to suggest one possibility for testing for the
presence of that in page 7 of
http://www.princeton.edu/~fpereira/Papers/searchmight.pdf
(pardon the self reference)

There will be code to help diagnose this in the next version of
Searchmight. That said, I think that this is more of a problem in
terms of making results that are marginal appear significant, rather
than introducing bias one way or another.

> doesn't seem unusual to have a few subjects with below-chance classification
> in large numbers of voxels; I don't think the current methods deal with this (or even explain it) very well.

Right, with a large number of voxels where the underlying accuracy is
at chance level a fraction of will them will appear much worse than
chance, just as a fraction will appear much better than chance. With
enough subjects...

Francisco