[pymvpa] Mixed effects and searchlight group significance thresholding

[Mike E. Klein]

Hi everyone,

I'm attempting to threshold group data for a searchlight-based MVPA . I am
performing the group-wise stats via a standard top-level analysis in SPM
(using single-subject searchlight accuracy maps as inputs). I am having
difficulty figuring out where to set significance thresholds. SPM is using
a purely random-effects calculation on the data (n=9, so df=8), leading to
enormous t-thresholds (~16), which are impossible to reach and seem way too
conservative. If I do small-volume corrections on our a-priori regions of
interest, the effects are significant (but not by much...t-thresholds in
the 8s), so this seems less than ideal (and ignores most of the brain).

Typically, for GLM analyses, we use a "mixed effects" model, which
incorporates both within- and between-subjects statistics, yielding an
"effective" degree of freedom (which is much higher than the number of
study participants, though much lower than the total number of trials in
the experiment). However, I am not sure (a) how to calculate this for an
MVPA study or (b) if the same set of assumptions hold. Our nine subjects
each underwent 9 functional runs (used for 8 -> 1 leave one out cross
validation). So each subjects searchlight map was reflecting an average of
these nine folds. We used 81 total examples per condition (9 per run),
which were temporally averaged, leaving 27 examples per condition that were
fed into an SVM. Single-subject results were warped into standard space and
also explicitly smoothed with a 7mm gaussian kernel, before being fed into
SPM.

We have strong results, so really I'm looking for the "most proper" way to
perform searchlight group significance testing. Because we're doing
35,000-45,000 spheres per subject, I don't think permutation testing is
feasible. There's also the option of reporting p<0.05 FWE stats for the
pre-defined ROIs, and p<0.001 (uncorrected) for the rest of the brain, for
completeness sake.

Any advice is greatly appreciated!

Best,
Mike
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20120705/6fb146dc/attachment.html>