[pymvpa] group level test on decoding accuracy

[Jingwen Jin]

Thank you very much Christopher!
These information is very helpful and reassuring. I tested a couple of
different minor modifications of the Lee et al., approach and found
consistent results.

Best,
Frances

On Sun, May 10, 2015 at 9:17 AM, Christopher J Markiewicz <effigies at bu.edu>
wrote:

> On 05/09/2015 12:49 PM, Jingwen Jin wrote:
> >
> > Hi MVPA experts,
> >
> > I have a general question about conducting group-level analysis on the
> > subjects' classification accuracy maps. Let's say I am doing a
> > one-sample t-test to find the voxels that have high classification
> > accuracy across subjects. Essentially, I am doing a t-test on percentage
> > numbers (SVM classification accuracy measured as percentage correct).
> > Since percentage is highly affected by the testing example numbers, and
> > in general would probably not meet the normal distribution assumption
> > for t-test.
> >
> > So my question is if people adjust for testing trial numbers or any sort
> > of transformation? For example, I converted each voxel's percentage
> > number to a z score at the individual subject's classification map
> > level, and then do group-level t-test on these z score maps. I wonder if
> > this is valid?
>
> What you describe sounds like the strategy used by Lee et al (2012)
> <https://dx.doi.org/10.1523/JNEUROSCI.3814-11.2012> so there's
> precedent. On the other hand, it's not clear to me how to do cluster
> thresholding for multiple comparisons correction properly, using this
> method. They use SPM8's random effects analysis, but if I recall
> correctly that requires smoothness assumptions, while MVPA analyses
> typically use unsmoothed volumes.
>
> There's also a much more intensive non-parametric test used by Stelzer
> et al. (2012) <https://dx.doi.org/10.1016/j.neuroimage.2012.09.063>, but
> it requires a lot of computing time and at least temporary storage space.
>
> --
> Christopher J Markiewicz
> Ph.D. Candidate, Quantitative Neuroscience Laboratory
> Boston University
>
>
> _______________________________________________
> Pkg-ExpPsy-PyMVPA mailing list
> Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
>



-- 
Jingwen Frances Jin
Department of Psychology
Stony Brook University
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20150511/ebad9374/attachment.html>