[pymvpa] group level test on decoding accuracy

[Christopher J Markiewicz]

On 05/09/2015 12:49 PM, Jingwen Jin wrote:
> 
> Hi MVPA experts,
> 
> I have a general question about conducting group-level analysis on the
> subjects' classification accuracy maps. Let's say I am doing a
> one-sample t-test to find the voxels that have high classification
> accuracy across subjects. Essentially, I am doing a t-test on percentage
> numbers (SVM classification accuracy measured as percentage correct).
> Since percentage is highly affected by the testing example numbers, and
> in general would probably not meet the normal distribution assumption
> for t-test. 
> 
> So my question is if people adjust for testing trial numbers or any sort
> of transformation? For example, I converted each voxel's percentage
> number to a z score at the individual subject's classification map
> level, and then do group-level t-test on these z score maps. I wonder if
> this is valid?

What you describe sounds like the strategy used by Lee et al (2012)
<https://dx.doi.org/10.1523/JNEUROSCI.3814-11.2012> so there's
precedent. On the other hand, it's not clear to me how to do cluster
thresholding for multiple comparisons correction properly, using this
method. They use SPM8's random effects analysis, but if I recall
correctly that requires smoothness assumptions, while MVPA analyses
typically use unsmoothed volumes.

There's also a much more intensive non-parametric test used by Stelzer
et al. (2012) <https://dx.doi.org/10.1016/j.neuroimage.2012.09.063>, but
it requires a lot of computing time and at least temporary storage space.

-- 
Christopher J Markiewicz
Ph.D. Candidate, Quantitative Neuroscience Laboratory
Boston University

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 455 bytes
Desc: OpenPGP digital signature
URL: <http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20150510/5873c589/attachment.sig>