[pymvpa] the effect of ROI size on classification accuracy

J.A. Etzel jetzel at artsci.wustl.edu
Fri Jul 18 15:03:15 UTC 2014

First, 30432 is a LOT of voxels for a ROI-based analysis. Linear svm can 
"pool" information from many voxels and have discontinuous information 
all of which makes it hard to interpret results when two ROIs are 
massively different sizes.

Also, 5 mm Gaussian smoothing isn't all that much at most voxel sizes; 
were your voxels very small? If the voxels were 3x3x3 mm or so I 
wouldn't expect a bit of smoothing to make much difference.

Regarding the actual accuracies you report, I find the dataset #1 ones 
fairly plausible; I'd interpret as ROI A informative, ROI B informative 
(surprisingly so, given how big it is), and ROI A + B informative. 
Probably the three accuracies are not significantly different; there are 
enough informative voxels in both ROIs to have good classification, and 
adding A to B just adds redundant information (or doesn't add enough 
uniquely-informative voxels to improve accuracy further).

But, combined with the accuracies for dataset #2, things are stranger. 
My main suggestion is to double-check everything: the most likely reason 
I can think of for such a big and inconsistent influence of smoothing is 
that something went wrong. For example, visually confirm that the 
smoothed images look like they have the proper amount of smoothing, and 
run both sets of images through the same code.

good luck,

On 7/18/2014 9:38 AM, Meng Liang wrote:
> Dear experts,
> I got some weird results when running MVPA (to distinguish two different
> stimulus categories) on two fMRI datasets with different smoothing using
> three different ROIs. I would like to know your opinion on why this
> could happen.
> I used linear SVM.
> The two datasets are from the same data acquisition but with different
> spatial smoothing: (1) without any spatial smoothing and (2) Gaussian
> smoothing with sigma=5mm
> Three ROIs: (A) brain area A containing 357 voxels, (B) brain area B
> containing 30432 voxels, and (C) brain areas A+B containing
> 357+30432=30789 voxels.
> The classification accuracies when using dataset#1:
>        0.750 for A,
>        0.792 for B,
>        0.792 for A+B
> The classification accuracy when using dataset#2:
>        0.875 for A,
>        0.667 for B,
>        0.583 for A+B
> so, using unsmoothed data, combining A and B did not change the
> classification accuracy. However, using smoothed data, combining A and B
> reduced the classification accuracy considerably and the accuracy was
> not significantly higher than chance level any more (all other
> accuracies were significantly higher than chance level according to
> permutation test).
> I would be grateful if anyone could let me know your thoughts why
> changing the ROI size has different effect on smoothed and unsmoothed data.
> Best,
> Meng
> _______________________________________________
> Pkg-ExpPsy-PyMVPA mailing list
> Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa

Joset A. Etzel, Ph.D.
Research Analyst
Cognitive Control & Psychopathology Lab
Washington University in St. Louis

More information about the Pkg-ExpPsy-PyMVPA mailing list