[pymvpa] Suspicious results

[Yaroslav Halchenko]

On Mon, 28 Feb 2011, Nynke van der Laan wrote:
> What I did is the following: I did a searchlight analysis (radius 10
> mm)

which makes it 20mm in diameter, altogether meaning that you could get
"legally" >chance performance in your searchlight center anywhere 1cm
apart from  the actual relevant activation point.  That would be one of
the effects which would add up to the heavy right tail in your resultant
distribution of the performances.  to see how much an effect of this one
-- reduce radius to 1mm and run the same searchlight -- is distribution
loosing its heavy >0.5 bias?

> brain mask). I used a NFoldCrossvalidation (no detrending or
> z-scoring).

well, depending on the actual data and experimental design, absent
detrending might add confounds.

Also, although you have mentioned that every chunk had labels balanced,
what is the output of

dataset.summary()
?


also, because of no z-scoring with not tuned RBF (non-linear) SVM, I am
not sure if it trained correctly per se.... what is the "picture" if you
use Linear SVM? what if you introduce zscoring and detrending?

> I use two stimuluscategories. The task I used consisted of 38 chunks
> (38 trials) with in each chunk two stimuluspresentations (one of each
> category). I have used blockaveraging to reduce features.

blockaveraging reduces samples, not features... ?

> Because I have two stimuluscategories the chance level accuracy would
> thus be 0.5

yes, unless samples are disbalanced across labels/chunks when
classifier might go for the 'overrepresented' class.

> correctly classified) So this would mean that there is predictive
> information in all regions of the brain.. 

well -- more precisely, "every voxel seems to find a relevant diagnostic
neighbor within 10mm radius", so not necessarily carrying predictive
information itself.

> The highest peaks are located at the borders of the brain.

was data motion corrected? was motion correlated with the design? (what
accuracy would obtain by using motion correction
parameters/characteristics such as displacement as your features)

-- 
=------------------------------------------------------------------=
Keep in touch                                     www.onerussian.com
Yaroslav Halchenko                 www.ohloh.net/accounts/yarikoptic