[pymvpa] Pattern localization

[Matthias Ekman]

Hi,

> Sorry about the delay Matthias, we have been traveling a bit ;)
> See comments embedded:
> 
>> I am trying to create a "discriminative map" which shows which features  
>> are driving the classifiers performance. The procedure (below) will give  
>> me one map per Nfold. My idea was, to calculate a mean map of those maps  
>> and (may be) threshold it at an arbitrary level.
> As for mean map, the easiest way is to use SplitClassifier and its
> sensitivity map which is by default is composed by taking mean
> sensitivity from all splits. Something like what we have done in
> http://www.frontiersin.org/neuroinformatics/paper/10.3389/neuro.11/003.2009/
> look into supplementals for full code / comments. But it would look like
> (clf can well be yours FeatureSelection one)
Thanks for the hint! I've had a look at those supplementals some time 
ago and forgot it in the meanwhile :-)

> 
>         sclf = SplitClassifier(clf, splitter,
>             enable_states=['confusion', 'training_confusion'])
> 
>         sensitivities = sclf.getSensitivityAnalyzer(
>             # do not combine sensitivities across splits, nor across classes
>             combiner=None, slave_combiner=None)(ds)
> 
> which would give you non-meaned set of sensitivities. If you like to have
> them meaned, just use 'combiner=FirstAxisMean', or in current version just
> don't specify combiner at all -- Mean value will be taken as the default
> behavior (it should change in upcoming versions where we would not impose such
> defaults -- better be explicit ;))
OK!

> 
>> The alternative I was thinking about, is a "stability analysis", that  
>> picks voxels which have consistent clf weights across cross-validation.
> 
> Well -- with feature selection it gets trickier since some features might
> not be selected in some of the splits, so you have a choice of judging
> stability may be just by the portion of the splits where this feature was
> selected at all
OK!
> 
>> What do you think would be an appropriate way (for linear classifiers)  
>> of visualizing voxels which most strongly affect classifier predictions?
> 
> so -- either those raw (mean) sensitivities, or their stability assessment
> across splits (just make sure that you normalized your features prior doing
> that, with zscore or smth else).  If there is no feature selection, than,
> permutation testing F- or converted z-scores might be also a nice one.
> In both cases though you need to decide on the thresholding scheme ;)
I looked through some papers and it seems to me, that this point is 
somehow arbitrary. Some guys are taking the "best" 20% features, some 
others discard those features with SD > 2, others show those features 
which have an overlap with other subjects features... those were noise 
perturbation shows an min. effect of 30% ... and so on.
> 
> I hope this helps
Yes! Thank you very much!


In your paper (Hanson & Halchenko, 2008) you used a recursive feature 
elimination algorithm to sequentially eliminated features with the 
smallest squared values of the separating plane normal coefficients. 
Then you derived weights for the FACE class by using only FACE class SVs 
(and for the HOUSE class in the same way).

Is there also some supplemental (may be I forgot about) where it is 
shown how to extract only special class SVs? Or would it be possible to 
explain this procedure?


Cheers,
Matthias

> -- 
> Yaroslav Halchenko
> Research Assistant, Psychology Department, Rutgers-Newark
> Student  Ph.D. @ CS Dept. NJIT
> Office: (973) 353-1412 | FWD: 82823 | Fax: (973) 353-1171
>         101 Warren Str, Smith Hall, Rm 4-105, Newark NJ 07102
> WWW:     http://www.linkedin.com/in/yarik        
>