[pymvpa] Sensitivity map with RFE?

marco tettamanti mrctttmnt at gmail.com
Wed Jul 17 21:16:09 UTC 2013


Dear Yaroslav,
thank you a lot for your help!
I was convinced it was my fault because of crappy snippet, I did not suspect 
this could be a bug.
I also confirm the problem with sensvm producing the same result for each fold: 
this is something I had noticed, but I again thought this was conventional 
absolute beginner's experience...

I think that what you are suggesting is that I should go to GitHub and build a 
new pymvpa version from the master. I never did that, but I can try to find out 
how. There should be a way to build a .deb package from git, right?

Anyway, below is the info about my system.

Thank you again and all the best,
Marco

In [3]: mvpa2.wtf()
Out[3]:
Current date:   2013-07-17 23:01
PyMVPA:
  Version:       2.2.0
  Hash:          ad955620e460965ce83c652bc690bea4dc2e21eb
  Path:          /usr/lib/pymodules/python2.7/mvpa2/__init__.pyc
  Version control (GIT):
  GIT information could not be obtained due 
"/usr/lib/pymodules/python2.7/mvpa2/.. is not under GIT"
SYSTEM:
  OS:            posix Linux 3.9-1-amd64 #1 SMP Debian 3.9.8-1
  Distribution:  debian/jessie/sid 

EXTERNALS: 

  Present:       atlas_fsl, cPickle, ctypes, good scipy.stats.rdist, good 
scipy.stats.rv_continuous._reduce_func(floc,fscale), good 
scipy.stats.rv_discrete.ppf, griddata, gzip, h5py, ipython, liblapack.so, 
libsvm, libsvm verbosity control, lxml, matplotlib, mdp, mdp ge 2.4, nibabel, 
nose, numpy, numpy_correct_unique, pprocess, pylab, pylab plottable, pywt, pywt 
wp reconstruct, reportlab, running ipython env, scipy, skl, weave 

  Absent:        atlas_pymvpa, cran-energy, elasticnet, glmnet, hcluster, lars, 
mass, nipy, nipy.neurospin, openopt, pywt wp reconstruct fixed, rpy2, sg ge 
0.6.4, sg ge 0.6.5, sg_fixedcachesize, shogun, shogun.krr, shogun.lightsvm, 
shogun.mpd, shogun.svmocas, shogun.svrlight, statsmodels
  Versions of critical externals: 

   reportlab   : 2.5 

   nibabel     : 1.3.0 

   matplotlib  : 1.1.1rc2 

   scipy       : 0.12.0 

   pprocess    : 0.5 

   ipython     : 0.13.2 

   skl         : 0.13.1 

   mdp         : 3.4 

   numpy       : 1.7.1 

   ctypes      : 1.1.0 

   matplotlib  : 1.1.1rc2 

   lxml        : 3.2.0 

   nifti       : failed to query due to "nifti is not a known dependency key." 

   numpy       : 1.7.1 

   pywt        : 0.2.0 

  Matplotlib backend: TkAgg 

RUNTIME: 

  PyMVPA Environment Variables: 

   PYTHONPATH          : 
":/usr/lib/python2.7/lib-old:/home/marco/data/bll/abstrasomat_pymvpa/abstrasomat_pymvpa_36s:/usr/lib/python2.7/plat-x86_64-linux-gnu:/usr/lib/python2.7/lib-tk:/usr/lib/python2.7/lib-dynload:/usr/bin:.:/usr/lib/python2.7/dist-packages:/usr/lib/python2.7/dist-packages/PIL:/usr/lib/pymodules/python2.7:/usr/lib/python2.7/dist-packages/IPython/extensions:/usr/lib/python2.7:/usr/lib/python2.7/dist-packages/wx-2.8-gtk2-unicode:/home/marco/.python27_compiled:/usr/lib/python2.7/dist-packages/gtk-2.0:/usr/local/lib/python2.7/dist-packages"
  PyMVPA Runtime Configuration:
   [general]
   verbose = 1

   [externals]
   have running ipython env = yes
   have numpy = yes
   have scipy = yes
   have matplotlib = yes
   have h5py = yes
   have reportlab = yes
   have weave = yes
   have good scipy.stats.rdist = yes
   have good scipy.stats.rv_discrete.ppf = yes
   have good scipy.stats.rv_continuous._reduce_func(floc,fscale) = yes
   have pylab = yes
   have lars = no
   have elasticnet = no
   have glmnet = no
   have skl = yes
   have ctypes = yes
   have libsvm = yes
   have shogun = no
   have openopt = no
   have nibabel = yes
   have mdp = yes
   have mdp ge 2.4 = yes
   have statsmodels = no
   have pywt = yes
   have cpickle = yes
   have gzip = yes
   have cran-energy = no
   have griddata = yes
   have nipy.neurospin = no
   have lxml = yes
   have atlas_fsl = yes
   have atlas_pymvpa = no
   have hcluster = no
   have ipython = yes
   have liblapack.so = yes
   have libsvm verbosity control = yes
   have mass = no
   have nipy = no
   have nose = yes
   have numpy_correct_unique = yes
   have pprocess = yes
   have pylab plottable = yes
   have pywt wp reconstruct = yes
   have pywt wp reconstruct fixed = no
   have rpy2 = no
   have sg ge 0.6.4 = no
   have sg ge 0.6.5 = no
   have sg_fixedcachesize = no
   have shogun.krr = no
   have shogun.lightsvm = no
   have shogun.mpd = no
   have shogun.svmocas = no
   have shogun.svrlight = no
  Process Information:
   Name: ipython
   State:        R (running)
   Tgid: 12965
   Pid:  12965
   PPid: 12923
   TracerPid:    0
   Uid:  1000    1000    1000    1000
   Gid:  1000    1000    1000    1000
   FDSize:       256
   Groups:       6 20 24 25 27 29 30 44 46 100 104 113 114 116 1000 1002
   VmPeak:         719900 kB
   VmSize:         712644 kB
   VmLck:               0 kB
   VmPin:               0 kB
   VmHWM:          101504 kB
   VmRSS:          100268 kB
   VmData:         233488 kB
   VmStk:             136 kB
   VmExe:            2280 kB
   VmLib:           60212 kB
   VmPTE:            1096 kB
   VmSwap:              0 kB
   Threads:      3
   SigQ: 0/254508
   SigPnd:       0000000000000000
   ShdPnd:       0000000000000000
   SigBlk:       0000000000000000
   SigIgn:       0000000001001000
   SigCgt:       0000000180000002
   CapInh:       0000000000000000
   CapPrm:       0000000000000000
   CapEff:       0000000000000000
   CapBnd:       0000001fffffffff
   Seccomp:      0
   Cpus_allowed: ff
   Cpus_allowed_list:    0-7
   Mems_allowed: 00000000,00000001
   Mems_allowed_list:    0
   voluntary_ctxt_switches:      1338
   nonvoluntary_ctxt_switches:   171



> [pymvpa] Sensitivity map with RFE?
> Yaroslav Halchenko debian at onerussian.com
> Wed Jul 17 18:49:41 UTC 2013
>
> Previous message: [pymvpa] Sensitivity map with RFE?
> Next message: [pymvpa] how to train/test on a single partition?
> Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
>
> sorry about the delay -- and thanks for buzzing back.
>
> I have tried your snippet with 2.2.0 version as available in Debian --
> reproduced your failure. But it works with the current master of PyMVPA, so we
> fixed something relevant since the release -- could you give it a try with your
> data? let us know if you need instructions (but then tell us about your system
> -- output of mvpa2.wtf() should be sufficient ;) )
>
> another BUT:
> it seems that your construct hits previously reported and presumably
> "fixed" issue:
> https://github.com/PyMVPA/PyMVPA/pull/53
>
> so whenever you check your code, please check -- is your senssvm
> "degenerate" in that all of its samples are the same, e.g. in my case on
> a dummy dataset
>
> *(Pydb) print senssvm.samples
> [[ 2.01502448 2.77870027 0.73660355 2.43673417 0.83915333 0.30925885]
> [ 2.01502448 2.77870027 0.73660355 2.43673417 0.83915333 0.30925885]
> [ 2.01502448 2.77870027 0.73660355 2.43673417 0.83915333 0.30925885]
> [ 2.01502448 2.77870027 0.73660355 2.43673417 0.83915333 0.30925885]]
>
>
>
> On Wed, 17 Jul 2013, marco tettamanti wrote:
>
>> Dear all, I apologize for the repost, but I didn't get any reply and I am
>> pretty stuck with this issue. Is there anybody that could kindly provide any
>> further advice?
>
>> Thank you and very best wishes, Marco
> --
> Yaroslav O. Halchenko, Ph.D.
> http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
> Senior Research Associate, Psychological and Brain Sciences Dept.
> Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
> Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
> WWW: http://www.linkedin.com/in/yarik
>



-------- Original Message --------
Subject: Re: Re: Sensitivity map with RFE?
Date: Fri, 12 Jul 2013 15:13:07 +0200
From: marco tettamanti <tettamanti.marco at hsr.it>
To: pkg-exppsy-pymvpa at lists.alioth.debian.org
<pkg-exppsy-pymvpa at lists.alioth.debian.org>


On 07/17/2013 10:28 AM, marco tettamanti wrote:
> Dear all, I apologize for the repost, but I didn't get any reply and I am
> pretty stuck with this issue.
> Is there anybody that could kindly provide any further advice?
>
> Thank you and very best wishes,
> Marco
>
> -------- Original Message --------
> Subject: Re: Re: Sensitivity map with RFE?
> Date: Fri, 12 Jul 2013 15:13:07 +0200
> From: marco tettamanti<tettamanti.marco at hsr.it>
> To: pkg-exppsy-pymvpa at lists.alioth.debian.org
> <pkg-exppsy-pymvpa at lists.alioth.debian.org>
>
> Dear Roberto,
> thank you very much for your reply!
> I have tried adding a line as you suggested, but nothing changed.
>
> Actually, I thought from the PyMVPA manual that it is the
> 'RepeatedMeasure(sensanasvm, NFoldPartitioner())' that takes care of the
> cross-validation.
>
> I have now tried modifying my code to make it more more similar to the example
> in the RFE help documentation, with two different outcomes:
>
> 1) The following is basically equivalent to the snippet I sent previously and,
> at least in my intentions, modelled upon the sensitivity analysis with feature
> selection.
> With this, I still get the error message "RuntimeError: Cannot reverse-map data
> since the original data shape is unknown. Either set `dshape` in the
> constructor, or call train()."
>
> #-----------------------------
> clfsvm = SplitClassifier(LinearCSVMC(), NFoldPartitioner())
>
> rfesvm = RFE(clfsvm.get_sensitivity_analyzer(postproc=maxofabs_sample()),
> ConfusionBasedError(clfsvm, confusion_state='stats'), Repeater(2),
> fselector=FractionTailSelector(0.30, mode='select', tail='upper'),
> stopping_criterion=NBackHistoryStopCrit(BestDetector(), 10),
> train_pmeasure=False, update_sensitivity=True)
>
> fclfsvm = FeatureSelectionClassifier(clfsvm, rfesvm)
>
> sensanasvm = fclfsvm.get_sensitivity_analyzer(postproc=maxofabs_sample())
>
> cv_sensana_svm = RepeatedMeasure(sensanasvm, NFoldPartitioner())
>
> senssvm = cv_sensana_svm(fds)
>
> print senssvm.shape
> #-----------------------------
>
>
> 2) I have also tried a different solution, which however I do not think that is
> suited to produce a sensitivity map. The following does not yield any errors and
> produces the correct map dimensionality. However, not suprisingly, the result
> does not make any sense, as I get a uniform value spread across all brain mask
> voxels.
>
> #-----------------------------
> clfsvm = SplitClassifier(LinearCSVMC(), NFoldPartitioner())
>
> rfesvm = RFE(clfsvm.get_sensitivity_analyzer(postproc=maxofabs_sample()),
> ConfusionBasedError(clfsvm, confusion_state='stats'), Repeater(2),
> fselector=FractionTailSelector(0.30, mode='select', tail='upper'),
> stopping_criterion=NBackHistoryStopCrit(BestDetector(), 10),
> train_pmeasure=False, update_sensitivity=True)
>
> fclfsvm = FeatureSelectionClassifier(clfsvm, rfesvm)
>
> cvtesvm = CrossValidation(fclfsvm, NFoldPartitioner(), errorfx=lambda p,
> t:np.mean(p == t), postproc=maxofabs_sample(), enable_ca=['confusion', 'stats'])
>
> cv_sensana_svm = RepeatedMeasure(cvtesvm, NFoldPartitioner())
>
> senssvm = cv_sensana_svm(fds)
> #-----------------------------
>
>
>
> Thank you all and very best wishes,
> Marco
>
>> Date: Fri, 12 Jul 2013 11:36:21 +0200
>> From: Roberto Guidotti<robbenson18 at gmail.com>
>> To: Development and support of PyMVPA
>> 	<pkg-exppsy-pymvpa at lists.alioth.debian.org>
>> Subject: Re: [pymvpa] Sensitivity map with RFE?
>> Message-ID:
>> 	<CAGj93cHSQy-SqXE7H7YwP4tGnCEvBHW7ypSX8ENNF6FWLrk_jQ at mail.gmail.com>
>> Content-Type: text/plain; charset="iso-8859-1"
>>
>> Dear Marco,
>>
>>>  From your snippet I notice that you get the sensitivity analyzer without
>> run any CrossValidation/Classification.
>>
>> So try to run
>> -------
>> err = fclfsvm(fds) #That let you to cross validate/classify/select features
>> on your dataset
>> -------
>> This let you to train your object and then you can get the sensitivity of
>> your classifier.
>>
>> I've never used RFE, but this is what I get by your snippet.
>>
>> Ciao
>> Roberto
>>
>>>>   Dear all,
>>>>   is there any manner to obtain a sensitivity map with RFE, similar to what
>>>>   can be done with feature selection?
>>>>
>>>>   I am trying to use the following code:
>>>>
>>>>   #-----------------------------**--------------------
>>>>   clfsvm = LinearCSVMC()
>>>>
>>>>   rfesvm = RFE(clfsvm.get_sensitivity_**analyzer(postproc=maxofabs_**sample()),
>>>>   CrossValidation(clfsvm, NFoldPartitioner(), errorfx=mean_mismatch_error,
>>>>   postproc=mean_sample()), Repeater(2), fselector=**FractionTailSelector(0.30,
>>>>   mode='select', tail='upper'), stopping_criterion=**NBackHistoryStopCrit(**BestDetector(),
>>>>   10), update_sensitivity=True)
>>>>
>>>>   fclfsvm = FeatureSelectionClassifier(**clfsvm, rfesvm)
>>>>
>>>>   sensanasvm = fclfsvm.get_sensitivity_**analyzer(postproc=maxofabs_**
>>>>   sample())
>>>>
>>>>   cv_sensana_svm = RepeatedMeasure(sensanasvm, NFoldPartitioner())
>>>>
>>>>   senssvm = cv_sensana_svm(fds)
>>>>   #-----------------------------**--------------------
>>>>
>>>>   However, after a while I get the following error:
>>>>
>>>>   RuntimeError: Cannot reverse-map data since the original data shape is
>>>>   unknown. Either set `dshape` in the constructor, or call train().
>>>>
>>>>
>>>>   Thank you in advance for any help!
>>>>   Best wishes,
>>>>   Marco
>

-- 
Marco Tettamanti, Ph.D.
Nuclear Medicine Department & Division of Neuroscience
San Raffaele Scientific Institute
Via Olgettina 58
I-20132 Milano, Italy
Phone ++39-02-26434888
Fax ++39-02-26434892
Email: tettamanti.marco at hsr.it
Skype: mtettamanti



More information about the Pkg-ExpPsy-PyMVPA mailing list