[pymvpa] Confusion Matrix for each Node with sphere_gnbsearchlight
basile pinsard
basile.pinsard at gmail.com
Fri Aug 28 18:40:47 UTC 2015
I wanted to do the same and had to make some changes to PyMVPA here:
https://github.com/bpinsard/PyMVPA/tree/gnbsearchlight_confusiontable
using it with:
errorfx = ConfusionMatrix(labels=ds.uniquetargets)
slght = GNBSearchlight(clf, prtnr, qe, errorfx=errorfx)
On Fri, Aug 28, 2015 at 2:23 PM, marco tettamanti <mrctttmnt at gmail.com>
wrote:
> Thanks again!
> I am on Debian testing (well, reverted on stable now, because of troubles
> with gcc5) and have version 2.3.1.
> I will give a try to the one from git.
> Best,
> Marco
>
>
> PyMVPA:
> Version: 2.3.1
> Hash: d1da5a749dc9cc606bd7f425d93d25464bf43454
> Path: /usr/lib/python2.7/dist-packages/mvpa2/__init__.pyc
> Version control (GIT):
> GIT information could not be obtained due
> "/usr/lib/python2.7/dist-packages/mvpa2/.. is not under GIT"
> SYSTEM:
> OS: posix Linux 4.1.0-1-amd64 #1 SMP Debian 4.1.3-1
> (2015-08-03)
> Distribution: debian/stretch/sid
>
>
> *Yaroslav Halchenko* debian at onerussian.com
> <pkg-exppsy-pymvpa%40lists.alioth.debian.org?Subject=Re%3A%20%5Bpymvpa%5D%20Confusion%20Matrix%20for%20each%20Node%20with%0A%20sphere_gnbsearchlight&In-Reply-To=%3C20150828161509.GS19455%40onerussian.com%3E>
> *Fri Aug 28 16:15:09 UTC 2015*
> ------------------------------
>
> On Fri, 28 Aug 2015, marco tettamanti wrote:
>
> >* Dear Yaroslav,
> *>* thank you very much for your reply. I have made several attempts, trying
> *>* to guess a solution, but it seems I always get a
> *>* 'TypeError: 'NoneType' object is not callable'.
> *
> oh shoot... forgotten that this one was implemented after the last 2.4.0
> release: in upstream/2.4.0-34-g55e147e this June... we should release I
> guess. what system are you on and what version of pymvpa currently?
> if you could use/try the one from git directly... ?
>
> >* Case 1:
> *>* slght = sphere_gnbsearchlight(clf, partitioner, radius=slradius,
> *>* space='voxel_indices', errorfx=None, postproc=mean_sample())
> *
> not the problem here BUT there should be no mean_sample() if errorfx is
> None -- you wouldn't want to average labels ;)
>
> --
> Yaroslav O. Halchenko, Ph.D.http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
> Research Scientist, Psychological and Brain Sciences Dept.
> Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
> Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
> WWW: http://www.linkedin.com/in/yarik
>
>
>
>
> On 08/28/2015 05:28 PM, marco tettamanti wrote:
>
> Dear Yaroslav,
> thank you very much for your reply. I have made several attempts, trying
> to guess a solution, but it seems I always get a
> 'TypeError: 'NoneType' object is not callable'.
>
> Any further advice is greatly appreciated!
> Best,
> Marco
>
>
> Case 1:
> slght = sphere_gnbsearchlight(clf, partitioner, radius=slradius,
> space='voxel_indices', errorfx=None, postproc=mean_sample())
> slght_map = slght(fds)
>
> In [70]: slght = sphere_gnbsearchlight(clf, partitioner, radius=slradius,
> space='voxel_indices', errorfx=None, postproc=mean_sample())
>
> In [71]: slght_map = slght(fds)
> [SLC] DBG: Phase 1. Initializing partitions using
> <NFoldPartitioner> on <Dataset: 108x111 at float32, <sa:
> chunks,targets,time_coords,time_indices>, <fa: voxel_indices>, <a:
> imgaffine,imghdr,imgtype,mapper,voxel_dim,voxel_eldim>>
> [SLC] DBG: Phase 2. Blocking data for 18 splits and 3 labels
> [SLC] DBG: Phase 3. Computing statistics for 54 blocks
> [SLC] DBG: Phase 4. Deducing neighbors information for 111 ROIs
> [SLC] DBG: Phase 4b. Converting neighbors to sparse matrix
> representation
> [SLC] DBG: Phase 5. Major loop
> [SLC] DBG: Split 0 out of 18
> [SLC] DBG: 'Training' is done
> [SLC] DBG: Doing 'Searchlight'
> [SLC] DBG: Assessing accuracies
> ---------------------------------------------------------------------------
> TypeError Traceback (most recent call last)
> <ipython-input-71-1146d298ca06> in <module>()
> ----> 1 slght_map = slght(fds)
>
> /usr/lib/python2.7/dist-packages/mvpa2/base/learner.pyc in __call__(self,
> ds)
> 257 "used and auto training is
> disabled."
> 258 % str(self))
> --> 259 return super(Learner, self).__call__(ds)
> 260
> 261
>
> /usr/lib/python2.7/dist-packages/mvpa2/base/node.pyc in __call__(self, ds)
> 119
> 120 self._precall(ds)
> --> 121 result = self._call(ds)
> 122 result = self._postcall(ds, result)
> 123
>
> /usr/lib/python2.7/dist-packages/mvpa2/measures/searchlight.pyc in
> _call(self, dataset)
> 141
> 142 # pass to subclass
> --> 143 results = self._sl_call(dataset, roi_ids, nproc)
> 144
> 145 if 'mapper' in dataset.a:
>
> /usr/lib/python2.7/dist-packages/mvpa2/measures/adhocsearchlightbase.pyc
> in _sl_call(self, dataset, roi_ids, nproc)
> 513 # error functions without a chance to screw up
> 514 for i, fpredictions in enumerate(predictions.T):
> --> 515 results[isplit, i] = errorfx(fpredictions,
> targets)
> 516
> 517
>
> TypeError: 'NoneType' object is not callable
>
>
>
>
> Similarly for other cases and combinations of them:
>
> Case 2:
> slght = sphere_gnbsearchlight(clf, partitioner, radius=slradius,
> space='voxel_indices', errorfx=ConfusionMatrixError(),
> postproc=mean_sample())
> slght_map = slght(fds)
>
>
> Case3:
> class KeepConfusionMatrix(Node):
> def _call(self, fds):
> out = np.zeros(1, dtype=object)
> out[0] = (fds.samples)
> return out
>
> slght = sphere_gnbsearchlight(clf, partitioner, errorfx=None,
> radius=slradius, space='voxel_indices',
> postproc=ChainNode([Confusion(labels=fds.UT)]))
> slght.postproc.append(KeepConfusionMatrix())
> slght_map = slght(fds)
>
>
> Case4:
> class KeepConfusionMatrix(Node):
> def _call(self, fds):
> out = np.zeros(1, dtype=object)
> out[0] = (fds.samples)
> return out
>
> slght = sphere_gnbsearchlight(clf, partitioner, errorfx=None,
> radius=slradius, space='voxel_indices',
> postproc=ChainNode([mean_sample(),Confusion(labels=fds.UT)]))
> slght.postproc.append(KeepConfusionMatrix())
> slght_map = slght(fds)
>
>
>
> Case5:
> class KeepConfusionMatrix(Node):
> def _call(self, fds):
> out = np.zeros(1, dtype=object)
> out[0] = (fds.samples)
> return out
>
> slght = sphere_gnbsearchlight(clf, partitioner,
> errorfx=ConfusionMatrixError(), radius=slradius, space='voxel_indices',
> postproc=ChainNode([mean_sample(),Confusion(labels=fds.UT)]))
> slght.postproc.append(KeepConfusionMatrix())
> slght_map = slght(fds)
>
>
>
> Yaroslav Halchenko debian at onerussian.com
> Fri Aug 28 13:16:38 UTC 2015
>
> quick an possible partial reply
>
> 1. "not sure" -- if it pukes then probably not, although judging from
> the code I foresaw arbitrary shape of the errorfx output
>
> 2. but you could make sphere_gnbsearchlight to return labels (not
> errors) and then post-process to get those confusion matrices. Just
> specify errorfx=None to it (not to CV). But you could also try
> passing errorfx=ConfusionMatrixError and see how that goes
>
> Please share what you discover/end up with.
> mvpa2/tests/test_usecases.py has more of usecase demos for gnb
> searchlights which might come handy
>
> --
> Yaroslav O. Halchenko, Ph.D.http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
> Research Scientist, Psychological and Brain Sciences Dept.
> Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
> Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
> WWW: http://www.linkedin.com/in/yarik
>
>
> On 08/28/2015 01:48 PM, marco tettamanti wrote:
>
> Dear all,
> is it possible to obtain confusion matrices for all nodes with
> "sphere_gnbsearchlight", as was suggested before with "sphere_searchlight":
>
> slcvte = CrossValidation(clf, partitioner, errorfx=None,
> postproc=ChainNode([Confusion(labels=fds.UT)]))
> class KeepConfusionMatrix(Node):
> def _call(self, fds):
> out = np.zeros(1, dtype=object)
> out[0] = (fds.samples)
> return out
>
> slcvte.postproc.append(KeepConfusionMatrix())
> slght = sphere_searchlight(slcvte, radius=slradius, space='voxel_indices',
> nproc=4, postproc=mean_sample())
> slght_map = slght(fds)
>
>
> Thank you and best wishes,
> Marco
>
> --
> Marco Tettamanti, Ph.D.
> Nuclear Medicine Department & Division of Neuroscience
> San Raffaele Scientific Institute
> Via Olgettina 58
> I-20132 Milano, Italy
> Phone ++39-02-26434888
> Fax ++39-02-26434892
> Email: tettamanti.marco at hsr.it
> Skype: mtettamanti
>
>
>
>
> _______________________________________________
> Pkg-ExpPsy-PyMVPA mailing list
> Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
>
--
Basile Pinsard
*PhD candidate, *
Laboratoire d'Imagerie Biomédicale, UMR S 1146 / UMR 7371, Sorbonne
Universités, UPMC, INSERM, CNRS
*Brain-Cognition-Behaviour Doctoral School **, *ED3C*, *UPMC, Sorbonne
Universités
Biomedical Sciences Doctoral School, Faculty of Medicine, Université de
Montréal
CRIUGM, Université de Montréal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20150828/fc793f7b/attachment-0001.html>
More information about the Pkg-ExpPsy-PyMVPA
mailing list