[pymvpa] Confusion Matrix for each Node with sphere_gnbsearchlight
marco tettamanti
mrctttmnt at gmail.com
Sat Aug 29 07:36:11 UTC 2015
Dear Basile,
thank you for sharing, I will try this out!
Best,
Marco
On 08/29/2015 09:26 AM, Marco Tettamanti wrote:
> Date: Fri, 28 Aug 2015 14:40:47 -0400
> From: basile pinsard <basile.pinsard at gmail.com>
> To: Development and support of PyMVPA <pkg-exppsy-pymvpa at lists.alioth.debian.org>
> Subject: Re: [pymvpa] Confusion Matrix for each Node with sphere_gnbsearchlight
>
> I wanted to do the same and had to make some changes to PyMVPA here:
> https://github.com/bpinsard/PyMVPA/tree/gnbsearchlight_confusiontable
> using it with:
> errorfx = ConfusionMatrix(labels=ds.uniquetargets)
> slght = GNBSearchlight(clf, prtnr, qe, errorfx=errorfx)
>
> On Fri, Aug 28, 2015 at 2:23 PM, marco tettamanti <mrctttmnt at gmail.com>
> wrote:
>
> Thanks again! I am on Debian testing (well, reverted on stable now,
> because of troubles with gcc5) and have version 2.3.1. I will give a try
> to the one from git. Best, Marco PyMVPA: Version: 2.3.1 Hash:
> d1da5a749dc9cc606bd7f425d93d25464bf43454 Path:
> /usr/lib/python2.7/dist-packages/mvpa2/__init__.pyc Version control (GIT):
> GIT information could not be obtained due
> "/usr/lib/python2.7/dist-packages/mvpa2/.. is not under GIT" SYSTEM: OS:
> posix Linux 4.1.0-1-amd64 #1 SMP Debian 4.1.3-1 (2015-08-03) Distribution:
> debian/stretch/sid *Yaroslav Halchenko* debian at onerussian.com
> <http://onerussian.com> <pkg-exppsy-pymvpa%40lists.alioth.debian.org
> <http://40lists.alioth.debian.org>?Subject=Re%3A%20%5Bpymvpa%5D%20Confusion%20Matrix%20for%20each%20Node%20with%0A%20sphere_gnbsearchlight&In-Reply-To=%3C20150828161509.GS19455%40onerussian.com
> <http://40onerussian.com>%3E> *Fri Aug 28 16:15:09 UTC 2015*
> --------------------------------------------------------------------------------
> On Fri, 28 Aug 2015, marco tettamanti wrote:
>
> * Dear Yaroslav,
>
> *>* thank you very much for your reply. I have made several attempts,
> trying *>* to guess a solution, but it seems I always get a *>*
> 'TypeError: 'NoneType' object is not callable'. * oh shoot... forgotten
> that this one was implemented after the last 2.4.0 release: in
> upstream/2.4.0-34-g55e147e this June... we should release I guess. what
> system are you on and what version of pymvpa currently? if you could
> use/try the one from git directly... ?
>
> * Case 1:
>
> *>* slght = sphere_gnbsearchlight(clf, partitioner, radius=slradius, *>*
> space='voxel_indices', errorfx=None, postproc=mean_sample()) * not the
> problem here BUT there should be no mean_sample() if errorfx is None --
> you wouldn't want to average labels ;) -- Yaroslav O. Halchenko,
> Ph.D.http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
> Research Scientist, Psychological and Brain Sciences Dept. Dartmouth
> College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1
> (603) 646-9834 Fax: +1 (603) 646-1419 WWW:
> http://www.linkedin.com/in/yarik On 08/28/2015 05:28 PM, marco tettamanti
> wrote: Dear Yaroslav, thank you very much for your reply. I have made
> several attempts, trying to guess a solution, but it seems I always get a
> 'TypeError: 'NoneType' object is not callable'. Any further advice is
> greatly appreciated! Best, Marco Case 1: slght =
> sphere_gnbsearchlight(clf, partitioner, radius=slradius,
> space='voxel_indices', errorfx=None, postproc=mean_sample()) slght_map =
> slght(fds) In [70]: slght = sphere_gnbsearchlight(clf, partitioner,
> radius=slradius, space='voxel_indices', errorfx=None,
> postproc=mean_sample()) In [71]: slght_map = slght(fds) [SLC] DBG: Phase
> 1. Initializing partitions using <NFoldPartitioner> on <Dataset:
> 108x111 at float32, <sa: chunks,targets,time_coords,time_indices>, <fa:
> voxel_indices>, <a:
> imgaffine,imghdr,imgtype,mapper,voxel_dim,voxel_eldim>> [SLC] DBG: Phase
> 2. Blocking data for 18 splits and 3 labels [SLC] DBG: Phase 3. Computing
> statistics for 54 blocks [SLC] DBG: Phase 4. Deducing neighbors
> information for 111 ROIs [SLC] DBG: Phase 4b. Converting neighbors to
> sparse matrix representation [SLC] DBG: Phase 5. Major loop [SLC] DBG:
> Split 0 out of 18 [SLC] DBG: 'Training' is done [SLC] DBG: Doing
> 'Searchlight' [SLC] DBG: Assessing accuracies
> --------------------------------------------------------------------------------
> TypeError Traceback (most recent call last)
> <ipython-input-71-1146d298ca06> in <module>() ----> 1 slght_map =
> slght(fds) /usr/lib/python2.7/dist-packages/mvpa2/base/learner.pyc in
> __call__(self, ds) 257 "used and auto training is disabled." 258 %
> str(self)) --> 259 return super(Learner, self).__call__(ds) 260 261
> /usr/lib/python2.7/dist-packages/mvpa2/base/node.pyc in __call__(self, ds)
> 119 120 self._precall(ds) --> 121 result = self._call(ds) 122 result =
> self._postcall(ds, result) 123
> /usr/lib/python2.7/dist-packages/mvpa2/measures/searchlight.pyc in
> _call(self, dataset) 141 142 # pass to subclass --> 143 results =
> self._sl_call(dataset, roi_ids, nproc) 144 145 if 'mapper' in dataset.a:
> /usr/lib/python2.7/dist-packages/mvpa2/measures/adhocsearchlightbase.pyc
> in _sl_call(self, dataset, roi_ids, nproc) 513 # error functions without a
> chance to screw up 514 for i, fpredictions in enumerate(predictions.T):
> --> 515 results[isplit, i] = errorfx(fpredictions, targets) 516 517
> TypeError: 'NoneType' object is not callable Similarly for other cases and
> combinations of them: Case 2: slght = sphere_gnbsearchlight(clf,
> partitioner, radius=slradius, space='voxel_indices',
> errorfx=ConfusionMatrixError(), postproc=mean_sample()) slght_map =
> slght(fds) Case3: class KeepConfusionMatrix(Node): def _call(self, fds):
> out = np.zeros(1, dtype=object) out[0] = (fds.samples) return out slght =
> sphere_gnbsearchlight(clf, partitioner, errorfx=None, radius=slradius,
> space='voxel_indices', postproc=ChainNode([Confusion(labels=fds.UT)]))
> slght.postproc.append(KeepConfusionMatrix()) slght_map = slght(fds) Case4:
> class KeepConfusionMatrix(Node): def _call(self, fds): out = np.zeros(1,
> dtype=object) out[0] = (fds.samples) return out slght =
> sphere_gnbsearchlight(clf, partitioner, errorfx=None, radius=slradius,
> space='voxel_indices',
> postproc=ChainNode([mean_sample(),Confusion(labels=fds.UT)]))
> slght.postproc.append(KeepConfusionMatrix()) slght_map = slght(fds) Case5:
> class KeepConfusionMatrix(Node): def _call(self, fds): out = np.zeros(1,
> dtype=object) out[0] = (fds.samples) return out slght =
> sphere_gnbsearchlight(clf, partitioner, errorfx=ConfusionMatrixError(),
> radius=slradius, space='voxel_indices',
> postproc=ChainNode([mean_sample(),Confusion(labels=fds.UT)]))
> slght.postproc.append(KeepConfusionMatrix()) slght_map = slght(fds)
> Yaroslav Halchenko debian at onerussian.com <http://onerussian.com> Fri
> Aug 28 13:16:38 UTC 2015 quick an possible partial reply 1. "not sure" --
> if it pukes then probably not, although judging from the code I foresaw
> arbitrary shape of the errorfx output 2. but you could make
> sphere_gnbsearchlight to return labels (not errors) and then post-process
> to get those confusion matrices. Just specify errorfx=None to it (not to
> CV). But you could also try passing errorfx=ConfusionMatrixError and see
> how that goes Please share what you discover/end up with.
> mvpa2/tests/test_usecases.py <http://usecases.py> has more of usecase
> demos for gnb searchlights which might come handy -- Yaroslav O.
> Halchenko, Ph.D.http://neuro.debian.net http://www.pymvpa.org
> http://www.fail2ban.org Research Scientist, Psychological and Brain
> Sciences Dept. Dartmouth College, 419 Moore Hall, Hinman Box 6207,
> Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW:
> http://www.linkedin.com/in/yarik On 08/28/2015 01:48 PM, marco tettamanti
> wrote: Dear all, is it possible to obtain confusion matrices for all nodes
> with "sphere_gnbsearchlight", as was suggested before with
> "sphere_searchlight": slcvte = CrossValidation(clf, partitioner,
> errorfx=None, postproc=ChainNode([Confusion(labels=fds.UT)])) class
> KeepConfusionMatrix(Node): def _call(self, fds): out = np.zeros(1,
> dtype=object) out[0] = (fds.samples) return out
> slcvte.postproc.append(KeepConfusionMatrix()) slght =
> sphere_searchlight(slcvte, radius=slradius, space='voxel_indices',
> nproc=4, postproc=mean_sample()) slght_map = slght(fds) Thank you and best
> wishes, Marco -- Marco Tettamanti, Ph.D. Nuclear Medicine Department &
> Division of Neuroscience San Raffaele Scientific Institute Via Olgettina
> 58 I-20132 Milano, Italy Phone ++39-02-26434888 Fax ++39-02-26434892
> Email: tettamanti.marco at hsr.it Skype: mtettamanti
> --------------------------------------------------------------------------------
> Pkg-ExpPsy-PyMVPA mailing list Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
>
>
>
>
>
> --
> Basile Pinsard
>
> *PhD
> candidate, *
> Laboratoire d'Imagerie Biom?dicale, UMR S 1146 / UMR 7371, Sorbonne
> Universit?s, UPMC, INSERM, CNRS
> *Brain-Cognition-Behaviour Doctoral School **, *ED3C*, *UPMC, Sorbonne
> Universit?s
> Biomedical Sciences Doctoral School, Faculty of Medicine, Universit? de
> Montr?al
> CRIUGM, Universit? de Montr?al
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20150828/fc793f7b/attachment.html>
>
> --------------------------------------------------------------------------------
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20150829/35ad1069/attachment.html>
More information about the Pkg-ExpPsy-PyMVPA
mailing list