<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Dear Basile,<br>
thank you for sharing, I will try this out!<br>
Best,<br>
Marco<br>
<br>
<div class="moz-cite-prefix">On 08/29/2015 09:26 AM, Marco
Tettamanti wrote:<br>
</div>
<blockquote
cite="mid:4D7DD337-3FD9-4E9F-81D0-2EA3F00BCC7A@gmail.com"
type="cite">
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
Date: Fri, 28 Aug 2015 14:40:47 -0400<br>
<pre class="k9mail">From: basile pinsard <a class="moz-txt-link-rfc2396E" href="mailto:basile.pinsard@gmail.com"><basile.pinsard@gmail.com></a>
To: Development and support of PyMVPA <a class="moz-txt-link-rfc2396E" href="mailto:pkg-exppsy-pymvpa@lists.alioth.debian.org"><pkg-exppsy-pymvpa@lists.alioth.debian.org></a>
Subject: Re: [pymvpa] Confusion Matrix for each Node with sphere_gnbsearchlight
I wanted to do the same and had to make some changes to PyMVPA here:
<a moz-do-not-send="true" href="https://github.com/bpinsard/PyMVPA/tree/gnbsearchlight_confusiontable">https://github.com/bpinsard/PyMVPA/tree/gnbsearchlight_confusiontable</a>
using it with:
errorfx = ConfusionMatrix(labels=ds.uniquetargets)
slght = GNBSearchlight(clf, prtnr, qe, errorfx=errorfx)
On Fri, Aug 28, 2015 at 2:23 PM, marco tettamanti <a class="moz-txt-link-rfc2396E" href="mailto:mrctttmnt@gmail.com"><mrctttmnt@gmail.com></a>
wrote:
<blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;"> Thanks again!
I
am on Debian testing (well, reverted on stable now, because of troubles
with gcc5) and have version 2.3.1.
I will give a try to the one from git.
Best,
Marco
PyMVPA:
Version: 2.3.1
Hash: d1da5a749dc9cc606bd7f425d93d25464bf43454
Path: /usr/lib/python2.7/dist-packages/mvpa2/__init__.pyc
Version control (GIT):
GIT information could not be obtained due
"/usr/lib/python2.7/dist-packages/mvpa2/.. is not under GIT"
SYSTEM:
OS: posix Linux 4.1.0-1-amd64 #1 SMP Debian 4.1.3-1
(2015-08-03)
Distribution: debian/stretch/sid
*Yaroslav Halchenko* debian at <a moz-do-not-send="true" href="http://onerussian.com">onerussian.com</a>
<pkg-exppsy-pymvpa%<a moz-do-not-send="true" href="http://40lists.alioth.debian.org">40lists.alioth.debian.org</a>?Subject=Re%3A%20%5Bpymvpa%5D%20Confusion%20Matrix%20for%20each%20Node%20with%0A%20sphere_gnbsearchlight&In-Reply-To=%3C20150828161509.GS19455%<a moz-do-not-send="true" href="http://40onerussian.com">40onerussian.com</a>%3E>
*Fri Aug 28 16:15:09 UTC 2015*
<hr>
On Fri, 28 Aug 2015, marco tettamanti wrote:
<blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #ad7fa8; padding-left: 1ex;">* Dear Yaroslav,
</blockquote> *>* thank you very much for your reply. I have made several attempts, trying
*>* to guess a solution, but it seems I always get a
*>* 'TypeError: 'NoneType' object is not callable'.
*
oh shoot... forgotten that this one was implemented after the last 2.4.0
release: in upstream/2.4.0-34-g55e147e this June... we should release I
guess. what system are you on and what version of pymvpa
currently?
if you could use/try the one from git directly... ?
<blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #ad7fa8; padding-left: 1ex;">* Case 1:
</blockquote> *>* slght = sphere_gnbsearchlight(clf, partitioner, radius=slradius,
*>* space='voxel_indices', errorfx=None, postproc=mean_sample())
*
not the problem here BUT there should be no mean_sample() if errorfx is
None -- you wouldn't want to average labels ;)
--
Yaroslav O. Halchenko, Ph.D.<a moz-do-not-send="true" href="http://neuro.debian.net">http://neuro.debian.net</a> <a moz-do-not-send="true" href="http://www.pymvpa.org">http://www.pymvpa.org</a> <a moz-do-not-send="true" href="http://www.fail2ban.org">http://www.fail2ban.org</a>
Research Scientist, Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
WWW: <a moz-do-not-send="true" href="http://www.linkedin.com/in/yarik">http://www.linkedin.com/in/yarik</a>
On 08/28/2015 05:28 PM, marco tettamanti wrote:
Dear Yaroslav,
thank you very much for your reply. I have made several attempts, trying
to guess a solution, but it seems I always get a
'TypeError: 'NoneType' object is not callable'.
Any further advice is greatly appreciated!
Best,
Marco
Case 1:
slght = sphere_gnbsearchlight(clf, partitioner, radius=slradius,
space='voxel_indices', errorfx=None, postproc=mean_sample())
slght_map = slght(fds)
In [70]: slght = sphere_gnbsearchlight(clf, partitioner, radius=slradius,
space='voxel_indices', errorfx=None, postproc=mean_sample())
In [71]: slght_map = slght(fds)
[SLC] DBG: Phase 1. Initializing partitions using
<NFoldPartitioner> on <Dataset: 108x111@float32, <sa:
chunks,targets,time_coords,time_indices>, <fa: voxel_indices>, <a:
imgaffine,imghdr,imgtype,mapper,voxel_dim,voxel_eldim>>
[SLC] DBG: Phase 2. Blocking data for 18 splits and 3 labels
[SLC] DBG: Phase 3. Computing statistics for 54 blocks
[SLC] DBG: Phase 4. Deducing neighbors information for 111 ROIs
[SLC] DBG: Phase 4b. Converting neighbors to sparse matrix
representation
[SLC] DBG: Phase 5. Major loop
[SLC] DBG: Split 0 out of 18
[SLC] DBG: 'Training' is done
[SLC] DBG: Doing 'Searchlight'
[SLC] DBG: Assessing accuracies
<hr>
TypeError Traceback (most recent call last)
<ipython-input-71-1146d298ca06> in <module>()
----> 1 slght_map = slght(fds)
/usr/lib/python2.7/dist-packages/mvpa2/base/learner.pyc
in __call__(self,
ds)
257 "used and auto training is
disabled."
258 % str(self))
--> 259 return super(Learner, self).__call__(ds)
260
261
/usr/lib/python2.7/dist-packages/mvpa2/base/node.pyc in __call__(self, ds)
119
120 self._precall(ds)
--> 121 result = self._call(ds)
122 result = self._postcall(ds, result)
123
/usr/lib/python2.7/dist-packages/mvpa2/measures/searchlight.pyc in
_call(self, dataset)
141
142 # pass to subclass
--> 143 results = self._sl_call(dataset, roi_ids, nproc)
144
145 if 'mapper' in dataset.a:
/usr/lib/python2.7/dist-packages/mvpa2/measures/adhocsearchlightbase.pyc
in _sl_call(self, dataset, roi_ids, nproc)
513
# error functions without a chance to screw up
514 for i, fpredictions in enumerate(predictions.T):
--> 515 results[isplit, i] = errorfx(fpredictions,
targets)
516
517
TypeError: 'NoneType' object is not callable
Similarly for other cases and combinations of them:
Case 2:
slght = sphere_gnbsearchlight(clf, partitioner, radius=slradius,
space='voxel_indices', errorfx=ConfusionMatrixError(),
postproc=mean_sample())
slght_map = slght(fds)
Case3:
class KeepConfusionMatrix(Node):
def _call(self, fds):
out = np.zeros(1, dtype=object)
out[0] = (fds.samples)
return out
slght = sphere_gnbsearchlight(clf, partitioner, errorfx=None,
radius=slradius, space='voxel_indices',
postproc=ChainNode([Confusion(labels=fds.UT)]))
slght.postproc.append(KeepConfusionMatrix())
slght_map = slght(fds)
Case4:
class KeepConfusionMatrix(Node):
def _call(self, fds):
out = np.zeros(1, dtype=object)
out[0] = (fds.samples)
return out
slght = sphere_gnbsearchlight(clf, partitioner, errorfx=None,
radius=slradius, space='voxel_indices',
postproc=ChainNode([mean_sample(),Confusion(labels=fds.UT)]))
slght.postproc.append(KeepConfusionMatrix())
slght_map = slght(fds)
Case5:
class KeepConfusionMatrix(Node):
def _call(self, fds):
out = np.zeros(1, dtype=object)
out[0] = (fds.samples)
return out
slght = sphere_gnbsearchlight(clf, partitioner,
errorfx=ConfusionMatrixError(), radius=slradius, space='voxel_indices',
postproc=ChainNode([mean_sample(),Confusion(labels=fds.UT)]))
slght.postproc.append(KeepConfusionMatrix())
slght_map = slght(fds)
Yaroslav Halchenko debian at <a moz-do-not-send="true" href="http://onerussian.com">onerussian.com</a>
Fri Aug 28 13:16:38 UTC 2015
quick an possible partial reply
1. "not sure" -- if it pukes then probably not, although judging from
the code I foresaw arbitrary shape of the errorfx output
2. but you could make sphere_gnbsearchlight to return labels (not
errors) and then post-process to get those confusion matrices. Just
specify errorfx=None to it (not to CV). But you could also try
passing errorfx=ConfusionMatrixError and see how that goes
Please share what you discover/end up with.
mvpa2/tests/test_<a moz-do-not-send="true" href="http://usecases.py">usecases.py</a> has more of usecase demos for gnb
searchlights which might come handy
--
Yaroslav O. Halchenko, Ph.D.<a moz-do-not-send="true" href="http://neuro.debian.net">http://neuro.debian.net</a> <a moz-do-not-send="true" href="http://www.pymvpa.org">http://www.pymvpa.org</a> <a moz-do-not-send="true" href="http://www.fail2ban.org">http://www.fail2ban.org</a>
Research Scientist, Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
WWW: <a moz-do-not-send="true" href="http://www.linkedin.com/in/yarik">http://www.linkedin.com/in/yarik</a>
On 08/28/2015 01:48 PM, marco tettamanti wrote:
Dear all,
is it possible to obtain confusion matrices for all nodes with
"sphere_gnbsearchlight", as was suggested before with "sphere_searchlight":
slcvte = CrossValidation(clf, partitioner, errorfx=None,
postproc=ChainNode([Confusion(labels=fds.UT)]))
class KeepConfusionMatrix(Node):
def _call(self, fds):
out = np.zeros(1, dtype=object)
out[0] = (fds.samples)
return out
slcvte.postproc.append(KeepConfusionMatrix())
slght = sphere_searchlight(slcvte, radius=slradius, space='voxel_indices',
nproc=4, postproc=mean_sample())
slght_map = slght(fds)
Thank you and best wishes,
Marco
--
Marco Tettamanti, Ph.D.
Nuclear Medicine Department & Division of Neuroscience
San Raffaele Scientific Institute
Via Olgettina 58
I-20132 Milano, Italy
Phone ++39-02-26434888
Fax ++39-02-26434892
Email: <a class="moz-txt-link-abbreviated" href="mailto:tettamanti.marco@hsr.it">tettamanti.marco@hsr.it</a>
Skype: mtettamanti
<hr>
Pkg-ExpPsy-PyMVPA mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Pkg-ExpPsy-PyMVPA@lists.alioth.debian.org">Pkg-ExpPsy-PyMVPA@lists.alioth.debian.org</a>
<a moz-do-not-send="true" href="http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa">http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa</a></blockquote>
--
Basile Pinsard
*PhD
candidate, *
Laboratoire d'Imagerie Biom?dicale, UMR S 1146 / UMR 7371, Sorbonne
Universit?s, UPMC, INSERM, CNRS
*Brain-Cognition-Behaviour Doctoral School **, *ED3C*, *UPMC, Sorbonne
Universit?s
Biomedical Sciences Doctoral School, Faculty of Medicine, Universit? de
Montr?al
CRIUGM, Universit? de Montr?al
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <<a moz-do-not-send="true" href="http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20150828/fc793f7b/attachment.html">http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20150828/fc793f7b/attachment.html</a>>
<hr>
</pre>
</blockquote>
<br>
</body>
</html>