[pymvpa] Confusion Matrix for each Node with sphere_gnbsearchlight
marco tettamanti
mrctttmnt at gmail.com
Sat Aug 29 15:12:45 UTC 2015
Dear Yaroslav,
I have dowloaded master version 2.4.0 from git, and I can now get classification
labels with errorfx=None.
The problem is now that "errorfx= " within "sphere_gnbsearchlight" behaves less
flexibly:
errorfx=None
errorfx=mean_mismatch_error
are ok, but:
errorfx=mean_match_accuracy
errorfx=ConfusionMatrixError(labels=fds.UT))
give an error of type:
ValueError: Collectable 'cvfolds' with length [108] does not match the required length [18] of collection '<SampleAttributesCollection>'.
As used for example in:
slght = sphere_gnbsearchlight(slghtclf, partitioner, radius=slradius,
space='voxel_indices', errorfx=mean_match_accuracy, postproc=mean_sample())
slght_map = slght(fds)
Thank you and best wishes,
Marco
On 08/28/2015 08:23 PM, marco tettamanti wrote:
> Thanks again!
> I am on Debian testing (well, reverted on stable now, because of troubles with
> gcc5) and have version 2.3.1.
> I will give a try to the one from git.
> Best,
> Marco
>
>
> PyMVPA:
> Version: 2.3.1
> Hash: d1da5a749dc9cc606bd7f425d93d25464bf43454
> Path: /usr/lib/python2.7/dist-packages/mvpa2/__init__.pyc
> Version control (GIT):
> GIT information could not be obtained due
> "/usr/lib/python2.7/dist-packages/mvpa2/.. is not under GIT"
> SYSTEM:
> OS: posix Linux 4.1.0-1-amd64 #1 SMP Debian 4.1.3-1 (2015-08-03)
> Distribution: debian/stretch/sid
>
>
>> *Yaroslav Halchenko* debian at onerussian.com
>> <mailto:pkg-exppsy-pymvpa%40lists.alioth.debian.org?Subject=Re%3A%20%5Bpymvpa%5D%20Confusion%20Matrix%20for%20each%20Node%20with%0A%20sphere_gnbsearchlight&In-Reply-To=%3C20150828161509.GS19455%40onerussian.com%3E>
>> /Fri Aug 28 16:15:09 UTC 2015/
>> --------------------------------------------------------------------------------
>> On Fri, 28 Aug 2015, marco tettamanti wrote:
>>
>> >/ Dear Yaroslav,
>> />/ thank you very much for your reply. I have made several attempts, trying
>> />/ to guess a solution, but it seems I always get a
>> />/ 'TypeError: 'NoneType' object is not callable'.
>> /
>> oh shoot... forgotten that this one was implemented after the last 2.4.0
>> release: in upstream/2.4.0-34-g55e147e this June... we should release I
>> guess. what system are you on and what version of pymvpa currently?
>> if you could use/try the one from git directly... ?
>>
>> >/ Case 1:
>> />/ slght = sphere_gnbsearchlight(clf, partitioner, radius=slradius,
>> />/ space='voxel_indices', errorfx=None, postproc=mean_sample())
>> /
>> not the problem here BUT there should be no mean_sample() if errorfx is
>> None -- you wouldn't want to average labels ;)
>>
>> --
>> Yaroslav O. Halchenko, Ph.D.
>> http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
>> Research Scientist, Psychological and Brain Sciences Dept.
>> Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
>> Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
>> WWW:http://www.linkedin.com/in/yarik
>>
>
>
>
> On 08/28/2015 05:28 PM, marco tettamanti wrote:
>> Dear Yaroslav,
>> thank you very much for your reply. I have made several attempts, trying to
>> guess a solution, but it seems I always get a
>> 'TypeError: 'NoneType' object is not callable'.
>>
>> Any further advice is greatly appreciated!
>> Best,
>> Marco
>>
>>
>> Case 1:
>> slght = sphere_gnbsearchlight(clf, partitioner, radius=slradius,
>> space='voxel_indices', errorfx=None, postproc=mean_sample())
>> slght_map = slght(fds)
>>
>> In [70]: slght = sphere_gnbsearchlight(clf, partitioner, radius=slradius,
>> space='voxel_indices', errorfx=None, postproc=mean_sample())
>>
>> In [71]: slght_map = slght(fds)
>> [SLC] DBG: Phase 1. Initializing partitions using
>> <NFoldPartitioner> on <Dataset: 108x111 at float32, <sa:
>> chunks,targets,time_coords,time_indices>, <fa: voxel_indices>, <a:
>> imgaffine,imghdr,imgtype,mapper,voxel_dim,voxel_eldim>>
>> [SLC] DBG: Phase 2. Blocking data for 18 splits and 3 labels
>> [SLC] DBG: Phase 3. Computing statistics for 54 blocks
>> [SLC] DBG: Phase 4. Deducing neighbors information for 111 ROIs
>> [SLC] DBG: Phase 4b. Converting neighbors to sparse matrix
>> representation
>> [SLC] DBG: Phase 5. Major loop
>> [SLC] DBG: Split 0 out of 18
>> [SLC] DBG: 'Training' is done
>> [SLC] DBG: Doing 'Searchlight'
>> [SLC] DBG: Assessing accuracies
>> ---------------------------------------------------------------------------
>> TypeError Traceback (most recent call last)
>> <ipython-input-71-1146d298ca06> in <module>()
>> ----> 1 slght_map = slght(fds)
>>
>> /usr/lib/python2.7/dist-packages/mvpa2/base/learner.pyc in __call__(self, ds)
>> 257 "used and auto training is
>> disabled."
>> 258 % str(self))
>> --> 259 return super(Learner, self).__call__(ds)
>> 260
>> 261
>>
>> /usr/lib/python2.7/dist-packages/mvpa2/base/node.pyc in __call__(self, ds)
>> 119
>> 120 self._precall(ds)
>> --> 121 result = self._call(ds)
>> 122 result = self._postcall(ds, result)
>> 123
>>
>> /usr/lib/python2.7/dist-packages/mvpa2/measures/searchlight.pyc in
>> _call(self, dataset)
>> 141
>> 142 # pass to subclass
>> --> 143 results = self._sl_call(dataset, roi_ids, nproc)
>> 144
>> 145 if 'mapper' in dataset.a:
>>
>> /usr/lib/python2.7/dist-packages/mvpa2/measures/adhocsearchlightbase.pyc
>> in _sl_call(self, dataset, roi_ids, nproc)
>> 513 # error functions without a chance to screw up
>> 514 for i, fpredictions in enumerate(predictions.T):
>> --> 515 results[isplit, i] = errorfx(fpredictions,
>> targets)
>> 516
>> 517
>>
>> TypeError: 'NoneType' object is not callable
>>
>>
>>
>>
>> Similarly for other cases and combinations of them:
>>
>> Case 2:
>> slght = sphere_gnbsearchlight(clf, partitioner, radius=slradius,
>> space='voxel_indices', errorfx=ConfusionMatrixError(), postproc=mean_sample())
>> slght_map = slght(fds)
>>
>>
>> Case3:
>> class KeepConfusionMatrix(Node):
>> def _call(self, fds):
>> out = np.zeros(1, dtype=object)
>> out[0] = (fds.samples)
>> return out
>>
>> slght = sphere_gnbsearchlight(clf, partitioner, errorfx=None,
>> radius=slradius, space='voxel_indices',
>> postproc=ChainNode([Confusion(labels=fds.UT)]))
>> slght.postproc.append(KeepConfusionMatrix())
>> slght_map = slght(fds)
>>
>>
>> Case4:
>> class KeepConfusionMatrix(Node):
>> def _call(self, fds):
>> out = np.zeros(1, dtype=object)
>> out[0] = (fds.samples)
>> return out
>>
>> slght = sphere_gnbsearchlight(clf, partitioner, errorfx=None,
>> radius=slradius, space='voxel_indices',
>> postproc=ChainNode([mean_sample(),Confusion(labels=fds.UT)]))
>> slght.postproc.append(KeepConfusionMatrix())
>> slght_map = slght(fds)
>>
>>
>>
>> Case5:
>> class KeepConfusionMatrix(Node):
>> def _call(self, fds):
>> out = np.zeros(1, dtype=object)
>> out[0] = (fds.samples)
>> return out
>>
>> slght = sphere_gnbsearchlight(clf, partitioner,
>> errorfx=ConfusionMatrixError(), radius=slradius, space='voxel_indices',
>> postproc=ChainNode([mean_sample(),Confusion(labels=fds.UT)]))
>> slght.postproc.append(KeepConfusionMatrix())
>> slght_map = slght(fds)
>>
>>
>>
>>> Yaroslav Halchenko debian at onerussian.com
>>> Fri Aug 28 13:16:38 UTC 2015
>>> quick an possible partial reply
>>>
>>> 1. "not sure" -- if it pukes then probably not, although judging from
>>> the code I foresaw arbitrary shape of the errorfx output
>>>
>>> 2. but you could make sphere_gnbsearchlight to return labels (not
>>> errors) and then post-process to get those confusion matrices. Just
>>> specify errorfx=None to it (not to CV). But you could also try
>>> passing errorfx=ConfusionMatrixError and see how that goes
>>>
>>> Please share what you discover/end up with.
>>> mvpa2/tests/test_usecases.py has more of usecase demos for gnb
>>> searchlights which might come handy
>>>
>>> --
>>> Yaroslav O. Halchenko, Ph.D.
>>> http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
>>> Research Scientist, Psychological and Brain Sciences Dept.
>>> Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
>>> Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
>>> WWW:http://www.linkedin.com/in/yarik
>>>
>>
>> On 08/28/2015 01:48 PM, marco tettamanti wrote:
>>> Dear all,
>>> is it possible to obtain confusion matrices for all nodes with
>>> "sphere_gnbsearchlight", as was suggested before with "sphere_searchlight":
>>>
>>> slcvte = CrossValidation(clf, partitioner, errorfx=None,
>>> postproc=ChainNode([Confusion(labels=fds.UT)]))
>>> class KeepConfusionMatrix(Node):
>>> def _call(self, fds):
>>> out = np.zeros(1, dtype=object)
>>> out[0] = (fds.samples)
>>> return out
>>>
>>> slcvte.postproc.append(KeepConfusionMatrix())
>>> slght = sphere_searchlight(slcvte, radius=slradius, space='voxel_indices',
>>> nproc=4, postproc=mean_sample())
>>> slght_map = slght(fds)
>>>
>>>
>>> Thank you and best wishes,
>>> Marco
>>> --
>>> Marco Tettamanti, Ph.D.
>>> Nuclear Medicine Department & Division of Neuroscience
>>> San Raffaele Scientific Institute
>>> Via Olgettina 58
>>> I-20132 Milano, Italy
>>> Phone ++39-02-26434888
>>> Fax ++39-02-26434892
>>> Email:tettamanti.marco at hsr.it
>>> Skype: mtettamanti
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20150829/82bb5deb/attachment.html>
More information about the Pkg-ExpPsy-PyMVPA
mailing list