[pymvpa] Confusion Matrix for each Node with sphere_gnbsearchlight

marco tettamanti mrctttmnt at gmail.com
Fri Aug 28 15:28:55 UTC 2015


**Dear Yaroslav,
thank you very much for your reply. I have made several attempts, trying to 
guess a solution, but it seems I always get a
'TypeError: 'NoneType' object is not callable'.

Any further advice is greatly appreciated!
Best,
Marco


Case 1:
slght = sphere_gnbsearchlight(clf, partitioner, radius=slradius, 
space='voxel_indices', errorfx=None, postproc=mean_sample())
slght_map = slght(fds)

    In [70]: slght = sphere_gnbsearchlight(clf, partitioner, radius=slradius,
    space='voxel_indices', errorfx=None, postproc=mean_sample())

    In [71]: slght_map = slght(fds)
    [SLC] DBG:             Phase 1. Initializing partitions using
    <NFoldPartitioner> on <Dataset: 108x111 at float32, <sa:
    chunks,targets,time_coords,time_indices>, <fa: voxel_indices>, <a:
    imgaffine,imghdr,imgtype,mapper,voxel_dim,voxel_eldim>>
    [SLC] DBG:             Phase 2. Blocking data for 18 splits and 3 labels
    [SLC] DBG:             Phase 3. Computing statistics for 54 blocks
    [SLC] DBG:             Phase 4. Deducing neighbors information for 111 ROIs
    [SLC] DBG:             Phase 4b. Converting neighbors to sparse matrix
    representation
    [SLC] DBG:             Phase 5. Major loop
    [SLC] DBG:              Split 0 out of 18
    [SLC] DBG:                'Training' is done
    [SLC] DBG:                Doing 'Searchlight'
    [SLC] DBG:               Assessing accuracies
    ---------------------------------------------------------------------------
    TypeError                                 Traceback (most recent call last)
    <ipython-input-71-1146d298ca06> in <module>()
    ----> 1 slght_map = slght(fds)

    /usr/lib/python2.7/dist-packages/mvpa2/base/learner.pyc in __call__(self, ds)
         257                                    "used and auto training is
    disabled."
         258                                    % str(self))
    --> 259         return super(Learner, self).__call__(ds)
         260
         261

    /usr/lib/python2.7/dist-packages/mvpa2/base/node.pyc in __call__(self, ds)
         119
         120         self._precall(ds)
    --> 121         result = self._call(ds)
         122         result = self._postcall(ds, result)
         123

    /usr/lib/python2.7/dist-packages/mvpa2/measures/searchlight.pyc in
    _call(self, dataset)
         141
         142         # pass to subclass
    --> 143         results = self._sl_call(dataset, roi_ids, nproc)
         144
         145         if 'mapper' in dataset.a:

    /usr/lib/python2.7/dist-packages/mvpa2/measures/adhocsearchlightbase.pyc in
    _sl_call(self, dataset, roi_ids, nproc)
         513                 # error functions without a chance to screw up
         514                 for i, fpredictions in enumerate(predictions.T):
    --> 515                     results[isplit, i] = errorfx(fpredictions, targets)
         516
         517

    TypeError: 'NoneType' object is not callable




Similarly for other cases and combinations of them:

Case 2:
slght = sphere_gnbsearchlight(clf, partitioner, radius=slradius, 
space='voxel_indices', errorfx=ConfusionMatrixError(), postproc=mean_sample())
slght_map = slght(fds)


Case3:
class KeepConfusionMatrix(Node):
       def _call(self, fds):
           out = np.zeros(1, dtype=object)
           out[0] = (fds.samples)
           return out

slght = sphere_gnbsearchlight(clf, partitioner, errorfx=None, radius=slradius, 
space='voxel_indices', postproc=ChainNode([Confusion(labels=fds.UT)]))
slght.postproc.append(KeepConfusionMatrix())
slght_map = slght(fds)


Case4:
class KeepConfusionMatrix(Node):
       def _call(self, fds):
           out = np.zeros(1, dtype=object)
           out[0] = (fds.samples)
           return out

slght = sphere_gnbsearchlight(clf, partitioner, errorfx=None, radius=slradius, 
space='voxel_indices', postproc=ChainNode([mean_sample(),Confusion(labels=fds.UT)]))
slght.postproc.append(KeepConfusionMatrix())
slght_map = slght(fds)



Case5:
class KeepConfusionMatrix(Node):
       def _call(self, fds):
           out = np.zeros(1, dtype=object)
           out[0] = (fds.samples)
           return out

slght = sphere_gnbsearchlight(clf, partitioner, errorfx=ConfusionMatrixError(), 
radius=slradius, space='voxel_indices', 
postproc=ChainNode([mean_sample(),Confusion(labels=fds.UT)]))
slght.postproc.append(KeepConfusionMatrix())
slght_map = slght(fds)



> Yaroslav Halchenko debian at onerussian.com
> Fri Aug 28 13:16:38 UTC 2015
> quick an possible partial reply
>
> 1. "not sure" -- if it pukes then probably not, although judging from
> the code I foresaw arbitrary shape of the errorfx output
>
> 2. but you could make sphere_gnbsearchlight to return labels (not
> errors) and then post-process to get those confusion matrices.  Just
> specify  errorfx=None  to it (not to CV).  But you could also try
> passing errorfx=ConfusionMatrixError and see how that goes
>
> Please share what you discover/end up with.
> mvpa2/tests/test_usecases.py  has more of usecase demos for gnb
> searchlights which might come handy
>
> -- 
> Yaroslav O. Halchenko, Ph.D.
> http://neuro.debian.net  http://www.pymvpa.org  http://www.fail2ban.org
> Research Scientist,            Psychological and Brain Sciences Dept.
> Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
> Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
> WWW:http://www.linkedin.com/in/yarik         
>

On 08/28/2015 01:48 PM, marco tettamanti wrote:
> Dear all,
> is it possible to obtain confusion matrices for all nodes with 
> "sphere_gnbsearchlight", as was suggested before with "sphere_searchlight":
>
> slcvte = CrossValidation(clf, partitioner, errorfx=None, 
> postproc=ChainNode([Confusion(labels=fds.UT)]))
> class KeepConfusionMatrix(Node):
>      def _call(self, fds):
>          out = np.zeros(1, dtype=object)
>          out[0] = (fds.samples)
>          return out
>
> slcvte.postproc.append(KeepConfusionMatrix())
> slght = sphere_searchlight(slcvte, radius=slradius, space='voxel_indices', 
> nproc=4, postproc=mean_sample())
> slght_map = slght(fds)
>
>
> Thank you and best wishes,
> Marco
> -- 
> Marco Tettamanti, Ph.D.
> Nuclear Medicine Department & Division of Neuroscience
> San Raffaele Scientific Institute
> Via Olgettina 58
> I-20132 Milano, Italy
> Phone ++39-02-26434888
> Fax ++39-02-26434892
> Email:tettamanti.marco at hsr.it
> Skype: mtettamanti

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20150828/8297787a/attachment.html>


More information about the Pkg-ExpPsy-PyMVPA mailing list