[pymvpa] outputting p-values from searchlight

[Nick Oosterhof]

> On 19 May 2017, at 16:24, Nick Oosterhof <n.n.oosterhof at googlemail.com> wrote:
> 
>> 
>> On 19 May 2017, at 16:16, William Graves <william.wyatt.graves at gmail.com> wrote:
>> 
>> Hi Nick,
>> 
>> Thanks for the quick reply!
>> 
>> The problem is, when I run the searchlight without setting “corrcoef_only” to “True”, I get this error (here are the last few lines of the output, let me know if it doesn’t show up properly):
>> 
>> [SLC] DBG:              +0:00:57 ======[100%]====== 0:00:00  ROI 1465 (1465/1465), 36 features
>> 
>> [SLC] DBG:               hstacking 1465 results of shape (1, 2)
>> [SLC] DBG:               hstacked shape (1, 2930)
>> ---------------------------------------------------------------------------
>> ValueError                                Traceback (most recent call last)
>> <ipython-input-185-a11e1def05a0> in <module>()
>> ----> 1 slres = sl(ds)
>> 
>> /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/mvpa2/base/learner.pyc in __call__(self, ds)
>>    256                                    "used and auto training is disabled."
>>    257                                    % str(self))
>> --> 258         return super(Learner, self).__call__(ds)
>>    259 
>>    260     is_trained = property(fget=lambda x: x.__is_trained, fset=_set_trained,
>> 
>> /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/mvpa2/base/node.pyc in __call__(self, ds, _call_kwargs)
>>    135 
>>    136         self._precall(ds)
>> --> 137         result = self._call(ds, **(_call_kwargs or self._get_call_kwargs(ds)))
>>    138         result = self._postcall(ds, result)
>>    139 
>> 
>> /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/mvpa2/measures/searchlight.pyc in _call(self, dataset)
>>    152 
>>    153         # pass to subclass
>> --> 154         results = self._sl_call(dataset, roi_ids, nproc)
>>    155 
>>    156         if 'mapper' in dataset.a:
>> 
>> /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/mvpa2/measures/searchlight.pyc in _sl_call(self, dataset, roi_ids, nproc)
>>    382                                     dataset=dataset,
>>    383                                     roi_ids=roi_ids,
>> --> 384                                     results=self.__handle_all_results(p_results))
>>    385 
>>    386         # Assure having a dataset (for paranoid ones)
>> 
>> /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/mvpa2/measures/searchlight.pyc in _concat_results(sl, dataset, roi_ids, results)
>>    262 
>>    263         # store the center ids as a feature attribute
>> --> 264         result_ds.fa['center_ids'] = roi_ids
>>    265 
>>    266         return result_ds
>> 
>> /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/mvpa2/base/collections.pyc in __setitem__(self, key, value)
>>    597                                 len(value.value),
>>    598                                 ulength,
>> --> 599                                 str(self)))
>>    600         # tell the attribute to maintain the desired length
>>    601         value.set_length_check(ulength)
>> 
>> ValueError: Collectable 'center_ids' with length [1465] does not match the required length [2930] of collection '<FeatureAttributesCollection: metrics>'.
>> 
>> Presumably there’s a clever way to get around this that I just haven’t figured out.
> 
> This could actually be a bug in mvpa2/measures/rsa.py, the line that causes trouble (git blame):
> 
> d3885f6c (Michael Hanke         2016-01-22 15:18:23 +0100 293)             return Dataset([[rho, p]], fa={'metrics': ['rho', 'p']})
> 
> where the dataset returned is of size 1x2 (1 sample, 2 features). To me it would make more sense to return a 2x1 dataset (2 samples, 1 feature) so that datasets can be stacked for the searchlight output.
> 
> @Michael Hanke, would you agree? 

So it's seems it's not a bug but intended (and tested!)  behaviour; I just found this in mvpa2/tests/test_rsa.py, line 225-226:

   # now with both but we need to transpose datasets
    tdcm1_both = PDistTargetSimilarity(tdsm, postproc=TransposeMapper())

This approach with the postproc works together with a searchlight.