[pymvpa] Searchlight Memory Error

[Jacob Elder]

Hi,

I am trying to run a whole-brain searchlight RSA (on a cluster with a lot
of processing cores available) but every time I do it returns a
memoryError. However, when I split it into hemispheres or run as an ROI, it
successfully runs without MemoryError. I have tried following some other
suggestions I found online but nothing has resolved the issue. Memory
shouldn't be an issue as the cluster has plenty of RAM available. We've
also tried to run this on another cluster at a separate institution using a
high mem option and it still didn't work. I am pretty stumped and was
wondering if anyone can help or has any potential solutions in mind?

Here is what the code looks like:

    subject = subjectData(inputs.rootDir, inputs.ID)


    *# load the trait similarity data*

    traitData = subject.importSimilarity()


    *# load the neural data*

    neuralData = subject.importNeural()



    neuralData = remove_invariant_features(neuralData)


    *print* ("Complete loading neural data")


    *# create an object from the neuroCorrelation class*

    matrixCorr = neuroCorrelation(traitData)


    *# set up searchlight parameters*

    *# set the radius based on the input*

    rad = inputs.rad


    *# Fisher's z-transformation for correlation coefficient*

    FisherTransform = FxMapper('features', *lambda* r: 0.5 * np.log((1 + r)
/ (1 - r)))


    *# create the search light*

    sl = sphere_searchlight(matrixCorr.correlate, rad, results_backend=
'hdf5', postproc=FisherTransform)


    *print* ("Complete search light set-up")


    *# apply the search light function to the neural data*

    sl_output = sl(neuralData)


    *print* ("Search light completed")


    *# transform the search light output to an image data*

    sl_image = map2nifti(data=sl_output, dataset=neuralData)


    *# save the output*

    outputDir = os.path.join(inputs.rootDir, "RSA", "searchLightResult",
"Subject{n}".format(n=inputs.ID),


"sub{n}_sl_results_{v}{h}_P_R{r}.nii.gz".format(n=inputs.ID,
v=inputs.valence, h=inputs.hemisphere, r=inputs.rad))

    sl_image.to_filename(outputDir)


    *print* ("Complete with subject{n}".format(n=inputs.ID))


Here is the error:

/home/user/anaconda2/lib/python2.7/site-packages/numpy/lib/type_check.py:546:
DeprecationWarning: np.asscalar(a) is deprecated since NumPy v1.16, use
a.item() instead

  'a.item() instead', DeprecationWarning, stacklevel=1)

Traceback (most recent call last):

  File "./script/RSA_rad_FB_test.py", line 54, in <module>

    sl_output = sl(neuralData)

  File
"/home/user/anaconda2/lib/python2.7/site-packages/mvpa2/base/learner.py",
line 258, in __call__

    return super(Learner, self).__call__(ds)

  File
"/home/user/anaconda2/lib/python2.7/site-packages/mvpa2/base/node.py", line
138, in __call__

    result = self._postcall(ds, result)

  File
"/home/user/anaconda2/lib/python2.7/site-packages/mvpa2/measures/base.py",
line 128, in _postcall

    result = super(Measure, self)._postcall(dataset, result)

  File
"/home/user/anaconda2/lib/python2.7/site-packages/mvpa2/base/node.py", line
179, in _postcall

    result = self._apply_postproc(ds, result)

  File
"/home/user/anaconda2/lib/python2.7/site-packages/mvpa2/base/node.py", line
253, in _apply_postproc

    result = self.__postproc(result)

  File
"/home/user/anaconda2/lib/python2.7/site-packages/mvpa2/base/learner.py",
line 258, in __call__

    return super(Learner, self).__call__(ds)

  File
"/home/user/anaconda2/lib/python2.7/site-packages/mvpa2/base/node.py", line
137, in __call__

    result = self._call(ds, **(_call_kwargs or self._get_call_kwargs(ds)))

  File
"/home/user/anaconda2/lib/python2.7/site-packages/mvpa2/mappers/base.py",
line 291, in _call

    return self.forward(ds)

  File
"/home/user/anaconda2/lib/python2.7/site-packages/mvpa2/mappers/base.py",
line 215, in forward

    return self._forward_dataset(data)

  File
"/home/user/anaconda2/lib/python2.7/site-packages/mvpa2/mappers/fx.py",
line 204, in _forward_dataset

    col[attr] = a

  File
"/home/user/anaconda2/lib/python2.7/site-packages/mvpa2/base/collections.py",
line 590, in __setitem__

    value = ArrayCollectable(value)

  File
"/home/user/anaconda2/lib/python2.7/site-packages/mvpa2/base/collections.py",
line 185, in __init__

    Collectable.__init__(self, value=value, name=name, doc=doc)

  File
"/home/user/anaconda2/lib/python2.7/site-packages/mvpa2/base/collections.py",
line 67, in __init__

    self._set(value)

  File
"/home/user/anaconda2/lib/python2.7/site-packages/mvpa2/base/collections.py",
line 291, in _set

    val = np.asanyarray(val)

  File
"/home/user/anaconda2/lib/python2.7/site-packages/numpy/core/numeric.py",
line 591, in asanyarray

    return array(a, dtype, copy=False, order=order, subok=True)

MemoryError


Any help would be greatly appreciated! Thank you.

-- 
Jacob Elder
Ph.D. Student, Dept. of Psychology
University of California, Riverside
The UCR Social Neuroscience Lab <https://www.hugheslab.org/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/pkg-exppsy-pymvpa/attachments/20200514/1d029540/attachment-0001.html>