[pymvpa] Searchlight hyperalignment crashes: "featuresBus error"
Michael Bannert
mbannert at tuebingen.mpg.de
Wed Oct 11 09:58:32 UTC 2017
Dear Yaroslav,
Thanks and sorry for my slow response. Our computing cluster underwent
major maintenance recently and I then wanted to do some testing to
provide some more detailed information.
I haven't encountered the error again. I'm usually testing my
hyperalignment-searchlight analysis only on a subset of subjects and
only on few voxel ("roi_ids" and "mask_node_ids" arguments,
respectively), which yielded no problems. This error occurred only when
running the analysis on all subjects and all voxels.
Is it hence possible that the "Bus error" occurred because of memory
limitations?
I am running hyperalignment searchlights in parallel on a cluster. I am
using the in-built parallelization via the "nproc" arguments. I think
the problem with using nproc=1 but accordingly parallelizing across more
numerious nodes is that each process will load the whole brain data
(taking up lots of memory) and then process only its small set of voxels
as specified in "mask_node_ids". This redundancy probably crashed my
jobs on the clusters. The error I mentioned in my previous email
occurred when test-running the code on a cluster node, which may have
happened to be too busy to load all whole-brain datasets (and which is
only meant for "light-weight" computing).
I now use fewer cluster nodes (5-10) with the maximum number of
processors (24) and let PyMVPA take care of the parallelization within
the specified subset of voxels. This seems to work.
So to cut a long story short and to answer your question, Yaroslav: I
don't think that choosing "nproc > 1" was the issue here. More likely
memory (?). It could also have something to do with the maintenance work
they have been undertaking at the cluster.
Thanks for your help!
Best,
Michael
On 05/10/17 15:40, Yaroslav Halchenko wrote:
> oh
>
> could you try please with nproc=1 (for the searchlight) and see if that
> avoids it (although would become slower)
>
> On Thu, 05 Oct 2017, Michael Bannert wrote:
>
>> Dear Swaroop, dear PyMVPA users,
>
>> I'm running a couple of searchlight hyperalignments but some of them crash.
>> The error message is "featuresBus error". What does this mean? What could be
>> causing this error?
>
>> Thanks and best,
>> Michael
>
>> PS: Here is the output of wtf()
>> Current date: 2017-10-05 13:35
>> PyMVPA:
>> Version: 2.6.0
>
> just FYI, there is 2.6.1 already, so if you can (e.g. not in the middle of
> the study), update
>
More information about the Pkg-ExpPsy-PyMVPA
mailing list