<html><head></head><body><div class="gmail_quote">On April 8, 2017 6:08:50 PM EDT, Alex Murphy <murphyalex@gmail.com> wrote:<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div dir="ltr">Hi all,<div><br /></div><div>I am currently using Anaconda 3.6 with pyMVPA and everything has been fine until I came to performing some computationally demanding full-brain searchlight analyses. I am trying to get this to work on the institution's cluster by qsub'ing a file that generates the event-related dataset from the NIFTY files and then performs a searchlight analysis. </div><div><br /></div><div>I am running into an annoying issue and I hope someone has already figured out a way to solve it. Basically, to have any chance of being able to solve this without going over the 12GB RAM limit imposed by the servers (which automatically kill any processes that exceed that amount) I need to use the multiple threads / cores and for that, I need the pprocess module. The version I downloaded is 0.5.1 which is apparently the latest, but the first issue I had is that it is written in python 2.7. So, I ran 2to3 on it and that seemed to solve one issue but then I had amother problem in socket.py</div><div><br /></div><div>The error that I get is:</div><div><br /></div><div> File "/users/alex/anaconda3/lib/python3.6/socket.py", line 245, in makefile<br /> raise ValueError("unbuffered streams must be binary")<br />ValueError: unbuffered streams must be binary<br /></div><div><br /></div><div>The problem seems to be that "pprocess" (which sphere_searchlight relies on) is written in Python 2.7 and incompatible with Python 3.X. I know it is possible to make a new virtual environment and recreate everything in 2.7 and then I *hope* that would solve the problem (but I am not sure and would like to see if I can get it to work with Python 3.X first). </div><div><br /></div><div>Basically, I cannot make use of the server's multi-threading procedures to allow me to do the searchlight analysis efficiently because of this. I do plan on using the GNB searchlight solution at some point but I would like to not be wholly restricted to using it because the server cannot work with "pprocess".</div><div><br /></div><div>Has anyone run into a similar problem and knows how to figure it out?</div><div><br /></div><div>** Summary of question **</div><div><br /></div><div>I run into errors using Python 3.X when doing searchlight analyses on my institution's servers because using only a single core (by default if pprocess isn't present) means I run over the RAM limit. The servers can handle really intensive workloads via parallel processing but I can't get it to work because "pprocess" gives me a few errors. The first I solved by convering pprocess.py to Python 3.X but there is an issue coming from socket.py about unbuffered streams must be binary. Is this a problem anyone has had or might know a solution to?</div><div><br /></div><div>Kind regards</div><div>Alex M</div></div>
</blockquote></div><br clear="all">Fwiw<br>
Indeed there might be issues with pprocess and python 3... We will need to look into it. Must work correctly with python 2, so if you could, please just install that version of anaconda<br>
<br>
gnbsearchlight isn't using it iirc and will be fast regardless<br>
-- <br>
Sent from a phone which beats iPhone.</body></html>