[pymvpa] RAM use

[Yaroslav Halchenko]

On Fri, 13 May 2016, Alyson Saenz wrote:

>    Hey guys, I'm getting in some problems for the RAM use. I don't have
>    access to fancy servers and I only have my main desktop PC to run the
>    analysis. I'm trying to run a searchlight just like in the tutorial but
>    with my own data. The thing is that I have about 30 runs of 4 minutes
>    each. So when I'm loading the data into the dataset, my PC gives up and
>    gives me "Memory Error..."
>    My guess is that the script loads everything in to the RAM, and as I only
>    have 8gb, it gets full. Is there a way to send the data directly to the
>    hard drive?

yes, it would need load data into RAM to be processed... depending at
which point it runs out of RAM, remedies could differ...

you could try running your script with

MVPA_DEBUG_METRICS=vmem,pid MVPA_DEBUG=.*

env variables to see debug output annotated with consumed at that point
memory -- it could point at which step you have consumed most of it

>    I will also try to get my hands on more RAM or use a flash drive, but I
>    don't think its going to be enough.

yeah -- you might want to add/increase amount of swap... but once again it all
depends on what exactly you are doing and at which step you run out of ram and
how much

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience     http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik