[Neurodebian-users] FSL and Condor

[Bertram Walter]

Hi,

here are my experiences with FSL and Condor:

On a Ubuntu 11.10 platform I have a working FSL-4.1.9-2-nd60+1  
installation and fsl-selftest (feeds 4.1.9.1) works fine.
/etc/fsl/fsl.sh is sourced in .bashrc

- I installed condor as described in  
http://neuro.debian.net/blog/2012/2012-03-09_parallelize_fsl_with_condor.html.

- condor_status gives now:

Name               OpSys      Arch   State     Activity LoadAv Mem    
ActvtyTime

slot1 at bion05.uni-g LINUX      X86_64 Owner     Idle     0.370  3960   
0+00:00:04
slot2 at bion05.uni-g LINUX      X86_64 Owner     Idle     0.000  3960   
0+00:00:05
                      Total Owner Claimed Unclaimed Matched Preempting  
Backfill

         X86_64/LINUX     2     2       0         0       0           
0        0

                Total     2     2       0         0       0           
0        0

- I started time fsl-selftest feat without FSLPARALLEL and got
...
start time = Fr 30. Mär 16:32:41 CEST 2012
hostname = bion05
os = Linux bion05 3.0.0-17-generic #30-Ubuntu SMP Thu Mar 8 20:45:39 UTC  
2012 x86_64 x86_64 x86_64 GNU/Linux
Starting FEAT at Fr 30. Mär 16:32:41 CEST 2012
...
end time = Fr 30. Mär 16:36:28 CEST 2012
...
real	3m46.880s
user	3m39.774s
sys	0m4.216s

- I added the line
     export FSLPARALLEL=condor
   to .bashrc

- I started time fsl-selftest feat in a newly opened terminal

condor_q showed

-- Submitter:  : <127.0.0.1:50413> : bion05.uni-giessen.de
  ID      OWNER            SUBMITTED     RUN_TIME ST PRI SIZE CMD
   12.0   walter          3/30 16:43   0+00:00:05 R  0   1.0  bash
   13.0   walter          3/30 16:43   0+00:00:00 H  0   1.0  bash
   14.0   walter          3/30 16:43   0+00:00:00 I  0   0.0   
cluster13_sentinel
   15.0   walter          3/30 16:43   0+00:00:00 H  0   1.0  bash
   16.0   walter          3/30 16:43   0+00:00:00 I  0   0.0   
cluster15_sentinel
   17.0   walter          3/30 16:43   0+00:00:00 H  0   1.0  bash
   18.0   walter          3/30 16:43   0+00:00:00 I  0   0.0   
cluster17_sentinel
   19.0   walter          3/30 16:43   0+00:00:00 H  0   1.0  bash
   20.0   walter          3/30 16:43   0+00:00:00 I  0   0.0   
cluster19_sentinel
   21.0   walter          3/30 16:43   0+00:00:00 H  0   1.0  bash
   22.0   walter          3/30 16:43   0+00:00:00 I  0   0.0   
cluster21_sentinel

11 jobs; 0 completed, 0 removed, 5 idle, 1 running, 5 held, 0 suspended

fsl-seftest resulted in
...
start time = Fr 30. Mär 16:43:48 CEST 2012
hostname = bion05
os = Linux bion05 3.0.0-17-generic #30-Ubuntu SMP Thu Mar 8 20:45:39 UTC  
2012 x86_64 x86_64 x86_64 GNU/Linux
...
Starting FEAT at Fr 30. Mär 16:43:48 CEST 2012
...
end time = Fr 30. Mär 16:49:01 CEST 2012
...
real	5m13.042s
user	0m6.588s
sys	0m0.712s

Oooh, much slower than without Condor

- I changed in /etc/condor/config.d/00debconf the lines
CONDOR_HOST = 127.0.0.1
ALLOW_WRITE = 127.0.0.1
as described in Neurodebian-users Digest, Vol 19, Issue 10
and restarted ubuntu for restarting condor completely.

- I started time fsl-selftest feat again.
After 15 minutes I checked condor_q:
-- Submitter:  : <127.0.0.1:45824> : bion05.uni-giessen.de
  ID      OWNER            SUBMITTED     RUN_TIME ST PRI SIZE CMD
   23.0   walter          3/30 17:05   0+00:00:00 I  0   1.0  bash
   24.0   walter          3/30 17:05   0+00:00:00 H  0   1.0  bash
   25.0   walter          3/30 17:05   0+00:15:00 R  0   0.0   
cluster24_sentinel
   26.0   walter          3/30 17:05   0+00:00:00 H  0   1.0  bash
   27.0   walter          3/30 17:05   0+00:15:00 R  0   0.0   
cluster26_sentinel
   28.0   walter          3/30 17:05   0+00:00:00 H  0   1.0  bash
   29.0   walter          3/30 17:05   0+00:15:00 R  0   0.0   
cluster28_sentinel
   30.0   walter          3/30 17:05   0+00:00:00 H  0   1.0  bash
   31.0   walter          3/30 17:05   0+00:15:00 R  0   0.0   
cluster30_sentinel
   32.0   walter          3/30 17:05   0+00:00:00 H  0   1.0  bash
   33.0   walter          3/30 17:05   0+00:15:00 R  0   0.0   
cluster32_sentinel

11 jobs; 0 completed, 0 removed, 1 idle, 5 running, 5 held, 0 suspended

- Quite disappointed I killed fsl-selftest.

What was going wrong?

Bertram


-- 
Dr. Bertram Walter
Bender Institute of Neuroimaging
University of Giessen
Otto-Behaghel-Str. 10H
35394 Giessen
Germany
Phone +49 (641) 99-26307
    or +49 (641) 99-26331 (Secretary)
Fax   +49 (641) 99-26309
www.bion.de