<div dir="ltr">Hi all,<div><br></div><div>I am currently working on basic classification problems on resting state fMRI datasets (like gender differences, for example) The dimensionality of feature vector is pretty large (128 time points X 33 X 64 X 64) I am working on a linux server with 8 cores and suff RAM. Following is the output of command 'free -m'<br>
<br>free -m<br> total used free shared buffers cached<br>Mem: 8006 1254 6752 0 100 854<br>-/+ buffers/cache: 299 7706<br>Swap: 23454 209 23244<br>
<br>When I try to train a simple linear SVM classifier on the dataset (I have 30 data points each of above said dimensionality) the program goes into some sort of infinite loop. (no errors) I had to reduce num of timepoints to like 50 in order to get back some results within reasonable time like 7 mins. Is that pymvpa limitation or is it a function of RAM/processor? Because, it does not throw out any error like insufficient memory, just takes an infinite time to process! (I waited for a day to get result on 1 specific pair of training and testing dataset, single fold cross validation)<br>
<br>I would like to know whats the max data size pymvpa can handle in terms of computation time? And how is it related to RAM/SWAP space/processor speed or any other factors? So that I can make relevant recommendations to our sys admin. Also, Is there is any possible workaround for this problem or do have I to go for some sort of dimensionality reduction?<br clear="all">
<br>extra info: output of cat /proc/cpuinfo<br><br>cat /proc/cpuinfo <br>processor : 0<br>vendor_id : GenuineIntel<br>cpu family : 6<br>model : 23<br>model name : Intel(R) Xeon(R) CPU E5405 @ 2.00GHz<br>
stepping : 6<br>cpu MHz : 1994.998<br>cache size : 6144 KB<br>physical id : 0<br>siblings : 4<br>core id : 0<br>cpu cores : 4<br>apicid : 0<br>initial apicid : 0<br>fpu : yes<br>
fpu_exception : yes<br>cpuid level : 10<br>wp : yes<br>flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall lm constant_tsc arch_perfmon pebs bts rep_good pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca sse4_1 lahf_lm<br>
bogomips : 3993.21<br>clflush size : 64<br>cache_alignment : 64<br>address sizes : 38 bits physical, 48 bits virtual<br>power management:<br><br>processor : 1<br>vendor_id : GenuineIntel<br>cpu family : 6<br>
model : 23<br>model name : Intel(R) Xeon(R) CPU E5405 @ 2.00GHz<br>stepping : 6<br>cpu MHz : 1994.998<br>cache size : 6144 KB<br>physical id : 1<br>siblings : 4<br>
core id : 0<br>cpu cores : 4<br>apicid : 4<br>initial apicid : 4<br>fpu : yes<br>fpu_exception : yes<br>cpuid level : 10<br>wp : yes<br>flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall lm constant_tsc arch_perfmon pebs bts rep_good pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca sse4_1 lahf_lm<br>
bogomips : 3990.05<br>clflush size : 64<br>cache_alignment : 64<br>address sizes : 38 bits physical, 48 bits virtual<br>power management:<br><br>processor : 2<br>vendor_id : GenuineIntel<br>cpu family : 6<br>
model : 23<br>model name : Intel(R) Xeon(R) CPU E5405 @ 2.00GHz<br>stepping : 6<br>cpu MHz : 1994.998<br>cache size : 6144 KB<br>physical id : 0<br>siblings : 4<br>core id : 2<br>
cpu cores : 4<br>apicid : 2<br>initial apicid : 2<br>fpu : yes<br>fpu_exception : yes<br>cpuid level : 10<br>wp : yes<br>flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall lm constant_tsc arch_perfmon pebs bts rep_good pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca sse4_1 lahf_lm<br>
bogomips : 3990.04<br>clflush size : 64<br>cache_alignment : 64<br>address sizes : 38 bits physical, 48 bits virtual<br>power management:<br><br>processor : 3<br>vendor_id : GenuineIntel<br>cpu family : 6<br>
model : 23<br>model name : Intel(R) Xeon(R) CPU E5405 @ 2.00GHz<br>stepping : 6<br>cpu MHz : 1994.998<br>cache size : 6144 KB<br>physical id : 1<br>siblings : 4<br>core id : 2<br>
cpu cores : 4<br>apicid : 6<br>initial apicid : 6<br>fpu : yes<br>fpu_exception : yes<br>cpuid level : 10<br>wp : yes<br>flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall lm constant_tsc arch_perfmon pebs bts rep_good pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca sse4_1 lahf_lm<br>
bogomips : 3990.06<br>clflush size : 64<br>cache_alignment : 64<br>address sizes : 38 bits physical, 48 bits virtual<br>power management:<br><br>processor : 4<br>vendor_id : GenuineIntel<br>cpu family : 6<br>
model : 23<br>model name : Intel(R) Xeon(R) CPU E5405 @ 2.00GHz<br>stepping : 6<br>cpu MHz : 1994.998<br>cache size : 6144 KB<br>physical id : 0<br>siblings : 4<br>core id : 1<br>
cpu cores : 4<br>apicid : 1<br>initial apicid : 1<br>fpu : yes<br>fpu_exception : yes<br>cpuid level : 10<br>wp : yes<br>flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall lm constant_tsc arch_perfmon pebs bts rep_good pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca sse4_1 lahf_lm<br>
bogomips : 3990.05<br>clflush size : 64<br>cache_alignment : 64<br>address sizes : 38 bits physical, 48 bits virtual<br>power management:<br><br>processor : 5<br>vendor_id : GenuineIntel<br>cpu family : 6<br>
model : 23<br>model name : Intel(R) Xeon(R) CPU E5405 @ 2.00GHz<br>stepping : 6<br>cpu MHz : 1994.998<br>cache size : 6144 KB<br>physical id : 1<br>siblings : 4<br>core id : 1<br>
cpu cores : 4<br>apicid : 5<br>initial apicid : 5<br>fpu : yes<br>fpu_exception : yes<br>cpuid level : 10<br>wp : yes<br>flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall lm constant_tsc arch_perfmon pebs bts rep_good pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca sse4_1 lahf_lm<br>
bogomips : 3990.05<br>clflush size : 64<br>cache_alignment : 64<br>address sizes : 38 bits physical, 48 bits virtual<br>power management:<br><br>processor : 6<br>vendor_id : GenuineIntel<br>cpu family : 6<br>
model : 23<br>model name : Intel(R) Xeon(R) CPU E5405 @ 2.00GHz<br>stepping : 6<br>cpu MHz : 1994.998<br>cache size : 6144 KB<br>physical id : 0<br>siblings : 4<br>core id : 3<br>
cpu cores : 4<br>apicid : 3<br>initial apicid : 3<br>fpu : yes<br>fpu_exception : yes<br>cpuid level : 10<br>wp : yes<br>flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall lm constant_tsc arch_perfmon pebs bts rep_good pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca sse4_1 lahf_lm<br>
bogomips : 3990.05<br>clflush size : 64<br>cache_alignment : 64<br>address sizes : 38 bits physical, 48 bits virtual<br>power management:<br><br>processor : 7<br>vendor_id : GenuineIntel<br>cpu family : 6<br>
model : 23<br>model name : Intel(R) Xeon(R) CPU E5405 @ 2.00GHz<br>stepping : 6<br>cpu MHz : 1994.998<br>cache size : 6144 KB<br>physical id : 1<br>siblings : 4<br>core id : 3<br>
cpu cores : 4<br>apicid : 7<br>initial apicid : 7<br>fpu : yes<br>fpu_exception : yes<br>cpuid level : 10<br>wp : yes<br>flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall lm constant_tsc arch_perfmon pebs bts rep_good pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca sse4_1 lahf_lm<br>
bogomips : 3990.06<br>clflush size : 64<br>cache_alignment : 64<br>address sizes : 38 bits physical, 48 bits virtual<br>power management:<br><br>Thanks in advance!<br>-- <br>Alok S. Deshpande<br>Graduate Student<br>
Electrical & Computer Engineering Department<br>University of Wisconsin Madison<br>
</div></div>