Bug#636075: boinc-client: Boinc is (too) disk intensive
franckr
franckr01 at online.de
Mon Aug 1 17:06:25 UTC 2011
Hi Steffen,
Thanks for your input.
As on your system you have low disk IO, I reinstalled boinc (this time manually on HDD) and checked one application at a time from project "Wold Community Grid".
<1> Conditions:
IO throughput for 1 task running at a time (if 2 runnings at same time => I divided by 2 the write IO/hr)
On a dual core Core2 at 3.2Ghz (faster CPU/more cores will have higher write IO).
Set "Task check point to disk at most every" 60000s
<2> Findings:
It is application dependant:
- Nice applications (write small amount)
(ca. 200kb/min) Help Conquer Cancer
(ca. 800kb/min) FightAIDS at Home
- Guilty application:
The Clean Energy Project - Phase 2
This one write *** 2 to 2.9 Gb/hr *** ! (again only 1 running on one core on an old CPU)
And nearly no read => that's a waste of participant's ressources and computer.
'System Requirements' announced on 'World Community Grid' says
Memory Available 1,024 MB
Disk Space 2,048 MB
But nowhere it is indicated that it is so write intensive !
Their might be other guilty applications, but I cannot check them all. Furthermore it may be task dependant too.
<3> Conclusion
I think Boinc should control applications, and allow the user to set 'max writing', as it todays allow 'max cpu', 'max memory'.
Thus suggested potential solutions:
a- Applications to announce estimated write I/O (read I/O too?)
b- Boinc allow user to limit max I/O per tasks (and if exceed, stop task & use other applications). When user set, can compare to Application announcement to avoid starting with limit < estimation.
c- install by default in /home (ex: /home/boinc) not in /var/lib/boinc-client. This will increase the chances to land on a HDD and not on a SSD.
<4> details 'Clean Energy Project - Phase 2' IO
Conditions:
- $ time iotop -u boinc -a
might need to install 'aptitude install iotop'
- check that Process ID are the same before and after measurements. Because some processes disappear and new ones appear (during same task!); but iotop only shows current running Processes, not anymore disappeared one => one can miss some IO.
- with Project 'World Community Grid'
Application 'The Clean Energy Project - Phase 2 6.40'
Starting task E202831_920_C.27.C21H11NOS3Si.00590497.3.set1d06_1 using cep2 version 640
- Set "Task check point to disk at most every" 60000s
Result1:
TID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND
5328 idle boinc 0.00 B 1180.60 M 0.00 % 0.01 % ../../projects/www.worldcommunitygrid.o~86.svp.n.opt 2378613406239751723 41381 0
5151 idle boinc 0.00 B 140.00 K 0.00 % 0.00 % ../../projects/www.worldcommunitygrid.o~chem_prod_linux.x86 -float 1 -stop 43200
5153 idle boinc 0.00 B 32.00 K 0.00 % 0.00 % ../../projects/www.worldcommunitygrid.o~chem_prod_linux.x86 -float 1 -stop 43200
5330 idle boinc 0.00 B 112.00 K 0.00 % 0.00 % ../../projects/www.worldcommunitygrid.o~86.svp.n.opt 2378613406239751723 41381 0
1951 idle boinc 0.00 B 84.00 K 0.00 % 0.00 % boinc --check_all_logins --redirectio --dir /var/lib/boinc-client
5152 idle boinc 0.00 B 0.00 B 0.00 % 0.00 % ../../projects/www.worldcommunitygrid.o~chem_prod_linux.x86 -float 1 -stop 43200
5329 idle boinc 0.00 B 0.00 B 0.00 % 0.00 % ../../projects/www.worldcommunitygrid.o~86.svp.n.opt 2378613406239751723 41381 0
real 26m45.474s
user 0m10.609s
sys 0m2.836s
1180.6/(26*60+455)*3600/1024 = 2.06Gb/hr
Result2:
TID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND
5328 idle boinc 0.00 B 829.65 M 0.00 % 0.01 % ../../projects/www.worldcommunitygrid.o~86.svp.n.opt 2378613406239751723 41381 0
5151 idle boinc 0.00 B 84.00 K 0.00 % 0.00 % ../../projects/www.worldcommunitygrid.o~chem_prod_linux.x86 -float 1 -stop 43200
5153 idle boinc 0.00 B 28.00 K 0.00 % 0.00 % ../../projects/www.worldcommunitygrid.o~chem_prod_linux.x86 -float 1 -stop 43200
5330 idle boinc 0.00 B 64.00 K 0.00 % 0.00 % ../../projects/www.worldcommunitygrid.o~86.svp.n.opt 2378613406239751723 41381 0
1951 idle boinc 0.00 B 56.00 K 0.00 % 0.00 % boinc --check_all_logins --redirectio --dir /var/lib/boinc-client
5152 idle boinc 0.00 B 0.00 B 0.00 % 0.00 % ../../projects/www.worldcommunitygrid.o~chem_prod_linux.x86 -float 1 -stop 43200
5329 idle boinc 0.00 B 0.00 B 0.00 % 0.00 % ../../projects/www.worldcommunitygrid.o~86.svp.n.opt 2378613406239751723 41381 0
real 16m55.249s
user 0m6.636s
sys 0m1.816s
829.65/(16*60+55)*3600/1024 = 2.87Gb/hr
ResultA: Just for the fun, I let it run
Total DISK READ: 0.00 B/s | Total DISK WRITE: 0.00 B/s
TID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND
5328 idle boinc 1352.00 K 3.37 G 0.00 % 0.01 % ../../projects/www.worldcommunitygrid.o~86.svp.n.opt 2378613406239751723 41381 0
5151 idle boinc 4.00 K 0.00 B 0.00 % 0.00 % ../../projects/www.worldcommunitygrid.o~chem_prod_linux.x86 -float 1 -stop 43200
5153 idle boinc 0.00 B 136.00 K 0.00 % 0.00 % ../../projects/www.worldcommunitygrid.o~chem_prod_linux.x86 -float 1 -stop 43200
5330 idle boinc 0.00 B 320.00 K 0.00 % 0.00 % ../../projects/www.worldcommunitygrid.o~86.svp.n.opt 2378613406239751723 41381 0
1951 idle boinc 0.00 B 248.00 K 0.00 % 0.00 % boinc --check_all_logins --redirectio --dir /var/lib/boinc-client
5152 idle boinc 0.00 B 0.00 B 0.00 % 0.00 % ../../projects/www.worldcommunitygrid.o~chem_prod_linux.x86 -float 1 -stop 43200
5329 idle boinc 0.00 B 0.00 B 0.00 % 0.00 % ../../projects/www.worldcommunitygrid.o~86.svp.n.opt 2378613406239751723 41381 0
This was at 11.25% of task, but as I said some process ID disappeared and at least several hundred Mb are missings.
(at least this one disappeared
5229 idle boinc 556.00 K 223.21 M 0.00 % 0.08 % ../../projects/www.worldc~188371360687645299 27960 0)
It is nice to see:
disk read: 1352.00 K
disk write: 3.37 G
=> Completely meaningless to write so much.
Hoping above will help to improve Boinc and its Projects.
Greetings,
Franck
More information about the pkg-boinc-devel
mailing list