[Teammetrics-discuss] Commit Stat.

[Andreas Tille]

On Thu, Oct 13, 2011 at 10:37:59PM +0530, Sukhbir Singh wrote:
> :) So I guess this is what was causing the problem. 2.3GB and 211MB is
> not exactly 'small', and that too when we are calling it via
> `subprocess`.

My guess is that this is binary data which is submitted (at least to a
large extend).

> I think the reason the script stops is that it _appears_ to stop
> because it is processing all this data. And this is the reason for ~
> 98% CPU usage, IMHO. If you think that this is the cause, the only
> probable way of solving that I see is that we dump the output of each
> revision to a temp file, parse it and then save it to `parse.info`.
> 
> It's worth a try if you feel that I should continue on this track...
> let me know!

Considering the fact that is binary data counting 'lines of code' does
not make sense anyway.  So I would count such submissions as "1
submission" "+1 line of code added" (it might be somehow unfair, but I
do not see a more reasonable approach).  So if you have a chance to
decide about the size (or even better the content beeing binary or not)
*before* you fetch the data and prevent trying to run your process into
large trouble I would just do it this way.

Try whatever approach seems to be more simple.

Kind regards

         Andreas.

-- 
http://fam-tille.de