[Teammetrics-discuss] Web Parser

Sukhbir Singh sukhbir.in at gmail.com
Thu Dec 1 11:38:46 UTC 2011


Hi,

Yesterday I had updated the web archive parser and fixed some issues
with the Date handling and all. Now it's in an operational state much
better than the previous one, however there is one issue that I should
discuss.

After a certain amount of messages downloaded (usually 100+),
lists.d.o stops responding for sometime. This causes the
urllib2.URLError to be thrown but lists.d.o is not responding; to be
sure that this problem was not in the code, I noticed that it failed
to load even through the browser. After a few seconds, it starts
responding again. This is totally random. I have handled this
exception but *not* implemented a mechanism that tries to download the
message again.

I was wondering, is this expected? I mean, did you face this issue
with your code, ever?

-- 
Sukhbir



More information about the Teammetrics-discuss mailing list