[Imaplib2-devel] imaplib2 debugging

[Piers Lauder]

On Sat, 07 May 2011 11:31:06 +0200, Sebastian Spaeth wrote:
  > 
  > On Sat, 07 May 2011 10:37:45 +1000, Piers Lauder <piers at janeelix.com> wrote:
  > > I have committed a new version of imaplib2 with extra logging, and a
  > > new paremeter to the class 'debug_buf_lvl' (default: 3), that controls
  > > log buffering.
  > > 
  > > So, to invoke IMAP4 with output to stderr at logging level 3, one would
  > > invoke as:
  > >   IMAP4(..., debug=3, debug_buf_lvl=2)
  > 
  > Thanks, I see that  debug=3, debug_buf_lvl=2, we would not actually log
  > _log(3,... messages, but we would add them to the circular buffer of 20
  > last debug messages, right?

Right.

  > > More lock traces now come out at debug level 3, which involved much less
  > > output than level 5 (which traces actual I/O).
  > 
  > Uhh, we probably should reduce our default logging level to 4 (it is 5
  > now in cases we have debug turned on).

4 would be ok as well (and add more useful info than 3).

  > > Hope that helps...
  > 
  > One question, in _log() you have:
  >                 self.debug_lock.acquire()
  >                 self._mesg(line, tn)
  >                 self.debug_lock.release()
  > 
  > So you have a per IMAP-instance lock protecting from concurrent writes
  > to stderr. However, given our multithreaded, multiconnection nature, we
  > can (and do) and multiple IMAP instances running simultanously, and they
  > share the same log file (debug_file). I am not sure this could cause
  > hangs, but could certainly lead to corrupted debug output, can't it?

It would mean than any one imaplib2 instance synchronises its own threads
when writing to the logging instance. If multiple objects are going to
write to the one logger, than that loogger itself needs to institute
synchronisation. There would be no corruption and no conflicts provided
it uses it's own semaphore in code like that above. (In which case of course,
the imaplib2 synchronisation becomes redundant.)


  > Sorry, threads and locks give me the creeps and make my head hurt :).
  > 
  > I will propose a patch to offlineimap, reducing our default debug level
  > and updating imaplib2 to the latest and greatest. As we plan to
  > introduce the IDLE support very soon. I see you have locking fixes for
  > IDLE too, are they crucial?

They are because the IDLE can be cancelled from more than one thread,
and though not crucial, I've included them for thoroughness sake. Ie:
the lack wouldn't cause a hang, just that the IDLE might not start
properly (only to be immediately cancelled by some activity). So a
piece of code added for the sake of correctness, rather than essential.

Piers.