IDLE again

[Dominic LoBue]

On Thu, Dec 3, 2009 at 2:47 PM, Dominic LoBue <dom.lobue at gmail.com> wrote:
> On Thu, Dec 3, 2009 at 9:58 AM,  <exarkun at twistedmatrix.com> wrote:
>> On 05:00 pm, jgoerzen at complete.org wrote:
>>>
>>> Dominic LoBue wrote:
>>>>
>>>> I am willing to do the legwork myself on this one to get it working,
>>>> though I could use your assistance in determining the best approach to
>>>> take.
>>>
>>> Great!
>>>
>>> I suspect that the easiest options would be:
>>>
>>> 1) Identify and fix the bugs from imaplib2
>>>
>>> 2) Add IDLE support to imaplib (or use imaplib to provide it)
>>>
>>> imaplib2 proved to be a massive user experience problem.  I don't like
>>> the idea of being "buggy by default" (defaulting to imaplib2 with an
>>> option to revert to imaplib), nor do I like a "more bugs please, but
>>> also enable IDLE" sort of option (imaplib default with imaplib2 as an
>>> option).  Enabling IDLE shouldn't cause people to have trouble.
>>>>
>>>> My other thought is to add IDLE support to the twisted imap client
>>>> library, and switch offlineimap over to twisted.
>>>
>>> I attempted that conversion myself some years ago, and had it mostly
>>> working, but it was terrible to maintain.  There also seem to be
>>> questions about the long-term viability of Twisted; I have heard that
>>> they intend to never port it to Python 3.
>>
>> Aside from there being questions about the long-term viability of Python 3
>> (wink), you can read about Twisted's Python 3 porting plans here:
>>
>>  http://twistedmatrix.com/trac/ticket/2484
>>
>> and some more general thoughts on Python 3 porting on stackoverflow:
>>
>>  http://bit.ly/6hWqYU
>>
>> The short version is that it'll happen when it's sufficiently useful.
>>
>> If we can momentarily pretend that the Python 3 question is answered, then
>> I'll just toss in that I'd be quite interested to see offlineimap use
>> Twisted for it's protocol needs.
>>
>> I know that twisted.mail.imap4 has issues (I recently filed a pretty major
>> ticket against it, in fact: <http://twistedmatrix.com/trac/ticket/4124>).
>>  I'm not trying to suggest that switching to Twisted's APIs will be a snap.
>>  I don't even have a good idea about whether it would be easier than fixing
>> the imaplib2 issues or adding the necessary features to imaplib (although I
>> do *feel* like imaplib is so low-level that, overall, using Twisted's APIs
>> would have to be an improvement).
>>
>> I do know that Twisted development is active, maintenance is ongoing, and if
>> there is interest in improving the IMAP4 code, I'm here to help with the
>> process.  So if you still decide not to use Twisted, do it for reasons other
>> than those. :)
>>
>> I'll go back to lurking now.
>>
>> Jean-Paul
>>
>
> Jean-Paul,
>
> Your offer is much appreciated, thank you!
>
> Just to give you fair warning: I've never worked with twisted before.
> I am very interested in learning however. If you're cool with
> answering some of my questions and showing me the ropes though, I
> totally accept your offer!
>
> I'm willing to put in the time required to porting OfflineIMAP over to
> Twisted if someone can smooth over Twisted's learning curve for me.
>
> --
> Dominic LoBue
>

Jean-Paul,

I read over the IMAP IDLE RFC and went through the IMAP4 twisted
library and I've sketched out a rough outline of how to implement
IDLE. I've run into some things in Twisted however that I don't really
understand well, and I'm hoping you can point me in the correct
direction.

First, just to verify my understanding: the IMAP4Client class is a
Protocol class. All IMAP commands are represented by at least two
methods in the IMAP4Client class - one for what to do when the command
is received from the server, and one for when the command is sent to
the server. Assuming this assertion to be true, the broad strokes of
the IDLE implementation is as follows:

-IDLE is engaged: command is sent to server turning on IDLE, schedule
an IDLE reset in 29 minutes, and an attribute ( _IDLE_Enabled for
example ) is set to True.

-In the command dispatcher code: checks if IDLE is enabled or not. If
enabled, it appends IDLE to all incoming commands and sends a DONE to
the server before any new commands are sent. (On the incoming commands
part: what I mean is if the method originally to be called was
"incoming_exists", instead it would go to "incoming_existsIDLE".)

-When IDLE is confirmed off by server: delete scheduled IDLE reset.

And that's basically it I think. Fancy stuff like downloading the
messages that IDLE notifies you about are handled in the
ClientFactory, right?

Some things that I'm not all that clear on and could use your help to
understand:
How do you cancel a previously queued/scheduled callback?
How are multiple connections to the same server handled? And more
importantly: how do you have a command in one session use a callback
on another already-open connection?

-- 
Dominic LoBue