Which encoding is used by offlineimap?

Wed Sep 7 13:19:21 BST 2011

Hi there,
but the error shown is not as the following one:

 Copy message 33702 Maildir[INBOX] -> Gmail[INBOX]
 WARNING: ERROR attempting to copy message 33702 for account
Acc-Main:Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/offlineimap/folder/Base.py", line
255, in copymessageto
    newuid = dstfolder.savemessage(uid, message, flags, rtime)
  File "/usr/lib/pymodules/python2.6/offlineimap/folder/IMAP.py", line
463, in savemessage
    (typ,dat) = imapobj.check()
  File "/usr/lib/pymodules/python2.6/offlineimap/imaplib2.py", line
624, in check
    return self._simple_command('CHECK', **kw)
  File "/usr/lib/pymodules/python2.6/offlineimap/imaplib2.py", line
1549, in _simple_command
    return self._command_complete(self._command(name, *args), kw)
  File "/usr/lib/pymodules/python2.6/offlineimap/imaplib2.py", line
1310, in _command_complete
    typ, dat = rqb.get_response('command: %s => %%s' % rqb.name)
  File "/usr/lib/pymodules/python2.6/offlineimap/imaplib2.py", line
172, in get_response
    raise typ(exc_fmt % str(val))
abort: command: CHECK => connection terminated

 WARNING: ERROR attempting to sync flags for account
Acc-Main:Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/offlineimap/folder/Base.py", line
421, in syncmessagesto
    action(dstfolder, statusfolder)
  File "/usr/lib/pymodules/python2.6/offlineimap/folder/Base.py", line
318, in syncmessagesto_copy
    self.copymessageto(uid, dstfolder, statusfolder, register = 0)
  File "/usr/lib/pymodules/python2.6/offlineimap/folder/Base.py", line
255, in copymessageto
    newuid = dstfolder.savemessage(uid, message, flags, rtime)
  File "/usr/lib/pymodules/python2.6/offlineimap/folder/IMAP.py", line
463, in savemessage
    (typ,dat) = imapobj.check()
  File "/usr/lib/pymodules/python2.6/offlineimap/imaplib2.py", line
624, in check
    return self._simple_command('CHECK', **kw)
  File "/usr/lib/pymodules/python2.6/offlineimap/imaplib2.py", line
1549, in _simple_command
    return self._command_complete(self._command(name, *args), kw)
  File "/usr/lib/pymodules/python2.6/offlineimap/imaplib2.py", line
1310, in _command_complete
    typ, dat = rqb.get_response('command: %s => %%s' % rqb.name)
  File "/usr/lib/pymodules/python2.6/offlineimap/imaplib2.py", line
172, in get_response
    raise typ(exc_fmt % str(val))
abort: command: CHECK => connection terminated

 WARNING: ERROR in syncfolder for Acc-Main folder INBOX: Traceback
(most recent call last):
  File "/usr/lib/pymodules/python2.6/offlineimap/accounts.py", line
355, in syncfolder
    localfolder.syncmessagesto(remotefolder, statusfolder)
  File "/usr/lib/pymodules/python2.6/offlineimap/folder/Base.py", line
421, in syncmessagesto
    action(dstfolder, statusfolder)
  File "/usr/lib/pymodules/python2.6/offlineimap/folder/Base.py", line
318, in syncmessagesto_copy
    self.copymessageto(uid, dstfolder, statusfolder, register = 0)
  File "/usr/lib/pymodules/python2.6/offlineimap/folder/Base.py", line
255, in copymessageto
    newuid = dstfolder.savemessage(uid, message, flags, rtime)
  File "/usr/lib/pymodules/python2.6/offlineimap/folder/IMAP.py", line
463, in savemessage
    (typ,dat) = imapobj.check()
  File "/usr/lib/pymodules/python2.6/offlineimap/imaplib2.py", line
624, in check
    return self._simple_command('CHECK', **kw)
  File "/usr/lib/pymodules/python2.6/offlineimap/imaplib2.py", line
1549, in _simple_command
    return self._command_complete(self._command(name, *args), kw)
  File "/usr/lib/pymodules/python2.6/offlineimap/imaplib2.py", line
1310, in _command_complete
    typ, dat = rqb.get_response('command: %s => %%s' % rqb.name)
  File "/usr/lib/pymodules/python2.6/offlineimap/imaplib2.py", line
172, in get_response
    raise typ(exc_fmt % str(val))
abort: command: CHECK => connection terminated

 Syncing [Gmail]/Starred: Gmail -> Maildir
 Establishing connection to imap.gmail.com:993.
 WARNING: ERROR in syncfolder for Acc-Main folder Gmail_Starred:
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/offlineimap/accounts.py", line
330, in syncfolder
    if not remotefolder.isuidvalidityok():
  File "/usr/lib/pymodules/python2.6/offlineimap/folder/Base.py", line
86, in isuidvalidityok
    return self.getsaveduidvalidity() == self.getuidvalidity()
  File "/usr/lib/pymodules/python2.6/offlineimap/folder/IMAP.py", line
74, in getuidvalidity
    imapobj = self.imapserver.acquireconnection()
  File "/usr/lib/pymodules/python2.6/offlineimap/imapserver.py", line
256, in acquireconnection
    self.plainauth(imapobj)
  File "/usr/lib/pymodules/python2.6/offlineimap/imapserver.py", line
133, in plainauth
    imapobj.login(self.username, self.getpassword())
  File "/usr/lib/pymodules/python2.6/offlineimap/imaplib2.py", line
793, in login
    self._deliver_exc(self.error, dat[-1], kw)
  File "/usr/lib/pymodules/python2.6/offlineimap/imaplib2.py", line
1356, in _deliver_exc
    raise exc(dat)
error: [ALERT] Account exceeded command or bandwidth limits. (Failure)

 Syncing [Gmail]/Sent Mail: Gmail -> Maildir
 Establishing connection to imap.gmail.com:993.
 WARNING: ERROR in syncfolder for Acc-Main folder Gmail_Sent_Mail:
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/offlineimap/accounts.py", line
330, in syncfolder
    if not remotefolder.isuidvalidityok():
  File "/usr/lib/pymodules/python2.6/offlineimap/folder/Base.py", line
86, in isuidvalidityok
    return self.getsaveduidvalidity() == self.getuidvalidity()
  File "/usr/lib/pymodules/python2.6/offlineimap/folder/IMAP.py", line
74, in getuidvalidity
    imapobj = self.imapserver.acquireconnection()
  File "/usr/lib/pymodules/python2.6/offlineimap/imapserver.py", line
256, in acquireconnection
    self.plainauth(imapobj)
  File "/usr/lib/pymodules/python2.6/offlineimap/imapserver.py", line
133, in plainauth
    imapobj.login(self.username, self.getpassword())
  File "/usr/lib/pymodules/python2.6/offlineimap/imaplib2.py", line
793, in login
    self._deliver_exc(self.error, dat[-1], kw)
  File "/usr/lib/pymodules/python2.6/offlineimap/imaplib2.py", line
1356, in _deliver_exc
    raise exc(dat)
error: [ALERT] Account exceeded command or bandwidth limits. (Failure)

.....

and the previous error (when copy Maildir -> Gmail) is solved by
deleting the affected email
and launching offlineimap again.

I've found that whenever this error happens to me the affected email
contains an
encoding different from utf8 (I can not see some of the characters by
using less)
and most of the times "file -bi filename" reports binary (but not always).

At first I though charset=binary could be used to say that emails
contain attachments,
but I've found some emails containing huge attachments where this
command reports
charset different from binary (for example charset=us-ascii).

Maybe I'm wrong ...

One question: is it normal to get the following result when counting
the number of
emails in each encoding? I mean, shouldn't all them be using the same encoding?

 Previous encoding of files was:
     74 /home/vpablos/secured/encrypted/Mail-Backup/Logs/encodings//binary
   6089 /home/vpablos/secured/encrypted/Mail-Backup/Logs/encodings//iso-8859-1
    681 /home/vpablos/secured/encrypted/Mail-Backup/Logs/encodings//unknown-8bit
  42843 /home/vpablos/secured/encrypted/Mail-Backup/Logs/encodings//us-ascii
   4789 /home/vpablos/secured/encrypted/Mail-Backup/Logs/encodings//utf-8
  54476 total

Thanks,

Victor

On Tue, Sep 6, 2011 at 9:00 AM, Sebastian Spaeth <Sebastian at sspaeth.de> wrote:
> On Sat, 3 Sep 2011 14:38:10 +0200, Victor Pablos Ceruelo <victorpablosceruelo at gmail.com> wrote:
>> abort: command: APPEND => socket error: <type 'exceptions.IOError'> -
>> Too many read 0
>
> Hi there,
> it's got nothing to do with encodings, you error is the same all
> over. Gmail dropped some connections (we are getting lots of reports of
> Gmail doing this), and we fail to handle it gracefully.
>
> Sebastian
>