Configuration boilerplate

[Dave Abrahams]

on Fri Aug 26 2011, Sebastian Spaeth <Sebastian-AT-SSpaeth.de> wrote:

> On Thu, 25 Aug 2011 07:17:12 -0800, Dave Abrahams <dave at boostpro.com> wrote:
>> Scenario: it takes hours to sync "All Mail."  INBOX syncs in seconds,
>> but I want to continue to receive mail in my INBOX while "All Mail" is
>> syncing.  Even if syncing starts with INBOX, it gets "stuck" doing hours
>> of work on "All Mail" and I can't read anything in INBOX in the
>> meantime.
>
> I see. Yes that would be difficult to achieve. With the new per-account
> locking system you could workaround this with different accounts like
> you do now and simply leave another offlineimap running in background
> syncing "All Mail"...
>  
>> > Sure, but this would require that something global keeps tab of all used
>> > maildir folders across accounts. 
>> 
>> Huh?  Just have a step at startup that validates the config file, neh?
>
> Some examples:
> 1)
> Account 1: IMAP1 <-> Maildir1
> Account 2: IMAP2 <-> Maildir2
>
> Maildir1: 
>  root = /home/u/mail
>  sep=/
>
> Maildir2:
>  root = /home/u/mail/Personal
>
> What should we do? Abort because if you have a folder Personal/Favorites
> on your IMAP1 you'll share a Maildir folder, but not if you don't have a
> "Personal" folder on IMAP1?

If the filterfolder and nametrans settings allow the possibility of
overlap, issue a big, ugly warning that encourages people to make
changes ensuring no
overlap

> 2)
> Account 1: IMAP1 <-> Maildir1 (root = /home/u/mail)
> Account 2: IMAP2 <-> Maildir2 (root = /home/u/mail)
>
> IMAP1: 
>  folderfilter = lambda foldername: foldername not in ['Spam']
>
> IMAP2:
>  folderfilter = lambda foldername: foldername in ['[Gmail]/All Mail']
>  nametrans = lambda folder: 'IMAP2.' + folder
>
> (Recognize that setup? its similar to yours) What should we do? 

The same as above.

> Abort because *if* you have a folder IMAP2 on your IMAP1 you'll share
> a Maildir folder, etc. With folderfilters and nametrans, it is
> non-trivial to detect in which cases we will be sharing a maildir. The
> best we could do is to detect whether a maildir root lies within
> another maildir root. But that would prevent e.g. your careful setup
> from running.

I wouldn't mind changing my setup to avoid a big ugly warning.

> Happy to accept solutions for sane validation mechanisms.
>
>> > The way OfflineImap is designed has Accounts be totally different
>> > operations that know nothing of each other, and even folders are
>> > pretty much separated (which is one of the reasons why it would be
>> > hard to detect moves of mails between folders).
>> 
>> I don't see the problem here, sorry.
>
> A Folder() object has no knowledge about mails in other Folder()
> objects, in fact they usually don't have the lists of containing mails
> loaded at the same time, so detecting a move from FolderA to FOlderB is
> non-trivial with the current architecture.
>
> Same with Accounts and as I have shown, without knowing which folders
> exist on the servers, it is not possible to detect if there will
> actually be sharing of maildirs.

Right... so use warnings?

Also, maybe the mapping should be from maildirs to accounts.  One way to
avoid allowing a many-to-one mapping is to point the arrows in the
opposite direction.  Just a suggesiton; not sure where it leads.

>> My first thought was to give me (documented) programmatic access to the
>> configuration so I can write Python code that builds whole sections of
>> the config file.  However, it occurs to me that a one syntactic
>> extension could reduce my config considerably.  I've rewritten it below
>> using an "inherit =" clause for each remote repository.
>
> inherit sounds like a good solution to me and should be possible to
> implement somehow.
>
> On top of the core functionality, SafeConfigParser supports
> interpolation. This means values can contain format strings which
> refer to other values in the same section, or values in a special
> DEFAULT section. Additional defaults can be provided on
> initialization.

The only problem with that is that I still need to list all the values
for each section.  The problem is that I have many whole sections that
are substantially identical.  The only place where this interpolation is
a big win is when the actual values in the settings are much much longer
than the key names.  If the key names are comparable in length to the
values, and I have to repeat the key names anyway, it's not such a big
help.

> So you could at least set common variables only once for all the
> accounts (although you would still need the boilerplate). This would be
> a single line of code change in OfflineImap.
>
> I've noted the 'inherited' idea down, I like it a lot.
>
> Sebastian

-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com