A comment on rewrites

[Stefano Canepa]

On Thu, May 20, 2010 at 08:34:20AM -0500, John Goerzen wrote:
> This is (collectively) your project now, so by all means feel free
> to ignore me entirely.  I wanted to share a bit of experience about
> rewrites of large free software projects.
>
> Most of the time, when there is an established Open Source project
> with a fairly significant codebase, and some people go off to
> rewrite it from scratch, the rewrite never gets done.  If enough
> people from the original project get invested in the rewrite, it may
> even kill the original project and wind up with no replacement.
> There are exceptions, of course, but I've seen this pattern several
> times.
>
> Part of the reason for this was already mentioned by someone else:
> if the project is large enough to merit thoughts of a rewrite,
> chances are it's also pretty large and complex.  OfflineIMAP has
> somewhere around 50 configuration options, three mail backend
> drivers, 4 UIs (plus a fifth Gnome one in development), and 8 years
> of testing and use.  It also has various hooks for extending it with
> custom Python code.
>
> This won't be duplicated overnight in any language with any library.
> In fact, it will take quite a bit of time to duplicate.  I should
> know: I've attempted a rewrite twice, running out of steam myself.
> The first was a few years back when I rewrote it to use Python
> Twisted.  I got it to actually work to a certain extent before
> giving up in frustration at the difficulty of debugging in Twisted.
> More recently I tried to rewrite it in Haskell, but ran out of time
> due to other commitments.
>
> There are some problems with OfflineIMAP, but a whole rewrite isn't
> needed to fix them.  The CPU-bound problem can be dealt with quite
> easily by using Sqlite or something like it for the LocalStatus
> storage.  There have been patches sent to the list in the past that
> got about 90% of the way to being accepted, in fact.  (If I recall
> correctly, they ultimately weren't because they didn't migrate
> existing LocalStatus data.)  There may be other ways to address it
> also.  The problem is unlikely to be language, but rather algorithm
> in a particular area.
>
> If a rewrite is attempted, it should be recognized that:
>
> 1) This will take a significant amount of time before it can begin
> to displace OfflineIMAP 6.2.x
>
> 2) In the meantime, the 6.2.x branch will need to be kept active or
> else the community will be lost before the rewrite is finished.
>
> 3) Ease of migration is important.
>
> I think it unlikely that choice of language will impact OfflineIMAP
> performance or reliability.  A well-written algorithm will always be
> much faster than the networked IMAP server.  OfflineIMAP's current
> algorithms were never designed with tens of thousands of messages
> per folder in mind, and it is natural that they aren't very fast in
> such situations.  (Note: I say that as someone with strong opinions
> in programming languages; choice of language *does* impact some
> things, but for this project, is unlikely to impact performance.)
>
> It may be that incremental changes result in, say, 3 years down the
> road none of my OfflineIMAP code survives.  That is fine, and a sign
> of an active project.  Such a process for change is more likely to
> succeed than an entire rewrite that is completed in 1 year, and also
> more likely to have high quality after 3.
>
> One other question is whether a rewritten project ought to be called
> "OfflineIMAP".  The answer probably depends upon how well it can
> grok ~/.offlineimaprc, the ~/.offlineimap metadata storage, and
> implements the full OfflineIMAP feature set.  If it doesn't do all
> of this pretty well, it probably ought to be named something else.
>
> Having said that: once again, this is your project, so to those of
> you that are willing to contribute to the community -- whether it be
> a rewrite or not, whether it is called OfflineIMAP or not -- more
> power to you, and thanks for the work you do.
>
> -- John

I'm reading this email really late becouse I was a little bit busy
during this period.

I agree completelly with John, I started studing offlineimap code
and I'm a great fun of python, if someone else is interested in
a co-maintainance of the code I can try to find some time to=20
work on the project.

Speaking about a rewrite in other languages there are offlineimap
alternatives written in perl or C.

Bye
Stefano

-- 
Stefano Canepa aka sc: sc at linux.it - http://www.stefanocanepa.it
Three great virtues of a programmer: laziness, impatience and hubris.
Le tre grandi virtù di un programmatore: pigrizia, impazienza e
arroganza. (Larry Wall)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://alioth-lists.debian.net/pipermail/offlineimap-project/attachments/20100702/ab926b04/attachment.sig>