Automated tests using dovecot
glasse at cs.rpi.edu
Tue Jun 14 12:01:53 BST 2011
On 06/09/2011 02:37 PM, Nicolas Sebrecht wrote:
> First of all, I didn't test the tests. I don't want to install any
> system-wide imap server on my laptop and the HDD on my very old server
> crashed, so I have to wait for a new full server. I've read the scripts.
Could you clarify what you would be willing to install on your laptop?
It's possible to install dovecot without it running as a system service
-- that's what I do. Are you unwilling to have dovecot installed at all?
> There are 3 ways to test an IMAP client:
> - use any IMAP server such as gmail, gmx, etc. (this is somewhat
> limited as we tester can't check what was done on the server);
> - install and configure a system-wide IMAP server;
> - use a chrooted environment.
I think the approach I took is closest to #3, but again, I didn't use an
actually chroot with a full OS install available on it. I just followed
the Dovecot wiki's instructions on running "rootless", i.e. as a
non-privileged user. While a chroot is a nice idea, I wonder if it's
really necessary or even useful. What's the payoff? Even in a chroot,
we still can't run on port 143 (for example) without having root privileges.
> In the long-term, we should be fine with any of the two latter
> possibilities, at least. The main consequence is that each solution
> should also provide the same API to easily check the results. I'm
> thinking about shell scripts, here. Something like:
> cooked.sh --number --new-mail # returns the number of new mail only
> # (since last checkpoint)
> cooked.sh --create-ckeckpoint
> cooked.sh --plain --all # return all emails as plain text
> cook.sh --new< mail.sample # add a new mail on server without any
> # IMAP client
I like this approach, and it feels cleaner and more flexible, but it
also does sound like a little more work. I am afraid we are going to
fall into the trap of "let's build a test harness", instead of "let's
use a test suite to test our real project" :)
> `tests-suites.conf # where testers configure the environment(s) they like
Meaning, "Please test against Gmail with these credentials", "Please
test against rootless Dovecot", and so on?
> `update-tests-suites.sh # fetch choosen environment, configuration
> | # sample files from an external repository
Does this mean, "download and install Dovecot"? I think that's outside
the scope of this test harness.
> `imap-server/ # where to store external repositories and
> | # high-level API scripts for cooking
> | |
> | `dovecot-vXYZ/ # configuration file samples, low-level
> | | # scripts for cooking, etc
> | |
> | `imapserver2-vXYZ/
So cook.sh would be a wrapper around simpler scripts that actually
implement the functionality (deliver, count mail, etc.)?
> It should not be hard work to achieve this. Again, I'm not thinking
> about something advanced. You could start a new public repository (which
> I could maintain in the longer run) for the server side configuration
> files and low-level scripts.
> What do you think?
I think it's a great idea but somewhat ambitious considering our limited
pool of manpower. Is it really likely that other projects will join in
and contribute? The perfect is the enemy of the good.
More information about the OfflineIMAP-project