[Nut-upsdev] NUT QA regression testing script (was: Upgraded Buildbot)

[Arnaud Quette]

Hey Charles and the list,

2011/11/13 Arnaud Quette <aquette.dev at gmail.com>:
>
> 2011/11/12 Charles Lepple <clepple at gmail.com>
>>
>> On Sat, Nov 12, 2011 at 9:41 AM, Arnaud Quette <aquette.dev at gmail.com>
>> wrote:
>> > since I'm working on improving the Ubuntu regression testing suite [1],
>> > which can also serve for general QA & integration purposes, we will have
>> > to
>> > discuss on its possible integration part of the "make check" and
>> > buildbot
>> > runs.
>>
>> Not a problem. I'd like to keep the commands the same across all
>> branches, though, so maybe we should put some dummy "make check" code
>> into the trunk?
>
> I'll have to look since it's still very ubuntu'ish.
> but apart from python requirements (unittest, subprocess, sys, os, time,
> testlib), only the paths and init methods remains to be adapted. So not that
> much, and the standard @python test should be enough with the deps being
> satisfied.
> I'd like to see the test plan and test cases formalized (in Asciidoc format)
> and applied as much as possible using this. In the end, we'll only have the
> actual hardware support remaining to be tested.

I've completed yesterday evening a first revamped version of this
regression testing script, still for Debian and Ubuntu only ATM.

It now tests the major NUT features, including:
- daemons startup (driver, upsd and upsmon)
- upsd reachability, through IPv4 and IPv6 (that would have saved us a
publication in a not so far away past)
- upsc and upsrw execution (upscmd still needs a dummy-ups counterpart)
- upsmon notifications (EXEC flag only ATM),
- upsmon simulated shutdown, on a single power source:
Ie, not an actual system shutdown, but only the implied mechanisms,
through the existence of POWERDOWNFLAG and the execution of
SHUTDOWNCMD

I have still a good TODO list on this.
All these information, including how to run the script, are located in
the script header.

I've attached this script, for reference, and it's also available as a
patch for Ubuntu QRT (QA Regression Testing):
https://bugs.launchpad.net/qa-regression-testing/+bug/894476

Part of the future improvements, I'm willing to tie this script to the
NUT test plan and test cases, at least the part that can be automated,
and to integrate it part of the 'check' target. Once done, we'll be
able to run it part of our Buildbot process...

Feedback and comments welcome, as usual...

cheers,
Arnaud
-- 
Linux / Unix Expert R&D - Eaton - http://powerquality.eaton.com
Network UPS Tools (NUT) Project Leader - http://www.networkupstools.org/
Debian Developer - http://www.debian.org
Free Software Developer - http://arnaud.quette.free.fr/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test-nut.py.gz
Type: application/x-gzip
Size: 4010 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/nut-upsdev/attachments/20111125/a32722d5/attachment.bin>