[sane-devel] Proposal for integration tests
rene at exactcode.de
Tue Feb 26 10:31:29 GMT 2019
On 14 Feb 2019, at 13:17, Povilas Kanapickas <povilas at radix.lt> wrote:
> I would like to propose adding integration tests to guard against accidental regressions especially in rarer scanners that few SANE developers have access to. We could run these tests on each pull request on gitlab without any scanner hardware.
> Given limited development resources and large variety of hardware that SANE supports, a major requirement of the tests would be that they are easy to create and use, otherwise no one will bother to deal with them.
> Therefore, I would like to propose a simple way of testing SANE by recording low-level communication with the scanner, replaying it at the test time and checking whether SANE behaves in an equivalent way.
> Additionally, I would like to propose that we distribute the recording functionality with release SANE builds. This way we could collect recordings from our users for scanners that no SANE developers have access to and then add to the test database.
> Finally, I expect that there will be a need for tools for simplify and automate the process of working with the recorded data. I would like to propose that we use python as a implementation language for any of such tools.
> Distribution of recording functionality together with release versions of SANE would require that we are able to change the communication backend on-demand. The best place to do that it seems to be at sanei layer. For example, in case of sanei_usb.h, I propose that instead of calling functions from that header we pass a pointer to a struct containing function pointers that match the current functionality one-to-one. Then, depending on startup options, we could create a USB backend that supports either of the following options:
> - calls directly to sanei_usb.c.
> - calls directly to sanei_usb.c, but records all parameters and responses to a file.
> - reads expected parameters and responses from a file and compares them with the current parameters. An error is recorded if anything does not match.
> If we agree with the above proposal, the use of the new, dynamic APIs would be completely opt-in. For example, the current API of sanei_usb.h would be left unmodified. Once the viability is proved with a single backend we could gradually introduce it to more backends.
> What do you think about this?
Sounds like a good idea, likely the devil is in the details, however you probably do not need to add a new “dynamic API”, but certainly can re-define the current sanei_ functions. You could even leave the current implementation as it, and add either add a second, new, conditionally compiled files with test record and reply functions, and/or use preprocessor macros to redefine the function names globally to ones with _test or moniker or whatever you see fit.
ExactCODE GmbH, Lietzenburger Str. 42, DE-10789 Berlin
http://exactcode.com | http://exactscan.com | http://ocrkit.com | http://t2-project.org | http://rene.rebe.de
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the sane-devel