[sane-devel] Proposal for integration tests
povilas at radix.lt
Thu Feb 14 12:17:14 GMT 2019
I would like to propose adding integration tests to guard against
accidental regressions especially in rarer scanners that few SANE
developers have access to. We could run these tests on each pull request
on gitlab without any scanner hardware.
Given limited development resources and large variety of hardware that
SANE supports, a major requirement of the tests would be that they are
easy to create and use, otherwise no one will bother to deal with them.
Therefore, I would like to propose a simple way of testing SANE by
recording low-level communication with the scanner, replaying it at the
test time and checking whether SANE behaves in an equivalent way.
Additionally, I would like to propose that we distribute the recording
functionality with release SANE builds. This way we could collect
recordings from our users for scanners that no SANE developers have
access to and then add to the test database.
Finally, I expect that there will be a need for tools for simplify and
automate the process of working with the recorded data. I would like to
propose that we use python as a implementation language for any of such
Distribution of recording functionality together with release versions
of SANE would require that we are able to change the communication
backend on-demand. The best place to do that it seems to be at sanei
layer. For example, in case of sanei_usb.h, I propose that instead of
calling functions from that header we pass a pointer to a struct
containing function pointers that match the current functionality
one-to-one. Then, depending on startup options, we could create a USB
backend that supports either of the following options:
- calls directly to sanei_usb.c.
- calls directly to sanei_usb.c, but records all parameters and
responses to a file.
- reads expected parameters and responses from a file and compares
them with the current parameters. An error is recorded if anything does
If we agree with the above proposal, the use of the new, dynamic APIs
would be completely opt-in. For example, the current API of sanei_usb.h
would be left unmodified. Once the viability is proved with a single
backend we could gradually introduce it to more backends.
What do you think about this?
More information about the sane-devel