[Freedombox-discuss] Heavy tests
Jonas Smedegaard
jonas at jones.dk
Sat Sep 11 08:54:30 BST 2021
Quoting Sunil Mohan Adapa (2021-09-11 04:07:41)
> On 10/09/21 4:02 pm, Fioddor Superconcentrado wrote:
> > I've coded some tests that result too heavy (slow) for unit testing.
> > But I think they still provide value in specific cases.
> >
> > So I've thought of renaming the file as *heavy_test_*.py*. This will
> > make default calls skip them but they can still be run explicitly
> > whenever the developer/tester wants to extend the coverage or the
> > reliability of the tests.
> >
> > I'd like to know what other developers/testers think of this
> > approach.
> >
> > If we do this for other modules and components we might create a
> > subfolder 'heavy_tests' or similar to collect separate them from
> > regular *tests_*.py* files.
> >
>
> It might be preferable to use pytest's "marks" to mark heavy tests
> (see WordPress' functional tests for example on how to mark). Then, we
> can introduce a '--include-heavy' option to include these tests (see
> /conftest.py for example on how --include-functional is done). This
> method gives us the flexibility organize tests via modules as we
> currently do and easily mark them as heavy to avoid them during
> regular runs.
I recommend to reuse the mechanism invented by Perl modules but in no
way specific to that: Check for environment variable EXTENDED_TESTING
Here is a concrete example of implementing EXTENDED_TESTING hinting in a
Python package:
https://salsa.debian.org/debian/python-aiortc/-/blob/debian/master/debian/patches/1001_skip_network_tests.patch
And here an example of how it can later be used:
https://salsa.debian.org/debian/python-aiortc/-/blob/debian/master/debian/tests/control
Here is an example of proposing the mechanism to an upstream:
https://github.com/etesync/etebase-py/issues/19
Her is the formal definition for the mechanism:
https://github.com/Perl-Toolchain-Gang/toolchain-site/blob/master/lancaster-consensus.md
- Jonas
--
* Jonas Smedegaard - idealist & Internet-arkitekt
* Tlf.: +45 40843136 Website: http://dr.jones.dk/
[x] quote me freely [ ] ask before reusing [ ] keep private
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: signature
URL: <http://alioth-lists.debian.net/pipermail/freedombox-discuss/attachments/20210911/d92d1ffe/attachment.sig>
More information about the Freedombox-discuss
mailing list