pytest-testmon is a pytest plugin which selects and executes only tests you need to run. It does this by collecting dependencies between tests and all executed code (internally using Coverage.py) and comparing the dependencies against changes. testmon updates its database on each test execution, so it works independently of version control.
# install pip install pytest-testmon # build the dependency database and save it to .testmondata pytest --testmon # change some of your code (with test coverage) # only run tests affected by the changes pytest --testmon
Before testmon can select the right subset of your tests you have to run all your tests with the –testmon option.
If you want to read more about how testmon collects and compares changes we wrote about it here.
If you want to run your tests on every file save we recommend using testmon with pytest-watch.
Testmon always re-executes tests which failed last time.
| option | help |
|---|---|
--testmon | Select tests affected by changes (based on previously collected data) and collect + write new data (.testmondata file). Either collection or selection might be deactivated (sometimes automatically). See below. |
--testmon-noselect | Run testmon but deactivate selection, so all tests selected by other means will be collected and executed. Forced if you use -k, -l, -lf, test_file.py::test_name |
--testmon-nocollect | Run testmon but deactivate the collection and writing of testmon data. Forced if you run under debugger or coverage. |
--testmon-forceselect | Run testmon and select only tests affected by changes and satisfying pytest selectors at the same time. |
--no-testmon | Turn off (even if activated from config by default). Forced if neither read nor write is possible (debugger plus test selector). |
--testmon-env=<STRING> | This allows you to have separate coverage data within one .testmondata file, e.g. when using the same source code serving different endpoints or Django settings. |
--tmnet | This is used for internal beta. Please don’t use. You can go to https://www.testmon.net/ to register. |
If you need to keep .testmondata anywhere else than in pytest rootdir please use TESTMON_DATAFILE environment variable.
Persistent configuration for testmon can be held in pytest config section like this (e.g. pytest.ini):
[pytest] addopts = --testmon # you can make --testmon a default if you want # If you want to separate different environments running the same sources.
Testmon selects too many tests for execution: Depending you your change it most likely is by design. If you changed a method parameter name, you effectively changed the whole hierarchy parameter -> method -> class -> module, so any test using anything from that module will be re-executed.
Tests are failing when running under testmon: It’s quite unlikely testmon influenced the execution of the test itself. However set of deselected and executed tests with testmon is highly variable, which means testmon is likely to expose undesired test dependencies. Please fix your test suite. We wrote down a couple of tips and tricks on how to tackle this challenge here.
You can also try if your test is influenced under pytest-cov (coverage) without testmon. For reporting a bug a repository with description of unexpected behavior is the best, but please don’t hesitate to report even if your project is closed source. We’ll try to fix it!
There are many things influencing the outcome of a test. testmon keeps track of these dependencies:
Testmon doesn’t know and doesn’t track changes of these categories:
Later versions can implement a mechanism to help reacting to these changes.