Batmen tests

Run the tests

We use Nix to enter an environment with all the dependencies.

batmen manually compiled

This will run the test suite with the version of batmen located in the folder build/

nix-shell ../default.nix -A test
pytest

Recompile automatically

If you want Nix to recompile batmen before entering the environment:

nix-shell ../default.nix -A test_rebuild
pytest

Browse the test outputs

Instead of reading the (long) outputs from the terminal, you can open test-out/test_report.html in a browser to see the test results.

Run an individual test

pytest -k EXPRESSION only runs the tests matching the given substring expression.

For example, pytest -k user will only run the tests whose name contain "user".

How does the testing work?

pytest interprets all the functions starting with test_ as tests. For one of these functions:

0. nix-shell puts you into an environment where batsched, batsim, robin, redis, etc. are available (code in default.nix)
1. pytest generates combinations of test input (code in conftest.py)
2. for each combination of inputs:
   1. pytest generates a robin input file
   2. pytest generates batsim and batmen input files if needed
   3. pytest executes robin or robintest on the generated file

Finally, we have special tests (in the file test_zexpected_output.py) verifying that the output of the previous tests correspond to the expected output.

Debugging a failed test

You can manually rerun a test with robin:

robin test-instances/FAILING-TEST.yaml

You can also run batsim and batsched in different terminals:

# feel free to hack these files — e.g., prepend commands with gdb, valgrind...
./test-out/FAILING-TEST/cmd/batsim.bash
./test-out/FAILING-TEST/cmd/sched.bash

Extra dependency

Some tests use a tool from the library batmen-tools developped separately. To do update it to the latest version:

cd batmen/test
curl -O https://gitlab.irit.fr/sepia-pub/mael/batmen-tools/-/raw/main/distance_batsim_output.py