After the execution, you should see the [Batsim outputs](https://batsim.readthedocs.io/en/latest/tuto-result-analysis/tuto.html) in the output directory.
After the execution, you should see the [Batsim outputs](https://batsim.readthedocs.io/en/latest/tuto-result-analysis/tuto.html) in the output directory.
## Run tests
## Run the tests
Batmen has a set of integration tests, located in the `test` directory. To run all the tests:
Batmen has a set of integration tests, located in the `test` directory. To run all the tests:
```bash
```bash
nix-shell -Atest
nix-shell -Atest--command"pytest"
pytest
```
```
For single tests: `pytest -k EXPRESSION` only runs the tests matching the given substring expression.
For more information about the tests, read the [test README](test/README.md)
Open `test-out/test_report.html` in a browser to see the test results.
cachix use batsim # add Batsim's cachix storage as a Nix remote cache
pytest
```
```
### How does it work?
### Browse the test outputs
Instead of reading the (long) outputs from the terminal, you can open `test-out/test_report.html` in a browser to see the test results.
### Run an individual test
`pytest -k EXPRESSION` only runs the tests matching the given substring expression.
For example, `pytest -k user` will only run the tests whose name contain "user".
## How does the testing work?
pytest interprets all the functions starting with `test_` as tests. For one of these functions:
0. nix-shell puts you into an environment where batsched, batsim, robin, redis, etc. are available (code in [default.nix])
0. nix-shell puts you into an environment where batsched, batsim, robin, redis, etc. are available (code in [default.nix])
1. pytest generates combinations of test input (code in [conftest.py])
1. pytest generates combinations of test input (code in [conftest.py])
2. for each combination of inputs: (code in [test_runner.py])
2. for each combination of inputs:
1. pytest generates a [robin] input file
1. pytest generates a [robin] input file
2. pytest generates batsim and batsched input files if needed
2. pytest generates batsim and batmen input files if needed
3. pytest executes [robin] or [robintest] on the generated file
3. pytest executes [robin] or [robintest] on the generated file
### Running a specific test
Finally, we have special tests (in the file [test_zexpected_output.py](./test_zexpected_output.py)) verifying that the output of the previous tests correspond to the expected output.
## Running a failed test
You can manually rerun a test with robin:
You can manually rerun a test with robin:
```
```
robin test-instances/FAILING-TEST.yaml
robin test-instances/FAILING-TEST.yaml
...
@@ -42,8 +52,17 @@ You can also run batsim and batsched in different terminals:
...
@@ -42,8 +52,17 @@ You can also run batsim and batsched in different terminals:
./test-out/FAILING-TEST/cmd/sched.bash
./test-out/FAILING-TEST/cmd/sched.bash
```
```
## Extra dependency
Some tests use a tool from the library `batmen-tools` developped separately. To do update it to the latest version: