Skip to content
Snippets Groups Projects
Commit 3c78e74f authored by Maël Madon's avatar Maël Madon
Browse files

doc: update test README (issue #3)

parent 1a0a5ac4
No related branches found
No related tags found
No related merge requests found
Pipeline #5199 passed
...@@ -91,17 +91,14 @@ path/to/batsim -p test/platforms/monocore/2machines.xml \ ...@@ -91,17 +91,14 @@ path/to/batsim -p test/platforms/monocore/2machines.xml \
After the execution, you should see the [Batsim outputs](https://batsim.readthedocs.io/en/latest/tuto-result-analysis/tuto.html) in the output directory. After the execution, you should see the [Batsim outputs](https://batsim.readthedocs.io/en/latest/tuto-result-analysis/tuto.html) in the output directory.
## Run tests ## Run the tests
Batmen has a set of integration tests, located in the `test` directory. To run all the tests: Batmen has a set of integration tests, located in the `test` directory. To run all the tests:
```bash ```bash
nix-shell -A test nix-shell -A test --command "pytest"
pytest
``` ```
For single tests: `pytest -k EXPRESSION` only runs the tests matching the given substring expression. For more information about the tests, read the [test README](test/README.md)
Open `test-out/test_report.html` in a browser to see the test results.
[Nix]: https://nixos.org/nix/ [Nix]: https://nixos.org/nix/
[Batsim]: https://batsim.org/ [Batsim]: https://batsim.org/
......
# Batmen tests # Batmen tests
### Extra dependencies ## Run the tests
Some tests use a tool from the external library `batmen-tools` developped separately. To update it to the latest version: We use [Nix] to enter an environment with all the dependencies.
```bash ### `batmen` manually compiled
cd test This will run the test suite with the version of `batmen` located in the folder `build/`
curl -O https://gitlab.irit.fr/sepia-pub/mael/batmen-tools/-/raw/main/distance_batsim_output.py
```
### Running tests
``` bash ``` bash
nix-shell ../default.nix -A test --command 'pytest' nix-shell ../default.nix -A test
# or just pytest, but you must prepare your env... pytest
``` ```
Optionally, use Batsim's binary cache to avoid recompiling many things (e.g., SimGrid). ### Recompile automatically
If you want Nix to recompile `batmen` before entering the environment:
``` bash ``` bash
nix-env -iA cachix -f https://cachix.org/api/v1/install # installs cachix nix-shell ../default.nix -A test_rebuild
cachix use batsim # add Batsim's cachix storage as a Nix remote cache pytest
``` ```
### How does it work? ### Browse the test outputs
Instead of reading the (long) outputs from the terminal, you can open `test-out/test_report.html` in a browser to see the test results.
### Run an individual test
`pytest -k EXPRESSION` only runs the tests matching the given substring expression.
For example, `pytest -k user` will only run the tests whose name contain "user".
## How does the testing work?
pytest interprets all the functions starting with `test_` as tests. For one of these functions:
0. nix-shell puts you into an environment where batsched, batsim, robin, redis, etc. are available (code in [default.nix]) 0. nix-shell puts you into an environment where batsched, batsim, robin, redis, etc. are available (code in [default.nix])
1. pytest generates combinations of test input (code in [conftest.py]) 1. pytest generates combinations of test input (code in [conftest.py])
2. for each combination of inputs: (code in [test_runner.py]) 2. for each combination of inputs:
1. pytest generates a [robin] input file 1. pytest generates a [robin] input file
2. pytest generates batsim and batsched input files if needed 2. pytest generates batsim and batmen input files if needed
3. pytest executes [robin] or [robintest] on the generated file 3. pytest executes [robin] or [robintest] on the generated file
### Running a specific test Finally, we have special tests (in the file [test_zexpected_output.py](./test_zexpected_output.py)) verifying that the output of the previous tests correspond to the expected output.
## Running a failed test
You can manually rerun a test with robin: You can manually rerun a test with robin:
``` ```
robin test-instances/FAILING-TEST.yaml robin test-instances/FAILING-TEST.yaml
...@@ -42,8 +52,17 @@ You can also run batsim and batsched in different terminals: ...@@ -42,8 +52,17 @@ You can also run batsim and batsched in different terminals:
./test-out/FAILING-TEST/cmd/sched.bash ./test-out/FAILING-TEST/cmd/sched.bash
``` ```
## Extra dependency
Some tests use a tool from the library `batmen-tools` developped separately. To do update it to the latest version:
```bash
cd batmen/test
curl -O https://gitlab.irit.fr/sepia-pub/mael/batmen-tools/-/raw/main/distance_batsim_output.py
```
[default.nix]: ../default.nix [default.nix]: ../default.nix
[conftest.py]: ./conftest.py [conftest.py]: ./conftest.py
[test_runner.py]: ./test_runner.py [test_runner.py]: ./test_runner.py
[robin]: https://framagit.org/batsim/batexpe [robin]: https://framagit.org/batsim/batexpe
[robintest]: https://framagit.org/batsim/batexpe [robintest]: https://framagit.org/batsim/batexpe
[Nix]: https://nixos.org/nix/
\ No newline at end of file
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment