Batmen
Batmen is a plugin to the scientific simulator Batsim. It allows the simulation of users of large-scale distributed systems. Batmen is originally a fork of batsched.
How does it work?
What differs from the original version of batsched?
New paradigm: we implemented a "cloud broker" allowing to model the interaction with the users. This cloud broker is able to dynamically register new jobs to sumbit on top of the static workload during the scheduling. This way we can represent different categories of users and model their behavior in response to feedback.
For now this paradigm is supported by all the schedulers inheriting from the class DynScheduler
i.e. all the custom schedulers located in folder src/scheds
. It uses the modules in the folder in folder src/broker
:
- dynscheduler: superclass managing the interaction with the broker from which the scheduling algorithms inherit
- broker: implement the broker itself, submitting the jobs on behalf of the users
- user: an individual user, submitting her jobs to the broker
New algorithms implemented (in folder src/scheds
):
- multicore_filler: one job at a time on only one multicore host. Developped for testing the multicore management in batsim/simgrid
- bin_packing: the scheduler packs the jobs on the least number of multicore hosts. Each job runs on only one host but can take several cores
- bin_packing_energy: same as bin_packing but switches of the hosts as soon as they are idle and power them on again when needed
Install
With Nix package manager. At the root of the project:
nix-shell -A batmen # enter a Nix environment with all dependencies managed
meson build # generate a build directory
ninja -C build # compile the project into the build directory
Manually: TDB...
Run a small instance
Follow the steps to install Batmen. The executable is at the location build/batmen
. We suppose that you also have Batsim installed on your machine (follow the installation guide here).
To run a simulation, we will need two terminals:
- one (Batmen) to simulate the users and the scheduler, and
- one (Batsim) to simulate the underlying infrastructure
Create the output directory:
mkdir -p out/first_simu
Launch Batmen with
- the scheduler FCFS
- one simulated user (described in the user description file
test/schedconf/routine_greedy_simple.json
)
build/batmen -v fcfs \
--variant_options_filepath test/schedconf/routine_greedy_simple.json
Launch Batsim with
- the platform file
test/platforms/monocore/2machines.xml
- an empty workload file
test/workloads/static/empty.json
(jobs are submitted dynamically by the simulated user) - the output directory
- all the options required for Batmen to work properly (see their description in Batsim CLI)
path/to/batsim -p test/platforms/monocore/2machines.xml \
-w test/workloads/static/empty.json \
-e out/first_simu/ \
--energy --enable-compute-sharing \
--enable-dynamic-jobs --acknowledge-dynamic-jobs \
--enable-profile-reuse
After the execution, you should see the Batsim outputs in the output directory.
Run tests
Batmen has a set of integration tests, located in the test
directory. To run all the tests:
nix-shell -A test
pytest
For single tests: pytest -k EXPRESSION
only runs the tests matching the given substring expression.
Open test-out/test_report.html
in a browser to see the test results.