Batmen

Batmen is a plugin to the scientific simulator Batsim. It allows to simulate users of large-scale distributed systems. Batmen is originally a fork of batsched.

How does it work?

A simulation with Batsim consists of two processes:

• Batsim itself, in charge of reading an input workload and simulating the platform (job arrival, job termination, energy consumed...)
• a scheduler, in charge of making the scheduling decisions. The scheduler communicates with Batsim via a ZeroMQ socket through messages defined in Batsim protocol.

In Batmen, we added a layer: the simulation of users.

User simulation

The interaction with users happens through a "broker". The broker manages a pool of simulated users and dynamically register new jobs to sumbit. These jobs come on top of the input workload read by Batsim.

Implementation, in the folder src/users:

• dynscheduler: superclass managing the interaction with the broker from which the scheduling algorithms inherit
• broker: implement the broker itself, submitting the jobs on behalf of the users
• user: an individual user, submitting her jobs to the broker

Types of user

• feedback users
• replay users
• modelled users

Available schedulers

Dynamic submission is supported by all the schedulers inheriting from the class DynScheduler i.e., all the schedulers located in the folder src/scheds. Some of them support machines with more than one core while others are only for monocore machines.

• monocore schedulers:
  • fcfs:
  • easy-bf:
• multicore schedulers:
  • bin_packing: the scheduler packs the jobs on the least number of multicore machines. Each job runs on only one host but can take several cores
  • bin_packing_energy: same as bin_packing, but saves power by switching off the machines as soon as they are idle and powering them on again when needed
  • multicore_filler: for test purposes. Schedules only one job at a time on only one multicore host.

Install

With Nix package manager. At the root of the project:

nix-shell -A batmen # enter a Nix environment with all dependencies managed
meson build # generate a build directory
ninja -C build # compile the project into the build directory

Manually: TDB...

Run a small instance

Follow the steps to install Batmen. The executable is at the location build/batmen. We suppose that you also have Batsim installed on your machine (follow the installation guide here).

To run a simulation, we will need two terminals:

• one (Batmen) to simulate the users and the scheduler, and
• one (Batsim) to simulate the underlying infrastructure

Create the output directory:

mkdir -p out/first_simu

Launch Batmen with

• the scheduler FCFS
• one simulated user (described in the user description file test/schedconf/routine_greedy_simple.json)

build/batmen -v fcfs \
    --variant_options_filepath test/schedconf/routine_greedy_simple.json

Launch Batsim with

• the platform file test/platforms/monocore/2machines.xml
• an empty workload file test/workloads/static/empty.json (jobs are submitted dynamically by the simulated user)
• the output directory
• all the options required for Batmen to work properly (see their description in Batsim CLI)

path/to/batsim -p test/platforms/monocore/2machines.xml \
    -w test/workloads/static/empty.json \
    -e out/first_simu/ \
    --energy --enable-compute-sharing \
    --enable-dynamic-jobs --acknowledge-dynamic-jobs \
    --enable-profile-reuse

After the execution, you should see the Batsim outputs in the output directory.

Run tests

Batmen has a set of integration tests, located in the test directory. To run all the tests:

nix-shell -A test
pytest

For single tests: pytest -k EXPRESSION only runs the tests matching the given substring expression.

Open test-out/test_report.html in a browser to see the test results.