RM4ES Practicals

This repository contains the material for the practical session of course RM4ES given in the Spring semester of 2024.

Session 1 : getting started

Objective for this session:

• setting up all the simulation environment
• understanding the workload representation in Batsim
• having a first scheduling algorithm running

Install

Ensure that you are in your home directory (run cd).

Clone this repository:

git clone https://gitlab.irit.fr/sepia-pub/mael/RM4ES-practicals.git

Install Nix, that will manage for you all the dependencies you need for the course:

# download and install nix-user-chroot, a hack to install nix without root privilege
curl -L  https://github.com/nix-community/nix-user-chroot/releases/download/1.2.2/nix-user-chroot-bin-1.2.2-x86_64-unknown-linux-musl --output nix-user-chroot
# give exec rights
chmod u+x nix-user-chroot
# download and install nix
./nix-user-chroot ~/.nix bash -c "curl -L https://nixos.org/nix/install | bash"
# enable cachix
mkdir -p ~/.config/nix && cp ~/RM4ES-practicals/util/nix.conf ~/.config/nix
# add a line to your bashrc
echo 'source ~/.nix-profile/etc/profile.d/nix.sh' >> ~/.bashrc

IMPORTANT: Now, each time you want to test your schedulers, you need to run, from the repository RM4ES-practicals:

# enter the nix-user-chroot shell
~/nix-user-chroot ~/.nix bash
# enter the nix shell defined in file default.nix, with all dependencies managed.
# can take a few minute the first time you run it, to download everything. 
nix-shell -A expe

Introduction to Batsim

The resource and job management simulator Batsim, that we are going to use during these practicals, needs three inputs:

• a platform: description of the IT infrastructure;
• a workload: list of jobs that are going to arrive and their characteristics;
• a scheduling algorithm: the "brain" of the system, taking decision on when and where to execute the jobs.

The platform is represented by an XML file. You have a simple example (monomachine platform) in the file expe/1machine.xml. This is enough for now! We will come back to it in more detail in session 3.

The workload is represented by a JSON file (see expe/2jobs.json):

{
    "jobs": [
        {"id": "1", "profile": "3UT", "res": 1, "walltime":10, "subtime": 0},
        {"id": "2", "profile": "4UT", "res": 1, "walltime":6, "subtime": 0},
    ],
    "profiles": {
        "3UT": {"delay": 3,"type": "delay"},
        "4UT": {"delay": 4,"type": "delay"}
    }
}

The meaning of each field in the JSON is explained in Batsim workload documentation.

The scheduling algorithm has to be developed separately, and communicates with Batsim through the messages defined in Batsim protocol. In this tutorial, we will develop our schedulers in Python thanks to pybatsim (version 3.2.0). The scheduler implementations will be located in the folder sched/.

Exercise 1

Represent the workload used previously in this course (exercise 1 from the exercise document) in Batsim JSON format. You will put it in a new file named expe/ex1.json. We will use the field "walltime" to store the "deadline".

Exercise 2

TODO run a simulation with pre-defined scheduler rejector. Look at the outputs...

Exercise 3

TODO develop a monomachine FCFS scheduler and run it on the inputs from ex1. Visualize the output thanks to evalys.

Session 2: monomachine schedulers

• séance 2 : monomachine + d'autres algos : RMS, EDF (3 cas: infra sous-dimensionnée / normalement dim / sur-dim, remarquer que sur du surdimensioné, l'algo change rien) => example simple avec 1 grosse tâche + 1 petite tâche avec deadline, en EDF sans preamption ça passe pas.

Session 3: multimachine schedulers

• séance 3 : multimachine hétérogènes en vitesse avec deadlines

Session 4: energy constraint

• séacnz 4 : EDF avec contrainte énergétique (powercap ou energycap)

Session 5: preamptive scheduling

• séance 5 : implémenter la préamption

Open bugs

• mismatch python version if I want to use batsim and evalys. Either python util/plot.py out/_jobs.csv or batsim -p expe/1machine.xml -w expe/ex1.json -e out/ make an error
• no working poetry in nix-shell
• will that work in the lab's machines?

Status rn

We can start a small expe!

On one shell:

ns -A dev
pybatsim newSched

On the other shell:

ns -A expe
batsim -p expe/1machine.xml -w expe/2jobs.json -e out/