Skip to content
Snippets Groups Projects
Select Git revision
  • main default protected
  • fgcs-submission
  • bug-jupyter
  • expe_half_infra_uncleaned
  • expe_half_infra
  • maelPhD
  • fgcs-2024
  • fgcs-submission
8 results
Compare
user avatar
add article title in README + update README
Maël Madon authored
8844a771
History
user avatar 8844a771
History
Name Last commit Last update
platform fork and clean repo
workload fork and clean repo
.gitignore rename delta -> additional lateness , in graphs
KTH.ipynb rename delta -> additional lateness , in graphs
README.md add article title in README + update README
SDSC.ipynb rename delta -> additional lateness , in graphs
default.nix nix: update kapack + use batsim v4.2 + add jupyter in nix
sched_metrics.ipynb delta per user figure for paper, stat table delta per user, suffix for fig path
README.md

Compare traditional replay and replay with feedback

This repository contains all the necessary material to reproduce the experiments and the graphs presented in the article "Replay with Feedback: How does the performance of HPC system impact user submission behavior?", submitted to the journal FGCS.

Contents

  • default.nix: Nix file describing the software dependencies and their specific versions
  • workload/: folder storing the workload files
  • platform/: folder containing the platform files
  • KTH.ipynb and SDSC.ipynb: Jupyter notebook running and analysing the experiments

The two notebooks are self-contained. They download the input data from the Parallel Workload Archive into the workload/ directory, run the experiments presented in the article, do the data analysis and plot the graphs. Both files contain essentially the same treatments, only the input workload changes.

Note: it shouldn't be very hard to run the experiments with another workload from Parallel Workload Archive. It requires adapting the variables set in the first cell of the notebook and creating a platform corresponding to the chosen workload. However, we use FCFS and EASY backfilling as scheduling algorithms, which might not correspond to the original scheduler used in the real infrastructure.

Steps to reproduce

Time needed to reproduce the experiments (on a i5-1135 processor):

  • downloading and compiling dependencies with Nix: ~10min
  • KTH.ipynb: 17min11
  • SDSC.ipynb: 41min53

Disk space: 6.2GB for experiment files and 3.4GB for dependencies (in /nix/store)

With Nix package manager

Clone the repository and go to the specific version (you can also download the version tagged fgcs-submission from gitlab web interface):

git clone git@gitlab.irit.fr:sepia-pub/open-science/expe-replay-feedback.git
cd expe-replay-feedback
git fetch --all --tags && git checkout tags/fgcs-submission

Enter a Nix-shell, where all the dependencies you might not have on your machine are carefully managed:

nix-shell -A exp_env

Start a jupyter notebook (or any IDE providing .ipynb support):

jupyter notebook

Open KTH.ipynb or SDSC.ipynb and run all the cells, in order.

Without Nix (not recommended)

You will need the following dependencies: