made a more precise README and small fixes on the notebook files

0_prepare_workload.ipynb

@@ -17,7 +17,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# Download the workload\n",
+    "# Download the workload (548.3 MB unzipped)\n",
     "!curl https://www.cs.huji.ac.il/labs/parallel/workload/l_metacentrum2/METACENTRUM-2013-3.swf.gz \\\n",
     "    --output workload/METACENTRUM-2013-3.swf.gz "
    ]
@@ -48,7 +48,7 @@
     "- we remove from the workload the jobs with an **execution time greater than one day**\n",
     "- we remove from the workload the jobs with a **number of requested cores greater than 16**\n",
     "\n",
-    "To do so, we use a home made SWF parser. "
+    "To do so, we use a the home-made SWF parser `swf_moulinette.py`:"
    ]
   },
   {
@@ -204,7 +204,7 @@
     "## Platform\n",
     "According to the system specifications given in the [corresponding page in Parallel Workload Archive](https://www.cs.huji.ac.il/labs/parallel/workload/l_metacentrum2/index.html): from June 1st 2014 to Nov 30th 2014 there is no change in the platform for the clusters considered in our study (<16 cores). There is a total of **6304 cores**.(1)\n",
     "\n",
-    "We build a platform file adapted to the remaining workload. We see above that the second selection cut 73.7\\% of core-hours from the original workload. We choose to make an homogeneous with 16-core nodes. To have a coherent number of nodes, we count:\n",
+    "We build a platform file adapted to the remaining workload. We see above that the second selection cuts 73.7\\% of core-hours from the original workload. We choose to make an homogeneous cluster with 16-core nodes. To have a coherent number of nodes, we count:\n",
     "\n",
     "$\\#nodes = \\frac{\\#cores_{total} * \\%kept_{core.hour}}{\\#corePerNode} = 6304 * .263 / 16 = 104$\n",
     "\n",
@@ -217,19 +217,11 @@
     "\n",
     "(1) clusters decomissionned before or comissionned after the 6-month period have been removed: $8+480+160+1792+256+576+88+416+108+168+752+112+588+48+152+160+192+24+224 = 6304$"
    ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "north-meeting",
-   "metadata": {},
-   "outputs": [],
-   "source": []
   }
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
    "name": "python3"
   },
@@ -243,7 +235,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.8.9"
+   "version": "3.9.6"
   }
  },
  "nbformat": 4,

README.md

-# Exploration of different user levers for demand response in data centers
-## List of experiments
-- TODO
+# Characterization of different user behaviors for demand response in data centers
+This repository contains the scripts and files needed to reproduce the experiments presented in the paper "Characterization of different user behaviors for demand response in data centers" submitted to [Euro-Par conference 2022](https://2022.euro-par.org/).
 
-## Install
-Clone the repository
+## Description of the main files
+- `0_prepare_workload.ipynb`: Jupyter notebook downloading and preparing the workload trace used in the experiments
+- `campaign2.py`: Python script preparing and launching in parallel the 105 experiments. Each experiment corresponds to one instance of `instance2.py`.
+- `analyse_campaign2.ipynb`: Jupyter notebook analysing the results and plotting the graphs shown in the article
 
-```bash
-git clone https://gitlab.irit.fr/sepia-pub/open-science/demand-response-user.git
-```
+(campaign1 is another set of experiements not discussed in the article. We kept it here but beware that some path might be broken...)
 
-For reproductibility, all the dependancies for these experiments and their version are managed with `nix` package manager. To install `nix`: 
+## Steps to reproduce
+You will need ~10GB disk space for the inputs, outputs and dependencies.
+
+### 1. Install
+For the sake reproductibility, all the dependancies for these experiments and their version (release tag or commit number) are managed with the package manager Nix. If you don't have it on your machine, the following command should install it. Otherwise, please refer to [their documentation](https://nixos.org/download.html).
 
 ```bash
 curl -L https://nixos.org/nix/install | sh
 ```
 
 The main software used (and configured in the file `default.nix`) are:
-- [batsim](https://batsim.org/) for the infrastructure simulation
-- [batmen](https://gitlab.irit.fr/sepia-pub/mael/batmen): our set of schedulers for batsim and plugin to simulate users
-- python3, .. . TODO for the data analysis
-
-## Start the simulation environment
-TODO
-``` bash
-nix-shell --pure ./env_nix.nix --show-trace #start the shell
-jupyter notebook                            #open jupyter notebook
+- [Batsim](https://batsim.org/) and [SimGrid](https://simgrid.org/) for the infrastructure simulation
+- [Batmen](https://gitlab.irit.fr/sepia-pub/mael/batmen): our set of schedulers for batsim and plugin to simulate users
+- python3, pandas, jupyter, matplotlib etc. for the data analysis
+
+Enter a shell with all dependencies managed. This will take some time to download and compile the first time you launch it, but then all the environment is cached for future use.
+```bash
+nix-shell -A exp_env
 ```
+
+### 2. Prepare input workload
+Inside the nix shell, start a notebook and follow the steps presented in `0_prepare_workload.ipynb`:
+```bash
+jupyter notebook 0_prepare_workload.ipynb
+```
+
+### 3. Launch the campaign
+Still inside the nix shell, launch the python script. It took about 2 hours for us to execute in parallel on a 16-core Intel Xeon E5-2630 v3 machine.
+```bash
+python3 campaign2.py
+```
+
+### 4. Analyse the results
+```bash
+jupyter notebook analyse_campaign2.ipynb
+```
\ No newline at end of file

default.nix

@@ -30,13 +30,11 @@ let self = rec {
   my_batsched = kapack.batsched.overrideAttrs (attr: rec {
     name = "batmen";
     version = "b63e95fad46962572b1f1d6db5bda979ca20dae0"; # commit "clean nix file and update README"
-    # version = "bab23fa5aa9c117d20fa74469147c1ce5fe58160";
     src = kapack.pkgs.fetchgit rec {
       url = "https://gitlab.irit.fr/sepia-pub/mael/batmen.git";
-      # url = "https://framagit.org/batsim/batsched";
       rev = version;
       sha256 = "1q2s1f68j4wd8vzzzirpjp7851l342hq6dvd9ky5g62zdps5mqlj";
-      # sha256 = "0r8qcvhy6ivx6flzl2lhxi1farkqp4w5w5635v83ckj7cc721njs";
+
     };
   });