@@ -98,9 +109,9 @@ The MD5 hashes have been computed by #link("https://www.gnu.org/software/coreuti
...
@@ -98,9 +109,9 @@ The MD5 hashes have been computed by #link("https://www.gnu.org/software/coreuti
= Getting Started Guide
= Getting Started Guide
All the software environments required to reproduce the analyses and experiments of article @lightpredenergy are open source and have been packaged with #link("https://nixos.org/", [Nix]).
All the software environments required to reproduce the analyses and experiments of article @lightpredenergy are open source and have been packaged with #link("https://nixos.org/", [Nix]).
Nix can build the *full* software stack needed for this experiment as long as source code remains available. As we also put our source code on #link("https://www.softwareheritage.org/")[Software Heritage] we hope that this artifact will have long-term longevity. For the sake of this artifact reviewers'qualityoflife,wehavesetupabinarycachewithprecompiledversionsofthesoftwareusedintheexperiments.
Nix can build the *full* software stack needed for this experiment as long as source code remains available. As we also put most of the source code needed by this artifact on #link("https://www.softwareheritage.org/")[Software Heritage] we hope that this artifact will have long-term longevity. For the sake of this artifact reviewers'qualityoflife,wehavesetupabinarycachewithprecompiledversionsofthesoftwareusedintheexperiments.
OursoftwareenvironmentslikelyworkonallplatformssupportedbyNix(Linuxon`i686`/`x86_64`/`aarch64`andMacOSon`x86_64`/`aarch64`asof2024-05-07)butwehaveonlytestedthemonLinuxon`x86_64`.Moreprecisely,wehaveusedthe#link("https://www.grid5000.fr/w/Grenoble:Hardware#dahu")[DahuGrid'5000 cluster] (Dell PowerEdge C6420, 2x Intel Xeon Gold 6130, 192 GiB of RAM) on the default operating system available on Grid'5000asof2024-05-07(Debian`5.10.209-2`usingLinuxkernel`5.10.0-28-amd64`).
OursoftwareenvironmentslikelyworkonallplatformssupportedbyNix(Linuxon`i686`/`x86_64`/`aarch64`andMacOSon`x86_64`/`aarch64`asof2024-05-07)butwehaveonlytestedthemonLinuxon`x86_64`.Moreprecisely,wehaveusedthe#link("https://www.grid5000.fr/w/Grenoble:Hardware#dahu")[DahuGrid'5000 cluster] (Dell PowerEdge C6420, 2x Intel Xeon Gold 6130, 192 GiB of RAM) on the default operating system available on Grid'5000asof2024-05-07(Debian`5.10.209-2`usingLinuxkernel`5.10.0-28-amd64`).
==InstallNix
==InstallNix
...
@@ -177,16 +188,13 @@ Please note that the way to enable flakes depend on whether you are on NixOS or
...
@@ -177,16 +188,13 @@ Please note that the way to enable flakes depend on whether you are on NixOS or
+ You can *check* the final analyses (code + plots) done in Article @lightpredenergy by reading the provided pre-rendered notebooks available on #link(zenodo-url)[Zenodo].
+ You can *reproduce* the *final analyses* by first downloading the provided aggregated results of the experiments from #link(zenodo-url)[Zenodo], and then by running the notebooks yourself.
This section downloads parts of the Marconi100 trace as archives from #link("https://gitlab.com/ecs-lab/exadata")[the ExaData Zenodo files], checks that the archives have the right content (via MD5 checksums), extracts the data needed by later stages of the pipeline (node power usage traces, jobs information traces), then finally removes unneeded extracted files and the downloaded archives.
#fullbox(footer:[#emph-overhead[Download+temporary disk: 254 Go.] Final disk: 928 Mo. #emph-overhead[Time: 00:40:00.]])[
#fullbox(footer:[#emph-overhead[Download+temporary disk: 254 Go.] Final disk: 928 Mo. #emph-overhead[Time: 00:40:00.]])[
```sh
```sh
...
@@ -313,7 +343,7 @@ This section downloads parts of the Marconi100 trace as archives from Zenodo, ch
...
@@ -313,7 +343,7 @@ This section downloads parts of the Marconi100 trace as archives from Zenodo, ch
The following command traverses all the Marconi100 power traces and counts how many times each node was at each power value.
The following command traverses all the Marconi100 power traces and counts how many times each node was at each power value.
The following commands generate workload parameters (_i.e._, when each workload should start and end), taking start points at random during the 2022 M100 trace.
Simulation instances are then generated from the workload parameters.
Required input files.
Required input files.
- `expe-sched/m100-platform.xml` (output of @sec-gen-sg-platform).
- `expe-sched/m100-platform.xml` (output of @sec-gen-sg-platform).
...
@@ -387,7 +418,7 @@ Required input files.
...
@@ -387,7 +418,7 @@ Required input files.
]
]
==== Merge job power predictions and jobs information into a single file
==== Merge job power predictions and jobs information into a single file
The job power predictions (outputs of @sec-job-power-pred, available on #link(zenodo-url)[Zenodo]) are two archives that we assume are on your disk in the `./user-power-predictions` directory.
These archives contain gzipped files for each user.
These archives contain gzipped files for each user.
To make things more convenient for the generation of simulation inputs, all the job power prediction files are merged into a single file with the following commands.
To make things more convenient for the generation of simulation inputs, all the job power prediction files are merged into a single file with the following commands.
Please note that all input files can be downloaded from #link(zenodo-url)[Zenodo] if you have not generated them yourself.
In particular to populate the `/tmp/wlds`directory you can *download file* `workloads.tar.xz` and then *extract it* into `/tmp/` via a command such as the following. `tar xf workloads.tar.xz --directory=/tmp/`
=== Analyze the simulation campaign results <sec-analyze-simu-campaign-results>
The following command runs a notebook that analyze the aggregated results of the simulation campaign, and outputs Figure 4 and Figure 5 of Article @lightpredenergy.
The following command runs a notebook that analyze the aggregated results of the simulation campaign, and outputs Figure 4 and Figure 5 of Article @lightpredenergy.