Reproduction prerequisites
Install Nix with the nix
command and Nix flakes
enabled.
TODO: explain how this is done currently and link to doc for later on
Reproduce M100 data analysis
Unless otherwise specified, all commands should be run from the root of this Git repository.
download M100 traces
TODO
extract M100 traces
TODO
aggregate 2022 power traces
filtering note
28249 measures about node 155 are odd: total_power=p0_power=p1_power=0 W. For information, the second lowest value observed on any node for total_power is 240 W. filtering: removed values with total_power=0. this concerns 28249 measures that are all about node=155
nix develop .#aggregate-power-m100-months --command m100-agg-power-months ./m100-data/ ./m100-data/22-agg_ 22-01 22-02 22-03 22-04 22-05 22-06 22-07 22-08 22-09
plot ecdf of power values
nix develop .#analyze-m100-agg-power --command Rscript ./scripts-r/visualize-m100-power-ecdf.R ./m100-data/22-agg_power_total.csv ./m100-data/viz-22-ecdf_power_total.pdf total
plot power distribution of each node (requires heavy computing)
nix develop .#analyze-m100-agg-power --command Rscript ./scripts-r/visualize-m100-power-distrib.R ./m100-data/22-agg_power_total.csv ./m100-data/viz-22-distrib_power_total.pdf total
plot power ecdf of each node
nix develop .#analyze-m100-agg-power --command Rscript ./scripts-r/visualize-m100-power-ecdf.R ./m100-data/22-agg_power_total.csv /tmp/meh.pdf
compute quantiles on aggregated 2022 power traces
nix develop .#analyze-m100-agg-power --command Rscript ./scripts-r/m100-per-machine-model.R ./m100-data/22-agg_power_total.csv ./m100-data/22-powermodel_total.csv
generate workload
power estimation of each job
This assumes that you have a dump from Danilo scripts :/.
nix develop .#use-python-scripts --command m100-agg-danilo-estimations danilo-dump ./m100-data/22-job-power-estimations.csv
put all useful job information into a single file
nix develop .#use-python-scripts --command m100-agg-jobs-info ./m100-data/ ./m100-data/22-jobs.csv 22-01 22-02 22-03 22-04 22-05 22-06 22-07 22-08 22-09
merge power estimations with job informations into a single file
nix develop .#use-python-scripts --command m100-join-usable-jobs-info ./m100-data/22-job-power-estimations.csv ./m100-data/22-jobs.csv ./m100-data/22-jobs-with-prediction.csv