README.md

FedEator_Results_analysis

Unzip the folders:

unzip Log.zip #original data
unzip Data_analysis.zip #processed data (output from all steps below)

Raw results of experiment are stored in /Log/Flower_campaign. There are 4 instances from our experiment.

Log/Flower_campaign
├── Flower_instance_fedAvg_cifar10_epoch1
├── Flower_instance_fedAvg_cifar10_epoch2
├── Flower_instance_fedAvg2Clients_cifar10_epoch1
├── Flower_instance_fedAvg2Clients_cifar10_epoch2
...

In each instance, the structure of folder is same:

Log/Flower_campaign
├── Flower_instance_fedAvg_cifar10_epoch1: each instance folder
│   ├── Expetator
|   |   ├── config_instance_1.json: meta data of instance
│   ├── Expetator_<host_info>_<timestamp>_mojitos: mojitos outputs
│   ├── Expetator_<host_info>_<timestamp>_power: wattmetter outputs
│   ├── Expetator_<host_info>_<timestamp>: measurement log
│   ├── Flwr_<timestamp>: each flower folder log from each setting/test in instance
│   │   ├── Client_<ip>
│   │   ├── Client_<ip>
│   │   ├── Server_<ip>
│   │   ├── training_results_<instance_name>_<time>.csv
...

For example: Flower_instance_fedAvg_cifar10_epoch1

Log/Flower_campaign/Flower_instance_fedAvg_cifar10_epoch1
├── Expetator
│   └── config_instance_1.json
├── Expetator_taurus-1.lyon.grid5000.fr_1740648134
├── Expetator_taurus-1.lyon.grid5000.fr_1740648134_mojitos 
│  < each file is data of each node * 4 nodes * 2 times repeat * 4 CPU freq setting -> 32 files>
│   ├── taurus-13.lyon.grid5000.fr_flower_1740648154
│   ├── taurus-13.lyon.grid5000.fr_flower_1740649591
│   ├── taurus-13.lyon.grid5000.fr_flower_1740650725
│   ├── taurus-13.lyon.grid5000.fr_flower_1740651708
│   ├── taurus-13.lyon.grid5000.fr_flower_1740652602
│   ├── taurus-13.lyon.grid5000.fr_flower_1740654039
│   ├── taurus-13.lyon.grid5000.fr_flower_1740655170
│   ├── taurus-13.lyon.grid5000.fr_flower_1740656160
│   ├── taurus-1.lyon.grid5000.fr_flower_1740648154
│   ├── taurus-1.lyon.grid5000.fr_flower_1740649591
│   ├── taurus-1.lyon.grid5000.fr_flower_1740650725
│   ├── taurus-1.lyon.grid5000.fr_flower_1740651708
│   ├── taurus-1.lyon.grid5000.fr_flower_1740652602
│   ├── taurus-1.lyon.grid5000.fr_flower_1740654039
│   ├── taurus-1.lyon.grid5000.fr_flower_1740655170
│   ├── taurus-1.lyon.grid5000.fr_flower_1740656160
│   ├── taurus-8.lyon.grid5000.fr_flower_1740648154
│   ├── taurus-8.lyon.grid5000.fr_flower_1740649591
│   ├── taurus-8.lyon.grid5000.fr_flower_1740650725
│   ├── taurus-8.lyon.grid5000.fr_flower_1740651708
│   ├── taurus-8.lyon.grid5000.fr_flower_1740652602
│   ├── taurus-8.lyon.grid5000.fr_flower_1740654039
│   ├── taurus-8.lyon.grid5000.fr_flower_1740655170
│   ├── taurus-8.lyon.grid5000.fr_flower_1740656160
│   ├── taurus-9.lyon.grid5000.fr_flower_1740648154
│   ├── taurus-9.lyon.grid5000.fr_flower_1740649591
│   ├── taurus-9.lyon.grid5000.fr_flower_1740650725
│   ├── taurus-9.lyon.grid5000.fr_flower_1740651708
│   ├── taurus-9.lyon.grid5000.fr_flower_1740652602
│   ├── taurus-9.lyon.grid5000.fr_flower_1740654039
│   ├── taurus-9.lyon.grid5000.fr_flower_1740655170
│   └── taurus-9.lyon.grid5000.fr_flower_1740656160
├── Expetator_taurus-1.lyon.grid5000.fr_1740648134_power
│ < each file contains data from 4 nodes * 2 times repeat * 4 CPU freq setting -> 8 files>
│   ├── taurus-1.lyon.grid5000.fr_flower_1740648154
│   ├── taurus-1.lyon.grid5000.fr_flower_1740649591
│   ├── taurus-1.lyon.grid5000.fr_flower_1740650725
│   ├── taurus-1.lyon.grid5000.fr_flower_1740651708
│   ├── taurus-1.lyon.grid5000.fr_flower_1740652602
│   ├── taurus-1.lyon.grid5000.fr_flower_1740654039
│   ├── taurus-1.lyon.grid5000.fr_flower_1740655170
│   └── taurus-1.lyon.grid5000.fr_flower_1740656160
< For Flwr logs: each folder contains training log from 4 nodes * 2 times repeat * 4 CPU freq setting -> 8 folders, mapping from flower data and system data by time of log>
├── Flwr_20250227_102234
│   ├── Client_172.16.48.13
│   ├── Client_172.16.48.8
│   ├── Client_172.16.48.9
│   ├── Server_172.16.48.1
│   └── training_results_fedAvg_15_20250227_102251.csv <-- main results are stored in csv 
├── Flwr_20250227_104631
│   ├── Client_172.16.48.13
│   ├── Client_172.16.48.8
│   ├── Client_172.16.48.9
│   ├── Server_172.16.48.1
│   └── training_results_fedAvg_15_20250227_104641.csv
├── Flwr_20250227_110525
│   ├── Client_172.16.48.13
│   ├── Client_172.16.48.8
│   ├── Client_172.16.48.9
│   ├── Server_172.16.48.1
│   └── training_results_fedAvg_15_20250227_110533.csv
├── Flwr_20250227_112148
│   ├── Client_172.16.48.13
│   ├── Client_172.16.48.8
│   ├── Client_172.16.48.9
│   ├── Server_172.16.48.1
│   └── training_results_fedAvg_15_20250227_112155.csv
├── Flwr_20250227_113643
│   ├── Client_172.16.48.13
│   ├── Client_172.16.48.8
│   ├── Client_172.16.48.9
│   ├── Server_172.16.48.1
│   └── training_results_fedAvg_15_20250227_113655.csv
├── Flwr_20250227_120039
│   ├── Client_172.16.48.13
│   ├── Client_172.16.48.8
│   ├── Client_172.16.48.9
│   ├── Server_172.16.48.1
│   └── training_results_fedAvg_15_20250227_120049.csv
├── Flwr_20250227_121930
│   ├── Client_172.16.48.13
│   ├── Client_172.16.48.8
│   ├── Client_172.16.48.9
│   ├── Server_172.16.48.1
│   └── training_results_fedAvg_15_20250227_121939.csv
└── Flwr_20250227_123600
    ├── Client_172.16.48.13
    ├── Client_172.16.48.8
    ├── Client_172.16.48.9
    ├── Server_172.16.48.1
    └── training_results_fedAvg_15_20250227_123607.csv
...

/Data_analysis stores the output of all data analysis steps.

Data Analysis Steps

0. Process Mojitos Folder:

• Calculate sum_package_dram for each timestamp in the Mojitos folder.
• Command to reproduce:

python3 0_mojitos.py --dir <path_to_instance_folder>

python3 0_mojitos.py --dir Log/Flower_campaign/Flower_instance_fedAvg_cifar10_epoch1
python3 0_mojitos.py --dir Log/Flower_campaign/Flower_instance_fedAvg_cifar10_epoch2
python3 0_mojitos.py --dir Log/Flower_campaign/Flower_instance_fedAvg2Clients_cifar10_epoch2
python3 0_mojitos.py --dir Log/Flower_campaign/Flower_instance_fedAvg2Clients_cifar10_epoch1

1. Analyze Processed Mojitos Folder:

• Calculate required statistics (e.g., standard deviation, sum, mean) on the processed Mojitos data.
• Command to reproduce:

python3 1_mojitos_ana.py --dir <path_to_instance_folder>

python3 1_mojitos_ana.py --dir Log/Flower_campaign/Flower_instance_fedAvg_cifar10_epoch1
python3 1_mojitos_ana.py --dir Log/Flower_campaign/Flower_instance_fedAvg_cifar10_epoch2
python3 1_mojitos_ana.py --dir Log/Flower_campaign/Flower_instance_fedAvg2Clients_cifar10_epoch2
python3 1_mojitos_ana.py --dir Log/Flower_campaign/Flower_instance_fedAvg2Clients_cifar10_epoch1

2. Analyze Power Data:

• Calculate statistics (e.g., standard deviation, sum, mean) on the power data.
• Command to reproduce:

python3 2_power_ana.py --dir <path_to_instance_folder>

python3 2_power_ana.py --dir Log/Flower_campaign/Flower_instance_fedAvg_cifar10_epoch1
python3 2_power_ana.py --dir Log/Flower_campaign/Flower_instance_fedAvg_cifar10_epoch2
python3 2_power_ana.py --dir Log/Flower_campaign/Flower_instance_fedAvg2Clients_cifar10_epoch2
python3 2_power_ana.py --dir Log/Flower_campaign/Flower_instance_fedAvg2Clients_cifar10_epoch1

3. Combine Mojitos and Power Data:

• Combine the outputs from steps 1 and 2 (Mojitos and Power processed data).
• Command to reproduce:

python3 3_mean_energy.py --power <power_file_processed> --mojitos <mojitos_file_processed>

python3 3_mean_energy.py --power Data_analysis/Output_level_1/merged_power_Flower_campaign_Flower_instance_fedAvg_cifar10_epoch1.csv --mojitos Data_analysis/Output_level_1/merged_mojitos_Flower_campaign_Flower_instance_fedAvg_cifar10_epoch1.csv
python3 3_mean_energy.py --power Data_analysis/Output_level_1/merged_power_Flower_campaign_Flower_instance_fedAvg_cifar10_epoch2.csv --mojitos Data_analysis/Output_level_1/merged_mojitos_Flower_campaign_Flower_instance_fedAvg_cifar10_epoch2.csv
python3 3_mean_energy.py --power Data_analysis/Output_level_1/merged_power_Flower_campaign_Flower_instance_fedAvg2Clients_cifar10_epoch2.csv --mojitos Data_analysis/Output_level_1/merged_mojitos_Flower_campaign_Flower_instance_fedAvg2Clients_cifar10_epoch2.csv
python3 3_mean_energy.py --power Data_analysis/Output_level_1/merged_power_Flower_campaign_Flower_instance_fedAvg2Clients_cifar10_epoch1.csv --mojitos Data_analysis/Output_level_1/merged_mojitos_Flower_campaign_Flower_instance_fedAvg2Clients_cifar10_epoch1.csv

4. Analyze Flower Performance Data:

• Process the performance data (accuracy, time).
• Command to reproduce:

python3 4_performance_ana.py --dir <path_to_instance_folder>

python3 4_flower_ana.py --dir Log/Flower_campaign/Flower_instance_fedAvg_cifar10_epoch1
python3 4_flower_ana.py --dir Log/Flower_campaign/Flower_instance_fedAvg2Clients_cifar10_epoch1
python3 4_flower_ana.py --dir Log/Flower_campaign/Flower_instance_fedAvg2Clients_cifar10_epoch2
python3 4_flower_ana.py --dir Log/Flower_campaign/Flower_instance_fedAvg_cifar10_epoch2

5. Combine Energy/Power and Performance Data:

• Combine the Energy/Power data with performance data.
• Command to reproduce:

python3 5_combine_energy_fl.py -a <accuracy_time_file> -e <energy_file> -m <metadata_file>

python3 5_combine_energy_fl.py -a Data_analysis/Output_level_2/flower_fedAvg2Clients_cifar10_epoch2_accuracy_time.csv -e /home/hdomai/Documents/RepoLab/Data/Data_analysis/Output_level_2/mean_energy_Flower_campaign_Flower_instance_fedAvg2Clients_cifar10_epoch2.csv -m Log/Flower_campaign/Flower_instance_fedAvg2Clients_cifar10_epoch2/Expetator_taurus-1.lyon.grid5000.fr_1740614797
python3 5_combine_energy_fl.py -a Data_analysis/Output_level_2/flower_fedAvg_cifar10_epoch2_accuracy_time.csv -e /home/hdomai/Documents/RepoLab/Data/Data_analysis/Output_level_2/mean_energy_Flower_campaign_Flower_instance_fedAvg_cifar10_epoch2.csv -m Log/Flower_campaign/Flower_instance_fedAvg_cifar10_epoch2/Expetator_taurus-1.lyon.grid5000.fr_1740599416
python3 5_combine_energy_fl.py -a Data_analysis/Output_level_2/flower_fedAvg2Clients_cifar10_epoch1_accuracy_time.csv -e Data_analysis/Output_level_2/mean_energy_Flower_campaign_Flower_instance_fedAvg2Clients_cifar10_epoch1.csv -m Log/Flower_campaign/Flower_instance_fedAvg2Clients_cifar10_epoch1/Expetator_taurus-1.lyon.grid5000.fr_1740657011
python3 5_combine_energy_fl.py -a Data_analysis/Output_level_2/flower_fedAvg_cifar10_epoch1_accuracy_time.csv -e Data_analysis/Output_level_2/mean_energy_Flower_campaign_Flower_instance_fedAvg_cifar10_epoch1.csv -m Log/Flower_campaign/Flower_instance_fedAvg_cifar10_epoch1/Expetator_taurus-1.lyon.grid5000.fr_1740648134

6. Compile All Instances into One Campaign File:

• Gather all the combined instance files into one campaign file.
• Command to reproduce:

python3 6_final.py -f <combined_file_1> <combined_file_n> -c <str_of_campaign_name>

python3 6_final.py -f Data_analysis/Output_level_3/combine_Flower_campaign_Flower_instance_fedAvg_cifar10_epoch1.csv Data_analysis/Output_level_3/combine_Flower_campaign_Flower_instance_fedAvg_cifar10_epoch2.csv Data_analysis/Output_level_3/combine_Flower_campaign_Flower_instance_fedAvg2Clients_cifar10_epoch1.csv Data_analysis/Output_level_3/combine_Flower_campaign_Flower_instance_fedAvg2Clients_cifar10_epoch2.csv -c Flower_full_campaign

7. Plot the Data (Visualization):

• Visualize the combined campaign data using the Jupyter notebook.
• Open 7_PLOT.ipynb and provide the path to the campaign file:

file = '<path_to_campaign_file>'

file = 'Data_analysis/Output_level_4/combine_Flower_full_campaign.csv'