This project provides tools to measure the energy consumption of Flower-based federated learning (FL) experiments on the Grid'5000 (G5K) testbed. It includes scripts to manage distributed nodes, run FL experiments, and monitor energy usage.
## Table of Contents
-[Getting Started](#getting-started)
-[Installation](#installation)
-[Usage](#usage)
-[Step 1. Configure](#step-1-configure)
-[Step 2. Reserve the Hosts in G5K](#step-2-reserve-the-hosts-in-g5k)
-[Step 3. Collect IP](#step-3-collect-ip)
-[Step 4. Run the Campaign or Instance](#step-4-run-the-campaign-or-instance)
-[Step 5. Output](#step-5-output)
-[Step 6. Clean Up](#step-6-clean-up)
-[Quickstart](#quickstart)
-[Output Structure](#output-structure)
-[License](#license)
## Getting Started
## Getting Started
These instructions will let you know how to run it.
The repository includes an example of Flower (using TensorFlow) in the `Flower_v1` directory and the source of measuring framework in `Run`. This example demonstrates how to use this framework to measure energy consumption.
An example of Flower (using tensorflow) is stored in Flower_v1. The test example will be use this source.
## Installation
### Prerequisites
This framework requires:
-**Python 3.9.2** or higher.
- Additional dependencies listed in `requirements.txt`. Install them with:
```bash
pip install-r requirements.txt
```
*Note:*`requirements.txt` includes TensorFlow for running the provided Flower example.
This framework requires **Python 3.9.2** or higher. Check your Python package version:
```bash
python3 --version
```
Other dependencies are stored in *requirements.txt*. Run:
```bash
pip install-r requirements.txt
```
This *requirements.txt* includes tensorflow to run the provided example Flwr.
### Installing
Download code:
Clone the repository and navigate to the `Run` directory:
**Remark:** EFLWR is configured for Flower only. Your Flower framework must contain server and client script to run on distributed nodes. The Flower_v1 directory provides an example. Once configured, EFLWR will automatically execute flower on the specified nodes and measure the energy consumption.
### Step 1. Configure
Configure the information of each exp in `config_instance*.json`, you can add more `config_instance*.json` with * is a number. Use your editor instead of *vim* (*nano* etc). See the examples in 2 config files which provied in Run directory.
```bash
vim config_instance1.json
```
```
"instance": the name of your exp, anything you want, I suggest to input the specific name of your testing.
"output_dir": where you want to store your log (your exp log and energy mornitoring log)
- FL scripts can be updated in `Flower_v1`.
- Configure each instance of experiment in `Run\config_instance*.json`.
- Follow these steps to config and run your experiment (or jump to [Quickstart](#quickstart) to run an example).
"server":
### Step 1. Reserve the Hosts in G5K
"command": which cmd sever use, my case is python3.
"args": includes file to run and arguments
"ip": address of server, this one will be automatic input when you run the code, just leave it blank
"port": default 8080m dont change it.
"clients":
Reserve the required number of hosts (*See the [document of G5K](https://www.grid5000.fr/w/Getting_Started#Reserving_resources_with_OAR:_the_basics) for more details*)
"name": numbering/name your client, should be client1.2.3...
```bash
"command": which cmd client use, my case is python3
"ip": address of client, this one will be automatic input when you run the code, just leave it blank.
```
```
Note that if you create *x* clients in json config then you have to researve *x+1* hosts in g5k (1 for server and *x* for *x* clients).
### Step 2. Configure
Edit the JSON configuration file (`config_instance*.json`) to specify experiment details. You can create multiple `config_instance*.json` files with the * is numbering of instance (the numbers must be consecutive positive integers starting from 1.)
- Update extraced frequencies value to configure files.
### Step 3. Collect IP
### Step 3. Collect IP
Run below cmd:
Run the following command to generate a node list:
```bash
```bash
uniq$OAR_NODEFILE> nodelist
uniq$OAR_NODEFILE> nodelist
```
```
Then automatically fill out missing IP addresses in `json`:
Automatically populate missing IP addresses in the JSON file:
```bash
```bash
python3 collect_ip.py
python3 collect_ip.py
```
```
### Step 4. Run the campaign or instance:
### Step 4. Run the Campaign or Single Instance
Now you can run the monitoring campaign by run_measure.py. Note that, this function will scan all the json file in the Run directory with the name "config_instance*.json".
`config_instance1.json` and `config_instance2.json` provide two examples of instance configuration. All fields are configured but "output_dir" and "args" must be updated with your directories setting.
Check the output in the directory where you set in json file. The output structure:
-`config_instance1.json`: fedAvg, cifar10, dvfs with min and max freq, 1 round.
```plaintext
-`config_instance1.json`: fedAvg2Clients, cifar10, dvfs with min and max freq, 1 round.
