slight changes in the workload selection

0_prepare_workload.ipynb

@@ -38,14 +38,51 @@
    "id": "graphic-rabbit",
    "metadata": {},
    "source": [
-    "As mentionned in the [original paper releasing the log](https://www.cs.huji.ac.il/~feit/parsched/jsspp15/p5-klusacek.pdf), the platform is **very heterogeneous**. For the purpose of our study, we perform the following selection:\n",
+    "It is a 2-year-long trace from MetaCentrum, the national grid of the Czech republic. As mentionned in the [original paper releasing the log](https://www.cs.huji.ac.il/~feit/parsched/jsspp15/p5-klusacek.pdf), the platform is **very heterogeneous** and underwent majors changes during the logging period. For the purpose of our study, we perform the following selection.\n",
+    "\n",
+    "First:\n",
     "- we remove from the workload all the clusters whose nodes have **more than 16 cores**\n",
-    "- we remove from the workload the jobs with a **execution time greater than one day**\n",
+    "- we truncate the workload to keep only 6 month (June to November 2014) where no major change was performed in the infrastructure (no cluster < 16 cores added nor removed, no reconfiguration in the scheduling system)\n",
+    "\n",
+    "Second:\n",
+    "- we remove from the workload the jobs with an **execution time greater than one day**\n",
     "- we remove from the workload the jobs with a **number of requested cores greater than 16**\n",
     "\n",
     "To do so, we use a home made SWF parser. "
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "ff40dcdd",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Unix Time Jun  1st 2014: 1401573600\n",
+      "Unix Time Nov 30th 2014: 1417388399\n",
+      "We should keep all the jobs submitted between 44578794 and 60393593\n"
+     ]
+    }
+   ],
+   "source": [
+    "from time import *\n",
+    "begin_trace = 1356994806 # according to original SWF header\n",
+    "jun1_unix_time, nov30_unix_time = mktime(strptime('Sun Jun  1 00:00:00 2014')), mktime(strptime('Sun Nov 30 23:59:59 2014'))\n",
+    "jun1, nov30 = (int) (jun1_unix_time - begin_trace), (int) (nov30_unix_time - begin_trace)\n",
+    "print(\"Unix Time Jun  1st 2014: {:.0f}\".format( jun1_unix_time ))\n",
+    "print(\"Unix Time Nov 30th 2014: {:.0f}\".format( nov30_unix_time ))\n",
+    "print(\"We should keep all the jobs submitted between {:d} and {:d}\".format(jun1, nov30))\n",
+    "\n",
+    "# Create a swf with only the selected clusters and the 6 selected months \n",
+    "! scripts/swf_moulinette.py workload/METACENTRUM-2013-3.swf \\\n",
+    "    -o workload/METACENTRUM_6months.swf \\\n",
+    "    --keep_only=\"submit_time >= {jun1} and submit_time <= {nov30}\" \\\n",
+    "    --partitions_to_select 1 2 3 5 7 8 9 10 11 12 14 15 18 19 20 21 22 23 25 26 31"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 9,
@@ -125,9 +162,10 @@
     }
    ],
    "source": [
-    "! ./0_prepare_workload/swf_moulinette.py workload/METACENTRUM-2013-3.swf -o workload/MC_selection_article.swf \\\n",
-    "    --keep_only=\"nb_res <= 16 and run_time <= 24*3600\" \\\n",
-    "    --partitions_to_select 1 2 3 5 7 8 9 10 11 12 14 15 18 19 20 21 22 23 25 26 31"
+    "# Keep only the selected jobs\n",
+    "! ./scripts/swf_moulinette.py workload/METACENTRUM_6months.swf \\\n",
+    "    -o workload/MC_selection_article.swf \\\n",
+    "    --keep_only=\"nb_res <= 16 and run_time <= 24*3600\""
    ]
   },
   {
@@ -169,7 +207,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.8.9"
+   "version": "3.8.10"
   }
  },
  "nbformat": 4,