Added results of experiments on MSR trace

code/plot-exp2.py

@@ -271,6 +271,8 @@ def main(wload, target_cache_size):
                             iops = [5.18, 15.15, 3.03, 2.22, 2.57, 1.81, 0.89, 0.39]
                         elif wload == 'scr_merg':
                             iops = [3.53, 0.12, 51.17, 55.22, 362.62, 0.88, 212.14, 11.97]
+                        elif wload == 'real1':
+                            iops = [767.01, 543.30, 945.15, 569.38 ,663.33 , 510.78]
                         else:
                             iops = [99.16, 24.56, 49.59, 199.25, 99.54, 149.74]
 
@@ -386,7 +388,7 @@ def main(wload, target_cache_size):
                         # beg_sec = 2400
                         # end_sec = 3600
                         beg_sec = 1200
-                        end_sec = 3600
+                        end_sec = 14400
                         if wload == 'wload_6gb':
                             plt.ylim([0.4, 0.9])
                         elif wload == 'wload_4gb':
@@ -400,8 +402,8 @@ def main(wload, target_cache_size):
                         plt.xlim([beg_sec, end_sec])
                         # plt.xticks(np.arange(beg_sec, end_sec + 1, 1800),
                         #            np.arange(beg_sec / 60, end_sec / 60 + 1, 30).astype(int))
-                        plt.xticks(np.arange(beg_sec, end_sec + 1, 300),
-                                   np.arange(beg_sec / 60, end_sec / 60 + 1, 5).astype(int))
+                        plt.xticks(np.arange(beg_sec, end_sec + 1, 2400),
+                                   np.arange(beg_sec / 60, end_sec / 60 + 1, 40).astype(int))
                         # plt.show()
                         plt.savefig(basedir_pre[:-1] + '-hr.png', dpi=300, bbox_inches='tight')
                         ############################################################################################
@@ -766,8 +768,9 @@ def main(wload, target_cache_size):
 if __name__ == '__main__':
     # matplotlib.use('agg')
     threads = []
-    wloads = [('wload_6gb', '6 GiB'), ('wload_4gb', '4 GiB'), ('real1', '6 GiB'), ('scr_merg', '64 MiB')]
-    wload = wloads[2]
+    wloads = [('wload_6gb', '6 GiB'), ('wload_4gb', '4 GiB'), ('real1', '6 GiB'), ('scr_merg', '64 MiB'), ('real2', '6 GiB')]
+    wload = wloads[4]
+    # wload = ('wss_iops100','6 GiB')
     # for wload in wloads:
     # x = threading.Thread(target=main, args=(wload[0], wload[1]))
     # threads.append(x)

experiments/real1/setup.txt

@@ -14,4 +14,9 @@ hr-15 -> shows the hit ratio during the 15 minutes after failure in the 3 cases
 hr -> shows the hit ratio for the entire 1 hour duration in the 3 cases
 vdisk-hr5-wop -> shows the hit ratio(without pre-warming) for each vdisk during the 5 minutes after failure
 vdisk-hr5-wp -> shows the hit ratio(with pre-warming) for each vdisk during the 5 minutes after failure
-vdisk-hr -> shows the hit ratio(with pre-warming) for each vdisk during entire 1 hour duration
\ No newline at end of file
+vdisk-hr -> shows the hit ratio(with pre-warming) for each vdisk during entire 1 hour duration
+
+Key observations
+1)Hit ratio, 5 minutes after failure is 0.85 without pre-warming and 0.87 with pre-warming
+2)Pre-warming doesnt seem to help much here. This maybe because IOPS(around 700-800) of the disks are very high and the hit ratio is recovered very quickly
+after failure.

experiments/real2/setup.txt

+Basic Info
+1)Experiment ran on trace taken from iotta.snia.org/traces/388 (MSR Cambridge trace)
+2)The above site has traces of 36 disks over a period of 1 week
+3)I used the first 4 hours of the 6 out of the 36 disks
+
+Results
+1)The relevant plots are included in the plots directory with the following nameing convention
+<prewarm-set-size>-<prewarm-rate>-<heuristic>-<info>
+where info can be
+hr-5 -> shows the hit ratio during the 5 minutes after failure in the 3 cases(with pre-warming, without pre-warming, no failure)
+hr-10 -> shows the hit ratio during the 10 minutes after failure in the 3 cases
+hr-15 -> shows the hit ratio during the 15 minutes after failure in the 3 cases
+hr -> shows the hit ratio for the entire 1 hour duration in the 3 cases
+vdisk-hr5-wop -> shows the hit ratio(without pre-warming) for each vdisk during the 5 minutes after failure
+vdisk-hr5-wp -> shows the hit ratio(with pre-warming) for each vdisk during the 5 minutes after failure
+vdisk-hr -> shows the hit ratio(with pre-warming) for each vdisk during entire 1 hour duration
+
+Key observations
+1)An important observation about the trace is that the iops values are very low. The largest IOPS value among the disk I chose is 18
+2)Pre-warming seems to take the hit ratio to a very large value almost intantly.
+3)The time taken to reach the hit ration just before failure in the case of no-prewarming is around 2 hours.
+4)All this(along with the results of real2) suggests IOPS maybe a very important factor in determing how effective pre-warming is going to be.
+5)In real1 we observed, IOPS were around 700-900 and pre-warming gave very litte benefit, but here with low values IOPS we are getting huge benefits.
+6)Small IOPS may also mean a small wss(if we define it to be the size of the active portion of the disk in a certain time window)
+
+Things to think about
+1)Should we experiment with all the 36 disks instead of choosing only 6. IOPS per disk maybe low because load is distributed over many disks and choosing only 
+a few disks may be giving positive results only because we aren't really using the true workload.
\ No newline at end of file