A repository that holds an ongoing work on Storm to tackle bursty workload and execution stragglers
This project comprises two parts --- the first part is a simulator that helps us to try out different strategies for operator adjustment and task scheduling, which aims to tackle bursty workload and mitigate node stragglers by dynamically matching the resource demands with the resource availability. The second part is the actual implementation of our adjustment framework on Storm, which only requires updates in the package storm core and remains transparent to the up-level logic.
This is repository is under active development as of June 2018.
As explained before, this section is broken into two parts focusing on the simulation and actual deployment, respectively.
To be continued...