Supervision : Pierre SENS

Co-supervision : Luciana Arantes

Dynamic Adaptation in Stream Processing System

Stream Processing Systems (SPSs) dynamically process input events. Since the input is usually not a constant flow, presenting rate fluctuations, many works in the literature propose to dynamically replicate SPS operators, aiming at reducing the processing bottleneck induced by such fluctuations. We present in this these a two approach to dynamically changes the number of replicas of an operator. First, a reactive approach, which based multiple metrics. And second, a predictive model, which based on input rate variation, execution time of operators, and queued events, dynamically defines the necessary current number of replicas of each operator. We also propose a Storm-based SPS, which uses such our model, not requiring reboot reconfiguration when the number of operators replica change.

2021-2023 Publications

• 2023
  • D. Wladdimiro, L. Arantes, N. Hidalgo, P. Sens : “PRESPS: a PREdictive model to determine the number of replicas of the operators in Stream Processing Systems”, Compas 2023 - Conférence francophone d'informatique en Parallélisme, Architecture et Système, Annecy, France (2023)
• 2022
  • D. Wladdimiro, L. Arantes, P. Sens, N. Hidalgo : “A predictive approach for dynamic replication of operators in distributed stream processing systems”, SBAC-PAD 2022 - IEEE 34^th International Symposium on Computer Architecture and High Performance Computing, Bordeaux, France, pp. 120-129, (IEEE) (2022)
  • D. Wladdimiro, L. Arantes, N. Hidalgo, P. Sens : “A predictive model for Stream Processing System that dynamically calibrates the number of operator replicas”, ComPAS 2022 - Conférence francophone d'informatique en Parallélisme, Architecture et Système, Amiens, France (2022)
• 2021
  • D. Wladdimiro, L. Arantes, P. Sens, N. Hidalgo : “A Multi-Metric Adaptive Stream Processing System”, NCA 2021 - 20^th IEEE International Symposium on Network Computing and Applications, Cambridge, Boston, United States (2021)