| 1. |
|
|
| 2. |
- Carbone, Paris, et al.
(författare)
-
Beyond Analytics : The Evolution of Stream Processing Systems
- 2020
-
Ingår i: Proceedings of the ACM SIGMOD International Conference on Management of Data. - New York, NY, USA : Association for Computing Machinery. - 9781450367356 ; , s. 2651-2658
-
Konferensbidrag (refereegranskat)abstract
- Stream processing has been an active research field for more than 20 years, but it is now witnessing its prime time due to recent successful efforts by the research community and numerous worldwide open-source communities. The goal of this tutorial is threefold. First, we aim to review and highlight noteworthy past research findings, which were largely ignored until very recently. Second, we intend to underline the differences between early ('00-'10) and modern ('11-'18) streaming systems, and how those systems have evolved through the years. Most importantly, we wish to turn the attention of the database community to recent trends: streaming systems are no longer used only for classic stream processing workloads, namely window aggregates and joins. Instead, modern streaming systems are being increasingly used to deploy general event-driven applications in a scalable fashion, challenging the design decisions, architecture and intended use of existing stream processing systems.
|
|
| 3. |
- Fragkoulis, Marios, et al.
(författare)
-
A survey on the evolution of stream processing systems
- 2024
-
Ingår i: The VLDB journal. - : Springer Science and Business Media Deutschland GmbH. - 1066-8888 .- 0949-877X. ; 33:2, s. 507-541
-
Tidskriftsartikel (refereegranskat)abstract
- Stream processing has been an active research field for more than 20 years, but it is now witnessing its prime time due to recent successful efforts by the research community and numerous worldwide open-source communities. This survey provides a comprehensive overview of fundamental aspects of stream processing systems and their evolution in the functional areas of out-of-order data management, state management, fault tolerance, high availability, load management, elasticity, and reconfiguration. We review noteworthy past research findings, outline the similarities and differences between the first (’00–’10) and second (’11–’23) generation of stream processing systems, and discuss future trends and open problems.
|
|
| 4. |
- Siachamis, George, et al.
(författare)
-
CheckMate : Evaluating Checkpointing Protocols for Streaming Dataflows
- 2024
-
Ingår i: Proceedings - 2024 IEEE 40th International Conference on Data Engineering, ICDE 2024. - : Institute of Electrical and Electronics Engineers (IEEE). ; , s. 4030-4043
-
Konferensbidrag (refereegranskat)abstract
- Stream processing in the last decade has seen broad adoption in both commercial and research settings. One key element for this success is the ability of modern stream processors to handle failures while ensuring exactly-once processing guarantees. At the moment of writing, virtually all stream processors that guarantee exactly-once processing implement a variant of Apache Flink's coordinated checkpoints -an extension of the original Chandy-Lamport checkpoints from 1985. However, the reasons behind this prevalence of the coordinated approach remain anecdotal, as reported by practitioners of the stream processing community. At the same time, common checkpointing approaches, such as the uncoordinated and the communication-induced ones, remain largely unexplored. This paper is the first to address this gap by i) shedding light on why practitioners have favored the coordinated approach and ii) investigating whether there are viable alternatives. To this end, we implement three checkpointing approaches that we surveyed and adapted for the distinct needs of streaming dataflows. Our analysis shows that the coordinated approach outperforms the uncoordinated and communication-induced protocols under uniformly distributed workloads. To our surprise, however, the uncoordinated approach is not only competitive to the coordinated one in uniformly distributed workloads, but it also outperforms the coordinated approach in skewed workloads. We conclude that rather than blindly employing coordinated checkpointing, research should focus on optimizing the very promising uncoordinated approach, as it can address issues with skew and support prevalent cyclic queries. We believe that our findings can trigger further research into checkpointing mechanisms.
|
|
