Large-scale data stream processing systems

QC 20171108
In our data-centric society, online services, decision making, and other aspects are increasingly becoming heavily dependent on trends and patterns extracted from data. A broad class of societal-scale data management problems requires system support for processing unbounded data with low latency and high throughput. Large-scale data stream processing systems perceive data as infinite streams and are designed to satisfy such requirements. They have further evolved substantially both in terms of expressive programming model support and also efficient and durable runtime execution on commodity clusters. Expressive programming models offer convenient ways to declare continuous data properties and applied computations, while hiding details on how these data streams are physically processed and orchestrated in a distributed environment. Execution engines provide a runtime for such models further allowing for scalable yet durable execution of any declared computation. In this chapter we introduce the major design aspects of large scale data stream processing systems, covering programming model abstraction levels and runtime concerns. We then present a detailed case study on stateful stream processing with Apache Flink, an open-source stream processor that is used for a wide variety of processing tasks. Finally, we address the main challenges of disruptive applications that large-scale data streaming enables from a systemic point of view.

Gévay, G. E. (författare)
Hermann, G. (författare)
Katsifodimos, A. (författare)
Soto, J. (författare)
Markl, V. (författare)
Haridi, Seif,1953-KTH,Programvaruteknik och Datorsystem, SCS(Swepub:kth)u1j6y2uf (författare)
KTHProgramvaruteknik och Datorsystem, SCS (creator_code:org_t)

Ingår i:Handbook of Big Data TechnologiesCham : Springer International Publishing, s. 219-26097833194934049783319493398

Av författaren/redakt...: Carbone, Paris; Gévay, G. E.; Hermann, G.; Katsifodimos, A.; Soto, J.; Markl, V.; visa fler...; Haridi, Seif, 19 ...; visa färre...

Om ämnet

TEKNIK OCH TEKNOLOGIER: TEKNIK OCH TEKNO ...; och Elektroteknik oc ...; och Datorsystem

Kungliga biblioteket hanterar dina personuppgifter i enlighet med EU:s dataskyddsförordning (2018), GDPR. Läs mer om hur det funkar här.
Så här hanterar KB dina uppgifter vid användning av denna tjänst.