1.
Havers, Bastian, 1991, et al.
(författare)
Proposing a framework for evaluating learning strategies in vehicular CPSs
2022
Ingår i: Middleware 2022 Industrial Track - Proceedings of the 23rd International Middleware Conference Industrial Track, Part of Middleware 2022. - New York, NY, USA : ACM. - 9781450399173 ; , s. 22-28
Konferensbidrag (refereegranskat) abstract
Highly-connected Vehicular Cyber-Physical Systems (VCPSs) offer manifold opportunities for distributing learning across the contained vehicles, road-side units and servers. However, simulating and evaluating particular distributed learning schemes poses a difficult problem in requiring realistic modeling of the vehicular fleet, communication, and the learning itself. In this work, we postulate a set of requirements for a framework simulating a complete learning workflow in a VCPS, and propose a modular architecture for it. Using a prototype implementation, we show with an example experiment the capabilities the proposed framework delivers for evaluating novel learning schemes in custom scenarios.
2.
Butun, Ismail, 1981, et al.
(författare)
Intrusion Detection in Industrial Networks via Data Streaming
2020
Ingår i: Industrial IoT: Challenges, Design Principles, Applications, and Security. - Cham : Springer International Publishing. ; , s. 213-238
Bokkapitel (övrigt vetenskapligt/konstnärligt) abstract
Given the increasing threat surface of industrial networks due to distributed, Internet-of-Things (IoT) based system architectures, detecting intrusions in Industrial IoT (IIoT) systems is all the more important, due to the safety implications of potential threats. The continuously generated data in such systems form both a challenge but also a possibility: data volumes/rates are high and require processing and communication capacity but they contain information useful for system operation and for detection of unwanted situations. In this chapter we explain that stream processing (a.k.a. data streaming) is an emerging useful approach both for general applications and for intrusion detection in particular, especially since it can enable data analysis to be carried out in the continuum of edge-fog-cloud distributed architectures of industrial networks, thus reducing communication latency and gradually filtering and aggregating data volumes. We argue that usefulness stems also due to facilitating provisioning of agile responses, i.e. due to potentially smaller latency for intrusion detection and hence also improved possibilities for intrusion mitigation. In the chapter we outline architectural features of IIoT networks, potential threats and examples of state-of-the art intrusion detection methodologies. Moreover, we give an overview of how leveraging distributed and parallel execution of streaming applications in industrial setups can influence the possibilities of protecting these systems. In these contexts, we give examples using electricity networks (a.k.a. Smart Grid systems). We conclude that future industrial networks, especially their Intrusion Detection Systems (IDSs), should take advantage of data streaming concept by decoupling semantics from the deployment.
3.
Gulisano, Vincenzo Massimiliano, 1984, et al.
(författare)
Efficient Data Streaming Multiway Aggregation through Concurrent Algorithmic Designs and New Abstract Data Types
2017
Ingår i: ACM Transactions on Parallel Computing. - : Association for Computing Machinery (ACM). - 2329-4949 .- 2329-4957. ; 4:2
Tidskriftsartikel (refereegranskat) abstract
Data streaming relies on continuous queries to process unbounded streams of data in a real-time fashion. It is commonly demanding in computation capacity, given that the relevant applications involve very large volumes of data. Data structures act as articulation points and maintain the state of data streaming operators, potentially supporting high parallelism and balancing the work among them. Prompted by this fact, in this work we study and analyze parallelization needs of these articulation points, focusing on the problem of streaming multiway aggregation, where large data volumes are received from multiple input streams. The analysis of the parallelization needs, as well as of the use and limitations of existing aggregate designs and their data structures, leads us to identify needs for appropriate shared objects that can achieve low-latency and high-throughput multiway aggregation. We present the requirements of such objects as abstract data types and we provide efficient lock-free linearizable algorithmic implementations of them, along with new multiway aggregate algorithmic designs that leverage them, supporting both deterministic order-sensitive and order-insensitive aggregate functions. Furthermore, we point out future directions that open through these contributions. The article includes an extensive experimental study, based on a variety of continuous aggregation queries on two large datasets extracted from SoundCloud, a music social network, and from a Smart Grid network. In all the experiments, the proposed data structures and the enhanced aggregate operators improved the processing performance significantly, up to one order of magnitude, in terms of both throughput and latency, over the commonly used techniques based on queues.
4.
Havers, Bastian, 1991, et al.
(författare)
DRIVEN: a Framework for Efficient Data Retrieval and Clustering in Vehicular Networks
2019
Ingår i: Proceedings - International Conference on Data Engineering. - 1084-4627. ; 2019-April, s. 1850-1861
Konferensbidrag (refereegranskat) abstract
Applications for adaptive (sometimes also called smart) Cyber-Physical Systems are blossoming thanks to the large volumes of data, sensed in a continuous fashion, in large distributed systems. The benefits of these applications come nonetheless with a price: the need for jointly addressing challenges in efficient data communication and analysis (among others). The goal of the DRIVEN framework, presented here, is to address these challenges for a data gathering and distance-based clustering tool in the context of vehicular networks. Because of the limited communication bandwidth (compared to the volume of sensed data) of vehicular networks and the monetary costs of data transmission, the intuition behind DRIVEN is to avoid gathering the data to be clustered in a raw format from each vehicle, but rather to allow for a streaming-based error-bounded approximation, through Piecewise Linear Approximation, to compress the volumes of data to be gathered. At the same time, rather than relying on a batch-based clustering algorithm that requires all the data to be first gathered (and then clustered), DRIVEN relies on and extends a streaming-based clustering algorithm that leverages the inherent ordering of the spatial and temporal data being collected, to perform the clustering in an online fashion, while data is being retrieved. As we show, based on our prototype implementation using Apache Flink and our evaluation with real-world data such as GPS and LiDAR, the accuracy loss for the clustering performed on the reconstructed data can be small, even when the raw data is compressed to 10- 35% of its original size, and the transferring of data itself can be completed in up to one-tenth of the duration observed when gathering raw data.
5.
Najdataei, Hannaneh, 1988, et al.
(författare)
Continuous and parallel LiDAR point-cloud clustering
2018
Ingår i: Proceedings - International Conference on Distributed Computing Systems. ; 2018-July, s. 671-684
Konferensbidrag (refereegranskat) abstract
In distributed digitalized environments in the context of the Internet of Things, we often need to do an analysis of big data originating at high rate-sensors at the edge of the infrastructure. A characteristic example is the light detection and ranging (LiDAR) technology, that allows sensing surrounding objects with fine-grained resolution in large areas. Their data (known as point clouds), generated continuously at very high rates, through appropriate analysis can provide information to support automated functionality in distributed cyber-physical? systems; clustering of point clouds is a key problem to extract this type of information. Methods for solving the problem in a continuous fashion can facilitate improved processing in fog architectures, through enabling low-latency, efficient continuous and streaming processing of data close to the sources; moreover, parallelism is a key requirement to exploit a variety of computing architectures in this context. We proposeLisco, a single-pass continuous Euclidean-distance-based clustering of LiDAR point clouds, that maximizes the granularity of the data processing pipeline and thus shows the potential for data-and pipeline-parallelism. We further present its parallel version, P-Lisco, that is architecture-independent and exploits the parallelism revealed byLisco'salgorithmic approach. Besides their algorithmic analysis, we provide a thorough experimental evaluation on architectures representative of high-end servers and of resource-constrained embedded devices and highlight the multiplicative improvements and scalability benefits of the proposed algorithms compared to the baseline, using both real-world datasets as well as synthetic ones to fully explore a wide spectrum of stress-levels for the algorithms.
6.
Najdataei, Hannaneh, 1988, et al.
(författare)
Stream-IT: Continuous and dynamic processing of production systems data - Throughput bottlenecks as a case-study
2019
Ingår i: IEEE International Symposium on Industrial Electronics. ; 2019-June, s. 1328-1333
Konferensbidrag (refereegranskat) abstract
Considering the needs for continuous availability of information out of data generated in Cyber-Physical production systems, we investigate the use of continuous stream processing as a paradigm for generating useful information out of the data, to support efficient and safe operation, as well as planning activities.Our contributions and expected benefits: (i) we show possibilities to automate and pipeline the validation and analysis of the data, hence providing an automated way to improve the quality of the latter and parallelizing the two phases; (ii) we show how to induce lower latency in generating the desired information, enabling it to be continuously made available, before whole batches of data are gathered, in cost-efficient ways; (iii) besides the automation of the above procedures that are commonly done in a batch fashion and with significant manual effort by the production system analysts, we show additional options for configuring ways in which to automate deeper analysis of the data; in particular, we provide evidences about how the rich semantics of stream processing frameworks can ease the development and deployment of data analysis applications in production systems.Moreover, using the problem of bottleneck detection as a sample scenario, we illustrate the above in a concrete fashion, on cost-efficient systems, that are plausible to have in existing deployments. The experimental study is on a 2-year data-set with more than 8.5 million entries, from a system including more than 30 interconnected machines and it demonstrates the benefits of the proposed methods, in providing timely and multidimensional information from the data, enabling possibilities for deeper analyses.
7.
Palyvos-Giannas, Dimitrios, 1991, et al.
(författare)
Erebus: Explaining the Outputs of Data Streaming Queries
2022
Ingår i: Proceedings of the VLDB Endowment. - : Association for Computing Machinery (ACM). - 2150-8097. ; 16:2, s. 230-242
Konferensbidrag (refereegranskat) abstract
In data streaming, why-provenance can explain why a given outcome is observed but offers no help in understanding why an expected outcome is missing. Explaining missing answers has been addressed in DBMSs, but these solutions are not directly applicable to the streaming setting, because of the extra challenges posed by limited storage and by the unbounded nature of data streams. With our framework, Erebus, we tackle the unaddressed challenges behind explaining missing answers in streaming applications. Erebus allows users to define expectations about the results of a query, verifying at runtime if such expectations hold, and also providing explanations when expected and observed outcomes diverge (missing answers). To the best of our knowledge, Erebus is the first such solution in data streaming. Our thorough evaluation on real data shows that Erebus can explain the (missing) answers with small overheads, both in low-and higher-end devices, even when large portions of the processed data are part of such explanations.
8.
Najdataei, Hannaneh, 1988, et al.
(författare)
pi-Lisco: parallel and incremental stream-based point-cloud clustering
2022
Ingår i: Proceedings of the ACM Symposium on Applied Computing. - New York, NY, USA : ACM. ; , s. 460-469
Konferensbidrag (refereegranskat) abstract
Point-cloud clustering is a key task in applications like autonomous vehicles and digital twins, where rotating LiDAR sensors commonly generate point-cloud measurements in data streams. The state-of-the-art algorithms, Lisco and its parallel equivalent P-Lisco, define a single-pass distance-based clustering. However, while outperforming other batch-based techniques, they cannot incrementally cluster point-clouds from consecutive LiDAR rotations, as they cannot exploit result-similarity between rotations. The simplicity of Lisco, along with the potential of improvements through utilization of computational overlaps, form the motivation of a more challenging objective studied here. We propose Parallel and Incremental Lisco (pi-Lisco), which, with a simple yet efficient approach, clusters LiDAR data in streaming sliding windows, reusing the results from overlapping portions of the data, thus, enabling single-window (i.e., in-place) processing. Moreover, pi-Lisco employs efficient work-sharing among threads, facilitated by the ScaleGate data structure, and embeds a customised version of the STINGER concurrent data structure. Through an orchestration of these key ideas, pi-Lisco is able to lead to significant performance improvements. We complement with an evaluation of pi-Lisco, using the Ford Campus real-world extensive data-set, showing (i) the computational benefits from incrementally processing the consecutive point-clouds; and (ii) the fact that pi-Lisco' parallelization leads to continuously increasing sustainable rates with increasing number of threads, shifting the saturation point of the baseline.
9.
Duvignau, Romaric, 1989, et al.
(författare)
Querying Large Vehicular Networks: How to Balance On-Board Workload and Queries Response Time?
2019
Ingår i: Proceedings of the 2019 IEEE Intelligent Transportation Systems Conference - ITSC 2019. - 9781538670248 ; , s. 2604-2611
Konferensbidrag (refereegranskat) abstract
Data analysis plays a key role in designing today’s Intelligent Transportation Systems (ITS) and is expected to become even more important in the future. Connected vehicles, one of the main instantiations of ITS, produce large volumes of data that are hard to gather by centralized analysis tools. The even larger volumes of data expected from autonomous driving will further exacerbate the bottleneck problem of data retrieval. When analysts issue queries that seek data from vehicles satisfying certain criteria (e.g. those driving above a certain speed or in a certain area), the problem can nonetheless be overcome by pushing to vehicles themselves the job of checking and reporting the compliance of their local data, hence avoiding a costly data retrieval phase. To efficiently provide answers for such queries, we present in this work configurable query-spreading algorithms tailored for vehicular networks. Our tunable algorithms, which we evaluate on two large datasets of real-world vehicular data, outperform baseline solutions and are able to trade-off the overall on-board workload and the response time needed to resolve a set of queries.
10.
Duvignau, Romaric, 1989, et al.
(författare)
Small-Scale Communities Are Sufficient for Cost- and Data-Efficient Peer-to-Peer Energy Sharing
2020
Ingår i: e-Energy 2020 - Proceedings of the 11th ACM International Conference on Future Energy Systems. - New York, NY, USA : ACM. ; , s. 35-46
Konferensbidrag (refereegranskat) abstract
Due to ever lower cost, investments in renewable electricity generation and storage have become more attractive to electricity consumers in recent years. At the same time, electricity generation and storage have become something to share or trade locally in energy communities or microgrid systems. In this context, peer-to-peer (P2P) sharing has gained attention, since it offers a way to optimize the cost-benefits from distributed resources, making them financially more attractive. However, it is not yet clear in which situations consumers do have interests to team up and how much cost is saved through cooperation in practical instances. While introducing realistic continuous decisions, through detailed analysis based on large-scale measured household data, we show that the financial benefit of cooperation does not require an accurate forecasting. Furthermore, we provide strong evidence, based on analysis of the same data, that even P2P networks with only 2--5 participants can reach a high fraction (96% in our study) of the potential gain, i.e., of the ideal offline (i.e., non-continuous) achievable gain. Maintaining such small communities results in much lower associated costs and better privacy, as each participant only needs to share its data with 1--4 other peers. These findings shed new light and motivate requirements for distributed, continuous and dynamic P2P matching algorithms for energy trading and sharing.
