| 1. |
- Alzghoul, Ahmad
(author)
-
Improving availability of industrial products through data stream mining
- 2011
-
Licentiate thesis (other academic/artistic)abstract
- Products of high quality are of great interest for industrial companies. The quality of a product can be considered in terms of production cost, operating cost, safety and product availability, for example. Product availability is a function of maintainability and reliability. Monitoring prevents unplanned stops, thus increasing product availability by decreasing needed maintenance. Through monitoring, failures can be detected and/or avoided. Detecting failures eliminates extra costs such as costs associated with machinery damage and dissatisfied customers, and time is saved since stops can be scheduled, instead of having unplanned stops. Product monitoring can be done through searching the data generated from sensors installed on products.Nowadays, the data can be collected at high rates as part of a data stream. Therefore, data stream management systems (DSMS) and data stream mining (DSM) are being used to control, manage and search the data stream. This work investigated how the availability of industrial products can be increased through the use of DSM and DSMS technologies.A review of the data stream mining algorithms and their applications in monitoring was conducted. Based on the review, a new data stream classification method, i.e. Grid-based classifier was proposed, tested and validated. Also, a fault detection system based on DSM and DSMS technologies was proposed. The proposed fault detection system was tested using data collected from Hägglunds Drives AB (HDAB) hydraulic motors. Thereafter, a data stream predictor was integrated into the proposed fault detection system to detect failures earlier, thus gaining more time for response actions. The modified fault detection system was tested and showed good performance. The results showed that the proposed fault detection system, which is based on DSM and DSMS technologies, achieved good performance (with classification accuracy around 95%) in detecting failures on time. Detecting failures on time prevents unplanned stops and may improve the maintainability of the industrial systems and, thus, their availability.
|
|
| 2. |
- Chen, Kunru, 1993-, et al.
(author)
-
From Publication to Production : Interactive Deployment of Forklift Activity Recognition
- 2024
-
In: 2024 IEEE International Conference on Industrial Technology (ICIT). - : IEEE. - 9798350340266
-
Conference paper (peer-reviewed)abstract
- As the rise of the Internet of Things has made a vast amount of sensory data readily available, research that develops data-driven methods for industrial applications has become increasingly popular. Yet, there are not many reports presenting the deployment of these methods. One can always expect “there is a gap between theory and reality,” but then, what is the gap? How big is it, and how to handle it? This paper demonstrates the deployment of machine learning (ML) models on a real forklift truck and the utilization of an interactive method that essentially bridges the gap between laboratory and realistic settings of the forklift application. The interactive method suggests a gradual adaptation to various user cases in practice: to test the offline method in an environment slightly different from what the training data presents and adjust the method according to these new usages. Additionally, the interactive model deployment allows modification of the offline method in the telematics unit of the forklift truck, which enables an immediate validation of the method adjustment. The result shows that the proposed method can effectively revise erroneous predictions from the ML method and provide quick adaptation to different forklift operations. It also gives a positive signal for further large-scale deployment of offline ML methods and shows their potential to create value and provide optimization in the industry. © 2024 IEEE.
|
|
| 3. |
- Gidofalvi, Gyözö, 1975-, et al.
(author)
-
Highly scalable trip grouping for large-scale collective transportation systems
- 2008
-
In: Advances in Database Technology - EDBT 2008 - 11th International Conference on Extending Database Technology, Proceedings. - New York, NY, USA : ACM Press. - 9781595939265 ; , s. 678-689
-
Conference paper (peer-reviewed)abstract
- Transportation–related problems, like road congestion, parking, and pollution, are increasing in most cities. In order to reduce traffic, recent work has proposed methods for vehicle sharing, for example for sharing cabs by grouping “closeby” cab requests and thus minimizing transportation cost and utilizing cab space. However, the methods published so far do not scale to large data volumes, which is necessary to facilitate large–scale collective transportation systems, e.g., ride–sharing systems for large cities. This paper presents highly scalable trip grouping algorithms, which generalize previous techniques and support input rates that can be orders of magnitude larger. The following three contributions make the grouping algorithms scalable. First, the basic grouping algorithm is expressed as a continuous stream query in a data stream management system to allow for a very large flow of requests. Second, following the divide–and–conquer paradigm, four space–partitioning policies for dividing the input data stream into sub–streams are developed and implemented using continuous stream queries. Third, using the partitioning policies, parallel implementations of the grouping algorithm in a parallel computing environment are described. Extensive experimental results show that the parallel implementation using simple adaptive partitioning methods can achieve speed–ups of several orders of magnitude without significantly degrading the quality of the grouping.
|
|
| 4. |
- Gidófalvi, Gyözö, et al.
(author)
-
Highly Scalable Trip Grouping for Large Scale Collective Transportation Systems
- 2008
-
In: Proc. 11th International Conference on Extending Database Technology, EDBT 2008.
-
Conference paper (peer-reviewed)abstract
- Transportation–related problems, like road congestion, park-ing, and pollution are increasing in most cities. In order toreduce traffic, recent work has proposed methods for vehiclesharing, for example for sharing cabs by grouping “closeby”cab requests and thus minimizing transportation cost andutilizing cab space. However, the methods proposed so fardo not scale to large data volumes, which is necessary tofacilitate large scale collective transportation systems, e.g.,ride–sharing systems for large cities.This paper presents highly scalable “trip grouping” algo-rithms, that generalize previous techniques and support in-put rates that can be orders of magnitude larger. The follow-ing three contributions make the grouping algorithms scal-able. First, the basic grouping algorithm is expressed as acontinuous stream query in a data stream management sys-tem to allow for very large flows of requests. Second, follow-ing the divide–and–conquer paradigm, four space–partition-ing policies for dividing the input data stream into sub–streams are developed and implemented using continuousstream queries. Third, using the partitioning policies, par-allel implementations of the grouping algorithm in a paral-lel computing environment are described. Extensive experi-mental results show that the parallel implementation usingsimple adaptive partitioning methods can achieve speed–upsof several orders of magnitudes without significantly effect-ing the quality of the grouping.
|
|
| 5. |
- Mahmood, Khalid
(author)
-
Scalable Data Management for Internet of Things
- 2021
-
Doctoral thesis (other academic/artistic)abstract
- Internet of Things (IoT) often involve considerable numbers of sensors that produce large volumes of data. In this context, efficient management of data could potentially enable automatic decision making based on analytics of sensors on equipment. However, these sensors are often geographically distributed and generate diverse formats of data in form of sensor streams at a high rate. The combination of these properties of IoT pose significant challenges for the existing database management systems (DBMSs) to provide scalable data storage and analytics.The problem of providing efficient data management of distributed IoT applications using DBMS technologies is addressed in this thesis. Initially, we developed a prototype system, Fused LOg database Query Processor (FLOQ), which enables general query processingover collections of relational databases that are deployed locally on distributed sites to store sensor measurement logs. Although FLOQ provides efficient query execution when scaling the number of distributed databases, it exhibits complexity and scalability issues for large IoT applications having heterogeneous data. The limitations of FLOQ are primarily inherent to its use of relational database backends for storage of sensor logs.When a relational database is used to store large-scale IoT data, it exhibits several challenges. The loading of massive logs produced at high rates is not fast enough due to its strong consistency mechanisms. Furthermore, it could demonstrate a single point of failure that limits the availability, and the inflexible schemas make it difficult to manage heterogeneity. In contrast to relational databases, distributed NoSQL data stores could provide scalable storage of heterogeneous data through data partitioning, replication, and high availability by sacrificing strong consistency. To understand the suitability of NoSQL databases, this thesis also investigates to what degree NoSQL DBMSs provide scalable storage and analytics of IoT applications by comparing a variety of state-of-the-art relational and NoSQL databases for real-world industrial IoT data. The experimental evaluations reveal that the scalability can be provided by the distributed NoSQL data stores; however, the support of advanced data analytics is difficult due to their limited query processing capabilities. Furthermore, data management of distributed IoT applications often requires seamless integration between a real-time edge analytics platform, a distributed storage manager, effective data integration, and query processing techniques for handling heterogeneity. Therefore, in order to provide a holistic data management solution, this thesis developed the Extended Query Processing (EQP) system, which enables advanced analytics for supporting both edge and offline analytics for large-scale IoT applications.These contributions enable efficient data management of large-scale heterogeneous IoT applications and supports advanced analytics.
|
|
| 6. |
|
|
| 7. |
- Rizk, Raya, et al.
(author)
-
Diftong : a tool for validating big data workflows
- 2019
-
In: Journal of Big Data. - : Springer Science and Business Media LLC. - 2196-1115. ; 6:1
-
Journal article (peer-reviewed)abstract
- Data validation is about verifying the correctness of data. When organisations update and refine their data transformations to meet evolving requirements, it is imperative to ensure that the new version of a workflow still produces the correct output. We motivate the need for workflows and describe the implementation of a validation tool called Diftong. This tool compares two tabular databases resulting from different versions of a workflow to detect and prevent potential unwanted alterations. Row-based and column-based statistics are used to quantify the results of the database comparison. Diftong was shown to provide accurate results in test scenarios, bringing benefits to companies that need to validate the outputs of their workflows. By automating this process, the risk of human error is also eliminated. Compared to the more labour-intensive manual alternative, it has the added benefit of improved turnaround time for the validation process. Together this allows for a more agile way of updating data transformation workflows.
|
|
| 8. |
- Zeitler, Erik, et al.
(author)
-
Massive scale-out of expensive continuous queries
- 2011
-
In: 36th International Conference on Very Large Data Bases.
-
Conference paper (peer-reviewed)abstract
- Scalable execution of expensive continuous queries over massive data streams requires input streams to be split into parallel sub-streams. The query operators are continuously executed in parallel over these sub-streams. Stream splitting involves both partitioning and replication of incoming tuples, depending on how the continuous query is parallelized. We provide a stream splitting operator that enables such customized stream splitting. However, it is critical that the stream splitting itself keeps up with input streams of high volume. This is a problem when the stream splitting predicates have some costs. Therefore, to enable customized splitting of high-volume streams, we introduce a parallelized stream splitting operator, called parasplit. We investigate the performance of parasplit using a cost model and experimentally. Based on these results, a heuristic is devised to automatically parallelize the execution of parasplit. We show that the maximum stream rate of parasplit approaches network speed, and that the parallelization is resource efficient. Finally, the scalability of our approach is experimentally demonstrated on the Linear Road Benchmark, showing an order of magnitude higher stream processing rate over previously published results, allowing at least 512 expressways.
|
|
| 9. |
- Zeitler, Erik, et al.
(author)
-
Processing High-Volume Stream Queries on a Supercomputer
- 2006
-
In: Processing High-Volume Stream Queries on a Supercomputer. - 0769525717 ; , s. 147-
-
Conference paper (peer-reviewed)abstract
- Scientific instruments, such as radio telescopes, colliders, sensor networks, and simulators generate very high volumes of data streams that scientists analyze to detect and understand physical phenomena. The high data volume and the need for advanced computations on the streams require substantial hardware resources and scalable stream processing. We address these challenges by developing data stream management technology to support high-volume stream queries utilizing massively parallel computer hardware. We have developed a data stream management system prototype for state-of-the-art parallel hardware. The performance evaluation uses real measurement data from LOFAR, a radio telescope antenna array being developed in the Netherlands.
|
|
| 10. |
- Zeitler, Erik, 1975-
(author)
-
Scalable Parallelization of Expensive Continuous Queries over Massive Data Streams
- 2011
-
Doctoral thesis (other academic/artistic)abstract
- Numerous applications in for example science, engineering, and financial analysis increasingly require online analysis over streaming data. These data streams are often of such a high rate that saving them to disk is not desirable or feasible. Therefore, search and analysis must be performed directly over the data in motion. Such on-line search and analysis can be expressed as continuous queries (CQs) that are defined over the streams. The result of a CQ is a stream itself, which is continuously updated as new data appears in the queried stream(s). In many cases, the applications require non-trivial analysis, leading to CQs involving expensive processing. To provide scalability of such expensive CQs over high-volume streams, the execution of the CQs must be parallelized.In order to investigate different approaches to parallel execution of CQs, a parallel data stream management system called SCSQ was implemented for this Thesis. Data and queries from space physics and traffic management applications are used in the evaluations, as well as synthetic data and the standard data stream benchmark; the Linear Road Benchmark. Declarative parallelization functions are introduced into the query language of SCSQ, allowing the user to specify customized parallelization. In particular, declarative stream splitting functions are introduced, which split a stream into parallel sub-streams, over which expensive CQ operators are continuously executed in parallel.Naïvely implemented, stream splitting becomes a bottleneck if the input streams are of high volume, if the CQ operators are massively parallelized, or if the stream splitting conditions are expensive. To eliminate this bottleneck, different approaches are investigated to automatically generate parallel execution plans for stream splitting functions. This Thesis shows that by parallelizing the stream splitting itself, expensive CQs can be processed at stream rates close to network speed. Furthermore, it is demonstrated how parallelized stream splitting allows orders of magnitude higher stream rates than any previously published results for the Linear Road Benchmark.
|
|
| 11. |
- Zeitler, Erik, et al.
(author)
-
Scalable Splitting of Massive Data Streams
- 2010
-
In: Database Systems for Advanced Applications. - Berlin : Springer-Verlag. - 9783642120978 ; , s. 184-198
-
Conference paper (peer-reviewed)abstract
- Scalable execution of continuous queries over massive data streams often requires splitting input streams into parallel sub-streams over which query operators are executed in parallel. Automatic stream splitting is in general very difficult, as the optimal parallelization may depend on application semantics. To enable application specific stream splitting, we introduce splitstream functions where the user specifies non-procedural stream partitioning and replication. For high-volume streams, the stream splitting itself becomes a performance bottleneck. A cost model is introduced that estimates the performance of splitstream functions with respect to throughput and CPU usage. We implement parallel splitstream functions, and relate experimental results to cost model estimates. Based on the results, a splitstream function called autosplit is proposed, which scales well for high degrees of parallelism, and is robust for varying proportions of stream partitioning and replication. We show how user defined parallelization using autosplit provides substantially improved scalability (L = 64) over previously published results for the Linear Road Benchmark.
|
|
| 12. |
|
|
| 13. |
- Zeitler, Erik
(author)
-
Working with Stella
- 2006
-
In: Astronnews. - 1871-6644. ; :1, s. 12-13
-
Journal article (pop. science, debate, etc.)
|
|
