| 1. |
- Cortinhal, Tiago, 1990-
(författare)
-
Semantics-aware Multi-modal Scene Perception for Autonomous Vehicles
- 2024
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Autonomous vehicles represent the pinnacle of modern technological innovation, navigating complex and unpredictable environments. To do so effectively, they rely on a sophisticated array of sensors. This thesis explores two of the most crucial sensors: LiDARs, known for their accuracy in generating detailed 3D maps of the environment, and RGB cameras, essential for processing visual cues critical for navigation. Together, these sensors form a comprehensive perception system that enables autonomous vehicles to operate safely and efficiently.However, the reliability of these vehicles has yet to be tested when key sensors fail. The abrupt failure of a camera, for instance, disrupts the vehicle’s perception system, creating a significant gap in sensory input. This thesis addresses this challenge by introducing a novel multi-modal domain translation framework that integrates LiDAR and RGB camera data while ensuring continuous functionality despite sensor failures. At the core of this framework is an innovative model capable of synthesizing RGB images and their corresponding segment maps from raw LiDAR data by exploiting the scene semantics. The proposed framework stands out as the first of its kind, demonstrating for the first time that the scene semantics can bridge the gap across different domains with distinct data structures, such as unorganized sparse 3D LiDAR point clouds and structured 2D camera data. Thus, this thesis represents a significant leap forward in the field, offering a robust solution to the challenge of RGB data recovery without camera sensors.The practical application of this model is thoroughly explored in the thesis. It involves testing the model’s capability to generate pseudo point clouds from RGB depth estimates, which, when combined with LiDAR data, create an enriched perception dataset. This enriched dataset is pivotal in enhancing object detection capabilities, a fundamental aspect of autonomous vehicle navigation. The quantitative and qualitative evidence reported in this thesis demonstrates that the synthetic generation of data not only compensates for the loss of sensory input but also considerably improves the performance of object detection systems compared to using raw LiDAR data only.By addressing the critical issue of sensor failure and presenting viable solutions, this thesis contributes to enhancing the safety, reliability, and efficiency of autonomous vehicles. It paves the way for further research and developiment, setting a new standard for autonomous vehicle technology in scenarios of sensor malfunctions or adverse environmental conditions.
|
|
| 2. |
- Altarabichi, Mohammed Ghaith, 1981-
(författare)
-
Evolving intelligence : Overcoming challenges for Evolutionary Deep Learning
- 2024
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Deep Learning (DL) has achieved remarkable results in both academic and industrial fields over the last few years. However, DL models are often hard to design and require proper selection of features and tuning of hyper-parameters to achieve high performance. These selections are tedious for human experts and require substantial time and resources. A difficulty that encouraged a growing number of researchers to use Evolutionary Computation (EC) algorithms to optimize Deep Neural Networks (DNN); a research branch called Evolutionary Deep Learning (EDL).This thesis is a two-fold exploration within the domains of EDL, and more broadly Evolutionary Machine Learning (EML). The first goal is to makeEDL/EML algorithms more practical by reducing the high computational costassociated with EC methods. In particular, we have proposed methods to alleviate the computation burden using approximate models. We show that surrogate-models can speed up EC methods by three times without compromising the quality of the final solutions. Our surrogate-assisted approach allows EC methods to scale better for both, expensive learning algorithms and large datasets with over 100K instances. Our second objective is to leverage EC methods for advancing our understanding of Deep Neural Network (DNN) design. We identify a knowledge gap in DL algorithms and introduce an EC algorithm precisely designed to optimize this uncharted aspect of DL design. Our analytical focus revolves around revealing avant-garde concepts and acquiring novel insights. In our study of randomness techniques in DNN, we offer insights into the design and training of more robust and generalizable neural networks. We also propose, in another study, a novel survival regression loss function discovered based on evolutionary search.
|
|
| 3. |
- Altarabichi, Mohammed Ghaith, 1981-, et al.
(författare)
-
Extracting Invariant Features for Predicting State of Health of Batteries in Hybrid Energy Buses
- 2021
-
Ingår i: 2021 IEEE 8th International Conference on Data Science and Advanced Analytics (DSAA), Porto, Portugal, 6-9 Oct., 2021. - : IEEE. ; , s. 1-6
-
Konferensbidrag (refereegranskat)abstract
- Batteries are a safety-critical and the most expensive component for electric vehicles (EVs). To ensure the reliability of the EVs in operation, it is crucial to monitor the state of health of those batteries. Monitoring their deterioration is also relevant to the sustainability of the transport solutions, through creating an efficient strategy for utilizing the remaining capacity of the battery and its second life. Electric buses, similar to other EVs, come in many different variants, including different configurations and operating conditions. Developing new degradation models for each existing combination of settings can become challenging from different perspectives such as unavailability of failure data for novel settings, heterogeneity in data, low amount of data available for less popular configurations, and lack of sufficient engineering knowledge. Therefore, being able to automatically transfer a machine learning model to new settings is crucial. More concretely, the aim of this work is to extract features that are invariant across different settings.In this study, we propose an evolutionary method, called genetic algorithm for domain invariant features (GADIF), that selects a set of features to be used for training machine learning models, in such a way as to maximize the invariance across different settings. A Genetic Algorithm, with each chromosome being a binary vector signaling selection of features, is equipped with a specific fitness function encompassing both the task performance and domain shift. We contrast the performance, in migrating to unseen domains, of our method against a number of classical feature selection methods without any transfer learning mechanism. Moreover, in the experimental result section, we analyze how different features are selected under different settings. The results show that using invariant features leads to a better generalization of the machine learning models to an unseen domain.
|
|
| 4. |
- Altarabichi, Mohammed Ghaith, 1981-, et al.
(författare)
-
Fast Genetic Algorithm for feature selection — A qualitative approximation approach
- 2023
-
Ingår i: Expert systems with applications. - Oxford : Elsevier. - 0957-4174 .- 1873-6793. ; 211
-
Tidskriftsartikel (refereegranskat)abstract
- Evolutionary Algorithms (EAs) are often challenging to apply in real-world settings since evolutionary computations involve a large number of evaluations of a typically expensive fitness function. For example, an evaluation could involve training a new machine learning model. An approximation (also known as meta-model or a surrogate) of the true function can be used in such applications to alleviate the computation cost. In this paper, we propose a two-stage surrogate-assisted evolutionary approach to address the computational issues arising from using Genetic Algorithm (GA) for feature selection in a wrapper setting for large datasets. We define “Approximation Usefulness” to capture the necessary conditions to ensure correctness of the EA computations when an approximation is used. Based on this definition, we propose a procedure to construct a lightweight qualitative meta-model by the active selection of data instances. We then use a meta-model to carry out the feature selection task. We apply this procedure to the GA-based algorithm CHC (Cross generational elitist selection, Heterogeneous recombination and Cataclysmic mutation) to create a Qualitative approXimations variant, CHCQX. We show that CHCQX converges faster to feature subset solutions of significantly higher accuracy (as compared to CHC), particularly for large datasets with over 100K instances. We also demonstrate the applicability of the thinking behind our approach more broadly to Swarm Intelligence (SI), another branch of the Evolutionary Computation (EC) paradigm with results of PSOQX, a qualitative approximation adaptation of the Particle Swarm Optimization (PSO) method. A GitHub repository with the complete implementation is available. © 2022 The Author(s)
|
|
| 5. |
- Altarabichi, Mohammed Ghaith, 1981-, et al.
(författare)
-
Fast Genetic Algorithm For Feature Selection — A Qualitative Approximation Approach
- 2023
-
Ingår i: Evolutionary Computation Conference Companion (GECCO ’23 Companion), July 15–19, 2023, Lisbon, Portugal. - New York, NY : Association for Computing Machinery (ACM). - 9798400701207 ; , s. 11-12
-
Konferensbidrag (refereegranskat)abstract
- We propose a two-stage surrogate-assisted evolutionary approach to address the computational issues arising from using Genetic Algorithm (GA) for feature selection in a wrapper setting for large datasets. The proposed approach involves constructing a lightweight qualitative meta-model by sub-sampling data instances and then using this meta-model to carry out the feature selection task. We define "Approximation Usefulness" to capture the necessary conditions that allow the meta-model to lead the evolutionary computations to the correct maximum of the fitness function. Based on our procedure we create CHCQX a Qualitative approXimations variant of the GA-based algorithm CHC (Cross generational elitist selection, Heterogeneous recombination and Cataclysmic mutation). We show that CHCQX converges faster to feature subset solutions of significantly higher accuracy, particularly for large datasets with over 100K instances. We also demonstrate the applicability of our approach to Swarm Intelligence (SI), with results of PSOQX, a qualitative approximation adaptation of the Particle Swarm Optimization (PSO) method. A GitHub repository with the complete implementation is available2. This paper for the Hot-off-the-Press track at GECCO 2023 summarizes the original work published at [3].References[1] Mohammed Ghaith Altarabichi, Yuantao Fan, Sepideh Pashami, Peyman Sheikholharam Mashhadi, and Sławomir Nowaczyk. 2021. Extracting invariant features for predicting state of health of batteries in hybrid energy buses. In 2021 ieee 8th international conference on data science and advanced analytics (dsaa). IEEE, 1–6.[2] Mohammed Ghaith Altarabichi, Sławomir Nowaczyk, Sepideh Pashami, and Peyman Sheikholharam Mashhadi. 2021. Surrogate-assisted genetic algorithm for wrapper feature selection. In 2021 IEEE Congress on Evolutionary Computation (CEC). IEEE, 776–785.[3] Mohammed Ghaith Altarabichi, Sławomir Nowaczyk, Sepideh Pashami, and Peyman Sheikholharam Mashhadi. 2023. Fast Genetic Algorithm for feature selection—A qualitative approximation approach. Expert systems with applications 211 (2023), 118528.© 2023 Copyright held by the owner/author(s).
|
|
| 6. |
- Altarabichi, Mohammed Ghaith, 1981-, et al.
(författare)
-
Rolling The Dice For Better Deep Learning Performance : A Study Of Randomness Techniques In Deep Neural Networks
- 2024
-
Ingår i: Information Sciences. - Philadelphia, PA : Elsevier. - 0020-0255 .- 1872-6291. ; 667, s. 1-17
-
Tidskriftsartikel (refereegranskat)abstract
- This paper presents a comprehensive empirical investigation into the interactions between various randomness techniques in Deep Neural Networks (DNNs) and how they contribute to network performance. It is well-established that injecting randomness into the training process of DNNs, through various approaches at different stages, is often beneficial for reducing overfitting and improving generalization. However, the interactions between randomness techniques such as weight noise, dropout, and many others remain poorly understood. Consequently, it is challenging to determine which methods can be effectively combined to optimize DNN performance. To address this issue, we categorize the existing randomness techniques into four key types: data, model, optimization, and learning. We use this classification to identify gaps in the current coverage of potential mechanisms for the introduction of noise, leading to proposing two new techniques: adding noise to the loss function and random masking of the gradient updates.In our empirical study, we employ a Particle Swarm Optimizer (PSO) to explore the space of possible configurations to answer where and how much randomness should be injected to maximize DNN performance. We assess the impact of various types and levels of randomness for DNN architectures applied to standard computer vision benchmarks: MNIST, FASHION-MNIST, CIFAR10, and CIFAR100. Across more than 30\,000 evaluated configurations, we perform a detailed examination of the interactions between randomness techniques and their combined impact on DNN performance. Our findings reveal that randomness in data augmentation and in weight initialization are the main contributors to performance improvement. Additionally, correlation analysis demonstrates that different optimizers, such as Adam and Gradient Descent with Momentum, prefer distinct types of randomization during the training process. A GitHub repository with the complete implementation and generated dataset is available. © 2024 The Author(s)
|
|
| 7. |
- Altarabichi, Mohammed Ghaith, 1981-, et al.
(författare)
-
Stacking Ensembles of Heterogenous Classifiers for Fault Detection in Evolving Environments
- 2020
-
Ingår i: Proceedings of the 30th European Safety and Reliability Conference and the 15th Probabilistic Safety Assessment and Management Conference. - Singapore : Research Publishing Services. - 9789811485930 ; , s. 1068-1068
-
Konferensbidrag (refereegranskat)abstract
- Monitoring the condition, detecting faults, and modeling the degradation of industrial equipment are important challenges in Prognostics and Health Management (PHM) field. Our solution to the challenge demonstrated a multi-stage approach for detecting faults in a group of identical industrial equipment, composed of four identical interconnected components, that have been deployed to the evolving environment with changes in operational and environmental conditions. In the first stage, a stacked ensemble of heterogeneous classifiers was applied to predict the state of each component of the equipment individually. In the second stage, a low pass filter was applied to smoothen the predictions cast by stacked ensembles, utilizing temporal information of the prediction sequence. © ESREL2020-PSAM15 Organizers. Published by Research Publishing, Singapore.
|
|
| 8. |
- Altarabichi, Mohammed Ghaith, 1981-, et al.
(författare)
-
Surrogate-Assisted Genetic Algorithm for Wrapper Feature Selection
- 2021
-
Ingår i: 2021 IEEE Congress on Evolutionary Computation (CEC). - : IEEE. - 9781728183930 ; , s. 776-785
-
Konferensbidrag (refereegranskat)abstract
- Feature selection is an intractable problem, therefore practical algorithms often trade off the solution accuracy against the computation time. In this paper, we propose a novel multi-stage feature selection framework utilizing multiple levels of approximations, or surrogates. Such a framework allows for using wrapper approaches in a much more computationally efficient way, significantly increasing the quality of feature selection solutions achievable, especially on large datasets. We design and evaluate a Surrogate-Assisted Genetic Algorithm (SAGA) which utilizes this concept to guide the evolutionary search during the early phase of exploration. SAGA only switches to evaluating the original function at the final exploitation phase.We prove that the run-time upper bound of SAGA surrogate-assisted stage is at worse equal to the wrapper GA, and it scales better for induction algorithms of high order of complexity in number of instances. We demonstrate, using 14 datasets from the UCI ML repository, that in practice SAGA significantly reduces the computation time compared to a baseline wrapper Genetic Algorithm (GA), while converging to solutions of significantly higher accuracy. Our experiments show that SAGA can arrive at near-optimal solutions three times faster than a wrapper GA, on average. We also showcase the importance of evolution control approach designed to prevent surrogates from misleading the evolutionary search towards false optima.
|
|
| 9. |
- Karlsson, Nellie, et al.
(författare)
-
Baseline Selection for Integrated Gradients in Predictive Maintenance of Volvo Trucks’ Turbocharger
- 2023
-
Ingår i: VEHICULAR 2023. - : International Academy, Research and Industry Association (IARIA). - 9781685580612 ; , s. 29-36
-
Konferensbidrag (refereegranskat)abstract
- The new advances in Vehicular Systems and Technologies have resulted in a sheer increase in the number of connected vehicles. These connected vehicles use IoT technologies to communicate operational signals with the OEMs, such as the vehicle’s speed, torque, temperature, load, RPM, etc. These signals have provided an unprecedented opportunity to adaptively monitor the status of each piece of the vehicle’s equipment and discover any possible risk of failure before it happens. This emerging field of study is called predictive maintenance (also known as condition-based maintenance) and has recently received much attention. In this paper, we apply Integrated Gradients (IG), an XAI method until now primarily used on image data, on datasets containing tabular and time-series data in the domain of predictive maintenance of trucks’ turbochargers. We evaluate how the results of IG differ, in these new settings, for various types of models. In particular, we investigate how the change of baseline can affect the outcome. Experimental results verify that IG can be applied successfully to both sequenced and non-sequenced data. Contrary to the opinion common in the literature, the gradient baseline does not affect the results of IG significantly, especially on models such as RNN, LSTM, and GRU, where the data contains time series; the effect is more visible for models like MLP with non-sequenced data. To confirm these findings, and to understand them deeper, we have also applied IG to SVM models, which gave the results that the choice of gradient baseline has a significant impact on the performance of SVM. (c) IARIA, 2023
|
|
| 10. |
- Kharazian, Zahra, et al.
(författare)
-
AID4HAI : Automatic Idea Detection for Healthcare-Associated Infections from Twitter, A Framework based on Active Learning and Transfer Learning
- 2023
-
Ingår i: Advances in Intelligent Data Analysis XXI. - Cham : Springer. - 9783031300462 - 9783031300479 ; , s. 195-207
-
Konferensbidrag (refereegranskat)abstract
- This research is an interdisciplinary work between data scientists, innovation management researchers, and experts from a Swedish hygiene and health company. Based on this collaboration, we have developed a novel package for automatic idea detection to control and prevent healthcare-associated infections (HAI). The principal idea of this study is to use machine learning methods to extract informative ideas from social media to assist healthcare professionals in reducing the rate of HAI. Therefore, the proposed package offers a corpus of data collected from Twitter, associated expert-created labels, and software implementation of an annotation framework based on the Active Learning paradigm. We employed Transfer Learning and built a two-step deep neural network model that incrementally extracts the semantic representation of the collected text data using the BERTweet language model in the first step and classifies these representations as informative or non-informative using a multi-layer perception (MLP) in the second step. The package is AID4HAI (Automatic Idea Detection for controlling and preventing Healthcare-Associated Infections) and is made fully available (software code and the collected data) through a public GitHub repository (https://github.com/XaraKar/AID4HAI). We believe that sharing our ideas and releasing these ready-to-use tools contributes to the development of the field and inspires future research.
|
|
| 11. |
- Khoshkangini, Reza, 1984-, et al.
(författare)
-
Early Prediction of Quality Issues in Automotive Modern Industry
- 2020
-
Ingår i: Information. - Basel : MDPI. - 2078-2489. ; 11:7
-
Tidskriftsartikel (refereegranskat)abstract
- Many industries today are struggling with early identification of quality issues, given the shortening of product design cycles and the desire to decrease production costs, coupled with customers' requirement for high uptime. The vehicle industry is no exception, as breakdowns often lead to on-road stops and delays in delivery missions. In this paper we consider quality issues to be an unexpected increase in failure rates of a particular component; those are particularly problematic for the Original Equipment Manufacturers (OEMs) since they lead to unplanned costs and can significantly affect brand value. We propose a new approach towards the early detection of quality issues using Machine Learning (ML) to forecast the failures of a given component across the large population of units.In this study, we combine the usage information of vehicles with the records of their failures. The former is continuously collected, as the usage statistics are transmitted over telematics connections. The latter is based on invoice and warranty information collected in the workshops. We compare two different ML approaches: the first is an auto-regression model of the failure ratios for vehicles based on past information, while the second is the aggregation of individual vehicle failure predictions based on their individual usage.We present experimental evaluations on the real data captured from heavy-duty trucks demonstrating how these two formulations have complementary strengths and weaknesses; in particular, they can outperform each other given different volumes of the data. The classification approach surpasses the regressor model whenever enough data is available, i.e., once the vehicles are in-service for a longer time. On the other hand, the regression shows better predictive performance with a smaller amount of data, i.e., for vehicles that have been deployed recently. © 2020 by the authors. Licensee MDPI, Basel, Switzerland.
|
|
| 12. |
- Khoshkangini, Reza, 1984-, et al.
(författare)
-
Optimal Task Grouping Approach in Multitask Learning
- 2024
-
Ingår i: Neural Information Processing. ICONIP 2023. - Heidelberg : Springer Nature. - 9789819980758 - 9789819980765 ; , s. 206-225
-
Konferensbidrag (refereegranskat)abstract
- Multi-task learning has become a powerful solution in which multiple tasks are trained together to leverage the knowledge learned from one task to improve the performance of the other tasks. However, the tasks are not always constructive on each other in the multi-task formulation and might play negatively during the training process leading to poor results. Thus, this study focuses on finding the optimal group of tasks that should be trained together for multi-task learning in an automotive context. We proposed a multi-task learning approach to model multiple vehicle long-term behaviors using low-resolution data and utilized gradient descent to efficiently discover the optimal group of tasks/vehicle behaviors that can increase the performance of the predictive models in a single training process. In this study, we also quantified the contribution of individual tasks in their groups and to the other groups’ performance. The experimental evaluation of the data collected from thousands of heavy-duty trucks shows that the proposed approach is promising. © 2024 Springer Nature
|
|
| 13. |
- Khoshkangini, Reza, et al.
(författare)
-
Predicting Vehicle Behavior Using Multi-task Ensemble Learning
- 2023
-
Ingår i: Expert systems with applications. - Oxford : Elsevier. - 0957-4174 .- 1873-6793. ; 212
-
Tidskriftsartikel (refereegranskat)abstract
- Vehicle utilization analysis is an essential tool for manufacturers to understand customer needs, improve equipment uptime, and to collect information for future vehicle and service development. Typically today, this behavioral modeling is done on high-resolution time-resolved data with features such as GPS position and fuel consumption. However, high-resolution data is costly to transfer and sensitive from a privacy perspective. Therefore, such data is typically only collected when the customer pays for extra services relying on that data. This motivated us to develop a multi-task ensemble approach to transfer knowledge from the high-resolution data and enable vehicle behavior prediction from low-resolution but high dimensional data that is aggregated over time in the vehicles.This study proposes a multi-task snapshot-stacked ensemble (MTSSE) deep neural network for vehicle behavior prediction by considering vehicles’ low-resolution operational life records. The multi-task ensemble approach utilizes the measurements to map the low-frequency vehicle usage to the vehicle behaviors defined from the high-resolution time-resolved data. Two data sources are integrated and used: high-resolution data called Dynafleet, and low-resolution so-called Logged Vehicle Data (LVD). The experimental results demonstrate the proposed approach’s effectiveness in predicting the vehicle behavior from low frequency data. With the suggested multi-task snapshot-stacked ensemble deep network, it is shown how low-resolution sensor data can highly contribute to predicting multiple vehicle behaviors simultaneously while using only one single training process. © 2022 The Author(s)
|
|
| 14. |
- Rahat, Mahmoud, 1985-, et al.
(författare)
-
Bridging the Gap: A Comparative Analysis of Regressive Remaining Useful Life Prediction and Survival Analysis Methods for Predictive Maintenance
- 2023
-
Ingår i: Vol. 4 No. 1 (2023): Proceedings of the Asia Pacific Conference of the PHM Society 2023. - New York : Prognostics and Health Management Society.
-
Konferensbidrag (refereegranskat)abstract
- Regressive Remaining Useful Life Prediction and Survival Analysis are two lines of research with similar goals but different origins; one from engineering and the other from survival study in clinical research. Although the two research paths share a common objective of predicting the time to an event, researchers from each path typically do not compare their methods with methods from the other direction. Given the mentioned gap, we propose a framework to compare methods from the two lines of research using run-to-failure datasets. Then by utilizing the proposed framework, we compare six models incorporating three widely recognized degradation models along with two learning algorithms. The first dataset used in this study is C-MAPSS which includes simulation data from aircraft turbofan engines. The second dataset is real-world data from streamed condition monitoring of turbocharger devices installed on a fleet of Volvo trucks.
|
|
| 15. |
- Rahat, Mahmoud, 1985-, et al.
(författare)
-
Domain Adaptation in Predicting Turbocharger Failures Using Vehicle's Sensor Measurements
- 2022
-
Ingår i: PHM Society European Conference. - State College, PA : PHM Society. - 9781936263363 ; , s. 432-439
-
Konferensbidrag (refereegranskat)abstract
- The discrepancy in the distribution of source and target domains is usually referred to as a domain shift. It is one of the reasons for the inferior performance of machine learning solutions at deployment. We illustrate that the domain shift issue is pertinent to the readings of the vehicles’ operational sensors. This is due to the fact that these measurements are collected over a period of time and are susceptible to various changes that happen in the meantime. Examples of these changes are usage pattern variations, aging of the vehicles, seasonal shifts, and driver changes. However, domain adversarial neural networks (DANN) have shown promising results to reduce the negative impact of the domain shift. The present study investigates domain adaptation (DA) in the predictive maintenance field by estimating the remaining useful life (RUL) of turbochargers. The devices are operating on a fleet of VOLVO trucks, and the information about their services is collected over four years between 2016 and 2019. The input features to the model are a set of bi-weekly collected measurements called logged vehicle data (LVD). The contributions of this paper are two-fold. First, we propose a new approach for detecting domain (covariate) shift using an autoencoder. Second, we adapt domain adversarial neural networks to the specific application of predicting turbocharger failures. Finally, we deploy a recurrent feature extraction layer in the DANN architecture to incorporate temporal aspect of the data. The experimental results demonstrate the superiority of the proposed method over the traditional approach.
|
|
| 16. |
- Sheikholharam Mashhadi, Peyman, 1982-, et al.
(författare)
-
Parallel orthogonal deep neural network
- 2021
-
Ingår i: Neural Networks. - Oxford : Elsevier BV. - 0893-6080 .- 1879-2782. ; 140, s. 167-183
-
Tidskriftsartikel (refereegranskat)abstract
- Ensemble learning methods combine multiple models to improve performance by exploiting their diversity. The success of these approaches relies heavily on the dissimilarity of the base models forming the ensemble. This diversity can be achieved in many ways, with well-known examples including bagging and boosting.It is the diversity of the models within an ensemble that allows the ensemble to correct the errors made by its members, and consequently leads to higher classification or regression performance. A mistake made by a base model can only be rectified if other members behave differently on that particular instance, and provide the aggregator with enough information to make an informed decision. On the contrary, lack of diversity not only lowers model performance, but also wastes computational resources. Nevertheless, in the current state of the art ensemble approaches, there is no guarantee on the level of diversity achieved, and no mechanism ensuring that each member will learn a different decision boundary from the others.In this paper, we propose a parallel orthogonal deep learning architecture in which diversity is enforced by design, through imposing an orthogonality constraint. Multiple deep neural networks are created, parallel to each other. At each parallel layer, the outputs of different base models are subject to Gram–Schmidt orthogonalization. We demonstrate that this approach leads to a high level of diversity from two perspectives. First, the models make different errors on different parts of feature space, and second, they exhibit different levels of uncertainty in their decisions. Experimental results confirm the benefits of the proposed method, compared to standard deep learning models and well-known ensemble methods, in terms of diversity and, as a result, classification performance. © 2021 The Author(s). Published by Elsevier Ltd.
|
|
| 17. |
- Sheikholharam Mashhadi, Peyman, 1982-, et al.
(författare)
-
Stacked Ensemble of Recurrent Neural Networks for Predicting Turbocharger Remaining Useful Life
- 2020
-
Ingår i: Applied Sciences. - Basel : MDPI. - 2076-3417. ; 10:1
-
Tidskriftsartikel (refereegranskat)abstract
- Predictive Maintenance (PM) is a proactive maintenance strategy that tries to minimize a system’s downtime by predicting failures before they happen. It uses data from sensors to measure the component’s state of health and make forecasts about its future degradation. However, existing PM methods typically focus on individual measurements. While it is natural to assume that a history of measurements carries more information than a single one. This paper aims at incorporating such information into PM models. In practice, especially in the automotive domain, diagnostic models have low performance, due to a large amount of noise in the data and limited sensing capability. To address this issue, this paper proposes to use a specific type of ensemble learning known as Stacked Ensemble. The idea is to aggregate predictions of multiple models—consisting of Long Short-Term Memory (LSTM) and Convolutional-LSTM—via a meta model, in order to boost performance. Stacked Ensemble model performs well when its base models are as diverse as possible. To this end, each such model is trained using a specific combination of the following three aspects: feature subsets, past dependency horizon, and model architectures. Experimental results demonstrate benefits of the proposed approach on a case study of heavy-duty truck turbochargers. © 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
|
|
| 18. |
- Tashtarian, Farzad, et al.
(författare)
-
Quality Optimization of Live Streaming Services over HTTP with Reinforcement Learning
- 2021
-
Ingår i: 2021 IEEE Global Communications Conference (GLOBECOM). - Piscataway : Institute of Electrical and Electronics Engineers (IEEE). - 9781728181042 ; , s. 1-6
-
Konferensbidrag (refereegranskat)abstract
- Recent years have seen tremendous growth in HTTP adaptive live video traffic over the Internet. In the presence of highly dynamic network conditions and diverse request patterns, existing yet simple hand-crafted heuristic approaches for serving client requests at the network edge might incur a large overhead and significant increase in time complexity. Therefore, these approaches might fail in delivering acceptable Quality of Experience (QoE) to end users. To bridge this gap, we propose ROPL, a learning-based client request management solution at the edge that leverages the power of the recent breakthroughs in deep reinforcement learning, to serve requests of concurrent users joining various HTTP-based live video channels. ROPL is able to react quickly to any changes in the environment, performing accurate decisions to serve clients requests, which results in achieving satisfactory user QoE. We validate the efficiency of ROPL through trace-driven simulations and a real-world setup. Experimental results from real-world scenarios confirm that ROPL outperforms existing heuristic-based approaches in terms of QoE, with a factor up to 3.7×. © 2021 IEEE
|
|
| 19. |
- Uddagiri, Venkata Sai Vivek, et al.
(författare)
-
Predicting hybrid vehicles' fuel and electric consumption using multitask learning
- 2021
-
Ingår i: 2021 IEEE 8th International Conference on Data Science and Advanced Analytics (DSAA). - : IEEE. - 9781665420990 - 9781665421003
-
Konferensbidrag (refereegranskat)abstract
- Predicting energy (fuel and electric) consumption of hybrid vehicles is important on different levels: vehicle industry as a whole, individuals, and can also pave the way towards a more sustainable future. Despite its importance, providing accurate predictions is quite a challenging task. Many essential factors impacting energy consumption, including travel time, average speed, etc., needless to say, these features are not available beforehand. However, these factors are available in our data-set. To use these factors effectively, in this paper, we propose including them as different tasks in a multitask setting to help our main problem of energy consumption. The promise of this approach is that since these tasks are relevant, learning them together would provide a common feature space sharing information about all tasks. More importantly, this shared feature space would carry important information helping energy consumption in particular. In multitask learning, two important issues are task dominance and conflicting gradients of different tasks. Different studies have addressed these two separately. In this paper, we propose a method tackling these two problems simultaneously. We show experimentally the success of this method in comparison to state-of-the-art.
|
|
