| 1. |
- Calikus, Ece, 1990-
(author)
-
Together We Learn More : Algorithms and Applications for User-Centric Anomaly Detection
- 2022
-
Doctoral thesis (other academic/artistic)abstract
- Anomaly detection is the problem of identifying data points or patterns that do not conform to normal behavior. Anomalies in data often correspond to important and actionable information such as frauds in financial applications, faults in production units, intrusions in computer systems, and serious diseases in patient records. One of the fundamental challenges of anomaly detection is that the exact notion of anomaly is subjective and varies greatly in different applications and domains. This makes distinguishing anomalies that match with the end-user's expectations from other observations difficult. As a result, anomaly detectors produce many false alarms that do not correspond to semantically meaningful anomalies for the analyst. Humans can help, in different ways, to bridge this gap between detected anomalies and ''anomalies-of-interest'': by giving clues on features more likely to reveal interesting anomalies or providing feedback to separate them from irrelevant ones. However, it is not realistic to assume a human to easily provide feedback without explaining why the algorithm classifies a certain sample as an anomaly. Interpretability of results is crucial for an analyst to be able to investigate the candidate anomaly and decide whether it is actually interesting or not. In this thesis, we take a step forward to improve the practical use of anomaly detection in real-life by leveraging human-algorithm collaboration. This thesis and appended papers study the problem of formulating and implementing algorithms for user-centric anomaly detection-- a setting in which people analyze, interpret, and learn from the detector's results, as well as provide domain knowledge or feedback. Throughout this thesis, we have described a number of diverse approaches, each addressing different challenges and needs of user-centric anomaly detection in the real world, and combined these methods into a coherent framework. By conducting different studies, this thesis finds that a comprehensive approach incorporating human knowledge and providing interpretable results can lead to more effective and practical anomaly detection and more successful real-world applications. The major contributions that result from the studies included in this work and led the above conclusion can be summarized into five categories: (1) exploring different data representations that are suitable for anomaly detection based on data characteristics and domain knowledge, (2) discovering patterns and groups in data that describe normal behavior in the current application, (3) implementing a generic and extensible framework enabling use-case-specific detectors suitable for different scenarios, (4) incorporating domain knowledge and expert feedback into anomaly detection, and (5) producing interpretable detection results that support end-users in understanding and validating the anomalies.
|
|
| 2. |
- Farouq, Shiraz, 1980-
(author)
-
Towards large-scale monitoring of operationally diverse thermal energy systems with data-driven techniques
- 2019
-
Licentiate thesis (other academic/artistic)abstract
- The core of many typical large-scale industrial infrastructures consists of hundreds or thousands of systems that are similar in their basic design and purpose. For instance, District Heating (DH) utilities rely on a large network of substations to deliver heat to their customers. Similarly, a factory may require a large fleet of specialized robots for manufacturing a certain product. Monitoring these systems is important for maintaining the overall efficiency of industrial operations by detecting various problems due to faults and misconfiguration. However, this can be challenging since a well-understood prior model for each system is rarely available.In most cases, each system in a fleet or network is fitted with a set of sensors to measure its state at different time intervals. Typically, a data-driven model for each system can be used for their monitoring. However, not all factors that can influence the operation of each system in a fleet have an associated sensor. Moreover, sufficient data instances of normal, atypical, and faulty behavior are rarely available to train such a model. These issues can impede the effectiveness of a system-level data-driven model. Alternatively, it can be assumed that since all the systems in a fleet are working on a similar task, they should all behave in a homogeneous manner. Any system that behaves differently from the majority is then considered an outlier. It is referred to as a global or fleet-level model. While the approach is simple, it is less effective in the presence of non-stationary working conditions. Hence, both system-level and fleet-level modeling approaches have their limitations.This thesis investigates system-level and fleet-level models for large-scale monitoring of systems. It proposes to rely on an alternative way, referred to as a reference-group based approach. Herein, the operational monitoring of a target system is delegated to a reference-group, which consists of systems experiencing a comparable operating regime along with the target system. Thus, the definition of a normal, atypical, or faulty operational behavior in a target system is described relative to its reference-group. This definition depends on the choice of the selected anomaly detection model. In this sense, if the target system is not behaving operationally in consort with the systems in its reference-group, then it can be inferred that this is either due to a fault or because of some atypical operation arising at the target system due to its local peculiarities. The application area for these investigations is the large-scale operational monitoring of thermal energy systems: network of DH substations and fleet of heat-pumps.The current findings indicate three advantages of a reference-group based approach. The first is that the reference operational behavior of a target system in the fleet does not need to be predefined. The second is that it provides a basis for what a target system’s operational behavior should have been and what it is. In this respect, each system in the reference-group provides evidence about a particular behavior during a particular period. It can be very useful when the description of a normal, atypical, and faulty operational behavior is not available. The third is that it can detect atypical and faulty operational behavior quickly compared to fleet-level models of anomaly detection.
|
|
| 3. |
- Fan, Yuantao, 1989-
(author)
-
A Self-Organized Fault Detection Method for Vehicle Fleets
- 2016
-
Licentiate thesis (other academic/artistic)abstract
- A fleet of commercial heavy-duty vehicles is a very interesting application arena for fault detection and predictive maintenance. With a highly digitized electronic system and hundreds of sensors mounted on-board a modern bus, a huge amount of data is generated from daily operations.This thesis and appended papers present a study of an autonomous framework for fault detection, using the data gathered from the regular operation of vehicles. We employed an unsupervised deviation detection method, called Consensus Self-Organising Models (COSMO), which is based on the concept of ‘wisdom of the crowd’. It assumes that the majority of the group is ‘healthy’; by comparing individual units within the group, deviations from the majority can be considered as potentially ‘faulty’. Information regarding detected anomalies can be utilized to prevent unplanned stops.This thesis demonstrates how knowledge useful for detecting faults and predicting failures can be autonomously generated based on the COSMO method, using different generic data representations. The case study in this work focuses on vehicle air system problems of a commercial fleet of city buses. We propose an approach to evaluate the COSMO method and show that it is capable of detecting various faults and indicates upcoming air compressor failures. A comparison of the proposed method with an expert knowledge based system shows that both methods perform equally well. The thesis also analyses the usage and potential benefits of using the Echo State Network as a generic data representation for the COSMO method and demonstrates the capability of Echo State Network to capture interesting characteristics in detecting different types of faults.
|
|
| 4. |
- Taghiyarrenani, Zahra, 1987-
(author)
-
Learning from Multiple Domains
- 2022
-
Licentiate thesis (other academic/artistic)abstract
- Domain adaptation (DA) transfers knowledge between domains by adapting them. The most well-known DA scenario in the literature is adapting two domains of source and target using the available labeled source samples to construct a model generalizable to the target domain. Although the primary purpose of DA is to compensate for the target domain’s labeled data shortage, the concept of adaptation can be utilized to solve other problems.One issue that may occur during adaptation is the problem of class misalignment, which would result in a negative transfer. Therefore, preventing negative transfer should be considered while designing DA methods. In addition, the sample availability in domains is another matter that should also be taken into account.Considering the two mentioned matters, this thesis aims to develop DA techniques to solve primary predictive maintenance problems.This thesis considers a spectrum of cases with different amounts of available target data. One endpoint is the case in which we have access to enough labeled target samples for all classes. In this case, we use the concept of DA for 1) Analyzing two different physical properties, i.e., vibration and current, to measure their robustness for fault identification and 2) Developing a denoising method to construct a robust model for a noisy test environment.Next, we consider the case where we have access to unlabeled and a few labeled target samples. Using the few labeled samples available, we aim to prevent negative transfer while adapting source and target domains. To achieve this, we construct a unified features representation using a few-shot and an adaptation learning technique.In the subsequent considered setting, we assume we only have access to very few labeled target samples, which are insufficient to train a domain-specific model. Furthermore, for the first time in the literature, we solve the DA for regression in a setting in which it adapts multiple domains with any arbitrary shift.Sometimes, due to the dynamic nature of the environment, we need to update a model to reflect the changes continuously. An example is in the field of computer network security. There is always the possibility of intrusion into a computer network, which makes each Intrusion Detection System (IDS) subject to concept shifts. In addition, different types of intrusions may occur in different networks. This thesis presents a framework for handling concept shift in one single network through incremental learning and simultaneously adapting samples from different networks to transfer knowledge about various intrusions. In addition, we employ active learning to use expert knowledge to label the samples for the adaptation purpose.During adaptation, all cases mentioned so far have the same label space for the source and target domains. Occasionally, this is not the case, and we do not have access to samples for specific classes, either in the source or target; This is the final scenario addressed in this thesis.One case is when we do not have access to some classes in the source domain. This setting is called Partial Domain Adaptation (PDA). This setting is beneficial to network traffic classification systems because, in general, every network has different types of applications and, therefore, different types of traffic. We develop a method for transferring knowledge from a source network to a target network even if the source network does not contain all types of traffic.Another case is when we have access to unlabeled target samples but not for all classes. We call this Limited Domain Adaptation (LDA) setting and propose a DA method for fault identification. The motivation behind this setting is that for developing a fault identification model for a system, we don’t want to wait until the occurrence of all faults for collecting even unlabeled samples; instead, we aim to use the knowledge about those faults from other domains.We provide results on synthetic and real-world datasets for the scenarios mentioned above. Results indicate that the proposed methods outperform the state-of-art and are effective and practical in solving real-world problems.For future works, we plan to extend the proposed methods to adapt domains with different input features, especially for solving predictive maintenance problems. Furthermore, we intend to extend our work to out-of-distribution learning methods, such as domain generalization.
|
|
| 5. |
- Altarabichi, Mohammed Ghaith, 1981-
(author)
-
Evolving intelligence : Overcoming challenges for Evolutionary Deep Learning
- 2024
-
Doctoral thesis (other academic/artistic)abstract
- Deep Learning (DL) has achieved remarkable results in both academic and industrial fields over the last few years. However, DL models are often hard to design and require proper selection of features and tuning of hyper-parameters to achieve high performance. These selections are tedious for human experts and require substantial time and resources. A difficulty that encouraged a growing number of researchers to use Evolutionary Computation (EC) algorithms to optimize Deep Neural Networks (DNN); a research branch called Evolutionary Deep Learning (EDL).This thesis is a two-fold exploration within the domains of EDL, and more broadly Evolutionary Machine Learning (EML). The first goal is to makeEDL/EML algorithms more practical by reducing the high computational costassociated with EC methods. In particular, we have proposed methods to alleviate the computation burden using approximate models. We show that surrogate-models can speed up EC methods by three times without compromising the quality of the final solutions. Our surrogate-assisted approach allows EC methods to scale better for both, expensive learning algorithms and large datasets with over 100K instances. Our second objective is to leverage EC methods for advancing our understanding of Deep Neural Network (DNN) design. We identify a knowledge gap in DL algorithms and introduce an EC algorithm precisely designed to optimize this uncharted aspect of DL design. Our analytical focus revolves around revealing avant-garde concepts and acquiring novel insights. In our study of randomness techniques in DNN, we offer insights into the design and training of more robust and generalizable neural networks. We also propose, in another study, a novel survival regression loss function discovered based on evolutionary search.
|
|
| 6. |
- Ashfaq, Awais, 1990-
(author)
-
Predicting clinical outcomes via machine learning on electronic health records
- 2019
-
Licentiate thesis (other academic/artistic)abstract
- The rising complexity in healthcare, exacerbated by an ageing population, results in ineffective decision-making leading to detrimental effects on care quality and escalates care costs. Consequently, there is a need for smart decision support systems that can empower clinician's to make better informed care decisions. Decisions, which are not only based on general clinical knowledge and personal experience, but also rest on personalised and precise insights about future patient outcomes. A promising approach is to leverage the ongoing digitization of healthcare that generates unprecedented amounts of clinical data stored in Electronic Health Records (EHRs) and couple it with modern Machine Learning (ML) toolset for clinical decision support, and simultaneously, expand the evidence base of medicine. As promising as it sounds, assimilating complete clinical data that provides a rich perspective of the patient's health state comes with a multitude of data-science challenges that impede efficient learning of ML models. This thesis primarily focuses on learning comprehensive patient representations from EHRs. The key challenges of heterogeneity and temporality in EHR data are addressed using human-derived features appended to contextual embeddings of clinical concepts and Long-Short-Term-Memory networks, respectively. The developed models are empirically evaluated in the context of predicting adverse clinical outcomes such as mortality or hospital readmissions. We also present evidence that, surprisingly, different ML models primarily designed for non-EHR analysis (like language processing and time-series prediction) can be combined and adapted into a single framework to efficiently represent EHR data and predict patient outcomes.
|
|
| 7. |
- Carpatorea, Iulian, 1982-
(author)
-
Methods to quantify and qualify truck driver performance
- 2017
-
Licentiate thesis (other academic/artistic)abstract
- Fuel consumption is a major economical component of vehicles, particularly for heavy-duty vehicles. It is dependent on many factors, such as driver and environment, and control over some factors is present, e.g. route, and we can try to optimize others, e.g. driver. The driver is responsible for around 30% of the operational cost for the fleet operator and is therefore important to have efficient drivers as they also inuence fuel consumption which is another major cost, amounting to around 40% of vehicle operation. The difference between good and bad drivers can be substantial, depending on the environment, experience and other factors.In this thesis, two methods are proposed that aim at quantifying and qualifying driver performance of heavy duty vehicles with respect to fuel consumption. The first method, Fuel under Predefined Conditions (FPC), makes use of domain knowledge in order to incorporate effect of factors which are not measured. Due to the complexity of the vehicles, many factors cannot be quantified precisely or even measured, e.g. wind speed and direction, tire pressure. For FPC to be feasible, several assumptions need to be made regarding unmeasured variables. The effect of said unmeasured variables has to be quantified, which is done by defining specific conditions that enable their estimation. Having calculated the effect of unmeasured variables, the contribution of measured variables can be estimated. All the steps are required to be able to calculate the influence of the driver. The second method, Accelerator Pedal Position - Engine Speed (APPES) seeks to qualify driver performance irrespective of the external factors by analyzing driver intention. APPES is a 2D histogram build from the two mentioned signals. Driver performance is expressed, in this case, using features calculated from APPES.The focus of first method is to quantify fuel consumption, giving us the possibility to estimate driver performance. The second method is more skewed towards qualitative analysis allowing a better understanding of driver decisions and how they affect fuel consumption. Both methods have the ability to give transferable knowledge that can be used to improve driver's performance or automatic driving systems.Throughout the thesis and attached articles we show that both methods are able to operate within the specified conditions and achieve the set goal.
|
|
| 8. |
- Mashad Nemati, Hassan, 1982-
(author)
-
Data analytics for weak spot detection in power distribution grids
- 2019
-
Doctoral thesis (other academic/artistic)abstract
- This research aims to develop data-driven methods that extract information from the available data in distribution grids for detecting weak spots, including the components with degraded reliability and areas with power quality problems. The results enable power distribution companies to change from reactive maintenance to predictive maintenance by deriving benefits from available data. In particular, the data is exploited for three purposes: (a) failure pattern discovery, (b) reliability evaluation of power cables, and (c) analyzing and modeling propagation of power quality disturbances (PQDs) in low-voltage grids.To analyze failure characteristics it is important to discover which failures share common features, e.g., if there are any types of failures that happen mostly in certain parts of the grid or at certain times. This analysis provides information about correlation between different features and identifying the most vulnerable components. In this case, we applied statistical analysis and association rules to discover failure patterns. Furthermore, we propose a visualization of the correlations between different factors representing failures by using an approximated Bayesian network. We show that the Bayesian Network constructed based on the interesting rules of two items is a good approximation of the real dataset.The main focus of reliability evaluation is on failure rate estimation and reliability ranking. In case of power cables, the limited amount of recorded events makes it difficult to perform failure rate modeling. Therefore, we propose a method for interpreting the results of goodness-of-fit measures with confidence intervals, estimated using synthetic data.To perform reliability ranking of power cables, in addition to the age of cables, we consider other factors. Then, we use the proportional hazard model (PHM) to assess the impact of the factors and calculate the failure rate of each individual cable. In reliability evaluation, it is important to consider the fact that power cables are repairable components. We discuss that the conclusions about different factors in PHM and cables ranking will be misleading if one considers the cables as non-repairable components.In low-voltage distribution grids, analyzing PQDs is important as we are moving towards smart grids with the next generation of producers and consumers. Installing Power Quality and Monitoring Systems (PQMS) at all the nodes in the network, for monitoring the impacts of the new consumer/producer, is prohibitively expensive. Instead, we demonstrate that power companies can utilize the available smart meters, which are widely deployed in the low-voltage grids, for monitoring power quality events and identifying areas with power quality problems. In particular, several models for propagation of PQDs, within neighbor customers in different levels of the grid topology, are investigated. The results show that meters data can be used to detect and describe propagation in low-voltage grids.The developed methods of (a) failure pattern discovery are applied on data from Halmstad Energi och Miljö (HEM Nät), Öresundskraft, Göteborg Energy, and Växjö Energy, four different distribution system operators in Sweden. The developed methods of (b) reliability evaluation of power cables and (c) analyzing and modeling propagation of PQDs are applied on data from HEM Nät.
|
|
| 9. |
- Taghiyarrenani, Zahra, 1987-
(author)
-
From Domain Adaptation to Federated Learning
- 2024
-
Doctoral thesis (other academic/artistic)abstract
- Data-driven methods have been gaining increasing attention; however, along with the benefits they offer, they also present several challenges, particularly concerning data availability, accessibility, and heterogeneity, the three factors that have shaped the development of this thesis. Data availability is the primary consideration when employing data-driven methodologies. Suppose we consider a system for which we aim to develop a Machine Learning (ML) model. Gathering labeled samples, particularly in the context of real-world problem-solving, consistently poses challenges. While collecting raw data may be feasible in certain situations, the process of labeling them is often difficult, leading to a shortage of labeled data. However, historical (outdated) data or labeled data may occasionally be available from different yet related systems. A feasible approach would be to leverage data from different but related sources to assist in situations in which data is scarce. The challenge with this approach is that data collected from various sources may exhibit statistical differences even if they have the same features, i.e., data heterogeneity. Data heterogeneity impacts the performance of ML models. This issue arises because conventional machine learning algorithms assume what’s known as the IID (Independently and Identically Distributed) assumption; training and test data come from the same underlying distribution and are independent and identically sampled. The IID assumption may not hold when data comes from different sources and can result in a trained model performing less effectively when used in another system or context. In such situations, Domain Adaptation (DA) is a solution. DA enhances the performance of ML models by minimizing the distribution distance between samples originating from diverse resources. Several factors come into play within the DA context, each necessitating distinct DA methods. In this thesis, we conduct an investigation and propose DA methods while considering various factors, including the number of domains involved, the quantity of data available (both labeled and unlabeled) within these domains, the task at hand (classification or regression), and the nature of statistical heterogeneity among samples from different domains, such as covariate shift or concept shift. It is crucial to emphasize that DA techniques work by assuming that we access the data from different resources. Data may be owned by different data owners, and data owners are willing to share their data. This data accessibility enables us to adapt data and optimize models accordingly. However, privacy concerns become a significant issue when addressing real-world problems, for example, where the data owners are from industry sectors. These privacy considerations necessitate the development of privacy-preserving techniques, such as Federated Learning (FL). FL is a privacy-preserving machine learning technique that enables different data owners to collaborate without sharing raw data samples. Instead, they share their ML models or model updates. Through this collaborative process, a global machine learning model is constructed, which can generalize and perform well across all participating domains. This approach addresses privacy concerns by keeping individual data localized while benefiting from collective knowledge to improve the global model. Among the most widely accepted FL methods is Federated Averaging (FedAvg). In this method, all clients connect with a central server. The server then computes the global model by aggregating the local models from each client, typically by calculating their average. Similar to DA, FL encounters issues when data from different domains exhibit statistical differences, i.e., heterogeneity, that can negatively affect the performance of the global model. A specialized branch known as Heterogeneous FL has emerged to tackle this situation. This thesis, alongside DA, considers the heterogeneous FL problem. This thesis examines FL scenarios where all clients possess labeled data. We begin by conducting experimental investigations to illustrate the impact of various types of heterogeneity on the outcomes of FL. Afterward, we perform a theoretical analysis and establish an upper bound for the risk of the global model for each client. Accordingly, we see that minimizing heterogeneity between the clients minimizes this upper bound. Building upon this insight, we develop a method aimed at minimizing this heterogeneity to personalize the global model for the clients, thereby enhancing the performance of the federated system. This thesis focuses on two practical applications that highlight the relevant challenges: Predictive Maintenance and Network Security. In predictive maintenance, the focus is on fault identification using both DA and FL. Additionally, the thesis investigates predicting the state of health of electric bus batteries using DA. Regarding network security applications, the thesis addresses network traffic classification and intrusion detection, employing DA. ©Zahra Taghiyarrenani.
|
|
| 10. |
- Ashfaq, Awais, 1990-
(author)
-
Deep Evidential Doctor
- 2022
-
Doctoral thesis (other academic/artistic)abstract
- Recent years have witnessed an unparalleled surge in deep neural networks (DNNs) research, surpassing traditional machine learning (ML) and statistical methods on benchmark datasets in computer vision, audio processing and natural language processing (NLP). Much of this success can be attributed to the availability of numerous open-source datasets, advanced computational resources and algorithms. These algorithms learn multiple levels of simple to complex abstractions (or representations) of data resulting in superior performances on downstream applications. This has led to an increasing interest in reaping the potential of DNNs in real-life safety-critical domains such as autonomous driving, security systems and healthcare. Each of them comes with their own set of complexities and requirements, thereby necessitating the development of new approaches to address domain-specific problems, even if building on common foundations.In this thesis, we address data science related challenges involved in learning effective prediction models from structured electronic health records (EHRs). In particular, questions related to numerical representation of complex and heterogeneous clinical concepts, modelling the sequential structure of EHRs and quantifying prediction uncertainties are studied. From a clinical perspective, the question of predicting onset of adverse outcomes for individual patients is considered to enable early interventions, improve patient outcomes, curb unnecessary expenditures and expand clinical knowledge.This is a compilation thesis including five articles. It begins by describing a healthcare information platform that encapsulates clinical, operational and financial data of patients across all public care delivery units in Halland, Sweden. Thus, the platform overcomes the technical and legislative data-related challenges inherent to the modern era's complex and fragmented healthcare sector. The thesis presents evidence that expert clinical features are powerful predictors of adverse patient outcomes. However, they are well complemented by clinical concept embeddings; gleaned via NLP inspired language models. In particular, a novel representation learning framework (KAFE: Knowledge And Frequency adapted Embeddings) has been proposed that leverages medical knowledge schema and adversarial principles to learn high quality embeddings of both frequent and rare clinical concepts. In the context of sequential EHR modelling, benchmark experiments on cost-sensitive recurrent nets have shown significant improvements compared to non-sequential networks. In particular, an attention based hierarchical recurrent net is proposed that represents individual patients as weighted sums of ordered visits, where visits are, in turn, represented as weighted sums of unordered clinical concepts. In the context of uncertainty quantification and building trust in models, the field of deep evidential learning has been extended. In particular for multi-label tasks, simple extensions to current neural network architecture are proposed, coupled with a novel loss criterion to infer prediction uncertainties without compromising on accuracy. Moreover, a qualitative assessment of the model behaviour has also been an important part of the research articles, to analyse the correlations learned by the model in relation to established clinical science.Put together, we develop DEep Evidential Doctor (DEED). DEED is a generic predictive model that learns efficient representations of patients and clinical concepts from EHRs and quantifies its confidence in individual predictions. It is also equipped to infer unseen labels.Overall, this thesis presents a few small steps towards solving the bigger goal of artificial intelligence (AI) in healthcare. The research has consistently shown impressive prediction performance for multiple adverse outcomes. However, we believe that there are numerous emerging challenges to be addressed in order to reap the full benefits of data and AI in healthcare. For future works, we aim to extend the DEED framework to incorporate wider data modalities such as clinical notes, signals and daily lifestyle information. We will also work to equip DEED with explainability features.
|
|
