| 1. |
- Bayram, Firas, et al.
(författare)
-
A domain-region based evaluation of ML performance robustness to covariate shift
- 2023
-
Ingår i: Neural Computing & Applications. - : Springer. - 0941-0643 .- 1433-3058. ; 35:24, s. 17555-17577
-
Tidskriftsartikel (refereegranskat)abstract
- Most machine learning methods assume that the input data distribution is the same in the training and testing phases.However, in practice, this stationarity is usually not met and the distribution of inputs differs, leading to unexpectedperformance of the learned model in deployment. The issue in which the training and test data inputs follow differentprobability distributions while the input–output relationship remains unchanged is referred to as covariate shift. In thispaper, the performance of conventional machine learning models was experimentally evaluated in the presence of covariateshift. Furthermore, a region-based evaluation was performed by decomposing the domain of probability density function ofthe input data to assess the classifier’s performance per domain region. Distributional changes were simulated in a twodimensional classification problem. Subsequently, a higher four-dimensional experiments were conducted. Based on theexperimental analysis, the Random Forests algorithm is the most robust classifier in the two-dimensional case, showing thelowest degradation rate for accuracy and F1-score metrics, with a range between 0.1% and 2.08%. Moreover, the resultsreveal that in higher-dimensional experiments, the performance of the models is predominantly influenced by the complexity of the classification function, leading to degradation rates exceeding 25% in most cases. It is also concluded that themodels exhibit high bias toward the region with high density in the input space domain of the training samples.
|
|
| 2. |
- Bayram, Firas, et al.
(författare)
-
A Drift Handling Approach for Self-Adaptive ML Software in Scalable Industrial Processes
- 2022
-
Ingår i: Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering. - New York, NY, USA : Association for Computing Machinery (ACM). - 9781450394758 ; , s. 1-5
-
Konferensbidrag (refereegranskat)abstract
- Most industrial processes in real-world manufacturing applications are characterized by the scalability property, which requires an automated strategy to self-adapt machine learning (ML) software systems to the new conditions. In this paper, we investigate an Electroslag Remelting (ESR) use case process from the Uddeholms AB steel company. The use case involves predicting the minimum pressure value for a vacuum pumping event. Taking into account the long time required to collect new records and efficiently integrate the new machines with the built ML software system. Additionally, to accommodate the changes and satisfy the non-functional requirement of the software system, namely adaptability, we propose an automated and adaptive approach based on a drift handling technique called importance weighting. The aim is to address the problem of adding a new furnace to production and enable the adaptability attribute of the ML software. The overall results demonstrate the improvements in ML software performance achieved by implementing the proposed approach over the classical non-adaptive approach.
|
|
| 3. |
- Bayram, Firas, et al.
(författare)
-
DA-LSTM: A dynamic drift-adaptive learning framework for interval load forecasting with LSTM networks
- 2023
-
Ingår i: Engineering applications of artificial intelligence. - : Elsevier. - 0952-1976 .- 1873-6769. ; 123
-
Tidskriftsartikel (refereegranskat)abstract
- Load forecasting is a crucial topic in energy management systems (EMS) due to its vital role in optimizing energy scheduling and enabling more flexible and intelligent power grid systems. As a result, these systems allow power utility companies to respond promptly to demands in the electricity market. Deep learning (DL) models have been commonly employed in load forecasting problems supported by adaptation mechanisms to cope with the changing pattern of consumption by customers, known as concept drift. A drift magnitude threshold should be defined to design change detection methods to identify drifts. While the drift magnitude in load forecasting problems can vary significantly over time, existing literature often assumes a fixed drift magnitude threshold, which should be dynamically adjusted rather than fixed during system evolution. To address this gap, in this paper, we propose a dynamic drift-adaptive Long Short-Term Memory (DA-LSTM) framework that can improve the performance of load forecasting models without requiring a drift threshold setting. We integrate several strategies into the framework based on active and passive adaptation approaches. To evaluate DA-LSTM in real-life settings, we thoroughly analyze the proposed framework and deploy it in a real-world problem through a cloud-based environment. Efficiency is evaluated in terms of the prediction performance of each approach and computational cost. The experiments show performance improvements on multiple evaluation metrics achieved by our framework compared to baseline methods from the literature. Finally, we present a trade-off analysis between prediction performance and computational costs.
|
|
| 4. |
- Bayram, Firas, et al.
(författare)
-
DQSOps : Data Quality Scoring Operations Framework for Data-Driven Applications
- 2023
-
Ingår i: EASE '23: Proceedings of the 27<sup>th</sup> International Conference on Evaluation and Assessment in Software Engineering. - : Association for Computing Machinery (ACM). - 9798400700446 ; , s. 32-41
-
Konferensbidrag (refereegranskat)abstract
- Data quality assessment has become a prominent component in the successful execution of complex data-driven artificial intelligence (AI) software systems. In practice, real-world applications generate huge volumes of data at speeds. These data streams require analysis and preprocessing before being permanently stored or used in a learning task. Therefore, significant attention has been paid to the systematic management and construction of high-quality datasets. Nevertheless, managing voluminous and high-velocity data streams is usually performed manually (i.e. offline), making it an impractical strategy in production environments. To address this challenge, DataOps has emerged to achieve life-cycle automation of data processes using DevOps principles. However, determining the data quality based on a fitness scale constitutes a complex task within the framework of DataOps. This paper presents a novel Data Quality Scoring Operations (DQSOps) framework that yields a quality score for production data in DataOps workflows. The framework incorporates two scoring approaches, an ML prediction-based approach that predicts the data quality score and a standard-based approach that periodically produces the ground-truth scores based on assessing several data quality dimensions. We deploy the DQSOps framework in a real-world industrial use case. The results show that DQSOps achieves significant computational speedup rates compared to the conventional approach of data quality scoring while maintaining high prediction performance.
|
|
| 5. |
- Bayram, Firas, et al.
(författare)
-
From concept drift to model degradation : An overview on performance-aware drift detectors
- 2022
-
Ingår i: Knowledge-Based Systems. - : Elsevier BV. - 0950-7051 .- 1872-7409. ; 245
-
Forskningsöversikt (refereegranskat)abstract
- The dynamicity of real-world systems poses a significant challenge to deployed predictive machine learning (ML) models. Changes in the system on which the ML model has been trained may lead to performance degradation during the system’s life cycle. Recent advances that study non-stationary environments have mainly focused on identifying and addressing such changes caused by a phenomenon called concept drift. Different terms have been used in the literature to refer to the same type of concept drift and the same term for various types. This lack of unified terminology is set out to create confusion on distinguishing between different concept drift variants. In this paper, we start by grouping concept drift types by their mathematical definitions and survey the different terms used in the literature to build a consolidated taxonomy of the field. We also review and classify performance-based concept drift detection methods proposed in the last decade. These methods utilize the predictive model’s performance degradation to signal substantial changes in the systems. The classification is outlined in a hierarchical diagram to provide an orderly navigation between the methods. We present a comprehensive analysis of the main attributes and strategies for tracking and evaluating the model’s performance in the predictive system. The paper concludes by discussing open research challenges and possible research directions.
|
|
| 6. |
- Bayram, Firas, 1992-
(författare)
-
Towards Robust and Adaptive Machine Learning : A Fresh Perspective on Evaluation and Adaptation Methodologies in Non-Stationary Environments
- 2023
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Machine learning (ML) has become ubiquitous in various disciplines and applications, serving as a powerful tool for developing predictive models to analyze diverse variables of interest. With the advent of the digital era, the proliferation of data has presented numerous opportunities for growth and expansion across various domains. However, along with these opportunities, there is a unique set of challenges that arises due to the dynamic and ever-changing nature of data. These challenges include concept drift, which refers to shifting data distributions over time, and other data-related issues that can be framed as learning problems. Traditional static models are inadequate in handling these issues, underscoring the need for novel approaches to enhance the performance robustness and reliability of ML models to effectively navigate the inherent non-stationarity in the online world. The field of concept drift is characterized by several intricate aspects that challenge learning algorithms, including the analysis of model performance, which requires evaluating and understanding how the ML model's predictive capability is affected by different problem settings. Additionally, determining the magnitude of drift necessary for change detection is an indispensable task, as it involves identifying substantial shifts in data distributions. Moreover, the integration of adaptive methodologies is essential for updating ML models in response to data dynamics, enabling them to maintain their effectiveness and reliability in evolving environments. In light of the significance and complexity of the topic, this dissertation offers a fresh perspective on the performance robustness and adaptivity of ML models in non-stationary environments. The main contributions of this research include exploring and organizing the literature, analyzing the performance of ML models in the presence of different types of drift, and proposing innovative methodologies for drift detection and adaptation that solve real-world problems. By addressing these challenges, this research paves the way for the development of more robust and adaptive ML solutions capable of thriving in dynamic and evolving data landscapes.
|
|
| 7. |
- Chahed, Hamza, et al.
(författare)
-
AIDA—Aholistic AI-driven networking and processing framework for industrial IoT applications
- 2023
-
Ingår i: Internet of Things. - : Elsevier. - 2542-6605. ; 22
-
Tidskriftsartikel (refereegranskat)abstract
- Industry 4.0 is characterized by digitalized production facilities, where a large volume of sensors collect a vast amount of data that is used to increase the sustainability of the production by e.g. optimizing process parameters, reducing machine downtime and material waste, and the like. However, making intelligent data-driven decisions under timeliness constraints requires the integration of time-sensitive networks with reliable data ingestion and processing infrastructure with plug-in support of Machine Learning (ML) pipelines. However, such integration is difficult due to the lack of frameworks that flexibly integrate and program the networking and computing infrastructures, while allowing ML pipelines to ingest the collected data and make trustworthy decisions in real time. In this paper, we present AIDA - a novel holistic AI-driven network and processing framework for reliable data-driven real-time industrial IoT applications. AIDA manages and configures Time-Sensitive networks (TSN) to enable real-time data ingestion into an observable AI-powered edge/cloud continuum. Pluggable and trustworthy ML components that make timely decisions for various industrial IoT applications and the infrastructure itself are an intrinsic part of AIDA. We introduce the AIDA architecture, demonstrate the building blocks of our framework and illustrate it with two use cases.
|
|
| 8. |
- Samoaa, Hazem Peter, et al.
(författare)
-
A systematic mapping study of source code representation for deep learning in software engineering
- 2022
-
Ingår i: Iet Software. - : Institution of Engineering and Technology (IET). - 1751-8806 .- 1751-8814. ; 16:4, s. 351-385
-
Tidskriftsartikel (refereegranskat)abstract
- The usage of deep learning (DL) approaches for software engineering has attracted much attention, particularly in source code modelling and analysis. However, in order to use DL, source code needs to be formatted to fit the expected input form of DL models. This problem is known as source code representation. Source code can be represented via different approaches, most importantly, the tree-based, token-based, and graph-based approaches. We use a systematic mapping study to investigate i detail the representation approaches adopted in 103 studies that use DL in the context of software engineering. Thus, studies are collected from 2014 to 2021 from 14 different journals and 27 conferences. We show that each way of representing source code can provide a different, yet orthogonal view of the same source code. Thus, different software engineering tasks might require different (combinations of) code representation approaches, depending on the nature and complexity of the task. Particularly, we show that it is crucial to define whether the DL approach requires lexical, syntactical, or semantic code information. Our analysis shows that a wide range of different representations and combinations of representations (hybrid representations) are used to solve a wide range of common software engineering problems. However, we also observe that current research does not generally attempt to transfer existing representations or models to other studies even though there are other contexts in which these representations and models may also be useful. We believe that there is potential for more reuse and the application of transfer learning when applying DL to software engineering tasks.
|
|
