| 1. |
- Zamli, K. Z., et al.
(författare)
-
Hybrid Henry gas solubility optimization algorithm with dynamic cluster-to-algorithm mapping
- 2021
-
Ingår i: Neural Computing & Applications. - : Springer Science+Business Media B.V.. - 0941-0643 .- 1433-3058. ; 33, s. 8389-8416
-
Tidskriftsartikel (refereegranskat)abstract
- This paper discusses a new variant of Henry Gas Solubility Optimization (HGSO) Algorithm, called Hybrid HGSO (HHGSO). Unlike its predecessor, HHGSO allows multiple clusters serving different individual meta-heuristic algorithms (i.e., with its own defined parameters and local best) to coexist within the same population. Exploiting the dynamic cluster-to-algorithm mapping via penalized and reward model with adaptive switching factor, HHGSO offers a novel approach for meta-heuristic hybridization consisting of Jaya Algorithm, Sooty Tern Optimization Algorithm, Butterfly Optimization Algorithm, and Owl Search Algorithm, respectively. The acquired results from the selected two case studies (i.e., involving team formation problem and combinatorial test suite generation) indicate that the hybridization has notably improved the performance of HGSO and gives superior performance against other competing meta-heuristic and hyper-heuristic algorithms.
|
|
| 2. |
- Ahmed, Bestoun S., 1982-, et al.
(författare)
-
A new approach to speed up combinatorial search strategies using stack and hash table
- 2016
-
Ingår i: Proceedings of 2016 SAI Computing Conference, SAI 2016. - : Institute of Electrical and Electronics Engineers (IEEE). - 9781467384605 ; , s. 1217-1222
-
Konferensbidrag (refereegranskat)abstract
- Owing to the significance of combinatorial search strategies both for academia and industry, the introduction of new techniques is a fast growing research field these days. These strategies have really taken different forms ranging from simple to complex strategies in order to solve all forms of combinatorial problems. Nonetheless, despite the kind of problem these approaches solve, they are prone to heavy computation with the number of combinations and growing search space dimensions. This paper presents a new approach to speed up the generation and search processes using a combination of stack and hash table data structures. This approach could be put to practice for the combinatorial approaches to speed up the generation of combinations and search process in the search space. Furthermore, this new approach proved its performance in diverse stages better than other known strategies. © 2016 IEEE.
|
|
| 3. |
- Ahmed, Bestoun S., 1982-, et al.
(författare)
-
An Automated Testing Framework For Smart TVapps Based on Model Separation
- 2020
-
Ingår i: IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW). - : IEEE Computer Society. ; , s. 62-73
-
Konferensbidrag (refereegranskat)abstract
- Smart TV application (app) is a new technological software app that can deal with smart TV devices to add more functionality and features. Despite its importance nowadays, far too little attention has been paid to present a systematic approach to test this kind of app so far. In this paper, we present a systematic model-based testing approach for smart TV app. We used our new notion of model separation to use sub-models based on the user preference instead of the exhaustive testing to generate the test cases. Based on the constructed model, we generated a set of test cases to assess the selected paths to the chosen destination in the app. We also defined new mutation operators for smart TV app to assess our testing approach. The evaluation results showed that our approach can generate more comprehensive models of smart TV apps with less time as compared to manual exploratory testing. The results also showed that our approach can generate effective test cases in term of fault detection.
|
|
| 4. |
- Ahmed, Bestoun S., 1982-, et al.
(författare)
-
An evaluation of Monte Carlo-based hyper-heuristic for interaction testing of industrial embedded software applications
- 2020
-
Ingår i: Soft Computing - A Fusion of Foundations, Methodologies and Applications. - : Springer. - 1432-7643 .- 1433-7479. ; 24:18, s. 13929-13954
-
Tidskriftsartikel (refereegranskat)abstract
- Hyper-heuristic is a new methodology for the adaptive hybridization of meta-heuristic algorithms to derive a general algorithm for solving optimization problems. This work focuses on the selection type of hyper-heuristic, called the exponential Monte Carlo with counter (EMCQ). Current implementations rely on the memory-less selection that can be counterproductive as the selected search operator may not (historically) be the best performing operator for the current search instance. Addressing this issue, we propose to integrate the memory into EMCQ for combinatorial t-wise test suite generation using reinforcement learning based on the Q-learning mechanism, called Q-EMCQ. The limited application of combinatorial test generation on industrial programs can impact the use of such techniques as Q-EMCQ. Thus, there is a need to evaluate this kind of approach against relevant industrial software, with a purpose to show the degree of interaction required to cover the code as well as finding faults. We applied Q-EMCQ on 37 real-world industrial programs written in Function Block Diagram (FBD) language, which is used for developing a train control management system at Bombardier Transportation Sweden AB. The results show that Q-EMCQ is an efficient technique for test case generation. Addition- ally, unlike the t-wise test suite generation, which deals with the minimization problem, we have also subjected Q-EMCQ to a maximization problem involving the general module clustering to demonstrate the effectiveness of our approach. The results show the Q-EMCQ is also capable of outperforming the original EMCQ as well as several recent meta/hyper-heuristic including modified choice function, Tabu high-level hyper-heuristic, teaching learning-based optimization, sine cosine algorithm, and symbiotic optimization search in clustering quality within comparable execution time.
|
|
| 5. |
- Ahmed, Bestoun S., 1982-, et al.
(författare)
-
Aspects of Quality in Internet of Things (IoT) Solutions : A Systematic Mapping Study
- 2019
-
Ingår i: IEEE Access. - : IEEE. - 2169-3536. ; 7, s. 13758-13780
-
Tidskriftsartikel (refereegranskat)abstract
- Internet of Things (IoT) is an emerging technology that has the promising power to change our future. Due to the market pressure, IoT systems may be released without sufficient testing. However, it is no longer acceptable to release IoT systems to the market without assuring the quality. As in the case of new technologies, the quality assurance process is a challenging task. This paper shows the results of the first comprehensive and systematic mapping study to structure and categories the research evidence in the literature starting in 2009 when the early publication of IoT papers for IoT quality assurance appeared. The conducted research is based on the most recent guidelines on how to perform systematic mapping studies. A set of research questions is defined carefully regarding the quality aspects of the IoT. Based on these questions, a large number of evidence and research papers is considered in the study (478 papers). We have extracted and analyzed different levels of information from those considered papers. Also, we have classified the topics addressed in those papers into categories based on the quality aspects. The study results carry out different areas that require more work and investigation in the context of IoT quality assurance. The results of the study can help in a further understanding of the research gaps. Moreover, the results show a roadmap for future research directions.
|
|
| 6. |
- Ahmed, Bestoun S., 1982-, et al.
(författare)
-
Code-Aware Combinatorial Interaction Testing
- 2019
-
Ingår i: IET Software. - London, England : Institution of Engineering and Technology. - 1751-8806 .- 1751-8814. ; 13:6, s. 600-609
-
Tidskriftsartikel (refereegranskat)abstract
- Combinatorial interaction testing (CIT) is a useful testing technique to address the interaction of input parameters in software systems. In many applications, the technique has been used as a systematic sampling technique to sample the enormous possibilities of test cases. In the last decade, most of the research activities focused on the generation of CIT test suites as it is a computationally complex problem. Although promising, less effort has been paid for the application of CIT. In general, to apply the CIT, practitioners must identify the input parameters for the Software-under-test (SUT), feed these parameters to the CIT tool to generate the test suite, and then run those tests on the application with some pass and fail criteria for verification. Using this approach, CIT is used as a black-box testing technique without knowing the effect of the internal code. Although useful, practically, not all the parameters having the same impact on the SUT. This paper introduces a different approach to use the CIT as a gray-box testing technique by considering the internal code structure of the SUT to know the impact of each input parameter and thus use this impact in the test generation stage. We applied our approach to five reliable case studies. The results showed that this approach would help to detect new faults as compared to the equal impact parameter approach.
|
|
| 7. |
- Ahmed, Bestoun S., 1982-, et al.
(författare)
-
Constrained interaction testing : A systematic literature study
- 2017
-
Ingår i: IEEE Access. - Sweden : IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC. - 2169-3536. ; 5, s. 25706-25730
-
Tidskriftsartikel (refereegranskat)abstract
- Interaction testing can be used to effectively detect faults that are otherwise difficult to find by other testing techniques. However, in practice, the input configurations of software systems are subjected to constraints, especially in the case of highly configurable systems. Handling constraints effectively and efficiently in combinatorial interaction testing is a challenging problem. Nevertheless, researchers have attacked this challenge through different techniques, and much progress has been achieved in the past decade. Thus, it is useful to reflect on the current achievements and shortcomings and to identify potential areas of improvements. This paper presents the first comprehensive and systematic literature study to structure and categorize the research contributions for constrained interaction testing. Following the guidelines of conducting a literature study, the relevant data are extracted from a set of 103 research papers belonging to constrained interaction testing. The topics addressed in constrained interaction testing research are classified into four categories of constraint test generation, application, generation and application, and model validation studies. The papers within each of these categories are extensively reviewed. Apart from answering several other research questions, this paper also discusses the applications of constrained interaction testing in several domains, such as software product lines, fault detection and characterization, test selection, security, and graphical user interface testing. This paper ends with a discussion of limitations, challenges, and future work in the area.
|
|
| 8. |
- Ahmed, Bestoun S., 1982-, et al.
(författare)
-
EvoCreeper: Automated Black-Box Model Generation for Smart TV Applications
- 2019
-
Ingår i: IEEE transactions on consumer electronics. - 0098-3063 .- 1558-4127. ; 65:2, s. 160-169
-
Tidskriftsartikel (refereegranskat)abstract
- Abstract—Smart TVs are coming to dominate the televisionmarket. This accompanied by an increase in the use of the smartTV applications (apps). Due to the increasing demand, developersneed modeling techniques to analyze these apps and assess theircomprehensiveness, completeness, and quality. In this paper, wepresent an automated strategy for generating models of smartTV apps based on a black-box reverse engineering. The strategycan be used to cumulatively construct a model for a given app byexploring the user interface in a manner consistent with the use ofa remote control device and extracting the runtime information.The strategy is based on capturing the states of the user interfaceto create a model during runtime without any knowledge ofthe internal structure of the app. We have implemented ourstrategy in a tool called EvoCreeper. The evaluation results showthat our strategy can automatically generate unique states anda comprehensive model that represents the real user interactionswith an app using a remote control device. The models thusgenerated can be used to assess the quality and completeness ofsmart TV apps in various contexts, such as the control of otherconsumer electronics in smart houses.
|
|
| 9. |
- Ahmed, Bestoun S., 1982-, et al.
(författare)
-
Handling constraints in combinatorial interaction testing in the presence of multi objective particle swarm and multithreading
- 2017
-
Ingår i: Information and Software Technology. - : Elsevier. - 0950-5849 .- 1873-6025. ; 86, s. 20-36
-
Tidskriftsartikel (refereegranskat)abstract
- Context: Combinatorial testing strategies have lately received a lot of attention as a result of their diverse applications. In its simple form, a combinatorial strategy can reduce several input parameters (configurations) of a system into a small set based on their interaction (or combination). In practice, the input configurations of software systems are subjected to constraints, especially in case of highly configurable systems. To implement this feature within a strategy, many difficulties arise for construction. While there are many combinatorial interaction testing strategies nowadays, few of them support constraints. Objective: This paper presents a new strategy, to construct combinatorial interaction test suites in the presence of constraints. Method: The design and algorithms are provided in detail. To overcome the multi-judgement criteria for an optimal solution, the multi-objective particle swarm optimisation and multithreading are used. The strategy and its associated algorithms are evaluated extensively using different benchmarks and comparisons. Results: Our results are promising as the evaluation results showed the efficiency and performance of each algorithm in the strategy. The benchmarking results also showed that the strategy can generate constrained test suites efficiently as compared to state-of-the-art strategies. Conclusion: The proposed strategy can form a new way for constructing of constrained combinatorial interaction test suites. The strategy can form a new and effective base for future implementations. (C) 2017 Elsevier B.V. All rights reserved.
|
|
| 10. |
- Ahmed, Bestoun S., 1982-
(författare)
-
Open-source Defect Injection Benchmark Testbed for the Evaluation of Testing
- 2020
-
Ingår i: IEEE 13th International Conference on Software Testing, Validation and Verification (ICST). - : IEEE Computer Society. - 9781728157771 ; , s. 442-447
-
Konferensbidrag (refereegranskat)abstract
- A natural method to evaluate the effectiveness of a testing technique is to measure the defect detection rate when applying the created test cases. Here, real or artificial software defects can be injected into the source code of software. For a more extensive evaluation, injection of artificial defects is usually needed and can be performed via mutation testing using code mutation operators. However, to simulate complex defects arising from a misunderstanding of design specifications, mutation testing might reach its limit in some cases. In this paper, we present an open-source benchmark testbed application that employs a complement method of artificial defect injection. The application is compiled after artificial defects are injected into its source code from predefined building blocks. The majority of the functions and user interface elements are covered by creating front-end-based automated test cases that can be used in experiments.
|
|
| 11. |
- Ahmed, Bestoun S., 1982-, et al.
(författare)
-
Optimum Design of (PID mu)-D-lambda controller for an automatic voltage regulator system using combinatorial test design
- 2016
-
Ingår i: PLOS ONE. - : PUBLIC LIBRARY OF SCIENCE. - 1932-6203. ; 11:11
-
Tidskriftsartikel (refereegranskat)abstract
- Combinatorial test design is a plan of test that aims to reduce the amount of test cases systematically by choosing a subset of the test cases based on the combination of input variables. The subset covers all possible combinations of a given strength and hence tries to match the effectiveness of the exhaustive set. This mechanism of reduction has been used successfully in software testing research with t-way testing (where t indicates the interaction strength of combinations). Potentially, other systems may exhibit many similarities with this approach. Hence, it could form an emerging application in different areas of research due to its usefulness. To this end, more recently it has been applied in a few research areas successfully. In this paper, we explore the applicability of combinatorial test design technique for Fractional Order (FO), Proportional-Integral-Derivative (PID) parameter design controller, named as FOPID, for an automatic voltage regulator (AVR) system. Throughout the paper, we justify this new application theoretically and practically through simulations. In addition, we report on first experiments indicating its practical use in this field. We design different algorithms and adapted other strategies to cover all the combinations with an optimum and effective test set. Our findings indicate that combinatorial test design can find the combinations that lead to optimum design. Besides this, we also found that by increasing the strength of combination, we can approach to the optimum design in a way that with only 4-way combinatorial set, we can get the effectiveness of an exhaustive test set. This significantly reduced the number of tests needed and thus leads to an approach that optimizes design of parameters quickly.
|
|
| 12. |
- Ahmed, Bestoun S., 1982-
(författare)
-
Test case minimization approach using fault detection and combinatorial optimization techniques for configuration-aware structural testing
- 2016
-
Ingår i: ENGINEERING SCIENCE AND TECHNOLOGY-AN INTERNATIONAL JOURNAL-JESTECH. - : Elsevier. - 2215-0986. ; 19:2, s. 737-753
-
Tidskriftsartikel (refereegranskat)abstract
- This paper presents a technique to minimize the number of test cases in configuration-aware structural testing. Combinatorial optimization is used first to generate an optimized test suite by sampling the input configuration. Second, for further optimization, the generated test suite is filtered based on an adaptive mechanism by using a mutation testing technique. The initialized test suite is optimized using cuckoo search (CS) along with combinatorial approach, and mutation testing is used to seed different faults to the software-under-test, as well as to filter the test cases based on the detected faults. To measure the effectiveness of the technique, an empirical study is conducted on a software system. The technique proves its effectiveness through the conducted case study. The paper also shows the application of combinatorial optimization and CS to the software testing. (C) 2016, Karabuk University. Publishing services by Elsevier B.V.
|
|
| 13. |
- Ahmed, Bestoun S., 1982-, et al.
(författare)
-
Testing of Smart TV applications : Key ingredients, challenges and proposed solutions
- 2019
-
Ingår i: Proceedings of the Future Technologies Conference. - Cham : Springer. ; , s. 241-256
-
Konferensbidrag (refereegranskat)abstract
- Smart TV applications are software applications that have been designed to run on smart TVs which are televisions with integrated Internet features. Nowadays, the smart TVs are going to dominate the television market, and the number of connected TVs is growing exponentially. This growth is accompanied by the increase of consumers and the use of smart TV applications that drive these devices. Due to the increasing demand for smart TV applications especially with the rise of the Internet of Things (IoT) services, it is essential to building an application with a certain level of quality. Despite the analogy between the smart TV and mobile apps, testing smart TV applications is different in many aspects due to the different nature of user interaction and development environment. To develop the field and formulate the concepts of smart TV application testing, this paper aims to provide the essential ingredients, solutions, answers to the most critical questions, and open problems. In addition, we offer initial results and proof of concepts for a creeper algorithm to detect essential views of the applications. This paper serves as an effort to report the key ingredients and challenges of the smart TV application testing systematically to the research community. © Springer Nature Switzerland AG 2019.
|
|
| 14. |
- Ahmed, Bestoun S., 1982-, et al.
(författare)
-
Towards an Automated Unified Framework to Run Applications for Combinatorial Interaction Testing
- 2019
-
Ingår i: EASE '19 Proceedings of the Evaluation and Assessment on Software Engineering. - NY, USA : Association for Computing Machinery (ACM). - 9781450371452 ; , s. 252-258
-
Konferensbidrag (refereegranskat)abstract
- Combinatorial interaction testing (CIT) is a well-known technique,but the industrial experience is needed to determine its effectivenessin different application domains. We present a case study introducinga unified framework for generating, executing and verifyingCIT test suites, based on the open-source Avocado test framework.In addition, we present a new industrial case study to demonstratethe effectiveness of the framework. This evaluation showed thatthe new framework can generate, execute, and verify effective combinatorialinteraction test suites for detecting configuration failures(invalid configurations) in a virtualization system.
|
|
| 15. |
- Alsarhan, Qusay, et al.
(författare)
-
Software module clustering : an in-depth literature analysis
- 2022
-
Ingår i: IEEE Transactions on Software Engineering. - : IEEE. - 0098-5589 .- 1939-3520. ; 48:6, s. 1905-1928
-
Tidskriftsartikel (refereegranskat)abstract
- Software module clustering is an unsupervised learning method used to cluster software entities (e.g., classes, modules, or files) of similar features. The obtained clusters may be used to study, analyze, and understand the structure and behavior of the software entities. Implementing software module clustering with optimal results is challenging. Accordingly, researchers have addressed many aspects of software module clustering in the last decade. Thus, it is essential to present research evidence that has been published in this area. In this study, 143 research papers that examined software module clustering from well-known literature databases were extensively reviewed to extract useful data. The obtained data were then used to answer several research questions regarding state-of-the-art clustering approaches, applications of clustering in software engineering, clustering process, clustering algorithms, and evaluation methods. Several research gaps and challenges in software module clustering are discussed in this paper to provide a useful reference for researchers in this field.
|
|
| 16. |
- Bayram, Firas, et al.
(författare)
-
A domain-region based evaluation of ML performance robustness to covariate shift
- 2023
-
Ingår i: Neural Computing & Applications. - : Springer. - 0941-0643 .- 1433-3058. ; 35:24, s. 17555-17577
-
Tidskriftsartikel (refereegranskat)abstract
- Most machine learning methods assume that the input data distribution is the same in the training and testing phases.However, in practice, this stationarity is usually not met and the distribution of inputs differs, leading to unexpectedperformance of the learned model in deployment. The issue in which the training and test data inputs follow differentprobability distributions while the input–output relationship remains unchanged is referred to as covariate shift. In thispaper, the performance of conventional machine learning models was experimentally evaluated in the presence of covariateshift. Furthermore, a region-based evaluation was performed by decomposing the domain of probability density function ofthe input data to assess the classifier’s performance per domain region. Distributional changes were simulated in a twodimensional classification problem. Subsequently, a higher four-dimensional experiments were conducted. Based on theexperimental analysis, the Random Forests algorithm is the most robust classifier in the two-dimensional case, showing thelowest degradation rate for accuracy and F1-score metrics, with a range between 0.1% and 2.08%. Moreover, the resultsreveal that in higher-dimensional experiments, the performance of the models is predominantly influenced by the complexity of the classification function, leading to degradation rates exceeding 25% in most cases. It is also concluded that themodels exhibit high bias toward the region with high density in the input space domain of the training samples.
|
|
| 17. |
- Bayram, Firas, et al.
(författare)
-
A Drift Handling Approach for Self-Adaptive ML Software in Scalable Industrial Processes
- 2022
-
Ingår i: Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering. - New York, NY, USA : Association for Computing Machinery (ACM). - 9781450394758 ; , s. 1-5
-
Konferensbidrag (refereegranskat)abstract
- Most industrial processes in real-world manufacturing applications are characterized by the scalability property, which requires an automated strategy to self-adapt machine learning (ML) software systems to the new conditions. In this paper, we investigate an Electroslag Remelting (ESR) use case process from the Uddeholms AB steel company. The use case involves predicting the minimum pressure value for a vacuum pumping event. Taking into account the long time required to collect new records and efficiently integrate the new machines with the built ML software system. Additionally, to accommodate the changes and satisfy the non-functional requirement of the software system, namely adaptability, we propose an automated and adaptive approach based on a drift handling technique called importance weighting. The aim is to address the problem of adding a new furnace to production and enable the adaptability attribute of the ML software. The overall results demonstrate the improvements in ML software performance achieved by implementing the proposed approach over the classical non-adaptive approach.
|
|
| 18. |
- Bayram, Firas, et al.
(författare)
-
DA-LSTM: A dynamic drift-adaptive learning framework for interval load forecasting with LSTM networks
- 2023
-
Ingår i: Engineering applications of artificial intelligence. - : Elsevier. - 0952-1976 .- 1873-6769. ; 123
-
Tidskriftsartikel (refereegranskat)abstract
- Load forecasting is a crucial topic in energy management systems (EMS) due to its vital role in optimizing energy scheduling and enabling more flexible and intelligent power grid systems. As a result, these systems allow power utility companies to respond promptly to demands in the electricity market. Deep learning (DL) models have been commonly employed in load forecasting problems supported by adaptation mechanisms to cope with the changing pattern of consumption by customers, known as concept drift. A drift magnitude threshold should be defined to design change detection methods to identify drifts. While the drift magnitude in load forecasting problems can vary significantly over time, existing literature often assumes a fixed drift magnitude threshold, which should be dynamically adjusted rather than fixed during system evolution. To address this gap, in this paper, we propose a dynamic drift-adaptive Long Short-Term Memory (DA-LSTM) framework that can improve the performance of load forecasting models without requiring a drift threshold setting. We integrate several strategies into the framework based on active and passive adaptation approaches. To evaluate DA-LSTM in real-life settings, we thoroughly analyze the proposed framework and deploy it in a real-world problem through a cloud-based environment. Efficiency is evaluated in terms of the prediction performance of each approach and computational cost. The experiments show performance improvements on multiple evaluation metrics achieved by our framework compared to baseline methods from the literature. Finally, we present a trade-off analysis between prediction performance and computational costs.
|
|
| 19. |
- Bayram, Firas, et al.
(författare)
-
DQSOps : Data Quality Scoring Operations Framework for Data-Driven Applications
- 2023
-
Ingår i: EASE '23: Proceedings of the 27<sup>th</sup> International Conference on Evaluation and Assessment in Software Engineering. - : Association for Computing Machinery (ACM). - 9798400700446 ; , s. 32-41
-
Konferensbidrag (refereegranskat)abstract
- Data quality assessment has become a prominent component in the successful execution of complex data-driven artificial intelligence (AI) software systems. In practice, real-world applications generate huge volumes of data at speeds. These data streams require analysis and preprocessing before being permanently stored or used in a learning task. Therefore, significant attention has been paid to the systematic management and construction of high-quality datasets. Nevertheless, managing voluminous and high-velocity data streams is usually performed manually (i.e. offline), making it an impractical strategy in production environments. To address this challenge, DataOps has emerged to achieve life-cycle automation of data processes using DevOps principles. However, determining the data quality based on a fitness scale constitutes a complex task within the framework of DataOps. This paper presents a novel Data Quality Scoring Operations (DQSOps) framework that yields a quality score for production data in DataOps workflows. The framework incorporates two scoring approaches, an ML prediction-based approach that predicts the data quality score and a standard-based approach that periodically produces the ground-truth scores based on assessing several data quality dimensions. We deploy the DQSOps framework in a real-world industrial use case. The results show that DQSOps achieves significant computational speedup rates compared to the conventional approach of data quality scoring while maintaining high prediction performance.
|
|
| 20. |
- Bayram, Firas, et al.
(författare)
-
From concept drift to model degradation : An overview on performance-aware drift detectors
- 2022
-
Ingår i: Knowledge-Based Systems. - : Elsevier BV. - 0950-7051 .- 1872-7409. ; 245
-
Forskningsöversikt (refereegranskat)abstract
- The dynamicity of real-world systems poses a significant challenge to deployed predictive machine learning (ML) models. Changes in the system on which the ML model has been trained may lead to performance degradation during the system’s life cycle. Recent advances that study non-stationary environments have mainly focused on identifying and addressing such changes caused by a phenomenon called concept drift. Different terms have been used in the literature to refer to the same type of concept drift and the same term for various types. This lack of unified terminology is set out to create confusion on distinguishing between different concept drift variants. In this paper, we start by grouping concept drift types by their mathematical definitions and survey the different terms used in the literature to build a consolidated taxonomy of the field. We also review and classify performance-based concept drift detection methods proposed in the last decade. These methods utilize the predictive model’s performance degradation to signal substantial changes in the systems. The classification is outlined in a hierarchical diagram to provide an orderly navigation between the methods. We present a comprehensive analysis of the main attributes and strategies for tracking and evaluating the model’s performance in the predictive system. The paper concludes by discussing open research challenges and possible research directions.
|
|
| 21. |
- Bayram, Firas, 1992-
(författare)
-
Towards Robust and Adaptive Machine Learning : A Fresh Perspective on Evaluation and Adaptation Methodologies in Non-Stationary Environments
- 2023
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Machine learning (ML) has become ubiquitous in various disciplines and applications, serving as a powerful tool for developing predictive models to analyze diverse variables of interest. With the advent of the digital era, the proliferation of data has presented numerous opportunities for growth and expansion across various domains. However, along with these opportunities, there is a unique set of challenges that arises due to the dynamic and ever-changing nature of data. These challenges include concept drift, which refers to shifting data distributions over time, and other data-related issues that can be framed as learning problems. Traditional static models are inadequate in handling these issues, underscoring the need for novel approaches to enhance the performance robustness and reliability of ML models to effectively navigate the inherent non-stationarity in the online world. The field of concept drift is characterized by several intricate aspects that challenge learning algorithms, including the analysis of model performance, which requires evaluating and understanding how the ML model's predictive capability is affected by different problem settings. Additionally, determining the magnitude of drift necessary for change detection is an indispensable task, as it involves identifying substantial shifts in data distributions. Moreover, the integration of adaptive methodologies is essential for updating ML models in response to data dynamics, enabling them to maintain their effectiveness and reliability in evolving environments. In light of the significance and complexity of the topic, this dissertation offers a fresh perspective on the performance robustness and adaptivity of ML models in non-stationary environments. The main contributions of this research include exploring and organizing the literature, analyzing the performance of ML models in the presence of different types of drift, and proposing innovative methodologies for drift detection and adaptation that solve real-world problems. By addressing these challenges, this research paves the way for the development of more robust and adaptive ML solutions capable of thriving in dynamic and evolving data landscapes.
|
|
| 22. |
- Bures, Miroslav, et al.
(författare)
-
Employment of multiple algorithms for optimal path-based test selection strategy
- 2019
-
Ingår i: Information and Software Technology. - : Elsevier. - 0950-5849 .- 1873-6025. ; 114, s. 21-36
-
Tidskriftsartikel (refereegranskat)abstract
- ContextExecuting various sequences of system functions in a system under test represents one of the primary techniques in software testing. The natural method for creating effective, consistent and efficient test sequences is to model the system under test and employ an algorithm to generate tests that satisfy a defined test coverage criterion. Several criteria for preferred test set properties can be defined. In addition, to optimize the test set from an economic viewpoint, the priorities of the various parts of the system model under test must be defined.ObjectiveUsing this prioritization, the test cases exercise the high-priority parts of the system under test by more path combinations than those with low priority (this prioritization can be combined with the test coverage criterion that determines how many path combinations of the individual parts of the system are tested). Evidence from the literature and our observations confirm that finding a universal algorithm that produces a test set with preferred properties for all test coverage criteria is a challenging task. Moreover, for different individual problem instances, different algorithms provide results with the best value of a preferred property. In this paper, we present a portfolio-based strategy to perform the best test selection.MethodThe proposed strategy first employs a set of current algorithms to generate test sets; then, a preferred property of each test set is assessed in terms of the selected criterion, and finally, the test set with the best value of a preferred property is chosen.ResultsThe experimental results confirm the validity and usefulness of this strategy. For individual instances of 50 system under test models, different algorithms provided results having the best preferred property value; these results varied by the required test coverage level, the size of the priority parts of the model, and the selected test set preferred property criteria.ConclusionIn addition to the used algorithms, the proposed strategy can be used to assess the optimality of different path-based testing algorithms and choose a suitable algorithm for the testing.
|
|
| 23. |
- Bures, M., et al.
(författare)
-
Internet of things : Current challenges in the quality assurance and testing methods
- 2019
-
Ingår i: Information Science and Applications 2018. ICISA. - Singapore : Springer Singapore. - 9789811310553 ; , s. 625-634
-
Konferensbidrag (refereegranskat)abstract
- Contemporary development of the Internet of Things (IoT) technology brings a number of challenges in the Quality Assurance area. Current issues related to security, user’s privacy, the reliability of the service, interoperability, and integration are discussed. All these create a demand for specific Quality Assurance methodology for the IoT solutions. In the paper, we present the state of the art of this domain and we discuss particular areas of system testing discipline, which is not covered by related work sufficiently so far. This analysis is supported by results of a recent survey we performed among ten IoT solutions providers, covering various areas of IoT applications. © 2019, Springer Nature Singapore Pte Ltd.
|
|
| 24. |
- Bures, M., et al.
(författare)
-
Interoperability and Integration Testing Methods for IoT Systems : A Systematic Mapping Study
- 2020
-
Ingår i: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). - Cham : Springer Science+Business Media B.V.. - 9783030587673 ; , s. 93-112
-
Konferensbidrag (refereegranskat)abstract
- The recent active development of Internet of Things (IoT) solutions in various domains has led to an increased demand for security, safety, and reliability of these systems. Security and data privacy are currently the most frequently discussed topics; however, other reliability aspects also need to be focused on to maintain smooth and safe operation of IoT systems. Until now, there has been no systematic mapping study dedicated to the topic of interoperability and integration testing of IoT systems specifically; therefore, we present such an overview in this study. We analyze 803 papers from four major primary databases and perform detailed assessment and quality check to find 115 relevant papers. In addition, recently published testing techniques and approaches are analyzed and classified; the challenges and limitations in the field are also identified and discussed. Research trends related to publication time, active researchers, and publication media are presented in this study. The results suggest that studies mainly focus only on general testing methods, which can be applied to integration and interoperability testing of IoT systems; thus, there are research opportunities to develop additional testing methods focused specifically on IoT systems, so that they are more effective in the IoT context.
|
|
| 25. |
- Bures, M., et al.
(författare)
-
On the Effectiveness of Combinatorial Interaction Testing : A Case Study
- 2017
-
Ingår i: Proceedings - 2017 IEEE International Conference on Software Quality, Reliability and Security Companion, QRS-C 2017. - 9781538620724 ; , s. 69-76
-
Konferensbidrag (refereegranskat)abstract
- Combinatorial interaction testing (CIT) stands as one of the efficient testing techniques that have been used in different applications recently. The technique is useful when there is a need to take the interaction of input parameters into consideration for testing a system. The key insight the technique is that not every single parameter may contribute to the failure of the system and there could be interactions among these parameters. Hence, there must be combinations of these input parameters based on the interaction strength. This technique has been used in many applications to assess its effectiveness. In this paper, we are addressing the effectiveness of CIT for a real-world case study using model-based mutation testing experiments. The contribution of the paper is threefold: First we introduce an effective testing application for CIT; Second, we address the effectiveness of increasing the interaction strength beyond the pairwise (i.e., interaction of more than two parameters); Third, model-based mutation testing is used to mutate the input model of the program in contrast to the traditional code-based mutation testing process. Experimental results showed that CIT is an effective testing technique for this kind of application. In addition, the results also showed the usefulness of model-based mutation testing to assess CIT applications. For the subject of this case study, the results also indicate that 3-way test suite (i.e., interaction of three parameters) could detect new faults that can not be detected by pairwise. © 2017 IEEE.
|
|
