| 1. |
- Ali, Nauman bin, et al.
(författare)
-
On the search for industry-relevant regression testing research
- 2019
-
Ingår i: Empirical Software Engineering. - New York, NY : Springer. - 1382-3256 .- 1573-7616. ; 24:4, s. 2020-2055
-
Tidskriftsartikel (refereegranskat)abstract
- Regression testing is a means to assure that a change in the software, or its execution environment, does not introduce new defects. It involves the expensive undertaking of rerunning test cases. Several techniques have been proposed to reduce the number of test cases to execute in regression testing, however, there is no research on how to assess industrial relevance and applicability of such techniques. We conducted a systematic literature review with the following two goals: firstly, to enable researchers to design and present regression testing research with a focus on industrial relevance and applicability and secondly, to facilitate the industrial adoption of such research by addressing the attributes of concern from the practitioners' perspective. Using a reference-based search approach, we identified 1068 papers on regression testing. We then reduced the scope to only include papers with explicit discussions about relevance and applicability (i.e. mainly studies involving industrial stakeholders). Uniquely in this literature review, practitioners were consulted at several steps to increase the likelihood of achieving our aim of identifying factors important for relevance and applicability. We have summarised the results of these consultations and an analysis of the literature in three taxonomies, which capture aspects of industrial-relevance regarding the regression testing techniques. Based on these taxonomies, we mapped 38 papers reporting the evaluation of 26 regression testing techniques in industrial settings. © The Author(s) 2019
|
|
| 2. |
- Beohar, Harsh, 1984-, et al.
(författare)
-
Basic behavioral models for software product lines : Expressiveness and testing pre-orders
- 2016
-
Ingår i: Science of Computer Programming. - Amsterdam : Elsevier. - 0167-6423 .- 1872-7964. ; 123, s. 42-60
-
Tidskriftsartikel (refereegranskat)abstract
- In order to provide a rigorous foundation for Software Product Lines (SPLs), several fundamental approaches have been proposed to their formal behavioral modeling. In this paper, we provide a structured overview of those formalisms based on labeled transition systems and compare their expressiveness in terms of the set of products they can specify. Moreover, we define the notion of tests for each of these formalisms and show that our notions of testing precisely capture product derivation, i.e., all valid products will pass the set of test cases of the product line and each invalid product fails at least one test case of the product line. © 2015 The Authors.
|
|
| 3. |
- Costa Paiva, Sofia, et al.
(författare)
-
Complete IOCO Test Cases : A Case Study
- 2016
-
Ingår i: A-TEST 2016 - Proceedings of the 7th International Workshop on Automating Test Case Design, Selection, and Evaluation, co-located with FSE 2016. - New York, NY : ACM Press. - 9781450344012 ; , s. 38-44
-
Konferensbidrag (refereegranskat)abstract
- Input/Output Transition Systems (IOTSs) have been widely used as test models in model-based testing. Traditionally, input output conformance testing (IOCO) has been used to generate random test cases from IOTSs. A recent test case generation method for IOTSs, called Complete IOCO, applies fault models to obtain complete test suites with guaranteed fault coverage for IOTSs. This paper measures the efficiency of Complete IOCO in comparison with the traditional IOCO test case generation implemented in the JTorX tool. To this end, we use a case study involving five specification models from the automotive and the railway domains. Faulty mutations of the specifications were produced in order to compare the efficiency of both test generation methods in killing them. The results indicate that Complete IOCO is more efficient in detecting deep faults in large state spaces while IOCO is more efficient in detecting shallow faults in small state spaces. © 2016 ACM.
|
|
| 4. |
- Kunze, Sebastian, 1990-, et al.
(författare)
-
Generation of Failure Models through Automata Learning
- 2016
-
Ingår i: Proceedings. - Los Alamitos : IEEE Computer Society. - 9781509025718 ; , s. 22-25
-
Konferensbidrag (refereegranskat)abstract
- In the context of the AUTO-CAAS project that deals with model-based testing techniques applied in the automotive domain, we present the preliminary ideas and results of building generalised failure models for non-conformant software components. These models are a necessary building block for our upcoming efforts to detect and analyse failure causes in automotive software built with AUTOSAR components. Concretely, we discuss how to build these generalised failure models using automata learning techniques applied to a guided model-based testing procedure of a failing component. We illustrate our preliminary findings and experiments on a simple integer queue implemented in the C programming language. © 2016 IEEE.
|
|
| 5. |
- Varshosaz, Mahsa, 1985-, et al.
(författare)
-
A classification of product sampling for software product lines
- 2018
-
Ingår i: SPLC '18 Proceedings of the 22nd International Systems and Software Product Line Conference - Volume 1. - New York, NY : Association for Computing Machinery (ACM). - 9781450363716 - 9781450364645 ; , s. 1-13
-
Konferensbidrag (refereegranskat)abstract
- The analysis of software product lines is challenging due to the potentially large number of products, which grow exponentially in terms of the number of features. Product sampling is a technique used to avoid exhaustive testing, which is often infeasible. In this paper, we propose a classification for product sampling techniques and classify the existing literature accordingly. We distinguish the important characteristics of such approaches based on the information used for sampling, the kind of algorithm, and the achieved coverage criteria. Furthermore, we give an overview on existing tools and evaluations of product sampling techniques. We share our insights on the state-of-the-art of product sampling and discuss potential future work. © 2018 Association for Computing Machinery.
|
|
| 6. |
- Varshosaz, Mahsa, 1985-, et al.
(författare)
-
Basic Behavioral Models For Software Product Lines : Revisited
- 2018
-
Ingår i: Science of Computer Programming. - Amsterdam : Elsevier. - 0167-6423 .- 1872-7964. ; 168, s. 171-185
-
Tidskriftsartikel (refereegranskat)abstract
- In Beohar et al. (2016) [9], we established an expressiveness hierarchy and studied the notions of refinement and testing for three fundamental behavioral models for software product lines. These models were featured transition systems, product line labeled transition systems, and modal transition systems. It turns out that our definition of product line labeled transition systems is more restrictive than the one introduced by Gruler, Leucker, and Scheidemann. Adopting the original and more liberal notion changes the expressiveness results, as we demonstrate in this paper. Namely, we show that the original notion of product line labeled transition systems and featured transition systems are equally expressive. As an additional result, we show that there are featured transition systems for which the size of the corresponding product line labeled transition system, resulting from any sound encoding, is exponentially larger than the size of the original model. Furthermore, we show that each product line labeled transition system can be encoded into a featured transition system, such that the size of featured transition system is linear in terms of the size of the corresponding model. To summarize, featured transition systems are equally expressive as, but exponentially more succinct than, product line labeled transition systems. © 2018 Elsevier B.V.
|
|
| 7. |
- Varshosaz, Mahsa, 1985-, et al.
(författare)
-
Delta-Oriented FSM-Based Testing
- 2015
-
Ingår i: Formal Methods and Software Engineering. - Cham : Springer. - 9783319254227 - 9783319254234 ; , s. 366-381
-
Konferensbidrag (refereegranskat)abstract
- We use the concept of delta-oriented programming to organize FSM-based test models in an incremental structure. We then exploit incremental FSM-based testing to make efficient use of this high-level structure in generating test cases. We show how our approach can lead to more efficient test-case generation, both by analyzing the complexity of the test-case generation algorithm and by applying the technique to a case study. © Springer International Publishing Switzerland 2015
|
|
| 8. |
- Varshosaz, Mahsa, 1985-
(författare)
-
Modeling and Model-Based Testing of Software Product Lines
- 2019
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Software product line (SPL) engineering has become common practice for mass production and customization of variability intensive systems. A software product line comprises a family of software systems which share a managed core set of artifacts and also have a set of well-defined variabilities. The main idea in SPL engineering is to enable systematic reuse in different phases of software development to reduce cost and time to release.Model-Based Testing (MBT) is a technique that is widely used for quality assurance of software systems. In MBT, an abstract model, which captures the desired behavior of the system, is used to generate test cases. The test cases are executed against a real implementation of the system and the conformance between the implementation and the specification is checked by comparing the observed outputs with the ones prescribed by the model.Software product lines have been applied in a number of domains with mission critical systems. MBT is one of the techniques that has been used for analysis of such systems. As the number of products can be potentially large in an SPL, using conventional approaches for MBT of the products of an SPL individually can be very costly and time consuming. To tackle this problem, several approaches have been proposed in order to enable systematic reuse in different phases of the MBT process.An efficient modeling technique is the first step towards an efficient MBT technique for SPLs. So far, several formalisms have been proposed for modeling SPLs. In this thesis, we conduct a study on such modeling techniques, focusing on four fundamental formalisms, namely featured transition systems, modal transition systems, product line calculus of communicating systems, and 1- selecting modal transition systems. We compare the expressive power and the succinctness of these formalisms.Furthermore, we investigate adapting existing MBT methods for efficient testing of SPLs. As a part of this line of our research, we adapt the test case generation algorithm of one of the well-known black-box testing approaches, namely, Harmonized State Identification (HSI) method by exploiting the idea of delta-oriented programming. We apply the adapted test case generation algorithm to a case study taken from industry and the results show up to 50 percent reduction of time in test case generation by using the delta-oriented HSI method.In line with our research on investigating existing MBT techniques, we compare the relative efficiency and effectiveness of the test case generation algorithms of the well-known Input-Output Conformance (ioco) testing approach and the complete ioco which is another testing technique used for input output transition systems that guarantees fault coverage. The comparison is done using three case studies taken from the automotive and railway domains. The obtained results show that complete ioco is more efficient in detecting deep faults (i.e., the faults reached through longer traces) in large state spaces while ioco is more efficient in detecting shallow faults (i.e., the faults reached through shorter traces) in small state spaces.Moreover, we conduct a survey on sampling techniques, which have been proposed as a solution for handling the large number of products in analysis. In general, in product sampling a subset of products that collectively cover the behavior of the product line are selected. Performing tests on well selected sample set can reveal most of the faults in all products. We provide a classification for a catalog of studies on product sampling for software product lines. Additionally, we present a number of insights on the studied work as well as gaps for the future research.
|
|
