| 1. |
- Edvardsson, Jon, 1973-
(författare)
-
Techniques for Automatic Generation of Tests from Programs and Specifications
- 2006
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Software testing is complex and time consuming. One way to reduce the effort associated with testing is to generate test data automatically. This thesis is divided into three parts. In the first part a mixed-integer constraint solver developed by Gupta et. al is studied. The solver, referred to as the Unified Numerical Approach (una), is an important part of their generator and it is responsible for solving equation systems that correspond to the program path currently under test.In this thesis it is shown that, in contrast to traditional optimization methods, the una is not bounded by the size of the solved equation system. Instead, it depends on how the system is composed. That is, even for very simple systems consisting of one variable we can easily get more than a thousand iterations. It is also shown that the una is not complete, that is, it does not always find a mixed-integer solution when there is one. It is found that a better approach is to use a traditional optimization method, like the simplex method in combination with branch-and-bound and/or a cutting-plane algorithm as a constraint solver.The second part explores a specification-based approach for generating tests developed by Meudec. Tests are generated by partitioning the specification input domain into a set of subdomains using a rule-based automatic partitioning strategy. An important step of Meudec’s method is to reduce the number of generated subdomains and find a minimal partition. This thesis shows that Meudec’s minimal partition algorithmis incorrect. Furthermore, two new efficient alternative algorithms are developed. In addition, an algorithm for finding the upper and lower bound on the number of subdomains in a partition is also presented.Finally, in the third part, two different designs of automatic testing tools are studied. The first tool uses a specification as an oracle. The second tool, on the other hand, uses a reference program. The fault-detection effectiveness of the tools is evaluated using both randomly and systematically generated inputs.
|
|
| 2. |
- Eriksson, Anders, et al.
(författare)
-
Model transformation impact on test artifacts : An empirical study
- 2012
-
Ingår i: Proceedings of the Workshop on Model-Driven Engineering, Verification and Validation, MoDeVVa 2012. - New York, NY, USA : Association for Computing Machinery (ACM). - 9781450318013 ; , s. 5-10
-
Konferensbidrag (refereegranskat)abstract
- Development environments that support Model-Driven Development often focus on model-level functional testing, enabling verification of design models against their specifications. However, developers of safety-critical software systems are also required to show that tests cover the structure of the implementation. Unfortunately, the implementation structure can diverge from the model depending on choices such as the model compiler or target language. Therefore, structural coverage at the model level may not guarantee coverage of the implementation. We present results from an industrial experiment that demonstrates the model-compiler effect on test artifacts in xtUML models when these models are transformed into C++. Test artifacts, i.e., predicates and clauses, are used to satisfy the structural code coverage criterion, in this case MCDC, which is required by the US Federal Aviation Administration. The results of the experiment show not only that the implementation contains more test artifacts than the model, but also that the test artifacts can be deterministically enumerated during translation. The analysis identifies two major sources for these additional test artifacts. © 2012 ACM.
|
|
| 3. |
- Eriksson, Anders, et al.
(författare)
-
Transformation rules for platform independent testing : An empirical study
- 2013
-
Ingår i: Proceedings of the Sixth IEEE International Conference on Software Testing, Verification and Validation, ICST 2013. - : IEEE conference proceedings. - 9781467359610 - 9780769549682 ; , s. 202-211
-
Konferensbidrag (refereegranskat)abstract
- Most Model-Driven Development projects focus on model-level functional testing. However, our recent study found an average of 67% additional logic-based test requirements from the code compared to the design model. The fact that full coverage at the design model level does not guarantee full coverage at the code level indicates that there are semantic behaviors in the model that model-based tests might miss, e.g., conditional behaviors that are not explicitly expressed as predicates and therefore not tested by logic-based coverage criteria. Avionics standards require that the structure of safety critical software is covered according to logic-based coverage criteria, including MCDC for the highest safety level. However, the standards also require that each test must be derived from the requirements. This combination makes designing tests hard, time-consuming and expensive to design. This paper defines a new model that uses transformation rules to help testers define tests at the platform independent model level. The transformation rules have been applied to six large avionic applications. The results show that the new model reduced the difference between model and code with respect to the number of additional test requirements from an average of67% to 0% in most cases and less than 1% for all applications. © 2013 IEEE.
|
|
| 4. |
- Ferrari, Fabiano C., et al.
(författare)
-
On transforming model-based tests into code : A systematic literature review
- 2023
-
Ingår i: Software testing, verification & reliability. - : John Wiley & Sons. - 0960-0833 .- 1099-1689. ; 33:8
-
Forskningsöversikt (refereegranskat)abstract
- Model-based test design is increasingly being applied in practice and studied in research. Model-based testing (MBT) exploits abstract models of the software behaviour to generate abstract tests, which are then transformed into concrete tests ready to run on the code. Given that abstract tests are designed to cover models but are run on code (after transformation), the effectiveness of MBT is dependent on whether model coverage also ensures coverage of key functional code. In this article, we investigate how MBT approaches generate tests from model specifications and how the coverage of tests designed strictly based on the model translates to code coverage. We used snowballing to conduct a systematic literature review. We started with three primary studies, which we refer to as the initial seeds. At the end of our search iterations, we analysed 30 studies that helped answer our research questions. More specifically, this article characterizes how test sets generated at the model level are mapped and applied to the source code level, discusses how tests are generated from the model specifications, analyses how the test coverage of models relates to the test coverage of the code when the same test set is executed and identifies the technologies and software development tasks that are on focus in the selected studies. Finally, we identify common characteristics and limitations that impact the research and practice of MBT: (i) some studies did not fully describe how tools transform abstract tests into concrete tests, (ii) some studies overlooked the computational cost of model-based approaches and (iii) some studies found evidence that bears out a robust correlation between decision coverage at the model level and branch coverage at the code level. We also noted that most primary studies omitted essential details about the experiments.
|
|
| 5. |
- González-Hernández, Loreto, et al.
(författare)
-
Using Mutant Stubbornness to Create Minimal and Prioritized Test Sets
- 2018
-
Ingår i: 2018 IEEE International Conference on Software Quality, Reliability and Security (QRS). - : IEEE Computer Society. - 9781538677575 - 9781538677582 ; , s. 446-457
-
Konferensbidrag (refereegranskat)abstract
- In testing, engineers want to run the most useful tests early (prioritization). When tests are run hundreds or thousands of times, minimizing a test set can result in significant savings (minimization). This paper proposes a new analysis technique to address both the minimal test set and the test case prioritization problems. This paper precisely defines the concept of mutant stubbornness, which is the basis for our analysis technique. We empirically compare our technique with other test case minimization and prioritization techniques in terms of the size of the minimized test sets and how quickly mutants are killed. We used seven C language subjects from the Siemens Repository, specifically the test sets and the killing matrices from a previous study. We used 30 different orders for each set and ran every technique 100 times over each set. Results show that our analysis technique performed significantly better than prior techniques for creating minimal test sets and was able to establish new bounds for all cases. Also, our analysis technique killed mutants as fast or faster than prior techniques. These results indicate that our mutant stubbornness technique constructs test sets that are both minimal in size, and prioritized effectively, as well or better than other techniques.
|
|
| 6. |
- Grindal, Mats, et al.
(författare)
-
An Evaluation of Combination Strategies for Test Case Selection
- 2006
-
Ingår i: Empirical Software Engineering. - : Springer. - 1382-3256 .- 1573-7616. ; 11:4, s. 583-611
-
Tidskriftsartikel (refereegranskat)abstract
- This paper presents results from a comparative evaluation of five combination strategies. Combination strategies are test case selection methods that combine “interesting” values of the input parameters of a test subject to form test cases. This research comparatively evaluated five combination strategies; the All Combination strategy (AC), the Each Choice strategy (EC), the Base Choice strategy (BC), Orthogonal Arrays (OA) and the algorithm from the Automatic Efficient Test Generator (AETG). AC satisfies n-wise coverage, EC and BC satisfy 1-wise coverage, and OA and AETG satisfy pair-wise coverage. The All Combinations strategy was used as a “gold standard” strategy; it subsumes the others but is usually too expensive for practical use. The others were used in an experiment that used five programs seeded with 128 faults. The combination strategies were evaluated with respect to the number of test cases, the number of faults found, failure size, and number of decisions covered. The strategy that requires the least number of tests, Each Choice, found the smallest number of faults. Although the Base Choice strategy requires fewer test cases than Orthogonal Arrays and AETG, it found as many faults. Analysis also shows some properties of the combination strategies that appear significant. The two most important results are that the Each Choice strategy is unpredictable in terms of which faults will be revealed, possibly indicating that faults are found by chance, and that the Base Choice and the pair-wise combination strategies to some extent target different types of faults.
|
|
| 7. |
- Grindal, Mats, et al.
(författare)
-
An Evaluation of Combination Strategies for Test Case Selection
- 2003
-
Rapport (övrigt vetenskapligt/konstnärligt)abstract
- In this report we present the results from a comparative evaluation of five combination strategies. Combination strategies are test case selection methods that combine interesting values of the input parameters of a test object to form test cases. One of the investigated combination strategies, namely the Each Choice strategy, satisfies 1-wise coverage, i.e., each interesting value of each parameter is represented at least once in the test suite. Two of the strategies, the Orthogonal Arrays and Heuristic Pair-Wise strategies both satisfy pair-wise coverage, i.e., every possible pair of interesting values of any two parameters are included in the test suite. The fourth combination strategy, the All Values strategy, generates all possible combinations of the interesting values of the input parameters. The fifth and last combination strategy, the Base Choice combination strategy, satisfies 1-wise coverage but in addition makes use of some semantic information to construct the test cases. Except for the All Values strategy, which is only used as a reference point with respect to the number of test cases, the combination strategies are evaluated and compared with respect to number of test cases, number of faults found, test suite failure density, and achieved decision coverage in an experiment comprising five programs, similar to Unix commands, seeded with 131 faults. As expected, the Each Choice strategy finds the smallest number of faults among the evaluated combination strategies. Surprisingly, the Base Choice strategy performs as well, in terms of detecting faults, as the pair-wise combination strategies, despite fewer test cases. Since the programs and faults in our experiment may not be representative of actual testing problems in an industrial setting, we cannot draw any general conclusions regarding the number of faults detected by the evaluated combination strategies. However, our analysis shows some properties of the combination strategies that appear significant in spite of the programs and faults not being representative. The two most important results are that the Each Choice strategy is unpredictable in terms of which faults will be detected, i.e., most faults found are found by chance, and that the Base Choice and the pair-wise combination strategies to some extent target different types of faults.
|
|
| 8. |
- Grindal, Mats, et al.
(författare)
-
Combination Testing Strategies : A Survey
- 2005
-
Ingår i: Software Testing, Verification, and Reliability. - : Wiley. - 0960-0833 .- 1099-1689. ; 15:3, s. 167-199
-
Tidskriftsartikel (refereegranskat)abstract
- Combination strategies are test case selection methods that identify test cases by combining values of the different test object input parameters based on some combinatorial strategy. This survey presents 16 different combination strategies, covering more than 40 papers that focus on one or several combination strategies. This collection represents most of the existing work performed on combination strategies. This survey describes the basic algorithms used by the combination strategies. Some properties of combination strategies, including coverage criteria and theoretical bounds on the size of test suites, are also included in this description. This survey paper also includes a subsumption hierarchy that attempts to relate the various coverage criteria associated with the identified combination strategies.
|
|
| 9. |
- Grindal, Mats, et al.
(författare)
-
Handling Constraints in the Input Space when Using Combination Strategies for Software Testing
- 2006
-
Rapport (övrigt vetenskapligt/konstnärligt)abstract
- This study compares seven different methods for handling constraints in input parameter models when using combination strategies to select test cases. Combination strategies are used to select test cases based on input parameter models. An input parameter model is a representation of the input space of the system under test via a set of parameters and values for these parameters. A test case is one specific combination of values for all the parameters. Sometimes the input parameter model may contain parameters that are not independent. Some sub-combinations of values of the dependent parameters may not be valid, i.e., these sub-combinations do not make sense. Combination strategies, in their basic forms, do not take into account any semantic information. Thus, invalid sub-combinations may be included in test cases in the test suite. This paper proposes four new constraint handling methods and compares these with three existing methods in an experiment in which the seven constraint handling methods are used to handle a number of different constraints in different sized input parameter models under three different coverage criteria. All in all, 2568 test suites with a total of 634,263 test cases have been generated within the scope of this experiment.
|
|
| 10. |
- Grindal, Mats, et al.
(författare)
-
Input Parameter Modeling for Combination Strategies
- 2007
-
Ingår i: Proceedings of the IASTED International Conference on Software Engineering (SE2007), Feb 13-15, Innsbruck, Austria. - : ACTA Press. - 9780889866416 - 9780889866430 ; , s. 255-260
-
Konferensbidrag (refereegranskat)abstract
- Combination strategies are test methods that generate test cases based on input parameter models. This paper suggests a structured modeling method used to translate requirements expressed in a general format into an input parameter model suitable for combination strategies.This paper also describes results from two initial experiments exploring the efficiency and effectiveness of the modeling method. These results indicate that the resulting models may contain enough information to detect the vast majority of faults in the system under test. Further, results indicate that the modeling method is simple enough to use in practical testing.
|
|
| 11. |
- Grindal, Mats, et al.
(författare)
-
Managing Conflicts when Using Combination Strategies to Test Software
- 2007
-
Ingår i: Proceedings of 18th Australian Conference on Software Engineering (ASWEC2007), 10-13 April, Melbourne, Australia. - : IEEE. ; , s. 255-264
-
Tidskriftsartikel (refereegranskat)abstract
- Testers often represent systems under test in input parameter models. These contain parameters with associated values. Combinations of parameter values, with one value for each parameter, are potential test cases. In most models, some values of two or more parameters cannot be combined. Testers must then detect and avoid or remove these conflicts. This paper proposes two new methods for automatically handling such conflicts and compares these with two existing methods, based on the sizes of the final conflict-free test suites. A test suite reduction method, usable with three of the four investigated methods is also included in the study, resulting in seven studied conflict handling methods. In the experiment, the number and types of conflicts, as well as the size of the input parameter model and the coverage criterion used, are varied. All in all, 3854 test suites with a total of 929, 158 test cases were generated. Two methods stand out as tractable and complementary. The best method (called the avoid methods) with respect to test suite size is to avoid selection of test cases with conflicts. However, this method cannot always be used. The second best method (called the replace method), removing conflicts from the final test suite, is completely general.
|
|
| 12. |
- Grindal, Mats, et al.
(författare)
-
On the Testing Maturity of Software Producing Organizations
- 2006
-
Ingår i: Proceedings - Testing: Academic and Industrial Conference. - Los Alamitos, California : IEEE Computer Society. - 0769526721 - 9780769526720 ; , s. 171-180
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
- This paper presents data from a study of the current state of practice of software testing. Test managers from twelve different software organizations were interviewed. The interviews focused on the amount of resources spent on testing, how the testing is conducted, and the knowledge of the personnel in the test organizations. The data indicate that the overall test maturity is low. Test managers are aware of this but have trouble improving. One problem is that the organizations are commercially successful, suggesting that products must already be "good enough". Also, the current lack of structured testing in practice makes it difficult to quantify the current level of maturity and thereby articulate the potential gain from increasing testing maturity to upper management and developers
|
|
| 13. |
- Grindal, Mats, et al.
(författare)
-
On the Testing Maturity of Software Producing Organizations
- 2006
-
Ingår i: Proceedings of the 1st International Conference Testing : Academia & Industry Conference Practice And Research Techniques (TAIC PART), 29th-31st August, Cumberland Lodge, Windsor, UK.
-
Konferensbidrag (refereegranskat)
|
|
| 14. |
|
|
| 15. |
- Lindström, Birgitta, et al.
(författare)
-
Generating Trace-Sets for Model-Based Testing
- 2007
-
Ingår i: Proceedings 18th IEEE International Symposiumon Software Reliability Engineering. - : IEEE. - 9780769530246 - 0769530249 ; , s. 171-180
-
Konferensbidrag (refereegranskat)abstract
- Model-checkers are powerful tools that can find individual traces through models to satisfy desired properties. These traces provide solutions to a number of problems. Instead of individual traces, software testing needs sets of traces that satisfy coverage criteria. Finding a trace set in a large model is difficult because model checkers generate single traces and use a lot of memory. Space and time requirements of modelchecking algorithms grow exponentially with respect to the number of variables and parallel automata of the model being analyzed. We present a method that generates a set of traces by iteratively invoking a model checker. The method mitigates the memory consumption problem by dynamically building partitions along the traces. This method was applied to a testability case study, and it generated the complete trace set, while ordinary model-checking could only generate 26%.
|
|
| 16. |
- Lindström, Birgitta, et al.
(författare)
-
Generating Trace-Sets for Model-based Testing
- 2007
-
Ingår i: Proceedings of the 18th IEEE International Symposium on Software Reliability. - 9780769530246 ; , s. 171-180
-
Konferensbidrag (refereegranskat)abstract
- Model-checkers are powerful tools that can find individual traces through models to satisfy desired properties. These traces provide solutions to a number of problems. Instead of individual traces, software testing needs sets of traces that satisfy coverage criteria. Finding a trace set in a large model is difficult because model checkers generate single traces and use a lot of memory. Space and time requirements of model-checking algorithms grow exponentially with respect to the number of variables and parallel automata of the model being analyzed. We present a method that generates a set of traces by iteratively invoking a model checker. The method mitigates the memory consumption problem by dynamically building partitions along the traces. This method was applied to a testability case study, and it generated the complete trace set, while ordinary model-checking could only generate 26%.
|
|
| 17. |
- Lindström, Birgitta, et al.
(författare)
-
Generating Trace-Sets for Model-based Testing
- 2007
-
Ingår i: The 18th IEEE International Symposium on Software Reliability (ISSRE '07). ; , s. 171-180
-
Konferensbidrag (populärvet., debatt m.m.)abstract
- Model-checkers are powerful tools that can find individual traces through models to satisfy desired properties. These traces provide solutions to a number of problems. Instead of individual traces, software testing needs sets of traces that satisfy coverage criteria. Finding a trace set in a large model is difficult because model checkers generate single traces and use a lot of memory. Space and time requirements of modelchecking algorithms grow exponentially with respect to the number of variables and parallel automata of the model being analyzed. We present a method that generates a set of traces by iteratively invoking a model checker. The method mitigates the memory consumption problem by dynamically building partitions along the traces. This method was applied to a testability case study, and it generated the complete trace set, while ordinary model-checking could only generate 26%.
|
|
| 18. |
- Lindström, Birgitta, et al.
(författare)
-
Identifying Useful Mutants to Test Time Properties
- 2018
-
Ingår i: 2018 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW). - : IEEE Computer Society. - 9781538663523 - 9781538663530 ; , s. 69-76
-
Konferensbidrag (refereegranskat)abstract
- Real-time systems have to be verified and tested for timely behavior as well as functional behavior. Thus, time is an extra dimension that adds to the complexity of software testing. A timed automata model with a model-checker can be used to generate timed test traces. To properly test the timely behavior, the set of test traces should challenge the different time constraints in the model. This paper describes and adapts mutation operators that target such time constraints in timed automata models. Time mutation operators apply a delta to the time constraints to help testers design tests that exceed the time constraints. We suggest that the size of this delta determines how easy the mutant is to kill and that the optimal delta varies by the program, mutation operator, and the individual mutant. To avoid trivial and equivalent time mutants, the delta should be set individually for each mutant. We discuss mutant subsumption and define the problem of finding dominator mutants in this new domain. In this position paper, we outline an iterative tuning process where a statistical model-checker, UPPAAL SMC, is used to: (i) create a tuned set of dominator time mutants, and (ii) generate test traces that kill the mutants.
|
|
| 19. |
- Lindström, Birgitta, et al.
(författare)
-
Message from the TestEd 2020 Chairs
- 2020
-
Ingår i: 2020 IEEE 13th International Conference on Software Testing, Verification and Validation Workshops. - : IEEE. - 9781728110752 - 9781728110769
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)
|
|
| 20. |
- Lindström, Birgitta, et al.
(författare)
-
Mutating Aspect-Oriented Models to Test Cross-Cutting Concerns
- 2015
-
Ingår i: 2015 IEEE 8th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2015 - Proceedings. - : IEEE conference proceedings. - 9781479918850 ; , s. Article number 7107456-
-
Konferensbidrag (refereegranskat)
|
|
| 21. |
- Lindström, Birgitta, et al.
(författare)
-
Six Issues in Testing Event-Triggered Real-Time Systems
- 2007
-
Rapport (övrigt vetenskapligt/konstnärligt)abstract
- Verification of real-time systems is a complex task, with problems coming from issues like concurrency. A previous paper suggested dealing with these problems by using a time-triggered design, which gives good support both for testing and formal analysis. However, atime-triggered solution is not always feasible and an event-triggered design is needed. Event-triggered systems are far more difficult to test than time-triggered systems.This paper revisits previously identified testing problems from a new perspective and identifies additional problems for event-triggered systems. The paper also presents an approach to deal with these problems. The TETReS project assumes a model-driven developmentprocess. We combine research within three different fields: (i) transformation of rule sets between timed automata specifications and ECA rules with maintained semantics, (ii) increasing testability in event-triggered system, and (iii) development of test case generation methods for event-triggered systems.
|
|
| 22. |
- Lindström, Birgitta, et al.
(författare)
-
Testability of Dynamic Real-Time Systems : An Empirical Study of Constrained Execution Environment Implications
- 2008
-
Ingår i: Proceedings of the First International Conference on Software Testing, Verification and Validation. - Los Alamitos : IEEE Computer Society. - 9780769531274 - 076953127X ; , s. 112-120
-
Konferensbidrag (refereegranskat)abstract
- Real-time systems must respond to events in a timely fashion; in hard real-time systems the penalty for a missed deadline is high. It is therefore necessary to design hard real-time systems so that the timing behavior of the tasks can be predicted. Static real-time systems have prior knowledge of the worst-case arrival patterns and resource usage. Therefore, a schedule can be calculated off-line and tasks can be guaranteed to have sufficient resources to complete (resource adequacy). Dynamic real-time systems, on the other hand, do not have such prior knowledge, and therefore must react to events when they occur. They also must adapt to changes in the urgencies of various tasks, and fairly allocate resources among the tasks. A disadvantage of static real-time systems is that a requirement on resource adequacy makes them expensive and often impractical. Dynamic realtime systems, on the other hand, have the disadvantage of being less predictable and therefore difficult to test. Hence, in dynamic systems, timeliness is hard to guarantee and reliability is often low. Using a constrained execution environment, we attempt to increase the testability of such systems. An initial step is to identify factors that affect testability. We present empirical results on how various factors in the execution environment impacts testability of real-time systems. The results show that some of the factors, previously identified as possibly impacting testability, do not have an impact, while others do.
|
|
| 23. |
- Lindström, Birgitta, 1958-
(författare)
-
Testability of Dynamic Real-Time Systems
- 2009
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- This dissertation concerns testability of event-triggered real-time systems. Real-time systems are known to be hard to test because they are required to function correct both with respect to what the system does and when it does it. An event-triggered real-time system is directly controlled by the events that occur in the environment, as opposed to a time-triggered system, which behavior with respect to when the system does something is constrained, and therefore more predictable. The focus in this dissertation is the behavior in the time domain and it is shown how testability is affected by some factors when the system is tested for timeliness.This dissertation presents a survey of research that focuses on software testability and testability of real-time systems. The survey motivates both the view of testability taken in this dissertation and the metric that is chosen to measure testability in an experiment. We define a method to generate sets of traces from a model by using a meta algorithm on top of a model checker. Defining such a method is a necessary step to perform the experiment. However, the trace sets generated by this method can also be used by test strategies that are based on orderings, for example execution orders.An experimental study is presented in detail. The experiment investigates how testability of an event-triggered real-time system is affected by some constraining properties of the execution environment. The experiment investigates the effect on testability from three different constraints regarding preemptions, observations and process instances. All of these constraints were claimed in previous work to be significant factors for the level of testability. Our results support the claim for the first two of the constraints while the third constraint shows no impact on the level of testability.Finally, this dissertation discusses the effect on the event-triggered semantics when the constraints are applied on the execution environment. The result from this discussion is that the first two constraints do not change the semantics while the third one does. This result indicates that a constraint on the number of process instances might be less useful for some event-triggered real-time systems.
|
|
| 24. |
- Lindström, Birgitta, et al.
(författare)
-
Using an Existing Suite of Test Objects : Experience from a Testing Experiment
- 2004
-
Ingår i: ACM SIGSOFT Software Engineering Notes. - : Association for Computing Machinery (ACM). ; , s. 1-3
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
- This workshop paper presents lessons learned from a recent experiment to compare several test strategies. The test strategies were compared in terms of the number of tests needed to satisfy them and in terms of faults found. The experimental design and conduct are discussed, and frank assessments of the decisions that were made are provided. The paper closes with a summary of the lessons that were learned.
|
|
| 25. |
- Lindström, Birgitta, et al.
(författare)
-
Using mutation to design tests for aspect-oriented models
- 2017
-
Ingår i: Information and Software Technology. - : Elsevier BV. - 0950-5849 .- 1873-6025. ; 81, s. 112-130
-
Tidskriftsartikel (refereegranskat)abstract
- Context: Testing for properties such as robustness or security is complicated because their concerns are often repeated in many locations and muddled with the normal code. Such “cross-cutting concerns” include things like interrupt events, exception handling, and security protocols. Aspect-oriented (AO) modeling allows developers to model the cross-cutting behavior independently of the normal behavior, thus supporting model-based testing of cross-cutting concerns. However, mutation operators defined for AO programs (source code) are usually not applicable to AO models (AOMs) and operators defined for models do not target the AO features. Objective: We present a method to design abstract tests at the aspect-oriented model level. We define mutation operators for aspect-oriented models and evaluate the generated mutants for an example system. Method: AOMs are mutated with novel operators that specifically target the AO modeling features. Test traces killing these mutant models are then generated. The generated and selected traces are abstract tests that can be transformed to concrete black-box tests and run on the implementation level, to evaluate the behavior of the woven cross-cutting concerns (combined aspect and base models). Results: This paper is a significant extension of our paper at Mutation 2015. We present a complete fault model, additional mutation operators, and a thorough analysis of the mutants generated for an example system. Conclusions: The analysis shows that some mutants are stillborn (syntactically illegal) but none is equivalent (exhibiting the same behavior as the original model). Additionally, our AOM-specific mutation operators can be combined with pre-existing operators to mutate code or models without any overlap.
|
|
| 26. |
- Nilsson, Robert, et al.
(författare)
-
Mutation-Based Testing Criteria for Timeliness
- 2004
-
Ingår i: Proceedings of the 28th Annual International Computer Software and Applications Conference, 2004. COMPSAC 2004. - : IEEE conference proceedings. - 0769522092 ; , s. 306-311
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)
|
|
| 27. |
- Nilsson, Robert, et al.
(författare)
-
Test case generation for mutation-based testing of timeliness
- 2006
-
Ingår i: Electronic Notes in Theoretical Computer Science. - : Elsevier. - 1571-0661. ; 164:4, s. 97-114
-
Tidskriftsartikel (refereegranskat)abstract
- Temporal correctness is crucial for real-time systems. Few methods exist to test temporal correctness and most methods used in practice are ad-hoc. A problem with testing real-time applications is the response-time dependency on the execution order of concurrent tasks. Execution order in turn depends on execution environment properties such as scheduling protocols, use of mutual exclusive resources as well as the point in time when stimuli is injected. Model based mutation testing has previously been proposed to determine the execution orders that need to be verified to increase confidence in timeliness. An effective way to automatically generate such test cases for dynamic real-time systems is still needed. This paper presents a method using heuristic-driven simulation to generate test cases.
|
|
| 28. |
- Nilsson, Robert, et al.
(författare)
-
Test case generation for testing of timeliness : Extended version
- 2005
-
Rapport (övrigt vetenskapligt/konstnärligt)abstract
- Temporal correctness is crucial for real-time systems. There are few methods to test temporal correctness and most methods used in practice are ad-hoc. A problem with testing real-time applications is the response-time dependency on the execution order of concurrent tasks. Execution orders in turn depends on scheduling protocols, task execution times, and use of mutual exclusive resources apart from the points in time when stimuli is injected. Model-based mutation testing has previously been proposed to determine the execution orders that need to be tested to increase confidence in timeliness. An effective way to automatically generate such test cases for dynamic real-time systems is still needed. This paper presents a method using heuristic-driven simulation for generation of test cases.
|
|
| 29. |
- Offutt, Jeff, et al.
(författare)
-
ICST 2009 special issue
- 2011
-
Ingår i: Software Testing Verification and Reliability. - : Wiley. - 0960-0833. ; 21:3, s. 153-154
-
Tidskriftsartikel (övrigt vetenskapligt/konstnärligt)
|
|
| 30. |
- Offutt, Jeff, et al.
(författare)
-
Teaching an International Distributed Discussion-Based Course
- 2019
-
Ingår i: Proceedings of the 2019 International Conference on Frontiers in Education. - : CSREA, Computer Science Research, Education, and Applications Press. - 1601324987 ; , s. 149-154
-
Konferensbidrag (refereegranskat)abstract
- Small discussion-based courses pose several challenges. Low enrollments make the course difficult to justify and can restrict active discussions. Impromptu discussions are hard to encourage. Students come to class tired, not well prepared, reluctant to speak out, or not able to verbalize abstract thoughts fast enough to fully engage. Sometimes a few students dominate the discussions while other students stay silent. This paper describes a novel teaching model that was created to allow one professor to teach the same course at multiple universities. As the course design emerged, the asynchronous online distributed nature of the course turned out to not only solve the initial problem, but also other challenges of discussion-based courses. Instructors and students found this model led to more engagement, increased learning, and higher performance.
|
|
