| 1. |
- Flemström, Daniel, 1971-
(författare)
-
Industrial System Level Test Automation
- 2021
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Vehicular software systems control and monitor many safety-critical functions, such as automated emergency brakes and anti-spin. These functions are integrated and tested at system level to ensure that the entire system works as intended.Traditionally, these functions or selected combinations of functions are tested in isolation. Although desired, rigorous testing of combinations of functions prior to deployment is seldom possible. One reason is the overwhelming work required to write new test cases for the nearly infinite combinations of functions and driving scenarios. Since software testing already accounts for up to 60% of the software development cost and the execution time in test rigs is expensive, further testing must be achieved by running test cases more in parallel. Moreover, new test cases must be reusable in different driving scenarios to cover more combinations of functions. One solution is to express the test logic in a new way so that it can be executed in parallel, independently of other tests and independent of the input stimuli. This would allow reusing the test logic for different sequences or drive scenarios. Passive testing is one such approach that has not yet been used much for vehicular software due to perceived difficulties, although it is well-established in other domains. One particular problem is how to specify and execute passive test cases for vehicular systems in an, for the test engineer, intuitive and straightforward manner. Further, there is a lack of tool support and knowledge on efficiently applying passive testing in an industrial context. Thus, the overall research goal of this thesis is to propose and evaluate industrially applicable methods and tools for passive testing at the system level of vehicular software systems. The research is based on a series of papers and case studies within the vehicular industry. In contrast to existing specification languages based on formal mathematical expressions, considerable effort was spent creating an intuitive language and an interactive tool, simple but powerful enough, to encourage the industrial application of passive testing. The main contributions include an easy-to-write and easy-to-read description language for passive test cases, an interactive development environment, and knowledge on how to succeed in passive testing in an industrial software development process. The findings of this thesis contribute to making passive testing a viable method for system-level testing and a way of reusing test logic. In the case of a studied train control management system, using passive testing may reuse up to 50% of the test logic for the safety-related requirements and 10% of the non-safety-related requirements.
|
|
| 2. |
- Irshad, Mohsin, 1984-
(författare)
-
Realizing the systematic reuse of automated acceptance tests in practice
- 2021
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Context: Automated acceptance testing has become a relevant practice of agile software development (e.g., Extreme Programming). Automated acceptance tests (AATs) are closely tied to requirements and provide a mechanism for continuous validation of requirements as system-level tests. Studies have shown that these tests are costly to develop, maintain, and reuse.Objectives: This thesis examines and supports the systematic reuse of automated acceptance tests by providing approaches to search, identity, adapt reusable test cases while considering the reuse costs. Additionally, the maintainability and usage of automated acceptance tests in largescale software projects are studied.Method: We used various research methods to investigate development, reuse, and maintaining the automated acceptance tests. Workshops and interviews of practitioners were used to identify the benefits and challenges of automated acceptance tests. Next, method engineering was used to construct a systematic reuse process for automated acceptance tests. Qualitative feedback collected using a survey, and industrial demonstration examined the reuse process’s performance expectancy, effort expectancy, and necessary facilitating conditions. Two systematic literature reviews are used to identify techniques to adapt tests for future reuse opportunities and calculate the reuse costs of automated acceptance tests. Later, we developed and evaluated an approach for refactoring behavior-driven development-based automated acceptance tests using action research.Results: A cost-aware systematic reuse process was constructed to support the reuse of automated acceptance tests containing eleven activities. For each activity, guidelines on expected input, expected output, the actors performing the activity, and techniques (automated using scripts) are suggested and evaluated. The techniques involve approaches to support development for reuse and methods to calculate the costs of reusing automated acceptance tests. The industrial evaluation of the reuse process and the techniques showed its usefulness and relevance for the industry. Furthermore, seven challenges (i.e., the scale of the software projects, ownership, lack of competence, cost benefits, specification of behaviors in large-scale projects, difficulty writing system-level test-cases, and versioning control of behaviors) and five benefits (i.e., understanding of a business aspect of requirements, improved quality of requirements, a guide to system-level use-cases, reuse of artifacts in large-scale projects, and help for test organization.) of using automated acceptance tests in large-scale projects are identified with software practitioners’ help. Later, we proposed a semiautomated four-step approach for pre-processing, measuring, ranking, and identifying refactoring candidates. The approach and the two proposed measures were successfully evaluated using two industrial projects. It was noted that similarity measures could support the maintenance of the specification base using refactoring.Conclusions: The studies show that automated acceptance tests are reusable and can be refactored using our proposed approach. The evaluation shows that the results apply to the software industry in the evaluated context. However, further work is required to evaluate the reuse process in different contexts.
|
|
