| 1. |
- Hassan, Mahdi Mohammad, 1977-, et al.
(författare)
-
Testability and Software Performance : A Systematic Mapping Study
- 2016
-
Ingår i: SAC '16. - New York, NY : Association for Computing Machinery (ACM). - 9781450337397 ; , s. 1566-1569
-
Konferensbidrag (refereegranskat)abstract
- In most of the research on software testability, functional correctness of the software has been the focus while the evidence regarding testability and non-functional properties such as performance is sporadic. The objective of this study is to present the current state-of-the-art related to issues of importance, types and domains of software under test, types of research, contribution types and design evaluation methods concerning testability and software performance. We find that observability, controllability and testing effort are the main testability issues while timeliness and response time (i.e., time constraints) are the main performance issues in focus. The primary studies in the area use diverse types of software under test within different domains, with realtime systems as being a dominant domain. The researchers have proposed many different methods in the area, however these methods lack implementation in practice.
|
|
| 2. |
- González-Hernández, Loreto, et al.
(författare)
-
Using Mutant Stubbornness to Create Minimal and Prioritized Test Sets
- 2018
-
Ingår i: 2018 IEEE International Conference on Software Quality, Reliability and Security (QRS). - : IEEE Computer Society. - 9781538677575 - 9781538677582 ; , s. 446-457
-
Konferensbidrag (refereegranskat)abstract
- In testing, engineers want to run the most useful tests early (prioritization). When tests are run hundreds or thousands of times, minimizing a test set can result in significant savings (minimization). This paper proposes a new analysis technique to address both the minimal test set and the test case prioritization problems. This paper precisely defines the concept of mutant stubbornness, which is the basis for our analysis technique. We empirically compare our technique with other test case minimization and prioritization techniques in terms of the size of the minimized test sets and how quickly mutants are killed. We used seven C language subjects from the Siemens Repository, specifically the test sets and the killing matrices from a previous study. We used 30 different orders for each set and ran every technique 100 times over each set. Results show that our analysis technique performed significantly better than prior techniques for creating minimal test sets and was able to establish new bounds for all cases. Also, our analysis technique killed mutants as fast or faster than prior techniques. These results indicate that our mutant stubbornness technique constructs test sets that are both minimal in size, and prioritized effectively, as well or better than other techniques.
|
|
| 3. |
- Hassan, Mahdi Mohammad, 1977-, et al.
(författare)
-
Testability and Software Robustness : A Systematic Literature Review
- 2015
-
Ingår i: 2015 41st Euromicro Conference on Software Engineering and Advanced Applications. - Funchal, Madeira, Portugal : IEEE. - 9781467375856 ; , s. 341-348
-
Konferensbidrag (refereegranskat)abstract
- The concept of software testability has been researched in several different dimensions, however the relation of this important concept with other quality attributes is a grey area where existing evidence is scattered. The objective of this study is to present a state-of-the-art with respect to issues of importance concerning software testability and an important quality attribute: software robustness. The objective is achieved by conducting a systematic literature review (SLR) on the topic. Our results show that a variety of testability issues are in focus with observability and controllability issues being most researched. Fault tolerance, exception handling and handling external influence are prominent robustness issues in focus.
|
|
| 4. |
- Lindström, Birgitta, et al.
(författare)
-
Mutating Aspect-Oriented Models to Test Cross-Cutting Concerns
- 2015
-
Ingår i: 2015 IEEE 8th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2015 - Proceedings. - : IEEE conference proceedings. - 9781479918850 ; , s. Article number 7107456-
-
Konferensbidrag (refereegranskat)
|
|
| 5. |
- Lindström, Birgitta, et al.
(författare)
-
Using mutation to design tests for aspect-oriented models
- 2017
-
Ingår i: Information and Software Technology. - : Elsevier BV. - 0950-5849 .- 1873-6025. ; 81, s. 112-130
-
Tidskriftsartikel (refereegranskat)abstract
- Context: Testing for properties such as robustness or security is complicated because their concerns are often repeated in many locations and muddled with the normal code. Such “cross-cutting concerns” include things like interrupt events, exception handling, and security protocols. Aspect-oriented (AO) modeling allows developers to model the cross-cutting behavior independently of the normal behavior, thus supporting model-based testing of cross-cutting concerns. However, mutation operators defined for AO programs (source code) are usually not applicable to AO models (AOMs) and operators defined for models do not target the AO features. Objective: We present a method to design abstract tests at the aspect-oriented model level. We define mutation operators for aspect-oriented models and evaluate the generated mutants for an example system. Method: AOMs are mutated with novel operators that specifically target the AO modeling features. Test traces killing these mutant models are then generated. The generated and selected traces are abstract tests that can be transformed to concrete black-box tests and run on the implementation level, to evaluate the behavior of the woven cross-cutting concerns (combined aspect and base models). Results: This paper is a significant extension of our paper at Mutation 2015. We present a complete fault model, additional mutation operators, and a thorough analysis of the mutants generated for an example system. Conclusions: The analysis shows that some mutants are stillborn (syntactically illegal) but none is equivalent (exhibiting the same behavior as the original model). Additionally, our AOM-specific mutation operators can be combined with pre-existing operators to mutate code or models without any overlap.
|
|
| 6. |
- Lisper, Björn, et al.
(författare)
-
Targeted Mutation : Efficient Mutation Analysis for Testing Non-Functional Properties
- 2017
-
Ingår i: Proceedings - 10th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2017. - : Institute of Electrical and Electronics Engineers Inc.. - 9781509066766 - 9781509066773 ; , s. 65-68
-
Konferensbidrag (refereegranskat)abstract
- Mutation analysis has proven to be a strong technique for software testing. Unfortunately, it is also computationally expensive and researchers have therefore proposed several different approaches to reduce the effort. None of these reduction techniques however, focuses on non-functional properties. Given that our goal is to create a strong test suite for testing a certain non-functional property, which mutants should be used? In this paper, we introduce the concept of targeted mutation, which focuses mutation effort to those parts of the code where a change can make a difference with respect to the targeted non-functional property. We show how targeted mutation can be applied to derive efficient test suites for estimating the Worst-Case Execution Time (WCET). We use program slicing to direct the mutations to the parts of the code that are likely to have the strongest influence on execution time. Finally, we outline an experimental procedure for how to evaluate the technique.
|
|
| 7. |
- Shah, S. M. A., et al.
(författare)
-
Robustness Testing of Embedded Software Systems : An Industrial Interview Study
- 2016
-
Ingår i: IEEE Access. - : IEEE. - 2169-3536. ; 4, s. 1859-1871
-
Tidskriftsartikel (refereegranskat)abstract
- Embedded software is at the core of current and future telecommunication, automotive, multimedia, and industrial automation systems. The success of practically any industrial application depends on the embedded software system's dependability, and one method to verify the dependability of a system is testing its robustness. The motivation behind this paper is to provide a knowledge base of the state of the practice in robustness testing of embedded software systems and to compare this to the state of the art. We have gathered the information on the state of the practice in robustness testing from seven different industrial domains (telecommunication, automotive, multimedia, critical infrastructure, aerospace, consumer products, and banking) by conducting 13 semi-structured interviews. We investigate the different aspects of robustness testing, such as the general view of robustness, relation to requirements engineering and design, test execution, failures, and tools. We highlight knowledge from the state of the practice of robustness testing of embedded software systems. We found different robustness testing practices that have not been previously described. This paper shows that the state of the practice, when it comes to robustness testing, differs between organizations and is quite different from the state of the art described in the scientific literature. For example, methods commonly described in the literature (e.g., the fuzzy approach) are not used in the organizations we studied. Instead, the interviewees described several ad hoc approaches that take specific scenarios into account (e.g., power failure or overload). Other differences we found concern the classification of robustness failures, the hypothesized root causes of robustness failures, and the types of tools used for robustness testing. This paper is a first step in capturing the state of the practice of robustness testing of embedded software systems. The results can be used by both researchers and practitioners. Researchers can use our findings to understand the gap between the state of the art and the state of the practice and develop their studies to fill this gap. Practitioners can also learn from this knowledge base regarding how they can improve their practice and acquire other practices.
|
|
| 8. |
- Atif, Yacine, 1967-, et al.
(författare)
-
Cyber-threat analysis for Cyber-Physical Systems : Technical report for Package 4, Activity 3 of ELVIRA project
- 2018
-
Rapport (övrigt vetenskapligt/konstnärligt)abstract
- Smart grid employs ICT infrastructure and network connectivity to optimize efficiency and deliver new functionalities. This evolu- tion is associated with an increased risk for cybersecurity threats that may hamper smart grid operations. Power utility providers need tools for assessing risk of prevailing cyberthreats over ICT infrastructures. The need for frameworks to guide the develop- ment of these tools is essential to define and reveal vulnerability analysis indicators. We propose a data-driven approach for design- ing testbeds to evaluate the vulnerability of cyberphysical systems against cyberthreats. The proposed framework uses data reported from multiple components of cyberphysical system architecture layers, including physical, control, and cyber layers. At the phys- ical layer, we consider component inventory and related physi- cal flows. At the control level, we consider control data, such as SCADA data flows in industrial and critical infrastructure control systems. Finally, at the cyber layer level, we consider existing secu- rity and monitoring data from cyber-incident event management tools, which are increasingly embedded into the control fabrics of cyberphysical systems.
|
|
| 9. |
- Atif, Yacine, 1967-, et al.
(författare)
-
Cyber-Threat Intelligence Architecture for Smart-Grid Critical Infrastructures Protection
- 2017
-
Konferensbidrag (refereegranskat)abstract
- Critical infrastructures (CIs) are becoming increasingly sophisticated with embedded cyber-physical systems (CPSs) that provide managerial automation and autonomic controls. Yet these advances expose CI components to new cyber-threats, leading to a chain of dysfunctionalities with catastrophic socio-economical implications. We propose a comprehensive architectural model to support the development of incident management tools that provide situation-awareness and cyber-threats intelligence for CI protection, with a special focus on smart-grid CI. The goal is to unleash forensic data from CPS-based CIs to perform some predictive analytics. In doing so, we use some AI (Artificial Intelligence) paradigms for both data collection, threat detection, and cascade-effects prediction.
|
|
| 10. |
- Atif, Yacine, 1967-, et al.
(författare)
-
Multi-agent Systems for Power Grid Monitoring : Technical report for Package 4.1 of ELVIRA project
- 2018
-
Rapport (övrigt vetenskapligt/konstnärligt)abstract
- This document reports a technical description of ELVIRA project results obtained as part of Work- package 4.1 entitled “Multi-agent systems for power Grid monitoring”. ELVIRA project is a collaboration between researchers in School of IT at University of Skövde and Combitech Technical Consulting Company in Sweden, with the aim to design, develop and test a testbed simulator for critical infrastructures cybersecurity. This report outlines intelligent approaches that continuously analyze data flows generated by Supervisory Control And Data Acquisition (SCADA) systems, which monitor contemporary power grid infrastructures. However, cybersecurity threats and security mechanisms cannot be analyzed and tested on actual systems, and thus testbed simulators are necessary to assess vulnerabilities and evaluate the infrastructure resilience against cyberattacks. This report suggests an agent-based model to simulate SCADA- like cyber-components behaviour when facing cyber-infection in order to experiment and test intelligent mitigation mechanisms.
|
|
