| 1. |
- Afzal, Zeeshan, 1991-, et al.
(author)
-
Automated Testing of IDS Rules
- 2015
-
In: Software Testing, Verification and Validation Workshops (ICSTW), 2015 IEEE Eighth International Conference on. - : IEEE. - 9781479918850
-
Conference paper (peer-reviewed)abstract
- As technology becomes ubiquitous, new vulnerabilities are being discovered at a rapid rate. Security experts continuously find ways to detect attempts to exploit those vulnerabilities. The outcome is an extremely large and complex rule set used by Intrusion Detection Systems (IDSs) to detect and prevent the vulnerabilities. The rule sets have become so large that it seems infeasible to verify their precision or identify overlapping rules. This work proposes a methodology consisting of a set of tools that will make rule management easier.
|
|
| 2. |
- Arts, Thomas, 1969, et al.
(author)
-
Graphical editing support for QuickCheck models
- 2015
-
In: 2015 IEEE 8th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2015 - Proceedings. - 2159-4848. - 9781479918850
-
Conference paper (peer-reviewed)abstract
- QuickCheck can test a system by specifying a state machine for the API exported by that system. This state machine specification includes a list of possible API calls. Each call is accompanied by a precondition, a postcondition, a generator for the arguments, and a description of how the state is changed. Based on this specification QuickCheck generates a random sequence of API calls. The preconditions ensure that a generated sequence is valid, and the postconditions check that the system behaves as expected. Many systems require an initialisation call before other calls, describing the transition from an uninitialized to an initialised state. QuickCheck offers two ways of specifying transitions between states: using preconditions or a finite state machine abstraction. In this paper we show, by means of an example, that the latter approach is to be preferred. In addition, we present a recent extension to QuickCheck that allows a user to graphically create and edit a finite state machine specification. This extension simplifies and speeds up the specification of a finite state machine, which can be regarded as a formal model of the system, considerably. The graphical representation makes the formal model easier to understand, and visualises the distribution of API calls. Moreover, the extension allows the user to change this distribution.
|
|
| 3. |
- Arts, Thomas, 1969, et al.
(author)
-
Testing AUTOSAR software with QuickCheck
- 2015
-
In: 2015 IEEE 8th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2015 - Proceedings. - 2159-4848. - 9781479918850
-
Conference paper (peer-reviewed)abstract
- AUTOSAR (AUTomotive Open System ARchitecture) is an evolving standard for embedded software in vehicles, defined by the automotive industry, and implemented by many different vendors. On behalf of Volvo Cars, we have developed model-based acceptance tests for some critical AUTOSAR components, to guarantee that implementations from different vendors are compatible. We translated over 3000 pages of textual specifications into QuickCheck models, and tested many different implementations using large volumes of generated tests. This exposed over 200 issues, which we raised with Volvo and the software vendors. Compared to an earlier manual approach, ours is more efficient, more effective, and more correct.
|
|
| 4. |
- Grinchtein, Olga, et al.
(author)
-
Testing of a telecommunication protocol using constraint programming
- 2015
-
In: 2015 IEEE Eighth International Conference On Software Testing, Verification And Validation Workshops (ICSTW). - 9781479918850
-
Conference paper (peer-reviewed)abstract
- We present an approach of implementing part of a telecommunication test harness using constraint programming. Our case study is the Public Warning System service, which is a part of the Long Term Evolution (LTE) 4G standard.
|
|
| 5. |
- Gustafsson, Thomas, et al.
(author)
-
Automotive System Testing by Independent Guarded Assertions
- 2015. - 10
-
In: 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW). - 9781479918850
-
Conference paper (peer-reviewed)abstract
- Testing is a key activity in industry to verify and validate products before they reach end customers. In hardwarein- the-loop system-level verification of automotive systems, testing is often performed using sequential execution of test scripts, each containing a mix of stimuli and assertions. In this paper, we propose and study an alternative approach for automated system-level testing automotive systems. In our approach, assertion-only test scripts and one (or several) stimulionly script(s), execute concurrently on the test driver. By separating the stimuli from the assertions, with each assertion independently determining when the system under test shall be verified, we seek to achieve three things: 1) tests that better represent real-world handling of the product, 2) reduced test execution time, and 3) increased defect detection. In addition to describing our proposed approach in detail, we provide experimental results from an industrial case study evaluating the approach in an automotive system test environment.
|
|
| 6. |
- Lindström, Birgitta, et al.
(author)
-
Mutating Aspect-Oriented Models to Test Cross-Cutting Concerns
- 2015
-
In: 2015 IEEE 8th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2015 - Proceedings. - : IEEE conference proceedings. - 9781479918850 ; , s. Article number 7107456-
-
Conference paper (peer-reviewed)
|
|
| 7. |
- Oliveira, R., et al.
(author)
-
Definition and evaluation of mutation operators for GUI-level mutation analysis
- 2015
-
In: 2015 IEEE 8th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2015 - Proceedings. - 2159-4848. - 9781479918850
-
Conference paper (peer-reviewed)abstract
- Automated testing has become essential in software industry to meet market demands for faster delivery and higher quality software. Testing is performed on many levels of system abstraction, from tests on source code to Graphical User Interface (GUI) tests. New testing techniques and frameworks are also continuously released to the market. Mutation analysis has been proposed as a way of assessing the quality of these new test techniques/frameworks as well as existing test suites in practice. The analysis is performed by seeding defects, referred to as mutants, into the system under test with the assumption that a technique/test suite of high quality will 'kill' the mutants. However, whilst support for mutation analysis exists for test techniques that operate on on lower levels of system abstraction, i.e. method-level mutation operators, the support for GUI-level mutation analysis is currently lacking. In this paper we perform an empirical analysis of 18 GUI-level mutation operators defined in our previous work and compare their efficiency and comprehensiveness to state-of-practice lower level mutation operators. The main findings of our analysis are (1) that traditional method-level mutation operators are not precise enough for GUI-level mutation; (2) the defined GUI-based mutation operators provide comprehensive support for GUI-level mutation; and (3) GUI-based mutation operators can be automated but are challenged by the dependencies between GUI widgets.
|
|
