| 1. |
- Börjesson, Emil, 1984, et al.
(författare)
-
Automated system testing using visual GUI testing tools: A comparative study in industry
- 2012
-
Ingår i: Proceedings - IEEE 5th International Conference on Software Testing, Verification and Validation, ICST 2012, Montreal, QC, 17 April through 21 April 2012. - 9780769546704 ; :Article number 6200127, s. 350-359
-
Konferensbidrag (refereegranskat)abstract
- Software companies are under continuous pressure to shorten time to market, raise quality and lower costs. More automated system testing could be instrumental in achieving these goals and in recent years testing tools have been developed to automate the interaction with software systems at the GUI level. However, there is a lack of knowledge on the usability and applicability of these tools in an industrial setting. This study evaluates two tools for automated visual GUI testing on a real-world, safety-critical software system developed by the company Saab AB. The tools are compared based on their properties as well as how they support automation of system test cases that have previously been conducted manually. The time to develop and the size of the automated test cases as well as their execution times have been evaluated. Results show that there are only minor differences between the two tools, one commercial and one open-source, but, more importantly, that visual GUI testing is an applicable technology for automated system testing with effort gains over manual system test practices. The study results also indicate that the technology has benefits over alternative GUI testing techniques and that it can be used for automated acceptance testing. However, visual GUI testing still has challenges that must be addressed, in particular the script maintenance costs and how to support robust test execution. © 2012 IEEE.
|
|
| 2. |
- Börjesson, Emil, 1984
(författare)
-
Industrial applicability of visual GUI testing for system and acceptance test automation
- 2012
-
Ingår i: Proceedings - IEEE 5th International Conference on Software Testing, Verification and Validation, ICST 2012, Montreal, QC,17 April through 21 April 2012. - 9780769546704 ; :Art. no. 6200141, s. 475-478
-
Konferensbidrag (refereegranskat)abstract
- The software market is becoming more challenging as demands for faster time-to-market and higher software quality continue to grow. These challenges are embedded in all areas of Software Engineering, including Verification and Validation where they are proposed as solvable with automated testing. However, most automated testing techniques focus on low system level testing and are not suitable for high level tests, i.e. System and Acceptance tests, leaving industrial needs for test automation unfulfilled. In this paper we present a research plan to evaluate a novel automated testing technique, called visual GUI testing, based on image recognition algorithms and scripts that interact through the system GUI to automate complex scenario based tests. The technique has been evaluated at the company Saab AB where industrial, safety critical, scenario based, test cases were automated showing the industrial applicability of the technique. However, many factors are still unknown regarding the techniques industrial applicability, i.e. script maintenance costs, usability and learn ability, etc. Our research aims to uncover these unknown factors with the final research goal to show that visual GUI testing is a viable and cost-effective technique that will fill the gap in industry for a cost-effective, simple, robust, high-level test automation technique.
|
|
| 3. |
- Eldh, Sigrid, 1961-, et al.
(författare)
-
Robustness Testing of Mobile Telecommunication Systems : A Case Study on Industrial Practice and Challenges
- 2012
-
Ingår i: Proceedings - IEEE 5th International Conference on Software Testing, Verification and Validation, ICST 2012. - : IEEE Press. - 9780769546704 ; , s. 895-900
-
Konferensbidrag (refereegranskat)abstract
- Robustness relates to the capability of a system to handle internal and external negative situations and disturbances. Robustness testing is the act of subjecting the system under test to such disturbances in a controlled manner. The objective of this study is to understand how robustness is considered in the development and testing of large-scale telecom systems, to identify the main challenges related to robustness and robustness testing, and to identify potential improvements to the current situation. We performed an exploratory case study of a telecom industry. Data was collected through interviews, study of documentation, and participant observation. Our result is a number of challenges related to robustness testing. The key challenge identified relates to understanding how robustness test can be broken down from a system-level to a low-level perspective. Our conclusion is that the area of robustness test is challenging for large complex systems, where the understanding of how to provoke complex failures and derive a root cause, as well as defining the correct level of robust design in software is not sufficiently explored.
|
|
| 4. |
- Wiklund, Kristian, 1972-, et al.
(författare)
-
Technical Debt in Test Automation
- 2012
-
Ingår i: Proceedings - IEEE 5th International Conference on Software Testing, Verification and Validation, ICST 2012. - : IEEE Computer Society. - 9780769546704 ; , s. 887-892
-
Konferensbidrag (refereegranskat)abstract
- Automated test execution is one of the more popular and available strategies to minimize the cost for software testing, and is also becoming one of the central concepts in modern software development as methods such as test-driven devel- opment gain popularity. Published studies on test automation indicate that the maintenance and development of test automation tools commonly encounter problems due to unforeseen issues. To further investigate this, we performed a case study on a telecommunication subsystem to seek factors that contribute to inefficiencies in use, maintenance, and development of the automated testing performed within the scope of responsibility of a software design team. A qualitative evaluation of the findings indicates that the main areas of improvement in this case are in the fields of interaction design and general software design principles, as applied to test execution system development.
|
|
