| 1. |
- Alégroth, Emil, 1984, et al.
(författare)
-
Industrial application of visual GUI testing: Lessons learned
- 2014
-
Ingår i: Continuous software engineering. - Cham : Springer International Publishing. ; 9783319112831, s. 127-140
-
Bokkapitel (övrigt vetenskapligt/konstnärligt)abstract
- A large body of academic knowledge has been devoted to automated software testing in order to support the software market's demands for continuous software delivery. However, most of these automated techniques approach testing from lower levels of system abstraction, e.g., component level, which limit their applicability for high-level regression testing of, for instance, system and acceptance tests, thus forcing companies to perform these test activities manually, which is considered time consuming, tedious, and error prone. In this book chapter, we present visual GUI testing (VGT), a tool driven test technique that uses image recognition in order to interact and assert the correctness of a system under test (SUT) through the bitmap graphical user interface (GUI) that is shown to the user on the computer monitor. This approach makes VGT flexible and applicable to any SUT with a GUI but also allows VGT tools to emulate end-user behavior and therefore automate previously manual system and acceptance test cases. In addition to presenting the technique itself, this chapter will also present some VGT tools and empirically identified problems with the technique and how these problems can be mitigated in practice. Finally we will discuss how VGT can be used in the context of continuous software development in order to support market demands for quicker software delivery.
|
|
| 2. |
- Alégroth, Emil, 1984, et al.
(författare)
-
JAutomate: A tool for system- and acceptance-test automation
- 2013
-
Ingår i: Proceedings - IEEE 6th International Conference on Software Testing, Verification and Validation, ICST 2013. - : IEEE. - 9780769549682 ; , s. 439-446
-
Konferensbidrag (refereegranskat)abstract
- System- and acceptance-testing are primarily performed with manual practices in current software industry. However, these practices have several issues, e.g. they are tedious, error prone and time consuming with costs up towards 40 percent of the total development cost. Automated test techniques have been proposed as a solution to mitigate these issues, but they generally approach testing from a lower level of system abstraction, leaving a gap for a flexible, high system-level test automation technique/tool. In this paper we present JAutomate, a Visual GUI Testing (VGT) tool that fills this gap by combining image recognition with record and replay functionality for high system-level test automation performed through the system under test's graphical user interface. We present the tool, its benefits compared to other similar techniques and manual testing. In addition, we compare JAutomate with two other VGT tools based on their static properties. Finally, we present the results from a survey with industrial practitioners that identifies testrelated problems that industry is currently facing and discuss how JAutomate can solve or mitigate these problems. © 2013 IEEE.
|
|
| 3. |
- Alégroth, Emil, 1984
(författare)
-
On the Industrial Applicability of Visual GUI Testing
- 2013
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- There is constant pressure on the software development industry to improve software quality and deliver new and innovative software faster and more efficiently. These demands affect all aspects of software development, from requirements engineering to testing. Today, much testing is performed withcostly, tedious and error-prone manual practices, especially on higher levels of system abstraction such as in system and acceptance testing. Increased test automation has been proposed as a key solution to help alleviate these test-related problems and has found increasing use. Automated tests typically consist of scripts that can be executed to give frequent feedback on the system's quality, ensure that previously correct functionality has not been negatively affected by changes, and also relieve resources e.g. human testers. However, as a system evolves, changes to the requirements also require maintenance of the tests, which can be costly. Furthermore, most automated test techniques approach testing from a lower level of system abstraction, e.g. testing the low-level components and functions, but their use for system testing has been questioned. Test techniques have been proposed that automate the interaction with the graphical user interface (GUI) of a system. For example, record and replay tools can be used to automate system testing by emulating a user interacting with the system. Existing literature have pointed out several shortcomings of these techniques, for example they are sensitive to GUI layout and code changes, require access to source code, are dependent on a specific platform or operating system access etc.In this thesis we evaluate if Visual GUI Testing (VGT), a novel technique that combines test scripts with image recognition, can be used to automate high-level, software testing. The strength of VGT lies in the use of image recognition that makes the technique robust in instances where previous GUIbased test techniques had limitations. Yet, VGT is only sparsely applied in industry and the academic body of knowledge on the technique is limited; there is a lack of empirical evaluation and experience reports from industrial use. To fill this gap in knowledge we have conducted three empirical case studies applying different VGT tools in industrial software projects at two Swedish companies. We have also conducted an industrial survey with participants from multiple companies. These studies provide evidence that VGT can be successfully applied in industrial software development projects, but also detail challenges, problems and limitations that need to be overcome for more widespread industrial adoption. Future work should focus on evaluating the long-term maintenance costs of VGT test scripts, since our studies only present initial evidence that these costs are manageable in relation to the benefits of the technique.
|
|
| 4. |
- Alégroth, Emil, 1984
(författare)
-
Random visual GUI testing: Proof of concept
- 2013
-
Ingår i: Proceedings of the International Conference on Software Engineering and Knowledge Engineering, SEKE. - 2325-9000 .- 2325-9086. ; 2013-January:January, s. 178-
-
Konferensbidrag (refereegranskat)abstract
- Market demands for higher quality software and shorter time-to-market delivery have resulted in a need for new automated software testing techniques. Most automated testing techniques are designed for regression testing that limit their fault finding ability to faults explicitly tested in scenarios/scripts. To overcome this limitation, companies define test processes with several test techniques, e.g. unit testing and random testing (RT). RT is a technique that can be performed manually or automatically with tools such as Fuzz, DART and Quickcheck. However, these tools operate on lower levels of system abstraction, leaving a gap for a Graphical User Interface (GUI), bitmap level, automated RT technique. In this paper we present proof of concept for Random Visual GUI testing (RVGT), a novel automated test technique that combines GUI based testing, Visual GUI Testing, with random testing. Proof of concept for RVGT is evaluated in a three phase study with results that show that RVGT is applicable for both functional and non-functional/quality requirement conformance testing. Furthermore, results from a survey performed in industry indicate that there is industrial need for the technique. These pivotal results show that further research into RVGT is warranted.
|
|
| 5. |
- Alégroth, Emil, 1984, et al.
(författare)
-
Transitioning manual system test suites to automated testing: An industrial case study
- 2013
-
Ingår i: Proceedings - IEEE 6th International Conference on Software Testing, Verification and Validation, ICST 2013. - : IEEE. - 9780769549682 ; , s. 56-65
-
Konferensbidrag (refereegranskat)abstract
- Visual GUI testing (VGT) is an emerging technique that provides software companies with the capability to automate previously time-consuming, tedious, and fault prone manual system and acceptance tests. Previous work on VGT has shown that the technique is industrially applicable, but has not addressed the real-world applicability of the technique when used by practitioners on industrial grade systems. This paper presents a case study performed during an industrial project with the goal to transition from manual to automated system testing using VGT. Results of the study show that the VGT transition was successful and that VGT could be applied in the industrial context when performed by practitioners but that there were several problems that first had to be solved, e.g. testing of a distributed system, tool volatility. These problems and solutions have been presented together with qualitative, and quantitative, data about the benefits of the technique compared to manual testing, e.g. greatly improved execution speed, feasible transition and maintenance costs, improved bug finding ability. The study thereby provides valuable, and previously missing, contributions about VGT to both practitioners and researchers. © 2013 IEEE.
|
|
| 6. |
- Liebel, Grischa, 1987, et al.
(författare)
-
State-of-practice in GUI-based system and acceptance testing: An industrial multiple-case study
- 2013
-
Ingår i: Proceedings - 39th Euromicro Conference Series on Software Engineering and Advanced Applications, SEAA 2013. Santander; Spain; 4 September 2013 through 6 September 2013. - 9780769550916 ; , s. 17-24
-
Konferensbidrag (refereegranskat)abstract
- Software testing is an essential means of evaluating software quality. System and acceptance tests aim to validate a system's conformance to its requirements on a high level of system abstraction. Therefore, they are generally performed by executing end-user scenarios through the system's graphical user interface (GUI). However, to the authors' best knowledge, there are no empirical studies that evaluate how GUI-based system and acceptance testing is performed in industrial practice. In this paper, we present a multiple-case study with the goal to investigate the state-of-practice of GUI-based system and acceptance testing at six software development companies of varying context. The main findings are that manual, GUI-based system testing is widespread and that automated GUI-based system and acceptance testing exists only on a small scale. Additionally, the study identifies core problems with GUI-based system and acceptance testing such as test tool limitations, high test costs and customer involvement in testing.
|
|
