Why many challenges with GUI test automation (will) remain

• Context: Automated testing is ubiquitous in modern software development and used to verify requirement conformance on all levels of system abstraction, including the system's graphical user interface (GUI). GUI-based test automation, like other automation, aims to reduce the cost and time for testing compared to alternative, manual approaches. Automation has been successful in reducing costs for other forms of testing (like unit- or integration testing) in industrial practice. However, we have not yet seen the same convincing results for automated GUI-based testing, which has instead been associated with multiple technical challenges. Furthermore, the software industry has struggled with some of these challenges for more than a decade with what seems like only limited progress. Objective: This systematic literature review takes a longitudinal perspective on GUI test automation challenges by identifying them and then investigating why the field has been unable to mitigate them for so many years. Method: The review is based on a final set of 49 publications, all reporting empirical evidence from practice or industrial studies. Statements from the publications are synthesized, based on a thematic coding, into 24 challenges related to GUI test automation. Results: The most reported challenges were mapped chronologically and further analyzed to determine how they and their proposed solutions have evolved over time. This chronological mapping of reported challenges shows that four of them have existed for almost two decades. Conclusion: Based on the analysis, we discuss why the key challenges with GUI-based test automation are still present and why some will likely remain in the future. For others, we discuss possible ways of how the challenges can be addressed. Further research should focus on finding solutions to the identified technical challenges with GUI-based test automation that can be resolved or mitigated. However, in parallel, we also need to acknowledge and try to overcome non-technical challenges. © 2021

Ämnesord

NATURVETENSKAP  -- Data- och informationsvetenskap -- Programvaruteknik (hsv//swe)

NATURAL SCIENCES  -- Computer and Information Sciences -- Software Engineering (hsv//eng)

TEKNIK OCH TEKNOLOGIER  -- Maskinteknik -- Produktionsteknik, arbetsvetenskap och ergonomi (hsv//swe)

ENGINEERING AND TECHNOLOGY  -- Mechanical Engineering -- Production Engineering, Human Work Science and Ergonomics (hsv//eng)

TEKNIK OCH TEKNOLOGIER  -- Annan teknik -- Övrig annan teknik (hsv//swe)

ENGINEERING AND TECHNOLOGY  -- Other Engineering and Technologies -- Other Engineering and Technologies not elsewhere specified (hsv//eng)

Nyckelord

GUI testing

System testing

Systematic literature review

Test automation

Automation

Cost reduction

Graphical user interfaces

Software design

Testing

Automated testing

Finding solutions

Graphical user interfaces (GUI)

Industrial practices

Software industry

Technical challenges

Integration testing

Publikations- och innehållstyp

ref (ämneskategori)

art (ämneskategori)