SwePub
Sök i SwePub databas

  Utökad sökning

Träfflista för sökning "WFRF:(Alégroth Emil) "

Sökning: WFRF:(Alégroth Emil)

  • Resultat 1-50 av 66
Sortera/gruppera träfflistan
   
NumreringReferensOmslagsbildHitta
1.
  • Lind, Emil, 1989-, et al. (författare)
  • Requirements Quality vs. Process and Stakeholders’ Well-Being : A Case of a Nordic Bank
  • 2023
  • Ingår i: Software Quality. - : Springer Science+Business Media B.V.. - 9783031314872 ; , s. 17-37
  • Konferensbidrag (refereegranskat)abstract
    • Requirements are key artefacts to describe the intended purpose of a software system. The quality of requirements is crucial for deciding what to do next, impacting the development process’ effectiveness and efficiency. However, we know very little about the connection between practitioners’ perceptions regarding requirements quality and its impact on the process or the feelings of the professionals involved in the development process. Objectives: This study investigates: i) How software development practitioners define requirements quality, ii) how the perceived quality of requirements impact process and stakeholders’ well-being, and iii) what are the causes and potential solutions for poor-quality requirements. Method: This study was performed as a descriptive interview study at a sub-organization of a Nordic bank that develops its own web and mobile apps. The data collection comprises interviews with 20 practitioners, including requirements engineers, developers, testers, and newly employed developers, with five interviewees from each group. Results: The results show that different roles have different views on what makes a requirement good quality. Participants highlighted that, in general, they experience negative emotions, more work, and overhead communication when they work with requirements they perceive to be of poor quality. The practitioners also describe positive effects on their performance and positive feelings when they work with requirements that they perceive to be good. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.
  •  
2.
  • Alégroth, Emil, 1984-, et al. (författare)
  • A Failed attempt at creating Guidelines for Visual GUI Testing : An industrial case study
  • 2021
  • Ingår i: Proceedings - 2021 IEEE 14th International Conference on Software Testing, Verification and Validation, ICST 2021. - : Institute of Electrical and Electronics Engineers Inc.. - 9781728168364 ; , s. 340-350
  • Konferensbidrag (refereegranskat)abstract
    • Software development is governed by guidelines that aim to improve the code's qualities, such as maintainability. However, whilst coding guidelines are commonplace for software, guidelines for testware are much less common. In particular, for GUI-based tests driven with image recognition, also referred to as Visual GUI Testing (VGT), explicit coding guidelines are missing.In this industrial case study, performed at the Swedish defence contractor Saab AB, we propose a set of coding guidelines for VGT and evaluate their impact on test scripts for an industrial, safety-critical system. To study the guidelines' effect on maintenance costs, five representative manual test cases are each translated with and without the proposed guidelines in the two VGT tools SikuliX and EyeAutomate. As such, 20 test scripts were developed, with a combined development cost of more than 100 man-hours. Three of the tests are then maintained by one researcher and two practitioners for another version of the system and costs measured to evaluate return on investment. This analysis is complemented with observations and interviews to elicit practitioners' perceptions and experiences with VGT.Results show that scripts developed with the guidelines had higher maintenance costs than scripts developed without guidelines. This is supported by qualitative results that many of the guidelines are considered inappropriate, superfluous or unnecessary due to the inherent properties of the scripts, e.g. their natural small size, linear flows, natural separation of concerns, and more. We conclude that there are differences between VGT scripts and software that prohibit direct translation of guidelines between the two. As such, we consider our study as a failure but argue that several lessons can be drawn from our results to guide future research into guidelines for VGT and GUI-based test automation. © 2021 IEEE.
  •  
3.
  •  
4.
  • Alégroth, Emil, 1984-, et al. (författare)
  • Characteristics that affect Preference of Decision Models for Asset Selection : An Industrial Questionnaire Survey
  • 2020
  • Ingår i: Software quality journal. - : Springer. - 0963-9314 .- 1573-1367. ; 28:4, s. 1675-1707
  • Tidskriftsartikel (refereegranskat)abstract
    • Modern software development relies on a combination of development and re-use of technical asset, e.g. software components, libraries and APIs.In the past, re-use was mostly conducted with internal assets but today external; open source, customer off-the-shelf (COTS) and assets developed through outsourcing are also common.This access to more asset alternatives presents new challenges regarding what assets to optimally chose and how to make this decision.To support decision-makers, decision-theory has been used to develop decision models for asset selection.However, very little industrial data has been presented in literature about the usefulness, or even perceived usefulness, of these models.Additionally, only limited information has been presented about what model characteristics that determine practitioner preference towards one model over another.Objective: The objective of this work is to evaluate what characteristics of decision models for asset selection that determine industrial practitioner preference of a model when given the choice of a decision-model of high precision or a model with high speed.Method: An industrial questionnaire survey is performed where a total of 33 practitioners, of varying roles, from 18 companies are tasked to compare two decision models for asset selection.Textual analysis and formal and descriptive statistics are then applied on the survey responses to answer the study's research questions.Results: The study shows that the practitioners had clear preference towards the decision model that emphasised speed over the one that emphasised decision precision.This conclusion was determined to be because one of the models was perceived faster, had lower complexity, had, was more flexible in use for different decisions, was more agile how it could be used in operation, its emphasis on people, its emphasis on ``good enough'' precision and ability to fail fast if a decision was a failure.Hence, seven characteristics that the practitioners considered important for their acceptance of the model.Conclusion: Industrial practitioner preference, which relates to acceptance, of decision models for asset selection is dependent on multiple characteristics that must be considered when developing a model for different types of decisions such as operational day-to-day decisions as well as more critical tactical or strategic decisions.The main contribution of this work are seven identified characteristics that can serve as industrial requirements for future research on decision models for asset selection.
  •  
5.
  • Alégroth, Emil, 1984, et al. (författare)
  • Conceptualization and evaluation of component-based testing unified with visual GUI testing: An empirical study
  • 2015
  • Ingår i: 2015 IEEE 8th International Conference on Software Testing, Verification and Validation, ICST 2015 - Proceedings. - 2159-4848. - 9781479971251
  • Konferensbidrag (refereegranskat)abstract
    • In this paper we present the results of a two-phase empirical study where we evaluate and compare the applicability of automated component-based Graphical User Interface (GUI) testing and Visual GUI Testing (VGT) in the tools GUITAR and a prototype tool we refer to as VGT GUITAR. First, GUI mutation operators are defined to create 18 faulty versions of an application on which both tools are then applied in an experiment. Results from 456 test case executions in each tool show, with statistical significance, that the component-based approach reports more false negatives than VGT for acceptance tests but that the VGT approach reports more false positives for system tests. Second, a case study is performed with larger open source applications, ranging from 8,803-55,006 lines of code. Results show that GUITAR is applicable in practice but has some challenges related to GUI component states. The results also show that VGT GUITAR is currently not applicable in practice and therefore requires further research and development. Based on the study's results we present areas of future work for both test approaches and conclude that the approaches have different benefits and drawbacks. The component-based approach is robust and executes tests faster than the VGT approach, with a factor of 3. However, the VGT approach can perform visual assertions and is perceived more flexible than the component- based approach. These conclusions let us hypothesize that a combination of the two approaches is the most suitable in practice and therefore warrants future research.
  •  
6.
  • Alégroth, Emil, et al. (författare)
  • Continuous Integration and Visual GUI Testing : Benefits and Drawbacks in Industrial Practice
  • 2018
  • Ingår i: Proceedings - 2018 IEEE 11th International Conference on Software Testing, Verification and Validation, ICST 2018. - : Institute of Electrical and Electronics Engineers Inc.. - 9781538650127 ; , s. 172-181
  • Konferensbidrag (refereegranskat)abstract
    • Continuous integration (CI) is growing in industrial popularity, spurred on by market trends towards faster delivery and higher quality software. A key facilitator of CI is automated testing that should be executed, automatically, on several levels of system abstraction. However, many systems lack the interfaces required for automated testing. Others lack test automation coverage of the system under test's (SUT) graphical user interface (GUI) as it is shown to the user. One technique that shows promise to solve these challenges is Visual GUI Testing (VGT), which uses image recognition to stimulate and assert the SUT's behavior. Research has presented the technique's applicability and feasibility in industry but only limited support, from an academic setting, that the technique is applicable in a CI environment. In this paper we presents a study from an industrial design research study with the objective to help bridge the gap in knowledge regarding VGT's applicability in a CI environment in industry. Results, acquired from interviews, observations and quantitative analysis of 17.567 test executions, collected over 16 weeks, show that VGT provides similar benefits to other automated test techniques for CI. However, several significant drawbacks, such as high costs, are also identified. The study concludes that, although VGT is applicable in an industrial CI environment, its severe challenges require more research and development before the technique becomes efficient in practice. © 2018 IEEE.
  •  
7.
  • Alégroth, Emil, 1984, et al. (författare)
  • Exploring the Presence of Technical Debt in Industrial GUI-based Testware: A Case Study
  • 2016
  • Ingår i: 2016 IEEE Ninth International Conference on Software Testing, Verification and Validation Workshops. - : IEEE. - 2159-4848. - 9781509036745 ; , s. 257-262
  • Konferensbidrag (refereegranskat)abstract
    • Technical debt (TD) is a concept used to describe a sub-optimal solution of a software artifact that negatively affects its comprehensibility, extendability and maintainability. As such, TD adversely affects the costs or quality associated with the artifact, which is also called interest. TD has through research been identified in all types of software artifacts, from architectural design to automated tests (Testware). However, research into testware technical debt (TTD) is limited and primarily focused on testing on lower level of system abstraction, i.e. unit-and integration tests, leaving a need for more TTD research on GUI-based testing. In this study we explore this gap in knowledge through an industrial case study at a Swedish avionics software development company. Four repositories are studied for the presence of TTD using expert interviews, semi-automated document analysis and automatic metric analysis. Results of the study provide initial support that the concept of TTD is applicable to GUI-based testware and show the presence of both TD items unique to GUI-based testware and items common to software. The implications of these results are that engineering best practices must be established for GUI-based testware to minimize TD interest.
  •  
8.
  •  
9.
  • Alégroth, Emil, 1984, et al. (författare)
  • Industrial application of visual GUI testing: Lessons learned
  • 2014
  • Ingår i: Continuous software engineering. - Cham : Springer International Publishing. ; 9783319112831, s. 127-140
  • Bokkapitel (övrigt vetenskapligt/konstnärligt)abstract
    • A large body of academic knowledge has been devoted to automated software testing in order to support the software market's demands for continuous software delivery. However, most of these automated techniques approach testing from lower levels of system abstraction, e.g., component level, which limit their applicability for high-level regression testing of, for instance, system and acceptance tests, thus forcing companies to perform these test activities manually, which is considered time consuming, tedious, and error prone. In this book chapter, we present visual GUI testing (VGT), a tool driven test technique that uses image recognition in order to interact and assert the correctness of a system under test (SUT) through the bitmap graphical user interface (GUI) that is shown to the user on the computer monitor. This approach makes VGT flexible and applicable to any SUT with a GUI but also allows VGT tools to emulate end-user behavior and therefore automate previously manual system and acceptance test cases. In addition to presenting the technique itself, this chapter will also present some VGT tools and empirically identified problems with the technique and how these problems can be mitigated in practice. Finally we will discuss how VGT can be used in the context of continuous software development in order to support market demands for quicker software delivery.
  •  
10.
  • Alégroth, Emil, 1984, et al. (författare)
  • JAutomate: A tool for system- and acceptance-test automation
  • 2013
  • Ingår i: Proceedings - IEEE 6th International Conference on Software Testing, Verification and Validation, ICST 2013. - : IEEE. - 9780769549682 ; , s. 439-446
  • Konferensbidrag (refereegranskat)abstract
    • System- and acceptance-testing are primarily performed with manual practices in current software industry. However, these practices have several issues, e.g. they are tedious, error prone and time consuming with costs up towards 40 percent of the total development cost. Automated test techniques have been proposed as a solution to mitigate these issues, but they generally approach testing from a lower level of system abstraction, leaving a gap for a flexible, high system-level test automation technique/tool. In this paper we present JAutomate, a Visual GUI Testing (VGT) tool that fills this gap by combining image recognition with record and replay functionality for high system-level test automation performed through the system under test's graphical user interface. We present the tool, its benefits compared to other similar techniques and manual testing. In addition, we compare JAutomate with two other VGT tools based on their static properties. Finally, we present the results from a survey with industrial practitioners that identifies testrelated problems that industry is currently facing and discuss how JAutomate can solve or mitigate these problems. © 2013 IEEE.
  •  
11.
  • Alégroth, Emil, 1984, et al. (författare)
  • Maintenance of automated test suites in industry: An empirical study on Visual GUI Testing
  • 2016
  • Ingår i: Information and Software Technology. - : Elsevier BV. - 0950-5849 .- 1873-6025. ; 73, s. 66-80
  • Tidskriftsartikel (refereegranskat)abstract
    • Context: Verification and validation (V&V) activities make up 20-50% of the total development costs of a software system in practice. Test automation is proposed to lower these V&V costs but available research only provides limited empirical data from industrial practice about the maintenance costs of automated tests and what factors affect these costs. In particular, these costs and factors are unknown for automated GUI-based testing. Objective: This paper addresses this lack of knowledge through analysis of the costs and factors associated with the maintenance of automated GUI-based tests in industrial practice. Method: An empirical study at two companies, Siemens and Saab, is reported where interviews about, and empirical work with, Visual GUI Testing is performed to acquire data about the technique's maintenance costs and feasibility. Results: 13 factors are observed that affect maintenance, e.g. tester knowledge/experience and test case complexity. Further, statistical analysis shows that developing new test scripts is costlier than maintenance but also that frequent maintenance is less costly than infrequent, big bang maintenance. In addition a cost model, based on previous work, is presented that estimates the time to positive return on investment (ROI) of test automation compared to manual testing. Conclusions: It is concluded that test automation can lower overall software development costs of a project while also having positive effects on software quality. However, maintenance costs can still be considerable and the less time a company currently spends on manual testing, the more time is required before positive, economic, ROI is reached after automation.
  •  
12.
  • Alégroth, Emil, 1984, et al. (författare)
  • On the industrial applicability of TextTest: An empirical case study
  • 2015
  • Ingår i: 2015 IEEE 8th International Conference on Software Testing, Verification and Validation, ICST 2015 - Proceedings. - 2159-4848. - 9781479971251
  • Konferensbidrag (refereegranskat)abstract
    • Software systems are becoming more complex, not least in their Graphical User Interfaces (GUIs), which presents challenges for existing testing practices. Pressure to reduce time to market leaves less time for manual testing and increases the importance of test automation. Previous research has identified several generations of automated GUI-based test approaches with different cost-benefit tradeoffs. Whilst test automation provides fast quality feedback it can be associated with high costs and inability to identify defects not explicitly anticipated by the test designer. TextTest is a capture-replay tool for GUI-based testing with a novel approach that overcomes several of the challenges experienced with previous approaches. Firstly the tool supports Approval Testing, an approach where ASCII-art representations of the GUI's visual state are used to verify correct application behavior at the system level. Secondly it records and replays test scripts in a user defined domain specific language (DSL) that is readable by all stakeholders. In this paper we present a three phase industrial case study that aims to identify TextTest's applicability in industrial practice. The paper reports that the tool is associated with (1) low script development costs due to recording functionality, (2) low maintenance costs, on average 7 minutes per test case, (3) better defect finding ability than manual system testing, (4) high test case execution performance (In this case 500 test cases in 20 minutes), (5) high script readability due to DSL defined scripts, and (6) test suites that are robust to change (In this case 93 percent per iteration). However, the tool requires a higher degree of technical skill for customization work, test maintainers need skills in designing regular expressions and the tool's applicability is currently restricted to Java and Python based applications.
  •  
13.
  • Alégroth, Emil, 1984 (författare)
  • On the Industrial Applicability of Visual GUI Testing
  • 2013
  • Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
    • There is constant pressure on the software development industry to improve software quality and deliver new and innovative software faster and more efficiently. These demands affect all aspects of software development, from requirements engineering to testing. Today, much testing is performed withcostly, tedious and error-prone manual practices, especially on higher levels of system abstraction such as in system and acceptance testing. Increased test automation has been proposed as a key solution to help alleviate these test-related problems and has found increasing use. Automated tests typically consist of scripts that can be executed to give frequent feedback on the system's quality, ensure that previously correct functionality has not been negatively affected by changes, and also relieve resources e.g. human testers. However, as a system evolves, changes to the requirements also require maintenance of the tests, which can be costly. Furthermore, most automated test techniques approach testing from a lower level of system abstraction, e.g. testing the low-level components and functions, but their use for system testing has been questioned. Test techniques have been proposed that automate the interaction with the graphical user interface (GUI) of a system. For example, record and replay tools can be used to automate system testing by emulating a user interacting with the system. Existing literature have pointed out several shortcomings of these techniques, for example they are sensitive to GUI layout and code changes, require access to source code, are dependent on a specific platform or operating system access etc.In this thesis we evaluate if Visual GUI Testing (VGT), a novel technique that combines test scripts with image recognition, can be used to automate high-level, software testing. The strength of VGT lies in the use of image recognition that makes the technique robust in instances where previous GUIbased test techniques had limitations. Yet, VGT is only sparsely applied in industry and the academic body of knowledge on the technique is limited; there is a lack of empirical evaluation and experience reports from industrial use. To fill this gap in knowledge we have conducted three empirical case studies applying different VGT tools in industrial software projects at two Swedish companies. We have also conducted an industrial survey with participants from multiple companies. These studies provide evidence that VGT can be successfully applied in industrial software development projects, but also detail challenges, problems and limitations that need to be overcome for more widespread industrial adoption. Future work should focus on evaluating the long-term maintenance costs of VGT test scripts, since our studies only present initial evidence that these costs are manageable in relation to the benefits of the technique.
  •  
14.
  • Alégroth, Emil, et al. (författare)
  • On the long-term use of visual gui testing in industrial practice : a case study
  • 2017
  • Ingår i: Empirical Software Engineering. - : Springer. - 1382-3256 .- 1573-7616. ; 22:6, s. 2937-2971
  • Tidskriftsartikel (refereegranskat)abstract
    • Visual GUI Testing (VGT) is a tool-driven technique for automated GUI-based testing that uses image recognition to interact with and assert the correctness of the behavior of a system through its GUI as it is shown to the user. The technique’s applicability, e.g. defect-finding ability, and feasibility, e.g. time to positive return on investment, have been shown through empirical studies in industrial practice. However, there is a lack of studies that evaluate the usefulness and challenges associated with VGT when used long-term (years) in industrial practice. This paper evaluates how VGT was adopted, applied and why it was abandoned at the music streaming application development company, Spotify, after several years of use. A qualitative study with two workshops and five well chosen employees is performed at the company, supported by a survey, which is analyzed with a grounded theory approach to answer the study’s three research questions. The interviews provide insights into the challenges, problems and limitations, but also benefits, that Spotify experienced during the adoption and use of VGT. However, due to the technique’s drawbacks, VGT has been abandoned for a new technique/framework, simply called the Test interface. The Test interface is considered more robust and flexible for Spotify’s needs but has several drawbacks, including that it does not test the actual GUI as shown to the user like VGT does. From the study’s results it is concluded that VGT can be used long-term in industrial practice but it requires organizational change as well as engineering best practices to be beneficial. Through synthesis of the study’s results, and results from previous work, a set of guidelines are presented that aim to aid practitioners to adopt and use VGT in industrial practice. However, due to the abandonment of the technique, future research is required to analyze in what types of projects the technique is, and is not, long-term viable. To this end, we also present Spotify’s Test interface solution for automated GUI-based testing and conclude that it has its own benefits and drawbacks.
  •  
15.
  • Alégroth, Emil, et al. (författare)
  • Overview of the ICST International Software Testing Contest
  • 2017
  • Ingår i: Proceedings - 10th IEEE International Conference on Software Testing, Verification and Validation, ICST 2017. - : IEEE Computer Society. - 9781509060313 ; , s. 550-551
  • Konferensbidrag (refereegranskat)abstract
    • In the software testing contest, practitioners and researcher's are invited to test their test approaches against similar approaches to evaluate pros and cons and which is perceivably the best. The 2017 iteration of the contest focused on Graphical User Interface-driven testing, which was evaluated on the testing tool TESTONA. The winner of the competition was announced at the closing ceremony of the international conference on software testing (ICST), 2017. © 2017 IEEE.
  •  
16.
  • Alégroth, Emil, 1984-, et al. (författare)
  • Practitioners' best practices to Adopt, Use or Abandon Model-based Testing with Graphical models for Software-intensive Systems
  • 2022
  • Ingår i: Empirical Software Engineering. - : SPRINGER. - 1382-3256 .- 1573-7616. ; 27:5
  • Tidskriftsartikel (refereegranskat)abstract
    • Model-based testing (MBT) has been extensively researched for software-intensive systems but, despite the academic interest, adoption of the technique in industry has been sparse. This phenomenon has been observed by our industrial partners for MBT with graphical models. They perceive one cause to be a lack of evidence-based MBT guidelines that, in addition to technical guidelines, also take non-technical aspects into account. This hypothesis is supported by a lack of such guidelines in the literature. Objective: The objective of this study is to elicit, and synthesize, MBT experts' best practices for MBT with graphical models. The results aim to give guidance to practitioners and aspire to give researchers new insights to inspire future research. Method: An interview survey is conducted using deep, semi-structured, interviews with an international sample of 17 MBT experts, in different roles, from software industry. Interview results are synthesised through semantic equivalence analysis and verified by MBT experts from industrial practice. Results: 13 synthesised conclusions are drawn from which 23 best-practice guidelines are derived for the adoption, use and abandonment of the technique. In addition, observations and expert insights are discussed that help explain the lack of wide-spread adoption of MBT with graphical models in industrial practice. Conclusions: Several technical aspects of MBT are covered by the results as well as conclusions that cover process- and organizational factors. These factors relate to the mindset, knowledge, organization, mandate and resources that enable the technique to be used effectively within an organization. The guidelines presented in this work complement existing knowledge and, as a primary objective, provide guidance for industrial practitioners to better succeed with MBT with graphical models.
  •  
17.
  • Alégroth, Emil, 1984 (författare)
  • Random visual GUI testing: Proof of concept
  • 2013
  • Ingår i: Proceedings of the International Conference on Software Engineering and Knowledge Engineering, SEKE. - 2325-9000 .- 2325-9086. ; 2013-January:January, s. 178-
  • Konferensbidrag (refereegranskat)abstract
    • Market demands for higher quality software and shorter time-to-market delivery have resulted in a need for new automated software testing techniques. Most automated testing techniques are designed for regression testing that limit their fault finding ability to faults explicitly tested in scenarios/scripts. To overcome this limitation, companies define test processes with several test techniques, e.g. unit testing and random testing (RT). RT is a technique that can be performed manually or automatically with tools such as Fuzz, DART and Quickcheck. However, these tools operate on lower levels of system abstraction, leaving a gap for a Graphical User Interface (GUI), bitmap level, automated RT technique. In this paper we present proof of concept for Random Visual GUI testing (RVGT), a novel automated test technique that combines GUI based testing, Visual GUI Testing, with random testing. Proof of concept for RVGT is evaluated in a three phase study with results that show that RVGT is applicable for both functional and non-functional/quality requirement conformance testing. Furthermore, results from a survey performed in industry indicate that there is industrial need for the technique. These pivotal results show that further research into RVGT is warranted.
  •  
18.
  • Alégroth, Emil, et al. (författare)
  • Replicating Rare Software Failures with Exploratory Visual GUI Testing
  • 2017
  • Ingår i: IEEE Software. - : IEEE Computer Society. - 0740-7459 .- 1937-4194. ; 34:5, s. 53-59
  • Tidskriftsartikel (refereegranskat)abstract
    • Saab AB developed software that had a defect that manifested itself only after months of continuous system use. After years of customer failure reports, the defect still persisted, until Saab developed failure replication based on visual GUI testing. © 1984-2012 IEEE.
  •  
19.
  • Alégroth, Emil, 1984, et al. (författare)
  • Special issue on new generations of UI testing
  • 2021
  • Ingår i: Software Testing Verification and Reliability. - : Wiley. - 0960-0833 .- 1099-1689. ; 31:3
  • Tidskriftsartikel (övrigt vetenskapligt/konstnärligt)
  •  
20.
  • Alégroth, Emil, 1984, et al. (författare)
  • Teaching scrum – what we did, what we will do and what impedes us
  • 2015
  • Ingår i: Lecture Notes in Business Information Processing. - 1865-1348 .- 1865-1356. - 9783319186115 ; 212, s. 361-362
  • Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
    • This paper analyses the way we teach Scrum. We reflect on our intended learning outcomes, which challenges we find in teaching Scrum and which lessons we have learned during the last four years. We also give an outlook on the way we want to introduce and apply Scrum in our teaching and how we intend to improve the curriculum.
  •  
21.
  •  
22.
  • Alégroth, Emil, et al. (författare)
  • Towards a mapping of software technical debt onto testware
  • 2017
  • Ingår i: Proceedings - 43rd Euromicro Conference on Software Engineering and Advanced Applications, SEAA 2017. - : Institute of Electrical and Electronics Engineers Inc.. - 9781538621400 ; , s. 404-411
  • Konferensbidrag (refereegranskat)abstract
    • Technical Debt (TD) is a metaphor used to explain the negative impacts that sub-optimal design decisions have in the long-term perspective of a software project. Although TD is acknowledged by both researchers and practitioners to have strong negative impact on Software development, its study on Testware has so far been very limited. A gap in knowledge that is important to address due to the growing popularity of Testware (scripted automated testing) in software development practice.In this paper we present a mapping analysis that connects 21 well-known, Software, object-oriented TD items to Testware, establishing them as Testware Technical Debt (TTD) items. The analysis indicates that most Software TD items are applicable or observable as TTD items, often in similar form and with roughly the same impact as for Software artifacts (e.g. reducing quality of the produced artifacts, lowering the effectiveness and efficiency of the development process whilst increasing costs). In the analysis, we also identify three types of connections between software TD and TTD items with varying levels of impact and criticality. Additionally, the study finds support for previous research results in which specific TTD items unique to Testware were identified. Finally, the paper outlines several areas of future research into TTD. © 2017 IEEE.
  •  
23.
  • Alégroth, Emil, 1984, et al. (författare)
  • Transitioning manual system test suites to automated testing: An industrial case study
  • 2013
  • Ingår i: Proceedings - IEEE 6th International Conference on Software Testing, Verification and Validation, ICST 2013. - : IEEE. - 9780769549682 ; , s. 56-65
  • Konferensbidrag (refereegranskat)abstract
    • Visual GUI testing (VGT) is an emerging technique that provides software companies with the capability to automate previously time-consuming, tedious, and fault prone manual system and acceptance tests. Previous work on VGT has shown that the technique is industrially applicable, but has not addressed the real-world applicability of the technique when used by practitioners on industrial grade systems. This paper presents a case study performed during an industrial project with the goal to transition from manual to automated system testing using VGT. Results of the study show that the VGT transition was successful and that VGT could be applied in the industrial context when performed by practitioners but that there were several problems that first had to be solved, e.g. testing of a distributed system, tool volatility. These problems and solutions have been presented together with qualitative, and quantitative, data about the benefits of the technique compared to manual testing, e.g. greatly improved execution speed, feasible transition and maintenance costs, improved bug finding ability. The study thereby provides valuable, and previously missing, contributions about VGT to both practitioners and researchers. © 2013 IEEE.
  •  
24.
  • Alégroth, Emil, 1984 (författare)
  • Visual GUI Testing: Automating High-level Software Testing in Industrial Practice
  • 2015
  • Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
    • Software Engineering is at the verge of a new era where continuous releases are becoming more common than planned long-term projects. In this context test automation will become essential on all levels of system abstraction to meet the market's demands on time-to-market and quality. Hence, automated tests are required from low-level software components, tested with unit tests, up to the pictorial graphical user interface (GUI), tested with user emulated system and acceptance tests. Thus far, research has provided industry with a plethora of automation solutions for lower level testing but GUI level testing is still primarily a manual, and therefore costly and tedious, activity in practice.We have identfied three generations of automated GUI-based testing. The first (1st) generation relies on GUI coordinates but is not used in practice due to unfeasible maintenance costs caused by fragility to GUI change. Second (2nd) generation tools instead operate against the system's GUI architecture, libraries or application programming interfaces. Whilst this approach is successfully used in practice, it does not verify the GUI's appearance and it isrestricted to specific GUI technologies, programming languages and platforms.The third (3rd) generation, referred to as Visual GUI Testing (VGT), is an emerging technique in industrial practice with properties that mitigate thechallenges experienced with previous techniques. VGT is defined as a tool-driven test technique where image recognition is used to interact with, andassert, a system's behavior through its pictorial GUI as it is shown to the user in user-emulated, automated, system or acceptance tests. Automated tests that produce results of quality on par with a human tester and is therefore an effective complement to reduce the aforementioned challenges with manualtesting. However, despite its benefits, the technique is only sparsely used inindustry and the academic body of knowledge contains little empirical supportfor the technique's industrial viability.This thesis presents a broad evaluation of VGT's capabilities, obtained through a series of case studies and experiments performed in academia andSwedish industry. The research follows an incremental methodology that began with experimentation with VGT, followed by industrial studies that wereconcluded with a study of VGT's use at a company over several years. Resultsof the research show that VGT is viable for use in industrial practice with better defect-finding ability than manual tests, ability to test any GUI basedsystem, high learnability, feasible maintenance costs and both short and longterm company benefits. However, there are still challenges associated with the successful adoption, use and long-term use of VGT in a company, the most crucial that suitable development and maintenance practices are used. This thesis thereby concludes that VGT can be used in industrial practice and aims to provides guidance to practitioners that seek to do so. Additionally, this work aims to be a stepping stone for academia to explore new test solutions that build on image recognition technology to improve the state-of-art.
  •  
25.
  • Alégroth, Emil, 1984, et al. (författare)
  • Visual GUI testing in practice: challenges, problemsand limitations
  • 2015
  • Ingår i: Empirical Software Engineering. - : Springer Science and Business Media LLC. - 1573-7616 .- 1382-3256. ; 20:3, s. 694-744
  • Tidskriftsartikel (refereegranskat)abstract
    • In today's software development industry, high-level tests such as Graphical User Interface (GUI) based system and acceptance tests are mostly performed with manual practices that are often costly, tedious and error prone. Test automation has been proposed to solve these problems but most automation techniques approach testing from a lower level of system abstraction. Their suitability for high-level tests has therefore been questioned. High-level test automation techniques such as Record and Replay exist, but studies suggest that these techniques suffer from limitations, e.g. sensitivity to GUI layout or code changes, system implementation dependencies, etc. Visual GUI Testing (VGT) is an emerging technique in industrial practice with perceived higher flexibility and robustness to certain GUI changes than previous high-level (GUI) test automation techniques. The core of VGT is image recognition which is applied to analyze and interact with the bitmap layer of a system's front end. By coupling image recognition with test scripts, VGT tools can emulate end user behavior on almost any GUI-based system, regardless of implementation language, operating system or platform. However, VGT is not without its own challenges, problems and limitations (CPLs) but, like for many other automated test techniques, there is a lack of empirically-based knowledge of these CPLs and how they impact industrial applicability. Crucially, there is also a lack of information on the cost of applying this type of test automation in industry. This manuscript reports an empirical, multi-unit case study performed at two Swedish companies that develop safety-critical software. It studies their transition from manual system test cases into tests automated with VGT. In total, four different test suites that together include more than 300 high-level system test cases were automated for two multi-million lines of code systems. The results show that the transitioned test cases could find defects in the tested systems and that all applicable test cases could be automated. However, during these transition projects a number of hurdles had to be addressed; a total of 58 different CPLs were identified and then categorized into 26 types. We present these CPL types and an analysis of the implications for the transition to and use of VGT in industrial software development practice. In addition, four high-level solutions are presented that were identified during the study, which would address about half of the identified CPLs. Furthermore, collected metrics on cost and return on investment of the VGT transition are reported together with information about the VGT suites' defect finding ability. Nine of the identified defects are reported, 5 of which were unknown to testers with extensive experience from using the manual test suites. The main conclusion from this study is that even though there are many challenges related to the transition and usage of VGT, the technique is still valuable, flexible and considered cost-effective by the industrial practitioners. The presented CPLs also provide decision support in the use and advancement of VGT and potentially other automated testing techniques similar to VGT, e.g. Record and Replay.
  •  
26.
  • Ardito, Luca, et al. (författare)
  • Towards automated translation between generations of GUI-based tests for mobile devices
  • 2018
  • Ingår i: Companion Proceedings for the ISSTA/ECOOP 2018 Workshops. - New York, NY, USA : Association for Computing Machinery, Inc. - 9781450359399 ; , s. 46-53
  • Konferensbidrag (refereegranskat)abstract
    • Market demands for faster delivery and higher software quality are progressively becoming more stringent. A key hindrance for software companies to meet such demands is how to test the software due to to the intrinsic costs of development, maintenance and evolution of testware. Especially since testware should be defined, and aligned, with all layers of system under test (SUT), including all graphical user interface (GUI) abstraction levels. These levels can be tested with different generations of GUI-based test approaches, where 2nd generation, or Layout-based, tests leverage GUI properties and 3rd generation, or Visual, tests make use of image recognition. The two approaches provide different benefits and drawbacks and are seldom used together because of the aforementioned costs, despite growing academic evidence of the complementary benefits. In this work we propose the proof of concept of a novel two-step translation approach for Android GUI testing that we aim to implement, where a translator first creates a technology independent script with actions and elements of the GUI, and then translates it to a script with the syntax chosen by the user. The approach enables users to translate Layout-based to Visual scripts and vice versa, to gain the benefits (e.g. robustness, speed and ability to emulate the user) of both generations, whilst minimizing the drawbacks (e.g. development and maintenance costs). We outline our approach from a technical perspective, discuss some of the key challenges with the realization of our approach, evaluate the feasibility and the advantages provided by our approach on an open-source Android application, and discuss the potential industrial impact of this work. © 2018 ACM.
  •  
27.
  • Bauer, Andreas, et al. (författare)
  • Augmented Testing to support Manual GUI-based Regression Testing : An Empirical Study
  • 2024
  • Ingår i: Empirical Software Engineering. - : Springer. - 1382-3256 .- 1573-7616. ; 29:6
  • Tidskriftsartikel (refereegranskat)abstract
    • Context: Manual graphical user interface (GUI) software testing presents a substantial part of the overall practiced testing efforts, despite various research efforts to further increase test automation. Augmented Testing (AT), a novel approach for GUI testing, aims to aid manual GUI-based testing through a tool-supported approach where an intermediary visual layer is rendered between the system under test (SUT) and the tester, superimposing relevant test information.Objective: The primary objective of this study is to gather empirical evidence regarding AT's efficiency compared to manual GUI-based regression testing. Existing studies involving testing approaches under the AT definition primarily focus on exploratory GUI testing, leaving a gap in the context of regression testing. As a secondary objective, we investigate AT's benefits, drawbacks, and usability issues when deployed with the demonstrator tool, Scout.Method: We conducted an experiment involving 13 industry professionals, from six companies, comparing AT to manual GUI-based regression testing. These results were complemented by interviews and Bayesian data analysis (BDA) of the study's quantitative results.Results: The results of the Bayesian data analysis revealed that the use of AT shortens test durations in 70% of the cases on average, concluding that AT is more efficient.When comparing the means of the total duration to perform all tests, AT reduced the test duration by 36% in total. Participant interviews highlighted nine benefits and eleven drawbacks of AT, while observations revealed four usability issues.Conclusion: This study makes an empirical contribution to understanding Augmented Testing, a promising approach to improve the efficiency of GUI-based regression testing in practice. Furthermore, it underscores the importance of continual refinements of AT.
  •  
28.
  • Bauer, Andreas, et al. (författare)
  • Code review guidelines for GUI-based testing artifacts
  • 2023
  • Ingår i: Information and Software Technology. - : Elsevier. - 0950-5849 .- 1873-6025. ; 163
  • Forskningsöversikt (refereegranskat)abstract
    • Context: Review of software artifacts, such as source or test code, is a common practice in industrial practice. However, although review guidelines are available for source and low-level test code, for GUI-based testing artifacts, such guidelines are missing. Objective: The goal of this work is to define a set of guidelines from literature about production and test code, that can be mapped to GUI-based testing artifacts. Method: A systematic literature review is conducted, using white and gray literature to identify guidelines for source and test code. These synthesized guidelines are then mapped, through examples, to create actionable, and applicable, guidelines for GUI-based testing artifacts. Results: The results of the study are 33 guidelines, summarized in nine guideline categories, that are successfully mapped as applicable to GUI-based testing artifacts. Of the collected literature, only 10 sources contained test-specific code review guidelines. These guideline categories are: perform automated checks, use checklists, provide context information, utilize metrics, ensure readability, visualize changes, reduce complexity, check conformity with the requirements and follow design principles and patterns. Conclusion: This pivotal set of guidelines provides an industrial contribution in filling the gap of general guidelines for review of GUI-based testing artifacts. Additionally, this work highlights, from an academic perspective, the need for future research in this area to also develop guidelines for other specific aspects of GUI-based testing practice, and to take into account other facets of the review process not covered by this work, such as reviewer selection. © 2023 The Author(s)
  •  
29.
  • Bauer, Andreas (författare)
  • Towards Collaborative GUI-based Testing
  • 2023
  • Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
    • Context:Contemporary software development is a socio-technical activity requiring extensive collaboration among individuals with diverse expertise.Software testing is an integral part of software development that also depends on various expertise.GUI-based testing allows assessing a system’s GUI and its behavior through its graphical user interface.Collaborative practices in software development, like code reviews, not only improve software quality but also promote knowledge exchange within teams.Similar benefits could be extended to other areas of software engineering, such as GUI-based testing.However, collaborative practices for GUI-based testing necessitate a unique approach since general software development practices, perceivably, can not be directly transferred to software testing.Goal:This thesis contributes towards a tool-supported approach enabling collaborative GUI-based testing.Our distinct goals are (1) to identify processes and guidelines to enable collaboration on GUI-based testing artifacts and (2) to operationalize tool support to aid this collaboration.Method:We conducted a systematic literature review identifying code review guidelines for GUI-based testing.Further, we conducted a controlled experiment to assess the efficiency and potential usability issues of Augmented Testing.Results:We provided guidelines for reviewing GUI-based testing artifacts, which aid contributors and reviewers during code reviews.We further provide empirical evidence that Augmented Testing is not only an efficient approach to GUI-based testing but also usable for non-technical users, making it a promising subject for further research in collaborative GUI-based testing.Conclusion:Code review guidelines aid collaboration through discussions, and a suitable testing approach can serve as a platform to operationalize collaboration.Collaborative GUI-based testing has the potential to improve the efficiency and effectiveness of such testing.
  •  
30.
  • Bauer, Andreas, et al. (författare)
  • We Tried and Failed : An Experience Report on a Collaborative Workflow for GUI-based Testing
  • 2023
  • Ingår i: Proceedings - 2023 IEEE 16th International Conference on Software Testing, Verification and Validation Workshops, ICSTW. - : Institute of Electrical and Electronics Engineers (IEEE). - 9798350333350 ; , s. 1-9
  • Konferensbidrag (refereegranskat)abstract
    • Modern software development is a team-based effort supported by tools, processes, and practices. One integral part is automated testing, where developers incorporate automated tests on multiple levels of system abstraction, from low-level unit tests to high-level system tests and Graphical User Interface (GUI) tests. Furthermore, the common practices of code reviews allow collaboration on artifacts based on discussions that improve the artifact's quality and to share information within the team. However, the characteristics of GUI-based tests, due to the level of abstraction and visual elements, introduce additional requirements and complexities compared to code or lower-level test code review, delimiting the practice benefits.The objective of this work is to propose a tool-supported workflow that enables active collaboration among stakeholders and improves the efficiency and effectiveness of team-based development of GUI-based tests.To evaluate the workflow, and show proof of concept, a technical demonstrator for merging of GUI-based tests was to be developed. However, during its development, we encountered several unforeseen challenges that forced us to halt its development. We report the negative results from this development and the main challenges we encountered, as well as the rationale and the decisions we took towards this workflow.In conclusion, this work presents a negative research result on a failed attempt to propose a tool-supported workflow that enables active collaboration on GUI-based tests. The outcome and learnings of this work are intended to guide future research and prevent researchers from falling into the same pitfalls we did. © 2023 IEEE.
  •  
31.
  • Borg, Markus, et al. (författare)
  • Selecting component sourcing options : A survey of software engineering's broader make-or-buy decisions
  • 2019
  • Ingår i: Information and Software Technology. - : Elsevier B.V.. - 0950-5849 .- 1873-6025. ; 112, s. 18-34
  • Tidskriftsartikel (refereegranskat)abstract
    • Context: Component-based software engineering (CBSE) is a common approach to develop and evolve contemporary software systems. When evolving a system based on components, make-or-buy decisions are frequent, i.e., whether to develop components internally or to acquire them from external sources. In CBSE, several different sourcing options are available: (1) developing software in-house, (2) outsourcing development, (3) buying commercial-off-the-shelf software, and (4) integrating open source software components. Objective: Unfortunately, there is little available research on how organizations select component sourcing options (CSO) in industry practice. In this work, we seek to contribute empirical evidence to CSO selection. Method: We conduct a cross-domain survey on CSO selection in industry, implemented as an online questionnaire. Results: Based on 188 responses, we find that most organizations consider multiple CSOs during software evolution, and that the CSO decisions in industry are dominated by expert judgment. When choosing between candidate components, functional suitability acts as an initial filter, then reliability is the most important quality. Conclusion: We stress that future solution-oriented work on decision support has to account for the dominance of expert judgment in industry. Moreover, we identify considerable variation in CSO decision processes in industry. Finally, we encourage software development organizations to reflect on their decision processes when choosing whether to make or buy components, and we recommend using our survey for a first benchmarking.
  •  
32.
  • Borg, Markus, et al. (författare)
  • Selecting Software Component Sourcing Options : Detailed Survey Description and Analysis
  • 2018
  • Rapport (övrigt vetenskapligt/konstnärligt)abstract
    • Component-based software engineering (CBSE) is a common approach to develop and evolve contemporary software systems. When evolving a system based on components, make-or-buy decisions are frequent, i.e., whether to develop components internally or to acquire them fromexternal sources. In CBSE, several different sourcing options are available: 1) developing software in-house, 2) outsourcing development, 3) buying commercial-off-the-shelf software, and 4) integrating open source software components. Unfortunately, there is little available research on howorganizations select component sourcing options (CSO) in industry practice. In this work, we seek to contribute empirical evidence to CSO selection. Method: We conduct a cross-domain survey on CSO selection in industry, implemented as an online questionnaire. Based on 188 responses, we find that most organizations consider multiple CSOs during software evolution, and that the CSO decisions in industry are dominated by expert judgment. When choosing between candidate components, functional suitability acts as an initial filter, then reliability is the most important quality. We stress that future solution-oriented work on decision support has to account for the dominance of expert judgment in industry. Moreover, we identify considerable variation in CSO decision processes in industry. Finally, we encourage software development organizations to reflect on their decision processes when choosing whether to make or buy components, and we recommend using our survey for a first benchmarking.
  •  
33.
  • Borg, Markus, et al. (författare)
  • Software Engineers' Information Seeking Behavior in Change Impact Analysis : An Interview Study
  • 2017
  • Ingår i: IEEE International Conference on Program Comprehension. - : IEEE Computer Society. - 9781538605356 ; , s. 12-22, s. 12-22
  • Konferensbidrag (refereegranskat)abstract
    • Software engineers working in large projects must navigate complex information landscapes. Change Impact Analysis (CIA) is a task that relies on engineers' successful information seeking in databases storing, e.g., source code, requirements, design descriptions, and test case specifications. Several previous approaches to support information seeking are task-specific, thus understanding engineers' seeking behavior in specific tasks is fundamental. We present an industrial case study on how engineers seek information in CIA, with a particular focus on traceability and development artifacts that are not source code. We show that engineers have different information seeking behavior, and that some do not consider traceability particularly useful when conducting CIA. Furthermore, we observe a tendency for engineers to prefer less rigid types of support rather than formal approaches, i.e., engineers value support that allows flexibility in how to practically conduct CIA. Finally, due to diverse information seeking behavior, we argue that future CIA support should embrace individual preferences to identify change impact by empowering several seeking alternatives, including searching, browsing, and tracing. © 2017 IEEE.
  •  
34.
  • Chatzipetrou, Panagiota, Assistant Professor, 1984-, et al. (författare)
  • Component attributes and their importance in decisions and component selection
  • 2020
  • Ingår i: Software quality journal. - : Springer Science and Business Media LLC. - 0963-9314 .- 1573-1367. ; 28, s. 567-593
  • Tidskriftsartikel (refereegranskat)abstract
    • Component-based software engineering is a common approach in the development and evolution of contemporary software systems. Different component sourcing options are available, such as: (1) Software developed internally (in-house), (2) Software developed outsourced, (3) Commercial off-the-shelf software, and (4) Open-Source Software. However, there is little available research on what attributes of a component are the most important ones when selecting new components. The objective of this study is to investigate what matters the most to industry practitioners when they decide to select a component. We conducted a cross-domain anonymous survey with industry practitioners involved in component selection. First, the practitioners selected the most important attributes from a list. Next, they prioritized their selection using the Hundred-Dollar ($100) test. We analyzed the results using compositional data analysis. The results of this exploratory analysis showed that cost was clearly considered to be the most important attribute for component selection. Other important attributes for the practitioners were: support of the component, longevity prediction, and level of off-the-shelf fit to product. Moreover, several practitioners still consider in-house software development to be the sole option when adding or replacing a component. On the other hand, there is a trend to complement it with other component sourcing options and, apart from cost, different attributes factor into their decision. Furthermore, in our analysis, nonparametric tests and biplots were used to further investigate the practitioners’ inherent characteristics. It seems that smaller and larger organizations have different views on what attributes are the most important, and the most surprising finding is their contrasting views on the cost attribute: larger organizations with mature products are considerably more cost aware.
  •  
35.
  • Chatzipetrou, Panagiota, Assistant Professor, 1984-, et al. (författare)
  • Component selection in software engineering - Which attributes are the most important in the decision process?
  • 2018
  • Ingår i: Proceedings - 44th Euromicro Conference on Software Engineering and Advanced Applications, SEAA 2018. - : IEEE conference proceedings. - 9781538673829 ; , s. 198-205, s. 198-205
  • Konferensbidrag (refereegranskat)abstract
    • Component-based software engineering is a common approach to develop and evolve contemporary software systems where different component sourcing options are available: 1)Software developed internally (in-house), 2)Software developed outsourced, 3)Commercial of the shelf software, and 4) Open Source Software. However, there is little available research on what attributes of a component are the most important ones when selecting new components. The object of the present study is to investigate what matters the most to industry practitioners during component selection. We conducted a cross-domain anonymous survey with industry practitioners involved in component selection. First, the practitioners selected the most important attributes from a list. Next, they prioritized their selection using the Hundred-Dollar ($100) test. We analyzed the results using Compositional Data Analysis. The descriptive results showed that Cost was clearly considered the most important attribute during the component selection. Other important attributes for the practitioners were: Support of the component, Longevity prediction, and Level of off-the-shelf fit to product. Next, an exploratory analysis was conducted based on the practitioners' inherent characteristics. Nonparametric tests and biplots were used. It seems that smaller organizations and more immature products focus on different attributes than bigger organizations and mature products which focus more on Cost. .
  •  
36.
  • Coppola, Riccardo, et al. (författare)
  • A taxonomy of metrics for GUI-based testing research : A systematic literature review
  • 2022
  • Ingår i: Information and Software Technology. - : Elsevier B.V.. - 0950-5849 .- 1873-6025. ; 152
  • Forskningsöversikt (refereegranskat)abstract
    • Context: GUI-based testing is a sub-field of software testing research that has emerged in the last three decades. GUI-based testing techniques focus on verifying the functional conformance of the system under test (SUT) through its graphical user interface. However, despite the research domains growth, studies in the field have low reproducibility and comparability. One observed cause of these phenomena is identified as a lack of research rigor and commonly used metrics, including coverage metrics. Objective: We aim to identify the most commonly used metrics in the field and formulate a taxonomy of coverage metrics for GUI-based testing research. Method: We adopt an evidence-based approach to build the taxonomy through a systematic literature review of studies in the GUI-based testing domain. Identified papers are then analyzed with Open and Axial Coding techniques to identify hierarchical and mutually exclusive categories of metrics with common characteristics, usages, and applications. Results: Through the analysis of 169 papers and 315 metric definitions, we obtained a taxonomy with 55 codes (common names for metrics), 17 metric categories, and 4 higher level categories: Functional Level, GUI Level, Model Level and Code Level. We measure a higher number of mentions of Model and Code level metrics over Functional and GUI level metrics. Conclusions: We propose a taxonomy for use in future GUI-based testing research to improve the general quality of studies in the domain. In addition, the taxonomy is perceived to help enable more replication studies as well as macro-analysis of the current body of research. © 2022 Elsevier B.V.
  •  
37.
  • Coppola, Riccardo, et al. (författare)
  • On Effectiveness and Efficiency of Gamified Exploratory GUI Testing
  • 2024
  • Ingår i: IEEE Transactions on Software Engineering. - : IEEE Computer Society. - 0098-5589 .- 1939-3520. ; 50:2, s. 322-337
  • Tidskriftsartikel (refereegranskat)abstract
    • Context: Gamification appears to improve enjoyment and quality of execution of software engineering activities, including software testing. Though commonly employed in industry, manual exploratory testing of web application GUIs was proven to be mundane and expensive. Gamification applied to that kind of testing activity has the potential to overcome its limitations, though no empirical research has explored this area yet.Goal: Collect preliminary insights on how gamification, when performed by novice testers, affects the effectiveness, efficiency, test case realism, and user experience in exploratory testing of web applications.Method: Common gamification features augment an existing exploratory testing tool: Final Score with Leaderboard, Injected Bugs, Progress Bar, and Exploration Highlights. The original tool and the gamified version are then compared in an experiment involving 144 participants. User experience is elicited using the Technology Acceptance Model (TAM) questionnaire instrument.Results: Statistical analysis identified several significant differences for metrics that represent the effectiveness and efficiency of tests showing an improvement in coverage when they were developed with gamification. Additionally, user experience is improved with gamification.Conclusions: Gamification of exploratory testing has a tangible effect on how testers create test cases for web applications. While the results are mixed, the effects are most beneficial and interesting and warrant more research in the future. Further research shall be aimed at confirming the presented results in the context of state-of-the-art testing tools and real-world development environments. 
  •  
38.
  • Coppola, Riccardo, et al. (författare)
  • Translation from layout-based to visual android test scripts : An empirical evaluation
  • 2021
  • Ingår i: Journal of Systems and Software. - : Elsevier Inc.. - 0164-1212 .- 1873-1228. ; 171
  • Tidskriftsartikel (refereegranskat)abstract
    • Mobile GUI tests can be classified as layout-based – i.e. using GUI properties as locators – or Visual – i.e. using widgets’ screen captures as locators –. Visual test scripts require significant maintenance efforts to be kept aligned with the tested application as it evolves or it is ported to different devices. This work aims to conceptualize a translation-based approach to automatically derive Visual tests from existing layout-based counterparts or repair them when graphical changes occur, and to develop a tool that implements and validates the approach. We present TOGGLE, a tool that translates Espresso layout-based tests for Android apps to Visual tests that conform to either SikuliX, EyeAutomate, or a combination of the two tools’ syntax. An experiment is conducted to measure the precision of the translation approach, which is evaluated on maintenance tasks triggered by graphical changes due to device diversity. Our results demonstrate the feasibility of a translation-based approach, show that script portability to different devices is improved (from 32% to 93%), and indicate that translation can repair up to 90% of Visual locators in failing tests. GUI test translation mitigates challenges with Visual tests like maintenance effort and portability, enabling their wider use in industrial practice. © 2020 Elsevier Inc.
  •  
39.
  • Coppola, Riccardo, et al. (författare)
  • Translation from Visual to Layout-based Android Test Cases : A Proof of Concept
  • 2020
  • Ingår i: Proceedings - 2020 IEEE 13th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2020. - : Institute of Electrical and Electronics Engineers Inc.. - 9781728110752 ; , s. 74-83
  • Konferensbidrag (refereegranskat)abstract
    • Context: 2nd generation (Layout-based) and 3rd generation (Visual) GUI testing are two approaches for testing mobile GUIs, both with individual benefits and drawbacks. Previous research has presented approaches to translate 2nd generation scripts to 3rd generation scripts but not the vice versa. Goal: The objective of this work is to provide Proof of Concept of the effectiveness of automatic translation between existing 3rd generation test scripts to 2nd generation test scripts. Method: A tool architecture is presented and implemented in a tool capable of translating most 3rd generation interactions with the GUI of an Android app into 2nd generation instructions and oracles for the Espresso testing tool.Results: We validate our approach on two test suites of our own creation, consisting of 30 test cases each. The measured success rate of the translation is 96.7% (58 working test cases out of 60 applications of the translator). Conclusion: The study provides support for the feasibility of a translation-based approach from 3rd to 2nd generation test cases. However, additional work is needed to make the approach applicable in real-world scenarios or larger open-source test suites. © 2020 IEEE.
  •  
40.
  • Fucci, Davide, 1985-, et al. (författare)
  • Evaluating software security maturity using OWASP SAMM : Different approaches and stakeholders perceptions
  • 2024
  • Ingår i: Journal of Systems and Software. - : Elsevier. - 0164-1212 .- 1873-1228. ; 214
  • Tidskriftsartikel (refereegranskat)abstract
    • Background: Recent years have seen a surge in cyber-attacks, which can be prevented or mitigated using software security activities. OWASP SAMM is a maturity model providing a versatile way for companies to assess their security posture and plan for improvements. Objective: We perform an initial SAMM assessment in collaboration with a company in the financial domain. Our objective is to assess a holistic inventory of the company security-related activities, focusing on how different roles perform the assessment and how they perceive the instrument used in the process. Methodology: We perform a case study to collect data using SAMM in a lightweight and novel manner through assessment using an online survey with 17 participants and a focus group with seven participants. Results: We show that different roles perceive maturity differently and that the two assessments deviate only for specific practices making the lightweight approach a viable and efficient solution in industrial practice. Our results indicate that the questions included in the SAMM assessment tool are answered easily and confidently across most roles. Discussion: Our results suggest that companies can productively use a lightweight SAMM assessment. We provide nine lessons learned for guiding industrial practitioners in the evaluation of their current security posture as well as for academics wanting to utilize SAMM as a research tool in industrial settings. Editor's note: Open Science material was validated by the Journal of Systems and Software Open Science Board. © 2024 The Author(s)
  •  
41.
  • Fucci, Davide, 1985-, et al. (författare)
  • When traceability goes awry : An industrial experience report?
  • 2022
  • Ingår i: Journal of Systems and Software. - : ELSEVIER SCIENCE INC. - 0164-1212 .- 1873-1228. ; 192
  • Tidskriftsartikel (refereegranskat)abstract
    • The concept of traceability between artifacts is considered an enabler for software project success. This concept has received plenty of attention from the research community and is by many perceived to always be available in an industrial setting. In this industry-academia collaborative project, a team of researchers, supported by testing practitioners from a large telecommunication company, sought to investigate the partner company's issues related to software quality. However, it was soon identified that the fundamental traceability links between requirements and test cases were missing. This lack of traceability impeded the implementation of a solution to help the company deal with its quality issues. In this experience report, we discuss lessons learned about the practical value of creating and maintaining traceability links in complex industrial settings and provide a cautionary tale for researchers. (c) 2022 The Author(s). Published by Elsevier Inc. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
  •  
42.
  • Karlsson, Stefan (författare)
  • Towards Augmented Exploratory Testing
  • 2021
  • Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
    • Software systems have an increasing presence in our society. With our infrastructure, such as food and water supply, controlled by complex software systems, it is essential to keep failures to a minimum. Exploratory testing has shown to be a good method of finding bugs that require more complex interactions, as well as to gain insights into how the system under test behaves. However, tool support in this area is lacking. Supporting exploratory testing with automation tools has the potential of freeing humans to spend their time where it is of the highest yield and also aid in covering the vast state space of testing a complex software system in a meaningful way. To be able to engineer such tools, targeting interaction models of contemporary systems, deeper knowledge is needed of the possibilities and limitations of applying available software testing methods. In this thesis we propose and evaluate several approaches that can be used to automatically perform black-box system-level testing, potentially augmenting humans in exploratory testing. We base these approaches on random-based methods, such as property-based testing, and present the results of evaluations on several industry-grade systems.Our results show that the proposed and evaluated methods automatically can find faults in real-world software systems. In addition to fault finding, our methods can also find insights such as deviations between the specification of the system under test and the actual behavior.The work in this thesis is a first step towards augmenting developers and testers in exploratory testing.
  •  
43.
  • Kitamura, Takashi, et al. (författare)
  • Industry-Academia Collaboration in Software Testing : An Overview of TAIC PART 2017
  • 2017
  • Ingår i: Proceedings - 10th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2017. - : Institute of Electrical and Electronics Engineers Inc.. - 9781509066766 ; , s. 42-43
  • Konferensbidrag (refereegranskat)abstract
    • Collaboration between industry and academia in software testing leads to improvement and innovation in industry, and it is the basis for achieving transferable and empirically evaluated results. Thus, the aim of TAIC PART is to forge collaboration between industry and academia on the challenging and exciting problem of real-world software testing. The workshop is promoted by representatives of both industry and academia, bringing together industrial software engineers and testers with researchers working on theory and practice of software testing. We present an overview of the 12th Workshop on Testing: Academia-Industry Collaboration, Practice and Research Techniques (TAIC PART 2017) and its contributions. © 2017 IEEE.
  •  
44.
  • Lenberg, Per, 1976, et al. (författare)
  • An initial analysis of differences in software engineers’ attitudes towards organizational change
  • 2016
  • Ingår i: Proceedings - 9th International Workshop on Cooperative and Human Aspects of Software Engineering, CHASE 2016. - New York, NY, USA : ACM Press. - 9781450341554 ; , s. 1-7
  • Konferensbidrag (refereegranskat)abstract
    • The ability to manage change is important in software engineering organizations, where rapid progress in technologies and constantly evolving methodologies create a turbulent environment. Research has identified employees’ attitudes towards organizational change as a key factor in the change process. Nonetheless, few studies exist that explore such attitudes in a software engineering context. The nature of change efforts is such that they often do not equally affect the various roles in the organization, which indicates that the roles may hold different attitudes. This study aimed to verify the existence of these presumed differences in attitudes towards organizational change between roles in a software engineering organization and to identify factors that contribute to these differences. The result of a survey (N=51) confirmed that there were significant differences, and that the software developers had a more positive attitude towards change and had deeper knowledge about the intended outcome compared to the line managers. The result of in-depth interviews (N=11) revealed that the software engineers evaluate the planned change in relation to the norms, values and standards of their peer group, meaning that an employee will have a positive attitude towards a change if its result is likely to make, or has made, it easier for him/her to uphold the peer group’s norms and values.
  •  
45.
  • Liebel, Grischa, 1987, et al. (författare)
  • State-of-practice in GUI-based system and acceptance testing: An industrial multiple-case study
  • 2013
  • Ingår i: Proceedings - 39th Euromicro Conference Series on Software Engineering and Advanced Applications, SEAA 2013. Santander; Spain; 4 September 2013 through 6 September 2013. - 9780769550916 ; , s. 17-24
  • Konferensbidrag (refereegranskat)abstract
    • Software testing is an essential means of evaluating software quality. System and acceptance tests aim to validate a system's conformance to its requirements on a high level of system abstraction. Therefore, they are generally performed by executing end-user scenarios through the system's graphical user interface (GUI). However, to the authors' best knowledge, there are no empirical studies that evaluate how GUI-based system and acceptance testing is performed in industrial practice. In this paper, we present a multiple-case study with the goal to investigate the state-of-practice of GUI-based system and acceptance testing at six software development companies of varying context. The main findings are that manual, GUI-based system testing is widespread and that automated GUI-based system and acceptance testing exists only on a small scale. Additionally, the study identifies core problems with GUI-based system and acceptance testing such as test tool limitations, high test costs and customer involvement in testing.
  •  
46.
  • Nass, Michel, et al. (författare)
  • Augmented testing : Industry feedback to shape a new testing technology
  • 2019
  • Ingår i: Proceedings - 2019 IEEE 12th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2019. - : Institute of Electrical and Electronics Engineers Inc.. - 9781728108889 ; , s. 176-183
  • Konferensbidrag (refereegranskat)abstract
    • Manual testing is the most commonly used approach in the industry today for acceptance-and system-testing of software applications. Test automation has been suggested to address drawbacks with manual testing but both test automation and manual testing have several challenges that limit their return of investment for system-and acceptance-test automation. Hence, there is still an industrial need for another approach to testing that can mitigate the challenges associated with system-and acceptance-testing and make it more efficient and cost effective for the industry. In this paper we present a novel technique we refer to as Augmented Testing (AT). AT is defined as testing through a visual layer between the tester and the System Under Test (SUT) that superimposes information on top of the GUI. We created a prototype for AT and performed an industrial workshop study with 10 software developers to get their perceived benefits and drawbacks of AT. The benefits and drawbacks will be useful for further development of the technique and prototype for AT. The workshop study identified more benefits than drawbacks with AT. Two of the identified benefits were: 'Know what to test and what has been tested' and 'Less manual work'. Due to these results, we believe that AT is a promising technique that deserves more research since it may provide industry with new benefits that current techniques lack. © 2019 IEEE.
  •  
47.
  • Nass, Michel, 1968-, et al. (författare)
  • Improving Web Element Localization by Using a Large Language Model
  • 2024
  • Ingår i: Software testing, verification & reliability. - : John Wiley & Sons. - 0960-0833 .- 1099-1689.
  • Tidskriftsartikel (refereegranskat)abstract
    • Web-based test automation heavily relies on accurately finding web elements. Traditional methods compare attributes but don't grasp the context and meaning of elements and words. The emergence of Large Language Models (LLMs) like GPT-4, which can show human-like reasoning abilities on some tasks, offers new opportunities for software engineering and web element localization. This paper introduces and evaluates VON Similo LLM, an enhanced web element localization approach. Using an LLM, it selects the most likely web element from the top-ranked ones identified by the existing VON Similo method, ideally aiming to get closer to human-like selection accuracy. An experimental study was conducted using 804 web element pairs from 48 real-world web applications. We measured the number of correctly identified elements as well as the execution times, comparing the effectiveness and efficiency of VON Similo LLM against the baseline algorithm. In addition, motivations from the LLM were recorded and analyzed for all instances where the original approach failed to find the right web element. VON Similo LLM demonstrated improved performance, reducing failed localizations from 70 to 39 (out of 804), a 44 percent reduction. Despite its slower execution time and additional costs of using the GPT-4 model, the LLMs human-like reasoning showed promise in enhancing web element localization. LLM technology can enhance web element identification in GUI test automation, reducing false positives and potentially lowering maintenance costs. However, further research is necessary to fully understand LLMs capabilities, limitations, and practical use in GUI testing.
  •  
48.
  • Nass, Michel, et al. (författare)
  • Improving Web Element Localization by Using a Large Language Model
  • 2024
  • Ingår i: Software Testing Verification and Reliability. - 0960-0833 .- 1099-1689. ; In Press
  • Tidskriftsartikel (refereegranskat)abstract
    • Web-based test automation heavily relies on accurately finding web elements. Traditional methods compare attributes but do not grasp the context and meaning of elements and words. The emergence of large language models (LLMs) like GPT-4, which can show human-like reasoning abilities on some tasks, offers new opportunities for software engineering and web element localization. This paper introduces and evaluates VON Similo LLM, an enhanced web element localization approach. Using an LLM, it selects the most likely web element from the top-ranked ones identified by the existing VON Similo method, ideally aiming to get closer to human-like selection accuracy. An experimental study was conducted using 804 web element pairs from 48 real-world web applications. We measured the number of correctly identified elements as well as the execution times, comparing the effectiveness and efficiency of VON Similo LLM against the baseline algorithm. In addition, motivations from the LLM were recorded and analysed for 140 instances. VON Similo LLM demonstrated improved performance, reducing failed localizations from 70 to 40 (out of 804), a 43% reduction. Despite its slower execution time and additional costs of using the GPT-4 model, the LLM's human-like reasoning showed promise in enhancing web element localization. LLM technology can enhance web element localization in GUI test automation, reducing false positives and potentially lowering maintenance costs. However, further research is necessary to fully understand LLMs' capabilities, limitations and practical use in GUI testing.
  •  
49.
  • Nass, Michel, 1968- (författare)
  • On overcoming challenges with GUI-based test automation
  • 2024
  • Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
    • Background: Automated testing is widely used in modern software development to check if the software, including its graphical user interface (GUI), meets the expectations in terms of quality and functionality. GUI-based test automation, like other automation, aims to save time and money compared to manual testing without reducing the software quality. While automation has successfully reduced costs for other types of testing (e.g., unit-or integration tests), GUI-based testing has faced technical challenges, some of which have lingered for over a decade. Objective: This thesis work aims to contribute to the software engineering body of knowledge by (1) identifying the main challenges in GUI-based test automation and (2) finding technical solutions to mitigate some of the main challenges. One such challenge is to reliably identify GUI elements during test execution to prevent unnecessary repairs. Another problem is the demand for test automation and programming skills when designing stable automated tests at scale. Method: We conducted several studies by adopting a multi-methodological approach. First, we performed a systematic literature review to identify the main challenges in GUI-based test automation, followed by multiple studies that propose and evaluate novel approaches to mitigate the main challenges. Results: Our first contribution is mapping the challenges in GUI-based test automation reported in academic literature. We mapped the main challenges (i.e. most reported) on a timeline and classified them as essential or accidental. This classification is valuable since future research can focus on the main challenges that we are more likely to mitigate using a technical solution (i.e., accidental). Our second contribution is several approaches that explore novel concepts or advance state-of-the-art techniques to mitigate some of the main accidental challenges. Testing an application through an augmented layer (Augmented Testing) can reduce the demand for test automation and programming skills and mitigate the challenges of creating and maintaining model based tests. Our proposed approach for locating web elements (Similo) can increase the robustness of automated test execution. Conclusion: Our results provide alternative approaches and concepts that can mitigate some of the main accidental challenges in GUI-based test automation. With a more robust test execution and tool support for test modeling, we can help reduce the manual labor spent on creating and maintaining automated GUI-based tests. With a reduced cost of automation, testers can focus more on other tasks like requirements, test design, and exploratory testing.
  •  
50.
  • Nass, Michel, 1968-, et al. (författare)
  • On the Industrial Applicability of Augmented Testing : An Empirical Study
  • 2020
  • Ingår i: Proceedings - 2020 IEEE 13th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2020. - : Institute of Electrical and Electronics Engineers Inc.. - 2159-4848. - 9781728110752 ; , s. 364-371
  • Konferensbidrag (refereegranskat)abstract
    • Testing applications with graphical user Interfaces (GUI) is an important but also a time-consuming task in practice. Tools and frameworks for GUI test automation can make the test execution more efficient and lower the manual labor required for regression testing. However, the test scripts used for automated GUI-based testing still require a substantial development effort and are often reported as sensitive to change, leading to frequent and costly maintenance. The efficiency of development, maintenance, and evolution of such tests are thereby dependent on the readability of scripts and the ease-of-use of test tools/frameworks in which the test scripts are defined. To address these shortcomings in existing state-of-practice techniques, a novel technique referred to as Augmented Testing (AT) has been proposed. AT is defined as testing the System Under Test (SUT) through an Augmented GUI that superimposes information on top of the SUT GUI. The Augmented GUI can provide the user with hints, test data, or other support while also observing and recording the tester's interactions. For this study, a prototype tool, called Scout, has been used that adheres to the AT concept that is evaluated in an industrial empirical study. In the evaluation, quasi-experiments and questionnaire surveys are performed in two workshops, with 12 practitioners from two Swedish companies (Ericsson and Inceptive). Results show that Scout can be used to create equivalent test cases faster, with statistical significance, than creating automated scripts in two popular state-of-practice tools. The study concludes that AT has cost-value benefits, applies to industrial-grade software, and overcomes several deficiencies of state-of-practice GUI testing technologies in terms of ease-of-use. © 2020 IEEE.
  •  
Skapa referenser, mejla, bekava och länka
  • Resultat 1-50 av 66
Typ av publikation
konferensbidrag (27)
tidskriftsartikel (25)
licentiatavhandling (4)
annan publikation (3)
rapport (2)
doktorsavhandling (2)
visa fler...
forskningsöversikt (2)
bokkapitel (1)
visa färre...
Typ av innehåll
refereegranskat (51)
övrigt vetenskapligt/konstnärligt (13)
populärvet., debatt m.m. (1)
Författare/redaktör
Alégroth, Emil, 1984 ... (46)
Alégroth, Emil (18)
Gorschek, Tony, 1972 ... (13)
Feldt, Robert, 1972 (10)
Coppola, Riccardo (7)
Feldt, Robert (6)
visa fler...
Wnuk, Krzysztof, 198 ... (5)
Ardito, Luca (5)
Axelsson, Jakob (3)
Petersen, Kai (3)
Nass, Michel (3)
Hammouda, Imed (2)
Gao, Z. (2)
Fucci, Davide, 1985- (2)
Felderer, Michael, 1 ... (2)
Šmite, Darja (2)
Steghöfer, Jan-Phili ... (2)
Mattsson, Michael (2)
Oliveira, R. (2)
Memon, A. (2)
Olsson Holmström, He ... (2)
Burden, Håkan, 1976 (2)
Olsson, Thomas (1)
Causevic, Adnan, 198 ... (1)
Unterkalmsteiner, Mi ... (1)
Steghöfer, Jan-Phili ... (1)
Mendez, Daniel (1)
Burden, Håkan (1)
Ciccozzi, Federico, ... (1)
Cicchetti, Antonio (1)
Nygren, Åse (1)
Wohlin, Claes (1)
Steiner, M (1)
Olsson, Nils (1)
Martini, Antonio, 19 ... (1)
Borch Petersen, Elin ... (1)
Tinnerholm, John (1)
Karlsson, Arvid (1)
Radway, Alexander (1)
Kolstrom, P. (1)
Bache, G. (1)
Bache, E. (1)
Matsuki, Shinsuke (1)
Vos, Tanja (1)
Akemine, Kinji (1)
Karl, Kristian (1)
Rosshagen, Helena (1)
Helmfridsson, Tomas (1)
Gustafsson, Johan (1)
Ivarsson, Henrik (1)
visa färre...
Lärosäte
Blekinge Tekniska Högskola (49)
Chalmers tekniska högskola (25)
RISE (8)
Örebro universitet (7)
Göteborgs universitet (3)
Mälardalens universitet (2)
visa fler...
Malmö universitet (2)
Linköpings universitet (1)
Lunds universitet (1)
Linnéuniversitetet (1)
visa färre...
Språk
Engelska (66)
Forskningsämne (UKÄ/SCB)
Naturvetenskap (62)
Teknik (8)
Samhällsvetenskap (4)

År

Kungliga biblioteket hanterar dina personuppgifter i enlighet med EU:s dataskyddsförordning (2018), GDPR. Läs mer om hur det funkar här.
Så här hanterar KB dina uppgifter vid användning av denna tjänst.

 
pil uppåt Stäng

Kopiera och spara länken för att återkomma till aktuell vy