| 1. |
- Borg, Markus, et al.
(författare)
-
Selecting Software Component Sourcing Options : Detailed Survey Description and Analysis
- 2018
-
Rapport (övrigt vetenskapligt/konstnärligt)abstract
- Component-based software engineering (CBSE) is a common approach to develop and evolve contemporary software systems. When evolving a system based on components, make-or-buy decisions are frequent, i.e., whether to develop components internally or to acquire them fromexternal sources. In CBSE, several different sourcing options are available: 1) developing software in-house, 2) outsourcing development, 3) buying commercial-off-the-shelf software, and 4) integrating open source software components. Unfortunately, there is little available research on howorganizations select component sourcing options (CSO) in industry practice. In this work, we seek to contribute empirical evidence to CSO selection. Method: We conduct a cross-domain survey on CSO selection in industry, implemented as an online questionnaire. Based on 188 responses, we find that most organizations consider multiple CSOs during software evolution, and that the CSO decisions in industry are dominated by expert judgment. When choosing between candidate components, functional suitability acts as an initial filter, then reliability is the most important quality. We stress that future solution-oriented work on decision support has to account for the dominance of expert judgment in industry. Moreover, we identify considerable variation in CSO decision processes in industry. Finally, we encourage software development organizations to reflect on their decision processes when choosing whether to make or buy components, and we recommend using our survey for a first benchmarking.
|
|
| 2. |
- Lind, Emil, 1989-, et al.
(författare)
-
Requirements Quality vs. Process and Stakeholders’ Well-Being : A Case of a Nordic Bank
- 2023
-
Ingår i: Software Quality. - : Springer Science+Business Media B.V.. - 9783031314872 ; , s. 17-37
-
Konferensbidrag (refereegranskat)abstract
- Requirements are key artefacts to describe the intended purpose of a software system. The quality of requirements is crucial for deciding what to do next, impacting the development process’ effectiveness and efficiency. However, we know very little about the connection between practitioners’ perceptions regarding requirements quality and its impact on the process or the feelings of the professionals involved in the development process. Objectives: This study investigates: i) How software development practitioners define requirements quality, ii) how the perceived quality of requirements impact process and stakeholders’ well-being, and iii) what are the causes and potential solutions for poor-quality requirements. Method: This study was performed as a descriptive interview study at a sub-organization of a Nordic bank that develops its own web and mobile apps. The data collection comprises interviews with 20 practitioners, including requirements engineers, developers, testers, and newly employed developers, with five interviewees from each group. Results: The results show that different roles have different views on what makes a requirement good quality. Participants highlighted that, in general, they experience negative emotions, more work, and overhead communication when they work with requirements they perceive to be of poor quality. The practitioners also describe positive effects on their performance and positive feelings when they work with requirements that they perceive to be good. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.
|
|
| 3. |
- Wohlin, Claes, et al.
(författare)
-
Towards evidence-based decision-making for identification and usage of assets in composite software : A research roadmap
- 2021
-
Ingår i: Journal of Software. - : John Wiley and Sons Ltd. - 2047-7473 .- 2047-7481. ; 33:6
-
Tidskriftsartikel (refereegranskat)abstract
- Software engineering is decision intensive. Evidence-based software engineering is suggested for decision-making concerning the use of methods and technologies when developing software. Software development often includes the reuse of software assets, for example, open-source components. Which components to use have implications on the quality of the software (e.g., maintainability). Thus, research is needed to support decision-making for composite software. This paper presents a roadmap for research required to support evidence-based decision-making for choosing and integrating assets in composite software systems. The roadmap is developed as an output from a 5-year project in the area, including researchers from three different organizations. The roadmap is developed in an iterative process and is based on (1) systematic literature reviews of the area; (2) investigations of the state of practice, including a case survey and a survey; and (3) development and evaluation of solutions for asset identification and selection. The research activities resulted in identifying 11 areas in need of research. The areas are grouped into two categories: areas enabling evidence-based decision-making and those related to supporting the decision-making. The roadmap outlines research needs in these 11 areas. The research challenges and research directions presented in this roadmap are key areas for further research to support evidence-based decision-making for composite software. © 2021 The Authors. Journal of Software: Evolution and Process published by John Wiley & Sons Ltd.
|
|
| 4. |
- Yu, Liang, 1986-, et al.
(författare)
-
Automated NFR testing in Continuous Integration Environments : a multi-case study of Nordic companies
-
Annan publikation (övrigt vetenskapligt/konstnärligt)abstract
- Context: Non-functional requirements (NFRs) (also referred to as system qualities) are essential for developing high-quality software.Notwithstanding its importance, NFR testing remains challenging, especially in terms of automation.Compared to manual verification, automated testing shows the potential to improve the efficiency and effectiveness of quality assurance, especially in the context of Continuous Integration (CI).However, studies on how companies manage automated NFR testing through CI are limited.Objective: This study examines how automated NFR testing can be enabledand supported using CI environments in software development companies.Method: We performed a multi-case study at four companies by conducting 22 semi-structured interviews with industrial practitioners.Results: Maintainability, reliability, performance, security and scalability, were found to be evaluated with automated tests in CI environments.Testing practices, quality metrics, and challenges for measuring NFRs were reported.Conclusions: This study presents an empirically derived model that shows how data produced by CI environments can be used for evaluation and monitoring of implemented NFR quality. Additionally, the manuscript presents explicit metrics, CI components, tools, and challenges that shall be considered while performing NFR testing in practice.
|
|
| 5. |
- Yu, Liang, 1986-, et al.
(författare)
-
Automated NFR testing in continuous integration environments : a multi-case study of Nordic companies
- 2023
-
Ingår i: Empirical Software Engineering. - : Springer. - 1382-3256 .- 1573-7616. ; 28:6
-
Tidskriftsartikel (refereegranskat)abstract
- Context: Non-functional requirements (NFRs) (also referred to as system qualities) are essential for developing high-quality software. Notwithstanding its importance, NFR testing remains challenging, especially in terms of automation. Compared to manual verification, automated testing shows the potential to improve the efficiency and effectiveness of quality assurance, especially in the context of Continuous Integration (CI). However, studies on how companies manage automated NFR testing through CI are limited. Objective: This study examines how automated NFR testing can be enabled and supported using CI environments in software development companies. Method: We performed a multi-case study at four companies by conducting 22 semi-structured interviews with industrial practitioners. Results: Maintainability, reliability, performance, security and scalability, were found to be evaluated with automated tests in CI environments. Testing practices, quality metrics, and challenges for measuring NFRs were reported. Conclusions: This study presents an empirically derived model that shows how data produced by CI environments can be used for evaluation and monitoring of implemented NFR quality. Additionally, the manuscript presents explicit metrics, CI components, tools, and challenges that shall be considered while performing NFR testing in practice. © 2023, The Author(s).
|
|
| 6. |
- Yu, Liang, 1986-, et al.
(författare)
-
Utilizing Continuous Integration environments for evaluation of software quality attributes : a practical road map
-
Annan publikation (övrigt vetenskapligt/konstnärligt)abstract
- The evaluation of quality attributes is often conducted in an ad-hoc manner in many companies.This practice, although often necessary to reduce cost, can lead to inefficient work and undetected quality defects.To mitigate these issues, approaches that make use of metrics/data from Continuous Integration (CI) environments have been proposed, offering many advantages, including timely feedback on code quality, early detection of quality issues, and visual representation of system quality trends.Despite these known benefits, there is a lack of frameworks that provide guidelines how to utilize CI environments, or their components, for quality attribute evaluation.Such guidelines are important since market demands on software quality factors, e.g., security or performance, are growing.Therefore, this study aims to explore the state-of-the-practice in using CI environments and identify shared guidelines underlying industrial practices.To achieve our research objectives, we conducted a multi-case study involving four selected companies and 22 interviews with industrial participants.In this paper, we first present a general model to offer an overview of how CI environments contribute to quality attribute evaluation.We then introduce common quality metrics used in the studied companies and from which CI components these metrics can be acquired.Finally, based on the study's findings, we propose a hierarchical and multi-layered decision support model for improving a CI environment's quality evaluation capabilities over time.
|
|
| 7. |
- Alégroth, Emil, 1984-, et al.
(författare)
-
A Failed attempt at creating Guidelines for Visual GUI Testing : An industrial case study
- 2021
-
Ingår i: Proceedings - 2021 IEEE 14th International Conference on Software Testing, Verification and Validation, ICST 2021. - : Institute of Electrical and Electronics Engineers Inc.. - 9781728168364 ; , s. 340-350
-
Konferensbidrag (refereegranskat)abstract
- Software development is governed by guidelines that aim to improve the code's qualities, such as maintainability. However, whilst coding guidelines are commonplace for software, guidelines for testware are much less common. In particular, for GUI-based tests driven with image recognition, also referred to as Visual GUI Testing (VGT), explicit coding guidelines are missing.In this industrial case study, performed at the Swedish defence contractor Saab AB, we propose a set of coding guidelines for VGT and evaluate their impact on test scripts for an industrial, safety-critical system. To study the guidelines' effect on maintenance costs, five representative manual test cases are each translated with and without the proposed guidelines in the two VGT tools SikuliX and EyeAutomate. As such, 20 test scripts were developed, with a combined development cost of more than 100 man-hours. Three of the tests are then maintained by one researcher and two practitioners for another version of the system and costs measured to evaluate return on investment. This analysis is complemented with observations and interviews to elicit practitioners' perceptions and experiences with VGT.Results show that scripts developed with the guidelines had higher maintenance costs than scripts developed without guidelines. This is supported by qualitative results that many of the guidelines are considered inappropriate, superfluous or unnecessary due to the inherent properties of the scripts, e.g. their natural small size, linear flows, natural separation of concerns, and more. We conclude that there are differences between VGT scripts and software that prohibit direct translation of guidelines between the two. As such, we consider our study as a failure but argue that several lessons can be drawn from our results to guide future research into guidelines for VGT and GUI-based test automation. © 2021 IEEE.
|
|
| 8. |
- Alégroth, Emil, 1984-, et al.
(författare)
-
Characteristics that affect Preference of Decision Models for Asset Selection : An Industrial Questionnaire Survey
- 2020
-
Ingår i: Software quality journal. - : Springer. - 0963-9314 .- 1573-1367. ; 28:4, s. 1675-1707
-
Tidskriftsartikel (refereegranskat)abstract
- Modern software development relies on a combination of development and re-use of technical asset, e.g. software components, libraries and APIs.In the past, re-use was mostly conducted with internal assets but today external; open source, customer off-the-shelf (COTS) and assets developed through outsourcing are also common.This access to more asset alternatives presents new challenges regarding what assets to optimally chose and how to make this decision.To support decision-makers, decision-theory has been used to develop decision models for asset selection.However, very little industrial data has been presented in literature about the usefulness, or even perceived usefulness, of these models.Additionally, only limited information has been presented about what model characteristics that determine practitioner preference towards one model over another.Objective: The objective of this work is to evaluate what characteristics of decision models for asset selection that determine industrial practitioner preference of a model when given the choice of a decision-model of high precision or a model with high speed.Method: An industrial questionnaire survey is performed where a total of 33 practitioners, of varying roles, from 18 companies are tasked to compare two decision models for asset selection.Textual analysis and formal and descriptive statistics are then applied on the survey responses to answer the study's research questions.Results: The study shows that the practitioners had clear preference towards the decision model that emphasised speed over the one that emphasised decision precision.This conclusion was determined to be because one of the models was perceived faster, had lower complexity, had, was more flexible in use for different decisions, was more agile how it could be used in operation, its emphasis on people, its emphasis on ``good enough'' precision and ability to fail fast if a decision was a failure.Hence, seven characteristics that the practitioners considered important for their acceptance of the model.Conclusion: Industrial practitioner preference, which relates to acceptance, of decision models for asset selection is dependent on multiple characteristics that must be considered when developing a model for different types of decisions such as operational day-to-day decisions as well as more critical tactical or strategic decisions.The main contribution of this work are seven identified characteristics that can serve as industrial requirements for future research on decision models for asset selection.
|
|
| 9. |
- Alégroth, Emil, 1984, et al.
(författare)
-
Conceptualization and evaluation of component-based testing unified with visual GUI testing: An empirical study
- 2015
-
Ingår i: 2015 IEEE 8th International Conference on Software Testing, Verification and Validation, ICST 2015 - Proceedings. - 2159-4848. - 9781479971251
-
Konferensbidrag (refereegranskat)abstract
- In this paper we present the results of a two-phase empirical study where we evaluate and compare the applicability of automated component-based Graphical User Interface (GUI) testing and Visual GUI Testing (VGT) in the tools GUITAR and a prototype tool we refer to as VGT GUITAR. First, GUI mutation operators are defined to create 18 faulty versions of an application on which both tools are then applied in an experiment. Results from 456 test case executions in each tool show, with statistical significance, that the component-based approach reports more false negatives than VGT for acceptance tests but that the VGT approach reports more false positives for system tests. Second, a case study is performed with larger open source applications, ranging from 8,803-55,006 lines of code. Results show that GUITAR is applicable in practice but has some challenges related to GUI component states. The results also show that VGT GUITAR is currently not applicable in practice and therefore requires further research and development. Based on the study's results we present areas of future work for both test approaches and conclude that the approaches have different benefits and drawbacks. The component-based approach is robust and executes tests faster than the VGT approach, with a factor of 3. However, the VGT approach can perform visual assertions and is perceived more flexible than the component- based approach. These conclusions let us hypothesize that a combination of the two approaches is the most suitable in practice and therefore warrants future research.
|
|
| 10. |
- Alégroth, Emil, 1984, et al.
(författare)
-
Exploring the Presence of Technical Debt in Industrial GUI-based Testware: A Case Study
- 2016
-
Ingår i: 2016 IEEE Ninth International Conference on Software Testing, Verification and Validation Workshops. - : IEEE. - 2159-4848. - 9781509036745 ; , s. 257-262
-
Konferensbidrag (refereegranskat)abstract
- Technical debt (TD) is a concept used to describe a sub-optimal solution of a software artifact that negatively affects its comprehensibility, extendability and maintainability. As such, TD adversely affects the costs or quality associated with the artifact, which is also called interest. TD has through research been identified in all types of software artifacts, from architectural design to automated tests (Testware). However, research into testware technical debt (TTD) is limited and primarily focused on testing on lower level of system abstraction, i.e. unit-and integration tests, leaving a need for more TTD research on GUI-based testing. In this study we explore this gap in knowledge through an industrial case study at a Swedish avionics software development company. Four repositories are studied for the presence of TTD using expert interviews, semi-automated document analysis and automatic metric analysis. Results of the study provide initial support that the concept of TTD is applicable to GUI-based testware and show the presence of both TD items unique to GUI-based testware and items common to software. The implications of these results are that engineering best practices must be established for GUI-based testware to minimize TD interest.
|
|
