| 1. |
- Ali, Nauman bin, et al.
(author)
-
On the search for industry-relevant regression testing research
- 2019
-
In: Empirical Software Engineering. - New York, NY : Springer. - 1382-3256 .- 1573-7616. ; 24:4, s. 2020-2055
-
Journal article (peer-reviewed)abstract
- Regression testing is a means to assure that a change in the software, or its execution environment, does not introduce new defects. It involves the expensive undertaking of rerunning test cases. Several techniques have been proposed to reduce the number of test cases to execute in regression testing, however, there is no research on how to assess industrial relevance and applicability of such techniques. We conducted a systematic literature review with the following two goals: firstly, to enable researchers to design and present regression testing research with a focus on industrial relevance and applicability and secondly, to facilitate the industrial adoption of such research by addressing the attributes of concern from the practitioners' perspective. Using a reference-based search approach, we identified 1068 papers on regression testing. We then reduced the scope to only include papers with explicit discussions about relevance and applicability (i.e. mainly studies involving industrial stakeholders). Uniquely in this literature review, practitioners were consulted at several steps to increase the likelihood of achieving our aim of identifying factors important for relevance and applicability. We have summarised the results of these consultations and an analysis of the literature in three taxonomies, which capture aspects of industrial-relevance regarding the regression testing techniques. Based on these taxonomies, we mapped 38 papers reporting the evaluation of 26 regression testing techniques in industrial settings. © The Author(s) 2019
|
|
| 2. |
- Bjarnason, Elizabeth, et al.
(author)
-
A Multi-Case Study of Agile Requirements Engineering and the Use of Test Cases as Requirements
- 2016. - 16
-
In: Information and Software Technology. - : Elsevier. - 0950-5849 .- 1873-6025. ; 77, s. 61-79
-
Journal article (peer-reviewed)abstract
- [Context] It is an enigma that agile projects can succeed "without requirements" when weak requirements engineering is a known cause for project failures. While agile development projects often manage well without extensive requirements test cases are commonly viewed as requirements and detailed requirements are documented as test cases. [Objective] We have investigated this agile practice of using test cases as requirements to understand how test cases can support the main requirements activities, and how this practice varies. [Method] We performed an iterative case study at three companies and collected data through 14 interviews and 2 focus groups. [Results] The use of test cases as requirements poses both benefits and challenges when eliciting, validating, verifying, and managing requirements, and when used as a documented agreement. We have identified five variants of the test-cases-as-requirements practice, namely de facto, behaviour-driven, story-test driven, stand-alone strict and stand-alone manual for which the application of the practice varies concerning the time frame of requirements documentation, the requirements format, the extent to which the test cases are a machine executable specification and the use of tools which provide specific support for the practice of using test cases as requirements. [Conclusions] The findings provide empirical insight into how agile development projects manage and communicate requirements. The identified variants of the practice of using test cases as requirements can be used to perform in-depth investigations into agile requirements engineering. Practitioners can use the provided recommendations as a guide in designing and improving their agile requirements practices based on project characteristics such as number of stakeholders and rate of change.
|
|
| 3. |
- Bjarnason, Elizabeth, et al.
(author)
-
A Theory of Distances in Software Engineering
- 2016
-
In: Information and Software Technology. - : Elsevier BV. - 0950-5849. ; 70, s. 204-219
-
Journal article (peer-reviewed)abstract
- [Context] Coordinating a software project across distances is challenging. Even without geographical and time zone distances, other distances within a project can cause communication gaps. For example, organisational and cognitive distances between product owners and development-near roles such as developers and testers can lead to differences in understanding and interpretation of the business requirements. Applying good software development practices, known to enhance alignment and coordination within development projects, can alleviate these challenges. [Objective] The aim of our research is to identify and describe underlying factors which can explain why certain practices support aligning and coordinating software development projects. [Method] We have inductively generated a theory analysing empirical data consisting of 15 interviews from 5 different companies. The systematic and iterative analysis was based on an initial hypothesis that distances affect development, and on results from previous research. [Results] We present a theory of distances that explains how practices improve the communication within a project by impacting distances between people, activities and artefacts. We also present a theoretical model of how specific alignment practices affect different types of distances. [Conclusions] The results provide a basis for further research and can be used by software organisations to improve on software practice.
|
|
| 4. |
- Bjarnason, Elizabeth, et al.
(author)
-
Alignment Practices Affect Distances in Software Development: A Theory and a Model
- 2014
-
In: GTSE 2014 Proceedings of the 3rd SEMAT Workshop on General Theories of Software Engineering. - New York, NY, USA : ACM. - 9781450328500 ; , s. 21-31
-
Conference paper (peer-reviewed)abstract
- Coordinating a software project across distances is challenging. Even without geographical and time zone distances, other distances within a project can cause communication gaps. For example, organisational and cognitive distances between product owners and development-near roles such as developers and testers can lead to weak alignment of the software and the business requirements. Applying good software development practices, known to enhance alignment, can alleviate these challenges. We present a theoretical model called the Gap Model of how alignment practices affect different types of distances. This model has been inductively generated from empirical data. We also present an initial version of a theory based on this model that explains, at a general level, how practices affect communication within a project by impacting distances between people, activities and artefacts. The presented results provide a basis for further research and can be used by software organisations to improve on software practice.
|
|
| 5. |
- Bjarnason, Elizabeth, et al.
(author)
-
An Industrial Case Study on Test Cases as Requirements
- 2015
-
In: Agile Processes in Software Engineering and Extreme Programming : 16th International Conference, XP 2015, Helsinki, Finland, May 25-29, 2015, Proceedings - 16th International Conference, XP 2015, Helsinki, Finland, May 25-29, 2015, Proceedings. - Cham : Springer International Publishing. - 1865-1356 .- 1865-1348. - 9783319186115 - 9783319186122 ; 212
-
Conference paper (peer-reviewed)abstract
- It is a conundrum that agile projects can succeed ‘without requirements’ when weak requirements engineering is a known cause for project failures. While Agile development projects often manage well without extensive requirements documentation, test cases are commonly used as requirements. We have investigated this agile practice at three companies in order to understand how test cases can fill the role of requirements. We performed a case study based on twelve interviews performed in a previous study. The findings include a range of benefits and challenges in using test cases for eliciting, validating, verifying, tracing and managing requirements. In addition, we identified three scenarios for applying the practice, namely as a mature practice, as a de facto practice and as part of an agile transition. The findings provide insights into how the role of requirements may be met in agile development including challenges to consider.
|
|
| 6. |
- Bjarnason, Elizabeth, et al.
(author)
-
An Industrial Case Study on the Use of Test Cases as Requirements
- 2015
-
In: Lecture Notes in Business Information<em></em>. - : Springer. ; , s. 27-39
-
Conference paper (peer-reviewed)abstract
- It is a conundrum that agile projects can succeed 'without requirements' when weak requirements engineering is a known cause for project failures. While Agile development projects often manage well without extensive requirements documentation, test cases are commonly used as requirements. We have investigated this agile practice at three companies in order to understandhow test cases can fill the role of requirements. We performed a case study based on twelve interviews performed in a previous study.The findings include a range of benefits and challenges in using test cases for eliciting, validating, verifying, tracing and managing requirements. In addition, we identified three scenarios for applying the practice, namely as a mature practice, as a de facto practice and as part of an agile transition. The findings provide insights into how the role of requirements may be met in agile development including challenges to consider.
|
|
| 7. |
- Bjarnason, Elizabeth, et al.
(author)
-
Challenges and practices in aligning requirements with verification and validation : a case study of six companies
- 2014
-
In: Empirical Software Engineering. - : Springer. - 1382-3256 .- 1573-7616. ; 19:6, s. 1809-1855
-
Journal article (peer-reviewed)abstract
- Weak alignment of requirements engineering (RE) with verification and validation (VV) may lead to problems in delivering the required products in time with the right quality. For example, weak communication of requirements changes to testers may result in lack of verification of new requirements and incorrect verification of old invalid requirements, leading to software quality problems, wasted effort and delays. However, despite the serious implications of weak alignment research and practice both tend to focus on one or the other of RE or VV rather than on the alignment of the two. We have performed a multi-unit case study to gain insight into issues around aligning RE and VV by interviewing 30 practitioners from 6 software developing companies, involving 10 researchers in a flexible research process for case studies. The results describe current industry challenges and practices in aligning RE with VV, ranging from quality of the individual RE and VV activities, through tracing and tools, to change control and sharing a common understanding at strategy, goal and design level. The study identified that human aspects are central, i.e. cooperation and communication, and that requirements engineering practices are a critical basis for alignment. Further, the size of an organisation and its motivation for applying alignment practices, e.g. external enforcement of traceability, are variation factors that play a key role in achieving alignment. Our results provide a strategic roadmap for practitioners improvement work to address alignment challenges. Furthermore, the study provides a foundation for continued research to improve the alignment of RE with VV.
|
|
| 8. |
|
|
| 9. |
- Borg, Markus, et al.
(author)
-
Adopting automated bug assignment in practice — a longitudinal case study at Ericsson
- 2024
-
In: Empirical Software Engineering. - : Springer. - 1382-3256 .- 1573-7616. ; 29:5
-
Journal article (peer-reviewed)abstract
- [Context] The continuous inflow of bug reports is a considerable challenge in large development projects. Inspired by contemporary work on mining software repositories, we designed a prototype bug assignment solution based on machine learning in 2011-2016. The prototype evolved into an internal Ericsson product, TRR, in 2017-2018. TRR’s first bug assignment without human intervention happened in April 2019. [Objective] Our study evaluates the adoption of TRR within its industrial context at Ericsson, i.e., we provide lessons learned related to the productization of a research prototype within a company. Moreover, we investigate 1) how TRR performs in the field, 2) what value TRR provides to Ericsson, and 3) how TRR has influenced the ways of working. [Method] We conduct a preregistered industrial case study combining interviews with TRR stakeholders, minutes from sprint planning meetings, and bug-tracking data. The data analysis includes thematic analysis, descriptive statistics, and Bayesian causal analysis. [Results] TRR is now an incorporated part of the bug assignment process. Considering the abstraction levels of the telecommunications stack, high-level modules are more positive while low-level modules experienced some drawbacks. Most importantly, some bug reports directly reach low-level modules without first having passed through fundamental root-cause analysis steps at higher levels. On average, TRR automatically assigns 30% of the incoming bug reports with an accuracy of 75%. Auto-routed TRs are resolved around 21% faster within Ericsson, and TRR has saved highly seasoned engineers many hours of work. Indirect effects of adopting TRR include process improvements, process awareness, increased communication, and higher job satisfaction. [Conclusions] TRR has saved time at Ericsson, but the adoption of automated bug assignment was more intricate compared to similar endeavors reported from other companies. We primarily attribute the difference to the very large size of the organization and the complex products. Key facilitators in the successful adoption include a gradual introduction, product champions, and careful stakeholder analysis.
|
|
| 10. |
- Börstler, Jürgen, 1960-, et al.
(author)
-
Acceptance behavior theories and models in software engineering — A mapping study
- 2024
-
In: Information and Software Technology. - : Elsevier. - 0950-5849 .- 1873-6025. ; 172
-
Journal article (peer-reviewed)abstract
- Context: The adoption or acceptance of new technologies or ways of working in software development activities is a recurrent topic in the software engineering literature. The topic has, therefore, been empirically investigated extensively. It is, however, unclear which theoretical frames of reference are used in this research to explain acceptance behaviors. Objective: In this study, we explore how major theories and models of acceptance behavior have been used in the software engineering literature to empirically investigate acceptance behavior.Method: We conduct a systematic mapping study of empirical studies using acceptance behavior theories in software engineering.Results: We identified 47 primary studies covering 56 theory uses. The theories were categorized into six groups. Technology acceptance models (TAM and its extensions) were used in 29 of the 47 primary studies, innovation theories in 10, and the theories of planned behavior/ reasoned action (TPB/TRA) in six. All other theories were used in at most two of the primary studies. The usage and operationalization of the theories were, in many cases, inconsistent with the underlying theories. Furthermore, we identified 77 constructs used by these studies of which many lack clear definitions. Conclusions: Our results show that software engineering researchers are aware of some of the leading theories and models of acceptance behavior, which indicates an attempt to have more theoretical foundations. However, we identified issues related to theory usage that make it difficult to aggregate and synthesize results across studies. We propose mitigation actions that encourage the consistent use of theories and emphasize the measurement of key constructs.
|
|
| 11. |
- da Mota Silveira Neto, Paulo Anselmo, et al.
(author)
-
Testing software product lines
- 2011
-
In: IEEE Software. - 0740-7459. ; 28:5, s. 16-20
-
Journal article (other academic/artistic)abstract
- Two studies of testing practices for software product lines identify gaps between required techniques and existing approaches in the available literature. This Web extra offers extra details for the main article (specifically, the bibliography for the two studies described).
|
|
| 12. |
- Ekelund, Edward Dunn, et al.
(author)
-
Efficient Regression Testing Based on Test History: An Industrial Evaluation
- 2015
-
In: 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME). - 9781467375320
-
Conference paper (peer-reviewed)abstract
- Due to changes in the development practices at Axis Communications, towards continuous integration, faster regression testing feedback is needed. The current automated regression test suite takes approximately seven hours to run which prevents developers from integrating code changes several times a day as preferred. Therefore we want to implement a highly selective yet accurate regression testing strategy. Traditional code coverage based techniques are not applicable due to the size and complexity of the software under test. Instead we decided to select tests based on regression test history. We developed a tool, the Difference Engine, which parses and analyzes results from previous test runs and outputs regression test recommendations. The Difference Engine correlates code and test cases at package level and recommends test cases that are strongly correlated to recently changed packages. We evaluated the technique with respect to correctness, precision, recall and efficiency. Our results are promising. On average the tool manages to identify 80% of the relevant tests while recommending only 4% of the test cases in the full regression test suite.
|
|
| 13. |
- Engström, Emelie, et al.
(author)
-
A qualitative survey of regression testing practices
- 2010
-
In: Product-Focused Software Process Improvement/Lecture Notes in Computer Science. - Berlin, Heidelberg : Springer Berlin Heidelberg. - 0302-9743 .- 1611-3349. - 9783642137914 ; 6156, s. 3-16
-
Conference paper (peer-reviewed)abstract
- Aim: Regression testing practices in industry have to be better understood, both for the industry itself and for the research community. Method: We conducted a qualitative industry survey by i) running a focus group meeting with 15 industry participants and ii) validating the outcome in an on line questionnaire with 32 respondents. Results: Regression testing needs and practices vary greatly between and within organizations and at different stages of a project. The importance and challenges of automation is clear from the survey. Conclusions: Most of the findings are general testing issues and are not specific to regression testing. Challenges and good practices relate to test automation and testability issues.
|
|
| 14. |
- Engström, Emelie, et al.
(author)
-
A systematic review on regression test selection techniques
- 2010
-
In: Information and Software Technology. - : Elsevier BV. - 0950-5849. ; 52:1, s. 14-30
-
Research review (peer-reviewed)abstract
- Regression testing is verifying that previously functioning software remains after a change. With the goal of finding a basis for further research in a joint industry-academia research project, we conducted a systematic review of empirical evaluations of regression test selection techniques. We identified 27 papers reporting 36 empirical studies, 21 experiments and 15 case studies. In total 28 techniques for regression test selection are evaluated. We present a qualitative analysis of the findings, an overview of techniques for regression test selection and related empirical evidence. No technique was found clearly superior since the results depend on many varying factors. We identified a need for empirical studies where concepts are evaluated rather than small variations in technical implementations.
|
|
| 15. |
- Engström, Emelie, et al.
(author)
-
An empirical evaluation of regression testing based on fix-cache recommendations
- 2010
-
In: [Host publication title missing].
-
Conference paper (peer-reviewed)abstract
- Background: The fix-cache approach to regression test selection was proposed to identify the most fault-prone files and corresponding test cases through analysis of fixed defect reports. Aim: The study aims at evaluating the efficiency of this approach, compared to the previous regression test selection strategy in a major corporation, developing embedded systems. Method: We launched a post-hoc case study applying the fix-cache selection method during six iterations of development of a multi- million LOC product. The test case execution was monitored through the test management and defect reporting systems of the company. Results: From the observations, we conclude that the fix-cache method is more efficient in four iterations. The difference is statistically significant at α = 0.05. Conclusions: The new method is significantly more efficient in our case study. The study will be replicated in an environment with better control of the test execution.efficiency of the regression testing based on these recommen- dations has not been evaluated earlier.
|
|
| 16. |
- Engström, Emelie, et al.
(author)
-
Decision support for test management and scope selection in a software product line context
- 2011
-
In: 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops. - 9781457700194 - 9780769543451
-
Conference paper (peer-reviewed)abstract
- In large software organizations with a product line development approach, system test planning and scope selection is a complex tasks for which tool support is needed. Due to repeated testing: across different testing levels, over time (test for regression) as well as of different variants, the risk of double testing is large as well as the risk of overlooking important tests, hidden by the huge amount of possible tests.This paper discusses the need and challenges of providing decision support for test planning and test selection in a product line context, and highlights possible paths towards a pragmatic implementation of context-specific decision support of various levels of automation. With existing regression testing approaches it is possible to provide automated decision support in a few specific cases, while test management in general may be supported through visualization of test execution coverage, the testing space and the delta between the sufficiently tested system and the system under test. A better understanding of the real world context and how to map research results to the same is needed.
|
|
| 17. |
- Engström, Emelie, et al.
(author)
-
Empirical evaluations of regression test selection techniques: a systematic review
- 2008
-
In: Proceedings of the Second ACM-IEEE international symposium on Empirical Software Engineering and Measurements. - New York, NY, USA : ACM. - 9781595939715 ; , s. 22-31
-
Conference paper (peer-reviewed)abstract
- Regression testing is the verification that previously functioning software remains after a change. In this paper we report on a systematic review of empirical evaluations of regression test selection techniques, published in major software engineering journals and conferences. Out of 2 923 papers analyzed in this systematic review, we identified 28 papers reporting on empirical comparative evaluations of regression test selection techniques. They report on 38 unique studies (23 experiments and 15 case studies), and in total 32 different techniques for regression test selection are evaluated. Our study concludes that no clear picture of the evaluated techniques can be provided based on existing empirical evidence, except for a small group of related techniques. Instead, we identified a need for more and better empirical studies were concepts are evaluated rather than small variations. It is also necessary to carefully consider the context in which studies are undertaken.
|
|
| 18. |
- Engström, Emelie
(author)
-
Exploring regression testing and software product line testing - research and state of practice
- 2010
-
Licentiate thesis (other academic/artistic)abstract
- In large software organizations with a product line development approach a selective testing of product variants is necessary in order to keep pace with the decreased development time for new products, enabled by the systematic reuse. The close relationship between products in product line indicates an option to reduce the testing effort due to redundancy. In many cases test selection is performed manually, based on test leaders’ expertise. This makes the cost and quality of the testing highly dependent on the skills and experience of the test leaders. There is a need in industry for systematic approaches to test selection. The goal of our research is to improve the control of the testing and reduce the amount of redundant testing in the product line context by applying regression test selection strategies. In this thesis, the state of art of regression testing and software product line testing are explored. Two extensive systematic reviews are conducted as well as an industrial survey of regression testing state of practice and an industrial evaluation of a pragmatic regression test selection strategy. Regression testing is not an isolated one-off activity, but rather an activity of varying scope and preconditions, strongly dependent on the context in which it is applied. Several techniques for regression test selection are proposed and evaluated empirically but in many cases the context is too specific for a technique to be easily applied directly by software developers. In order to improve the possibility for generalizing empirical results on regression test selection, guidelines for reporting the testing context are discussed in this thesis. Software product line testing is a relatively new research area. The understanding about challenges is well established but when looking for solutions to these challenges, we mostly find proposals, and empirical evaluations are sparse. Regression test selection strategies proposed in literature are not easily applicable in the product line context. Instead, control may be increased by increased visibility of the effects of testing and proper measurements of software quality. Focus of our future work will be on how to guide the planning and assessment of regression testing activities in large, complex reuse based systems, by visualizing the quality achieved in different parts of the system and evaluating the effects of different selection strategies when applied in various regression testing situations.
|
|
| 19. |
- Engström, Emelie, et al.
(author)
-
How software engineering research aligns with design science : a review
- 2020
-
In: Empirical Software Engineering. - : Springer Science and Business Media LLC. - 1573-7616 .- 1382-3256. ; 25:4
-
Journal article (peer-reviewed)abstract
- Background: Assessing and communicating software engineering research can be challenging. Design science is recognized as an appropriate research paradigm for applied research, but is rarely explicitly used as a way to present planned or achieved research contributions in software engineering. Applying the design science lens to software engineering research may improve the assessment and communication of research contributions. Aim: The aim of this study is 1) to understand whether the design science lens helps summarize and assess software engineering research contributions, and 2) to characterize different types of design science contributions in the software engineering literature. Method: In previous research, we developed a visual abstract template, summarizing the core constructs of the design science paradigm. In this study, we use this template in a review of a set of 38 award winning software engineering publications to extract, analyze and characterize their design science contributions. Results: We identified five clusters of papers, classifying them according to their different types of design science contributions. Conclusions: The design science lens helps emphasize the theoretical contribution of research output—in terms of technological rules—and reflect on the practical relevance, novelty and rigor of the rules proposed by the research.
|
|
| 20. |
- Engström, Emelie, et al.
(author)
-
Improving regression testing transparency and efficiency with history-based prioritization – an industrial case study
- 2011
-
In: [Host publication title missing]. - 9781612841748 ; , s. 367-376
-
Conference paper (peer-reviewed)abstract
- Abstract—Background: History based regression testing was proposed as a basis for automating regression test selection, for the purpose of improving transparency and test efficiency, at the function test level in a large scale software development organization. Aim: The study aims at investigating the current manual regression testing process as well as adopting, implementing and evaluating the effect of the proposed method. Method: A case study was launched including: identification of important factors for prioritization and selection of test cases, implementation of the method, and a quantitative and qualitative evaluation. Results: 10 different factors, of which two are history-based, are identified as important for selection. Most of the information needed is available in the test management and error reporting systems while some is embedded in the process. Transparency is increased through a semi-automated method. Our quantitative evaluation indicates a possibility to improve efficiency, while the qualitative evaluation supports the general principles of history-based testing but suggests changes in implementation details.
|
|
| 21. |
- Engström, Emelie, et al.
(author)
-
Indirect effects in evidential assessment: A case study on regression test technology adoption
- 2012
-
In: 2nd International Workshop on Evidential Assessment of Software Technologies, EAST 2012. Lund, 22 September 2012. - New York, NY, USA : ACM. - 9781450315098 ; , s. 15-20
-
Conference paper (peer-reviewed)abstract
- Background: There is a need for effcient regression testing in most software development organizations. Often the proposed solutions involve automation. However, despite this being a well researched area, research results are rarely applied in industrial practice. Aim: In this paper we aim to bridge the gap between research and practice by providing examples of how evidence-based regression testing approaches can be adopted in industry. We also discuss challenges for the research community. Method: An industrial case study was carried out to evaluate the possibility to improve regression testing at Sony Ericsson Mobile Communications. We analyse the procedure undertaken based on frameworks from the evidence based software engineering, EBSE, paradigm (with a focus on the evidence) and automation literature (with a focus on the practical effects). Results: Our results pinpoint the need for systematic approaches when introducing a new tool. Practitioners and researchers need congruent guidelines supporting the appraisal of both the evidence base and the pragmatic effects, both direct but also indirect, of the changes. This is illustrated by the introduction of the automation perspective.
|
|
| 22. |
- Engström, Emelie, et al.
(author)
-
Mapping software testing practice with software testing research—SERP-test taxonomy
- 2015
-
In: International conference on Software Testing, Verification and Validation Workshops. - : IEEE Computer Society.
-
Conference paper (peer-reviewed)abstract
- There is a gap between software testing research and practice. One reason is the discrepancy between how testing research is reported and how testing challenges are perceived in industry. We propose the SERP-test taxonomy to structure information on testing interventions and practical testing challenges from a common perspective and thus bridge the communication gap. To develop the taxonomy we follow a systematic incremental approach. The SERP-test taxonomy may be used by both researchers and practitioners to classify and search for testing challenges or interventions. The SERP-test taxonomy also supports comparison of testing interventions by providing an instrument for assessing the distance between them and thus identify relevant points of comparisons.
|
|
| 23. |
- Engström, Emelie
(author)
-
Regression test selection and product line system testing
- 2010
-
In: [Host publication title missing]. ; , s. 512-515
-
Conference paper (peer-reviewed)abstract
- Abstract— Context: Software product lines (SPL) are used in industry to achieve more efficient software development. To test a SPL is complex and costly and often becomes a bottleneck in the product line organization. Objective: This research aims to develop and evaluate strategies for improving system test selection in a SPL. Method: Initially industrial practices and research in both SPL testing and traditional regression test selection have been surveyed. Two systematic literature reviews, two industrial exploratory surveys and one industrial evaluation of a pragmatic test selection approach have been conducted. Results: There is a lack of industrial evaluations as well as of useful solutions, both regarding regression test selection and SPL testing. Test selection is an activity of varying scope and preconditions, strongly dependent on the context in which it is applied. Conclusions: Continued research will be done in close cooperation with industry with the goal to define a tool for visualizing system test coverage in a product line and the delta between a product and the covered part of the product line.
|
|
| 24. |
- Engström, Emelie, et al.
(author)
-
SERP-test : a taxonomy for supporting industry-academia communication
- 2017
-
In: Software quality journal. - : Springer-Verlag New York. - 0963-9314 .- 1573-1367. ; 25:4, s. 1269-1305
-
Journal article (peer-reviewed)abstract
- This paper presents the construction and evaluation of SERP-test, a taxonomy aimed to improve communication between researchers and practitioners in the area of software testing. SERP-test can be utilized for direct communication in industry academia collaborations. It may also facilitate indirect communication between practitioners adopting software engineering research and researchers who are striving for industry relevance. SERP-test was constructed through a systematic and goal-oriented approach which included literature reviews and interviews with practitioners and researchers. SERP-test was evaluated through an online survey and by utilizing it in an industry–academia collaboration project. SERP-test comprises four facets along which both research contributions and practical challenges may be classified: Intervention, Scope, Effect target and Context constraints. This paper explains the available categories for each of these facets (i.e., their definitions and rationales) and presents examples of categorized entities. Several tasks may benefit from SERP-test, such as formulating research goals from a problem perspective, describing practical challenges in a researchable fashion, analyzing primary studies in a literature review, or identifying relevant points of comparison and generalization of research.
|
|
| 25. |
- Engström, Emelie, et al.
(author)
-
Software product line testing - a systematic mapping study
- 2011
-
In: Information and Software Technology. - : Elsevier BV. - 0950-5849. ; 53:1, s. 2-13
-
Journal article (peer-reviewed)abstract
- Context: Software product lines (SPL) are used in industry to achieve more efficient software development. However, the testing side of SPL is underdeveloped. Objective: This study aims at surveying existing research on SPL testing in order to identify useful approaches and needs for future research. Method: A systematic mapping study is launched to find as much literature as possible, and the 64 papers found are classified with respect to focus, research type and contribution type. Results: A majority of the papers are of proposal research types (64 %). System testing is the largest group with respect to research focus (40%), followed by management (23%). Method contributions are in majority. Conclusions: More validation and evaluation research is needed to provide a better foundation for SPL testing.
|
|
| 26. |
- Engström, Emelie
(author)
-
Supporting Decisions on Regression test Scoping in a Software Product Line Context - from Evidence to Practice
- 2013
-
Doctoral thesis (other academic/artistic)abstract
- Large software organizations with a product line development approach face many challenges regarding testing. Test managers need to make tradeoffs along three dimensions of repeated testing (abstraction level, time and product space) and consider a range of goals and constraints. In order to keep pace with the decreased development time for new products, which is enabled by the systematic reuse, selective testing of product variants is necessary. A common industrial practice is to base test scoping decisions on practitioners’ expertise and experience. However, as software systems grow in size and complexity, the combinatorial explosion of test possibilities makes it infeasible to assess the reasonableness of the decisions without tool support. Research on regression test selection propose several systematic strategies for setting a proper test scope when verifying changes of previously tested code. The goal of our research is to improve test management and reduce the amount of redundant testing in the product line context by applying regression test selection strategies. However, despite extensive research on regression testing, gained knowledge has not impacted on industry practices. Thus a secondary goal of our research is to bridge the gap between research and practice in the field of regression testing. Test planning support, like any decision support system, must operate in a complex context and need to be pragmatic, adapted to the context and evolve incrementally within the context. This thesis explores state of art and state of practice of regression testing and software product line testing, and proposes and evaluates a visualization strategy to support regression test scoping in the product line context. Two extensive systematic literature reviews are conducted as well as four empirical studies in close cooperation with industry. Through visualization of relevant information at a proper level of detail, test management in general may be supported. A visual analytics tool for test management would also provide a framework which enables research based and context specific regression testing improvements.
|
|
| 27. |
- Engström, Emelie, et al.
(author)
-
Supporting Regression Test Scoping with Visual Analytics
- 2014
-
In: [Host publication title missing]. ; , s. 283-292
-
Conference paper (peer-reviewed)abstract
- Test managers have to repeatedly select test cases for test activities during evolution of large software systems. Researchers have widely studied automated test scoping, but have not fully investigated decision support with human interaction. We previously proposed the introduction of visual analytics for this purpose. Aim: In this empirical study we investigate how to design such decision support. Method: We explored the use of visual analytics using heat maps of historical test data for test scoping support by letting test managers evaluate prototype visualizations in three focus groups with in total nine industrial test experts. Results: All test managers in the study found the visual analytics useful for supporting test planning. However, our results show that different tasks and contexts require different types of visualizations. Conclusion: Important properties for test planning support are: ability to overview testing from different perspectives, ability to filter and zoom to compare subsets of the testing with respect to various attributes and the ability to manipulate the subset under analysis by selecting and deselecting test cases. Our results may be used to support the introduction of visual test analytics in practice.
|
|
| 28. |
- Engström, Emelie, et al.
(author)
-
Test Overlay in an Emerging Software Product Line – An Industrial Case Study
- 2013
-
In: Information and Software Technology. - : Elsevier BV. - 0950-5849. ; 55:3, s. 581-594
-
Journal article (peer-reviewed)abstract
- Context: In large software organizations with a product line development approach, system test planning and scope selection is a complex task. Due to repeated testing: across different testing levels, over time (test for regression) as well as of different variants, the risk of redundant testing is large as well as the risk of overlooking important tests, hidden by the huge amount of possible tests. Aims: This study assesses the amount and type of overlaid manual test- ing across feature, integration and system test in such context, it explores the causes of potential redundancy and elaborates on how to provide decision sup- port in terms of visualization for the purpose of avoiding redundancy. Method: An in-depth case study was launched including both qualitative and quantitative observations. Results: A high degree of test overlay is identified originating from distributed test responsibilities, poor documentation and structure of test cases, parallel work and insufficient delta analysis. The amount of test overlay depends on which level of abstraction is studied. Conclusions: Avoiding re- dundancy requires tool support, e.g. visualization of test design coverage, test execution progress, priorities of coverage items as well as visualized priorities of variants to support test case selection.
|
|
| 29. |
- Flemström, Daniel, 1971-
(author)
-
Similarity-Based Test Effort Reduction
- 2017
-
Licentiate thesis (other academic/artistic)abstract
- Embedded computer systems are all around us. We find them in everything, from dishwashers to cars and airplanes. They must always work correctly and moreover, often within certain time constraints. The software of such a system can be very large and complex, e.g. in the case of a car or a train. Hence, we develop the software for embedded systems in smaller, manageable, parts. These parts can be successively integrated, until they form the complete software for the embedded system, possibly at different levels. This phase of the development process is called the system integration phase and is one of the most critical phases in the development of embedded systems. In this phase, substantial effort is spent on testing activities.Studies have found that a considerable amount of test effort is wasteful due to people, unknowingly or by necessity, performing similar (or even overlapping) test activities. Consequently, test cases may end up to be similar, partially or wholly. We identified such test similarities in a case study of 2500 test cases, written in natural language, from four different projects in the embedded vehicular domain. Such information can be used for reducing effort when maintaining or automating similar test cases.In another case study in the same domain, we investigated several approaches for prioritizing test cases to automate with the objective to reduce manual test effort as quick as possible given that similar automated tests could be reused (similarity-based reuse). We analyzed how the automation order affects the test effort for four projects with a total of 3919 integration test cases, written in natural language. The results showed that similarity-based reuse of automated test case script code, and the best-performing automation order can reduce the expected manual test effort with 20 percentage points.Another way of reducing test effort is to reuse test artifacts from one level of integration to another, instead of duplicating them. We studied such reuse methods, that we denote vertical reuse, in a systematic mapping study. While the results from of our systematic mapping study showed the viability of vertical test reuse methods, our industrial case studies showed that keeping track of similarities and test overlaps is both possible and feasible for test effort reduction. We further conclude that the test case automation order affects the manual test execution effort when there exist similar steps that cannot be removed, but are possible to reuse with respect to test script code.
|
|
| 30. |
- Helgesson, Daniel, et al.
(author)
-
Cognitive Load Drivers in Large Scale Software Development
- 2019
-
In: 2019 IEEE/ACM 12th International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE). - 9781728122397 - 9781728122403
-
Conference paper (peer-reviewed)abstract
- Software engineers handle a lot of information intheir daily work. We explore how software engineers interact withinformation management systems/tools, and to what extent thesesystems expose users to increased cognitive load. We reviewed theliterature of cognitive aspects, relevant for software engineering,and performed an exploratory case study on how software engi-neers perceive information systems. Data was collected throughfive semistructured interviews. We present empirical evidence ofthe presence of cognitive load drivers, as a consequence of tooluse in large scale software engineering.
|
|
| 31. |
- Hrusto, Adha, et al.
(author)
-
Closing the Feedback Loop in DevOps Through Autonomous Monitors in Operations
- 2021
-
In: SN Computer Science. - : Springer Science and Business Media LLC. - 2662-995X .- 2661-8907. ; 2:6
-
Journal article (peer-reviewed)abstract
- DevOps represent the tight connection between development and operations. To address challenges that arise on the bor- derline between development and operations, we conducted a study in collaboration with a Swedish company responsible for ticket management and sales in public transportation. The aim of our study was to explore and describe the existing DevOps environment, as well as to identify how the feedback from operations can be improved, specifically with respect to the alerts sent from system operations. Our study complies with the basic principles of the design science paradigm, such as understanding and improving design solutions in the specific areas of practice. Our diagnosis, based on qualitative data collected through interviews and observations, shows that alert flooding is a challenge in the feedback loop, i.e. too much signals from operations create noise in the feedback loop. Therefore, we design a solution to improve the alert management by optimizing when to raise alerts and accordingly introducing a new element in the feedback loop, a smart filter. Moreover, we implemented a prototype of the proposed solution design and showed that a tighter relation between operations and development can be achieved, using a hybrid method which combines rule-based and unsupervised machine learning for operations data analysis.
|
|
| 32. |
- Hrusto, Adha, et al.
(author)
-
Optimization of Anomaly Detection in a Microservice System Through Continuous Feedback from Development
- 2022
-
In: IEEE/ACM 10th International Workshop on Software Engineering for Systems-of-Systems and Software Ecosystems (SESoS 2022). - New York, NY, USA : ACM. - 9781665462389 - 9781450393348 ; , s. 13-20
-
Conference paper (peer-reviewed)abstract
- Monitoring a microservice system may bring a lot of benefits to development teams such as early detection of run-time errors and various performance anomalies. In this study, we explore deep learning (DL) solutions for detection of anomalous system's behavior based on collected monitoring data that consists of applications' and systems' performance metrics. The study is conducted in a collaboration with a Swedish company responsible for ticket and payment management in public transportation. Moreover, we specifically address a shortage of approaches for evaluating DL models without any ground truth data. Hence, we propose a solution design for anomaly detection and reporting alerts inspired by state-of-the-art DL solutions. Furthermore, we propose a plan for its in-context implementation and evaluation empowered by feedback from the development team. Through continuous feedback from development, the labeled data is generated and used for optimization of the DL model. In this way, a microservice system may leverage DL solutions to address rising challenges within its architecture.
|
|
| 33. |
- Hrusto, Adha, et al.
(author)
-
Towards optimization of anomaly detection in DevOps
- 2023
-
In: Information and Software Technology. - 0950-5849. ; 160
-
Journal article (peer-reviewed)abstract
- Context: DevOps has recently become a mainstream solution for bridging the gaps between development (Dev) and operations (Ops) enabling cross-functional collaboration. The DevOps concept of continuous monitoring may bring a lot of benefits to development teams such as early detection of run-time errors and various performance anomalies. Objective: We aim to explore deep learning (DL) solutions for detection of anomalous systems behavior based on collected monitoring data that consists of applications’ and systems’ performance metrics. Moreover, we specifically address a shortage of approaches for evaluating DL models without any ground truth data. Methods: We perform a case study in a real DevOps environment, following the principles of the design science paradigm. The research activities span from practice to theory and from problem to solution domain, including problem conceptualization, solution design, instantiation, and empirical validation. Results: We proposed and implemented a cloud solution for DL model deployment and evaluation empowered by feedback from the development team. The labeled data generated through the feedback was used for evaluation of current and training of new DL models in several iterations. The overall results showed that reconstruction-based models such as autoencoders, are quite robust to any parameter modification and are among the preferred for anomaly detection in multivariate monitoring data. Conclusion: Leveraging raw monitoring data and DL-inspired solutions, DevOps teams may get critical insights into the software and its operation. In our case, this proved to be an efficient way of discovering early signs of production failures.
|
|
| 34. |
- Jernberg, Hugo, et al.
(author)
-
Getting Started with Chaos Engineering – design of an implementation framework in practice
- 2020
-
In: Proceedings of the ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM) Industry Track. - New York, NY, USA : ACM. - 9781450375801
-
Conference paper (peer-reviewed)abstract
- Background. Chaos Engineering is proposed as a practice to verify a system’s resilience under real, operational conditions. It employs fault injection, is originally developed at Netflix, and supported by several tools from there and other sources. Aims. We aim to intro- duce Chaos Engineering at ICA Gruppen AB, a group of companies whose core business is grocery retail, to improve their systems’ resilience, and to capture our knowledge gained from literature and interviews in a process framework for the introduction of Chaos Engineering. Method. The research is conducted under the design science paradigm, where the problem is conceptualized through a literature study of Chaos Engineering and exploratory interviews in the company. The solution framework is designed based on the literature and a tool survey, and validated by letting software en- gineers at ICA apply parts of it to the software systems of ica.se website, including its e-shop. Results. The main contributions are a synthesis of Chaos Engineering literature and tools, in depth un- derstanding of the needs of the case company, and guidelines for introducing Chaos Engineering. Conclusions. The applied parts were concluded to be feasible and they successfully discovered a set of initial improvement opportunities for the system’s resilience, as well as a suitable Chaos Engineering practice for future resilience testing of the system. We recommend companies using the frame- work as a guide for the implementation of Chaos Engineering.
|
|
| 35. |
- Laiq, Muhammad, et al.
(author)
-
A data-driven approach for understanding invalid bug reports : An industrial case study
- 2023
-
In: Information and Software Technology. - : Elsevier. - 0950-5849 .- 1873-6025. ; 164
-
Journal article (peer-reviewed)abstract
- Context: Bug reports created during software development and maintenance do not always describe deviations from a system's valid behavior. Such invalid bug reports may consume significant resources and adversely affect the prioritization and resolution of valid bug reports. There is a need to identify preventive actions to reduce the inflow of invalid bug reports. Existing research has shown that manually analyzing invalid bug report descriptions provides cues regarding preventive actions. However, such a manual approach is not cost-effective due to the time required to analyze a sufficiently large number of bug reports needed to identify useful patterns. Furthermore, the analysis needs to be repeated as the underlying causes of invalid bug reports change over time. Objective: In this study, we propose and evaluate the use of Latent Dirichlet Allocation (LDA), a topic modeling approach, to support practitioners in suggesting preventive actions to avoid the creation of similar invalid bug reports in the future. Method: In an industrial case study, we first manually analyzed descriptions of invalid bug reports to identify common patterns in their descriptions. We further investigated to what extent LDA can support this manual process. We used expert-based validation to evaluate the relevance of identified common patterns and their usefulness in suggesting preventive measures. Results: We found that invalid bug reports have common patterns that are perceived as relevant, and they can be used to devise preventive measures. Furthermore, the identification of common patterns can be supported with automation. Conclusion: Using LDA, practitioners can effectively identify representative groups of bug reports (i.e., relevant common patterns) from a large number of bug reports and analyze them further to devise preventive measures. © 2023 The Author(s)
|
|
| 36. |
- Laiq, Muhammad, et al.
(author)
-
Early Identification of Invalid Bug Reports in Industrial Settings – A Case Study
- 2022
-
In: Product-Focused Software Process Improvement. - Cham : Springer Science+Business Media B.V.. - 1611-3349 .- 0302-9743. - 9783031213878 ; , s. 497-507
-
Conference paper (peer-reviewed)abstract
- Software development companies spend considerable time resolving bug reports. However, bug reports might be invalid, i.e., not point to a valid flaw. Expensive resources and time might be expended on invalid bug reports before discovering that they are invalid. In this case study, we explore the impact of invalid bug reports and develop and assess the use of machine learning (ML) to indicate whether a bug report is likely invalid. We found that about 15% of bug reports at the case company are invalid, and that their resolution time is similar to valid bug reports. Among the ML-based techniques we used, logistic regression and SVM show promising results. In the feedback, practitioners indicated an interest in using the tool to identify invalid bug reports at early stages. However, they emphasized the need to improve the explainability of ML-based recommendations and to reduce the maintenance cost of the tool. © 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.
|
|
| 37. |
- Laiq, Muhammad, et al.
(author)
-
Industrial adoption of machine learning techniques for early identification of invalid bug reports
- 2024
-
In: Empirical Software Engineering. - 1573-7616.
-
Journal article (peer-reviewed)abstract
- Despite the accuracy of machine learning (ML) techniques in predict-ing invalid bug reports, as shown in earlier research, and the importance of earlyidentification of invalid bug reports in software maintenance, the adoption of MLtechniques for this task in industrial practice is yet to be investigated. In this study,we used a technology transfer model to guide the adoption of an ML technique at acompany for the early identification of invalid bug reports. In the process, we alsoidentify necessary conditions for adopting such techniques in practice. We followeda case study research approach with various design and analysis iterations for tech-nology transfer activities. We collected data from bug repositories, through focusgroups, a questionnaire, and a presentation and feedback session with an expert.As expected, we found that an ML technique can identify invalid bug reports withacceptable accuracy at an early stage. However, the technique’s accuracy dropsover time in its operational use due to changes in the product, the used technolo-gies, or the development organization. Such changes may require retraining theML model. During validation, practitioners highlighted the need to understandthe ML technique’s predictions to trust the predictions. We found that a visual(using a state-of-the-art ML interpretation framework) and descriptive explana-tion of the prediction increases the trustability of the technique compared to justpresenting the results of the validity predictions. We conclude that trustability,integration with the existing toolchain, and maintaining the techniques’ accuracyover time are critical for increasing the likelihood of adoption.
|
|
| 38. |
- Mäntylä, Mika, et al.
(author)
-
On Rapid Releases and Software Testing
- 2013
-
In: [Host publication title missing]. - Eindhoven, Netherlands. - 1063-6773. ; , s. 20-29
-
Conference paper (peer-reviewed)abstract
- Abstract—Large open and closed source organizations like Google, Facebook and Mozilla are migrating their products towards rapid releases. While this allows faster time-to-market and user feedback, it also implies less time for testing and bug fixing. Since initial research results indeed show that rapid releases fix proportionally less reported bugs than traditional releases, this paper investigates the changes in software testing effort after moving to rapid releases. We analyze the results of 312,502 execution runs of the 1,547 mostly manual systemlevel test cases of Mozilla Firefox from 2006 to 2012 (5 major traditional and 9 major rapid releases), and triangulated our findings with a Mozilla QA engineer. In rapid releases, testing has a narrower scope that enables deeper investigation of the features and regressions with the highest risk, while traditional releases run the whole test suite. Furthermore, rapid releases make it more difficult to build a large testing community, forcing Mozilla to increase contractor resources in order to sustain testing for rapid releases.
|
|
| 39. |
- Mäntylä, M.V., et al.
(author)
-
On rapid releases and software testing : a case study and a semi-systematic literature review
- 2015
-
In: Empirical Software Engineering. - : Springer Netherlands. - 1382-3256 .- 1573-7616. ; 20:5, s. 1384-1425
-
Journal article (peer-reviewed)abstract
- Large open and closed source organizations like Google, Facebook and Mozilla are migrating their products towards rapid releases. While this allows faster time-to-market and user feedback, it also implies less time for testing and bug fixing. Since initial research results indeed show that rapid releases fix proportionally less reported bugs than traditional releases, this paper investigates the changes in software testing effort after moving to rapid releases in the context of a case study on Mozilla Firefox, and performs a semi-systematic literature review. The case study analyzes the results of 312,502 execution runs of the 1,547 mostly manual system-level test cases of Mozilla Firefox from 2006 to 2012 (5 major traditional and 9 major rapid releases), and triangulates our findings with a Mozilla QA engineer. We find that rapid releases have a narrower test scope that enables a deeper investigation of the features and regressions with the highest risk. Furthermore, rapid releases make testing more continuous and have proportionally smaller spikes before the main release. However, rapid releases make it more difficult to build a large testing community , and they decrease test suite diversity and make testing more deadline oriented. In addition, our semi-systematic literature review presents the benefits, problems and enablers of rapid releases from 24 papers found using systematic search queries and a similar amount of papers found through other means. The literature review shows that rapid releases are a prevalent industrial practice that are utilized even in some highly critical domains of software engineering, and that rapid releases originated from several software development methodologies such as agile, open source, lean and internet-speed software development. However, empirical studies proving evidence of the claimed advantages and disadvantages of rapid releases are scarce.
|
|
| 40. |
- Petersen, Kai, et al.
(author)
-
Finding relevant research solutions for practical problems : the serp taxonomy architecture
- 2014
-
Conference paper (peer-reviewed)abstract
- Background: Experience and research indicates that there exist a communication gap between research and industry in software engineering. Objective: We propose the Software Engineering Research and Practice (SERP) taxonomy architecture to support communication between practitioners and researchers. The taxonomy architecture provides a basis for classifying research from a problem perspective which in turn supports the breaking down of complex practical challenges to researchable units. Thus such taxonomy may support the mapping of challenges in industry to research solutions in the software engineering context. Method: In this paper we present SERP and exemplifies its usage based on two literature studies in the field of software engineering. Further, we discuss how a taxonomy based on this architecture could have helped us in two past research projects that were conducted in close collaboration with industry. Finally we validate SERP by applying it to the area of software testing, developing SERP-test, and interviewing two industry practitioners and two researchers. Results: The taxonomy architecture has been applied to two problems in software testing, and has been assessed through interviews with practitioners and researchers. The interviews provided suggestions of how to improve the taxonomy architecture, which have been incorporated. With two examples, we demonstrated how the taxonomy architecture could be used to find solutions for industrial problems, and to find the problems addressed by a particular solution. Conclusion: SERP may be useful in multiple ways: (1) Given that SERP taxonomies are populated with industrial problems and scientific solutions, we could rapidly identify candidate research solutions for industrial practice. (2) Researchers could benefit from the taxonomy in the reporting of their research to ease the mapping to industrial challenges.
|
|
| 41. |
|
|
| 42. |
- Petersen, Kai, et al.
(author)
-
Revisiting the construct and assessment of industrial relevance in software engineering research within
- 2024
-
In: 1st International Workshop on Methodological Issues with Empirical Studies in Software Engineering.
-
Conference paper (peer-reviewed)abstract
- Industrial relevance is essential for an applied research area like software engineering. However, it is unclear how to achieve industrial relevance and how we communicate and assess it. We propose a reasoning framework to support the design, reporting, and assessment of research for industrial relevance.
|
|
| 43. |
- Rico, Sergio, et al.
(author)
-
A case study of industry–academia communication in a joint software engineering research project
- 2021
-
In: Journal of software: Evolution and Process. - : Wiley. - 2047-7481 .- 2047-7473.
-
Journal article (peer-reviewed)abstract
- Empirical software engineering research relies on good communication with industrial partners. Conducting joint research both requires and contributes to bridging the communication gap between industry and academia (IA) in software engineering. This study aims to explore communication between the two parties in such a setting. To better understand what facilitates good IA communication and what project outcomes such communication promotes, we performed a case study, in the context of a long-term IA joint project, followed by a validating survey among practitioners and researchers with experience of working in similar settings. We identified five facilitators of IA communication and nine project outcomes related to this communication. The facilitators concern the relevance of the research, practitioners' attitude and involvement in research, frequency of communication and longevity of the collaboration. The project outcomes promoted by this communication include, for researchers, changes in teaching and new scientific venues, and for practitioners, increased awareness, changes to practice, and new tools and source code. Besides, both parties gain new knowledge and develop social-networks through IA communication. Our study presents empirically based insights that can provide advise on how to improve communication in IA research projects and thus the co-creation of software engineering knowledge that is anchored in both practice and research.
|
|
| 44. |
- Rico, Sergio, et al.
(author)
-
A taxonomy for improving industry-academia communication in IoT vulnerability management
- 2019
-
In: 2019 45th Euromicro Conference on Software Engineering and Advanced Applications (SEAA). - 9781728134222 - 9781728134215 ; , s. 38-45
-
Conference paper (peer-reviewed)abstract
- Background: In software engineering, industry-academia is a symbiotic relationship. Researchers need to be aware of the industry to produce relevant research, while practitioners are educated in academia and could take advantage of empirical research. The SERP taxonomy architecture is designed to support communication between practitioners and researchers in software engineering. Objective: The purpose of this study is to analyze to what extent the SERP taxonomy architecture is useful for improving communication between researchers and practitioners in IoT vulnerability management. Method: We developed a SERP taxonomy for IoT vulerability management, SERP-MENTION, in an incremental way. Along the development, we evaluated the developed taxonomy in a project of industry academia collaboration. Results: In addition to the taxonomy itself we elaborate on the taxonomy development process and the potential of SERP-MENTION to support communication between researchers and practitioners within the area.Conclusion: The SERP architecture can be used in a new field, it is perceived as useful by potential users to better describe and communicate research outputs and practical challenges in software vulnerability management.
|
|
| 45. |
- Rico, Sergio, et al.
(author)
-
Experiences from conducting rapid reviews in collaboration with practitioners — Two industrial cases
- 2024
-
In: Information and Software Technology. - : Elsevier. - 0950-5849 .- 1873-6025. ; 167
-
Journal article (peer-reviewed)abstract
- Context: Evidence-based software engineering (EBSE) aims to improve research utilization in practice. It relies on systematic methods to identify, appraise, and synthesize existing research findings to answer questions of interest for practice. However, the lack of practitioners’ involvement in these studies’ design, execution, and reporting indicates a lack of appreciation for the need for knowledge exchange between researchers and practitioners. The resultant systematic literature studies often lack relevance for practice. Objective: This paper explores the use of Rapid Reviews (RRs), in fostering knowledge exchange between academia and industry. Through the lens of two case studies, we delve into the practical application and experience of conducting RRs. Methods: We analyzed the conduct of two rapid reviews by two different groups of researchers and practitioners. We collected data through interviews, and the documents produced during the review (like review protocols, search results, and presentations). The interviews were analyzed using thematic analysis. Results: We report how the two groups of researchers and practitioners performed the rapid reviews. We observed some benefits, like promoting dialogue and paving the way for future collaborations. We also found that practitioners entrusted the researchers to develop and follow a rigorous approach and were more interested in the applicability of the findings in their context. The problems investigated in these two cases were relevant but not the most immediate ones. Therefore, rapidness was not a priority for the practitioners. Conclusion: The study illustrates that rapid reviews can support researcher-practitioner communication and industry-academia collaboration. Furthermore, the recommendations based on the experiences from the two cases complement the detailed guidelines researchers and practitioners may follow to increase interaction and knowledge exchange. © 2023 The Author(s)
|
|
| 46. |
- Rico, Sergio, et al.
(author)
-
Guidelines for conducting interactive rapid reviews in software engineering -- from a focus on technology transfer to knowledge exchange
-
Reports (other academic/artistic)abstract
- Evidence-based software engineering (EBSE) aims to improve research utilization in practice. It relies on systematic methods (like systematic literature reviews, systematic mapping studies, and rapid reviews) to identify, appraise, and synthesize existing research findings to answer questions of interest. However, the lack of practitioners' involvement in the design, execution, and reporting of these methods indicates a lack of appreciation for knowledge exchange between researchers and practitioners. Within EBSE, the main reason for conducting these systematic studies is to answer the practitioner's questions and impact practice. However, in many cases, academics have undertaken these studies without any direct involvement of practitioners.This report focuses on the rapid review guidelines and presents practical advice on conducting these with practitioner involvement to facilitate knowledge co-creation.Based on a literature review of rapid reviews and stakeholders engagement in medicine and our experience of using secondary studies in software engineering, we propose extensions to an existing proposal for rapid reviews in software engineering to increase researchers-practitioners knowledge exchange. We refer to the extended method as an interactive rapid review.An interactive rapid review is a streamlined approach to conduct agile literature reviews in close collaboration between researchers and practitioners in software engineering. This report describes the process and discusses possible usage scenarios and some reflections from the proposal's ongoing evaluation.The proposed guidelines will potentially boost knowledge co-creation through active researcher-practitioner interaction by streamlining practitioners' involvement and recognizing the need for an agile process.
|
|
| 47. |
- Runeson, Per, et al.
(author)
-
Regression Testing in Software Product Line Engineering
- 2012
-
In: Advances in Computers. - 0065-2458. ; 86, s. 223-263
-
Journal article (peer-reviewed)abstract
- Software product line engineering is an approach to cost-efficiently derive tailored products to markets and customers, utilizing common components and services in a planned manner. Product lines have been applied to other engineering fields for decades, while being quite recently introduced in software engineering. For software product lines, productivity gains are mostly related to the development process. Especially, software product line testing faces challenges in the vast number of versions and variants of software products to be tested, originating from a software product line, and consequently the risk for redundant testing. The testing challenges resemble those of regression testing in one-off software development, although adding the complexity of parallel variants. Ongoing research provide some support for software product line test selection, although they are too small-scale and require more formalism than generally available in practice. We propose a visualization approach to help test managers improve communication about testing in order to utilize test resources efficiently.
|
|
| 48. |
- Runeson, Per, et al.
(author)
-
Software Product Line Testing – A 3D Regression Testing Problem
- 2012
-
In: [Host publication title missing]. ; , s. 742-746
-
Conference paper (peer-reviewed)abstract
- In software product line engineering, testing for regression concerns not only versions, as in one-off product development, but also regression across variants. We propose a 3D process model, with the dimensions of level, version and variant, to help analyze, plan and manage software product line testing. We derive the model from empirical observations of regression testing practice and software product line testing theory and practice, and look forward to see the model evaluated in practitioner-oriented research.
|
|
| 49. |
|
|
| 50. |
- Runeson, Per, et al.
(author)
-
The Design Science Paradigm as a Frame for Empirical Software Engineering
- 2020
-
In: Contemporary Empirical Methods in Software Engineering. - Cham : Springer International Publishing. - 9783030324889 - 9783030324896 ; , s. 127-147
-
Book chapter (peer-reviewed)abstract
- Software engineering research aims to help improve real-world practice. With the adoption of empirical software engineering research methods, the understanding of real-world needs and validation of solution proposals have evolved. However, the philosophical perspective on what constitutes theoretical knowledge and research contributions in software engineering is less discussed in the community. In this chapter, we use the design science paradigm as a frame for articulating and communicating prescriptive software engineering research contributions. Design science embraces problem conceptualization, solution (or artifact) design, and validation of solution proposals, with recommendations for practice phrased as technological rules. Design science is used in related research areas, particularly information systems and management theory. We elaborate the constructs of design science for software engineering, relate them to different conceptualizations of design science, and provide examples of possible research methods. We outline how the assessment of research contributions, industry–academia communication, and theoretical knowledge building may be supported by the design science paradigm. Finally, we provide examples of software engineering research presented through a design science lens.
|
|
