| 1. |
- Fagerholm, F., et al.
(författare)
-
Cognition in Software Engineering: A Taxonomy and Survey of a Half-Century of Research
- 2022
-
Ingår i: Acm Computing Surveys. - : Association for Computing Machinery (ACM). - 0360-0300 .- 1557-7341. ; 54:11
-
Tidskriftsartikel (refereegranskat)abstract
- Cognition plays a fundamental role in most software engineering activities. This article provides a taxonomy of cognitive concepts and a survey of the literature since the beginning of the Software Engineering discipline. The taxonomy comprises the top-level concepts of perception, attention, memory, cognitive load, reasoning, cognitive biases, knowledge, social cognition, cognitive control, and errors, and procedures to assess them both qualitatively and quantitatively. The taxonomy provides a useful tool to filter existing studies, classify new studies, and support researchers in getting familiar with a (sub) area. In the literature survey, we systematically collected and analysed 311 scientific papers spanning five decades and classified them using the cognitive concepts from the taxonomy. Our analysis shows that the most developed areas of research correspond to the four life-cycle stages, software requirements, design, construction, and maintenance. Most research is quantitative and focuses on knowledge, cognitive load, memory, and reasoning. Overall, the state of the art appears fragmented when viewed from the perspective of cognition. There is a lack of use of cognitive concepts that would represent a coherent picture of the cognitive processes active in specific tasks. Accordingly, we discuss the research gap in each cognitive concept and provide recommendations for future research.
|
|
| 2. |
- Gomes, Francisco, 1987, et al.
(författare)
-
Evolution of statistical analysis in empirical software engineering research : Current state and steps forward
- 2019
-
Ingår i: Journal of Systems and Software. - : Elsevier Inc.. - 0164-1212 .- 1873-1228. ; 156, s. 246-267
-
Tidskriftsartikel (refereegranskat)abstract
- Software engineering research is evolving and papers are increasingly based on empirical data from a multitude of sources, using statistical tests to determine if and to what degree empirical evidence supports their hypotheses. To investigate the practices and trends of statistical analysis in empirical software engineering (ESE), this paper presents a review of a large pool of papers from top-ranked software engineering journals. First, we manually reviewed 161 papers and in the second phase of our method, we conducted a more extensive semi-automatic classification of papers spanning the years 2001–2015 and 5196 papers. Results from both review steps was used to: i) identify and analyse the predominant practices in ESE (e.g., using t-test or ANOVA), as well as relevant trends in usage of specific statistical methods (e.g., nonparametric tests and effect size measures) and, ii) develop a conceptual model for a statistical analysis workflow with suggestions on how to apply different statistical methods as well as guidelines to avoid pitfalls. Lastly, we confirm existing claims that current ESE practices lack a standard to report practical significance of results. We illustrate how practical significance can be discussed in terms of both the statistical analysis and in the practitioner's context. © 2019 Elsevier Inc.
|
|
| 3. |
- Gren, Lucas, 1984, et al.
(författare)
-
Group development and group maturity when building agile teams: A qualitative and quantitative investigation at eight large companies
- 2017
-
Ingår i: Journal of Systems and Software. - : Elsevier BV. - 0164-1212 .- 1873-1228. ; 124, s. 104-119
-
Tidskriftsartikel (refereegranskat)abstract
- The agile approach to projects focuses more on close-knit teams than traditional waterfall projects, which means that aspects of group maturity become even more important. This psychological aspect is not much researched in connection to the building of an "agile team." The purpose of this study is to investigate how building agile teams is connected to a group development model taken from social psychology. We conducted ten semi-structured interviews with coaches, Scrum Masters, and managers responsible for the agile process from seven different companies, and collected survey data from 66 group-members from four companies (a total of eight different companies). The survey included an agile measurement tool and the one part of the Group Development Questionnaire. The results show that the practitioners define group developmental aspects as key factors to a successful agile transition. Also, the quantitative measurement of agility was significantly correlated to the group maturity measurement. We conclude that adding these psychological aspects to the description of the "agile team" could increase the understanding of agility and partly help define an "agile team." We propose that future work should develop specific guidelines for how software development teams at different maturity levels might adopt agile principles and practices differently. (C) 2016 Elsevier Inc. All rights reserved.
|
|
| 4. |
- Gren, Lucas, 1984, et al.
(författare)
-
The Prospects of a Quantitative Measurement of Agility: A Validation Study on an Agile Maturity Model
- 2015
-
Ingår i: Journal of Systems and Software. - : Elsevier BV. - 0164-1212 .- 1873-1228. ; 107, s. 38-49
-
Tidskriftsartikel (refereegranskat)abstract
- Agile development has now become a well-known approach to collaboration in professional work life. Both researchers and practitioners want validated tools to measure agility. This study sets out to validate an agile maturity measurement model with statistical tests and empirical data. First, a pretest was conducted as a case study including a survey and focus group. Second, the main study was conducted with 45 employees from two SAP customers in the US. We used internal consistency (by a Cronbach’s alpha) as the main measure for reliability and analyzed construct validity by exploratory principal factor analysis (PFA). The results suggest a new categorization of a subset of items existing in the tool and provides empirical support for these new groups of factors. However, we argue that more work is needed to reach the point where a maturity models with quantitative data can be said to validly measure agility, and even then, such a measurement still needs to include some deeper analysis with cultural and contextual items.
|
|
| 5. |
- Lenberg, Per, 1976, et al.
(författare)
-
An initial analysis of software engineers’ attitudes towards organizational change
- 2017
-
Ingår i: Empirical Software Engineering. - : Springer Science and Business Media LLC. - 1382-3256 .- 1573-7616. ; 22, s. 2179-2205
-
Tidskriftsartikel (refereegranskat)abstract
- © 2016, The Author(s). Employees’ attitudes towards organizational change are a critical determinant in the change process. Researchers have therefore tried to determine what underlying concepts that affect them. These extensive efforts have resulted in the identification of several antecedents. However, no studies have been conducted in a software engineering context and the research has provided little information on the relative impact and importance of the identified concepts. In this study, we have combined results from previous social science research with results from software engineering research, and thereby identified three underlying concepts with an expected significant impact on software engineers’ attitudes towards organizational change, i.e. their knowledge about the intended change outcome, their understanding of the need for change, and their feelings of participation in the change process. The result of two separate multiple regression analysis, where we used industrial questionnaire data (N=56), showed that the attitude concept openness to change is predicted by all three concepts, while the attitude concept readiness for change is predicted by need for change and participation. Our research provides an empirical baseline to an important area of software engineering and the result can be a starting-point for future organizational change research. In addition, the proposed model prescribes practical directions for software engineering organizations to adopt in improving employees’ responses to change and, thus, increase the probability of a successful change.
|
|
| 6. |
- Marculescu, Bogdan, et al.
(författare)
-
An initial industrial evaluation of interactive search-based testing for embedded software
- 2015
-
Ingår i: Applied Soft Computing. - : Elsevier. - 1568-4946 .- 1872-9681. ; 29, s. 26-39
-
Tidskriftsartikel (refereegranskat)abstract
- Search-based software testing promises the ability to generate and evaluate large numbers of test cases at minimal cost. From an industrial perspective, this could enable an increase in product quality without a matching increase in the time and effort required to do so. Search-based software testing, however, is a set of quite complex techniques and approaches that do not immediately translate into a process for use with most companies. For example, even if engineers receive the proper education and training in these new approaches, it can be hard to develop a general fitness function that covers all contingencies. Furthermore, in industrial practice, the knowledge and experience of domain specialists are often key for effective testing and thus for the overall quality of the final software system. But it is not clear how such domain expertise can be utilized in a search-based system. This paper presents an interactive search-based software testing (ISBST) system designed to operate in an industrial setting and with the explicit aim of requiring only limited expertise in software testing. It uses SBST to search for test cases for an industrial software module, while also allowing domain specialists to use their experience and intuition to interactively guide the search. In addition to presenting the system, this paper reports on an evaluation of the system in a company developing a framework for embedded software controllers. A sequence of workshops provided regular feedback and validation for the design and improvement of the ISBST system. Once developed, the ISBST system was evaluated by four electrical and system engineers from the company (the ’domain specialists’ in this context) used the system to develop test cases for a commonly used controller module. As well as evaluating the utility of the ISBST system, the study generated interaction data that were used in subsequent laboratory experimentation to validate the underlying search-based algorithm in the presence of realistic, but repeatable, interactions. The results validate the importance that automated software testing tools in general, and search-based tools, in particular, can leverage input from domain specialists while generating tests. Furthermore, the evaluation highlighted benefits of using such an approach to explore areas that the current testing practices do not cover or cover insufficiently. © 2014 Elsevier B.V. All rights reserved.
|
|
| 7. |
- Marculescu, Bogdan, et al.
(författare)
-
Tester interactivity makes a difference in search-based software testing : A controlled experiment
- 2016
-
Ingår i: Information and Software Technology. - : Elsevier. - 0950-5849 .- 1873-6025. ; 78, s. 66-82
-
Tidskriftsartikel (refereegranskat)abstract
- Context: Search-based software testing promises to provide users with the ability to generate high quality test cases, and hence increase product quality, with a minimal increase in the time and effort required. The development of the Interactive Search-Based Software Testing (ISBST) system was motivated by a previous study to investigate the application of search-based software testing (SBST) in an industrial setting. ISBST allows users to interact with the underlying SBST system, guiding the search and assessing the results. An industrial evaluation indicated that the ISBST system could find test cases that are not created by testers employing manual techniques. The validity of the evaluation was threatened, however, by the low number of participants. Objective: This paper presents a follow-up study, to provide a more rigorous evaluation of the ISBST system. Method: To assess the ISBST system a two-way crossover controlled experiment was conducted with 58 students taking a Verification and Validation course. The NASA Task Load Index (NASA-TLX) is used to assess the workload experienced by the participants in the experiment. Results:The experimental results validated the hypothesis that the ISBST system generates test cases that are not found by the same participants employing manual testing techniques. A follow-up laboratory experiment also investigates the importance of interaction in obtaining the results. In addition to this main result, the subjective workload was assessed for each participant by means of the NASA-TLX tool. The evaluation showed that, while the ISBST system required more effort from the participants, they achieved the same performance. Conclusions: The paper provides evidence that the ISBST system develops test cases that are not found by manual techniques, and that interaction plays an important role in achieving that result. (C) 2016 Elsevier B.V. All rights reserved.
|
|
| 8. |
- Marculescu, Bogdan, 1984, et al.
(författare)
-
Transferring interactive search-based software testing to industry
- 2018
-
Ingår i: Journal of Systems and Software. - : Elsevier BV. - 0164-1212 .- 1873-1228. ; 142, s. 156-170
-
Tidskriftsartikel (refereegranskat)abstract
- Context: Search-Based Software Testing (SBST), and the wider area of Search-Based Software Engineering (SBSE), is the application of optimization algorithms to problems in software testing, and software engineering, respectively. New algorithms, methods, and tools are being developed and validated on benchmark problems. In previous work, we have also implemented and evaluated Interactive Search-Based Software Testing (ISBST) tool prototypes, with a goal to successfully transfer the technique to industry. Objective: While SBST and SBSE solutions are often validated on benchmark problems, there is a need to validate them in an operational setting, and to assess their performance in practice. The present paper discusses the development and deployment of SBST tools for use in industry, and reflects on the transfer of these techniques to industry. Method: In addition to previous work discussing the development and validation of an ISBST prototype, a new version of the prototype ISBST system was evaluated in the laboratory and in industry. This evaluation is based on an industrial System under Test (SUT) and was carried out with industrial practitioners. The Technology Transfer Model is used as a framework to describe the progression of the development and evaluation of the ISBST system, as it progresses through the first five of its seven steps. Results: The paper presents a synthesis of previous work developing and evaluating the ISBST prototype, as well as presenting an evaluation, in both academia and industry, of that prototype's latest version. In addition to the evaluation, the paper also discusses the lessons learned from this transfer. Conclusions: This paper presents an overview of the development and deployment of the ISBST system in an industrial setting, using the framework of the Technology Transfer Model. We conclude that the ISBST system is capable of evolving useful test cases for that setting, though improvements in the means the system uses to communicate that information to the user are still required. In addition, a set of lessons learned from the project are listed and discussed. Our objective is to help other researchers that wish to validate search-based systems in industry, and provide more information about the benefits and drawbacks of these systems.
|
|
| 9. |
- Torkar, Richard, 1971, et al.
(författare)
-
A Method to Assess and Argue for Practical Significance in Software Engineering
- 2022
-
Ingår i: IEEE Transactions on Software Engineering. - : IEEE Computer Society. - 0098-5589 .- 1939-3520. ; 48:6, s. 2053-2065
-
Tidskriftsartikel (refereegranskat)abstract
- A key goal of empirical research in software engineering is to assess practical significance, which answers the question whether the observed effects of some compared treatments show a relevant difference in practice in realistic scenarios. Even though plenty of standard techniques exist to assess statistical significance, connecting it to practical significance is not straightforward or routinely done; indeed, only a few empirical studies in software engineering assess practical significance in a principled and systematic way. In this paper, we argue that Bayesian data analysis provides suitable tools to assess practical significance rigorously. We demonstrate our claims in a case study comparing different test techniques. The case study's data was previously analyzed (Afzal et al., 2015) using standard techniques focusing on statistical significance. Here, we build a multilevel model of the same data, which we fit and validate using Bayesian techniques. Our method is to apply cumulative prospect theory on top of the statistical model to quantitatively connect our statistical analysis output to a practically meaningful context. This is then the basis both for assessing and arguing for practical significance. Our study demonstrates that Bayesian analysis provides a technically rigorous yet practical framework for empirical software engineering. A substantial side effect is that any uncertainty in the underlying data will be propagated through the statistical model, and its effects on practical significance are made clear. Thus, in combination with cumulative prospect theory, Bayesian analysis supports seamlessly assessing practical significance in an empirical software engineering context, thus potentially clarifying and extending the relevance of research for practitioners.
|
|
| 10. |
- Torkar, Richard, 1971, et al.
(författare)
-
Prediction of faults-slip-through in large software projects: An empirical evaluation
- 2014
-
Ingår i: Software quality journal. - Netherlands : Springer Science and Business Media LLC. - 0963-9314 .- 1573-1367. ; 22:1, s. 51-86
-
Tidskriftsartikel (refereegranskat)abstract
- A large percentage of the cost of rework can be avoided by finding more faults earlier in a software test process. Therefore, determination of which software test phases to focus improvement work on has considerable industrial interest. We evaluate a number of prediction techniques for predicting the number of faults slipping through to unit, function, integration, and system test phases of a large industrial project. The objective is to quantify improvement potential in different test phases by striving toward finding the faults in the right phase. The results show that a range of techniques are found to be useful in predicting the number of faults slipping through to the four test phases; however, the group of search-based techniques (genetic programming, gene expression programming, artificial immune recognition system, and particle swarm optimization–based artificial neural network) consistently give better predictions, having a representation at all of the test phases. Human predictions are consistently better at two of the four test phases. We conclude that the human predictions regarding the number of faults slipping through to various test phases can be well supported by the use of search-based techniques. A combination of human and an automated search mechanism (such as any of the search-based techniques) has the potential to provide improved prediction results.
|
|
