Search: onr:"swepub:oai:DiVA.org:bth-24154" >
A Method to Assess ...
A Method to Assess and Argue for Practical Significance in Software Engineering
-
- Torkar, Richard, 1971 (author)
- Gothenburg University,Göteborgs universitet,Institutionen för data- och informationsteknik, Software Engineering (GU),Institutionen för data- och informationsteknik, Software Engineering (GU),Chalmers Univ Technol
-
- Furia, Carlo A, 1979 (author)
- USI Univ Svizzera Italiana, CHE,Universita della Svizzera italiana
-
- Feldt, Robert, 1972 (author)
- Chalmers Univ Technol,Chalmers tekniska högskola,Chalmers University of Technology
-
show more...
-
- de Oliveira Neto, Francisco Gomes (author)
- Gothenburg University,Göteborgs universitet,Institutionen för data- och informationsteknik, Software Engineering (GU),Institutionen för data- och informationsteknik, Software Engineering (GU),Chalmers Univ Technol
-
- Gren, Lucas, 1984- (author)
- Gothenburg University,Blekinge Tekniska Högskola,Institutionen för programvaruteknik,Göteborgs universitet,University of Gothenburg,Institutionen för data- och informationsteknik, Software Engineering (GU),Institutionen för data- och informationsteknik, Software Engineering (GU)
-
- Lenberg, Per, 1976 (author)
- Chalmers Univ Technol,Göteborgs universitet,University of Gothenburg
-
- Ernst, Neil A. (author)
- Univ Victoria, CAN,University of Victoria
-
show less...
-
(creator_code:org_t)
- IEEE Computer Society, 2022
- 2022
- English.
-
In: IEEE Transactions on Software Engineering. - : IEEE Computer Society. - 0098-5589 .- 1939-3520. ; 48:6, s. 2053-2065
- Related links:
-
https://doi.org/10.1...
-
show more...
-
https://bth.diva-por... (primary) (Raw object)
-
https://research.cha... (primary) (free)
-
https://urn.kb.se/re...
-
https://doi.org/10.1...
-
https://research.cha...
-
https://gup.ub.gu.se...
-
show less...
Abstract
Subject headings
Close
- A key goal of empirical research in software engineering is to assess practical significance, which answers the question whether the observed effects of some compared treatments show a relevant difference in practice in realistic scenarios. Even though plenty of standard techniques exist to assess statistical significance, connecting it to practical significance is not straightforward or routinely done; indeed, only a few empirical studies in software engineering assess practical significance in a principled and systematic way. In this paper, we argue that Bayesian data analysis provides suitable tools to assess practical significance rigorously. We demonstrate our claims in a case study comparing different test techniques. The case study's data was previously analyzed (Afzal et al., 2015) using standard techniques focusing on statistical significance. Here, we build a multilevel model of the same data, which we fit and validate using Bayesian techniques. Our method is to apply cumulative prospect theory on top of the statistical model to quantitatively connect our statistical analysis output to a practically meaningful context. This is then the basis both for assessing and arguing for practical significance. Our study demonstrates that Bayesian analysis provides a technically rigorous yet practical framework for empirical software engineering. A substantial side effect is that any uncertainty in the underlying data will be propagated through the statistical model, and its effects on practical significance are made clear. Thus, in combination with cumulative prospect theory, Bayesian analysis supports seamlessly assessing practical significance in an empirical software engineering context, thus potentially clarifying and extending the relevance of research for practitioners.
Subject headings
- NATURVETENSKAP -- Data- och informationsvetenskap -- Programvaruteknik (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Software Engineering (hsv//eng)
- NATURVETENSKAP -- Matematik -- Sannolikhetsteori och statistik (hsv//swe)
- NATURAL SCIENCES -- Mathematics -- Probability Theory and Statistics (hsv//eng)
Keyword
- Bayes methods
- Data models
- Software engineering
- Statistical analysis
- Analytical models
- Testing
- Decision making
- Practical significance
- statistical significance
- Bayesian analysis
- empirical software engineering
- Analytical models
- Bayesian analysis
- Data models
- Decision making
- empirical software engineering
- practical significance
- Software engineering
- Statistical analysis
- statistical significance
Publication and Content Type
- ref (subject category)
- art (subject category)
Find in a library
To the university's database
- By the author/editor
-
Torkar, Richard, ...
-
Furia, Carlo A, ...
-
Feldt, Robert, 1 ...
-
de Oliveira Neto ...
-
Gren, Lucas, 198 ...
-
Lenberg, Per, 19 ...
-
show more...
-
Ernst, Neil A.
-
show less...
- About the subject
-
- NATURAL SCIENCES
-
NATURAL SCIENCES
-
and Computer and Inf ...
-
and Software Enginee ...
-
- NATURAL SCIENCES
-
NATURAL SCIENCES
-
and Mathematics
-
and Probability Theo ...
- Articles in the publication
-
IEEE Transaction ...
- By the university
-
Blekinge Institute of Technology
-
Chalmers University of Technology
-
University of Gothenburg