| 1. |
- Berntsson Svensson, Richard, 1978, et al.
(författare)
-
Not all requirements prioritization criteria are equal at all times: A quantitative analysis
- 2024
-
Ingår i: Journal of Systems and Software. - 0164-1212. ; 209
-
Tidskriftsartikel (refereegranskat)abstract
- Requirement prioritization is recognized as an important decision-making activity in requirements engineering. Requirement prioritization is applied to determine which requirements should be implemented and released. In order to prioritize requirements, there are several approaches/techniques/tools that use different requirements prioritization criteria, which are often identified by gut feeling instead of an in-depth analysis of which criteria are most important to use. Therefore, in this study we investigate which requirements prioritization criteria are most important to use in industry when determining which requirements are implemented and released, and if the importance of the criteria change depending on how far a requirement has reached in the development process. We conducted a quantitative study where quantitative data was collected through a case study of one completed project from one software developing company by extracting 32,139 requirements prioritization decisions based on eight requirements prioritization criteria for 11,110 requirements. The results show that not all requirements prioritization criteria are equally important, and this change depending on how far a requirement has reached in the development process. For example, for requirements prioritization decisions before iteration/sprint planning, having high Business value had an impact on the decisions, but after iteration/sprint planning, having high Business value had no impact. Editor's note: Open Science material was validated by the Journal of Systems and Software Open Science Board.
|
|
| 2. |
- Berntsson Svensson, Richard, et al.
(författare)
-
Prioritization of quality requirements : State of practice in eleven companies
- 2011
-
Ingår i: 2011 IEEE 19th International Requirements Engineering Conference, RE 2011; Trento; 29 August 2011 through 2 September 2011. - Trento : IEEE. - 9781457709234 ; , s. 69-78, s. 69-78
-
Konferensbidrag (refereegranskat)abstract
- Requirements prioritization is recognized as an important but challenging activity in software product development. For a product to be successful, it is crucial to find the right balance among competing quality requirements. Although literature offers many methods for requirements prioritization, the research on prioritization of quality requirements is limited. This study identifies how quality requirements are prioritized in practice at 11 successful companies developing software intensive systems. We found that ad-hoc prioritization and priority grouping of requirements are the dominant methods for prioritizing quality requirements. The results also show that it is common to use customer input as criteria for prioritization but absence of any criteria was also common. The results suggests that quality requirements by default have a lower priority than functional requirements, and that they only get attention in the prioritizing process if decision-makers are dedicated to invest specific time and resources on QR prioritization. The results of this study may help future research on quality requirements to focus investigations on industry-relevant issues.
|
|
| 3. |
- Berntsson Svensson, Richard, et al.
(författare)
-
Quality Requirements in Industrial Practice – An Extended Interview Study at Eleven Companies
- 2012
-
Ingår i: IEEE Transactions on Software Engineering. - : IEEE. - 0098-5589 .- 1939-3520. ; 38:4, s. 923-935
-
Tidskriftsartikel (refereegranskat)abstract
- In order to create a successful software product and assure its quality, it is not enough to fulfill the functional requirements, it is also crucial to find the right balance among competing quality requirements (QR). An extended, previosluy piloted, interview study was performed to identify specific challenges associated with the selection, trade-off, and management of QR in industrial practice. Data was collected through semi-structured interviews with eleven product managers and eleven project leaders from eleven software companies. The contribution of this study is fourfold: First, it compares how QR are handled in two cases, companies working in business-to-business markets, and companies that are working in business-to-consumer markets. These two are also compared in terms of impact on the handling of QRs. Second, it compares the perceptions and priorities of QR by product and project management respectively. Third, it includes an examination of the interdependencies among quality requirements perceived as most important by the practitioners. Fourth, it characterizes the selection and management of QR in down-stream development activities.
|
|
| 4. |
- Berntsson Svensson, Richard, 1978, et al.
(författare)
-
Quality Requirements in Industrial Practice - An Extended Interview Study at Eleven Companies
- 2011
-
Ingår i: IEEE Transactions on Software Engineering. - 0098-5589 .- 1939-3520.
-
Tidskriftsartikel (refereegranskat)abstract
- In order to create a successful software product and assure its quality, it is not enough to fulfill the functional requirements, it is also crucial to find the right balance among competing quality requirements (QR). An extended, previously piloted, interview study was performed to identify specific challenges associated with the selection, trade-off, and management of QR in industrial practice. Data was collected through semi-structured interviews with eleven product managers and eleven project leaders from eleven software companies. The contribution of this study is fourfold: First, it compares how QR are handled in two cases, companies working in business-to-business markets, and companies that are working in business-to-consumer markets. These two are also compared in terms of impact on the handling of QRs. Second, it compares the perceptions and priorities of QR by product and project management respectively. Third, it includes an examination of the interdependencies among quality requirements perceived as most important by the practitioners. Fourth, it characterizes the selection and management of QR in down-stream development activities.
|
|
| 5. |
- Berntsson Svensson, Richard, 1978, et al.
(författare)
-
The unfulfilled potential of data-driven decision making in agile software development
- 2019
-
Ingår i: Lecture Notes in Business Information Processing. - Cham : Springer International Publishing. - 1865-1356 .- 1865-1348. - 9783030190330 ; 355, s. 69-85
-
Konferensbidrag (refereegranskat)abstract
- With the general trend towards data-driven decision making (DDDM), organizations are looking for ways to use DDDM to improve their decisions. However, few studies have looked into the practitioners view of DDDM, in particular for agile organizations. In this paper we investigated the experiences of using DDDM, and how data can improve decision making. An emailed questionnaire was sent out to 124 industry practitioners in agile software developing companies, of which 84 answered. The results show that few practitioners indicated a wide-spread use of DDDM in their current decision making practices. The practitioners were more positive to its future use for higher-level and more general decision making, fairly positive to its use for requirements elicitation and prioritization decisions, while being less positive to its future use at the team level. The practitioners do see a lot of potential for DDDM in an agile context; however, currently unfulfilled.
|
|
| 6. |
- Afzal, Wasif, et al.
(författare)
-
A Comparative Evaluation of Using Genetic Programming for Predicting Fault Count Data
- 2008
-
Ingår i: Proceedings - The 3rd International Conference on Software Engineering Advances, ICSEA 2008, Includes ENTISY 2008: International Workshop on Enterprise Information Systems. - : IEEE. - 9780769533728 ; , s. 407-414
-
Konferensbidrag (refereegranskat)abstract
- There have been a number of software reliability growth models (SRGMs) proposed in literature. Due to several reasons, such as violation of models' assumptions and complexity of models, the practitioners face difficulties in knowing which models to apply in practice. This paper presents a comparative evaluation of traditional models and use of genetic programming (GP) for modeling software reliability growth based on weekly fault count data of three different industrial projects. The motivation of using a GP approach is its ability to evolve a model based entirely on prior data without the need of making underlying assumptions. The results show the strengths of using GP for predicting fault count data.
|
|
| 7. |
|
|
| 8. |
- Afzal, Wasif, et al.
(författare)
-
A systematic mapping study on non-functional search-based software testing
- 2008
-
Ingår i: 20th International Conference on Software Engineering and Knowledge Engineering.
-
Konferensbidrag (refereegranskat)abstract
- Automated software test generation has been applied across the spectrum of test case design methods; this includes white-box (structural), black-box (functional), greybox (combination of structural and functional) and nonfunctional testing. In this paper, we undertake a systematic mapping study to present a broad review of primary studies on the application of search-based optimization techniques to non-functional testing. The motivation is to identify the evidence available on the topic and to identify gaps in the application of search-based optimization techniques to different types of non-functional testing. The study is based on a comprehensive set of 35 papers obtained after using a multi-stage selection criteria and are published in workshops, conferences and journals in the time span 1996–2007. We conclude that the search-based software testing community needs to do more and broader studies on nonfunctional search-based software testing (NFSBST) and the results from our systematic map can help direct such efforts.
|
|
| 9. |
- Afzal, Wasif, et al.
(författare)
-
A systematic review of search-based testing for non-functional system properties
- 2009
-
Ingår i: Information and Software Technology. - : Butterworth-Heinemann Newton, MA, USA. - 0950-5849 .- 1873-6025. ; 51:6, s. 957-976
-
Tidskriftsartikel (refereegranskat)abstract
- Search-based software testing is the application of metaheuristic search techniques to generate software tests. The test adequacy criterion is transformed into a fitness function and a set of solutions in the search space are evaluated with respect to the fitness function using a metaheuristic search technique. The application of metaheuristic search techniques for testing is promising due to the fact that exhaustive testing is infeasible considering the size and complexity of software under test. Search-based software testing has been applied across the spectrum of test case design methods; this includes white-box (structural), black-box (functional) and grey-box (combination of structural and functional) testing. In addition, metaheuristic search techniques have also been applied to test non-functional properties. The overall objective of undertaking this systematic review is to examine existing work into non-functional search-based software testing (NFSBST). We are interested in types of non-functional testing targeted using metaheuristic search techniques, different fitness functions used in different types of search-based non-functional testing and challenges in the application of these techniques. The systematic review is based on a comprehensive set of 35 articles obtained after a multi-stage selection process and have been published in the time span 1996-2007. The results of the review show that metaheuristic search techniques have been applied for non-functional testing of execution time, quality of service, security, usability and safety. A variety of metaheuristic search techniques are found to be applicable for non-functional testing including simulated annealing, tabu search, genetic algorithms, ant colony methods, grammatical evolution, genetic programming (and its variants including linear genetic programming) and swarm intelligence methods. The review reports on different fitness functions used to guide the search for each of the categories of execution time, safety, usability, quality of service and security; along with a discussion of possible challenges in the application of metaheuristic search techniques.
|
|
| 10. |
- Afzal, Wasif, et al.
(författare)
-
An experiment on the effectiveness and efficiency of exploratory testing
- 2015
-
Ingår i: Empirical Software Engineering. - : Springer. - 1382-3256 .- 1573-7616. ; 20:3, s. 844-878
-
Tidskriftsartikel (refereegranskat)abstract
- The exploratory testing (ET) approach is commonly applied in industry, but lacks scientific research. The scientific community needs quantitative results on the performance of ET taken from realistic experimental settings. The objective of this paper is to quantify the effectiveness and efficiency of ET vs. testing with documented test cases (test case based testing, TCT). We performed four controlled experiments where a total of 24 practitioners and 46 students performed manual functional testing using ET and TCT. We measured the number of identified defects in the 90-minute testing sessions, the detection difficulty, severity and types of the detected defects, and the number of false defect reports. The results show that ET found a significantly greater number of defects. ET also found significantly more defects of varying levels of difficulty, types and severity levels. However, the two testing approaches did not differ significantly in terms of the number of false defect reports submitted. We conclude that ET was more efficient than TCT in our experiment. ET was also more effective than TCT when detection difficulty, type of defects and severity levels are considered. The two approaches are comparable when it comes to the number of false defect reports submitted.
|
|
| 11. |
- Afzal, Wasif, et al.
(författare)
-
Genetic programming for cross-release fault count predictions in large and complex software projects
- 2010
-
Ingår i: Evolutionary Computation and Optimization Algorithms in Software Engineering. - : IGI Global, Hershey, USA. - 9781615208098
-
Bokkapitel (refereegranskat)abstract
- Software fault prediction can play an important role in ensuring software quality through efficient resource allocation. This could, in turn, reduce the potentially high consequential costs due to faults. Predicting faults might be even more important with the emergence of short-timed and multiple software releases aimed at quick delivery of functionality. Previous research in software fault prediction has indicated that there is a need i) to improve the validity of results by having comparisons among number of data sets from a variety of software, ii) to use appropriate model evaluation measures and iii) to use statistical testing procedures. Moreover, cross-release prediction of faults has not yet achieved sufficient attention in the literature. In an attempt to address these concerns, this paper compares the quantitative and qualitative attributes of 7 traditional and machine-learning techniques for modeling the cross-release prediction of fault count data. The comparison is done using extensive data sets gathered from a total of 7 multi-release open-source and industrial software projects. These software projects together have several years of development and are from diverse application areas, ranging from a web browser to a robotic controller software. Our quantitative analysis suggests that genetic programming (GP) tends to have better consistency in terms of goodness of fit and accuracy across majority of data sets. It also has comparatively less model bias. Qualitatively, ease of configuration and complexity are less strong points for GP even though it shows generality and gives transparent models. Artificial neural networks did not perform as well as expected while linear regression gave average predictions in terms of goodness of fit and accuracy. Support vector machine regression and traditional software reliability growth models performed below average on most of the quantitative evaluation criteria while remained on average for most of the qualitative measures.
|
|
| 12. |
- Afzal, Wasif, et al.
(författare)
-
Incorporating Metrics in an Organizational Test Strategy
- 2008
-
Ingår i: International Conference on Software Testing, Verification and Validation. - : IEEE. - 9780769533889 ; , s. 304-315
-
Konferensbidrag (refereegranskat)abstract
- An organizational level test strategy needs to incorporate metrics to make the testing activities visible and available to process improvements. The majority of testing measurements that are done are based on faults found in the test execution phase. In contrast, this paper investigates metrics to support software test planning and test design processes. We have assembled metrics in these two process types to support management in carrying out evidence-based test process improvements and to incorporate suitable metrics as part of an organization level test strategy. The study is composed of two steps. The first step creates a relevant context by analyzing key phases in the software testing lifecycle, while the second step identifies the attributes of software test planning and test design processes along with metric(s) support for each of the identified attributes.
|
|
| 13. |
- Afzal, Wasif, et al.
(författare)
-
Lessons from applying experimentation in software engineering prediction systems
- 2008
-
Ingår i: Proceedings of The 2nd International workshop on Software Productivity Analysis and Cost Estimation (SPACE'08), Collocated with 15th Asia-Pacific Software Engineering Conference. - Beijing : State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences.
-
Konferensbidrag (refereegranskat)abstract
- Within software engineering prediction systems, experiments are undertaken primarliy to investigate relationships and to measure/compare models' accuracy. This paper discusses our experience and presents useful lessons/guidelines in experimenting with software engineering prediction systems. For this purpose, we use a typical software engineering experimentation process as a baseline. We found that the typical software engineering experimentation process in software engineering is supportive in developing prediction systems and have highlighted issues more central to the domain of software engineering prediction systems.
|
|
| 14. |
- Afzal, Wasif, et al.
(författare)
-
On the application of genetic programming for software engineering predictive modeling : A systematic review
- 2011
-
Ingår i: Expert Systems with Applications. - : Pergamon-Elsevier Science Ltd. - 0957-4174 .- 1873-6793. ; 38:9, s. 11984-11997
-
Forskningsöversikt (refereegranskat)abstract
- The objective of this paper is to investigate the evidence for symbolic regression using genetic programming (GP) being an effective method for prediction and estimation in software engineering, when compared with regression/machine learning models and other comparison groups (including comparisons with different improvements over the standard GP algorithm). We performed a systematic review of literature that compared genetic programming models with comparative techniques based on different independent project variables. A total of 23 primary studies were obtained after searching different information sources in the time span 1995-2008. The results of the review show that symbolic regression using genetic programming has been applied in three domains within software engineering predictive modeling: (i) Software quality classification (eight primary studies). (ii) Software cost/effort/size estimation (seven primary studies). (iii) Software fault prediction/software reliability growth modeling (eight primary studies). While there is evidence in support of using genetic programming for software quality classification, software fault prediction and software reliability growth modeling: the results are inconclusive for software cost/effort/size estimation.
|
|
| 15. |
- Afzal, Wasif, et al.
(författare)
-
On using grey literature and google scholar in systematic literature reviews in software engineering
- 2020
-
Ingår i: IEEE Access. - United States. - 2169-3536. ; 8, s. 36226-36243
-
Tidskriftsartikel (refereegranskat)abstract
- © 2013 IEEE. Context: The inclusion of grey literature (GL) is important to remove publication bias while gathering available evidence regarding a certain topic. The number of systematic literature reviews (SLRs) in Software Engineering (SE) is increasing but we do not know about the extent of GL usage in these SLRs. Moreover, Google Scholar is rapidly becoming a search engine of choice for many researchers but the extent to which it can find the primary studies is not known. Objective: This tertiary study is an attempt to i) measure the usage of GL in SLRs in SE. Furthermore this study proposes strategies for categorizing GL and a quality checklist to use for GL in future SLRs; ii) explore if it is feasible to use only Google Scholar for finding scholarly articles for academic research. Method: We have conducted a systematic mapping study to measure the extent of GL usage in SE SLRs as well as to measure the feasibility of finding primary studies using Google Scholar. Results and conclusions: a) Grey Literature: 76.09% SLRs (105 out of 138) in SE have included one or more GL studies as primary studies. Among total primary studies across all SLRs (6307), 582 are classified as GL, making the frequency of GL citing as 9.23%. The intensity of GL use indicate that each SLR contains 5 primary studies on average (total intensity of GL use being 5.54). The ranking of GL tells us that conference papers are the most used form 43.3% followed by technical reports 28.52%. Universities, research institutes, labs and scientific societies together make up 67.7% of GL used, indicating that these are useful sources for searching GL. We additionally propose strategies for categorizing GL and criteria for evaluating GL quality, which can become a basis for more detailed guidelines for including GL in future SLRs. b) Google Scholar Results: The results show that Google Scholar was able to retrieve 96% of primary studies of these SLRs. Most of the primary studies that were not found using Google Scholar were from grey sources.
|
|
| 16. |
- Afzal, Wasif, et al.
(författare)
-
Prediction of fault count data using genetic programming
- 2008
-
Ingår i: IEEE INMIC 2008: 12th IEEE International Multitopic Conference - Conference Proceedings. - Karachi, Pakistan : IEEE. ; , s. 349-356
-
Konferensbidrag (refereegranskat)abstract
- Software reliability growth modeling helps in deciding project release time and managing project resources. A large number of such models have been presented in the past. Due to the existence of many models, the models' inherent complexity, and their accompanying assumptions; the selection of suitable models becomes a challenging task. This paper presents empirical results of using genetic programming (GP) for modeling software reliability growth based on weekly fault count data of three different industrial projects. The goodness of fit (adaptability) and predictive accuracy of the evolved model is measured using five different measures in an attempt to present a fair evaluation. The results show that the GP evolved model has statistically significant goodness of fit and predictive accuracy.
|
|
| 17. |
- Afzal, Wasif, et al.
(författare)
-
Resampling Methods in Software Quality Classification
- 2012
-
Ingår i: International Journal of Software Engineering and Knowledge Engineering. - Sweden : World Scientific. - 0218-1940. ; 22:2, s. 203-223
-
Tidskriftsartikel (refereegranskat)abstract
- In the presence of a number of algorithms for classification and prediction in software engineering, there is a need to have a systematic way of assessing their performances. The performance assessment is typically done by some form of partitioning or resampling of the original data to alleviate biased estimation. For predictive and classification studies in software engineering, there is a lack of a definitive advice on the most appropriate resampling method to use. This is seen as one of the contributing factors for not being able to draw general conclusions on what modeling technique or set of predictor variables are the most appropriate. Furthermore, the use of a variety of resampling methods make it impossible to perform any formal meta-analysis of the primary study results. Therefore, it is desirable to examine the influence of various resampling methods and to quantify possible differences. Objective and method: This study empirically compares five common resampling methods (hold-out validation, repeated random sub-sampling, 10-fold cross-validation, leave-one-out cross-validation and non-parametric bootstrapping) using 8 publicly available data sets with genetic programming (GP) and multiple linear regression (MLR) as software quality classification approaches. Location of (PF, PD) pairs in the ROC (receiver operating characteristics) space and area under an ROC curve (AUC) are used as accuracy indicators. Results: The results show that in terms of the location of (PF, PD) pairs in the ROC space, bootstrapping results are in the preferred region for 3 of the 8 data sets for GP and for 4 of the 8 data sets for MLR. Based on the AUC measure, there are no significant differences between the different resampling methods using GP and MLR. Conclusion: There can be certain data set properties responsible for insignificant differences between the resampling methods based on AUC. These include imbalanced data sets, insignificant predictor variables and high-dimensional data sets. With the current selection of data sets and classification techniques, bootstrapping is a preferred method based on the location of (PF, PD) pair data in the ROC space. Hold-out validation is not a good choice for comparatively smaller data sets, where leave-one-out cross-validation (LOOCV) performs better. For comparatively larger data sets, 10-fold cross-validation performs better than LOOCV.
|
|
| 18. |
- Afzal, Wasif, et al.
(författare)
-
Search-based prediction of fault count data
- 2009
-
Ingår i: Proceedings - 1st International Symposium on Search Based Software Engineering, SSBSE 2009. - Windsor : IEEE Computer Society. - 9780769536750 ; , s. 35-38
-
Konferensbidrag (refereegranskat)abstract
- Symbolic regression, an application domain of genetic programming (GP), aims to find a function whose output has some desired property, like matching target values of a particular data set. While typical regression involves finding the coefficients of a pre-defined function, symbolic regression finds a general function, with coefficients, fitting the given set of data points. The concepts of symbolic regression using genetic programming can be used to evolve a model for fault count predictions. Such a model has the advantages that the evolution is not dependent on a particular structure of the model and is also independent of any assumptions, which are common in traditional time-domain parametric software reliability growth models. This research aims at applying experiments targeting fault predictions using genetic programming and comparing the results with traditional approaches to compare efficiency gains.
|
|
| 19. |
- Afzal, Wasif, et al.
(författare)
-
Search-based prediction of fault-slip-through in large software projects
- 2010
-
Ingår i: Proceedings - 2nd International Symposium on Search Based Software Engineering, SSBSE 2010. - : IEEE. - 9780769541952 ; , s. 79-88
-
Konferensbidrag (refereegranskat)abstract
- A large percentage of the cost of rework can be avoided by finding more faults earlier in a software testing process. Therefore, determination of which software testing phases to focus improvements work on, has considerable industrial interest. This paper evaluates the use of five different techniques, namely particle swarm optimization based artificial neural networks (PSO-ANN), artificial immune recognition systems (AIRS), gene expression programming (GEP), genetic programming (GP) and multiple regression (MR), for predicting the number of faults slipping through unit, function, integration and system testing phases. The objective is to quantify improvement potential in different testing phases by striving towards finding the right faults in the right phase. We have conducted an empirical study of two large projects from a telecommunication company developing mobile platforms and wireless semiconductors. The results are compared using simple residuals, goodness of fit and absolute relative error measures. They indicate that the four search-based techniques (PSO-ANN, AIRS, GEP, GP) perform better than multiple regression for predicting the fault-slip-through for each of the four testing phases. At the unit and function testing phases, AIRS and PSO-ANN performed better while GP performed better at integration and system testing phases. The study concludes that a variety of search-based techniques are applicable for predicting the improvement potential in different testing phases with GP showing more consistent performance across two of the four test phases.
|
|
| 20. |
- Afzal, Wasif, et al.
(författare)
-
Software Test Process Improvement Approaches: A Systematic Literature Review and an Industrial Case Study
- 2016
-
Ingår i: Journal of Systems and Software. - : Elsevier BV. - 0164-1212. ; 111, s. 1-33
-
Tidskriftsartikel (refereegranskat)abstract
- Software Test Process Improvement (STPI) approaches are frameworks that guide software development organizations to improve their software testing process. We have identified existing STPI approaches and their characteristics (such as completeness of development, availability of information and assessment instruments, and domain limitations of the approaches) using a systematic literature review (SLR). Furthermore, two selected approaches (TPI Next and TMMi) are evaluated with respect to their content and assessment results in industry. As a result of this study, we have identified 18 STPI approaches and their characteristics. A detailed comparison of the content of TPI Next and TMMi is done. We found that many of the STPI approaches do not provide sufficient information or the approaches do not include assessment instruments. This makes it difficult to apply many approaches in industry. Greater similarities were found between TPI Next and TMMi and fewer differences. We conclude that numerous STPI approaches are available but not all are generally applicable for industry. One major difference between available approaches is their model representation. Even though the applied approaches generally show strong similarities, differences in the assessment results arise due to their different model representations.
|
|
| 21. |
- Afzal, Wasif, et al.
(författare)
-
Suitability of Genetic Programming for Software Reliability Growth Modeling
- 2008
-
Ingår i: Proceedings - International Symposium on Computer Science and Its Applications, CSA 2008. - : IEEE. - 9780769534282 ; , s. 114-117
-
Konferensbidrag (refereegranskat)abstract
- Genetic programming (GP) has been found to be effective in finding a model that fits the given data points without making any assumptions about the model structure. This makes GP a reasonable choice for software reliability growth modeling. This paper discusses the suitability of using GP for software reliability growth modeling and highlights the mechanisms that enable GP to progressively search for fitter solutions.
|
|
| 22. |
- Afzal, Wasif, et al.
(författare)
-
Towards benchmarking feature subset selection methods for software fault prediction
- 2016
-
Ingår i: Computational Intelligence and Quantitative Software Engineering. - Berlin, Heidelberg : Springer. - 9783319259642 - 9783319259628 ; , s. 33-58
-
Bokkapitel (refereegranskat)abstract
- Despite the general acceptance that software engineering datasets often contain noisy, irrele- vant or redundant variables, very few benchmark studies of feature subset selection (FSS) methods on real-life data from software projects have been conducted. This paper provides an empirical comparison of state-of-the-art FSS methods: information gain attribute ranking (IG); Relief (RLF); principal com- ponent analysis (PCA); correlation-based feature selection (CFS); consistency-based subset evaluation (CNS); wrapper subset evaluation (WRP); and an evolutionary computation method, genetic programming (GP), on five fault prediction datasets from the PROMISE data repository. For all the datasets, the area under the receiver operating characteristic curve—the AUC value averaged over 10-fold cross- validation runs—was calculated for each FSS method-dataset combination before and after FSS. Two diverse learning algorithms, C4.5 and na ̈ıve Bayes (NB) are used to test the attribute sets given by each FSS method. The results show that although there are no statistically significant differences between the AUC values for the different FSS methods for both C4.5 and NB, a smaller set of FSS methods (IG, RLF, GP) consistently select fewer attributes without degrading classification accuracy. We conclude that in general, FSS is beneficial as it helps improve classification accuracy of NB and C4.5. There is no single best FSS method for all datasets but IG, RLF and GP consistently select fewer attributes without degrading classification accuracy within statistically significant boundaries.
|
|
| 23. |
- Aguayo, Claudio, et al.
(författare)
-
Contextualizando el uso de tecnologías inteligentes móviles para el monitoreo y educación de visitantes [Contextualising the use of smart mobile technologies for visitor monitoring and education]
- 2019
-
Ingår i: XI SOCIETUR [Chilean Society for Tourism Research] Conference 2019, 24-26 April, Punta Arenas, Chile: SOCIETUR. - Santiago : XI SOCIETUR [Chilean Society for Tourism Research].
-
Konferensbidrag (refereegranskat)abstract
- Con una expansión de la recreación al aire libre y el desarrollo del turismo en muchos países hoy en día, el monitoreo de visitantes y la educación pueden considerarse como dos partes integrales de la gestión recreativa contemporánea. El monitoreo de visitantes se refiere a la documentación profesional de actividades recreativas y comportamiento en contextos de áreas recreativas. Esto se ha convertido en una tarea de gestión cada vez más importante para garantizar que los intereses y las experiencias recreativas de los visitantes se incluyan en diversas políticas y estrategias de gestión (Hansen, 2016). La educación de los visitantes, por otro lado, se refiere a cómo las áreas recreativas pueden ofrecer importantes oportunidades de aprendizaje experiencial para la educación relevante basada en el contexto local. La educación dirigida a los visitantes puede complementar y reforzar las experiencias al aire libre, pudiendo ser una forma importante de promover objetivos de sostenibilidad, como la adaptación local al cambio climático (Lück, 2015). Las estrategias de monitoreo y educación de los visitantes se han establecido y utilizado durante mucho tiempo en la gestión de diferentes contextos recreativos en todo el mundo. Sin embargo, el uso de nuevas tecnologías inteligentes para propósitos de monitoreo y educación ha recibido poca atención. Actualmente hay muy poca información disponible sobre el uso potencial de la tecnología móvil, como teléfonos inteligentes y tablets, para fines de monitoreo y educación dentro de contextos de áreas recreativas. La tecnología móvil puede ofrecer muchas opciones novedosas para actividades de monitoreo pasivo y activo de visitantes (Ahas et al., 2010). Del mismo modo, las tecnologías de aprendizaje móvil de hoy en día ofrecen herramientas y posibilidades sin precedentes para complementar y reforzar las experiencias de aprendizaje recreativo al aire libre. Además, éste proceso puede conllevar y una adaptación del aprendizaje a las temáticas relevantes a nivel local, incluyendo elementos culturalmente significativo (Aguayo, 2016). Sin embargo, este tipo de aplicación de las tecnologías móviles inteligentes sigue siendo un área poco explorada de investigación y desarrollo, sobre todo en el área de gestión turística. Desde este proyecto en curso, se propone un marco teórico conceptual inicial para el uso de tecnologías móviles inteligentes para el monitoreo y educación de visitantes en contextos recreacionales. Éste marco se ha desarrollado originalmente a partir de los contextos de turismo de mar costero en Suecia y Nueva Zelanda, encontrándose aún en etapa de conceptualización. En esta sesión se presentará el proyecto, incluyendo indicadores tempranos propuestos por actores en gobernanza y gestión de destinos turísticos de Suecia y Nueva Zelanda que han definido el marco teórico; y en segunda parte se llevará a cabo una breve sesión interactiva de lluvia de ideas para recoger las ideas y propuestas que surjan desde la audiencia en torno a la aplicación del marco teórico en el contexto de la Patagonia y otros destinos, según los presentes.
|
|
| 24. |
- Ali, Nauman bin, et al.
(författare)
-
The impact of a proposal for innovation measurement in the software industry
- 2020
-
Ingår i: International Symposium on Empirical Software Engineering and Measurement. - New York, NY, USA : IEEE Computer Society. - 1949-3789 .- 1949-3770. - 9781450375801
-
Konferensbidrag (refereegranskat)abstract
- Background: Measuring an organization's capability to innovate and assessing its innovation output and performance is a challenging task. Previously, a comprehensive model and a suite of measurements to support this task were proposed. Aims: In the current paper, seven years since the publication of the paper titled Towards innovation measurement in the software industry, we have reflected on the impact of thework. Method:We have mainly relied on quantitative and qualitative analysis of the citations of the paper using an established classification schema. Results: We found that the article has had a significant scientific impact (indicated by the number of citations), i.e., (1) cited in literature from both software engineering and other fields, (2) cited in grey literature and peerreviewed literature, and (3) substantial citations in literature not published in the English language. However, we consider a majority of the citations in the peer-reviewed literature (75 out of 116) as neutral, i.e., they have not used the innovation measurement paper in any substantial way. All in all, 38 out of 116 have used, modified or based their work on the definitions, measurements or the model proposed in the article. This analysis revealed a significant weakness of the citing work, i.e., among the citing papers, we found only two explicit comparisons to the innovation measurement proposal, and we found no papers that identify weaknesses of said proposal. Conclusions: This work highlights the need for being cautious of relying solely on the number of citations for understanding impact, and the need for further improving and supporting the peer-review process to identify unwarranted citations in papers. © 2020 IEEE Computer Society. All rights reserved.
|
|
| 25. |
|
|
| 26. |
|
|
| 27. |
- Darwish, Rashid, 1980, et al.
(författare)
-
A Controlled Experiment on Coverage Maximization of Automated Model-Based Software Test Cases in the Automotive Industry
- 2017
-
Ingår i: Proceedings - 10th IEEE International Conference on Software Testing, Verification and Validation, ICST 2017. ; , s. 546-547
-
Konferensbidrag (refereegranskat)abstract
- © 2017 IEEE. In the automotive industry, as the complexity of electronic control units (ECUs) increase, there is a need for the creation of models that facilitate early tests to ensure functionality, but there is little guidance on how to write these tests in order to achieve maximum coverage. Our prototype CANoe+, which builds on the CANoe and GraphWalker tools, was evaluated against CANoe with regard to coverage maximization of generated test cases from the viewpoint of both software developers and software testers.
|
|
| 28. |
- de Oliveira Neto, Francisco Gomes, et al.
(författare)
-
Full modification coverage through automatic similarity-based test case selection
- 2016
-
Ingår i: Information and Software Technology. - : Elsevier BV. - 0950-5849. ; 80, s. 124-137
-
Tidskriftsartikel (refereegranskat)abstract
- Context: This paper presents the similarity approach for regression testing (SART), where a similarity-based test case selection technique is used in a model-based testing process to provide selection of test cases exercising modified parts of a specification model. Unlike other model-based regression testing techniques, SART relies on similarity analysis among test cases to identify modifications, instead of comparing models, hence reducing the dependency on specific types of model. Objective: To present convincing evidence of the usage of similarity measures for modification-traversing test case selection. Method: We investigate SART in a case study and an experiment. The case study uses artifacts from industry and should be seen as a sanity check of SART, while the experiment focuses on gaining statistical power through the generation of synthetical models in order to provide convincing evidence of SART’s effectiveness. Through posthoc analysis we obtain p-values and effect sizes to observe statistically significant differences between treatments with respect to transition and modification coverage. Results: The case study with industrial artifacts revealed that SART is able to uncover the same number of defects as known similarity-based test case selection techniques. In turn, the experiment shows that SART, unlike the other investigated techniques, presents 100% modification coverage. In addition, all techniques covered a similar percentage of model transitions. Conclusions: In summary, not only does SART provide transition and defect coverage equal to known STCS techniques, but it exceeds greatly in covering modified parts of the specification model, being a suitable candidate for model-based regression testing. Keywords: Regression testing, Test case selection, Model-based testing, Experimental Study
|
|
| 29. |
- de Oliveira Neto, Francisco Gomes, et al.
(författare)
-
Searching for models to evaluate software technology
- 2013
-
Ingår i: 2013 1st International Workshop on Combining Modelling and Search-Based Software Engineering, CMSBSE 2013 - Proceedings. - 9781467362849 ; , s. 12-15
-
Konferensbidrag (refereegranskat)abstract
- Modeling and abstraction is key in all engineering processes and have found extensive use also in software engineering. When developing new methodologies and techniques to support software engineers we want to evaluate them on realistic models. However, this is a challenge since (1) it is hard to get industry to give access to their models, and (2) we need a large number of models to systematically evaluate a technology. This paper proposes that search-based techniques can be used to search for models with desirable properties, which can then be used to systematically evaluate model-based technologies. By targeting properties seen in industrial models we can then get the best of both worlds: models that are similar to models used in industry but in quantities that allow extensive experimentation. To exemplify our ideas we consider a specific case in which a model generator is used to create models to test a regression test optimization technique. © 2013 IEEE.
|
|
| 30. |
- Dobslaw, Felix, 1983, et al.
(författare)
-
Estimating Return on Investment for GUI Test Automation Frameworks
- 2019
-
Ingår i: Proceedings - International Symposium on Software Reliability Engineering, ISSRE. - 1071-9458. - 9781728149813 ; 2019-October
-
Konferensbidrag (refereegranskat)abstract
- Automated graphical user interface (GUI) tests can reduce manual testing activities and increase test frequency. This motivates the conversion of manual test cases into automated GUI tests. However, it is not clear whether such automation is cost-effective given that GUI automation scripts add to the code base and demand maintenance as a system evolves. In this paper, we introduce a method for estimating maintenance cost and Return on Investment (ROI) for Automated GUI Testing (AGT). The method utilizes the existing source code change history and has the potential to be used for the evaluation of other testing or quality assurance automation technologies. We evaluate the method for a real-world, industrial software system and compare two fundamentally different AGT frameworks, namely Selenium and EyeAutomate, to estimate and compare their ROI. We also report on their defect-finding capabilities and usability. The quantitative data is complemented by interviews with employees at the company the study has been conducted at. The method was successfully applied, and estimated maintenance cost and ROI for both frameworks are reported. Overall, the study supports earlier results showing that implementation time is the leading cost for introducing AGT. The findings further suggest that, while EyeAutomate tests are significantly faster to implement, Selenium tests require more of a programming background but less maintenance.
|
|
| 31. |
- Dobslaw, F., et al.
(författare)
-
Estimating Return on Investment for GUI Test Automation Frameworks
- 2019
-
Ingår i: 2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE). - : IEEE. - 9781728149820
-
Konferensbidrag (refereegranskat)abstract
- Automated graphical user interface (GUI) tests can reduce manual testing activities and increase test frequency. This motivates the conversion of manual test cases into automated GUI tests. However, it is not clear whether such automation is cost-effective given that GUI automation scripts add to the code base and demand maintenance as a system evolves. In this paper, we introduce a method for estimating maintenance cost and Return on Investment (ROI) for Automated GUI Testing (AGT). The method utilizes the existing source code change history and has the potential to be used for the evaluation of other testing or quality assurance automation technologies. We evaluate the method for a real-world, industrial software system and compare two fundamentally different AGT frameworks, namely Selenium and EyeAutomate, to estimate and compare their ROI. We also report on their defect-finding capabilities and usability. The quantitative data is complemented by interviews with employees at the company the study has been conducted at. The method was successfully applied, and estimated maintenance cost and ROI for both frameworks are reported. Overall, the study supports earlier results showing that implementation time is the leading cost for introducing AGT. The findings further suggest that, while EyeAutomate tests are significantly faster to implement, Selenium tests require more of a programming background but less maintenance.
|
|
| 32. |
|
|
| 33. |
- Edison, Henry, et al.
(författare)
-
Towards innovation measurement in the software industry
- 2013
-
Ingår i: Journal of Systems and Software. - : Elsevier. - 0164-1212. ; 86:5, s. 1390-1407
-
Tidskriftsartikel (refereegranskat)abstract
- In today's highly competitive business environments with shortened product and technology life cycle, it is critical for software industry to continuously innovate. This goal can be achieved by developing a better understanding and control of the activities and determinants of innovation. Innovation measurement initiatives assess innovation capability, output and performance to help develop such an understanding. This study explores various aspects relevant to innovation measurement ranging from definitions, measurement frameworks and metrics that have been proposed in literature and used in practice. A systematic literature review followed by an online questionnaire and interviews with practitioners and academics were employed to identify a comprehensive definition of innovation that can be used in software industry. The metrics for the evaluation of determinants, inputs, outputs and performance were also aggregated and categorised. Based on these findings, a conceptual model of the key measurable elements of innovation was constructed from the findings of the systematic review. The model was further refined after feedback from academia and industry through interviews.
|
|
| 34. |
- Engström, Emelie, et al.
(författare)
-
Indirect effects in evidential assessment: A case study on regression test technology adoption
- 2012
-
Ingår i: 2nd International Workshop on Evidential Assessment of Software Technologies, EAST 2012. Lund, 22 September 2012. - New York, NY, USA : ACM. - 9781450315098 ; , s. 15-20
-
Konferensbidrag (refereegranskat)abstract
- Background: There is a need for effcient regression testing in most software development organizations. Often the proposed solutions involve automation. However, despite this being a well researched area, research results are rarely applied in industrial practice. Aim: In this paper we aim to bridge the gap between research and practice by providing examples of how evidence-based regression testing approaches can be adopted in industry. We also discuss challenges for the research community. Method: An industrial case study was carried out to evaluate the possibility to improve regression testing at Sony Ericsson Mobile Communications. We analyse the procedure undertaken based on frameworks from the evidence based software engineering, EBSE, paradigm (with a focus on the evidence) and automation literature (with a focus on the practical effects). Results: Our results pinpoint the need for systematic approaches when introducing a new tool. Practitioners and researchers need congruent guidelines supporting the appraisal of both the evidence base and the pragmatic effects, both direct but also indirect, of the changes. This is illustrated by the introduction of the automation perspective.
|
|
| 35. |
- Felderer, Michael, 1978-, et al.
(författare)
-
A testability analysis framework for non-functional properties
- 2018
-
Ingår i: 2018 IEEE 11TH INTERNATIONAL CONFERENCE ON SOFTWARE TESTING, VERIFICATION AND VALIDATION WORKSHOPS (ICSTW). - : Institute of Electrical and Electronics Engineers Inc.. - 9781538663523 ; , s. 54-58
-
Konferensbidrag (refereegranskat)abstract
- This paper presents background, the basic steps and an example for a testability analysis framework for non-functional properties.
|
|
| 36. |
- Feldt, Robert, et al.
(författare)
-
Challenges with Software Verification and Validation Activities in the Space Industry
- 2010
-
Konferensbidrag (refereegranskat)abstract
- Developing software for high-dependable space applications and systems is a formidable task. With new political and market pressures on the space industry to deliver more software at a lower cost, optimization of their methods and standards need to be investigated. The industry has to follow standards that strictly set quality goals and prescribes engineering processes and methods to fulfill them. The overall goal of this study is to evaluate if current use of the standards from the European Cooperation for Space Standardization (ECSS) is cost efficient and if there are ways to make the process leaner while still maintaining quality and to analyze if their verification and validation (V&V) activities can be optimized. This paper presents results from two industrial case studies of companies in the European space industry that are following ECSS standards in various V&V activities. The case studies reported here focus on how ECSS standards are used by the companies, how that affects their processes and, in the end, how their V&V activities can be further optimized.
|
|
| 37. |
- Feldt, Robert, et al.
(författare)
-
Links between the personalities, views and attitudes of software engineers
- 2010
-
Ingår i: Information and Software Technology. - : Elsevier BV. - 0950-5849 .- 1873-6025. ; 52:6, s. 611-624
-
Tidskriftsartikel (refereegranskat)abstract
- Context:Successful software development and management depends not only on the technologies, methods and processes employed but also on the judgments and decisions of the humans involved. These, in turn, are affected by the basic views and attitudes of the individual engineers.Objective:The objective of this paper is to establish if these views and attitudes can be linked to the personalities of software engineers.Methods:We summarize the literature on personality and software engineering and then describe an empirical study on 47 professional engineers in ten different Swedish software development companies. The study evaluated the personalities of these engineers via the IPIP 50-item five-factor personality test and prompted them on their attitudes towards and basic views on their professional activities.Results:We present extensive statistical analyses of their responses to show that there are multiple, significant associations between personality factors and software engineering attitudes. The tested individuals are more homogeneous in personality than a larger sample of individuals from the general population.Conclusion:Taken together, the methodology and personality test we propose and the associated statistical analyses can help find and quantify relations between complex factors in software engineering projects in both research and practice.
|
|
| 38. |
- Feldt, Robert, et al.
(författare)
-
Optimizing Verification and Validation Activities for Software in the Space Industry
- 2010
-
Ingår i: European Space Agency, (Special Publication) ESA SP. Conference on DAta Systems in Aerospace, DASIA 2010; Budapest; 1-4 June 2010. - Budapest : European Space Agency. - 0379-6566. - 9789290922469 ; 682 SP, s. 309-313
-
Konferensbidrag (refereegranskat)abstract
- Software for space applications has special requirements in terms of reliability and dependability and the verification & validation activities (VAs) of these systems often account for more than 50% of the develop- ment effort. The industry is also faced with political and market pressure to deliver software faster and cheaper. Thus new ways are needed to optimize these activities so that high quality can be retained even with reduced costs and effort. Here we present a framework for the management and optimization of verification & validation activities (VAMOS). An initial evaluation of the framework based on historical data as well as data extracted with a new tool has been done and are described briefly.
|
|
| 39. |
- Feldt, Robert, et al.
(författare)
-
Searching for Cognitively Diverse Tests : Towards Universal Test Diversity Metrics
- 2008
-
Ingår i: International Conference on Software Testing, Verification and Validation. - Lillehammer, Norge : IEEE. - 0769533884 ; , s. 178-186
-
Konferensbidrag (refereegranskat)abstract
- Search-based software testing (SBST) has shown a potential to decrease cost and increase quality of testingrelated software development activities. Research in SBST has so far mainly focused on the search for isolated tests that are optimal according to a fitness function that guides the search. In this paper we make the case for fitness functions that measure test fitness in relation to existing or previously found tests; a test is good if it is diverse from other tests. We present a model for test variability and propose the use of a theoretically optimal diversity metric at variation points in the model. We then describe how to apply a practically useful approximation to the theoretically optimal metric. The metric is simple and powerful and can be adapted to a multitude of different test diversity measurement scenarios. We present initial results from an experiment to compare how similar to human subjects, the metric can cluster a set of test cases. To carry out the experiment we have extended an existing framework for test automation in an object-oriented, dynamic programming language.
|
|
| 40. |
|
|
| 41. |
|
|
| 42. |
- Feldt, Robert, 1972, et al.
(författare)
-
Ways of applying artificial intelligence in software engineering
- 2018
-
Ingår i: Proceedings - International Conference on Software Engineering. - New York : IEEE. - 0270-5257. ; Part F137725, s. 35-41
-
Konferensbidrag (refereegranskat)abstract
- As Artificial Intelligence (AI) techniques become more powerful and easier to use they are increasingly deployed as key components of modern software systems. While this enables new functionality and often allows better adaptation to user needs it also creates additional problems for software engineers and exposes companies to new risks. Some work has been done to better understand the interaction between Software Engineering and AI but we lack methods to classify ways of applying AI in software systems and to analyse and understand the risks this poses. Only by doing so can we devise tools and solutions to help mitigate them. This paper presents the AI in SE Application Levels (AI-SEAL) taxonomy that categorises applications according to their point of application, the type of AI technology used and the automation level allowed. We show the usefulness of this taxonomy by classifying 15 papers from previous editions of the RAISE workshop. Results show that the taxonomy allows classification of distinct AI applications and provides insights concerning the risks associated with them. We argue that this will be important for companies in deciding how to apply AI in their software applications and to create strategies for its use.
|
|
| 43. |
|
|
| 44. |
- Furia, Carlo A., et al.
(författare)
-
Applying Bayesian analysis guidelines to empirical software engineering data
- 2021
-
Ingår i: ACM Transactions on Software Engineering and Methodology. - : Association for Computing Machinery (ACM). - 1049-331X .- 1557-7392. ; 31:3
-
Tidskriftsartikel (refereegranskat)abstract
- Statistical analysis is the tool of choice to turn data into information and then information into empirical knowledge. However, the process that goes from data to knowledge is long, uncertain, and riddled with pitfalls. To be valid, it should be supported by detailed, rigorous guidelines that help ferret out issues with the data or model and lead to qualified results that strike a reasonable balance between generality and practical relevance. Such guidelines are being developed by statisticians to support the latest techniques for Bayesian data analysis. In this article, we frame these guidelines in a way that is apt to empirical research in software engineering. To demonstrate the guidelines in practice, we apply them to reanalyze a GitHub dataset about code quality in different programming languages. The dataset’s original analysis [Ray et al. 55] and a critical reanalysis [Berger et al. 6] have attracted considerable attention—in no small part because they target a topic (the impact of different programming languages) on which strong opinions abound. The goals of our reanalysis are largely orthogonal to this previous work, as we are concerned with demonstrating, on data in an interesting domain, how to build a principled Bayesian data analysis and to showcase its benefits. In the process, we will also shed light on some critical aspects of the analyzed data and of the relationship between programming languages and code quality—such as the impact of project-specific characteristics other than the used programming language. The high-level conclusions of our exercise will be that Bayesian statistical techniques can be applied to analyze software engineering data in a way that is principled, flexible, and leads to convincing results that inform the state-of-the-art while highlighting the boundaries of its validity. The guidelines can support building solid statistical analyses and connecting their results. Thus, they can help buttress continued progress in empirical software engineering research.
|
|
| 45. |
- Furia, Carlo A, 1979, et al.
(författare)
-
Applying Bayesian Analysis Guidelines to Empirical Software Engineering Data: The Case of Programming Languages and Code Quality
- 2022
-
Ingår i: ACM Transactions on Software Engineering and Methodology. - : Association for Computing Machinery (ACM). - 1049-331X .- 1557-7392. ; 31:3
-
Tidskriftsartikel (refereegranskat)abstract
- Statistical analysis is the tool of choice to turn data into information and then information into empirical knowledge. However, the process that goes from data to knowledge is long, uncertain, and riddled with pitfalls. To be valid, it should be supported by detailed, rigorous guidelines that help ferret out issues with the data or model and lead to qualified results that strike a reasonable balance between generality and practical relevance. Such guidelines are being developed by statisticians to support the latest techniques for Bayesian data analysis. In this article, we frame these guidelines in a way that is apt to empirical research in software engineering.To demonstrate the guidelines in practice, we apply them to reanalyze a GitHub dataset about code quality in different programming languages. The dataset's original analysis [Ray et al. 55] and a critical reanalysis [Berger et al. 6] have attracted considerable attention-in no small part because they target a topic (the impact of different programming languages) on which strong opinions abound. The goals of our reanalysis are largely orthogonal to this previous work, as we are concerned with demonstrating, on data in an interesting domain, how to build a principled Bayesian data analysis and to showcase its benefits. In the process, we will also shed light on some critical aspects of the analyzed data and of the relationship between programming languages and code quality-such as the impact of project-specific characteristics other than the used programming language.The high-level conclusions of our exercise will be that Bayesian statistical techniques can be applied to analyze software engineering data in a way that is principled, flexible, and leads to convincing results that inform the state-of-The-Art while highlighting the boundaries of its validity. The guidelines can support building solid statistical analyses and connecting their results. Thus, they can help buttress continued progress in empirical software engineering research.
|
|
| 46. |
- Furia, Carlo A, 1979, et al.
(författare)
-
Bayesian Data Analysis in Empirical Software Engineering Research
- 2021
-
Ingår i: IEEE Transactions on Software Engineering. - 0098-5589 .- 1939-3520. ; 47:9, s. 1786-1810
-
Tidskriftsartikel (refereegranskat)abstract
- IEEE Statistics comes in two main flavors: frequentist and Bayesian. For historical and technical reasons, frequentist statistics have traditionally dominated empirical data analysis, and certainly remain prevalent in empirical software engineering. This situation is unfortunate because frequentist statistics suffer from a number of shortcomings---such as lack of flexibility and results that are unintuitive and hard to interpret---that curtail their effectiveness when dealing with the heterogeneous data that is increasingly available for empirical analysis of software engineering practice. In this paper, we pinpoint these shortcomings, and present Bayesian data analysis techniques that provide tangible benefits---as they can provide clearer results that are simultaneously robust and nuanced. After a short, high-level introduction to the basic tools of Bayesian statistics, we present the reanalysis of two empirical studies on the effectiveness of automatically generated tests and the performance of programming languages, respectively. By contrasting the original frequentist analyses with our new Bayesian analyses, we demonstrate the concrete advantages of the latter. To conclude we advocate a more prominent role for Bayesian statistical techniques in empirical software engineering research and practice.
|
|
| 47. |
- Furia, Carlo A, 1979, et al.
(författare)
-
Towards causal analysis of empirical software engineering data: The impact of programming languages on coding competitions
- 2024
-
Ingår i: ACM Transactions on Software Engineering and Methodology. - 1049-331X .- 1557-7392. ; 33:1, s. 1-35
-
Tidskriftsartikel (refereegranskat)abstract
- There is abundant observational data in the software engineering domain, whereas running large-scale controlled experiments is often practically impossible. Thus, most empirical studies can only report statistical correlations—instead of potentially more insightful and robust causal relations. To support analyzing purely observational data for causal relations, and to assess any differences between purely predictive and causal models of the same data, this paper discusses some novel techniques based on structural causal models (such as directed acyclic graphs of causal Bayesian networks). Using these techniques, one can rigorously express, and partially validate, causal hypotheses; and then use the causal information to guide the construction of a statistical model that captures genuine causal relations—such that correlation does imply causation. We apply these ideas to analyzing public data about programmer performance in Code Jam, a large world- wide coding contest organized by Google every year. Specifically, we look at the impact of different program- ming languages on a participant’s performance in the contest. While the overall effect associated with programming languages is weak compared to other variables—regardless of whether we consider correlational or causal links—we found considerable differences between a purely associational and a causal analysis of the very same data. The takeaway message is that even an imperfect causal analysis of observational data can help answer the salient research questions more precisely and more robustly than with just purely predictive techniques— where genuine causal effects may be confounded.
|
|
| 48. |
- Ghazi, Ahmad Nauman, et al.
(författare)
-
Information sources and their importance to prioritize test cases in heterogeneous systems context
- 2014
-
Ingår i: Communications in Computer and Information Science. - Berlin, Heidelberg : Springer. - 1865-0929. - 9783662438954 - 9783662438961 ; 425, s. 86-98
-
Konferensbidrag (refereegranskat)abstract
- Context: Testing techniques proposed in the literature rely on various sources of information for test case selection (e.g., require- ments, source code, system structure, etc.). The challenge of test selection is amplified in the context of heterogeneous systems, where it is unknown which information/data sources are most important. Contribution: (1) Achieve in-depth understanding of test processes in heterogeneous systems; (2) Elicit information sources for test selection in the context of heterogeneous systems. (3) Capture the relative importance of the identified information sources. Method: Case study research is used for the elicitation and understanding of which information sources are relevant for test case privatization, followed by an exploratory survey capturing the relative importance of information sources for testing heterogeneous systems. Results: We classified different information sources that play a vital role in the test selection process, and found that their importance differs largely for the different test levels observed in heterogeneous testing. However, overall all sources were considered essential in test selection for heterogeneous systems. Conclusion: Heterogeneous system testing requires solutions that take all information sources into account when suggesting test cases for selection. Such approaches need to be developed and compared with existing solutions.
|
|
| 49. |
- Gomes, Francisco, 1987, et al.
(författare)
-
Evolution of statistical analysis in empirical software engineering research : Current state and steps forward
- 2019
-
Ingår i: Journal of Systems and Software. - : Elsevier Inc.. - 0164-1212 .- 1873-1228. ; 156, s. 246-267
-
Tidskriftsartikel (refereegranskat)abstract
- Software engineering research is evolving and papers are increasingly based on empirical data from a multitude of sources, using statistical tests to determine if and to what degree empirical evidence supports their hypotheses. To investigate the practices and trends of statistical analysis in empirical software engineering (ESE), this paper presents a review of a large pool of papers from top-ranked software engineering journals. First, we manually reviewed 161 papers and in the second phase of our method, we conducted a more extensive semi-automatic classification of papers spanning the years 2001–2015 and 5196 papers. Results from both review steps was used to: i) identify and analyse the predominant practices in ESE (e.g., using t-test or ANOVA), as well as relevant trends in usage of specific statistical methods (e.g., nonparametric tests and effect size measures) and, ii) develop a conceptual model for a statistical analysis workflow with suggestions on how to apply different statistical methods as well as guidelines to avoid pitfalls. Lastly, we confirm existing claims that current ESE practices lack a standard to report practical significance of results. We illustrate how practical significance can be discussed in terms of both the statistical analysis and in the practitioner's context. © 2019 Elsevier Inc.
|
|
| 50. |
- Gorschek, Tony, et al.
(författare)
-
Introduction of a Process Maturity Model for Market-driven Product Management and Requirements engineering
- 2012
-
Ingår i: Journal of Software Maintenance and Evolution. - : John Wiley and Sons. - 1532-060X .- 1532-0618. ; 24:1, s. 83-113
-
Tidskriftsartikel (refereegranskat)abstract
- The area of software product development of software intensive products has received much attention, especially in the area of requirements engineering and product management. Many companies are faced with new challenges when operating in an environment where potential requirements number in thousands or even tens of thousands, and where a product does not have a customer, but any number of customers or markets. The development organization carries not only all the costs of development, but also takes all the risks. In this environment traditional bespoke requirements engineering, together with traditional process assessment and improvement models fall short as they do not address the unique challenges of a market-driven environment. This paper introduces the Market-driven Requirements Engineering Process Model, aimed at enabling process improvement and process assurance for organizations faced with these new challenges. The model is also validated in the industry through three case studies where the model is used for process assessment and improvement suggestion. Initial results show that the model is appropriate for process improvement for organizations operating in a market-driven environment. In addition, the model was designed to be light weight in terms of low cost and thus adapted not only for large organizations but suitable for small and medium enterprises as well.
|
|
