| 1. |
- Bennin, Kwabena Ebo, 1987-, et al.
(författare)
-
Revisiting the Impact of Concept Drift on Just-in-Time Quality Assurance
- 2020
-
Ingår i: Proceedings - 2020 IEEE 20th International Conference on Software Quality, Reliability, and Security, QRS 2020. - : Institute of Electrical and Electronics Engineers Inc.. - 9781728189130 ; , s. 53-59
-
Konferensbidrag (refereegranskat)abstract
- The performance of software defect prediction(SDP) models is known to be dependent on the datasets used for training the models. Evolving data in a dynamic software development environment such as significant refactoring and organizational changes introduces new concept to the prediction model, thus making improved classification performance difficult. In this study, we investigate and assess the existence and impact of concept drift on SDP performances. We empirically asses the prediction performance of five models by conducting cross-version experiments using fifty-five releases of five open-source projects. Prediction performance fluctuated as the training datasets changed over time. Our results indicate that the quality and the reliability of defect prediction models fluctuate over time and that this instability should be considered by software quality teams when using historical datasets. The performance of a static predictor constructed with data from historical versions may degrade over time due to the challenges posed by concept drift. © 2020 IEEE.
|
|
| 2. |
- Kabir, Md Alamgir, et al.
(författare)
-
A Drift Propensity Detection Technique to Improve the Performance for Cross-Version Software Defect Prediction
- 2020
-
Ingår i: Proceedings - 2020 IEEE 44th Annual Computers, Software, and Applications Conference, COMPSAC 2020. - : Institute of Electrical and Electronics Engineers Inc.. - 9781728173030 ; , s. 882-891
-
Konferensbidrag (refereegranskat)abstract
- In cross-version defect prediction (CVDP), historical data is derived from the prior version of the same project to predict defects of the current version. Recent studies in CVDP focus on subset selection to deal with the changes of the data distributions. No prior study has focused on training data arriving in streaming fashion across the versions where the significant differences between versions make the prediction unreliable. We refer to this situation as Drift Propensity (DP). By identifying DP, necessary steps can be taken (e.g., updating or retraining the model) to improve the prediction performance. In this paper, we investigate the chronological defect datasets and identify DP in the datasets. The no-memory data management technique is employed to manage the data distributions and a DP detection technique is proposed. The idea behind the proposed DP detection technique is to monitor the algorithm's error-rate. The DP detector triggers DP, warning, and control flags to take necessary steps. The proposed technique is significantly superior in identifying the distribution differences (p-value < 0.05). The DP's identified in the data distributions achieve large effect sizes (Hedges' g ≥ 0.80) during the pair-wise comparisons. We observe that if the error-rate exponentially increases, it causes DP, resulting in prediction performance deterioration. We thus recommend researches and practitioners to address DP in the chronological datasets. Due to its potential effects in the datasets, the prediction models could be enhanced to get the best results in CVDP. © 2020 IEEE.
|
|
| 3. |
- Kabir, Md Alamdir, et al.
(författare)
-
Assessing the significant impact of concept drift in software defect prediction
- 2019
-
Ingår i: Proceedings - International Computer Software and Applications Conference. - : IEEE Computer Society. - 9781728126074 ; , s. 53-58
-
Konferensbidrag (refereegranskat)abstract
- Concept drift is a known phenomenon in software data analytics. It refers to the changes in the data distribution over time. The performance of analytic and prediction models degrades due to the changes in the data over time. To improve prediction performance, most studies propose that the prediction model be updated when concept drift occurs. In this work, we investigate the existence of concept drift and its associated effects on software defect prediction performance. We adopt the strategy of an empirically proven method DDM (Drift Detection Method) and evaluate its statistical significance using the chi-square test with Yates continuity correction. The objective is to empirically determine the concept drift and to calibrate the base model accordingly. The empirical study indicates that the concept drift occurs in software defect datasets, and its existence subsequently degrades the performance of prediction models. Two types of concept drifts (gradual and sudden drifts) were identified using the chi-square test with Yates continuity correction in the software defect datasets studied. We suggest concept drift should be considered by software quality assurance teams when building prediction models. © 2019 IEEE.
|
|
| 4. |
- Yu, Xiao, et al.
(författare)
-
Improving Ranking-Oriented Defect Prediction Using a Cost-Sensitive Ranking SVM
- 2020
-
Ingår i: IEEE Transactions on Reliability. - : Institute of Electrical and Electronics Engineers Inc.. - 0018-9529 .- 1558-1721. ; 69:1, s. 139-153
-
Tidskriftsartikel (refereegranskat)abstract
- Context: Ranking-oriented defect prediction (RODP) ranks software modules to allocate limited testing resources to each module according to the predicted number of defects. Most RODP methods overlook that ranking a module with more defects incorrectly makes it difficult to successfully find all of the defects in the module due to fewer testing resources being allocated to the module, which results in much higher costs than incorrectly ranking the modules with fewer defects, and the numbers of defects in software modules are highly imbalanced in defective software datasets. Cost-sensitive learning is an effective technique in handling the cost issue and data imbalance problem for software defect prediction. However, the effectiveness of cost-sensitive learning has not been investigated in RODP models. Aims: In this article, we propose a cost-sensitive ranking support vector machine (SVM) (CSRankSVM) algorithm to improve the performance of RODP models. Method: CSRankSVM modifies the loss function of the ranking SVM algorithm by adding two penalty parameters to address both the cost issue and the data imbalance problem. Additionally, the loss function of the CSRankSVM is optimized using a genetic algorithm. Results: The experimental results for 11 project datasets with 41 releases show that CSRankSVM achieves 1.12%-15.68% higher average fault percentile average (FPA) values than the five existing RODP methods (i.e., decision tree regression, linear regression, Bayesian ridge regression, ranking SVM, and learning-to-rank (LTR)) and 1.08%-15.74% higher average FPA values than the four data imbalance learning methods (i.e., random undersampling and a synthetic minority oversampling technique; two data resampling methods; RankBoost, an ensemble learning method; IRSVM, a CSRankSVM method for information retrieval). Conclusion: CSRankSVM is capable of handling the cost issue and data imbalance problem in RODP methods and achieves better performance. Therefore, CSRankSVM is recommended as an effective method for RODP. © 1963-2012 IEEE.
|
|
| 5. |
- Zabardast, Ehsan, et al.
(författare)
-
Further Investigation of the Survivability of Code Technical Debt Items
- 2022
-
Ingår i: JOURNAL OF SOFTWARE-EVOLUTION AND PROCESS. - : John Wiley & Sons. - 2047-7473 .- 2047-7481. ; 34:2
-
Tidskriftsartikel (refereegranskat)abstract
- Context: Technical Debt (TD) discusses the negative impact of sub-optimal decisions to cope with the need-for-speed in software development. Code Technical Debt Items (TDI) are atomic elements of TD that can be observed in code artifacts. Empirical results on open-source systems demonstrated how code-smells, which are just one type of TDIs, are introduced and "survive" during release cycles. However, little is known about whether the results on the survivability of code-smells hold for other types of code TDIs (i.e., bugs and vulnerabilities) and in industrial settings.Goal: Understanding the survivability of code TDIs by conducting an empirical study analyzing two industrial cases and 31 open-source systems from Apache Foundation. Method: We analyzed 144,476 code TDIs (35,372 from the industrial systems) detected by Sonarqube (in 193,196 commits) to assess their survivability using survivability models.Results: In general, code TDIs tend to remain and linger for long periods in open-source systems, whereas they are removed faster in industrial systems. Code TDIs that survive over a certain threshold tend to remain much longer, which confirms previous results. Our results also suggest that bugs tend to be removed faster, while code smells and vulnerabilities tend to survive longer.
|
|
