Revisiting the Impact of Concept Drift on Just-in-Time Quality Assurance

• The performance of software defect prediction(SDP) models is known to be dependent on the datasets used for training the models. Evolving data in a dynamic software development environment such as significant refactoring and organizational changes introduces new concept to the prediction model, thus making improved classification performance difficult. In this study, we investigate and assess the existence and impact of concept drift on SDP performances. We empirically asses the prediction performance of five models by conducting cross-version experiments using fifty-five releases of five open-source projects. Prediction performance fluctuated as the training datasets changed over time. Our results indicate that the quality and the reliability of defect prediction models fluctuate over time and that this instability should be considered by software quality teams when using historical datasets. The performance of a static predictor constructed with data from historical versions may degrade over time due to the challenges posed by concept drift. © 2020 IEEE.

Subject headings

NATURVETENSKAP  -- Data- och informationsvetenskap -- Programvaruteknik (hsv//swe)

NATURAL SCIENCES  -- Computer and Information Sciences -- Software Engineering (hsv//eng)

Keyword

Concept drift

Defect prediction

Just-in-Time Quality assurance

Computer software selection and evaluation

Defects

Forecasting

Open source software

Software design

Software quality

Software reliability

Classification performance

Defect prediction models

Dynamic softwares

Open source projects

Organizational change

Prediction performance

Software defect prediction

Training data sets

Predictive analytics

Publication and Content Type

ref (subject category)

kon (subject category)