| 1. |
- Al Sabbagh, Khaled, 1987, et al.
(författare)
-
A classification of code changes and test types dependencies for improving machine learning based test selection
- 2021
-
Ingår i: SIGPLAN Notices (ACM Special Interest Group on Programming Languages). - New York, NY, USA : ACM. - 0730-8566. ; , s. 40-49
-
Konferensbidrag (refereegranskat)abstract
- Machine learning has been increasingly used to solve various software engineering tasks. One example of their usage is in regression testing, where a classifier is built using historical code commits to predict which test cases require execution. In this paper, we address the problem of how to link specific code commits to test types to improve the predictive performance of learning models in improving regression testing. We design a dependency taxonomy of the content of committed code and the type of a test case. The taxonomy focuses on two types of code commits: changing memory management and algorithm complexity. We reviewed the literature, surveyed experienced testers from three Swedish-based software companies, and conducted a workshop to develop the taxonomy. The derived taxonomy shows that memory management code should be tested with tests related to performance, load, soak, stress, volume, and capacity; the complexity changes should be tested with the same dedicated tests and maintainability tests. We conclude that this taxonomy can improve the effectiveness of building learning models for regression testing.
|
|
| 2. |
- Al Sabbagh, Khaled, 1987, et al.
(författare)
-
Improving Data Quality for Regression Test Selection by Reducing Annotation Noise
- 2020
-
Ingår i: Proceedings - 46th Euromicro Conference on Software Engineering and Advanced Applications, SEAA 2020. ; , s. 191-194
-
Konferensbidrag (refereegranskat)abstract
- Big data and machine learning models have been increasingly used to support software engineering processes and practices. One example is the use of machine learning models to improve test case selection in continuous integration. However, one of the challenges in building such models is the identification and reduction of noise that often comes in large data. In this paper, we present a noise reduction approach that deals with the problem of contradictory training entries. We empirically evaluate the effectiveness of the approach in the context of selective regression testing. For this purpose, we use a curated training set as input to a tree-based machine learning ensemble and compare the classification precision, recall, and f-score against a non-curated set. Our study shows that using the noise reduction approach on the training instances gives better results in prediction with an improvement of 37% on precision, 70% on recall, and 59% on f-score.
|
|
| 3. |
- Al Sabbagh, Khaled, 1987, et al.
(författare)
-
Improving Software Regression Testing Using a Machine Learning-Based Method for Test Type Selection
- 2022
-
Ingår i: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). - Cham : Springer International Publishing. - 1611-3349 .- 0302-9743. ; 13709 LNCS, s. 480-496
-
Konferensbidrag (refereegranskat)abstract
- Since only a limited time is available for performing software regression testing, a subset of crucial test cases from the test suites has to be selected for execution. In this paper, we introduce a method that uses the relation between types of code changes and regression tests to select test types that require execution. We work closely with a large power supply company to develop and evaluate the method and measure the total regression testing time taken by our method and its effectiveness in selecting the most relevant test types. The results show that the method reduces the total regression time by an average of 18,33% when compared with the approach used by our industrial partner. The results also show that using a medium window size in the method configuration results in an improved recall rate from 61,11% to 83,33%, but not in considerable time reduction of testing. We conclude that our method can potentially be used to steer the testing effort at software development companies by guiding testers into which regression test types are essential for execution.
|
|
| 4. |
- Al-Sabbagh, Khaled, 1987, et al.
(författare)
-
Mobile language learning applications for Arabic speaking migrants - A usability perspective
- 2018
-
Ingår i: Language Learning in Higher Education. - : Walter de Gruyter GmbH. - 2191-611X .- 2191-6128. ; 9:1, s. 71-95
-
Tidskriftsartikel (refereegranskat)abstract
- Usability testing with mobile applications (apps) plays an important role in determining the ease of use of an app as well as in informing their design and development. The purpose of this research is to investigate the usability of three language learning apps with a sample of Arabic speaking migrants who have recently arrived to Sweden. In addition, We surveyed available apps for the target group on both Android and Apple market stores and used a framework for categorizing language learning apps to guide the design and development of our own app. The outcomes show that in order for users to engage in an app, there are a number of obstacles that need to be overcome to make an app motivating enough to use, such as lack of variation in functionality, instructions and feedback. Our app, Minclusion, was developed from the input obtained from the usability studies.
|
|
| 5. |
- Al Sabbagh, Khaled, 1987, et al.
(författare)
-
Predicting build outcomes in continuous integration using textual analysis of source code commits
- 2022
-
Ingår i: PROMISE 2022 - Proceedings of the 18th International Conference on Predictive Models and Data Analytics in Software Engineering, co-located with ESEC/FSE 2022. - New York, NY, USA : ACM. ; , s. 42-51
-
Konferensbidrag (refereegranskat)abstract
- Machine learning has been increasingly used to solve various software engineering tasks. One example of its usage is to predict the outcome of builds in continuous integration, where a classifier is built to predict whether new code commits will successfully compile. The aim of this study is to investigate the effectiveness of fifteen software metrics in building a classifier for build outcome prediction. Particularly, we implemented an experiment wherein we compared the effectiveness of a line-level metric and fourteen other traditional software metrics on 49,040 build records that belong to 117 Java projects. We achieved an average precision of 91% and recall of 80% when using the line-level metric for training, compared to 90% precision and 76% recall for the next best traditional software metric. In contrast, using file-level metrics was found to yield a higher predictive quality (average MCC for the best software metric= 68%) than the line-level metric (average MCC= 16%) for the failed builds. We conclude that file-level metrics are better predictors of build outcomes for the failed builds, whereas the line-level metric is a slightly better predictor of passed builds.
|
|
| 6. |
- Al Sabbagh, Khaled, 1987, et al.
(författare)
-
Predicting Test Case Verdicts Using TextualAnalysis of Commited Code Churns
- 2019
-
Ingår i: CEUR Workshop Proceedings. - 1613-0073. ; 2476, s. 138-153
-
Konferensbidrag (refereegranskat)abstract
- Background: Continuous Integration (CI) is an agile software development practice that involves producing several clean builds of the software per day. The creation of these builds involve running excessive executions of automated tests, which is hampered by high hardware cost and reduced development velocity. Goal: The goal of our research is to develop a method that reduces the number of executed test cases at each CI cycle.Method: We adopt a design research approach with an infrastructure provider company to develop a method that exploits Ma-chine Learning (ML) to predict test case verdicts for committed sourcecode. We train five different ML models on two data sets and evaluate their performance using two simple retrieval measures: precision and recall. Results: While the results from training the ML models on the first data-set of test executions revealed low performance, the curated data-set for training showed an improvement on performance with respect to precision and recall. Conclusion: Our results indicate that the method is applicable when training the ML model on churns of small sizes
|
|
| 7. |
- Al Sabbagh, Khaled, 1987, et al.
(författare)
-
Selective Regression Testing based on Big Data: Comparing Feature Extraction Techniques
- 2020
-
Ingår i: IEEE Software. - 1937-4194 .- 0740-7459. ; , s. 322-329
-
Konferensbidrag (refereegranskat)abstract
- Regression testing is a necessary activity in continuous integration (CI) since it provides confidence that modified parts of the system are correct at each integration cycle. CI provides large volumes of data which can be used to support regression testing activities. By using machine learning, patterns about faulty changes in the modified program can be induced, allowing test orchestrators to make inferences about test cases that need to be executed at each CI cycle. However, one challenge in using learning models lies in finding a suitable way for characterizing source code changes and preserving important information. In this paper, we empirically evaluate the effect of three feature extraction algorithms on the performance of an existing ML-based selective regression testing technique. We designed and performed an experiment to empirically investigate the effect of Bag of Words (BoW), Word Embeddings (WE), and content-based feature extraction (CBF). We used stratified cross validation on the space of features generated by the three FE techniques and evaluated the performance of three machine learning models using the precision and recall metrics. The results from this experiment showed a significant difference between the models' precision and recall scores, suggesting that the BoW-fed model outperforms the other two models with respect to precision, whereas a CBF-fed model outperforms the rest with respect to recall.
|
|
| 8. |
- Al Sabbagh, Khaled, 1987, et al.
(författare)
-
The connections between group maturity, software development velocity, and planning effectiveness
- 2018
-
Ingår i: Journal of Software-Evolution and Process. - : Wiley. - 2047-7473 .- 2047-7481. ; 30:1
-
Tidskriftsartikel (refereegranskat)abstract
- Empirical evidence regarding the connection between group development (maturity) and the success of software development teams is lacking. The purpose of this research is to gain a qualitative and quantitative understanding of how velocity and planning effectiveness of software teams connect to a group development model. The Group Development Questionnaire was given to 19 software developers from 4 work groups to assess their group development maturity. The work groups' responses to the survey were checked for correlation with development velocity and planning effectiveness. Semistructured interviews were conducted with 16 individuals from the same 4 work groups to explore issues about their group maturity and to validate the responses of the interviewees in the Group Development Questionnaire. The measurement of the fourth stage of group development had a strong association with the planning effectiveness measurement, which means that a team with less issues in the fourth phase of group development is more effective in adhering to its plans. On the other hand, group development and velocity showed no significant convergent validity. We conclude that the dynamics within software development teams might correlate to their ability to deliver the expected outcome as planned but not to their ability to develop tasks faster.
|
|
| 9. |
- Al Sabbagh, Khaled, 1987, et al.
(författare)
-
The Effect of Class Noise on Continuous Test Case Selection: A Controlled Experiment on Industrial Data
- 2020
-
Ingår i: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). - 1611-3349 .- 0302-9743. ; 12562, s. 287-303
-
Konferensbidrag (refereegranskat)abstract
- Continuous integration and testing produce a large amount of data about defects in code revisions, which can be utilized for training a predictive learner to effectively select a subset of test suites. One challenge in using predictive learners lies in the noise that comes in the training data, which often leads to a decrease in classification performances. This study examines the impact of one type of noise, called class noise, on a learner’s ability for selecting test cases. Understanding the impact of class noise on the performance of a learner for test case selection would assist testers decide on the appropriateness of different noise handling strategies. For this purpose, we design and implement a controlled experiment using an industrial data-set to measure the impact of class noise at six different levels on the predictive performance of a learner. We measure the learning performance using the Precision, Recall, F-score, and Mathew Correlation Coefficient (MCC) metrics. The results show a statistically significant relationship between class noise and the learners performance for test case selection. Particularly, a significant difference between the three performance measures (Precision, F-score, and MCC)under all the six noise levels and at 0% level was found, whereas a similar relationship between recall and class noise was found at a level above30%. We conclude that higher class noise ratios lead to missing out more tests in the predicted subset of test suite and increases the rate of false alarms when the class noise ratio exceeds 30%
|
|
| 10. |
- Gren, Lucas, 1984, et al.
(författare)
-
Group developmental psychology and software development performance
- 2017
-
Ingår i: IEEE/ACM 39th International Conference on Software Engineering Companion, ICSE-C 2017. 20-28 May 2017, Buenos Aires, Argentina. - : IEEE. - 9781538615898
-
Konferensbidrag (refereegranskat)abstract
- © 2017 IEEE. Due to the fact that software development is a product of team effort it is important to investigate the influence of group developmental psychology on software development performance. In this case study we wanted to test how performance (i.e. velocity and planning effectiveness) are related to the group's maturity level. We gave the Group Development Questionnaire (the GDQ) to 19 software developers to assess their group maturity (i.e. their progress in their group development) and ran correlation analysis against the development velocity and planning effectiveness (i.e. earned points over planned points). The results show that group maturity is correlated to planning effectiveness but not velocity, meaning that group development is connected to the team's ability to plan well, but not their ability to implement tasks fast.
|
|
