| 1. |
- Afzal, Wasif, et al.
(författare)
-
A Comparative Evaluation of Using Genetic Programming for Predicting Fault Count Data
- 2008
-
Ingår i: Proceedings - The 3rd International Conference on Software Engineering Advances, ICSEA 2008, Includes ENTISY 2008: International Workshop on Enterprise Information Systems. - : IEEE. - 9780769533728 ; , s. 407-414
-
Konferensbidrag (refereegranskat)abstract
- There have been a number of software reliability growth models (SRGMs) proposed in literature. Due to several reasons, such as violation of models' assumptions and complexity of models, the practitioners face difficulties in knowing which models to apply in practice. This paper presents a comparative evaluation of traditional models and use of genetic programming (GP) for modeling software reliability growth based on weekly fault count data of three different industrial projects. The motivation of using a GP approach is its ability to evolve a model based entirely on prior data without the need of making underlying assumptions. The results show the strengths of using GP for predicting fault count data.
|
|
| 2. |
- Afzal, Wasif, et al.
(författare)
-
A systematic mapping study on non-functional search-based software testing
- 2008
-
Ingår i: 20th International Conference on Software Engineering and Knowledge Engineering.
-
Konferensbidrag (refereegranskat)abstract
- Automated software test generation has been applied across the spectrum of test case design methods; this includes white-box (structural), black-box (functional), greybox (combination of structural and functional) and nonfunctional testing. In this paper, we undertake a systematic mapping study to present a broad review of primary studies on the application of search-based optimization techniques to non-functional testing. The motivation is to identify the evidence available on the topic and to identify gaps in the application of search-based optimization techniques to different types of non-functional testing. The study is based on a comprehensive set of 35 papers obtained after using a multi-stage selection criteria and are published in workshops, conferences and journals in the time span 1996–2007. We conclude that the search-based software testing community needs to do more and broader studies on nonfunctional search-based software testing (NFSBST) and the results from our systematic map can help direct such efforts.
|
|
| 3. |
- Afzal, Wasif, et al.
(författare)
-
A systematic review of search-based testing for non-functional system properties
- 2009
-
Ingår i: Information and Software Technology. - : Butterworth-Heinemann Newton, MA, USA. - 0950-5849 .- 1873-6025. ; 51:6, s. 957-976
-
Tidskriftsartikel (refereegranskat)abstract
- Search-based software testing is the application of metaheuristic search techniques to generate software tests. The test adequacy criterion is transformed into a fitness function and a set of solutions in the search space are evaluated with respect to the fitness function using a metaheuristic search technique. The application of metaheuristic search techniques for testing is promising due to the fact that exhaustive testing is infeasible considering the size and complexity of software under test. Search-based software testing has been applied across the spectrum of test case design methods; this includes white-box (structural), black-box (functional) and grey-box (combination of structural and functional) testing. In addition, metaheuristic search techniques have also been applied to test non-functional properties. The overall objective of undertaking this systematic review is to examine existing work into non-functional search-based software testing (NFSBST). We are interested in types of non-functional testing targeted using metaheuristic search techniques, different fitness functions used in different types of search-based non-functional testing and challenges in the application of these techniques. The systematic review is based on a comprehensive set of 35 articles obtained after a multi-stage selection process and have been published in the time span 1996-2007. The results of the review show that metaheuristic search techniques have been applied for non-functional testing of execution time, quality of service, security, usability and safety. A variety of metaheuristic search techniques are found to be applicable for non-functional testing including simulated annealing, tabu search, genetic algorithms, ant colony methods, grammatical evolution, genetic programming (and its variants including linear genetic programming) and swarm intelligence methods. The review reports on different fitness functions used to guide the search for each of the categories of execution time, safety, usability, quality of service and security; along with a discussion of possible challenges in the application of metaheuristic search techniques.
|
|
| 4. |
- Afzal, Wasif, et al.
(författare)
-
An experiment on the effectiveness and efficiency of exploratory testing
- 2015
-
Ingår i: Empirical Software Engineering. - : Springer. - 1382-3256 .- 1573-7616. ; 20:3, s. 844-878
-
Tidskriftsartikel (refereegranskat)abstract
- The exploratory testing (ET) approach is commonly applied in industry, but lacks scientific research. The scientific community needs quantitative results on the performance of ET taken from realistic experimental settings. The objective of this paper is to quantify the effectiveness and efficiency of ET vs. testing with documented test cases (test case based testing, TCT). We performed four controlled experiments where a total of 24 practitioners and 46 students performed manual functional testing using ET and TCT. We measured the number of identified defects in the 90-minute testing sessions, the detection difficulty, severity and types of the detected defects, and the number of false defect reports. The results show that ET found a significantly greater number of defects. ET also found significantly more defects of varying levels of difficulty, types and severity levels. However, the two testing approaches did not differ significantly in terms of the number of false defect reports submitted. We conclude that ET was more efficient than TCT in our experiment. ET was also more effective than TCT when detection difficulty, type of defects and severity levels are considered. The two approaches are comparable when it comes to the number of false defect reports submitted.
|
|
| 5. |
- Afzal, Wasif, et al.
(författare)
-
Cloud-Based Architectures for Model-Based Simulation Testing of Embedded Software
- 2021
-
Ingår i: 2021 10th Mediterranean Conference on Embedded Computing, MECO 2021. - 9780738133614
-
Konferensbidrag (refereegranskat)abstract
- Model-based testing (MBT) generates many test cases for validating a system under test against the user-defined requirements. Cloud computing provides powerful resources that can be utilised to execute these many test cases that would otherwise take much resources locally. Other benefits of utilizing cloud-based resources are elastic and on-demand, rapid provisioning and release of new, potentially value-adding services. Although cloud providers such as Amazon Web Services (AWS) have provided the necessary technologies for successful cloud-based operation, it remains difficult to migrate and hence achieve the realisation of MBT as a service for traditional in-house testing operations, especially for embedded software. In this paper, we present a series of cloud-based architectures powered by AWS and an open-source MBT tool, GraphWalker. These architectures are realized at simulation testing stage for real-world embedded software and particularly cater for online MBT, whereby the model-based tool is deployed as a RESTful web service, accessible through a number of REST API commands. The presented architectures as well as their realization through AWS can be adopted in future for more advanced levels of simulation testing of embedded software.
|
|
| 6. |
- Afzal, Wasif, et al.
(författare)
-
Genetic programming for cross-release fault count predictions in large and complex software projects
- 2010
-
Ingår i: Evolutionary Computation and Optimization Algorithms in Software Engineering. - : IGI Global, Hershey, USA. - 9781615208098
-
Bokkapitel (refereegranskat)abstract
- Software fault prediction can play an important role in ensuring software quality through efficient resource allocation. This could, in turn, reduce the potentially high consequential costs due to faults. Predicting faults might be even more important with the emergence of short-timed and multiple software releases aimed at quick delivery of functionality. Previous research in software fault prediction has indicated that there is a need i) to improve the validity of results by having comparisons among number of data sets from a variety of software, ii) to use appropriate model evaluation measures and iii) to use statistical testing procedures. Moreover, cross-release prediction of faults has not yet achieved sufficient attention in the literature. In an attempt to address these concerns, this paper compares the quantitative and qualitative attributes of 7 traditional and machine-learning techniques for modeling the cross-release prediction of fault count data. The comparison is done using extensive data sets gathered from a total of 7 multi-release open-source and industrial software projects. These software projects together have several years of development and are from diverse application areas, ranging from a web browser to a robotic controller software. Our quantitative analysis suggests that genetic programming (GP) tends to have better consistency in terms of goodness of fit and accuracy across majority of data sets. It also has comparatively less model bias. Qualitatively, ease of configuration and complexity are less strong points for GP even though it shows generality and gives transparent models. Artificial neural networks did not perform as well as expected while linear regression gave average predictions in terms of goodness of fit and accuracy. Support vector machine regression and traditional software reliability growth models performed below average on most of the quantitative evaluation criteria while remained on average for most of the qualitative measures.
|
|
| 7. |
- Afzal, Wasif, et al.
(författare)
-
Incorporating Metrics in an Organizational Test Strategy
- 2008
-
Ingår i: International Conference on Software Testing, Verification and Validation. - : IEEE. - 9780769533889 ; , s. 304-315
-
Konferensbidrag (refereegranskat)abstract
- An organizational level test strategy needs to incorporate metrics to make the testing activities visible and available to process improvements. The majority of testing measurements that are done are based on faults found in the test execution phase. In contrast, this paper investigates metrics to support software test planning and test design processes. We have assembled metrics in these two process types to support management in carrying out evidence-based test process improvements and to incorporate suitable metrics as part of an organization level test strategy. The study is composed of two steps. The first step creates a relevant context by analyzing key phases in the software testing lifecycle, while the second step identifies the attributes of software test planning and test design processes along with metric(s) support for each of the identified attributes.
|
|
| 8. |
- Afzal, Wasif, et al.
(författare)
-
Lessons from applying experimentation in software engineering prediction systems
- 2008
-
Ingår i: Proceedings of The 2nd International workshop on Software Productivity Analysis and Cost Estimation (SPACE'08), Collocated with 15th Asia-Pacific Software Engineering Conference. - Beijing : State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences.
-
Konferensbidrag (refereegranskat)abstract
- Within software engineering prediction systems, experiments are undertaken primarliy to investigate relationships and to measure/compare models' accuracy. This paper discusses our experience and presents useful lessons/guidelines in experimenting with software engineering prediction systems. For this purpose, we use a typical software engineering experimentation process as a baseline. We found that the typical software engineering experimentation process in software engineering is supportive in developing prediction systems and have highlighted issues more central to the domain of software engineering prediction systems.
|
|
| 9. |
- Afzal, Wasif, et al.
(författare)
-
On the application of genetic programming for software engineering predictive modeling : A systematic review
- 2011
-
Ingår i: Expert Systems with Applications. - : Pergamon-Elsevier Science Ltd. - 0957-4174 .- 1873-6793. ; 38:9, s. 11984-11997
-
Forskningsöversikt (refereegranskat)abstract
- The objective of this paper is to investigate the evidence for symbolic regression using genetic programming (GP) being an effective method for prediction and estimation in software engineering, when compared with regression/machine learning models and other comparison groups (including comparisons with different improvements over the standard GP algorithm). We performed a systematic review of literature that compared genetic programming models with comparative techniques based on different independent project variables. A total of 23 primary studies were obtained after searching different information sources in the time span 1995-2008. The results of the review show that symbolic regression using genetic programming has been applied in three domains within software engineering predictive modeling: (i) Software quality classification (eight primary studies). (ii) Software cost/effort/size estimation (seven primary studies). (iii) Software fault prediction/software reliability growth modeling (eight primary studies). While there is evidence in support of using genetic programming for software quality classification, software fault prediction and software reliability growth modeling: the results are inconclusive for software cost/effort/size estimation.
|
|
| 10. |
- Afzal, Wasif, et al.
(författare)
-
On using grey literature and google scholar in systematic literature reviews in software engineering
- 2020
-
Ingår i: IEEE Access. - United States. - 2169-3536. ; 8, s. 36226-36243
-
Tidskriftsartikel (refereegranskat)abstract
- © 2013 IEEE. Context: The inclusion of grey literature (GL) is important to remove publication bias while gathering available evidence regarding a certain topic. The number of systematic literature reviews (SLRs) in Software Engineering (SE) is increasing but we do not know about the extent of GL usage in these SLRs. Moreover, Google Scholar is rapidly becoming a search engine of choice for many researchers but the extent to which it can find the primary studies is not known. Objective: This tertiary study is an attempt to i) measure the usage of GL in SLRs in SE. Furthermore this study proposes strategies for categorizing GL and a quality checklist to use for GL in future SLRs; ii) explore if it is feasible to use only Google Scholar for finding scholarly articles for academic research. Method: We have conducted a systematic mapping study to measure the extent of GL usage in SE SLRs as well as to measure the feasibility of finding primary studies using Google Scholar. Results and conclusions: a) Grey Literature: 76.09% SLRs (105 out of 138) in SE have included one or more GL studies as primary studies. Among total primary studies across all SLRs (6307), 582 are classified as GL, making the frequency of GL citing as 9.23%. The intensity of GL use indicate that each SLR contains 5 primary studies on average (total intensity of GL use being 5.54). The ranking of GL tells us that conference papers are the most used form 43.3% followed by technical reports 28.52%. Universities, research institutes, labs and scientific societies together make up 67.7% of GL used, indicating that these are useful sources for searching GL. We additionally propose strategies for categorizing GL and criteria for evaluating GL quality, which can become a basis for more detailed guidelines for including GL in future SLRs. b) Google Scholar Results: The results show that Google Scholar was able to retrieve 96% of primary studies of these SLRs. Most of the primary studies that were not found using Google Scholar were from grey sources.
|
|
