| 1. |
- Lehman, Joel, et al.
(författare)
-
The Surprising Creativity of Digital Evolution: A Collection of Anecdotes from the Evolutionary Computation and Artificial Life Research Communities
- 2020
-
Ingår i: Artificial Life. - : MIT Press - Journals. - 1530-9185 .- 1064-5462. ; 26:2, s. 274-306
-
Tidskriftsartikel (refereegranskat)abstract
- Evolution provides a creative fount of complex and subtle adaptations that often surprise the scientists who discover them. However, the creativity of evolution is not limited to the natural world: Artificial organisms evolving in computational environments have also elicited surprise and wonder from the researchers studying them. The process of evolution is an algorithmic process that transcends the substrate in which it occurs. Indeed, many researchers in the field of digital evolution can provide examples of how their evolving algorithms and organisms have creatively subverted their expectations or intentions, exposed unrecognized bugs in their code, produced unexpectedly adaptations, or engaged in behaviors and outcomes, uncannily convergent with ones found in nature. Such stories routinely reveal surprise and creativity by evolution in these digital worlds, but they rarely fit into the standard scientific narrative. Instead they are often treated as mere obstacles to be overcome, rather than results that warrant study in their own right. Bugs are fixed, experiments are refocused, and one-off surprises are collapsed into a single data point. The stories themselves are traded among researchers through oral tradition, but that mode of information transmission is inefficient and prone to error and outright loss. Moreover, the fact that these stories tend to be shared only among practitioners means that many natural scientists do not realize how interesting and lifelike digital organisms are and how natural their evolution can be. To our knowledge, no collection of such anecdotes has been published before. This article is the crowd-sourced product of researchers in the fields of artificial life and evolutionary computation who have provided first-hand accounts of such cases. It thus serves as a written, fact-checked collection of scientifically important and even entertaining stories. In doing so we also present here substantial evidence that the existence and importance of evolutionary surprises extends beyond the natural world, and may indeed be a universal property of all complex evolving systems.
|
|
| 2. |
|
|
| 3. |
|
|
| 4. |
- Afzal, Wasif, et al.
(författare)
-
A systematic mapping study on non-functional search-based software testing
- 2008
-
Ingår i: 20th International Conference on Software Engineering and Knowledge Engineering.
-
Konferensbidrag (refereegranskat)abstract
- Automated software test generation has been applied across the spectrum of test case design methods; this includes white-box (structural), black-box (functional), greybox (combination of structural and functional) and nonfunctional testing. In this paper, we undertake a systematic mapping study to present a broad review of primary studies on the application of search-based optimization techniques to non-functional testing. The motivation is to identify the evidence available on the topic and to identify gaps in the application of search-based optimization techniques to different types of non-functional testing. The study is based on a comprehensive set of 35 papers obtained after using a multi-stage selection criteria and are published in workshops, conferences and journals in the time span 1996–2007. We conclude that the search-based software testing community needs to do more and broader studies on nonfunctional search-based software testing (NFSBST) and the results from our systematic map can help direct such efforts.
|
|
| 5. |
- Afzal, Wasif, et al.
(författare)
-
A systematic review of search-based testing for non-functional system properties
- 2009
-
Ingår i: Information and Software Technology. - : Butterworth-Heinemann Newton, MA, USA. - 0950-5849 .- 1873-6025. ; 51:6, s. 957-976
-
Tidskriftsartikel (refereegranskat)abstract
- Search-based software testing is the application of metaheuristic search techniques to generate software tests. The test adequacy criterion is transformed into a fitness function and a set of solutions in the search space are evaluated with respect to the fitness function using a metaheuristic search technique. The application of metaheuristic search techniques for testing is promising due to the fact that exhaustive testing is infeasible considering the size and complexity of software under test. Search-based software testing has been applied across the spectrum of test case design methods; this includes white-box (structural), black-box (functional) and grey-box (combination of structural and functional) testing. In addition, metaheuristic search techniques have also been applied to test non-functional properties. The overall objective of undertaking this systematic review is to examine existing work into non-functional search-based software testing (NFSBST). We are interested in types of non-functional testing targeted using metaheuristic search techniques, different fitness functions used in different types of search-based non-functional testing and challenges in the application of these techniques. The systematic review is based on a comprehensive set of 35 articles obtained after a multi-stage selection process and have been published in the time span 1996-2007. The results of the review show that metaheuristic search techniques have been applied for non-functional testing of execution time, quality of service, security, usability and safety. A variety of metaheuristic search techniques are found to be applicable for non-functional testing including simulated annealing, tabu search, genetic algorithms, ant colony methods, grammatical evolution, genetic programming (and its variants including linear genetic programming) and swarm intelligence methods. The review reports on different fitness functions used to guide the search for each of the categories of execution time, safety, usability, quality of service and security; along with a discussion of possible challenges in the application of metaheuristic search techniques.
|
|
| 6. |
- Afzal, Wasif, et al.
(författare)
-
Genetic programming for cross-release fault count predictions in large and complex software projects
- 2010
-
Ingår i: Evolutionary Computation and Optimization Algorithms in Software Engineering. - : IGI Global, Hershey, USA. - 9781615208098
-
Bokkapitel (refereegranskat)abstract
- Software fault prediction can play an important role in ensuring software quality through efficient resource allocation. This could, in turn, reduce the potentially high consequential costs due to faults. Predicting faults might be even more important with the emergence of short-timed and multiple software releases aimed at quick delivery of functionality. Previous research in software fault prediction has indicated that there is a need i) to improve the validity of results by having comparisons among number of data sets from a variety of software, ii) to use appropriate model evaluation measures and iii) to use statistical testing procedures. Moreover, cross-release prediction of faults has not yet achieved sufficient attention in the literature. In an attempt to address these concerns, this paper compares the quantitative and qualitative attributes of 7 traditional and machine-learning techniques for modeling the cross-release prediction of fault count data. The comparison is done using extensive data sets gathered from a total of 7 multi-release open-source and industrial software projects. These software projects together have several years of development and are from diverse application areas, ranging from a web browser to a robotic controller software. Our quantitative analysis suggests that genetic programming (GP) tends to have better consistency in terms of goodness of fit and accuracy across majority of data sets. It also has comparatively less model bias. Qualitatively, ease of configuration and complexity are less strong points for GP even though it shows generality and gives transparent models. Artificial neural networks did not perform as well as expected while linear regression gave average predictions in terms of goodness of fit and accuracy. Support vector machine regression and traditional software reliability growth models performed below average on most of the quantitative evaluation criteria while remained on average for most of the qualitative measures.
|
|
| 7. |
- Afzal, Wasif, et al.
(författare)
-
Prediction of fault count data using genetic programming
- 2008
-
Ingår i: IEEE INMIC 2008: 12th IEEE International Multitopic Conference - Conference Proceedings. - Karachi, Pakistan : IEEE. ; , s. 349-356
-
Konferensbidrag (refereegranskat)abstract
- Software reliability growth modeling helps in deciding project release time and managing project resources. A large number of such models have been presented in the past. Due to the existence of many models, the models' inherent complexity, and their accompanying assumptions; the selection of suitable models becomes a challenging task. This paper presents empirical results of using genetic programming (GP) for modeling software reliability growth based on weekly fault count data of three different industrial projects. The goodness of fit (adaptability) and predictive accuracy of the evolved model is measured using five different measures in an attempt to present a fair evaluation. The results show that the GP evolved model has statistically significant goodness of fit and predictive accuracy.
|
|
| 8. |
- Afzal, Wasif, et al.
(författare)
-
Resampling Methods in Software Quality Classification
- 2012
-
Ingår i: International Journal of Software Engineering and Knowledge Engineering. - Sweden : World Scientific. - 0218-1940. ; 22:2, s. 203-223
-
Tidskriftsartikel (refereegranskat)abstract
- In the presence of a number of algorithms for classification and prediction in software engineering, there is a need to have a systematic way of assessing their performances. The performance assessment is typically done by some form of partitioning or resampling of the original data to alleviate biased estimation. For predictive and classification studies in software engineering, there is a lack of a definitive advice on the most appropriate resampling method to use. This is seen as one of the contributing factors for not being able to draw general conclusions on what modeling technique or set of predictor variables are the most appropriate. Furthermore, the use of a variety of resampling methods make it impossible to perform any formal meta-analysis of the primary study results. Therefore, it is desirable to examine the influence of various resampling methods and to quantify possible differences. Objective and method: This study empirically compares five common resampling methods (hold-out validation, repeated random sub-sampling, 10-fold cross-validation, leave-one-out cross-validation and non-parametric bootstrapping) using 8 publicly available data sets with genetic programming (GP) and multiple linear regression (MLR) as software quality classification approaches. Location of (PF, PD) pairs in the ROC (receiver operating characteristics) space and area under an ROC curve (AUC) are used as accuracy indicators. Results: The results show that in terms of the location of (PF, PD) pairs in the ROC space, bootstrapping results are in the preferred region for 3 of the 8 data sets for GP and for 4 of the 8 data sets for MLR. Based on the AUC measure, there are no significant differences between the different resampling methods using GP and MLR. Conclusion: There can be certain data set properties responsible for insignificant differences between the resampling methods based on AUC. These include imbalanced data sets, insignificant predictor variables and high-dimensional data sets. With the current selection of data sets and classification techniques, bootstrapping is a preferred method based on the location of (PF, PD) pair data in the ROC space. Hold-out validation is not a good choice for comparatively smaller data sets, where leave-one-out cross-validation (LOOCV) performs better. For comparatively larger data sets, 10-fold cross-validation performs better than LOOCV.
|
|
| 9. |
- Afzal, Wasif, et al.
(författare)
-
Search-based prediction of fault count data
- 2009
-
Ingår i: Proceedings - 1st International Symposium on Search Based Software Engineering, SSBSE 2009. - Windsor : IEEE Computer Society. - 9780769536750 ; , s. 35-38
-
Konferensbidrag (refereegranskat)abstract
- Symbolic regression, an application domain of genetic programming (GP), aims to find a function whose output has some desired property, like matching target values of a particular data set. While typical regression involves finding the coefficients of a pre-defined function, symbolic regression finds a general function, with coefficients, fitting the given set of data points. The concepts of symbolic regression using genetic programming can be used to evolve a model for fault count predictions. Such a model has the advantages that the evolution is not dependent on a particular structure of the model and is also independent of any assumptions, which are common in traditional time-domain parametric software reliability growth models. This research aims at applying experiments targeting fault predictions using genetic programming and comparing the results with traditional approaches to compare efficiency gains.
|
|
| 10. |
- Afzal, Wasif, et al.
(författare)
-
Search-based prediction of fault-slip-through in large software projects
- 2010
-
Ingår i: Proceedings - 2nd International Symposium on Search Based Software Engineering, SSBSE 2010. - : IEEE. - 9780769541952 ; , s. 79-88
-
Konferensbidrag (refereegranskat)abstract
- A large percentage of the cost of rework can be avoided by finding more faults earlier in a software testing process. Therefore, determination of which software testing phases to focus improvements work on, has considerable industrial interest. This paper evaluates the use of five different techniques, namely particle swarm optimization based artificial neural networks (PSO-ANN), artificial immune recognition systems (AIRS), gene expression programming (GEP), genetic programming (GP) and multiple regression (MR), for predicting the number of faults slipping through unit, function, integration and system testing phases. The objective is to quantify improvement potential in different testing phases by striving towards finding the right faults in the right phase. We have conducted an empirical study of two large projects from a telecommunication company developing mobile platforms and wireless semiconductors. The results are compared using simple residuals, goodness of fit and absolute relative error measures. They indicate that the four search-based techniques (PSO-ANN, AIRS, GEP, GP) perform better than multiple regression for predicting the fault-slip-through for each of the four testing phases. At the unit and function testing phases, AIRS and PSO-ANN performed better while GP performed better at integration and system testing phases. The study concludes that a variety of search-based techniques are applicable for predicting the improvement potential in different testing phases with GP showing more consistent performance across two of the four test phases.
|
|
| 11. |
- Ahmad, E., et al.
(författare)
-
Assessment and support for software capstone projects at the undergraduate level: A survey and rubrics
- 2011
-
Ingår i: Frontiers of Information Technology (FIT), 2011, Islamabad, 19-21 Dec. 2011. - Islamabad. - 9780769546254 ; , s. 25 - 32
-
Konferensbidrag (refereegranskat)abstract
- Software engineering and computer science students conduct a capstone project during the final year of their degree programs. These projects are essential in validating that students have gained required knowledge and they can synthesize and use that knowledge to solve real world problems. However, the external requirements on educational programs often do not provide detailed guidelines for how to conduct or support these capstone projects, which may lead to variations among universities. This paper presents the results from a survey conducted at 19 different Pakistani universities of the current management practices and assessment criteria used for the capstone project courses at Undergraduate level. Based upon the results of this survey and similar work on Master Thesis capstone projects in Sweden, we present assessment rubrics for software-related undergraduate capstone projects. We also present recommendations for the continuous improvement of capstone projects.
|
|
| 12. |
- Alégroth, Emil, 1984, et al.
(författare)
-
Industrial application of visual GUI testing: Lessons learned
- 2014
-
Ingår i: Continuous software engineering. - Cham : Springer International Publishing. ; 9783319112831, s. 127-140
-
Bokkapitel (övrigt vetenskapligt/konstnärligt)abstract
- A large body of academic knowledge has been devoted to automated software testing in order to support the software market's demands for continuous software delivery. However, most of these automated techniques approach testing from lower levels of system abstraction, e.g., component level, which limit their applicability for high-level regression testing of, for instance, system and acceptance tests, thus forcing companies to perform these test activities manually, which is considered time consuming, tedious, and error prone. In this book chapter, we present visual GUI testing (VGT), a tool driven test technique that uses image recognition in order to interact and assert the correctness of a system under test (SUT) through the bitmap graphical user interface (GUI) that is shown to the user on the computer monitor. This approach makes VGT flexible and applicable to any SUT with a GUI but also allows VGT tools to emulate end-user behavior and therefore automate previously manual system and acceptance test cases. In addition to presenting the technique itself, this chapter will also present some VGT tools and empirically identified problems with the technique and how these problems can be mitigated in practice. Finally we will discuss how VGT can be used in the context of continuous software development in order to support market demands for quicker software delivery.
|
|
| 13. |
- Alégroth, Emil, 1984, et al.
(författare)
-
Maintenance of automated test suites in industry: An empirical study on Visual GUI Testing
- 2016
-
Ingår i: Information and Software Technology. - : Elsevier BV. - 0950-5849 .- 1873-6025. ; 73, s. 66-80
-
Tidskriftsartikel (refereegranskat)abstract
- Context: Verification and validation (V&V) activities make up 20-50% of the total development costs of a software system in practice. Test automation is proposed to lower these V&V costs but available research only provides limited empirical data from industrial practice about the maintenance costs of automated tests and what factors affect these costs. In particular, these costs and factors are unknown for automated GUI-based testing. Objective: This paper addresses this lack of knowledge through analysis of the costs and factors associated with the maintenance of automated GUI-based tests in industrial practice. Method: An empirical study at two companies, Siemens and Saab, is reported where interviews about, and empirical work with, Visual GUI Testing is performed to acquire data about the technique's maintenance costs and feasibility. Results: 13 factors are observed that affect maintenance, e.g. tester knowledge/experience and test case complexity. Further, statistical analysis shows that developing new test scripts is costlier than maintenance but also that frequent maintenance is less costly than infrequent, big bang maintenance. In addition a cost model, based on previous work, is presented that estimates the time to positive return on investment (ROI) of test automation compared to manual testing. Conclusions: It is concluded that test automation can lower overall software development costs of a project while also having positive effects on software quality. However, maintenance costs can still be considerable and the less time a company currently spends on manual testing, the more time is required before positive, economic, ROI is reached after automation.
|
|
| 14. |
- Alégroth, Emil, et al.
(författare)
-
On the long-term use of visual gui testing in industrial practice : a case study
- 2017
-
Ingår i: Empirical Software Engineering. - : Springer. - 1382-3256 .- 1573-7616. ; 22:6, s. 2937-2971
-
Tidskriftsartikel (refereegranskat)abstract
- Visual GUI Testing (VGT) is a tool-driven technique for automated GUI-based testing that uses image recognition to interact with and assert the correctness of the behavior of a system through its GUI as it is shown to the user. The technique’s applicability, e.g. defect-finding ability, and feasibility, e.g. time to positive return on investment, have been shown through empirical studies in industrial practice. However, there is a lack of studies that evaluate the usefulness and challenges associated with VGT when used long-term (years) in industrial practice. This paper evaluates how VGT was adopted, applied and why it was abandoned at the music streaming application development company, Spotify, after several years of use. A qualitative study with two workshops and five well chosen employees is performed at the company, supported by a survey, which is analyzed with a grounded theory approach to answer the study’s three research questions. The interviews provide insights into the challenges, problems and limitations, but also benefits, that Spotify experienced during the adoption and use of VGT. However, due to the technique’s drawbacks, VGT has been abandoned for a new technique/framework, simply called the Test interface. The Test interface is considered more robust and flexible for Spotify’s needs but has several drawbacks, including that it does not test the actual GUI as shown to the user like VGT does. From the study’s results it is concluded that VGT can be used long-term in industrial practice but it requires organizational change as well as engineering best practices to be beneficial. Through synthesis of the study’s results, and results from previous work, a set of guidelines are presented that aim to aid practitioners to adopt and use VGT in industrial practice. However, due to the abandonment of the technique, future research is required to analyze in what types of projects the technique is, and is not, long-term viable. To this end, we also present Spotify’s Test interface solution for automated GUI-based testing and conclude that it has its own benefits and drawbacks.
|
|
| 15. |
- Alégroth, Emil, et al.
(författare)
-
Replicating Rare Software Failures with Exploratory Visual GUI Testing
- 2017
-
Ingår i: IEEE Software. - : IEEE Computer Society. - 0740-7459 .- 1937-4194. ; 34:5, s. 53-59
-
Tidskriftsartikel (refereegranskat)abstract
- Saab AB developed software that had a defect that manifested itself only after months of continuous system use. After years of customer failure reports, the defect still persisted, until Saab developed failure replication based on visual GUI testing. © 1984-2012 IEEE.
|
|
| 16. |
- Alégroth, Emil, 1984, et al.
(författare)
-
Special issue on new generations of UI testing
- 2021
-
Ingår i: Software Testing Verification and Reliability. - : Wiley. - 0960-0833 .- 1099-1689. ; 31:3
-
Tidskriftsartikel (övrigt vetenskapligt/konstnärligt)
|
|
| 17. |
- Alégroth, Emil, 1984, et al.
(författare)
-
Transitioning manual system test suites to automated testing: An industrial case study
- 2013
-
Ingår i: Proceedings - IEEE 6th International Conference on Software Testing, Verification and Validation, ICST 2013. - : IEEE. - 9780769549682 ; , s. 56-65
-
Konferensbidrag (refereegranskat)abstract
- Visual GUI testing (VGT) is an emerging technique that provides software companies with the capability to automate previously time-consuming, tedious, and fault prone manual system and acceptance tests. Previous work on VGT has shown that the technique is industrially applicable, but has not addressed the real-world applicability of the technique when used by practitioners on industrial grade systems. This paper presents a case study performed during an industrial project with the goal to transition from manual to automated system testing using VGT. Results of the study show that the VGT transition was successful and that VGT could be applied in the industrial context when performed by practitioners but that there were several problems that first had to be solved, e.g. testing of a distributed system, tool volatility. These problems and solutions have been presented together with qualitative, and quantitative, data about the benefits of the technique compared to manual testing, e.g. greatly improved execution speed, feasible transition and maintenance costs, improved bug finding ability. The study thereby provides valuable, and previously missing, contributions about VGT to both practitioners and researchers. © 2013 IEEE.
|
|
| 18. |
- Alégroth, Emil, 1984, et al.
(författare)
-
Visual GUI testing in practice: challenges, problemsand limitations
- 2015
-
Ingår i: Empirical Software Engineering. - : Springer Science and Business Media LLC. - 1573-7616 .- 1382-3256. ; 20:3, s. 694-744
-
Tidskriftsartikel (refereegranskat)abstract
- In today's software development industry, high-level tests such as Graphical User Interface (GUI) based system and acceptance tests are mostly performed with manual practices that are often costly, tedious and error prone. Test automation has been proposed to solve these problems but most automation techniques approach testing from a lower level of system abstraction. Their suitability for high-level tests has therefore been questioned. High-level test automation techniques such as Record and Replay exist, but studies suggest that these techniques suffer from limitations, e.g. sensitivity to GUI layout or code changes, system implementation dependencies, etc. Visual GUI Testing (VGT) is an emerging technique in industrial practice with perceived higher flexibility and robustness to certain GUI changes than previous high-level (GUI) test automation techniques. The core of VGT is image recognition which is applied to analyze and interact with the bitmap layer of a system's front end. By coupling image recognition with test scripts, VGT tools can emulate end user behavior on almost any GUI-based system, regardless of implementation language, operating system or platform. However, VGT is not without its own challenges, problems and limitations (CPLs) but, like for many other automated test techniques, there is a lack of empirically-based knowledge of these CPLs and how they impact industrial applicability. Crucially, there is also a lack of information on the cost of applying this type of test automation in industry. This manuscript reports an empirical, multi-unit case study performed at two Swedish companies that develop safety-critical software. It studies their transition from manual system test cases into tests automated with VGT. In total, four different test suites that together include more than 300 high-level system test cases were automated for two multi-million lines of code systems. The results show that the transitioned test cases could find defects in the tested systems and that all applicable test cases could be automated. However, during these transition projects a number of hurdles had to be addressed; a total of 58 different CPLs were identified and then categorized into 26 types. We present these CPL types and an analysis of the implications for the transition to and use of VGT in industrial software development practice. In addition, four high-level solutions are presented that were identified during the study, which would address about half of the identified CPLs. Furthermore, collected metrics on cost and return on investment of the VGT transition are reported together with information about the VGT suites' defect finding ability. Nine of the identified defects are reported, 5 of which were unknown to testers with extensive experience from using the manual test suites. The main conclusion from this study is that even though there are many challenges related to the transition and usage of VGT, the technique is still valuable, flexible and considered cost-effective by the industrial practitioners. The presented CPLs also provide decision support in the use and advancement of VGT and potentially other automated testing techniques similar to VGT, e.g. Record and Replay.
|
|
| 19. |
- Alizadeh, Zeinab, et al.
(författare)
-
Alignment of requirements specification and testing: A systematic mapping study
- 2011
-
Ingår i: Proceedings of the ICST Workshop on Requirements and Validation, Verification and Testing (REVVERT'11).
-
Konferensbidrag (refereegranskat)abstract
- Requirements should specify expectations on a software system and testing should ensure these expectations are met. Thus, to enable high product quality and efficient development it is crucial that requirements and testing activities and information are aligned. A lot of research has been done in the respective fields of Requirements Engineering and Testing but there is a lack of summaries of the current state of the art on how to link the two. This study presents a systematic mapping of the alignment of specification and testing of functional or nonfunctional requirements in order to identify useful approaches and needs for future research. In particular we focus on results relevant for nonfunctional requirements but since only a few studies was found on alignment in total we also cover the ones on functional requirements. We summarize the 35 relevant papers found and discuss them within six major sub categories with model-based testing and traceability being the ones with most prior results.
|
|
| 20. |
- Axelsson, Stefan, et al.
(författare)
-
Detecting Defects with an Interactive Code Review Tool Based on Visualisation and Machine Learning
- 2009
-
Konferensbidrag (refereegranskat)abstract
- Code review is often suggested as a means of improving code quality. Since humans are poor at repetitive tasks, some form of tool support is valuable. To that end we developed a prototype tool to illustrate the novel idea of applying machine learning (based on Normalised Compression Distance) to the problem of static analysis of source code. Since this tool learns by example, it is rivially programmer adaptable. As machine learning algorithms are notoriously difficult to understand operationally (they are opaque) we applied information visualisation to the results of the learner. In order to validate the approach we applied the prototype to source code from the open-source project Samba and from an industrial, telecom software system. Our results showed that the tool did indeed correctly find and classify problematic sections of code based on training examples.
|
|
| 21. |
- Baigent, Colin, et al.
(författare)
-
The effects of lowering LDL cholesterol with simvastatin plus ezetimibe in patients with chronic kidney disease (Study of Heart and Renal Protection) : a randomised placebo-controlled trial
- 2011
-
Ingår i: The Lancet. - 0140-6736 .- 1474-547X. ; 377:9784, s. 2181-2192
-
Tidskriftsartikel (refereegranskat)abstract
- Background Lowering LDL cholesterol with statin regimens reduces the risk of myocardial infarction, ischaemic stroke, and the need for coronary revascularisation in people without kidney disease, but its effects in people with moderate-to-severe kidney disease are uncertain. The SHARP trial aimed to assess the efficacy and safety of the combination of simvastatin plus ezetimibe in such patients. Methods This randomised double-blind trial included 9270 patients with chronic kidney disease (3023 on dialysis and 6247 not) with no known history of myocardial infarction or coronary revascularisation. Patients were randomly assigned to simvastatin 20 mg plus ezetimibe 10 mg daily versus matching placebo. The key prespecified outcome was first major atherosclerotic event (non-fatal myocardial infarction or coronary death, non-haemorrhagic stroke, or any arterial revascularisation procedure). All analyses were by intention to treat. This trial is registered at ClinicalTrials.gov, NCT00125593, and I SRCTN54137607. Findings 4650 patients were assigned to receive simvastatin plus ezetimibe and 4620 to placebo. Allocation to simvastatin plus ezetimibe yielded an average LDL cholesterol difference of 0.85 mmol/L (SE 0.02; with about two-thirds compliance) during a median follow-up of 4.9 years and produced a 17% proportional reduction in major atherosclerotic events (526 [11.3%] simvastatin plus ezetimibe vs 619 [13.4%] placebo; rate ratio [RR] 0.83, 95% CI 0.74-0.94; log-rank p=0.0021). Non-significantly fewer patients allocated to simvastatin plus ezetimibe had a non-fatal myocardial infarction or died from coronary heart disease (213 [4.6%] vs 230 [5.0%]; RR 0.92,95% CI 0.76-1.11; p=0.37) and there were significant reductions in non-haemorrhagic stroke (131 [2.8%] vs 174 [3.8%]; RR 0.75,95% CI 0.60-0.94; p=0.01) and arterial revascularisation procedures (284 [6.1%] vs 352 [7.6%]; RR 0.79, 95% CI 0.68-0.93; p=0.0036). After weighting for subgroup-specific reductions in LDL cholesterol, there was no good evidence that the proportional effects on major atherosclerotic events differed from the summary rate ratio in any subgroup examined, and, in particular, they were similar in patients on dialysis and those who were not. The excess risk of myopathy was only two per 10 000 patients per year of treatment with this combination (9 [0.2%] vs 5 [0.1%]). There was no evidence of excess risks of hepatitis (21 [0.5%] vs 18 [0.4%]), gallstones (106 [2.3%] vs 106 [2.3%]), or cancer (438 [9.4%] vs 439 [9.5%], p=0.89) and there was no significant excess of death from any non-vascular cause (668 [14.4%] vs 612 [13.2%], p=0.13). Interpretation Reduction of LDL cholesterol with simvastatin 20 mg plus ezetimibe 10 mg daily safely reduced the incidence of major atherosclerotic events in a wide range of patients with advanced chronic kidney disease.
|
|
| 22. |
- Berntsson Svensson, Richard, et al.
(författare)
-
Prioritization of quality requirements : State of practice in eleven companies
- 2011
-
Ingår i: 2011 IEEE 19th International Requirements Engineering Conference, RE 2011; Trento; 29 August 2011 through 2 September 2011. - Trento : IEEE. - 9781457709234 ; , s. 69-78, s. 69-78
-
Konferensbidrag (refereegranskat)abstract
- Requirements prioritization is recognized as an important but challenging activity in software product development. For a product to be successful, it is crucial to find the right balance among competing quality requirements. Although literature offers many methods for requirements prioritization, the research on prioritization of quality requirements is limited. This study identifies how quality requirements are prioritized in practice at 11 successful companies developing software intensive systems. We found that ad-hoc prioritization and priority grouping of requirements are the dominant methods for prioritizing quality requirements. The results also show that it is common to use customer input as criteria for prioritization but absence of any criteria was also common. The results suggests that quality requirements by default have a lower priority than functional requirements, and that they only get attention in the prioritizing process if decision-makers are dedicated to invest specific time and resources on QR prioritization. The results of this study may help future research on quality requirements to focus investigations on industry-relevant issues.
|
|
| 23. |
- Berntsson Svensson, Richard, et al.
(författare)
-
Quality Requirements in Industrial Practice – An Extended Interview Study at Eleven Companies
- 2012
-
Ingår i: IEEE Transactions on Software Engineering. - : IEEE. - 0098-5589 .- 1939-3520. ; 38:4, s. 923-935
-
Tidskriftsartikel (refereegranskat)abstract
- In order to create a successful software product and assure its quality, it is not enough to fulfill the functional requirements, it is also crucial to find the right balance among competing quality requirements (QR). An extended, previosluy piloted, interview study was performed to identify specific challenges associated with the selection, trade-off, and management of QR in industrial practice. Data was collected through semi-structured interviews with eleven product managers and eleven project leaders from eleven software companies. The contribution of this study is fourfold: First, it compares how QR are handled in two cases, companies working in business-to-business markets, and companies that are working in business-to-consumer markets. These two are also compared in terms of impact on the handling of QRs. Second, it compares the perceptions and priorities of QR by product and project management respectively. Third, it includes an examination of the interdependencies among quality requirements perceived as most important by the practitioners. Fourth, it characterizes the selection and management of QR in down-stream development activities.
|
|
| 24. |
- Berntsson Svensson, Richard, 1978, et al.
(författare)
-
Quality Requirements in Industrial Practice - An Extended Interview Study at Eleven Companies
- 2011
-
Ingår i: IEEE Transactions on Software Engineering. - 0098-5589 .- 1939-3520.
-
Tidskriftsartikel (refereegranskat)abstract
- In order to create a successful software product and assure its quality, it is not enough to fulfill the functional requirements, it is also crucial to find the right balance among competing quality requirements (QR). An extended, previously piloted, interview study was performed to identify specific challenges associated with the selection, trade-off, and management of QR in industrial practice. Data was collected through semi-structured interviews with eleven product managers and eleven project leaders from eleven software companies. The contribution of this study is fourfold: First, it compares how QR are handled in two cases, companies working in business-to-business markets, and companies that are working in business-to-consumer markets. These two are also compared in terms of impact on the handling of QRs. Second, it compares the perceptions and priorities of QR by product and project management respectively. Third, it includes an examination of the interdependencies among quality requirements perceived as most important by the practitioners. Fourth, it characterizes the selection and management of QR in down-stream development activities.
|
|
| 25. |
- Berntsson Svensson, Richard, 1978, et al.
(författare)
-
The unfulfilled potential of data-driven decision making in agile software development
- 2019
-
Ingår i: Lecture Notes in Business Information Processing. - Cham : Springer International Publishing. - 1865-1356 .- 1865-1348. - 9783030190330 ; 355, s. 69-85
-
Konferensbidrag (refereegranskat)abstract
- With the general trend towards data-driven decision making (DDDM), organizations are looking for ways to use DDDM to improve their decisions. However, few studies have looked into the practitioners view of DDDM, in particular for agile organizations. In this paper we investigated the experiences of using DDDM, and how data can improve decision making. An emailed questionnaire was sent out to 124 industry practitioners in agile software developing companies, of which 84 answered. The results show that few practitioners indicated a wide-spread use of DDDM in their current decision making practices. The practitioners were more positive to its future use for higher-level and more general decision making, fairly positive to its use for requirements elicitation and prioritization decisions, while being less positive to its future use at the team level. The practitioners do see a lot of potential for DDDM in an agile context; however, currently unfulfilled.
|
|
| 26. |
- Bjarnason, Elizabeth, et al.
(författare)
-
Challenges and practices in aligning requirements with verification and validation : a case study of six companies
- 2014
-
Ingår i: Empirical Software Engineering. - : Springer. - 1382-3256 .- 1573-7616. ; 19:6, s. 1809-1855
-
Tidskriftsartikel (refereegranskat)abstract
- Weak alignment of requirements engineering (RE) with verification and validation (VV) may lead to problems in delivering the required products in time with the right quality. For example, weak communication of requirements changes to testers may result in lack of verification of new requirements and incorrect verification of old invalid requirements, leading to software quality problems, wasted effort and delays. However, despite the serious implications of weak alignment research and practice both tend to focus on one or the other of RE or VV rather than on the alignment of the two. We have performed a multi-unit case study to gain insight into issues around aligning RE and VV by interviewing 30 practitioners from 6 software developing companies, involving 10 researchers in a flexible research process for case studies. The results describe current industry challenges and practices in aligning RE with VV, ranging from quality of the individual RE and VV activities, through tracing and tools, to change control and sharing a common understanding at strategy, goal and design level. The study identified that human aspects are central, i.e. cooperation and communication, and that requirements engineering practices are a critical basis for alignment. Further, the size of an organisation and its motivation for applying alignment practices, e.g. external enforcement of traceability, are variation factors that play a key role in achieving alignment. Our results provide a strategic roadmap for practitioners improvement work to address alignment challenges. Furthermore, the study provides a foundation for continued research to improve the alignment of RE with VV.
|
|
| 27. |
- Briand, Lionel, et al.
(författare)
-
JF Welcome: ICSE 2021
- 2021
-
Ingår i: Proceedings - International Conference on Software Engineering. - : IEEE Computer Society. - 0270-5257. ; May 2021, s. xxxiii-
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)
|
|
| 28. |
|
|
| 29. |
- Börjesson, Emil, 1984, et al.
(författare)
-
Automated system testing using visual GUI testing tools: A comparative study in industry
- 2012
-
Ingår i: Proceedings - IEEE 5th International Conference on Software Testing, Verification and Validation, ICST 2012, Montreal, QC, 17 April through 21 April 2012. - 9780769546704 ; :Article number 6200127, s. 350-359
-
Konferensbidrag (refereegranskat)abstract
- Software companies are under continuous pressure to shorten time to market, raise quality and lower costs. More automated system testing could be instrumental in achieving these goals and in recent years testing tools have been developed to automate the interaction with software systems at the GUI level. However, there is a lack of knowledge on the usability and applicability of these tools in an industrial setting. This study evaluates two tools for automated visual GUI testing on a real-world, safety-critical software system developed by the company Saab AB. The tools are compared based on their properties as well as how they support automation of system test cases that have previously been conducted manually. The time to develop and the size of the automated test cases as well as their execution times have been evaluated. Results show that there are only minor differences between the two tools, one commercial and one open-source, but, more importantly, that visual GUI testing is an applicable technology for automated system testing with effort gains over manual system test practices. The study results also indicate that the technology has benefits over alternative GUI testing techniques and that it can be used for automated acceptance testing. However, visual GUI testing still has challenges that must be addressed, in particular the script maintenance costs and how to support robust test execution. © 2012 IEEE.
|
|
| 30. |
- Börjesson, Emil, 1984, et al.
(författare)
-
Structuring Software Engineering Case Studies to Cover Multiple Perspectives
- 2011
-
Ingår i: SEKE 2011, Proceedings of the Twenty-Third International Conference on Software Engineering & Knowledge Engineering. - 9781891706295
-
Konferensbidrag (refereegranskat)abstract
- Case studies are used in software engineering (SE) research for detailed study of phenomena in their real-world context. There are guidelines listing important factors to consider when designing case studies, but there is a lack of advice on how to structure the collected information and ensure its breadth. Without considering multiple perspectives, such as business and organization, there is a risk that too few perspectives are covered.The objective of this paper is to develop a framework to give structure and ensure breadth of a SE case study.For an analysis of the verification and validation practices of a Swedish software company we developed an analytical framework based on two dimensions. The matrix spanned by the dimensions (perspective and time) helped structure data collection and connect different findings. A six-step process was defined to adapt and execute the framework at the company and we exemplify its use and describe its perceived advantages and disadvantages.The framework simplified the analysis and gave a broader understanding of the studied practices but there is a trade- off with the depth of the results, making the framework more suitable for explorative, open-ended studies.
|
|
| 31. |
- Čaušević, Adnan, 1981-
(författare)
-
Software Testing in Agile Development : Technological and Organisational Challenges
- 2011
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- The emerging industrial trend towards agile software development processes brings forth new concerns, challenges as well as opportunities. One of the main concerns is with respect to the achievable quality levels of the final product, for which testing is the well-known assurance mechanism. However, it is not well defined for the community on how to perform testing using existing expertise in an agile environment. This uncertainty may create confusion and contra productivity that can lead to testing teams and their practices considered as an obstacle for full implementation of agile processes within an organisation. This thesis outlines our current research activities towards identifying and addressing important organisational and technical challenges in the agile environment. In this context, we propose a new role for traditional testers which will enable them to integrate into the agile team as well as to fully exploit their knowledge in the new context. We have conducted an elaborate industrial survey on the preferences and practices with respect to the contemporary aspects of software testing and identified test-driven development as an important technical area for improvement. A systematic review on empirical evidences related to test-driven development was performed subsequently, which revealed a list of factors limiting its widespread industrial acceptance. Knowledge of testing was identified as one of those factors and was further investigated in a controlled experiment performed with undergraduate students. Our future works aim to confirm these research findings in wider as well as industrial settings and investigate other limiting factors in detail, with the aim of providing guidelines for achieving better utilisation of testers and testing practices.
|
|
| 32. |
- Chunduri, Annapurna, et al.
(författare)
-
An effective verification strategy for testing distributed automotive embedded software functions : A case study
- 2016
-
Ingår i: Lecture Notes in Computer Science. - Cham : Springer. - 9783319490939 ; , s. 233-248
-
Konferensbidrag (refereegranskat)abstract
- Integration testing of automotive embedded software functions that are distributed across several Electronic Control Unit (ECU) system software modules is a complex and challenging task in today’s automotive industry. They neither have infinite resources, nor have the time to carry out exhaustive testing of these functions. On the other hand, the traditional approach of implementing an ad-hoc selection of test scenarios based on the testers’ experience typically leads to both test gaps and test redundancies. Here, we address this challenge by proposing a verification strategy that enhances the process in order to identify and mitigate such gaps and redundancies in automotive system software testing. This helps increase test coverage by taking more data-driven decisions for integration testing of the functions. The strategy was developed in a case study at a Swedish automotive company that involved multiple data collection steps. After static validation of the proposed strategy it was evaluated on one distributed automotive software function, the Fuel Level Display, and found to be both feasible and effective. © Springer International Publishing AG 2016.
|
|
| 33. |
- Clark, David, et al.
(författare)
-
Information Transformation : An Underpinning Theory for Software Engineering
- 2015
-
Ingår i: 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, Vol 2. - : IEEE. - 9781479919345 ; , s. 599-602
-
Konferensbidrag (refereegranskat)abstract
- Software engineering lacks underpinning scientific theories both for the software it produces and the processes by which it does so. We propose that an approach based on information theory can provide such a theory, or rather many theories. We envision that such a benefit will be realised primarily through research based on the quantification of information involved and a mathematical study of the limiting laws that arise. However, we also argue that less formal but more qualitative uses for information theory will be useful. The main argument in support of our vision is based on the fact that both a program and an engineering process to develop such a program are fundamentally processes that transform information. To illustrate our argument we focus on software testing and develop an initial theory in which a test suite is input/output adequate if it achieves the channel capacity of the program as measured by the mutual information between its inputs and its outputs. We outline a number of problems, metrics and concrete strategies for improving software engineering, based on information theoretical analyses. We find it likely that similar analyses and subsequent future research to detail them would be generally fruitful for software engineering.
|
|
| 34. |
- de Oliveira Neto, Francisco Gomes, et al.
(författare)
-
Searching for models to evaluate software technology
- 2013
-
Ingår i: 2013 1st International Workshop on Combining Modelling and Search-Based Software Engineering, CMSBSE 2013 - Proceedings. - 9781467362849 ; , s. 12-15
-
Konferensbidrag (refereegranskat)abstract
- Modeling and abstraction is key in all engineering processes and have found extensive use also in software engineering. When developing new methodologies and techniques to support software engineers we want to evaluate them on realistic models. However, this is a challenge since (1) it is hard to get industry to give access to their models, and (2) we need a large number of models to systematically evaluate a technology. This paper proposes that search-based techniques can be used to search for models with desirable properties, which can then be used to systematically evaluate model-based technologies. By targeting properties seen in industrial models we can then get the best of both worlds: models that are similar to models used in industry but in quantities that allow extensive experimentation. To exemplify our ideas we consider a specific case in which a model generator is used to create models to test a regression test optimization technique. © 2013 IEEE.
|
|
| 35. |
- Dobslaw, Felix, 1983-, et al.
(författare)
-
Automated black-box boundary value detection
- 2023
-
Ingår i: PeerJ Computer Science. - : PeerJ. - 2376-5992. ; 9
-
Tidskriftsartikel (refereegranskat)abstract
- Abstract Software systems typically have an input domain that can be subdivided into sub-domains, each of which generates similar or related outputs. Testing it on the boundaries between these sub-domains is critical to ensure high-quality software. Therefore, boundary value analysis and testing have been a fundamental part of the software testing toolbox for a long time and are typically taught early to software engineering students. Despite its many argued benefits, boundary value analysis for a given software specification or application is typically described in abstract terms. This allows for variation in how testers apply it and in the benefits they see. Additionally, its adoption has been limited since it requires a specification or model to be analysed. We propose an automated black-box boundary value detection method to support software testers in performing systematic boundary value analysis. This dynamic method can be utilized even without a specification or model. The proposed method is based on a metric referred to as the program derivative, which quantifies the level of boundariness of test inputs. By combining this metric with search algorithms, we can identify and rank pairs of inputs as good boundary candidates, i.e., inputs that are in close proximity to each other but with outputs that are far apart. We have implemented the AutoBVA approach and evaluated it on a curated dataset of example programs. Furthermore, we have applied the approach broadly to a sample of 613 functions from the base library of the Julia programming language. The approach could identify boundary candidates that highlight diverse boundary behaviours in over 70% of investigated systems under test. The results demonstrate that even a simple variant of the program derivative, combined with broad sampling and search over the input space, can identify interesting boundary candidates for a significant portion of the functions under investigation. In conclusion, we also discuss the future extension of the approach to encompass more complex systems under test cases and datatypes.
|
|
| 36. |
- Dobslaw, Felix, et al.
(författare)
-
Automated black-box boundary value detection
- 2023
-
Ingår i: PeerJ Computer Science. - 2376-5992. ; 9
-
Tidskriftsartikel (refereegranskat)abstract
- Software systems typically have an input domain that can be subdivided into subdomains, each of which generates similar or related outputs. Testing it on the boundaries between these sub-domains is critical to ensure high-quality software. Therefore, boundary value analysis and testing have been a fundamental part of the software testing toolbox for a long time and are typically taught early to software engineering students. Despite its many argued benefits, boundary value analysis for a given software specification or application is typically described in abstract terms. This allows for variation in how testers apply it and in the benefits they see. Additionally, its adoption has been limited since it requires a specification or model to be analysed. We propose an automated black-box boundary value detection method to support software testers in performing systematic boundary value analysis. This dynamic method can be utilized even without a specification or model. The proposed method is based on a metric referred to as the program derivative, which quantifies the level of boundariness of test inputs. By combining this metric with search algorithms, we can identify and rank pairs of inputs as good boundary candidates, i.e., inputs that are in close proximity to each other but with outputs that are far apart. We have implemented the AutoBVA approach and evaluated it on a curated dataset of example programs. Furthermore, we have applied the approach broadly to a sample of 613 functions from the base library of the Julia programming language. The approach could identify boundary candidates that highlight diverse boundary behaviours in over 70% of investigated systems under test. The results demonstrate that even a simple variant of the program derivative, combined with broad sampling and search over the input space, can identify interesting boundary candidates for a significant portion of the functions under investigation. In conclusion, we also discuss the future extension of the approach to encompass more complex systems under test cases and datatypes.
|
|
| 37. |
- Dobslaw, Felix, et al.
(författare)
-
Automated black-box boundary value detection
- 2023
-
Ingår i: PeerJ Computer Science. - 2376-5992. ; 9
-
Tidskriftsartikel (refereegranskat)abstract
- Software systems typically have an input domain that can be subdivided into subdomains, each of which generates similar or related outputs. Testing it on the boundaries between these sub-domains is critical to ensure high-quality software. Therefore, boundary value analysis and testing have been a fundamental part of the software testing toolbox for a long time and are typically taught early to software engineering students. Despite its many argued benefits, boundary value analysis for a given software specification or application is typically described in abstract terms. This allows for variation in how testers apply it and in the benefits they see. Additionally, its adoption has been limited since it requires a specification or model to be analysed. We propose an automated black-box boundary value detection method to support software testers in performing systematic boundary value analysis. This dynamic method can be utilized even without a specification or model. The proposed method is based on a metric referred to as the program derivative, which quantifies the level of boundariness of test inputs. By combining this metric with search algorithms, we can identify and rank pairs of inputs as good boundary candidates, i.e., inputs that are in close proximity to each other but with outputs that are far apart. We have implemented the AutoBVA approach and evaluated it on a curated dataset of example programs. Furthermore, we have applied the approach broadly to a sample of 613 functions from the base library of the Julia programming language. The approach could identify boundary candidates that highlight diverse boundary behaviours in over 70% of investigated systems under test. The results demonstrate that even a simple variant of the program derivative, combined with broad sampling and search over the input space, can identify interesting boundary candidates for a significant portion of the functions under investigation. In conclusion, we also discuss the future extension of the approach to encompass more complex systems under test cases and datatypes.
|
|
| 38. |
- Dobslaw, Felix, 1983-, et al.
(författare)
-
Automated Boundary Identification for Machine Learning Classifiers
- 2024
-
Ingår i: The 17th Intl. Workshop on Search-Based and Fuzz Testing.
-
Konferensbidrag (refereegranskat)abstract
- AI and Machine Learning (ML) models are increasingly used as(critical) components in software systems, even safety-critical ones.This puts new demands on the degree to which we need to test themand requires new and expanded testing methods. Recent boundary-value identification methods have been developed and shown toautomatically find boundary candidates for traditional, non-MLsoftware: pairs of nearby inputs that result in (highly) differingoutputs. These can be shown to developers and testers, who canjudge if the boundary is where it is supposed to be.Here, we explore how this method can identify decision bound-aries of ML classification models. The resulting ML Boundary Span-ning Algorithm (ML-BSA) is a search-based method extendingprevious work in two main ways.We empirically evaluate ML-BSAon seven ML datasets and show that it better spans and thus bet-ter identifies the entire classification boundary(ies). The diversityobjective helps spread out the boundary pairs more broadly andevenly. This, we argue, can help testers and developers better judgewhere a classification boundary actually is, compare to expecta-tions, and then focus further testing, validation, and even furthertraining and model refinement on parts of the boundary wherebehaviour is not ideal.
|
|
| 39. |
- Dobslaw, Felix, 1983, et al.
(författare)
-
Boundary Value Exploration for Software Analysis
- 2020
-
Ingår i: Proceedings - 2020 IEEE 13th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2020. - : IEEE. ; , s. 346-353
-
Konferensbidrag (refereegranskat)abstract
- For software to be reliable and resilient, it is widely accepted that tests must be created and maintained alongside the software itself. One safeguard from vulnerabilities and failures in code is to ensure correct behavior on the boundaries between subdomains of the input space. So-called boundary value analysis (BVA) and boundary value testing (BVT) techniques aim to exercise those boundaries and increase test effectiveness. However, the concepts of BVA and BVT themselves are not generally well defined, and it is not clear how to identify relevant sub-domains, and thus the boundaries delineating them, given a specification. This has limited adoption and hindered automation. We clarify BVA and BVT and introduce Boundary Value Exploration (BVE) to describe techniques that support them by helping to detect and identify boundary inputs. Additionally, we propose two concrete BVE techniques based on information-theoretic distance functions: (i) an algorithm for boundary detection and (ii) the usage of software visualization to explore the behavior of the software under test and identify its boundary behavior. As an initial evaluation, we apply these techniques on a much used and well-tested date handling library. Our results reveal questionable behavior at boundaries highlighted by our techniques. In conclusion, we argue that the boundary value exploration that our techniques enable is a step towards automated boundary value analysis and testing, which can foster their wider use and improve test effectiveness and efficiency.
|
|
| 40. |
- Dobslaw, Felix, 1983, et al.
(författare)
-
Estimating Return on Investment for GUI Test Automation Frameworks
- 2019
-
Ingår i: Proceedings - International Symposium on Software Reliability Engineering, ISSRE. - 1071-9458. - 9781728149813 ; 2019-October
-
Konferensbidrag (refereegranskat)abstract
- Automated graphical user interface (GUI) tests can reduce manual testing activities and increase test frequency. This motivates the conversion of manual test cases into automated GUI tests. However, it is not clear whether such automation is cost-effective given that GUI automation scripts add to the code base and demand maintenance as a system evolves. In this paper, we introduce a method for estimating maintenance cost and Return on Investment (ROI) for Automated GUI Testing (AGT). The method utilizes the existing source code change history and has the potential to be used for the evaluation of other testing or quality assurance automation technologies. We evaluate the method for a real-world, industrial software system and compare two fundamentally different AGT frameworks, namely Selenium and EyeAutomate, to estimate and compare their ROI. We also report on their defect-finding capabilities and usability. The quantitative data is complemented by interviews with employees at the company the study has been conducted at. The method was successfully applied, and estimated maintenance cost and ROI for both frameworks are reported. Overall, the study supports earlier results showing that implementation time is the leading cost for introducing AGT. The findings further suggest that, while EyeAutomate tests are significantly faster to implement, Selenium tests require more of a programming background but less maintenance.
|
|
| 41. |
|
|
| 42. |
- Dobslaw, Felix, 1983-, et al.
(författare)
-
Similarities of Testing Programmed and Learnt Software
- 2023
-
Ingår i: Proceedings - 2023 IEEE 16th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2023. - : IEEE conference proceedings. ; , s. 78-81
-
Konferensbidrag (refereegranskat)abstract
- This study examines to what extent the testing of traditional software components and machine learning (ML) models fundamentally differs or not. While some researchers argue that ML software requires new concepts and perspectives for testing, our analysis highlights that, at a fundamental level, the specification and testing of a software component are not dependent on the development process used or on implementation details. Although the software engineering/computer science (SE/CS) and Data Science/ML (DS/ML) communities have developed different expectations, unique perspectives, and varying testing methods, they share clear commonalities that can be leveraged. We argue that both areas can learn from each other, and a non-dual perspective could provide novel insights not only for testing ML but also for testing traditional software. Therefore, we call upon researchers from both communities to collaborate more closely and develop testing methods and tools that can address both traditional and ML software components. While acknowledging their differences has merits, we believe there is great potential in working on unified methods and tools that can address both types of software.
|
|
| 43. |
|
|
| 44. |
- Dodig-Crnkovic, Gordana, et al.
(författare)
-
Professional and Ethical Issues of Software Engineering Curriculum Applied in Swedish Academic Context
- 2009
-
Konferensbidrag (refereegranskat)abstract
- The increasing dependence on computers for critical infrastructures essential for the functioning of a society and its economy has given rise to host of ethical, social, and legal issues. The ability to make sound ethical decisions is thus an important part of Computing and Software engineer’s professional skills. This paper argues for the significance of teaching professional, social and ethical issues in Software Engineering in a Swedish context and practice. Examples are presented of teaching materials and experiences from the course Professional Ethics in Science and Engineering at Malardalen University, and the PIFF project for support of Software Engineering Master Theses running at Malardalen University, Blekinge Institute of Technology and Lund University.
|
|
| 45. |
- Doganay, Kivanc
(författare)
-
Applications of Optimization Methods in Industrial Maintenance Scheduling and Software Testing
- 2014
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- As the world is getting more and more competitive, efficiency has become a bigger concern than ever for many businesses. Certain efficiency concerns can naturally be expressed as optimization problems, which is a well studied field in the academia. However, optimization algorithms are not as widely employed in industrial practice as they could. There are various reasons for the lack of widespread adoption. For example, it can be difficult or even impossible for non-experts to formulate a detailed mathematical model of the problem. On the other hand, a scientist usually does not have a deep enough understanding of critical business details, and may fail to capture enough details of the real- world phenomenon of concern. While a model at an arbitrary abstraction level is often good enough to demonstrate the optimization approach, ignoring relevant aspects can easily render the solution impractical for the industry. This is an important problem, because applicability concerns hinder the possible gains that can be achieved by using the academic knowledge in industrial practice. In this thesis, we study the challenges of industrial optimization problems in the form of four case studies at four different companies, in the domains of maintenance schedule optimization and search-based software testing. Working with multiple case studies in different domains allows us to better understand the possible gains and practical challenges in applying optimization methods in an industrial setting. Often there is a need to trade precision for applicability, which is typically very context dependent. Therefore, we compare our results against base values, e.g., results from simpler algorithms or the state of the practice in the given context, where applicable. Even though we cannot claim that optimization methods are applicable in all situations, our work serves as an empirical evidence for the usability of optimization methods for improvements in different industrial contexts. We hope that our work can encourage the adoption of optimization techniques by more industrial practitioners.
|
|
| 46. |
- Engström, Emelie, et al.
(författare)
-
Indirect effects in evidential assessment: A case study on regression test technology adoption
- 2012
-
Ingår i: 2nd International Workshop on Evidential Assessment of Software Technologies, EAST 2012. Lund, 22 September 2012. - New York, NY, USA : ACM. - 9781450315098 ; , s. 15-20
-
Konferensbidrag (refereegranskat)abstract
- Background: There is a need for effcient regression testing in most software development organizations. Often the proposed solutions involve automation. However, despite this being a well researched area, research results are rarely applied in industrial practice. Aim: In this paper we aim to bridge the gap between research and practice by providing examples of how evidence-based regression testing approaches can be adopted in industry. We also discuss challenges for the research community. Method: An industrial case study was carried out to evaluate the possibility to improve regression testing at Sony Ericsson Mobile Communications. We analyse the procedure undertaken based on frameworks from the evidence based software engineering, EBSE, paradigm (with a focus on the evidence) and automation literature (with a focus on the practical effects). Results: Our results pinpoint the need for systematic approaches when introducing a new tool. Practitioners and researchers need congruent guidelines supporting the appraisal of both the evidence base and the pragmatic effects, both direct but also indirect, of the changes. This is illustrated by the introduction of the automation perspective.
|
|
| 47. |
- Enoiu, Eduard Paul, et al.
(författare)
-
Mutation-based test generation for PLC embedded software using model checking
- 2016
-
Ingår i: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). - Cham : Springer. - 9783319474427 ; , s. 155-171
-
Konferensbidrag (refereegranskat)abstract
- Testing is an important activity in engineering of industrial embedded software. In certain application domains (e.g., railway industry) engineering software is certified according to safety standards that require extensive software testing procedures to be applied for the development of reliable systems. Mutation analysis is a technique for creating faulty versions of a software for the purpose of examining the fault detection ability of a test suite. Mutation analysis has been used for evaluating existing test suites, but also for generating test suites that detect injected faults (i.e., mutation testing). To support developers in software testing, we propose a technique for producing test cases using an automated test generation approach that operates using mutation testing for software written in IEC 61131-3 language, a programming standard for safety-critical embedded software, commonly used for Programmable Logic Controllers (PLCs). This approach uses the Uppaal model checker and is based on a combined model that contains all the mutants and the original program. We applied this approach in a tool for testing industrial PLC programs and evaluated it in terms of cost and fault detection. For realistic validation we collected industrial experimental evidence on how mutation testing compares with manual testing as well as automated decision-coverage adequate test generation. In the evaluation, we used manually seeded faults provided by four industrial engineers. The results show that even if mutation-based test generation achieves better fault detection than automated decision coverage-based test generation, these mutation-adequate test suites are not better at detecting faults than manual test suites. However, the mutation-based test suites are significantly less costly to create, in terms of testing time, than manually created test suites. Our results suggest that the fault detection scores could be improved by considering some new and improved mutation operators (e.g., Feedback Loop Insertion Operator (FIO)) for PLC programs as well as higher-order mutations.
|
|
| 48. |
- Enoiu, Eduard Paul, PhD, et al.
(författare)
-
Towards a Model of Testers' Cognitive Processes: Software Testing as a Problem Solving Approach
- 2020
-
Ingår i: Proceedings - Companion of the 2020 IEEE 20th International Conference on Software Quality, Reliability, and Security, QRS-C 2020. ; , s. 272-279
-
Konferensbidrag (refereegranskat)abstract
- Software testing is a complex, intellectual activity based (at least) on analysis, reasoning, decision making, abstraction and collaboration performed in a highly demanding environment. Naturally, it uses and allocates multiple cognitive resources in software testers. However, while a cognitive psychology perspective is increasingly used in the general software engineering literature, it has yet to find its place in software testing. To the best of our knowledge, no theory of software testers' cognitive processes exists. Here, we take the first step towards such a theory by presenting a cognitive model of software testing based on how problem solving is conceptualized in cognitive psychology. Our approach is to instantiate a general problem solving process for the specific problem of creating test cases. We then propose an experiment for testing our cognitive test design model. The experiment makes use of verbal protocol analysis to understand the mechanisms by which human testers choose, design, implement and evaluate test cases. An initial evaluation was then performed with five software engineering master students as subjects. The results support a problem solving-based model of test design for capturing testers' cognitive processes.
|
|
| 49. |
- Enoiu, Eduard Paul, PhD, et al.
(författare)
-
Towards Human-Like Automated Test Generation: Perspectives from Cognition and Problem Solving
- 2021
-
Ingår i: Proceedings - 2021 IEEE/ACM 13th International Workshop on Cooperative and Human Aspects of Software Engineering, CHASE 2021. - Virtual and Madrid, Spain. ; , s. 123-124
-
Konferensbidrag (refereegranskat)abstract
- Automated testing tools typically create test cases that are different from what human testers create. This often makes the tools less effective, the created tests harder to understand, and thus results in tools providing less support to human testers. Here, we propose a framework based on cognitive science and, in particular, an analysis of approaches to problem solving, for identifying cognitive processes of testers. The framework helps map test design steps and criteria used in human test activities and thus to better understand how effective human testers perform their tasks. Ultimately, our goal is to be able to mimic how humans create test cases and thus to design more human-like automated test generation systems. We posit that such systems can better augment and support testers in a way that is meaningful to them.
|
|
| 50. |
- Enoiu, Eduard Paul, PhD, et al.
(författare)
-
Understanding Problem Solving in Software Testing : An Exploration of Tester Routines and Behavior
- 2023
-
Ingår i: Lecture Notes Computer Science. - : Springer Science and Business Media Deutschland GmbH. - 9783031432392 ; 14131 LNCS, s. 143-159
-
Konferensbidrag (refereegranskat)abstract
- Software testing is a difficult, intellectual activity performed in a social environment. Naturally, testers use and allocate multiple cognitive resources towards this task. The goal of this study is to understand better the routine and behaviour of human testers and their mental models when performing testing. We investigate this topic by surveying 38 software testers and developers in Sweden. The survey explores testers’ cognitive processes when performing testing by investigating the knowledge they bring, the activities they select and perform, and the challenges they face in their routine. By analyzing the survey results, we provide a characterization of tester practices and identify insights regarding the problem-solving process. We use these descriptions to further enhance a cognitive model of software testing. © 2023, IFIP International Federation for Information Processing.
|
|
