| 1. |
- Heckmann Barbalho de Figueroa, Laiz, et al.
(author)
-
A Modeling Approach for Bioinformatics Workflows
- 2019
-
In: The Practice of Enterprise Modeling - 12th {IFIP} Working Conference, PoEM 2019, Luxembourg, Luxembourg, November 27-29, 2019, Proceedings. - Cham : Springer.
-
Conference paper (peer-reviewed)
|
|
| 2. |
- de Oliveira Neto, Francisco Gomes, et al.
(author)
-
Challenges of Aligning Requirements Engineering and System Testing in Large-Scale Agile: A Multiple Case Study
- 2017
-
In: 2017 IEEE 25th International Requirements Engineering Conference Workshops (REW), 4-8 Sept. 2017, Lisbon, Portugal. - : IEEE. - 9781538634882
-
Conference paper (peer-reviewed)abstract
- Abstract: As agile methods become more pervasive, agile practices are applied to more large-scale systems with a scope that goes beyond pure software. The expansion of agile in these contexts provides benefits, but creates new challenges. Widespread use of agile has changed the way we must think about practices both in Requirements Engineering (RE) and in System Testing (ST). Our experience shows that many challenges in the application of large-scale agile development relate to either RE or ST, and in particular to the alignment between these areas. In this paper we present large-scale agile-related challenges from a multiple case study which relate to REST alignment. We map our challenges to an existing framework for REST alignment, and make an initial attempt to suggest agile RE practices from the literature which may alleviate these challenges. Our results show that the interviewed companies need to first adopt more agile RE practices to enhance REST alignment and then leverage agile testing. Future work will look more towards evaluating these best practices.
|
|
| 3. |
- de Oliveira Neto, Francisco Gomes, et al.
(author)
-
Full modification coverage through automatic similarity-based test case selection
- 2016
-
In: Information and Software Technology. - : Elsevier BV. - 0950-5849. ; 80, s. 124-137
-
Journal article (peer-reviewed)abstract
- Context: This paper presents the similarity approach for regression testing (SART), where a similarity-based test case selection technique is used in a model-based testing process to provide selection of test cases exercising modified parts of a specification model. Unlike other model-based regression testing techniques, SART relies on similarity analysis among test cases to identify modifications, instead of comparing models, hence reducing the dependency on specific types of model. Objective: To present convincing evidence of the usage of similarity measures for modification-traversing test case selection. Method: We investigate SART in a case study and an experiment. The case study uses artifacts from industry and should be seen as a sanity check of SART, while the experiment focuses on gaining statistical power through the generation of synthetical models in order to provide convincing evidence of SART’s effectiveness. Through posthoc analysis we obtain p-values and effect sizes to observe statistically significant differences between treatments with respect to transition and modification coverage. Results: The case study with industrial artifacts revealed that SART is able to uncover the same number of defects as known similarity-based test case selection techniques. In turn, the experiment shows that SART, unlike the other investigated techniques, presents 100% modification coverage. In addition, all techniques covered a similar percentage of model transitions. Conclusions: In summary, not only does SART provide transition and defect coverage equal to known STCS techniques, but it exceeds greatly in covering modified parts of the specification model, being a suitable candidate for model-based regression testing. Keywords: Regression testing, Test case selection, Model-based testing, Experimental Study
|
|
| 4. |
- de Oliveira Neto, Francisco Gomes, et al.
(author)
-
Improving continuous integration with similarity-based test case selection
- 2018
-
In: Proceedings of the 13th International Workshop on Automation of Software Test. - New York : ACM Digital Library. - 0270-5257. - 9781450357432 ; , s. 39-45
-
Conference paper (peer-reviewed)abstract
- Automated testing is an essential component of Continuous Integration (CI) and Delivery (CD), such as scheduling automated test sessions on overnight builds. That allows stakeholders to execute entire test suites and achieve exhaustive test coverage, since running all tests is often infeasible during work hours, i.e., in parallel to development activities. On the other hand, developers also need test feedback from CI servers when pushing changes, even if not all test cases are executed. In this paper we evaluate similarity-based test case selection (SBTCS) on integration-level tests executed on continuous integration pipelines of two companies. We select test cases that maximise diversity of test coverage and reduce feedback time to developers. Our results confirm existing evidence that SBTCS is a strong candidate for test optimisation, by reducing feedback time (up to 92% faster in our case studies) while achieving full test coverage using only information from test artefacts themselves.
|
|
| 5. |
- de Oliveira Neto, Francisco Gomes, et al.
(author)
-
Visualisation to Support Fault Localisation in Distributed Embedded Systems within the Automotive Industry
- 2018
-
In: 2018 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW). - : IEEE. - 9781538694435
-
Conference paper (peer-reviewed)abstract
- This paper presents the design, development and evaluation of a software tool to assist the localisation of root causes of test case failures in distributed embedded systems, specifically vehicle systems controlled by a network of electronic control units (ECUs). We use data visualising to provide sensible information from a large number of test execution logs from large-scale software integration testing under a continuous integration process. Our goal is to allow more efficient root-cause identification of failures and foster a continuous feedback loop in the fault localisation process. We evaluate our solution in-situ at the Research and Development division of Volvo Car Corporation (VCC). Our prototype helps the failure debugging procedures by presenting clear and concise data and by allowing stakeholders to filter and control which information is displayed. Moreover, it encourages a systematic and continuous analysis of the current state of testing by aggregating and categorising historical data from test harnesses to identify patterns and trends in test results.
|
|
| 6. |
- Dobslaw, F., et al.
(author)
-
Estimating Return on Investment for GUI Test Automation Frameworks
- 2019
-
In: 2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE). - : IEEE. - 9781728149820
-
Conference paper (peer-reviewed)abstract
- Automated graphical user interface (GUI) tests can reduce manual testing activities and increase test frequency. This motivates the conversion of manual test cases into automated GUI tests. However, it is not clear whether such automation is cost-effective given that GUI automation scripts add to the code base and demand maintenance as a system evolves. In this paper, we introduce a method for estimating maintenance cost and Return on Investment (ROI) for Automated GUI Testing (AGT). The method utilizes the existing source code change history and has the potential to be used for the evaluation of other testing or quality assurance automation technologies. We evaluate the method for a real-world, industrial software system and compare two fundamentally different AGT frameworks, namely Selenium and EyeAutomate, to estimate and compare their ROI. We also report on their defect-finding capabilities and usability. The quantitative data is complemented by interviews with employees at the company the study has been conducted at. The method was successfully applied, and estimated maintenance cost and ROI for both frameworks are reported. Overall, the study supports earlier results showing that implementation time is the leading cost for introducing AGT. The findings further suggest that, while EyeAutomate tests are significantly faster to implement, Selenium tests require more of a programming background but less maintenance.
|
|
| 7. |
|
|
| 8. |
|
|
| 9. |
- Felderer, Michael, 1978-, et al.
(author)
-
A testability analysis framework for non-functional properties
- 2018
-
In: 2018 IEEE 11TH INTERNATIONAL CONFERENCE ON SOFTWARE TESTING, VERIFICATION AND VALIDATION WORKSHOPS (ICSTW). - : Institute of Electrical and Electronics Engineers Inc.. - 9781538663523 ; , s. 54-58
-
Conference paper (peer-reviewed)abstract
- This paper presents background, the basic steps and an example for a testability analysis framework for non-functional properties.
|
|
| 10. |
- Stikkolorum, Dave, et al.
(author)
-
Evaluating didactic approaches used by teaching assistants for software analysis and design using UML
- 2018
-
In: Proceedings of the 3rd European Conference of Software Engineering Education. - New York, NY, USA : ACM. - 9781450363839
-
Conference paper (peer-reviewed)abstract
- © 2018 Copyright held by the owner/author(s). Teaching assistants (TAs) are employed for supporting various activities of a course, from assisting with course material, grading and, predominantly, bridging the interaction between teachers and students. This role is even more significant in large courses with dozens of students, particularly during lab sessions with practical assignments. Therefore, here we explore how does the role of the TA support students to overcome the challenges they face when learning software design. During a bachelor course on Analysis and Design, we collected data from i) weekly interviews with TAs, ii) student's assignment hand-ins and iii) the corresponding feedback reports written by TAs to students. By analysing these three different sources we propose guidelines and practices for effective deployment of TAs.
|
|
| 11. |
- Torkar, Richard, 1971, et al.
(author)
-
Statistical and practical significance of empirical software engineering research: A maturity model
- 2017
-
Journal article (other academic/artistic)abstract
- Software engineering research is maturing and papers increasingly support their arguments with empirical data from a multitude of sources, using statistical tests to judge if and to what degree empirical evidence supports their hypotheses. This paper presents trends and current state of art regarding statistical maturity in empirical software engineering research; the objective being the development of a statistical maturity model. First, we manually reviewed papers from four well-known and top ranked journals producing a review protocol along with the view of current (2015) state of art concerning statistical maturity, practical significance and reproducibility of empirical software engineering research. Our protocol was then used as ground truth, i.e., training set, for a semi-automatic classification of studies for the years 2001--2015 using a total of 3,011 papers. We used the extracted data to develop a statistical maturity model which also includes a model for how to argue for practical significance. The statistical maturity of empirical software engineering research has an upward trend in certain areas (e.g., use of nonparametric statistics, but also more generally in the usage of quantitative analysis). However, we also see how our research area currently often fails to connect the statistical analysis to practical significance. For instance, we argue that conclusions should explicitly state contributions to software engineering practice, e.g., the return on investment for practitioners. We argue that the statistical maturity model can be used by researchers and practitioners to build a coherent statistical analysis and guide them in the choice of statistical approaches of its steps. The final goal for a researcher would be to, in a clearer way, present and argue for the practical significance of their findings. Bayesian analysis, we believe, has a role to play in this.
|
|
