| 1. |
- Heckmann Barbalho de Figueroa, Laiz, et al.
(författare)
-
A Modeling Approach for Bioinformatics Workflows
- 2019
-
Ingår i: The Practice of Enterprise Modeling - 12th {IFIP} Working Conference, PoEM 2019, Luxembourg, Luxembourg, November 27-29, 2019, Proceedings. - Cham : Springer.
-
Konferensbidrag (refereegranskat)
|
|
| 2. |
- Ahmad, Azeem, et al.
(författare)
-
A Multi-factor Approach for Flaky Test Detection and Automated Root Cause Analysis
- 2021
-
Ingår i: Proceedings - Asia-Pacific Software Engineering Conference, APSEC. - : IEEE COMPUTER SOC. - 1530-1362. ; , s. 338-348
-
Konferensbidrag (refereegranskat)abstract
- Developers often spend time to determine whether test case failures are real failures or flaky. The flaky tests, also known as non-deterministic tests, switch their outcomes without any modification in the codebase, hence reducing the confidence of developers during maintenance as well as in the quality of a product. Re-running test cases to reveal flakiness is resource-consuming, unreliable and does not reveal the root causes of test flakiness. Our paper evaluates a multi-factor approach to identify flaky test executions implemented in a tool named MDF laker. The four factors are: trace-back coverage, flaky frequency, number of test smells, and test size. Based on the extracted factors, MDFlaker uses k-Nearest Neighbor (KNN) to determine whether failed test executions are flaky. We investigate MDFlaker in a case study with 2166 test executions from different open-source repositories. We evaluate the effectiveness of our flaky detection tool. We illustrate how the multi-factor approach can be used to reveal root causes for flakiness, and we conduct a qualitative comparison between MDF laker and other tools proposed in literature. Our results show that the combination of different factors can be used to identify flaky tests. Each factor has its own trade-off, e.g., trace-back leads to many true positives, while flaky frequency yields more true negatives. Therefore, specific combinations of factors enable classification for testers with limited information (e.g., not enough test history information).
|
|
| 3. |
- Ahmad, Azeem, et al.
(författare)
-
An Industrial Study on the Challenges and Effects of Diversity-Based Testing in Continuous Integration
- 2023
-
Ingår i: IEEE International Conference on Software Quality, Reliability and Security, QRS. - 2693-9177. - 9798350319583
-
Konferensbidrag (refereegranskat)abstract
- Many test prioritisation techniques have been proposed in order to improve test effectiveness of Continuous Integration (CI) pipelines. Particularly, diversity-based testing (DBT) has shown promising and competitive results to improve test effectiveness. However, the technical and practical challenges of introducing test prioritisation in CI pipelines are rarely discussed, thus hindering the applicability and adoption of those proposed techniques. This research builds on our prior work in which we evaluated diversity-based techniques in an industrial setting. This work investigates the factors that influence the adoption of DBT both in connection to improvements in test cost-effectiveness, as well as the process and human related challenges to transfer and use DBT prioritisation in CI pipelines. We report on a case study considering the CI pipeline of Axis Communications in Sweden. We performed a thematic analysis of a focus group interview with senior practitioners at the company to identify the challenges and perceived benefits of using test prioritisation in their test process. Our thematic analysis reveals a list of ten challenges and seven perceived effects of introducing test prioritisation in CI cycles. For instance, our participants emphasized the importance of introducing comprehensible and transparent techniques that instill trust in its users. Moreover, practitioners prefer techniques compatible with their current test infrastructure (e.g., test framework and environments) in order to reduce instrumentation efforts and avoid disrupting their current setup. In conclusion, we have identified tradeoffs between different test prioritisation techniques pertaining to the technical, process and human aspects of regression testing in CI. We summarize those findings in a list of seven advantages that refer to specific stakeholder interests and describe the effects of adopting DBT in CI pipelines.
|
|
| 4. |
- Cvitic, Petya Hristova, et al.
(författare)
-
Investigating Software Testing and Maintenance of Open-Source Distributed Ledger
- 2023
-
Ingår i: Proceedings - 2023 IEEE International Conference on Software Analysis, Evolution and Reengineering, SANER 2023. - : IEEE conference proceedings. - 9781665452786 ; , s. 886-896
-
Konferensbidrag (refereegranskat)abstract
- A distributed ledger is the backbone of all blockchain solutions. It provides a shared database spreading across a network of nodes. The number of DL solutions and their implementations has grown in recent years. Besides the architectural and performance promises of these solutions, organizations seekingto implement DL also need to consider the overall quality of the software available and its ecosystem. Particularly, previous research has identified the need to better understand the testing and maintenance practices behind these types of technologies. This paper investigates the testing and maintenance of 18 different open-source projects that implement distributed ledgers. We perform a manual inspection of test artefacts and mine the history of commits, issues and contributors of the chosen projects to understand the landscape of testing and maintenance in these projects. Our findings suggest that unit and integration tests are present in most projects, they do not follow a holistic system testing approach. Moreover, projects rely on a small team of core contributors (5 on average). While the projects are continuously maintained, larger changes are uncommon. Our results can be used for benchmarking and pinpointing areas of improvement for the development of distributed ledgers.
|
|
| 5. |
- Cvitic, Petya Hristova, et al.
(författare)
-
Investigating Software Testing and Maintenance of Open-Source Distributed Ledger
- 2023
-
Ingår i: Proceedings - 2023 IEEE International Conference on Software Analysis, Evolution and Reengineering, SANER 2023. - : IEEE. - 2833-9061. - 9781665452786
-
Konferensbidrag (refereegranskat)abstract
- A distributed ledger is the backbone of all blockchain solutions. It provides a shared database spreading across a network of nodes. The number of DL solutions and their implementations has grown in recent years. Besides the architectural and performance promises of thesesolutions, organizations seekingto implement DL also need to consider the overall quality of the software available and its ecosystem. Particularly, previous research has identified the need to better understand the testing and maintenance practices behind these types of technologies. This paper investigates the testing and maintenance of 18 different open-source projects that implement distributed ledgers. We perform a manual inspection of test artefacts and mine the history of commits, issues and contributors of the chosen projects to understand the landscape of testing and maintenance in these projects. Our findings suggest that unit and integration tests are present in most projects, they do not follow a holistic system testing approach. Moreover, projects rely on a small team of core contributors (5 on average). While the projects are continuously maintained, larger changes are uncommon. Our results can be used for benchmarking and pinpointing areas of improvement for the development of distributed ledgers.
|
|
| 6. |
- de Oliveira Neto, Francisco Gomes, et al.
(författare)
-
Challenges of Aligning Requirements Engineering and System Testing in Large-Scale Agile: A Multiple Case Study
- 2017
-
Ingår i: 2017 IEEE 25th International Requirements Engineering Conference Workshops (REW), 4-8 Sept. 2017, Lisbon, Portugal. - : IEEE. - 9781538634882
-
Konferensbidrag (refereegranskat)abstract
- Abstract: As agile methods become more pervasive, agile practices are applied to more large-scale systems with a scope that goes beyond pure software. The expansion of agile in these contexts provides benefits, but creates new challenges. Widespread use of agile has changed the way we must think about practices both in Requirements Engineering (RE) and in System Testing (ST). Our experience shows that many challenges in the application of large-scale agile development relate to either RE or ST, and in particular to the alignment between these areas. In this paper we present large-scale agile-related challenges from a multiple case study which relate to REST alignment. We map our challenges to an existing framework for REST alignment, and make an initial attempt to suggest agile RE practices from the literature which may alleviate these challenges. Our results show that the interviewed companies need to first adopt more agile RE practices to enhance REST alignment and then leverage agile testing. Future work will look more towards evaluating these best practices.
|
|
| 7. |
- de Oliveira Neto, Francisco Gomes, et al.
(författare)
-
Full modification coverage through automatic similarity-based test case selection
- 2016
-
Ingår i: Information and Software Technology. - : Elsevier BV. - 0950-5849. ; 80, s. 124-137
-
Tidskriftsartikel (refereegranskat)abstract
- Context: This paper presents the similarity approach for regression testing (SART), where a similarity-based test case selection technique is used in a model-based testing process to provide selection of test cases exercising modified parts of a specification model. Unlike other model-based regression testing techniques, SART relies on similarity analysis among test cases to identify modifications, instead of comparing models, hence reducing the dependency on specific types of model. Objective: To present convincing evidence of the usage of similarity measures for modification-traversing test case selection. Method: We investigate SART in a case study and an experiment. The case study uses artifacts from industry and should be seen as a sanity check of SART, while the experiment focuses on gaining statistical power through the generation of synthetical models in order to provide convincing evidence of SART’s effectiveness. Through posthoc analysis we obtain p-values and effect sizes to observe statistically significant differences between treatments with respect to transition and modification coverage. Results: The case study with industrial artifacts revealed that SART is able to uncover the same number of defects as known similarity-based test case selection techniques. In turn, the experiment shows that SART, unlike the other investigated techniques, presents 100% modification coverage. In addition, all techniques covered a similar percentage of model transitions. Conclusions: In summary, not only does SART provide transition and defect coverage equal to known STCS techniques, but it exceeds greatly in covering modified parts of the specification model, being a suitable candidate for model-based regression testing. Keywords: Regression testing, Test case selection, Model-based testing, Experimental Study
|
|
| 8. |
- de Oliveira Neto, Francisco Gomes, et al.
(författare)
-
Improving continuous integration with similarity-based test case selection
- 2018
-
Ingår i: Proceedings of the 13th International Workshop on Automation of Software Test. - New York : ACM Digital Library. - 0270-5257. - 9781450357432 ; , s. 39-45
-
Konferensbidrag (refereegranskat)abstract
- Automated testing is an essential component of Continuous Integration (CI) and Delivery (CD), such as scheduling automated test sessions on overnight builds. That allows stakeholders to execute entire test suites and achieve exhaustive test coverage, since running all tests is often infeasible during work hours, i.e., in parallel to development activities. On the other hand, developers also need test feedback from CI servers when pushing changes, even if not all test cases are executed. In this paper we evaluate similarity-based test case selection (SBTCS) on integration-level tests executed on continuous integration pipelines of two companies. We select test cases that maximise diversity of test coverage and reduce feedback time to developers. Our results confirm existing evidence that SBTCS is a strong candidate for test optimisation, by reducing feedback time (up to 92% faster in our case studies) while achieving full test coverage using only information from test artefacts themselves.
|
|
| 9. |
- de Oliveira Neto, Francisco Gomes, et al.
(författare)
-
Searching for models to evaluate software technology
- 2013
-
Ingår i: 2013 1st International Workshop on Combining Modelling and Search-Based Software Engineering, CMSBSE 2013 - Proceedings. - 9781467362849 ; , s. 12-15
-
Konferensbidrag (refereegranskat)abstract
- Modeling and abstraction is key in all engineering processes and have found extensive use also in software engineering. When developing new methodologies and techniques to support software engineers we want to evaluate them on realistic models. However, this is a challenge since (1) it is hard to get industry to give access to their models, and (2) we need a large number of models to systematically evaluate a technology. This paper proposes that search-based techniques can be used to search for models with desirable properties, which can then be used to systematically evaluate model-based technologies. By targeting properties seen in industrial models we can then get the best of both worlds: models that are similar to models used in industry but in quantities that allow extensive experimentation. To exemplify our ideas we consider a specific case in which a model generator is used to create models to test a regression test optimization technique. © 2013 IEEE.
|
|
| 10. |
- de Oliveira Neto, Francisco Gomes, et al.
(författare)
-
Visualisation to Support Fault Localisation in Distributed Embedded Systems within the Automotive Industry
- 2018
-
Ingår i: 2018 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW). - : IEEE. - 9781538694435
-
Konferensbidrag (refereegranskat)abstract
- This paper presents the design, development and evaluation of a software tool to assist the localisation of root causes of test case failures in distributed embedded systems, specifically vehicle systems controlled by a network of electronic control units (ECUs). We use data visualising to provide sensible information from a large number of test execution logs from large-scale software integration testing under a continuous integration process. Our goal is to allow more efficient root-cause identification of failures and foster a continuous feedback loop in the fault localisation process. We evaluate our solution in-situ at the Research and Development division of Volvo Car Corporation (VCC). Our prototype helps the failure debugging procedures by presenting clear and concise data and by allowing stakeholders to filter and control which information is displayed. Moreover, it encourages a systematic and continuous analysis of the current state of testing by aggregating and categorising historical data from test harnesses to identify patterns and trends in test results.
|
|
