| 1. |
- Bjarnason, Elizabeth, et al.
(författare)
-
A Multi-Case Study of Agile Requirements Engineering and the Use of Test Cases as Requirements
- 2016. - 16
-
Ingår i: Information and Software Technology. - : Elsevier. - 0950-5849 .- 1873-6025. ; 77, s. 61-79
-
Tidskriftsartikel (refereegranskat)abstract
- [Context] It is an enigma that agile projects can succeed "without requirements" when weak requirements engineering is a known cause for project failures. While agile development projects often manage well without extensive requirements test cases are commonly viewed as requirements and detailed requirements are documented as test cases. [Objective] We have investigated this agile practice of using test cases as requirements to understand how test cases can support the main requirements activities, and how this practice varies. [Method] We performed an iterative case study at three companies and collected data through 14 interviews and 2 focus groups. [Results] The use of test cases as requirements poses both benefits and challenges when eliciting, validating, verifying, and managing requirements, and when used as a documented agreement. We have identified five variants of the test-cases-as-requirements practice, namely de facto, behaviour-driven, story-test driven, stand-alone strict and stand-alone manual for which the application of the practice varies concerning the time frame of requirements documentation, the requirements format, the extent to which the test cases are a machine executable specification and the use of tools which provide specific support for the practice of using test cases as requirements. [Conclusions] The findings provide empirical insight into how agile development projects manage and communicate requirements. The identified variants of the practice of using test cases as requirements can be used to perform in-depth investigations into agile requirements engineering. Practitioners can use the provided recommendations as a guide in designing and improving their agile requirements practices based on project characteristics such as number of stakeholders and rate of change.
|
|
| 2. |
- Jones, Robert P., et al.
(författare)
-
Patterns of Recurrence After Resection of Pancreatic Ductal Adenocarcinoma : A Secondary Analysis of the ESPAC-4 Randomized Adjuvant Chemotherapy Trial
- 2019
-
Ingår i: JAMA Surgery. - : AMER MEDICAL ASSOC. - 2168-6254 .- 2168-6262. ; 154:11, s. 1038-1048
-
Tidskriftsartikel (refereegranskat)abstract
- Importance: The patterns of disease recurrence after resection of pancreatic ductal adenocarcinoma with adjuvant chemotherapy remain unclear.Objective: To define patterns of recurrence after adjuvant chemotherapy and the association with survival.Design, Setting, and Participants: Prospectively collected data from the phase 3 European Study Group for Pancreatic Cancer 4 adjuvant clinical trial, an international multicenter study. The study included 730 patients who had resection and adjuvant chemotherapy for pancreatic cancer. Data were analyzed between July 2017 and May 2019.Interventions: Randomization to adjuvant gemcitabine or gemcitabine plus capecitabine.Main Outcomes and Measures: Overall survival, recurrence, and sites of recurrence.Results: Of the 730 patients, median age was 65 years (range 37-81 years), 414 were men (57%), and 316 were women (43%). The median follow-up time from randomization was 43.2 months (95% CI, 39.7-45.5 months), with overall survival from time of surgery of 27.9 months (95% CI, 24.8-29.9 months) with gemcitabine and 30.2 months (95% CI, 25.8-33.5 months) with the combination (HR, 0.81; 95% CI, 0.68-0.98; P=.03). The 5-year survival estimates were 17.1% (95% CI, 11.6%-23.5%) and 28.0% (22.0%-34.3%), respectively. Recurrence occurred in 479 patients (65.6%); another 78 patients (10.7%) died without recurrence. Local recurrence occurred at a median of 11.63 months (95% CI, 10.05-12.19 months), significantly different from those with distant recurrence with a median of 9.49 months (95% CI, 8.44-10.71 months) (HR, 1.21; 95% CI, 1.01-1.45; P=.04). Following recurrence, the median survival was 9.36 months (95% CI, 8.08-10.48 months) for local recurrence and 8.94 months (95% CI, 7.82-11.17 months) with distant recurrence (HR, 0.89; 95% CI, 0.73-1.09; P=.27). The median overall survival of patients with distant-only recurrence (23.03 months; 95% CI, 19.55-25.85 months) or local with distant recurrence (23.82 months; 95% CI, 17.48-28.32 months) was not significantly different from those with only local recurrence (24.83 months; 95% CI, 22.96-27.63 months) (P=.85 and P=.35, respectively). Gemcitabine plus capecitabine had a 21% reduction of death following recurrence compared with monotherapy (HR, 0.79; 95% CI, 0.64-0.98; P=.03).Conclusions and Relevance: There were no significant differences between the time to recurrence and subsequent and overall survival between local and distant recurrence. Pancreatic cancer behaves as a systemic disease requiring effective systemic therapy after resection.Trial Registration: ClinicalTrials.gov identifier: NCT00058201, EudraCT 2007-004299-38, and ISRCTN 96397434. This secondary analysis of a randomized clinical trial investigates patterns of recurrence after adjuvant chemotherapy in pancreatic cancer and the association with survival.
|
|
| 3. |
- Borg, Martin, et al.
(författare)
-
Supporting Change Impact Analysis Using a Recommendation System : An Industrial Case Study in a Safety-Critical Context
- 2017
-
Ingår i: IEEE Transactions on Software Engineering. - : IEEE. - 0098-5589 .- 1939-3520. ; 43:7, s. 675-700
-
Tidskriftsartikel (refereegranskat)abstract
- Change Impact Analysis (CIA) during software evolution of safety-critical systems is a labor-intensive task. Several authors have proposed tool support for CIA, but very few tools were evaluated in industry. We present a case study on ImpRec, a recommendation System for Software Engineering (RSSE), tailored for CIA at a process automation company. ImpRec builds on assisted tracing, using information retrieval solutions and mining software repositories to recommend development artifacts, potentially impacted when resolving incoming issue reports. In contrast to the majority of tools for automated CIA, ImpRec explicitly targets development artifacts that are not source code. We evaluate ImpRec in a two-phase study. First, we measure the correctness of ImpRec's recommendations by a simulation based on 12 years' worth of issue reports in the company. Second, we assess the utility of working with ImpRec by deploying the RSSE in two development teams on different continents. The results suggest that ImpRec presents about 40 percent of the true impact among the top-10 recommendations. Furthermore, user log analysis indicates that ImpRec can support CIA in industry, and developers acknowledge the value of ImpRec in interviews. In conclusion, our findings show the potential of reusing traceability associated with developers' past activities in an RSSE.
|
|
| 4. |
- Helali Moghadam, Mahshid, et al.
(författare)
-
Adaptive Runtime Response Time Control in PLC-based Real-Time Systems using Reinforcement Learning
- 2018
-
Ingår i: ACM/IEEE 13th International Symposium on Software Engineering for Adaptive and Self-Managing Systems, SEAMS 2018, , co-located with International Conference on Software Engineering, ICSE 2018; Gothenburg; Sweden; 28 May 2018 through 29 May 2018; Code 138312. - New York, NY, USA : ACM. ; , s. 217-223
-
Konferensbidrag (refereegranskat)abstract
- Timing requirements such as constraints on response time are key characteristics of real-time systems and violations of these requirements might cause a total failure, particularly in hard real-time systems. Runtime monitoring of the system properties is of great importance to detect and mitigate such failures. Thus, a runtime control to preserve the system properties could improve the robustness of the system with respect to timing violations. Common control approaches may require a precise analytical model of the system which is difficult to be provided at design time. Reinforcement learning is a promising technique to provide adaptive model-free control when the environment is stochastic, and the control problem could be formulated as a Markov Decision Process. In this paper, we propose an adaptive runtime control using reinforcement learning for real-time programs based on Programmable Logic Controllers (PLCs), to meet the response time requirements. We demonstrate through multiple experiments that our approach could control the response time efficiently to satisfy the timing requirements.
|
|
| 5. |
- Helali Moghadam, Mahshid, et al.
(författare)
-
An autonomous performance testing framework using self-adaptive fuzzy reinforcement learning
- 2022
-
Ingår i: Software quality journal. - : Springer. - 0963-9314 .- 1573-1367. ; , s. 127-159
-
Tidskriftsartikel (refereegranskat)abstract
- Test automation brings the potential to reduce costs and human effort, but several aspects of software testing remain challenging to automate. One such example is automated performance testing to find performance breaking points. Current approaches to tackle automated generation of performance test cases mainly involve using source code or system model analysis or use-case-based techniques. However, source code and system models might not always be available at testing time. On the other hand, if the optimal performance testing policy for the intended objective in a testing process instead could be learned by the testing system, then test automation without advanced performance models could be possible. Furthermore, the learned policy could later be reused for similar software systems under test, thus leading to higher test efficiency. We propose SaFReL, a self-adaptive fuzzy reinforcement learning-based performance testing framework. SaFReL learns the optimal policy to generate performance test cases through an initial learning phase, then reuses it during a transfer learning phase, while keeping the learning running and updating the policy in the long term. Through multiple experiments in a simulated performance testing setup, we demonstrate that our approach generates the target performance test cases for different programs more efficiently than a typical testing process and performs adaptively without access to source code and performance models. © 2021, The Author(s).
|
|
| 6. |
- Helali Moghadam, Mahshid
(författare)
-
Intelligence-Driven Software Performance Assurance
- 2022
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Software performance assurance is of great importance for the success of software products, which are nowadays involved in many parts of our life. Performance evaluation approaches such as performance modeling, testing, as well as runtime performance control methods, all can contribute to the realization of software performance assurance. Many of the common approaches to tackle challenges in this area involve relying on performance models or using system models and source code. Although modeling provides a deep insight into the system behavior, developing a detailed model is challenging. Furthermore, software artifacts such as models and source code might not be readily available at all times in the development lifecycle. This thesis focuses on leveraging the potential of machine learning (ML) and evolutionary search-based techniques to provide viable solutions for addressing the challenges in different aspects of software performance assurance efficiently and effectively.In this thesis, we first investigate the capabilities of model-free reinforcement learning to address the objectives in robustness testing problems. We develop two self-adaptive reinforcement learning-driven test agents called SaFReL and RELOAD. They generate effective platform-based test scenarios and test workloads, respectively. The output scenarios and workloads help testers and software engineers meet their objectives efficiently without relying on models or source code. SaFReL and RELOAD learn the optimal policies (ways) to meet the test objectives and can reuse the learned policies adaptively in other testing settings. Policy reuse can lead to higher test efficiency and cost savings, for example, when testing similar test objectives or software systems with comparable performance sensitivity.Next, we leverage the potential of evolutionary computation algorithms, i.e., genetic algorithms, evolution strategies, and particle swarm optimization, to generate failure-revealing test scenarios for robustness testing of AI systems. In this part, we choose autonomous driving systems as a prevailing example of contemporary AI systems. We study the efficacy of the proposed evolutionary search-based test generation techniques and evaluate primarily to what extent they can trigger failures. Moreover, we investigate the diversity of those failures and compare them to existing baseline solutions. Finally, we again use the potential of model-free reinforcement learning to develop adaptive ML-driven runtime performance control approaches. We present a response time preservation method for a sample type of industrial applications and a resource allocation technique for dynamic workloads in a data grid application. The proposed ML-driven techniques learn how to adjust the tunable parameters and resource configuration at runtime to keep the performance continually compliant with the requirements and to further optimize the runtime performance. We evaluate the efficacy of the approaches and show how effectively they can improve the performance and keep the performance requirements satisfied under varying conditions such as dynamic workloads and the occurrence of runtime events that lead to substantial response time deviations.
|
|
| 7. |
- Helali Moghadam, Mahshid, et al.
(författare)
-
Intelligent Load Testing: Self-adaptive Reinforcement Learning-driven Load Runner
-
Annan publikation (övrigt vetenskapligt/konstnärligt)abstract
- Load testing with the aim of generating an effective workload to identify performance issues is a time-consuming and complex challenge, particularly for evolving software systems. Current automated approaches mainly rely on analyzing system models and source code, or modeling of the real system usage. However, that information might not be available all the time or obtaining it might require considerable effort. On the other hand, if the optimal policy for generating the proper test workload resulting in meeting the objectives of the testing can be learned by the testing system, testing would be possible without access to system models or source code. We propose a self-adaptive reinforcement learning-driven load testing agent that learns the optimal policy for test workload generation. The agent can reuse the learned policy in subsequent testing activities such as meeting different types of testing targets. It generates an efficient test workload resulting in meeting the objective of the testing adaptively without access to system models or source code. Our experimental evaluation shows that the proposed self-adaptive intelligent load testing can reach the testing objective with lower cost in terms of the workload size, i.e. the number of generated users, compared to a typical load testing process, and results in productivity benefits in terms of higher efficiency.
|
|
| 8. |
- Helali Moghadam, Mahshid, et al.
(författare)
-
Learning-based Response Time Analysis in Real-Time Embedded Systems : A Simulation-based Approach
- 2018
-
Ingår i: 1st International Workshop on Software Qualities and their Dependencies, located at the International Conference of Software Engineering (ICSE) 2018 SQUADE'18. - New York, NY, USA : ACM. - 9781450357371 ; , s. 21-24
-
Konferensbidrag (refereegranskat)abstract
- Response time analysis is an essential task to verify the behavior of real-time systems. Several response time analysis methods have been proposed to address this challenge, particularly for real-time systems with different levels of complexity. Static analysis is a popular approach in this context, but its practical applicability is limited due to the high complexity of the industrial real-time systems, as well as many unpredictable runtime events in these systems. In this work-in-progress paper, we propose a simulationbased response time analysis approach using reinforcement learning to find the execution scenarios leading to the worst-case response time. The approach learns how to provide a practical estimation of the worst-case response time through simulating the program without performing static analysis. Our initial study suggests that the proposed approach could be applicable in the simulation environments of the industrial real-time control systems to provide a practical estimation of the execution scenarios leading to the worst-case response time.
|
|
| 9. |
- Helali Moghadam, Mahshid, et al.
(författare)
-
Learning-Based Self-Adaptive Assurance of Timing Properties in a Real-Time Embedded System
- 2018
-
Ingår i: ICST Workshop on Testing Extra-Functional Properties and Quality Characteristics of Software Systems ITEQS'18. - 9781538663523 ; , s. 77-80
-
Konferensbidrag (refereegranskat)abstract
- Providing an adaptive runtime assurance technique to meet the performance requirements of a real-time system without the need for a precise model could be a challenge. Adaptive performance assurance based on monitoring the status of timing properties can bring more robustness to the underlying platform. At the same time, the results or the achieved policy of this adaptive procedure could be used as feedback to update the initial model, and consequently for producing proper test cases. Reinforcement-learning has been considered as a promising adaptive technique for assuring the satisfaction of the performance properties of software-intensive systems in recent years. In this work-in-progress paper, we propose an adaptive runtime timing assurance procedure based on reinforcement learning to satisfy the performance requirements in terms of response time. The timing control problem is formulated as a Markov Decision Process and the details of applying the proposed learning-based timing assurance technique are described.
|
|
| 10. |
- Helali Moghadam, Mahshid
(författare)
-
Machine Learning-Assisted Performance Assurance
- 2020
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- With the growing involvement of software systems in our life, assurance of performance, as an important quality characteristic, rises to prominence for the success of software products. Performance testing, preservation, and improvement all contribute to the realization of performance assurance. Common approaches to tackle challenges in testing, preservation, and improvement of performance mainly involve techniques relying on performance models or using system models or source code. Although modeling provides a deep insight into the system behavior, drawing a well-detailed model is challenging. On the other hand, those artifacts such as models and source code might not be available all the time. These issues are the motivations for using model-free machine learning techniques such as model-free reinforcement learning to address the related challenges in performance assurance.Reinforcement learning implies that if the optimal policy (way) for achieving the intended objective in a performance assurance process could instead be learnt by the acting system (e.g., the tester system), then the intended objective could be accomplished without advanced performance models. Furthermore, the learnt policy could later be reused in similar situations, which leads to efficiency improvement by saving computation time while reducing the dependency on the models and source code.In this thesis, our research goal is to develop adaptive and efficient performance assurance techniques meeting the intended objectives without access to models and source code. We propose three model-free learning-based approaches to tackle the challenges; efficient generation of performance test cases, runtime performance (response time) preservation, and performance improvement in terms of makespan (completion time) reduction. We demonstrate the efficiency and adaptivity of our approaches based on experimental evaluations conducted on the research prototype tools, i.e. simulation environments that we developed or tailored for our problems, in different application areas.
|
|
