| 1. |
- Bjarnason, Elizabeth, et al.
(författare)
-
A Multi-Case Study of Agile Requirements Engineering and the Use of Test Cases as Requirements
- 2016. - 16
-
Ingår i: Information and Software Technology. - : Elsevier. - 0950-5849 .- 1873-6025. ; 77, s. 61-79
-
Tidskriftsartikel (refereegranskat)abstract
- [Context] It is an enigma that agile projects can succeed "without requirements" when weak requirements engineering is a known cause for project failures. While agile development projects often manage well without extensive requirements test cases are commonly viewed as requirements and detailed requirements are documented as test cases. [Objective] We have investigated this agile practice of using test cases as requirements to understand how test cases can support the main requirements activities, and how this practice varies. [Method] We performed an iterative case study at three companies and collected data through 14 interviews and 2 focus groups. [Results] The use of test cases as requirements poses both benefits and challenges when eliciting, validating, verifying, and managing requirements, and when used as a documented agreement. We have identified five variants of the test-cases-as-requirements practice, namely de facto, behaviour-driven, story-test driven, stand-alone strict and stand-alone manual for which the application of the practice varies concerning the time frame of requirements documentation, the requirements format, the extent to which the test cases are a machine executable specification and the use of tools which provide specific support for the practice of using test cases as requirements. [Conclusions] The findings provide empirical insight into how agile development projects manage and communicate requirements. The identified variants of the practice of using test cases as requirements can be used to perform in-depth investigations into agile requirements engineering. Practitioners can use the provided recommendations as a guide in designing and improving their agile requirements practices based on project characteristics such as number of stakeholders and rate of change.
|
|
| 2. |
- Jones, Robert P., et al.
(författare)
-
Patterns of Recurrence After Resection of Pancreatic Ductal Adenocarcinoma : A Secondary Analysis of the ESPAC-4 Randomized Adjuvant Chemotherapy Trial
- 2019
-
Ingår i: JAMA Surgery. - : AMER MEDICAL ASSOC. - 2168-6254 .- 2168-6262. ; 154:11, s. 1038-1048
-
Tidskriftsartikel (refereegranskat)abstract
- Importance: The patterns of disease recurrence after resection of pancreatic ductal adenocarcinoma with adjuvant chemotherapy remain unclear.Objective: To define patterns of recurrence after adjuvant chemotherapy and the association with survival.Design, Setting, and Participants: Prospectively collected data from the phase 3 European Study Group for Pancreatic Cancer 4 adjuvant clinical trial, an international multicenter study. The study included 730 patients who had resection and adjuvant chemotherapy for pancreatic cancer. Data were analyzed between July 2017 and May 2019.Interventions: Randomization to adjuvant gemcitabine or gemcitabine plus capecitabine.Main Outcomes and Measures: Overall survival, recurrence, and sites of recurrence.Results: Of the 730 patients, median age was 65 years (range 37-81 years), 414 were men (57%), and 316 were women (43%). The median follow-up time from randomization was 43.2 months (95% CI, 39.7-45.5 months), with overall survival from time of surgery of 27.9 months (95% CI, 24.8-29.9 months) with gemcitabine and 30.2 months (95% CI, 25.8-33.5 months) with the combination (HR, 0.81; 95% CI, 0.68-0.98; P=.03). The 5-year survival estimates were 17.1% (95% CI, 11.6%-23.5%) and 28.0% (22.0%-34.3%), respectively. Recurrence occurred in 479 patients (65.6%); another 78 patients (10.7%) died without recurrence. Local recurrence occurred at a median of 11.63 months (95% CI, 10.05-12.19 months), significantly different from those with distant recurrence with a median of 9.49 months (95% CI, 8.44-10.71 months) (HR, 1.21; 95% CI, 1.01-1.45; P=.04). Following recurrence, the median survival was 9.36 months (95% CI, 8.08-10.48 months) for local recurrence and 8.94 months (95% CI, 7.82-11.17 months) with distant recurrence (HR, 0.89; 95% CI, 0.73-1.09; P=.27). The median overall survival of patients with distant-only recurrence (23.03 months; 95% CI, 19.55-25.85 months) or local with distant recurrence (23.82 months; 95% CI, 17.48-28.32 months) was not significantly different from those with only local recurrence (24.83 months; 95% CI, 22.96-27.63 months) (P=.85 and P=.35, respectively). Gemcitabine plus capecitabine had a 21% reduction of death following recurrence compared with monotherapy (HR, 0.79; 95% CI, 0.64-0.98; P=.03).Conclusions and Relevance: There were no significant differences between the time to recurrence and subsequent and overall survival between local and distant recurrence. Pancreatic cancer behaves as a systemic disease requiring effective systemic therapy after resection.Trial Registration: ClinicalTrials.gov identifier: NCT00058201, EudraCT 2007-004299-38, and ISRCTN 96397434. This secondary analysis of a randomized clinical trial investigates patterns of recurrence after adjuvant chemotherapy in pancreatic cancer and the association with survival.
|
|
| 3. |
- Borg, Martin, et al.
(författare)
-
Supporting Change Impact Analysis Using a Recommendation System : An Industrial Case Study in a Safety-Critical Context
- 2017
-
Ingår i: IEEE Transactions on Software Engineering. - : IEEE. - 0098-5589 .- 1939-3520. ; 43:7, s. 675-700
-
Tidskriftsartikel (refereegranskat)abstract
- Change Impact Analysis (CIA) during software evolution of safety-critical systems is a labor-intensive task. Several authors have proposed tool support for CIA, but very few tools were evaluated in industry. We present a case study on ImpRec, a recommendation System for Software Engineering (RSSE), tailored for CIA at a process automation company. ImpRec builds on assisted tracing, using information retrieval solutions and mining software repositories to recommend development artifacts, potentially impacted when resolving incoming issue reports. In contrast to the majority of tools for automated CIA, ImpRec explicitly targets development artifacts that are not source code. We evaluate ImpRec in a two-phase study. First, we measure the correctness of ImpRec's recommendations by a simulation based on 12 years' worth of issue reports in the company. Second, we assess the utility of working with ImpRec by deploying the RSSE in two development teams on different continents. The results suggest that ImpRec presents about 40 percent of the true impact among the top-10 recommendations. Furthermore, user log analysis indicates that ImpRec can support CIA in industry, and developers acknowledge the value of ImpRec in interviews. In conclusion, our findings show the potential of reusing traceability associated with developers' past activities in an RSSE.
|
|
| 4. |
- Helali Moghadam, Mahshid, et al.
(författare)
-
Adaptive Runtime Response Time Control in PLC-based Real-Time Systems using Reinforcement Learning
- 2018
-
Ingår i: ACM/IEEE 13th International Symposium on Software Engineering for Adaptive and Self-Managing Systems, SEAMS 2018, , co-located with International Conference on Software Engineering, ICSE 2018; Gothenburg; Sweden; 28 May 2018 through 29 May 2018; Code 138312. - New York, NY, USA : ACM. ; , s. 217-223
-
Konferensbidrag (refereegranskat)abstract
- Timing requirements such as constraints on response time are key characteristics of real-time systems and violations of these requirements might cause a total failure, particularly in hard real-time systems. Runtime monitoring of the system properties is of great importance to detect and mitigate such failures. Thus, a runtime control to preserve the system properties could improve the robustness of the system with respect to timing violations. Common control approaches may require a precise analytical model of the system which is difficult to be provided at design time. Reinforcement learning is a promising technique to provide adaptive model-free control when the environment is stochastic, and the control problem could be formulated as a Markov Decision Process. In this paper, we propose an adaptive runtime control using reinforcement learning for real-time programs based on Programmable Logic Controllers (PLCs), to meet the response time requirements. We demonstrate through multiple experiments that our approach could control the response time efficiently to satisfy the timing requirements.
|
|
| 5. |
- Helali Moghadam, Mahshid, et al.
(författare)
-
Learning-based Response Time Analysis in Real-Time Embedded Systems : A Simulation-based Approach
- 2018
-
Ingår i: 1st International Workshop on Software Qualities and their Dependencies, located at the International Conference of Software Engineering (ICSE) 2018 SQUADE'18. - New York, NY, USA : ACM. - 9781450357371 ; , s. 21-24
-
Konferensbidrag (refereegranskat)abstract
- Response time analysis is an essential task to verify the behavior of real-time systems. Several response time analysis methods have been proposed to address this challenge, particularly for real-time systems with different levels of complexity. Static analysis is a popular approach in this context, but its practical applicability is limited due to the high complexity of the industrial real-time systems, as well as many unpredictable runtime events in these systems. In this work-in-progress paper, we propose a simulationbased response time analysis approach using reinforcement learning to find the execution scenarios leading to the worst-case response time. The approach learns how to provide a practical estimation of the worst-case response time through simulating the program without performing static analysis. Our initial study suggests that the proposed approach could be applicable in the simulation environments of the industrial real-time control systems to provide a practical estimation of the execution scenarios leading to the worst-case response time.
|
|
| 6. |
- Helali Moghadam, Mahshid, et al.
(författare)
-
Learning-Based Self-Adaptive Assurance of Timing Properties in a Real-Time Embedded System
- 2018
-
Ingår i: ICST Workshop on Testing Extra-Functional Properties and Quality Characteristics of Software Systems ITEQS'18. - 9781538663523 ; , s. 77-80
-
Konferensbidrag (refereegranskat)abstract
- Providing an adaptive runtime assurance technique to meet the performance requirements of a real-time system without the need for a precise model could be a challenge. Adaptive performance assurance based on monitoring the status of timing properties can bring more robustness to the underlying platform. At the same time, the results or the achieved policy of this adaptive procedure could be used as feedback to update the initial model, and consequently for producing proper test cases. Reinforcement-learning has been considered as a promising adaptive technique for assuring the satisfaction of the performance properties of software-intensive systems in recent years. In this work-in-progress paper, we propose an adaptive runtime timing assurance procedure based on reinforcement learning to satisfy the performance requirements in terms of response time. The timing control problem is formulated as a Markov Decision Process and the details of applying the proposed learning-based timing assurance technique are described.
|
|
| 7. |
- Helali Moghadam, Mahshid, et al.
(författare)
-
Machine Learning to Guide Performance Testing : An Autonomous Test Framework
- 2019
-
Ingår i: ICST Workshop on Testing Extra-Functional Properties and Quality Characteristics of Software Systems ITEQS'19, 2019.
-
Konferensbidrag (refereegranskat)abstract
- Satisfying performance requirements is of great importance for performance-critical software systems. Performance analysis to provide an estimation of performance indices and ascertain whether the requirements are met is essential for achieving this target. Model-based analysis as a common approach might provide useful information but inferring a precise performance model is challenging, especially for complex systems. Performance testing is considered as a dynamic approach for doing performance analysis. In this work-in-progress paper, we propose a self-adaptive learning-based test framework which learns how to apply stress testing as one aspect of performance testing on various software systems to find the performance breaking point. It learns the optimal policy of generating stress test cases for different types of software systems, then replays the learned policy to generate the test cases with less required effort. Our study indicates that the proposed learning-based framework could be applied to different types of software systems and guides towards autonomous performance testing.
|
|
| 8. |
- Ademuyiwa, Adesoji O., et al.
(författare)
-
Determinants of morbidity and mortality following emergency abdominal surgery in children in low-income and middle-income countries
- 2016
-
Ingår i: BMJ Global Health. - : BMJ Publishing Group Ltd. - 2059-7908. ; 1:4
-
Tidskriftsartikel (refereegranskat)abstract
- Background: Child health is a key priority on the global health agenda, yet the provision of essential and emergency surgery in children is patchy in resource-poor regions. This study was aimed to determine the mortality risk for emergency abdominal paediatric surgery in low-income countries globally.Methods: Multicentre, international, prospective, cohort study. Self-selected surgical units performing emergency abdominal surgery submitted prespecified data for consecutive children aged <16 years during a 2-week period between July and December 2014. The United Nation's Human Development Index (HDI) was used to stratify countries. The main outcome measure was 30-day postoperative mortality, analysed by multilevel logistic regression.Results: This study included 1409 patients from 253 centres in 43 countries; 282 children were under 2 years of age. Among them, 265 (18.8%) were from low-HDI, 450 (31.9%) from middle-HDI and 694 (49.3%) from high-HDI countries. The most common operations performed were appendectomy, small bowel resection, pyloromyotomy and correction of intussusception. After adjustment for patient and hospital risk factors, child mortality at 30 days was significantly higher in low-HDI (adjusted OR 7.14 (95% CI 2.52 to 20.23), p<0.001) and middle-HDI (4.42 (1.44 to 13.56), p=0.009) countries compared with high-HDI countries, translating to 40 excess deaths per 1000 procedures performed.Conclusions: Adjusted mortality in children following emergency abdominal surgery may be as high as 7 times greater in low-HDI and middle-HDI countries compared with high-HDI countries. Effective provision of emergency essential surgery should be a key priority for global child health agendas.
|
|
| 9. |
- Andersen, Lars L, et al.
(författare)
-
Effect of physical exercise on workplace social capital: cluster randomized controlled trial
- 2015
-
Ingår i: Scandinavian Journal of Public Health. - : SAGE Publications. - 1651-1905 .- 1403-4948. ; 43:8, s. 810-818
-
Tidskriftsartikel (refereegranskat)abstract
- While workplace health promotion with group-based physical exercise can improve workers' physical health, less is known about potential carry-over effects to psychosocial factors. This study investigates the effect of physical exercise on social capital at work.
|
|
| 10. |
- Assar, Saïd, et al.
(författare)
-
Using Text Clustering to Predict Defect Resolution Time: A Conceptual Replication and an Evaluation of Prediction Accuracy
- 2015
-
Ingår i: Empirical Software Engineering. - : Springer Science and Business Media LLC. - 1573-7616 .- 1382-3256.
-
Tidskriftsartikel (refereegranskat)abstract
- Defect management is a central task in software maintenance. When a defect is reported, appropriate resources must be allocated to analyze and resolve the defect. An important issue in resource allocation is the estimation of Defect Resolution Time (DRT). Prior research has considered different approaches for DRT prediction exploiting information retrieval techniques and similarity in textual defect descriptions. In this article, we investigate the potential of text clustering for DRT prediction. We build on a study published by Raja (2013) which demonstrated that clusters of similar defect reports had statistically significant differences in DRT. Raja’s study also suggested that this difference between clusters could be used for DRT prediction. Our aims are twofold: First, to conceptually replicate Raja’s study and to assess the repeatability of its results in different settings; Second, to investigate the potential of textual clustering of issue reports for DRT prediction with focus on accuracy. Using different data sets and a different text mining tool and clustering technique, we first conduct an independent replication of the original study. Then we design a fully automated prediction method based on clustering with a simulated test scenario to check the accuracy of our method. The results of our independent replication are comparable to those of the original study and we confirm the initial findings regarding significant differences in DRT between clusters of defect reports. However, the simulated test scenario used to assess our prediction method yields poor results in terms of DRT prediction accuracy. Although our replication confirms the main finding from the original study, our attempt to use text clustering as the basis for DRT prediction did not achieve practically useful levels of accuracy.
|
|
| 11. |
- Bjarnason, Elizabeth, et al.
(författare)
-
2nd International Workshop on Requirements Engineering and Testing (RET 2015)
- 2015
-
Ingår i: 2015 IEEE/ACM 37TH IEEE INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING, VOL 2. - : IEEE. - 9781479919345 ; , s. 997-998
-
Konferensbidrag (refereegranskat)abstract
- The RET (Requirements Engineering and Testing) workshop provides a meeting point for researchers and practitioners from the two separate fields of Requirements Engineering (RE) and Testing. The goal is to improve the connection and alignment of these two areas through an exchange of ideas, challenges, practices, experiences and results. The long term aim is to build a community and a body of knowledge within the intersection of RE and Testing. One of the main outputs of the 1st workshop was a collaboratively constructed map of the area of RET showing the topics relevant to RET for these. The 2nd workshop will continue in the same interactive vein and include a keynote, paper presentations with ample time for discussions, and a group exercise. For true impact and relevance this cross-cutting area requires contribution from both RE and Testing, and from both researchers and practitioners. For that reason we welcome a range of paper contributions from short experience papers to full research papers that both clearly cover connections between the two fields.
|
|
| 12. |
- Bjarnason, Elizabeth, et al.
(författare)
-
Aligning Requirements and Testing : Working Together toward the Same Goal
- 2017
-
Ingår i: IEEE Software. - 0740-7459 .- 1937-4194. ; 34:1, s. 20-23
-
Tidskriftsartikel (övrigt vetenskapligt/konstnärligt)abstract
- The proper alignment of requirements engineering and testing (RET) can be key to software's success. Three practices can provide effective RET alignment: using test cases as requirements, harvesting trace links, and reducing distances between requirements engineers and testers. The Web extra https://youtu.be/M65ZKxfxqME is an audio podcast of author Elizabeth Bjarnason reading the the Requirements column she cowrote with Markus Borg.
|
|
| 13. |
- Bjarnason, Elizabeth, et al.
(författare)
-
An Industrial Case Study on Test Cases as Requirements
- 2015
-
Ingår i: Agile Processes in Software Engineering and Extreme Programming : 16th International Conference, XP 2015, Helsinki, Finland, May 25-29, 2015, Proceedings - 16th International Conference, XP 2015, Helsinki, Finland, May 25-29, 2015, Proceedings. - Cham : Springer International Publishing. - 1865-1356 .- 1865-1348. - 9783319186115 - 9783319186122 ; 212
-
Konferensbidrag (refereegranskat)abstract
- It is a conundrum that agile projects can succeed ‘without requirements’ when weak requirements engineering is a known cause for project failures. While Agile development projects often manage well without extensive requirements documentation, test cases are commonly used as requirements. We have investigated this agile practice at three companies in order to understand how test cases can fill the role of requirements. We performed a case study based on twelve interviews performed in a previous study. The findings include a range of benefits and challenges in using test cases for eliciting, validating, verifying, tracing and managing requirements. In addition, we identified three scenarios for applying the practice, namely as a mature practice, as a de facto practice and as part of an agile transition. The findings provide insights into how the role of requirements may be met in agile development including challenges to consider.
|
|
| 14. |
- Bjarnason, Elizabeth, et al.
(författare)
-
An Industrial Case Study on the Use of Test Cases as Requirements
- 2015
-
Ingår i: Lecture Notes in Business Information<em></em>. - : Springer. ; , s. 27-39
-
Konferensbidrag (refereegranskat)abstract
- It is a conundrum that agile projects can succeed 'without requirements' when weak requirements engineering is a known cause for project failures. While Agile development projects often manage well without extensive requirements documentation, test cases are commonly used as requirements. We have investigated this agile practice at three companies in order to understandhow test cases can fill the role of requirements. We performed a case study based on twelve interviews performed in a previous study.The findings include a range of benefits and challenges in using test cases for eliciting, validating, verifying, tracing and managing requirements. In addition, we identified three scenarios for applying the practice, namely as a mature practice, as a de facto practice and as part of an agile transition. The findings provide insights into how the role of requirements may be met in agile development including challenges to consider.
|
|
| 15. |
- Bjarnason, Elizabeth, et al.
(författare)
-
Supervising for Independence – A Case Study of Master Science Projects in Higher Education
- 2015. - 12
-
Ingår i: LU:s femte högskolepedagogiska utvecklingskonferens.
-
Konferensbidrag (refereegranskat)abstract
- Students completing a Swedish Master's degree in engineering should have knowledge and skills to independently solve engineering issues. This autonomy should be developed and demonstrated within the M.Sc. project course. But, how can supervisors encourage independence? We have explored this in a case study through semi-structured interviews with students, supervisors and examiners of two M.Sc. projects. We investigated their view of independence, and how supervision correlates to independence. The results identify areas relevant to independence, namely supervision roles and relationships, student characteristics, M.Sc. process, and view on independence. The results confirm previous findings that students' knowledge of and motivation for the topic support independence. The supervisor's role is to guide and support through frequent peer-level discussions and to act as a discussion partner, while the student should have the main responsibility for the project. We conclude that it is important for supervisors to encourage students to take ownership of their M.Sc. projects and to design their own solutions, while providing the overall process and timelines.
|
|
| 16. |
- Bjarnason, Elizabeth, et al.
(författare)
-
Supervising Towards Independence
- 2016. - 10
-
Ingår i: Lärande i LTH. - : Lunds Tekniska Högskola. ; 32, s. 5-6
-
Annan publikation (övrigt vetenskapligt/konstnärligt)abstract
- Supervising a student can be compared to teaching someone to drive a car. The student is in the driver's seat while the supervisor provides structure and guidance, and can intervene in risky and unsafe situations. It is a learning process in which the student gradually gains experience and sufficient skill to obtain a driving license, and to drive without an instructor. Similarly, a student attending the MSc project course at the technical faculty of Lund University is to "develop and demonstrate knowledge and ability required to autonomously work as an engineer" (from MSc course plan). But what factors affect a MSc project, and how can we as supervisors support students in their learning process towards independence? We performed a case study of two completed MSc projects where we interviewed the student, the supervisor and the examiner for each case. In this article we present the main conclusions drawn from the cross-case analysis of this study. Details on the studied cases and the results on which these conclusions are based can be found in our previous publication of this study.
|
|
| 17. |
- Borg, Markus, et al.
(författare)
-
An Analytical View ofTest Results Using CityScapes
- 2018
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
- t— In this paper we map test results from a real ASIC project on to the file structure of the design under test andpresent it as a cityscape. In the cityscape each house is a file where its height reflects the number of commits to that file. Thecolor reflects the fraction of bad commits.We identify error prone areas (red "bad" neighborhoods) as well as the most active areas (tall "downtown" areas). Thecityscape also allows us to identify potential test coverage holes (tall green buildings) where there are a lot of activities but nofailures.
|
|
| 18. |
|
|
| 19. |
- Borg, Markus, et al.
(författare)
-
Digitalization of Swedish Government Agencies : Detailed Census Description and Analysis
- 2018
-
Rapport (övrigt vetenskapligt/konstnärligt)abstract
- Software engineering is at the core of the digitalization of society. Ill-informed decisions can have major consequences, as made evident in the 2017 government crisis in Sweden, originating in a data breach caused by an outsourcing deal made by the Swedish Transport Agency. Many Government Agencies (GovAgs) in Sweden are rapidly undergoing a digital transition, thus it is important to overview how widespread, and mature, software development is in this part of the public sector. We present a software development census of Swedish GovAgs, complemented by document analysis and a survey. We show that 39.2% of the GovAgs develop software internally, some matching the number of developers in large companies. Our findings suggest that the development largely resembles private sector counterparts, and that established best practices are implemented. Still, we identify improvement potential in the areas of strategic sourcing, openness, collaboration across GovAgs, and quality requirements. The Swedish Government has announced the establishment of a new digitalization agency next year, and our hope is that the software engineering community will contribute its expertise with a clear voice.
|
|
| 20. |
- Borg, Markus, et al.
(författare)
-
Digitalization of Swedish Government Agencies : A Perspective Through the Lens of a Software Development Census
- 2018
-
Ingår i: Proceedings of the 40th International Conference on Software Engineering. - New York, NY, USA : ACM. ; , s. 37-46
-
Konferensbidrag (refereegranskat)abstract
- Software engineering is at the core of the digitalization of society. Ill-informed decisions can have major consequences, as made evident in the 2017 government crisis in Sweden, originating in a data breach caused by an outsourcing deal made by the Swedish Transport Agency. Many Government Agencies (GovAgs) in Sweden are rapidly undergoing a digital transition, thus it is important to overview how widespread, and mature, software development is in this part of the public sector. We present a software development census of Swedish GovAgs, complemented by document analysis and a survey. We show that 39.2% of the GovAgs develop software internally, some matching the number of developers in large companies. Our findings suggest that the development largely resembles private sector counterparts, and that established best practices are implemented. Still, we identify improvement potential in the areas of strategic sourcing, openness, collaboration across GovAgs, and quality requirements. The Swedish Government has announced the establishment of a new digitalization agency next year, and our hope is that the software engineering community will contribute its expertise with a clear voice.
|
|
| 21. |
- Borg, Markus, et al.
(författare)
-
Do Take it Personal : It's Not What You Say, It's Who (and Where) You Are!
- 2016. - 6
-
Ingår i: Tiny Transactions on Computer Science. ; 4
-
Tidskriftsartikel (refereegranskat)abstract
- Issue management in market-driven software projects is constantly under time pressure. A limited set of developers must share their time between developing features for the next release and resolving reported issues. Project managers need to find the appropriate balance between a high quality product and fast time to market. We study a telecom company in Sweden developing embedded systems for a consumer market. The project managers report that developers resolve approximately 10% of the issues reported during a project. Consequently, it is critical to properly prioritize the issues to receive the best possible return on investment, and above all to remove all bugs that might impact the market's reception of the product. We use machine learning to investigate what features of an issue report are the best predictors of changes to production code during its corresponding resolution. After removing all features jeopardizing the confidentiality of individual engineers, the issue reports are characterized by 19 features (apart from text). We extract 80,000 issue reports, an equal mix of positive and negative examples, and train a Bayesian Network classifier [2], obtaining 73% classification accuracy. Moreover, it reveals that the feature with the highest predictive value is from which physical site the issue was submitted. The general priority feature however, is only ranked 17 out of 19, whereas the submitting team is ranked 12. Our findings confirm a suspicion in the company: the priority set by the issue submitter is indeed a poor predictor of a future code change.
|
|
| 22. |
- Borg, Markus, et al.
(författare)
-
Enabling Visual Design Verification Analytics– From Prototype Visualizations to anAnalytics Tool using the Unity Game Engine
- 2018
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
- The ever-increasing architectural complexity in contemporary ASIC projects turns Design Verification (DV)into a highly advanced endeavor. Pressing needs for short time-to-market has made automation a key solution in DV.However, recurring execution of large regression suites inevitably leads to challenging amounts of test results. Following thedesign science paradigm, we present an action research study to introduce visual analytics in a commercial ASIC project. Wedevelop a cityscape visualization tool using the game engine Unity. Initial evaluations are promising, suggesting that the tooloffers a novel approach to identify error-prone parts of the design, as well as coverage holes.
|
|
| 23. |
- Borg, Markus
(författare)
-
From Bugs to Decision Support – Leveraging Historical Issue Reports in Software Evolution
- 2015
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Software developers in large projects work in complex information landscapes and staying on top of all relevant software artifacts is an acknowledged challenge. As software systems often evolve over many years, a large number of issue reports is typically managed during the lifetime of a system, representing the units of work needed for its improvement, e.g., defects to fix, requested features, or missing documentation. Efficient management of incoming issue reports requires the successful navigation of the information landscape of a project. In this thesis, we address two tasks involved in issue management: Issue Assignment (IA) and Change Impact Analysis (CIA). IA is the early task of allocating an issue report to a development team, and CIA is the subsequent activity of identifying how source code changes affect the existing software artifacts. While IA is fundamental in all large software projects, CIA is particularly important to safety-critical development. Our solution approach, grounded on surveys of industry practice as well as scientific literature, is to support navigation by combining information retrieval and machine learning into Recommendation Systems for Software Engineering (RSSE). While the sheer number of incoming issue reports might challenge the overview of a human developer, our techniques instead benefit from the availability of ever-growing training data. We leverage the volume of issue reports to develop accurate decision support for software evolution. We evaluate our proposals both by deploying an RSSE in two development teams, and by simulation scenarios, i.e., we assess the correctness of the RSSEs' output when replaying the historical inflow of issue reports. In total, more than 60,000 historical issue reports are involved in our studies, originating from the evolution of five proprietary systems for two companies. Our results show that RSSEs for both IA and CIA can help developers navigate large software projects, in terms of locating development teams and software artifacts. Finally, we discuss how to support the transfer of our results to industry, focusing on addressing the context dependency of our tool support by systematically tuning parameters to a specific operational setting.
|
|
| 24. |
- Borg, Markus, et al.
(författare)
-
From LiDAR to Underground Maps via 5G - Business Models Enabling a System-of-Systems Approach to Mapping the Kankberg Mine
- 2017
-
Rapport (övrigt vetenskapligt/konstnärligt)abstract
- With ever-increasing productivity targets in mining operations, there is a growing interest in mining automation. The PIMM project addresses the fundamental challenge of network communication by constructing a pilot 5G network in the underground mine Kankberg. In this report, we discuss how such a 5G network could constitute the essential infrastructure to organize existing systems in Kankberg into a system-of-systems (SoS). In this report, we analyze a scenario in which LiDAR equipped vehicles operating in the mine are connected to existing mine mapping and positioning solutions. The approach is motivated by the approaching era of remote controlled, or even autonomous, vehicles in mining operations. The proposed SoS could ensure continuously updated maps of Kankberg, rendered in unprecedented detail, supporting both productivity and safety in the underground mine. We present four different SoS solutions from an organizational point of view, discussing how development and operations of the constituent systems could be distributed among Boliden and external stakeholders, e.g., the vehicle suppliers, the hauling company, and the developers of the mapping software. The four scenarios are compared from both technical and business perspectives, and based on trade-off discussions and SWOT analyses. We conclude our report by recommending continued research along two future paths, namely a closer cooperation with the vehicle suppliers, and further feasibility studies regarding establishing a Kankberg software ecosystem.
|
|
| 25. |
- Borg, Markus, et al.
(författare)
-
On using active learning and self-training when mining performance discussions on stack overflow
- 2017
-
Ingår i: EASE'17 Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering. ACM International Conference Proceeding Series. - New York, NY, USA : ACM. - 9781450348041 ; , s. 308-313
-
Konferensbidrag (refereegranskat)abstract
- Abundant data is the key to successful machine learning. However, supervised learning requires annotated data that are often hard to obtain. In a classification task with limited resources, Active Learning (AL) promises to guide annotators to examples that bring the most value for a classifier. AL can be successfully combined with self-training, i.e., extending a training set with the unlabelled examples for which a classifier is the most certain. We report our experiences on using AL in a systematic manner to train an SVM classifier for Stack Overflow posts discussing performance of software components. We show that the training examples deemed as the most valuable to the classifier are also the most difficult for humans to annotate. Despite carefully evolved annotation criteria, we report low inter-rater agreement, but we also propose mitigation strategies. Finally, based on one annotator's work, we show that self-training can improve the classification accuracy. We conclude the paper by discussing implication for future text miners aspiring to use AL and self-training.
|
|
| 26. |
- Borg, Markus, et al.
(författare)
-
Piggybacking on an Autonomous Hauler : Business Models Enabling a System-of-Systems Approach to Mapping an Underground Mine
- 2017
-
Ingår i: 2017 IEEE 25TH INTERNATIONAL REQUIREMENTS ENGINEERING CONFERENCE (RE). - : IEEE Press. - 9781538631911 ; , s. 372-381
-
Konferensbidrag (refereegranskat)abstract
- With ever-increasing productivity targets in mining operations, there is a growing interest in mining automation. In future mines, remote-controlled and autonomous haulers will operate underground guided by LiDAR sensors. We envision reusing LiDAR measurements to maintain accurate mine maps that would contribute to both safety and productivity. Extrapolating from a pilot project on reliable wireless communication in Boliden's Kankberg mine, we propose establishing a system-of-systems (SoS) with LIDAR-equipped haulers and existing mapping solutions as constituent systems. SoS requirements engineering inevitably adds a political layer, as independent actors are stakeholders both on the system and SoS levels. We present four SoS scenarios representing different business models, discussing how development and operations could be distributed among Boliden and external stakeholders, e.g., the vehicle suppliers, the hauling company, and the developers of the mapping software. Based on eight key variation points, we compare the four scenarios from both technical and business perspectives. Finally, we validate our findings in a seminar with participants from the relevant stakeholders. We conclude that to determine which scenario is the most promising for Boliden, trade-offs regarding control, costs, risks, and innovation must be carefully evaluated.
|
|
| 27. |
- Borg, Markus, et al.
(författare)
-
Practitioners' Perspectives on Change Impact Analysis for Safety-Critical Software - A Preliminary Analysis
- 2016. - 11
-
Ingår i: Computer Safety, Reliability, and Security. - Cham : Springer International Publishing. - 9783319454795 - 9783319454801 ; , s. 346-358
-
Konferensbidrag (refereegranskat)abstract
- Safety standards prescribe change impact analysis (CIA) during evolution of safety-critical software systems. Although CIA is a fundamental activity, there is a lack of empirical studies about how it is performed in practice. We present a case study on CIA in the context of an evolving automation system, based on 14 interviews in Sweden and India. Our analysis suggests that engineers on average spend 50-100 hours on CIA per year, but the effort varies considerably with the phases of projects. Also, the respondents presented different connotations to CIA and perceived the importance of CIA differently. We report the most pressing CIA challenges, and several ideas on how to support future CIA. However, we show that measuring the effect of such improvement solutions is non-trivial, as CIA is intertwined with other development activities. While this paper only reports preliminary results, our work contributes empirical insights into practical CIA.
|
|
| 28. |
- Borg, Markus, et al.
(författare)
-
Safely Entering the Deep: A Review of Verification and Validation for Machine Learning and a Challenge Elicitation in the Automotive Industry
- 2019
-
Ingår i: Journal of Automotive Software Engineering. - Paris : Athena International Publishing B.V.. - 2589-2258. ; 1:1, s. 1-13
-
Tidskriftsartikel (refereegranskat)abstract
- Deep neural networks (DNNs) will emerge as a cornerstone in automotive software engineering. However, developing systems with DNNs introduces novel challenges for safety assessments. This paper reviews the state-of-the-art in verification and validation of safety-critical systems that rely on machine learning. Furthermore, we report from a workshop series on DNNs for perception with automotive experts in Sweden, confirming that ISO 26262 largely contravenes the nature of DNNs. We recommend aerospace-to-automotive knowledge transfer and systems-based safety approaches, for example, safety cage architectures and simulated system test cases.
|
|
| 29. |
- Borg, Markus, et al.
(författare)
-
Selecting component sourcing options : A survey of software engineering's broader make-or-buy decisions
- 2019
-
Ingår i: Information and Software Technology. - : Elsevier B.V.. - 0950-5849 .- 1873-6025. ; 112, s. 18-34
-
Tidskriftsartikel (refereegranskat)abstract
- Context: Component-based software engineering (CBSE) is a common approach to develop and evolve contemporary software systems. When evolving a system based on components, make-or-buy decisions are frequent, i.e., whether to develop components internally or to acquire them from external sources. In CBSE, several different sourcing options are available: (1) developing software in-house, (2) outsourcing development, (3) buying commercial-off-the-shelf software, and (4) integrating open source software components. Objective: Unfortunately, there is little available research on how organizations select component sourcing options (CSO) in industry practice. In this work, we seek to contribute empirical evidence to CSO selection. Method: We conduct a cross-domain survey on CSO selection in industry, implemented as an online questionnaire. Results: Based on 188 responses, we find that most organizations consider multiple CSOs during software evolution, and that the CSO decisions in industry are dominated by expert judgment. When choosing between candidate components, functional suitability acts as an initial filter, then reliability is the most important quality. Conclusion: We stress that future solution-oriented work on decision support has to account for the dominance of expert judgment in industry. Moreover, we identify considerable variation in CSO decision processes in industry. Finally, we encourage software development organizations to reflect on their decision processes when choosing whether to make or buy components, and we recommend using our survey for a first benchmarking.
|
|
| 30. |
- Borg, Markus, et al.
(författare)
-
Selecting Software Component Sourcing Options : Detailed Survey Description and Analysis
- 2018
-
Rapport (övrigt vetenskapligt/konstnärligt)abstract
- Component-based software engineering (CBSE) is a common approach to develop and evolve contemporary software systems. When evolving a system based on components, make-or-buy decisions are frequent, i.e., whether to develop components internally or to acquire them fromexternal sources. In CBSE, several different sourcing options are available: 1) developing software in-house, 2) outsourcing development, 3) buying commercial-off-the-shelf software, and 4) integrating open source software components. Unfortunately, there is little available research on howorganizations select component sourcing options (CSO) in industry practice. In this work, we seek to contribute empirical evidence to CSO selection. Method: We conduct a cross-domain survey on CSO selection in industry, implemented as an online questionnaire. Based on 188 responses, we find that most organizations consider multiple CSOs during software evolution, and that the CSO decisions in industry are dominated by expert judgment. When choosing between candidate components, functional suitability acts as an initial filter, then reliability is the most important quality. We stress that future solution-oriented work on decision support has to account for the dominance of expert judgment in industry. Moreover, we identify considerable variation in CSO decision processes in industry. Finally, we encourage software development organizations to reflect on their decision processes when choosing whether to make or buy components, and we recommend using our survey for a first benchmarking.
|
|
| 31. |
- Borg, Markus, et al.
(författare)
-
Software Engineers' Information Seeking Behavior in Change Impact Analysis : An Interview Study
- 2017
-
Ingår i: IEEE International Conference on Program Comprehension. - : IEEE Computer Society. - 9781538605356 ; , s. 12-22, s. 12-22
-
Konferensbidrag (refereegranskat)abstract
- Software engineers working in large projects must navigate complex information landscapes. Change Impact Analysis (CIA) is a task that relies on engineers' successful information seeking in databases storing, e.g., source code, requirements, design descriptions, and test case specifications. Several previous approaches to support information seeking are task-specific, thus understanding engineers' seeking behavior in specific tasks is fundamental. We present an industrial case study on how engineers seek information in CIA, with a particular focus on traceability and development artifacts that are not source code. We show that engineers have different information seeking behavior, and that some do not consider traceability particularly useful when conducting CIA. Furthermore, we observe a tendency for engineers to prefer less rigid types of support rather than formal approaches, i.e., engineers value support that allows flexibility in how to practically conduct CIA. Finally, due to diverse information seeking behavior, we argue that future CIA support should embrace individual preferences to identify change impact by empowering several seeking alternatives, including searching, browsing, and tracing. © 2017 IEEE.
|
|
| 32. |
- Borg, Markus, et al.
(författare)
-
Summary of the 4th International Workshop on Requirements Engineering and Testing (RET 2017)
- 2018
-
Ingår i: Software Engineering Notes. - : Association for Computing Machinery (ACM). - 0163-5948 .- 1943-5843. ; 42:4, s. 28-31
-
Tidskriftsartikel (övrigt vetenskapligt/konstnärligt)abstract
- The RET (Requirements Engineering and Testing) workshop series provides a meeting point for researchers and practitioners from the two separate fields of Requirements Engineering (RE) and Testing. The long term aim is to build a community and a body of knowledge within the intersection of RE and Testing, i.e., RET. The 4th workshop was co-located with the 25th International Requirements Engineering Conference (RE'17) in Lisbon, Portugal and attracted about 20 participants. In line with the previous workshop instances, RET 2017 o ered an interactive setting with a keynote, an invited talk, paper presentations, and a concluding hands-on exercise.
|
|
| 33. |
- Borg, Markus, et al.
(författare)
-
SZZ unleashed: an open implementation of the SZZ algorithm : featuring example usage in a study of just-in-time bug prediction for the Jenkins project
- 2019
-
Konferensbidrag (refereegranskat)abstract
- Machine learning applications in software engineering often rely on detailed information about bugs. While issue trackers often contain information about when bugs were fixed, details about when they were introduced to the system are often absent. As a remedy, researchers often rely on the SZZ algorithm as a heuristic approach to identify bug-introducing software changes. Unfortunately, as reported in a recent systematic literature review, few researchers have made their SZZ implementations publicly available. Consequently, there is a risk that research effort is wasted as new projects based on SZZ output need to initially reimplement the approach. Furthermore, there is a risk that newly developed (closed source) SZZ implementations have not been properly tested, thus conducting research based on their output might introduce threats to validity. We present SZZ Unleashed, an open implementation of the SZZ algorithm for git repositories. This paper describes our implementation along with a usage example for the Jenkins project, and conclude with an illustrative study on just-in-time bug prediction. We hope to continue evolving SZZ Unleashed on GitHub, and warmly invite the community to contribute.
|
|
| 34. |
- Borg, Markus, et al.
(författare)
-
The More the Merrier: Leveraging on the Bug Inflow to Guide Software Maintenance
- 2015
-
Ingår i: Tiny Transactions on Computer Science. ; 3
-
Tidskriftsartikel (refereegranskat)abstract
- Issue management, a central part of software maintenance, requires much effort for complex software systems. The continuous inflow of issue reports makes it hard for developers to stay on top of the situation, and the threatening information overload makes activities such as duplicate management, Issue Assignment (IA), and Change Impact Analysis (CIA) tedious and error-prone. Still, most practitioners work with tools that act as little more than issue containers. Machine Learning encompasses approaches that identify patterns or make predictions based on empirical data. While humans have limited ability to work with big data, ML instead tends to improve the more training data that is available. Consequently, we argue that the challenge of information overload in issue management appears to be particularly suitable for ML-based tool support. While others have initially explored the area, we develop two ML-based tools, and evaluate them in proprietary software engineering contexts. We replicated [1] for five projects in two companies, and our automated IA obtains an accuracy matching the current manual processes. Thus, as our solution delivers instantaneous IA, an organization can potentially save considerable analysis effort. Moreover, for the most comprehensive of the five projects, we implemented automated CIA in the tool ImpRec [3]. We evaluated the tool in a longitudinal in situ study, i.e., deployment in two development teams in industry. Based on log analysis and complementary interviews using the QUPER model [2] for utility assessment, we conclude that ImpRec offered helpful support in the CIA task.
|
|
| 35. |
|
|
| 36. |
- Borg, Markus
(författare)
-
TuneR : A Framework for Tuning Software Engineering Tools with Hands-on Instructions in R
- 2016. - 15
-
Ingår i: Journal of Software. - : John Wiley & Sons Ltd. - 2047-7473 .- 2047-7481. ; 28:6, s. 427-459
-
Tidskriftsartikel (refereegranskat)abstract
- Numerous tools automating various aspects of software engineering have been developed, and many of the tools are highly configurable through parameters. Understanding the parameters of advanced tools often requires deep understanding of complex algorithms. Unfortunately, suboptimal parameter settings limit the performance of tools and hinder industrial adaptation, but still few studies address the challenge of tuning software engineering tools. We present TuneR, an experiment framework that supports finding feasible parameter settings using empirical methods. The framework is accompanied by practical guidelines of how to use R to analyze the experimental outcome. As a proof-of-concept, we apply TuneR to tune ImpRec, a recommendation system for change impact analysis in a software system that has evolved for more than two decades. Compared with the output from the default setting, we report a 20.9% improvement in the response variable reflecting recommendation accuracy. Moreover, TuneR reveals insights into the interaction among parameters, as well as nonlinear effects. TuneR is easy to use, thus the framework has potential to support tuning of software engineering tools in both academia and industry.
|
|
| 37. |
- Borg, Sixten, et al.
(författare)
-
Cost effectiveness of pomalidomide in patients with relapsed and refractory multiple myeloma in Sweden
- 2016
-
Ingår i: Acta Oncologica. - 0284-186X. ; 55:5, s. 554-560
-
Tidskriftsartikel (refereegranskat)abstract
- Background: Multiple myeloma (MM) patients who have progressed following treatment with both bortezomib and lenalidomide have a poor prognosis. In this late stage, other effective alternatives are limited, and patients in Sweden are often left with best supportive care. Pomalidomide is a new anti-angiogenic and immunomodulatory drug for the treatment of MM. Our objective was to evaluate the cost effectiveness of pomalidomide as an add-on to best supportive care in patients with relapsed and refractory MM in Sweden. Material and methods: We developed a health-economic discrete event simulation model of a patient’s course through stable disease and progressive disease, until death. It estimates life expectancy, quality-adjusted life years (QALYs) and costs from a societal perspective. Effectiveness data and utilities were taken from the MM-003 trial comparing pomalidomide plus low-dose dexamethasone with high-dose dexamethasone (HIDEX). Cost data were taken from official Swedish price lists, government sources and literature. Results: The model estimates that, if a patient is treated with HIDEX, life expectancy is 1.12 years and the total cost is SEK 179 976 (€19 100), mainly indirect costs. With pomalidomide plus low-dose dexamethasone, life expectancy is 2.33 years, with a total cost of SEK 767 064 (€81 500), mainly in drug and indirect costs. Compared to HIDEX, pomalidomide treatment gives a QALY gain of 0.7351 and an incremental cost of SEK 587 088 (€62 400) consisting of increased drug costs (59%), incremental indirect costs (33%) and other healthcare costs (8%). The incremental cost-effectiveness ratio is SEK 798 613 (€84 900) per QALY gained. Conclusion: In a model of late-stage MM patients with a poor prognosis in the Swedish setting, pomalidomide is associated with a relatively high incremental cost per QALY gained. This model was accepted by the national Swedish reimbursement authority TLV, and pomalidomide was granted reimbursement in Sweden.
|
|
| 38. |
- Chatzipetrou, Panagiota, Assistant Professor, 1984-, et al.
(författare)
-
Component selection in software engineering - Which attributes are the most important in the decision process?
- 2018
-
Ingår i: Proceedings - 44th Euromicro Conference on Software Engineering and Advanced Applications, SEAA 2018. - : IEEE conference proceedings. - 9781538673829 ; , s. 198-205, s. 198-205
-
Konferensbidrag (refereegranskat)abstract
- Component-based software engineering is a common approach to develop and evolve contemporary software systems where different component sourcing options are available: 1)Software developed internally (in-house), 2)Software developed outsourced, 3)Commercial of the shelf software, and 4) Open Source Software. However, there is little available research on what attributes of a component are the most important ones when selecting new components. The object of the present study is to investigate what matters the most to industry practitioners during component selection. We conducted a cross-domain anonymous survey with industry practitioners involved in component selection. First, the practitioners selected the most important attributes from a list. Next, they prioritized their selection using the Hundred-Dollar ($100) test. We analyzed the results using Compositional Data Analysis. The descriptive results showed that Cost was clearly considered the most important attribute during the component selection. Other important attributes for the practitioners were: Support of the component, Longevity prediction, and Level of off-the-shelf fit to product. Next, an exploratory analysis was conducted based on the practitioners' inherent characteristics. Nonparametric tests and biplots were used. It seems that smaller organizations and more immature products focus on different attributes than bigger organizations and mature products which focus more on Cost. .
|
|
| 39. |
- Cicchetti, Antonio, et al.
(författare)
-
Towards Software Assets Origin Selection Supported by a Knowledge Repository
- 2016
-
Ingår i: Proceedings - 2016 1st International Workshop on Decision Making in Software ARCHitecture, MARCH 2016. - : IEEE Computer Society. - 9781509025732 ; , s. 22-29
-
Konferensbidrag (refereegranskat)abstract
- Software architecture is no more a mere system specification as resulting from the design phase, but it includes the process by which its specification was carried out. In this respect, design decisions in component-based software engineering play an important role: They are used to enhance the quality of the system, keep the current market level, keep partnership relationships, reduce costs, and so forth. For non trivial systems, a recurring situation is the selection of an asset origin, that is if going for in-house, outsourcing, open-source, or COTS, when in the need of a certain missing functionality. Usually, the decision making process follows a case-by-case approach, in which historical information is largely neglected: hence, it is avoided the overhead of keeping detailed documentation about past decisions, but it is hampered consistency among multiple, possibly related, decisions.The ORION project aims at developing a decision support framework in which historical decision information plays a pivotal role: it is used to analyse current decision scenarios, take well-founded decisions, and store the collected data for future exploitation. In this paper, we outline the potentials of such a knowledge repository, including the information it is intended to be stored in it, and when and how to retrieve it within a decision case.
|
|
| 40. |
- Cirenajwis, Helena, et al.
(författare)
-
Molecular stratification of metastatic melanoma using gene expression profiling: prediction of survival outcome and benefit from molecular targeted therapy.
- 2015
-
Ingår i: Oncotarget. - : Impact Journals, LLC. - 1949-2553. ; 6:14, s. 12297-12309
-
Tidskriftsartikel (refereegranskat)abstract
- Melanoma is currently divided on a genetic level according to mutational status. However, this classification does not optimally predict prognosis. In prior studies, we have defined gene expression phenotypes (high-immune, pigmentation, proliferative and normal-like), which are predictive of survival outcome as well as informative of biology. Herein, we employed a population-based metastatic melanoma cohort and external cohorts to determine the prognostic and predictive significance of the gene expression phenotypes. We performed expression profiling on 214 cutaneous melanoma tumors and found an increased risk of developing distant metastases in the pigmentation (HR, 1.9; 95% CI, 1.05-3.28; P=0.03) and proliferative (HR, 2.8; 95% CI, 1.43-5.57; P=0.003) groups as compared to the high-immune response group. Further genetic characterization of melanomas using targeted deep-sequencing revealed similar mutational patterns across these phenotypes. We also used publicly available expression profiling data from melanoma patients treated with targeted or vaccine therapy in order to determine if our signatures predicted therapeutic response. In patients receiving targeted therapy, melanomas resistant to targeted therapy were enriched in the MITF-low proliferative subtype as compared to pre-treatment biopsies (P=0.02). In summary, the melanoma gene expression phenotypes are highly predictive of survival outcome and can further help to discriminate patients responding to targeted therapy.
|
|
| 41. |
- Cito, Jurgen, et al.
(författare)
-
Feedback from operations to software development—a devops perspective on runtime metrics and logs
- 2019
-
Ingår i: Lect. Notes Comput. Sci.. - Cham : Springer International Publishing. - 9783030060183 ; , s. 184-195
-
Konferensbidrag (refereegranskat)abstract
- DevOps achieve synergy between software development and operations engineers. This synergy can only happen if the right culture is in place to foster communication between these roles. We investigate the relationship between runtime data generated during production and how this data can be used as feedback in the software development process. For that, we want to discuss case study organizations that have different needs on their operations-to-development feedback pipeline, from which we abstract and propose a more general, higher-level feedback process. Given such a process, we discuss a technical environment required to support this process. We sketch out different scenarios in which feedback is useful in different phases of the software development life-cycle.
|
|
| 42. |
- de la Vara, José Luis, et al.
(författare)
-
An Industrial Survey of Safety Evidence Change Impact Analysis Practice
- 2016. - 13
-
Ingår i: IEEE Transactions on Software Engineering. - : IEEE. - 0098-5589 .- 1939-3520. ; 42:12, s. 1095-1117
-
Tidskriftsartikel (refereegranskat)abstract
- In many application domains, critical systems must comply with safety standards. This involves gathering safety evidence in the form of artefacts such as safety analyses, system specifications, and testing results. These artefacts can evolve during a system's lifecycle, creating a need for impact analysis to guarantee that system safety and compliance are not jeopardised. Although extensive research has been conducted on change impact analysis and on safety evidence management, the knowledge about how safety evidence change impact analysis is addressed in practice is limited. This paper reports on a survey targeted at filling this gap by analysing the circumstances under which safety evidence change impact analysis is addressed, the tool support used, and the challenges faced. We obtained 97 valid responses representing 16 application domains, 28 countries, and 47 safety standards. The results suggest that most practitioners deal with safety evidence change impact analysis during system development and mainly from system specifications. Furthermore, the level of automation in the process is low and insufficient tool support is the most frequent challenge. Other notable findings include that the different artefact types used as safety evidence seem to co-evolve, the evolution of safety case should probably be better managed, and no commercial impact analysis tool has been reported as used for all artefact types. Finally, we identified over 20 areas where the state of the practice in safety evidence change impact analysis can be improved.
|
|
| 43. |
- Erman, Nicklas, et al.
(författare)
-
Navigating Information Overload Caused by Automated Testing – A Clustering Approach in Multi-Branch Development
- 2015
-
Ingår i: 2015 IEEE 8th International Conference on Software Testing, Verification and Validation, ICST 2015 - Proceedings. - 9781479971251
-
Konferensbidrag (refereegranskat)abstract
- Background. Test automation is a widely used technique to increase the efficiency of software testing. However, executing more test cases increases the effort required to analyze test results. At Qlik, automated tests run nightly for up to 20 development branches, each containing thousands of test cases, resulting in information overload. Aim. We therefore develop a tool that supports the analysis of test results. Method. We create NIOCAT, a tool that clusters similar test case failures, to help the analyst identify underlying causes. To evaluate the tool, experiments on manually created subsets of failed test cases representing different use cases are conducted, and a focus group meeting is held with test analysts at Qlik. Results. The case study shows that NIOCAT creates accurate clusters, in line with analyses performed by human analysts. Further, the potential time-savings of our approach is confirmed by the participants in the focus group. Conclusions. NIOCAT provides a feasible complement to current automated testing practices at Qlik by reducing information overload.
|
|
| 44. |
- Hamilton, Paul, et al.
(författare)
-
Striatal dopamine deficits predict reductions in striatal functional connectivity in major depression: a concurrent C-11-raclopride positron emission tomography and functional magnetic resonance imaging investigation
- 2018
-
Ingår i: Translational Psychiatry. - : NATURE PUBLISHING GROUP. - 2158-3188. ; 8
-
Tidskriftsartikel (refereegranskat)abstract
- Major depressive disorder (MDD) is characterized by the altered integration of reward histories and reduced responding of the striatum. We have posited that this reduced striatal activation in MDD is due to tonically decreased stimulation of striatal dopamine synapses which results in decremented propagation of information along the corticostriatal-pallido-thalamic (CSPT) spiral. In the present investigation, we tested predictions of this formulation by conducting concurrent functional magnetic resonance imaging (fMRI) and C-11-raclopride positron emission tomography (PET) in depressed and control (CTL) participants. We scanned 16 depressed and 14 CTL participants with simultaneous fMRI and C-11-raclopride PET. We estimated raclopride binding potential (BPND), voxel-wise, and compared MDD and CTL samples with respect to BPND in the striatum. Using striatal regions that showed significant between-group BPND differences as seeds, we conducted whole-brain functional connectivity analysis using the fMRI data and identified brain regions in each group in which connectivity with striatal seed regions scaled linearly with BPND from these regions. We observed increased BPND in the ventral striatum, bilaterally, and in the right dorsal striatum in the depressed participants. Further, we found that as BPND increased in both the left ventral striatum and right dorsal striatum in MDD, connectivity with the cortical targets of these regions (default-mode network and salience network, respectively) decreased. Deficits in stimulation of striatal dopamine receptors in MDD could account in part for the failure of transfer of information up the CSPT circuit in the pathophysiology of this disorder.
|
|
| 45. |
- Henriksson, Jens, 1991, et al.
(författare)
-
Automotive safety and machine learning : Initial results from a study on how to adapt the ISO 26262 safety standard
- 2018
-
Ingår i: 2018 IEEE/ACM 1st International Workshop on Software Engineering for AI in Autonomous Systems (SEFAIAS). - New York, NY : ACM Publications. - 9781450357395 - 9781538662618 ; May 2018, s. 47-49
-
Konferensbidrag (refereegranskat)abstract
- Machine learning (ML) applications generate a continuous stream of success stories from various domains. ML enables many novel applications, also in safety-critical contexts. However, the functional safety standards such as ISO 26262 did not evolve to cover ML. We conduct an exploratory study on which parts of ISO 26262 represent the most critical gaps between safety engineering and ML development. While this paper only reports the first steps toward a larger research endeavor, we report three adaptations that are critically needed to allow ISO 26262 compliant engineering, and related suggestions on how to evolve the standard. © 2018 ACM.
|
|
| 46. |
- Henriksson, Jens, et al.
(författare)
-
Performance Analysis of Out-of-Distribution Detection on Various Trained Neural Networks
- 2019
-
Ingår i: Proceedings. 45th Euromicro Conference on Software Engineering and Advanced Applications. SEAA 2019. - Piscataway : IEEE. - 9781728134215 - 9781728134222 - 9781728132853 ; , s. 113-120
-
Konferensbidrag (refereegranskat)abstract
- Several areas have been improved with Deep Learning during the past years. For non-safety related products adoption of AI and ML is not an issue, whereas in safety critical applications, robustness of such approaches is still an issue. A common challenge for Deep Neural Networks (DNN) occur when exposed to out-of-distribution samples that are previously unseen, where DNNs can yield high confidence predictions despite no prior knowledge of the input. In this paper we analyse two supervisors on two well-known DNNs with varied setups of training and find that the outlier detection performance improves with the quality of the training procedure. We analyse the performance of the supervisor after each epoch during the training cycle, to investigate supervisor performance as the accuracy converges. Understanding the relationship between training results and supervisor performance is valuable to improve robustness of the model and indicates where more work has to be done to create generalized models for safety critical applications. © 2019 IEEE
|
|
| 47. |
- Henriksson, Jens, 1991, et al.
(författare)
-
Towards Structured Evaluation of Deep Neural Network Supervisors
- 2019
-
Ingår i: Proceedings - 2019 IEEE International Conference on Artificial Intelligence Testing, AITest 2019. - New York : Institute of Electrical and Electronics Engineers Inc.. - 9781728104928 ; 1
-
Konferensbidrag (refereegranskat)abstract
- Deep Neural Networks (DNN) have improved the quality of several non-safety related products in the past years. However, before DNNs should be deployed to safety-critical applications, their robustness needs to be systematically analyzed. A common challenge for DNNs occurs when input is dissimilar to the training set, which might lead to high confidence predictions despite proper knowledge of the input. Several previous studies have proposed to complement DNNs with a supervisor that detects when inputs are outside the scope of the network. Most of these supervisors, however, are developed and tested for a selected scenario using a specific performance metric. In this work, we emphasize the need to assess and compare the performance of supervisors in a structured way. We present a framework constituted by four datasets organized in six test cases combined with seven evaluation metrics. The test cases provide varying complexity and include data from publicly available sources as well as a novel dataset consisting of images from simulated driving scenarios. The latter we plan to make publicly available. Our framework can be used to support DNN supervisor evaluation, which in turn could be used to motive development, validation, and deployment of DNNs in safety-critical applications.
|
|
| 48. |
|
|
| 49. |
- Jonsson, Leif, et al.
(författare)
-
Automated Bug Assignment: Ensemble-based Machine Learning in Large Scale Industrial Contexts
- 2015
-
Ingår i: Empirical Software Engineering. - : Springer Science and Business Media LLC. - 1573-7616 .- 1382-3256. ; 21:4
-
Tidskriftsartikel (refereegranskat)abstract
- Bug report assignment is an important part of software maintenance. In particular, incorrect assignments of bug reports to development teams can be very expensive in large software development projects. Several studies propose automating bug assignment techniques using machine learning in open source software contexts, but no study exists for large-scale proprietary projects in industry. The goal of this study is to evaluate automated bug assignment techniques that are based on machine learning classification. In particular, we study the state-of-the-art ensemble learner Stacked Generalization (SG) that combines several classifiers. We collect more than 50,000 bug reports from five development projects from two companies in different domains. We implement automated bug assignment and evaluate the performance in a set of controlled experiments. We show that SG scales to large scale industrial application and that it outperforms the use of individual classifiers for bug assignment, reaching prediction accuracies from 50 % to 89 % when large training sets are used. In addition, we show how old training data can decrease the prediction accuracy of bug assignment. We advice industry to use SG for bug assignment in proprietary contexts, using at least 2,000 bug reports for training. Finally, we highlight the importance of not solely relying on results from cross-validation when evaluating automated bug assignment.
|
|
| 50. |
- Larsson, Jacob, et al.
(författare)
-
Testing Quality Requirements of a System-of-Systems in the Public Sector - Challenges and Potential Remedies
- 2016. - 8
-
Ingår i: CEUR Workshop Proceedings.
-
Konferensbidrag (refereegranskat)abstract
- Quality requirements is a difficult concept in software projects, and testing software qualities is a well-known challenge. Without proper management of quality requirements, there is an increased risk that the software product un-der development will not meet the expectations of its future users. In this pa-per, we share experiences from testing quality requirements when developing a large system-of-systems in the public sector in Sweden. We complement the experience reporting by analyzing documents from the case under study. As a final step, we match the identified challenges with solution proposals from the literature. We report five main challenges covering inadequate re-quirements engineering and disconnected test managers. Finally, we match the challenges to solutions proposed in the scientific literature, including in-tegrated requirements engineering, the twin peaks model, virtual plumblines, the QUPER model, and architecturally significant requirements. Our experi-ences are valuable to other large development projects struggling with testing of quality requirements. Furthermore, the report could be used by as input to process improvement activities in the case under study.
|
|
