| 1. |
- Alégroth, Emil, 1984-, et al.
(författare)
-
Characteristics that affect Preference of Decision Models for Asset Selection : An Industrial Questionnaire Survey
- 2020
-
Ingår i: Software quality journal. - : Springer. - 0963-9314 .- 1573-1367. ; 28:4, s. 1675-1707
-
Tidskriftsartikel (refereegranskat)abstract
- Modern software development relies on a combination of development and re-use of technical asset, e.g. software components, libraries and APIs.In the past, re-use was mostly conducted with internal assets but today external; open source, customer off-the-shelf (COTS) and assets developed through outsourcing are also common.This access to more asset alternatives presents new challenges regarding what assets to optimally chose and how to make this decision.To support decision-makers, decision-theory has been used to develop decision models for asset selection.However, very little industrial data has been presented in literature about the usefulness, or even perceived usefulness, of these models.Additionally, only limited information has been presented about what model characteristics that determine practitioner preference towards one model over another.Objective: The objective of this work is to evaluate what characteristics of decision models for asset selection that determine industrial practitioner preference of a model when given the choice of a decision-model of high precision or a model with high speed.Method: An industrial questionnaire survey is performed where a total of 33 practitioners, of varying roles, from 18 companies are tasked to compare two decision models for asset selection.Textual analysis and formal and descriptive statistics are then applied on the survey responses to answer the study's research questions.Results: The study shows that the practitioners had clear preference towards the decision model that emphasised speed over the one that emphasised decision precision.This conclusion was determined to be because one of the models was perceived faster, had lower complexity, had, was more flexible in use for different decisions, was more agile how it could be used in operation, its emphasis on people, its emphasis on ``good enough'' precision and ability to fail fast if a decision was a failure.Hence, seven characteristics that the practitioners considered important for their acceptance of the model.Conclusion: Industrial practitioner preference, which relates to acceptance, of decision models for asset selection is dependent on multiple characteristics that must be considered when developing a model for different types of decisions such as operational day-to-day decisions as well as more critical tactical or strategic decisions.The main contribution of this work are seven identified characteristics that can serve as industrial requirements for future research on decision models for asset selection.
|
|
| 2. |
- Borg, Markus, et al.
(författare)
-
Ergo, SMIRK is safe : a safety case for a machine learning component in a pedestrian automatic emergency brake system
- 2023
-
Ingår i: Software quality journal. - : Springer. - 0963-9314 .- 1573-1367. ; 31:2, s. 335-
-
Tidskriftsartikel (refereegranskat)abstract
- Integration of machine learning (ML) components in critical applications introduces novel challenges for software certification and verification. New safety standards and technical guidelines are under development to support the safety of ML-based systems, e.g., ISO 21448 SOTIF for the automotive domain and the Assurance of Machine Learning for use in Autonomous Systems (AMLAS) framework. SOTIF and AMLAS provide high-level guidance but the details must be chiseled out for each specific case. We initiated a research project with the goal to demonstrate a complete safety case for an ML component in an open automotive system. This paper reports results from an industry-academia collaboration on safety assurance of SMIRK, an ML-based pedestrian automatic emergency braking demonstrator running in an industry-grade simulator. We demonstrate an application of AMLAS on SMIRK for a minimalistic operational design domain, i.e., we share a complete safety case for its integrated ML-based component. Finally, we report lessons learned and provide both SMIRK and the safety case under an open-source license for the research community to reuse. © 2023, The Author(s).
|
|
| 3. |
|
|
| 4. |
- Butler, Simon, et al.
(författare)
-
On business adoption and use of reproducible builds for open and closed source software
- 2023
-
Ingår i: Software quality journal. - : Springer Nature Switzerland AG. - 0963-9314 .- 1573-1367. ; 31:3, s. 687-719
-
Tidskriftsartikel (refereegranskat)abstract
- Reproducible builds (R-Bs) are software engineering practices that reliably create bit-for-bit identical binary executable files from specified source code. R-Bs are applied in someopen source software (OSS) projects and distributions to allow verification that the distrib-uted binary has been built from the released source code. The use of R-Bs has been advo-cated in software maintenance and R-Bs are applied in the development of some OSS secu-rity applications. Nonetheless, industry application of R-Bs appears limited, and we seekto understand whether awareness is low or if significant technical and business reasonsprevent wider adoption. Through interviews with software practitioners and business man-agers, this study explores the utility of applying R-Bs in businesses in the primary and sec-ondary software sectors and the business and technical reasons supporting their adoption.We find businesses use R-Bs in the safety-critical and security domains, and R-Bs are valu-able for traceability and support collaborative software development. We also found thatR-Bs are valued as engineering processes and are seen as a badge of software quality, butwithout a tangible value proposition. There are good engineering reasons to use R-Bs inindustrial software development, and the principle of establishing correspondence betweensource code and binary offers opportunities for the development of further applications.
|
|
| 5. |
- Chatzipetrou, Panagiota, Assistant Professor, 1984-, et al.
(författare)
-
Component attributes and their importance in decisions and component selection
- 2020
-
Ingår i: Software quality journal. - : Springer Science and Business Media LLC. - 0963-9314 .- 1573-1367. ; 28, s. 567-593
-
Tidskriftsartikel (refereegranskat)abstract
- Component-based software engineering is a common approach in the development and evolution of contemporary software systems. Different component sourcing options are available, such as: (1) Software developed internally (in-house), (2) Software developed outsourced, (3) Commercial off-the-shelf software, and (4) Open-Source Software. However, there is little available research on what attributes of a component are the most important ones when selecting new components. The objective of this study is to investigate what matters the most to industry practitioners when they decide to select a component. We conducted a cross-domain anonymous survey with industry practitioners involved in component selection. First, the practitioners selected the most important attributes from a list. Next, they prioritized their selection using the Hundred-Dollar ($100) test. We analyzed the results using compositional data analysis. The results of this exploratory analysis showed that cost was clearly considered to be the most important attribute for component selection. Other important attributes for the practitioners were: support of the component, longevity prediction, and level of off-the-shelf fit to product. Moreover, several practitioners still consider in-house software development to be the sole option when adding or replacing a component. On the other hand, there is a trend to complement it with other component sourcing options and, apart from cost, different attributes factor into their decision. Furthermore, in our analysis, nonparametric tests and biplots were used to further investigate the practitioners’ inherent characteristics. It seems that smaller and larger organizations have different views on what attributes are the most important, and the most surprising finding is their contrasting views on the cost attribute: larger organizations with mature products are considerably more cost aware.
|
|
| 6. |
- Eken, Beyza, et al.
(författare)
-
An empirical study on the effect of community smells on bug prediction
- 2021
-
Ingår i: Software quality journal. - : Springer. - 0963-9314 .- 1573-1367. ; 29, s. 159-194
-
Tidskriftsartikel (refereegranskat)abstract
- Community-aware metrics through socio-technical developer networks or organizational structures have already been studied in the software bug prediction field. Community smells are also proposed to identify communication and collaboration patterns in developer communities. Prior work reports a statistical association between community smells and code smells identified in software modules. We investigate the contribution of community smells on predicting bug-prone classes and compare their contribution with that of code smell-related information and state-of-the-art process metrics. We conduct our empirical analysis on ten open-source projects with varying sizes, buggy and smelly class ratios. We build seven different bug prediction models to answer three RQs: a baseline model including a state-of-the-art metric set used, three models incorporating a particular metric set, namely community smells, code smells, code smell intensity, into the baseline, and three models incorporating a combination of smell-related metrics into the baseline. The performance of these models is reported in terms of recall, false positive rates, F-measure and AUC and statistically compared using Scott-Knott ESD tests. Community smells improve the prediction performance of a baseline model by up to 3% in terms of AUC, while code smell intensity improves the baseline models by up to 40% in terms of F-measure and up to 17% in terms of AUC. The conclusions are significantly influenced by the validation strategies used, algorithms and the selected projects' data characteristics. While the code smell intensity metric captures the most information about technical flaws in predicting bug-prone classes, the community smells also contribute to bug prediction models by revealing communication and collaboration flaws in software development teams. Future research is needed to capture the communication patterns through multiple channels and to understand whether socio-technical flaws could be used in a cross-project bug prediction setting.
|
|
| 7. |
- Helali Moghadam, Mahshid, et al.
(författare)
-
An autonomous performance testing framework using self-adaptive fuzzy reinforcement learning
- 2022
-
Ingår i: Software quality journal. - : Springer. - 0963-9314 .- 1573-1367. ; , s. 127-159
-
Tidskriftsartikel (refereegranskat)abstract
- Test automation brings the potential to reduce costs and human effort, but several aspects of software testing remain challenging to automate. One such example is automated performance testing to find performance breaking points. Current approaches to tackle automated generation of performance test cases mainly involve using source code or system model analysis or use-case-based techniques. However, source code and system models might not always be available at testing time. On the other hand, if the optimal performance testing policy for the intended objective in a testing process instead could be learned by the testing system, then test automation without advanced performance models could be possible. Furthermore, the learned policy could later be reused for similar software systems under test, thus leading to higher test efficiency. We propose SaFReL, a self-adaptive fuzzy reinforcement learning-based performance testing framework. SaFReL learns the optimal policy to generate performance test cases through an initial learning phase, then reuses it during a transfer learning phase, while keeping the learning running and updating the policy in the long term. Through multiple experiments in a simulated performance testing setup, we demonstrate that our approach generates the target performance test cases for different programs more efficiently than a typical testing process and performs adaptively without access to source code and performance models. © 2021, The Author(s).
|
|
| 8. |
- Karlsson, Stefan, et al.
(författare)
-
Exploring API behaviours through generated examples
- 2024
-
Ingår i: Software Quality Journal. - : SPRINGER. - 1573-1367 .- 0963-9314. ; 32:2, s. 729-763
-
Tidskriftsartikel (refereegranskat)abstract
- Understanding the behaviour of a system's API can be hard. Giving users access to relevant examples of how an API behaves has been shown to make this easier for them. In addition, such examples can be used to verify expected behaviour or identify unwanted behaviours. Methods for automatically generating examples have existed for a long time. However, state-of-the-art methods rely on either white-box information, such as source code, or on formal specifications of the system behaviour. But what if you do not have access to either? This may be the case, for example, when interacting with a third-party API. In this paper, we present an approach to automatically generate relevant examples of behaviours of an API, without requiring either source code or a formal specification of behaviour. Evaluation on an industry-grade REST API shows that our method can produce small and relevant examples that can help engineers to understand the system under exploration.
|
|
| 9. |
- Karlsson, Stefan, et al.
(författare)
-
Exploring behaviours of RESTful APIs in an industrial setting
- 2024
-
Ingår i: Software quality journal. - : Springer. - 0963-9314 .- 1573-1367.
-
Tidskriftsartikel (refereegranskat)abstract
- A common way of exposing functionality in contemporary systems is by providing a Web-API based on the REST API architectural guidelines. To describe REST APIs, the industry standard is currently OpenAPI-specifications. Test generation and fuzzing methods targeting OpenAPI-described REST APIs have been a very active research area in recent years. An open research challenge is to aid users in better understanding their API, in addition to finding faults and to cover all the code. In this paper, we address this challenge by proposing a set of behavioural properties, common to REST APIs, which are used to generate examples of behaviours that these APIs exhibit. These examples can be used both (i) to further the understanding of the API and (ii) as a source of automatic test cases. Our evaluation shows that our approach can generate examples deemed relevant for understanding the system and for a source of test generation by practitioners. In addition, we show that basing test generation on behavioural properties provides tests that are less dependent on the state of the system, while at the same time yielding a similar code coverage as state-of-the-art methods in REST API fuzzing in a given time limit.
|
|
| 10. |
- Minhas, Nasir Mehmood, et al.
(författare)
-
Lessons learned from replicating a study on information-retrieval based test case prioritization
- 2023
-
Ingår i: Software quality journal. - : Springer. - 0963-9314 .- 1573-1367. ; 31:4, s. 1527-1559
-
Tidskriftsartikel (refereegranskat)abstract
- Replication studies help solidify and extend knowledge by evaluating previous studies’ findings. Software engineering literature showed that too few replications are conducted focusing on software artifacts without the involvement of humans. This study aims to replicate an artifact-based study on software testing to address the gap related to replications. In this investigation, we focus on (i) providing a step-by-step guide of the replication, reflecting on challenges when replicating artifact-based testing research and (ii) evaluating the replicated study concerning the validity and robustness of the findings. We replicate a test case prioritization technique proposed by Kwon et al. We replicated the original study using six software programs, four from the original study and two additional software programs. We automated the steps of the original study using a Jupyter notebook to support future replications. Various general factors facilitating replications are identified, such as (1) the importance of documentation; (2) the need for assistance from the original authors; (3) issues in the maintenance of open-source repositories (e.g., concerning needed software dependencies, versioning); and (4) availability of scripts. We also noted observations specific to the study and its context, such as insights from using different mutation tools and strategies for mutant generation. We conclude that the study by Kwon et al. is partially replicable for small software programs and could be automated to facilitate software practitioners, given the availability of required information. However, it is hard to implement the technique for large software programs with the current guidelines. Based on lessons learned, we suggest that the authors of original studies need to publish their data and experimental setup to support the external replications. © 2023, The Author(s).
|
|
| 11. |
- Saeeda, Hina, et al.
(författare)
-
Navigating social debt and its link with technical debt in large-scale agile software development projects
- 2024
-
Ingår i: Software quality journal. - : Springer. - 0963-9314 .- 1573-1367.
-
Tidskriftsartikel (refereegranskat)abstract
- Agile methodologies have emerged as transformative paradigms in the ever-evolving software development landscape, emphasizing iterative development, customer collaboration, and adaptability. As the scope and complexity of projects and organizations expand, applying agile principles within the context of Large-Scale Agile Development (LSAD) encounters distinctive challenges. The majority of challenges encountered in LSAD, technical and non-technical, are attributed to the accrual of social debt. However, a conspicuous gap remains in understanding and addressing social debt in LSAD. This study aims to fill this void by investigating social debt in LSAD through an in-depth industrial case study with a leading Nordic company specializing in telecommunications software and services and focusing on producing secure 5G network solutions. The study investigates the causes of LSAD's social debt and examines its impacts on secure 5G telecom software development. By addressing these objectives, this research sheds light on a critical aspect of LSAD's social debt, caused by 3C challenges(communication, coordination and collaboration), social confines challenges, community smells challenges, and organisational social challenges in the telecom sector that have been underrepresented in the existing literature.
|
|
| 12. |
- Song, Qunying, et al.
(författare)
-
Critical scenario identification for realistic testing of autonomous driving systems
- 2023
-
Ingår i: Software quality journal. - : Springer Nature. - 0963-9314 .- 1573-1367. ; 31:2, s. 441-469
-
Tidskriftsartikel (refereegranskat)abstract
- Autonomous driving has become an important research area for road traffic, whereas testing of autonomous driving systems to ensure a safe and reliable operation remains an open challenge. Substantial real-world testing or massive driving data collection does not scale since the potential test scenarios in real-world traffic are infinite, and covering large shares of them in the test is impractical. Thus, critical ones have to be prioritized. We have developed an approach for critical test scenario identification and in this study, we implement the approach and validate it on two real autonomous driving systems from industry by integrating it into their tool-chain. Our main contribution in this work is the demonstration and validation of our approach for critical scenario identification for testing real autonomous driving systems.
|
|
| 13. |
- Ulan, Maria, et al.
(författare)
-
Copula-based software metrics aggregation
- 2021
-
Ingår i: Software quality journal. - : Springer. - 0963-9314 .- 1573-1367. ; 29, s. 863-899
-
Tidskriftsartikel (refereegranskat)abstract
- A quality model is a conceptual decomposition of an abstract notion of quality into relevant, possibly conflicting characteristics and further into measurable metrics. For quality assessment and decision making, metrics values are aggregated to characteristics and ultimately to quality scores. Aggregation has often been problematic as quality models do not provide the semantics of aggregation. This makes it hard to formally reason about metrics, characteristics, and quality. We argue that aggregation needs to be interpretable and mathematically well defined in order to assess, to compare, and to improve quality. To address this challenge, we propose a probabilistic approach to aggregation and define quality scores based on joint distributions of absolute metrics values. To evaluate the proposed approach and its implementation under realistic conditions, we conduct empirical studies on bug prediction of ca. 5000 software classes, maintainability of ca. 15000 open-source software systems, and on the information quality of ca. 100000 real-world technical documents. We found that our approach is feasible, accurate, and scalable in performance.
|
|
| 14. |
- Kudran Pradhan, Shameer, et al.
(författare)
-
Identifying and managing data quality requirements: a design science study in the field of automated driving
- 2023
-
Ingår i: Software quality journal. - 0963-9314.
-
Tidskriftsartikel (refereegranskat)abstract
- Good data quality is crucial for any data-driven system’s effective and safe operation. For critical safety systems, the significance of data quality is even higher since incorrect or low-quality data may cause fatal faults. However, there are challenges in identifying and managing data quality. In particular, there is no accepted process to define and continuously test data quality concerning what is necessary for operating the system. This lack is problematic because even safety-critical systems become increasingly dependent on data. Here, we propose a Candidate Framework for Data Quality Assessment and Maintenance (CaFDaQAM) to systematically manage data quality and related requirements based on design science research. The framework is constructed based on an advanced driver assistance system (ADAS) case study. The study is based on empirical data from a literature review, focus groups, and design workshops. The proposed framework consists of four components: a Data Quality Workflow, a List of Data Quality Challenges, a List of Data Quality Attributes, and Solution Candidates. Together, the components act as tools for data quality assessment and maintenance. The candidate framework and its components were validated in a focus group.
|
|
