| 1. |
- de la Vara, Jose Luis, et al.
(författare)
-
A Proposal for the Classification of Methods for Verification and Validation of Safety, Cybersecurity, and Privacy of Automated Systems
- 2021
-
Ingår i: 14th International Conference on the Quality of Information and Communications Technology (QUATIC 2021).
-
Konferensbidrag (refereegranskat)abstract
- As our dependence on automated systems grows, so does the need for guaranteeing their safety, cybersecurity, and privacy (SCP). Dedicated methods for verification and validation (V&V) must be used to this end and it is necessary that the methods and their characteristics can be clearly differentiated. This can be achieved via method classifications. However, we have experienced that existing classifications are not suitable to categorise V&V methods for SCP of automated systems. They do not pay enough attention to the distinguishing characteristics of this system type and of these quality concerns. As a solution, we present a new classification developed in the scope of a large-scale industry-academia project. The classification considers both the method type, e.g., testing, and the concern addressed, e.g., safety. Over 70 people have successfully used the classification on 53 methods. We argue that the classification is a more suitable means to categorise V&V methods for SCP of automated systems and that it can help other researchers and practitioners.
|
|
| 2. |
- Agirre, Joseba A, et al.
(författare)
-
Multidimensional Framework for Characterizing Verification and Validation of Automated Systems
- 2022
-
Ingår i: 18<sup>th</sup> European dependable computing conference (EDCC 2022).
-
Konferensbidrag (refereegranskat)abstract
- Verification and Validation (V&V) of automated systems is becoming more costly and time-consuming because of the increasing size and complexity of these systems. Moreover, V&V of these systems can be hindered if the methods and processes are not properly described, analysed, and selected. It is essential that practitioners use suitable V&V methods and enact adequate V&V processes to confirm that these systems work as intended and in a cost-effective manner. Previous works have created different taxonomies and models considering different aspects of V&V that can be used to classify V&V methods and tools. The aim of this work is to provide a broad, comprehensive and a easy to use framework that addresses characterisation needs, rather than focusing on individual aspects of V&V methods and processes.To this end, in this paper, we present a multi-domain and multi-dimensional framework to characterize and classify V&V methods and tools in a structured way. The framework considers a comprehensive characterization of different relevant aspects of V&V. A web-based repository has been implemented on the basis of the framework, as an example of use, in order to collect information about the application of V&V methods and tools. This way, practitioners and researchers can easily learn about and identify suitable V&V processes.
|
|
| 3. |
- Borg, Markus, et al.
(författare)
-
Practitioners' Perspectives on Change Impact Analysis for Safety-Critical Software - A Preliminary Analysis
- 2016. - 11
-
Ingår i: Computer Safety, Reliability, and Security. - Cham : Springer International Publishing. - 9783319454795 - 9783319454801 ; , s. 346-358
-
Konferensbidrag (refereegranskat)abstract
- Safety standards prescribe change impact analysis (CIA) during evolution of safety-critical software systems. Although CIA is a fundamental activity, there is a lack of empirical studies about how it is performed in practice. We present a case study on CIA in the context of an evolving automation system, based on 14 interviews in Sweden and India. Our analysis suggests that engineers on average spend 50-100 hours on CIA per year, but the effort varies considerably with the phases of projects. Also, the respondents presented different connotations to CIA and perceived the importance of CIA differently. We report the most pressing CIA challenges, and several ideas on how to support future CIA. However, we show that measuring the effect of such improvement solutions is non-trivial, as CIA is intertwined with other development activities. While this paper only reports preliminary results, our work contributes empirical insights into practical CIA.
|
|
| 4. |
|
|
| 5. |
|
|
| 6. |
- de la Vara, José Luis, et al.
(författare)
-
An Industrial Survey of Safety Evidence Change Impact Analysis Practice
- 2016. - 13
-
Ingår i: IEEE Transactions on Software Engineering. - : IEEE. - 0098-5589 .- 1939-3520. ; 42:12, s. 1095-1117
-
Tidskriftsartikel (refereegranskat)abstract
- In many application domains, critical systems must comply with safety standards. This involves gathering safety evidence in the form of artefacts such as safety analyses, system specifications, and testing results. These artefacts can evolve during a system's lifecycle, creating a need for impact analysis to guarantee that system safety and compliance are not jeopardised. Although extensive research has been conducted on change impact analysis and on safety evidence management, the knowledge about how safety evidence change impact analysis is addressed in practice is limited. This paper reports on a survey targeted at filling this gap by analysing the circumstances under which safety evidence change impact analysis is addressed, the tool support used, and the challenges faced. We obtained 97 valid responses representing 16 application domains, 28 countries, and 47 safety standards. The results suggest that most practitioners deal with safety evidence change impact analysis during system development and mainly from system specifications. Furthermore, the level of automation in the process is low and insufficient tool support is the most frequent challenge. Other notable findings include that the different artefact types used as safety evidence seem to co-evolve, the evolution of safety case should probably be better managed, and no commercial impact analysis tool has been reported as used for all artefact types. Finally, we identified over 20 areas where the state of the practice in safety evidence change impact analysis can be improved.
|
|
| 7. |
- de la Vara, José Luis, et al.
(författare)
-
Survey on Safety Evidence Change Impact Analysis in Practice: Detailed Description and Analysis
- 2014
-
Rapport (övrigt vetenskapligt/konstnärligt)abstract
- Critical systems must comply with safety standards in many application domains. This involves gathering safety evidence in the form of artefacts such as safety analyses, system specifications, and testing results. These artefacts can evolve during a system’s lifecycle, and impact analysis might be necessary to guarantee that system safety and compliance are not jeopardised. Although extensive research has been conducted on impact analysis and on safety evidence management, the knowledge about how safety evidence change impact analysis is addressed in practice is limited. This technical report presents a survey targeted at filling this gap by analysing the circumstances under which safety evidence change impact analysis is addressed, the tool support used, and the challenges faced. We obtained 97 valid responses representing 16 application domains, 28 countries, and 47 safety standards. The results suggest that most projects deal with safety evidence change impact analysis during system development and mainly from system specifications, the level of automation in the process is low, and insufficient tool support is the most frequent challenge. Other notable findings are that safety case evolution should probably be better managed, no commercial impact analysis tool has been reported as used for all artefact types, and experience and automation do not seem to greatly help in avoiding challenges.
|
|
| 8. |
- Ferrari, Enrico, et al.
(författare)
-
Criteria for the Analysis of Gaps and Limitations of V&V Methods for Safety- and Security-Critical Systems
- 2022
-
Konferensbidrag (refereegranskat)abstract
- As society increasingly relies on safety- and security- critical systems, the need for confirming their dependability becomes essential. Adequate V&V (verification and validation) methods must be employed, e.g., for system testing. When selecting and using the methods, it is important to analyze their possible gaps and limitations, such as scalability issues. However, and as we have experienced, common, explicitly defined criteria are seldom used for such analyses. This results in analyses that consider different aspects and to a different extent, hindering their comparison and thus the comparison of the V&V methods. As a solution, we present a set of criteria for the analysis of gaps and limitations of V&V methods for safety- and security-critical systems. The criteria have been identified in the scope of the VALU3S project. Sixty-two people from 33 organizations agreed upon the use of nine criteria: functionality, accuracy, scalability, deployment, learning curve, automation, reference environment, cost, and standards. Their use led to more homogeneous and more detailed analyses when compared to similar previous efforts. We argue that the proposed criteria can be helpful to others when having to deal with similar activities.
|
|
| 9. |
- Luis de la Vara, Jose, et al.
(författare)
-
The AMASS Approach for Assurance and Certification of Critical Systems
- 2019
-
Ingår i: embedded world 2019 ewC-2019.
-
Konferensbidrag (refereegranskat)abstract
- Safety-critical systems are subject to rigorous assurance and certification processes to guarantee that they do not pose unreasonable risks to people, property, or the environment. The associated activities are usually complex and time-consuming, thus they need adequate support for their execution. The activities are further becoming more challenging as the systems are evolving towards open, interconnected systems with new features, e.g. Internet connectivity, and new assurance needs, e.g. compliance with several assurance standards for different dependability attributes. This requires the development of novel approaches for cost-effective assurance and certification. With the overall goal of lowering assurance and certification costs in face of rapidly changing features and market needs, the AMASS project has created and consolidated the de-facto European-wide open solution for assurance and certification of critical systems. This has been achieved by establishing a novel holistic and reuse-oriented approach for architecture-driven assurance, multi-concern assurance, and for seamless interoperability between assurance and engineering activities along with third-party activities. This paper introduces the main elements of the AMASS approach and how to use them and benefit from them.
|
|
| 10. |
- Ruiz, Alejandra, et al.
(författare)
-
Architecture-driven, Multi-concern and Seamless Assurance and Certification of Cyber-Physical Systems
- 2016
-
Ingår i: COMPUTER SAFETY, RELIABILITY, AND SECURITY, SAFECOMP 2016. - Cham : Springer International Publishing. - 9783319454795 - 9783319454801 ; , s. 311-321
-
Konferensbidrag (refereegranskat)abstract
- Unlike practices in electrical and mechanical equipment engineering, Cyber-Physical Systems (CPS) do not have a set of standardized and harmonized practices for assurance and certification that ensures safe, secure and reliable operation with typical software and hardware architectures. This paper presents a recent initiative called AMASS (Architecture-driven, Multi-concern and Seamless Assurance and Certification of Cyber-Physical Systems) to promote harmonization, reuse and automation of labour-intensive certification-oriented activities via using model-based approaches and incremental techniques. AMASS will develop an integrated and holistic approach, a supporting tool ecosystem and a self-sustainable community for assurance and certification of CPS. The approach will be driven by architectural decisions (fully compatible with standards, e.g. AUTOSAR and IMA), including multiple assurance concerns such as safety, security and reliability. AMASS will support seamless interoperability between assurance/certification and engineering activities along with third-party activities (external assessments, supplier assurance). The ultimate aim is to lower certification costs in face of rapidly changing product features and market needs.
|
|
| 11. |
|
|
| 12. |
- Smrčka, Aleš, et al.
(författare)
-
Towards an extensive set of criteria for safety and cyber-security evaluation of cyber-physical systems
- 2023
-
Ingår i: Open Research Europe. - 2732-5121. - 9781665485555 ; 3
-
Tidskriftsartikel (refereegranskat)abstract
- Verification and validation (V&V) are complex processes combining different approaches and incorporating many different methods including many activities. System engineers regularly face the question if their V&V activities lead to better products, and having appropriate criteria at hand for evaluation of safety and cybersecurity of the systems would help to answer such a question. Additionally, when there is a demand to improve the quality of an already managed V&V process, there is a struggle over what criteria to use in order to measure the improvement. This paper presents an extensive set of criteria suitable for safety and cybersecurity evaluation of cyberphysical systems. The evaluation criteria are agreed upon by 60 researchers from 32 academic and industrial organizations jointly working in a large-scale European research project on 13 real-world use cases from the domains of automotive, railway, aerospace, agriculture, healthcare, and industrial robotics.
|
|
| 13. |
- Sulaman, Sardar Muhammad, et al.
(författare)
-
Development of Safety-Critical Software Systems Using Open Source Software - A Systematic Map
- 2014
-
Ingår i: [Host publication title missing]. - 1089-6503. ; , s. 17-24
-
Konferensbidrag (refereegranskat)abstract
- The popularity of Open Source Software (OSS) has increased the interest in using it in safety critical applications. The aim of this study is to review research carried out on usage of open source code in development of safety-critical software and systems. We conducted a systematic mapping study through searches in library databases and manual identification of articles from open source conferences.We have identified 22 studies about using open source software, mainly in automotive, aerospace, medical and nuclear domains. Moreover, only a few studies present complete safety systems that are released as OSS in full. The most commonly used OSS functionalities are operating systems, imaging, control and data management. Finally most of the integrated OSS have mature code bases and a commit history of more than five years.
|
|
| 14. |
- Wagner, Stefan, et al.
(författare)
-
Status quo in requirements engineering : A theory and a global family of surveys
- 2019
-
Ingår i: ACM Transactions on Software Engineering and Methodology. - : Association for Computing Machinery (ACM). - 1049-331X .- 1557-7392. ; 28:2, s. 1-48
-
Tidskriftsartikel (refereegranskat)abstract
- Requirements Engineering (RE) has established itself as a software engineering discipline over the past decades. While researchers have been investigating the RE discipline with a plethora of empirical studies, attempts to systematically derive an empirical theory in context of the RE discipline have just recently been started. However, such a theory is needed if we are to define and motivate guidance in performing high quality RE research and practice. We aim at providing an empirical and externally valid foundation for a theory of RE practice, which helps software engineers establish effective and efficient RE processes in a problem-driven manner. We designed a survey instrument and an engineer-focused theory that was first piloted in Germany and, after making substantial modifications, has now been replicated in 10 countries worldwide. We have a theory in the form of a set of propositions inferred from our experiences and available studies, as well as the results from our pilot study in Germany. We evaluate the propositions with bootstrapped confidence intervals and derive potential explanations for the propositions. In this article, we report on the design of the family of surveys, its underlying theory, and the full results obtained from the replication studies conducted in 10 countries with participants from 228 organisations. Our results represent a substantial step forward towards developing an empirical theory of RE practice. The results reveal, for example, that there are no strong differences between organisations in different countries and regions, that interviews, facilitated meetings and prototyping are the most used elicitation techniques, that requirements are often documented textually, that traces between requirements and code or design documents are common, that requirements specifications themselves are rarely changed and that requirements engineering (process) improvement endeavours are mostly internally driven. Our study establishes a theory that can be used as starting point for many further studies for more detailed investigations. Practitioners can use the results as theory-supported guidance on selecting suitable RE methods and techniques.
|
|
| 15. |
- Wagner, Stefan, et al.
(författare)
-
Status Quo in Requirements Engineering: A Theory and a Global Family of Surveys
- 2020
-
Ingår i: Lecture Notes in Informatics. - 1617-5468. ; P-310, s. 115-116
-
Konferensbidrag (refereegranskat)abstract
- While researchers have been investigating the Requirements Engineering (RE) discipline with a plethora of empirical studies, attempts to systematically derive an empirical theory in context of the RE discipline have just recently been started. We aim at providing an empirical and externally valid foundation for a theory of RE practice, which helps software engineers establish effective and efficient RE processes in a problem-driven manner. We designed a survey instrument and an engineer-focused theory that has been conducted in 10 countries. We have a theory in the form of a set of propositions inferred from our experiences and available studies, as well as the results from our pilot study in Germany. We evaluate the propositions with bootstrapped confidence intervals and derive potential explanations for the propositions.
|
|
