| 1. |
- Carlsson, Magnus, et al.
(författare)
-
Enabling Uncertainty Quantification of Large Aircraft System Simulation Models
- 2013
-
Ingår i: 4:th CEAS conference, 2013. - : Linköping University Electronic Press. - 9789175195193
-
Konferensbidrag (refereegranskat)abstract
- A common viewpoint in both academia and industry is that that Verification, Validation and Uncertainty Quantification (VV&UQ) of simulation models are vital activities for a successful deployment of model-based system engineering. In the literature, there is no lack of advice regarding methods for VV&UQ. However, for industrial applications available methods for Uncertainty Quantification (UQ) often seem too detailed or tedious to even try. The consequence is that no UQ is performed, resulting in simulation models not being used to their full potential.In this paper, the effort required for UQ of a detailed aircraft vehicle system model is estimated. A number of methodological steps that aim to achieve a more feasible UQ are proposed. The paper is focused on 1‑D dynamic simulation models of physical systems with or without control software, typically described by Ordinary Differential Equations (ODEs) or Differential Algebraic Equations (DAEs). An application example of an aircraft vehicle system model is used for method evaluation.
|
|
| 2. |
- Carlsson, Magnus, et al.
(författare)
-
Evaluating Model Uncertainty Based on Probabilistic Analysis and Component Output Uncertainty Descriptions
- 2012
-
Ingår i: Proceedings of the ASME 2012 International Mechanical Engineering Congress & Exposition.
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
- To support early model validation, this paper describes a method utilizing information obtained from the common practice component level validation to assess uncertainties on model top level. Initiated in previous research, a generic output uncertainty description component, intended for power-port based simulation models of physical systems, has been implemented in Modelica. A set of model components has been extended with the generic output uncertainty description, and the concept of using component level output uncertainty to assess model top level uncertainty has been applied on a simulation model of a radar liquid cooling system. The focus of this paper is on investigating the applicability of combining the output uncertainty method with probabilistic techniques, not only to provide upper and lower bounds on model uncertaintiesbut also to accompany the uncertainties with estimated probabilities.It is shown that the method may result in a significant improvement in the conditions for conducting an assessment of model uncertainties. The primary use of the method, in combination with either deterministic or probabilistic techniques, is in the early development phases when system level measurement data are scarce. The method may also be used to point out which model components contribute most to the uncertainty on model top level. Such information can be used to concentrate physical testing activities to areas where it is needed most. In this context, the method supports the concept of Virtual Testing.
|
|
| 3. |
- Carlsson, Magnus, et al.
(författare)
-
Methodology for Development and Validation of Multipurpose Simulation Models
- 2012
-
Ingår i: 50th AIAA Aerospace Sciences Meeting Online Proceedings including the New Horizons Forum and Aerospace Exposition (2012). - Reston, Virigina : AIAA.
-
Konferensbidrag (refereegranskat)abstract
- This paper describes a framework for development and validation of multipurpose simulation models. The presented methodology enables reuse of models in different applications with different purposes. The scope is simulation models representing physical environment, physical aircraft systems or subsystems, avionics equipment, and electronic hardware. The methodology has been developed by a small interdisciplinary team, with experience from Modeling and Simulation (M&S) of vehicle systems as well as development of simulators for verification and training. Special care has been taken to ensure usability of the workflow and method descriptions, mainly by means of 1) a user friendly format, easy to overview and update, 2) keeping the amount of text down, and 3) providing relevant examples, templates, and checklists. A simulation model of the Environmental Control System (ECS) of a military fighter aircraft, the Saab Gripen, is used as an example to guide the reader through the workflow of developing and validating multipurpose simulation models. The methods described in the paper can be used in both military and civil applications, and are not limited to the aircraft industry.
|
|
| 4. |
- Carlsson, Magnus
(författare)
-
Methods for Early Model Validation : Applied on Simulation Models of Aircraft Vehicle Systems
- 2013
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Simulation models of physical systems, with or without control software, are widely used in the aeronautic industry in applications ranging from system development to verification and end-user training. With the main drivers of reducing the cost of physical testing and in general enhancing the ability to take early model-based design decisions, there is an ongoing trend of further increasing the portion of modeling and simulation.The work presented in this thesis is focused on development of methodology for model validation, which is a key enabler for successfully reducing the amount of physical testing without compromising safety. Reducing the amount of physical testing is especially interesting in the aeronautic industry, where each physical test commonly represents a significant cost. Besides the cost aspect, it may also be difficult or hazardous to carry out physical testing. Specific to the aeronautic industry are also the relatively long development cycles, implying long periods of uncertainty during product development. In both industry and academia a common viewpoint is that verification, validation, and uncertainty quantification of simulation models are critical activities for a successful deployment of model-based systems engineering. However, quantification of simulation results uncertainty commonly requires a large amount of certain information, and for industrial applications available methods often seem too detailed or tedious to even try. This in total constitutes more than sufficient reason to invest in research on methodology for model validation, with special focus on simplified methods for use in early development phases when system measurement data are scarce.Results from the work include a method supporting early model validation. When sufficient system level measurement data for validation purposes is unavailable, this method provides a means to use knowledge of component level uncertainty for assessment of model top level uncertainty. Also, the common situation of lacking data for characterization of parameter uncertainties is to some degree mitigated. A novel concept has been developed for integrating uncertainty information obtained from component level validation directly into components, enabling assessment of model level uncertainty. In this way, the level of abstraction is raised from uncertainty of component input parameters to uncertainty of component output characteristics. The method is integrated in a Modelica component library for modeling and simulation of aircraft vehicle systems, and is evaluated in both deterministic and probabilistic frameworks using an industrial application example. Results also include an industrial applicable process for model development, validation, and export, and the concept of virtual testing and virtual certification is discussed.
|
|
| 5. |
- Carlsson, Magnus, et al.
(författare)
-
Utilizing Uncertainty Information in Early Model Validation
- 2012
-
Ingår i: AIAA Modeling and Simulation Technologies Conference. - Reston, Virigina : American Institute of Aeronautics and Astronautics. - 9781624101823
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
- This paper proposes a pragmatic approach enabling early model validation activities with a limited availability of system level measurement data. The method utilizes information obtained from the common practice of component validation to assess uncertainties on model top level. Focusing on industrial applicability, the method makes use of information normally available to engineers developing simulation models of existing or not yet existing systems. This is in contrast to the traditional sensitivity analysis requiring the user to quantify component parameter uncertainties – a task which, according to the authors’ experience, may be far from intuitive. As the proposed method enables uncertainties to be defined for a component’s outputs (characteristics) rather than its inputs (parameters), it is hereby termed output uncertainty. The method is primarily intended for use in large-scale mathematical 1-D dynamic simulation models of physical systems with or without control software, typically described by Ordinary Differential Equations (ODE) or Differential Algebraic Equations (DAE).It is shown that the method may result in a significant reduction in the number of uncertain parameters that require consideration in a simulation model. The uncertainty quantification of these parameters also becomes more intuitive. Since this implies a substantial improvement in the conditions of conducting sensitivity analysis or optimization on large-scale simulation models, the method facilitates early model validation. In contrast to sensitivity analysis with respect to a model’s original component parameters, which only covers one aspect of model uncertainty, the output uncertainty method enables assessment also of other kinds of uncertainties, such as uncertainties in underlying equations or uncertainties due to model simplifications. To increase the relevance of the method, a simulation model of a radar liquid cooling system is used as an industrial application example.
|
|
| 6. |
- Eek, Magnus, 1980-, et al.
(författare)
-
A Concept for Credibility Assessment of Aircraft System Simulators
- 2016
-
Ingår i: Journal of Aerospace Information Systems. - : American Institute of Aeronautics and Astronautics. - 1940-3151. ; 13:6, s. 219-233
-
Tidskriftsartikel (refereegranskat)abstract
- An efficient methodology for verification, validation, and credibility assessment of simulation models and simulator applications is an enabler for the aeronautical industrys increasing reliance on modeling and simulation in system design and verification and on training. As a complement to traditional document-centric approaches, this paper presents a method for credibility assessment of simulator applications, in which credibility information is presented to end users directly during simulation. The central idea is that each model in a simulator is extended with a metamodel describing different aspects of credibility. The metamodel includes a number of static credibility measures and a dynamic measure that may vary during simulation. The concept is implemented and tested in two system simulators for the Saab Gripen fighter aircraft. According to the evaluation, the concept facilitates an intuitive overview of model dependencies, as well as credibility information for individual models and for a simulator as a whole. This implies a support for detecting test plan deficiencies or that a simulator configuration is not a suitable platform for the execution of a particular test. Furthermore, model developers and end users are encouraged to reflect upon central credibility aspects like intended use, model fidelity, and test worthiness in their daily work.
|
|
| 7. |
- Eek, Magnus, 1980-, et al.
(författare)
-
Definition and Implementation of a Method for Uncertainty Aggregation in Component-Based System Simulation Models
- 2017
-
Ingår i: Journal of Verification, Validation and Uncertainty Quantification. - New York : ASME Press. - 2377-2158 .- 2377-2166. ; 2:1
-
Tidskriftsartikel (refereegranskat)abstract
- Component-based system simulation models are used throughout all development phases for design and verification of both physical systems and control software, not least in the aeronautical industry. However, the application of structured methods for uncertainty quantification (UQ) of system simulation models is rarely seen. To enable dimensionality reduction of a UQ problem and to thereby make UQ more feasible for industry-grade system simulation models, this paper describes a pragmatic method for uncertainty aggregation. The central idea of the proposed aggregation method is to integrate information obtained during common practice component-level validation directly into the components, and to utilize this information in model-level UQ. A generic component output uncertainty description has been defined and implemented in a Modelica library for modeling and simulation (M&S) of aircraft vehicle systems. An example is provided on how to characterize and quantify a component’s aggregated output uncertainty based on the component-level bench test measurement data. Furthermore, the industrial applicability of the uncertainty aggregation method is demonstrated in an approximate UQ of an aircraft liquid cooling system simulation model. For cases when the concept of thorough UQ resulting in probability boxes is not feasible, the demonstrated approximate UQ using aggregated uncertainties is considered to be a pragmatic alternative fairly in reach for the common M&S practitioner within the area of system simulation
|
|
| 8. |
- Eek, Magnus
(författare)
-
On Credibility Assessment in Aircraft System Simulation
- 2016
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- The aeronautical industry is becoming increasingly reliant on Modeling and Simulation (M&S) for use throughout all system development phases, for system verification, and end-user training. To justify and to utilize the full potential of today’s model-based approach, the development of efficient and industrially applicable methods for credibility assessment of M&S efforts is a key challenge.This work addresses methods facilitating credibility assessment of simulation models and simulator applications used in aircraft system development. For models of individual aircraft subsystems, an uncertainty aggregation method is proposed that facilitates early model validation through approximate uncertainty quantification. The central idea is to integrate information obtained during component level validation directly into the component equations, and to utilize this information in model level uncertainty quantification.In addition to methods intended for models of individual subsystems, this work also proposes a method and an associated tool for credibility assessment of large-scale simulator applications. As a complement to traditional document-centric approaches, static and dynamic credibility information is here presented to end-users directly during simulation. This implies a support for detecting test plan deficiencies, or that a simulator configuration is not a suitable platform for the execution of a particular test. The credibility assessment tool has been implemented and evaluated in two large-scale system simulators for the Saab Gripen fighter aircraft. The work presented herein also includes an industrially applicable workflow for development, validation, and export of simulation models.
|
|
| 9. |
- Eek, Magnus
(författare)
-
On Credibility Assessment in Aircraft System Simulation
- 2016
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- The aeronautical industry is becoming increasingly reliant on Modeling and Simulation (M&S) for use throughout all system development phases, for system verification, and end-user training. To justify and to utilize the full potential of today’s model-based approach, the development of efficient and industrially applicable methods for credibility assessment of M&S efforts is a key challenge.This work addresses methods facilitating credibility assessment of simulation models and simulator applications used in aircraft system development. For models of individual aircraft subsystems, an uncertainty aggregation method is proposed that facilitates early model validation through approximate uncertainty quantification. The central idea is to integrate information obtained during component level validation directly into the component equations, and to utilize this information in model level uncertainty quantification.In addition to methods intended for models of individual subsystems, this work also proposes a method and an associated tool for credibility assessment of large-scale simulator applications. As a complement to traditional document-centric approaches, static and dynamic credibility information is here presented to end-users directly during simulation. This implies a support for detecting test plan deficiencies, or that a simulator configuration is not a suitable platform for the execution of a particular test. The credibility assessment tool has been implemented and evaluated in two large-scale system simulators for the Saab Gripen fighter aircraft. The work presented herein also includes an industrially applicable workflow for development, validation, and export of simulation models.
|
|
| 10. |
- Eek, Magnus, et al.
(författare)
-
Study of industrially applied methods for verification, validation and uncertainty quantification of simulator models
- 2015
-
Ingår i: International Journal of Modeling, Simulation, and Scientific Computing. - : World Scientific. - 1793-9623 .- 1793-9615. ; 6:2
-
Tidskriftsartikel (refereegranskat)abstract
- To better utilize the potential of system simulation models and simulators, industrially applicable methods for Verification, Validation and Uncertainty Quantification (VV&UQ) are crucial. This paper presents an exploratory case study of VV&UQ techniques applied on models integrated in aircraft system simulators at Saab Aeronautics and in driving simulators at the Swedish National Road and Transport Research Institute (VTI). Results show that a large number of Verification and Validation (V&V) techniques are applied, some of which are promising for further development and use in simulator credibility assessment. Regarding the application of UQ, a large gap between academia and this part of industry has been identified, and simplified methods are needed. The applicability of the NASA Credibility Assessment Scale (CAS) at the studied organizations is also evaluated and it can be concluded that the CAS is considered to be a usable tool for achieving a uniform level of V&V for all models included in a simulator, although its implementation at the studied organizations requires tailoring and coordination.
|
|
