| 1. |
|
|
| 2. |
- Atzori, Marco, et al.
(författare)
-
In-situ visualization of large-scale turbulence simulations in Nek5000 with ParaView Catalyst
- 2021
-
Rapport (övrigt vetenskapligt/konstnärligt)abstract
- In-situ visualization on HPC systems allows us to analyze simulation results that would otherwise be impossible, given the size of the simulation data sets and offline post-processing execution time. We design and develop in-situ visualization with Paraview Catalyst in Nek5000, a massively parallel Fortran and C code for computational fluid dynamics applications. We perform strong scalability tests up to 2,048 cores on KTH's Beskow Cray XC40 supercomputer and assess in-situ visualization's impact on the Nek5000 performance. In our study case, a high-fidelity simulation of turbulent flow, we observe that in-situ operations significantly limit the strong scalability of the code, reducing the relative parallel efficiency to only ~21\% on 2,048 cores (the relative efficiency of Nek5000 without in-situ operations is ~99\%). Through profiling with Arm MAP, we identified a bottleneck in the image composition step (that uses Radix-kr algorithm) where a majority of the time is spent on MPI communication. We also identified an imbalance of in-situ processing time between rank 0 and all other ranks. Better scaling and load-balancing in the parallel image composition would considerably improve the performance and scalability of Nek5000 with in-situ capabilities in large-scale simulation.
|
|
| 3. |
- Atzori, Marco, 1992-, et al.
(författare)
-
In situ visualization of large-scale turbulence simulations in Nek5000 with ParaView Catalyst
- 2022
-
Ingår i: Journal of Supercomputing. - : Springer. - 0920-8542 .- 1573-0484. ; 78:3, s. 3605-3620
-
Tidskriftsartikel (refereegranskat)abstract
- In situ visualization on high-performance computing systems allows us to analyze simulation results that would otherwise be impossible, given the size of the simulation data sets and offline post-processing execution time. We develop an in situ adaptor for Paraview Catalyst and Nek5000, a massively parallel Fortran and C code for computational fluid dynamics. We perform a strong scalability test up to 2048 cores on KTH’s Beskow Cray XC40 supercomputer and assess in situ visualization’s impact on the Nek5000 performance. In our study case, a high-fidelity simulation of turbulent flow, we observe that in situ operations significantly limit the strong scalability of the code, reducing the relative parallel efficiency to only ≈ 21 % on 2048 cores (the relative efficiency of Nek5000 without in situ operations is ≈ 99 %). Through profiling with Arm MAP, we identified a bottleneck in the image composition step (that uses the Radix-kr algorithm) where a majority of the time is spent on MPI communication. We also identified an imbalance of in situ processing time between rank 0 and all other ranks. In our case, better scaling and load-balancing in the parallel image composition would considerably improve the performance of Nek5000 with in situ capabilities. In general, the result of this study highlights the technical challenges posed by the integration of high-performance simulation codes and data-analysis libraries and their practical use in complex cases, even when efficient algorithms already exist for a certain application scenario.
|
|
| 4. |
- Brynjell-Rahkola, Mattias, 1986-, et al.
(författare)
-
On the stability of flat plate boundary layers subject to localized suction
- 2015
-
Rapport (övrigt vetenskapligt/konstnärligt)abstract
- The stability of the Blasius boundary layer subject to localized suction is revisited. Using tools of global stability analysis, the leading direct and adjoint eigenpairs are determined, and novel insight into the sensitivity and receptivity of the flow is obtained. The problem is addressed through high-order spectral element simulations, which enables the inclusion of a suction pipe into the domain. Due to this, a detailed investigation of the connection between the pipe flow and the boundary layer flow is possible. For all cases investigated, the former always turns out to transition for a lower Reynolds number and suction rate than the latter, and the transition scenario is found to be due to a global instability originating inside a separation bubble at the pipe inlet. Identification of such regions, provides information that is valuable in further development of algorithms for laminar flow control.
|
|
| 5. |
- Chauvat, Guillaume, et al.
(författare)
-
Global linear analysis of a jet in cross-flow at low velocity ratios
- 2020
-
Ingår i: Journal of Fluid Mechanics. - : CAMBRIDGE UNIV PRESS. - 0022-1120 .- 1469-7645. ; 889
-
Tidskriftsartikel (refereegranskat)abstract
- The stability of the jet in cross-flow is investigated using a complete set-up including the flow inside the pipe. First, direct simulations were performed to find the critical velocity ratio as a function of the Reynolds number, keeping the boundary-layer displacement thickness fixed. At all Reynolds numbers investigated, there exists a steady regime at low velocity ratios. As the velocity ratio is increased, a bifurcation to a limit cycle composed of hairpin vortices is observed. The critical bulk velocity ratio is found at approximately for the Reynolds number , above which a global mode of the system becomes unstable. An impulse response analysis was performed and characteristics of the generated wave packets were analysed, which confirmed results of our global mode analysis. In order to study the sensitivity of this flow, we performed transient growth computations and also computed the optimal periodic forcing and its response. Even well below this stability limit, at , large transient growth ( in energy amplification) is possible and the resolvent norm of the linearized Navier-Stokes operator peaks above . This is accompanied with an extreme sensitivity of the spectrum to numerical details, making the computation of a few tens of eigenvalues close to the limit of what can be achieved with double precision arithmetic. We demonstrate that including the meshing of the jet pipe in the simulations does not change qualitatively the dynamics of the flow when compared to the simple Dirichlet boundary condition representing the jet velocity profile. This is in agreement with the recent experimental results of Klotz et al. (J. Fluid Mech., vol. 863, 2019, pp. 386-406) and in contrast to previous studies of Cambonie & Aider (Phys. Fluids, vol. 26, 2014, 084101). Our simulations also show that a small amount of noise at subcritical velocity ratios may trigger the shedding of hairpin vortices.
|
|
| 6. |
|
|
| 7. |
|
|
| 8. |
|
|
| 9. |
|
|
| 10. |
- de Val Borro, Miguel, et al.
(författare)
-
Vortex generation in protoplanetary disks with an embedded giant planet
- 2007
-
Ingår i: Astronomy and Astrophysics. - : EDP Sciences. - 0004-6361 .- 1432-0746. ; 471:3, s. 1043-1055
-
Tidskriftsartikel (refereegranskat)abstract
- Context: Vortices in protoplanetary disks can capture solid particles and form planetary cores within shorter timescales than those involved in the standard core-accretion model. Aims: We investigate vortex generation in thin unmagnetized protoplanetary disks with an embedded giant planet with planet to star mass ratio 10-4 and 10-3.Methods: Two-dimensional hydrodynamical simulations of a protoplanetary disk with a planet are performed using two different numerical methods. The results of the non-linear simulations are compared with a time-resolved modal analysis of the azimuthally averaged surface density profiles using linear perturbation theory.Results: Finite-difference methods implemented in polar coordinates generate vortices moving along the gap created by Neptune-mass to Jupiter-mass planets. The modal analysis shows that unstable modes are generated with growth rate of order 0.3 ΩK for azimuthal numbers m=4,5,6, where ΩK is the local Keplerian frequency. Shock-capturing Cartesian-grid codes do not generate very much vorticity around a giant planet in a standard protoplanetary disk. Modal calculations confirm that the obtained radial profiles of density are less susceptible to the growth of linear modes on timescales of several hundreds of orbital periods. Navier-Stokes viscosity of the order ν=10-5 (in units of a2 Ωp) is found to have a stabilizing effect and prevents the formation of vortices. This result holds at high resolution runs and using different types of boundary conditions.Conclusions: Giant protoplanets of Neptune-mass to Jupiter-mass can excite the Rossby wave instability and generate vortices in thin disks. The presence of vortices in protoplanetary disks has implications for planet formation, orbital migration, and angular momentum transport in disks.
|
|
| 11. |
- Gong, Jing, et al.
(författare)
-
Nek5000 with OpenACC
- 2015
-
Ingår i: Solving software challenges for exascale. - Cham : Springer International Publishing. - 9783319159751 - 9783319159768 ; , s. 57-68
-
Konferensbidrag (refereegranskat)abstract
- Nek5000 is a computational fluid dynamics code based on the spectral element method used for the simulation of incompressible flows. We follow up on an earlier study which ported the simplified version of Nek5000 to a GPU-accelerated system by presenting the hybrid CPU/GPU implementation of the full Nek5000 code using OpenACC. The matrix-matrix multiplication, the Nek5000 gather-scatter operator and a preconditioned Conjugate Gradient solver have implemented using OpenACC for multi-GPU systems. We report an speed-up of 1.3 on single node of a Cray XK6 when using OpenACC directives in Nek5000. On 512 nodes of the Titan supercomputer, the speed-up can be approached to 1.4. A performance analysis of the Nek5000 code using Score-P and Vampir performance monitoring tools shows that overlapping of GPU kernels with host-accelerator memory transfers would considerably increase the performance of the OpenACC version of Nek5000 code.
|
|
| 12. |
|
|
| 13. |
|
|
| 14. |
- Mallor, Fermin, et al.
(författare)
-
In-Situ Analysis of Backflow Events and Their Relation to Separation in Wings Through Well-Resolved LES
- 2024
-
Ingår i: ERCOFTAC Series. - : Springer Science and Business Media B.V.. ; , s. 17-22
-
Bokkapitel (övrigt vetenskapligt/konstnärligt)abstract
- Wall-bounded turbulent flows as those occurring in transportation (e.g. aviation) or industrial applications (e.g turbomachinery), are usually subjected to pressure gradients (PGs). The presence of such PGs affects greatly the development and physics of the turbulent boundary layer (TBL), making it an open research area. An important phenomena associated with the presence of strong adverse PGs (APGs) as appearing in wings, is the separation of the boundary layer, which can lead to stall.
|
|
| 15. |
- Mallor, Fermin, et al.
(författare)
-
Tracking backflow events and flow separation in wings
-
Annan publikation (övrigt vetenskapligt/konstnärligt)abstract
- This study presents an in-depth analysis of backflow event dynamics within non-equilibrium turbulent boundary layers under strong adverse pressure gradients. Large Eddy Simulation (LES) of the flow around a NACA 4412 wing profile at a chord-based Reynolds number of 4×105 and two angles of attack (5 and 11 degrees) are carried out with the spectral-element code Nek5000. The research focuses on the identification and time-tracking of backflow events, their growth mechanisms, and their impact on mean flow separation. Directed Acyclic Graphs (DAGs) are constructed to map the interconnectivity of backflow events over the wing surface, providing a framework to describe their evolution.The study reveals that while most backflow events are small, disconnected and localized, a significant portion of events (around 20%) display significant growth due to their merging with other backflows—a phenomenon not extensively reported in canonical flow studies. The major contribution of this research is the discovery of a single dynamical process, encapsulated by a unique DAG, that characterizes both the separation and incipient flow separation. The importance of merging in the development of large-scale separated regions is highlighted, suggesting that the merging process is a critical factor in the flow separation mechanism. The findings offer potential applications in flow control, suggesting that mitigating the merging process of backflow events could be an effective strategy to delay or prevent flow separation. This insight opens new avenues for the development of flow control techniques aimed at improving aerodynamic efficiency and performance. Further investigations could yield significant advancements in active flow control systems, contributing to the optimized design of aerodynamic surfaces and energy-efficient aerodynamic operations.
|
|
| 16. |
- Massaro, Daniele, et al.
(författare)
-
A comprehensive framework to enhance numerical simulations in the spectral-element code Nek5000
- 2024
-
Ingår i: Computer Physics Communications. - : Elsevier BV. - 0010-4655 .- 1879-2944. ; 302
-
Tidskriftsartikel (refereegranskat)abstract
- A framework is presented for the spectral-element code Nek5000, which has been, and still is, widely used in the computational fluid dynamics (CFD) community to perform high-fidelity numerical simulations of transitional and high Reynolds number flows. Despite the widespread usage, there is a deficiency in having a comprehensive set of tools specifically designed for conducting simulations using Nek5000. To address this issue, we have created a unique framework that allows, inter alia, to perform stability analysis and compute statistics of a turbulent flow. The framework encapsulates modules that provide tools, run-time parameters and memory structures, defining interfaces and performing different tasks. First, the framework architecture is described, showing its non-intrusive approach. Then, the modules are presented, explaining the main tools that have been implemented and describing some of the test cases. The code is open-source and available online, with proper documentation, to-run instructions and related examples.
|
|
| 17. |
|
|
| 18. |
- Massaro, Daniele, et al.
(författare)
-
Coherent structures in the turbulent stepped cylinder flow at ReD=5000
- 2023
-
Ingår i: International Journal of Heat and Fluid Flow. - : Elsevier BV. - 0142-727X .- 1879-2278. ; 102, s. 109144-
-
Tidskriftsartikel (refereegranskat)abstract
- The coherent structures arising in the turbulent flow around a three-dimensional stepped (or step) cylinder are studied through direct numerical simulation. This geometry is widespread in many applications and the junction substantially modifies the wake behaviour, generating three main cells. The mechanisms of vortex connections on the junction are difficult to be captured and interpreted. We thus use a high-order spectral -element methodology (SEM), and the adaptive mesh refinement technique (AMR) to adequately resolve each region of the domain, capturing the smallest turbulent scales. In this way, we can analyse the vortical interactions on the junction via the A2-criterion and understand the evolution of the train of hairpins, which appears only when the cylinder shear layer gets unstable. Together with the hairpins, four horseshoe and edge vortices coexist on the flat junction surface. A complete picture of the vortices' evolution in time is provided. To extract the large-scale, and most energetic, structures in the wake we perform a three-dimensional proper orthogonal decomposition (POD) of the flow. The first six POD modes correspond to three travelling modes which identify the large (L), the small (S) and the modulation (N) cells. The ReD trend shows that these cells persist at higher Reynolds numbers with a larger separation between the vortex shedding frequencies fN and fL. At the same time, the downwash POD mode gets less strong with a more intense and localised modulation region which affects a more extended portion of the large cylinder wake.
|
|
| 19. |
- Massaro, Daniele, et al.
(författare)
-
Direct Numerical Simulation of Turbulent Flow Around 3D Stepped Cylinder wth Adaptive Mesh Refinement
- 2022
-
Ingår i: 12th International Symposium on Turbulence and Shear Flow Phenomena, TSFP 2022. - : International Symposium on Turbulence and Shear Flow Phenomena, TSFP.
-
Konferensbidrag (refereegranskat)abstract
- In the present study, we investigate the turbulent three-dimensional flow around a stepped cylinder, namely two cylinders of different diameters joint at one extremity. We perform a direct numerical simulation with the spectral element code Nek5000 that uses a high order spatial discretisation (the polynomial order is p = 7). The adaptive mesh refinement technique is employed in the error-driven meshing procedure, allowing an adequately refined mesh everywhere. We consider the Reynolds number ReD = 1000, based on the large cylinder diameter and the uniform inflow velocity. We compare our results with the previous experimental campaign by Morton & Yarusevych (2014b). The results agree very well and we can identify the three main wake regions: the S, N and L cell with a Strouhal number StS = 0.408, StN = 0.188 and StL = 0.201 respectively. The instantaneous mean flow properties are studied showing that the junction dynamics is more similar to the previous laminar studies at ReD = 150 rather than at higher ReD = 3900. Moreover, proper orthogonal decomposition is used to detect the most energetic coherent structures, that resemble the three wake cells.
|
|
| 20. |
- Massaro, Daniele, et al.
(författare)
-
Global stability of 180-bend pipe flow with mesh adaptivity
- 2023
-
Ingår i: Physical Review Fluids. - : American Physical Society (APS). - 2469-990X. ; 8:11
-
Tidskriftsartikel (refereegranskat)abstract
- The global stability of the flow in a spatially developing 180∘-bend pipe with curvature δ=R/Rc=1/3 is investigated by performing direct numerical simulations to understand the underlying transitional mechanism. A unique application of the adaptive mesh refinement technique is used during the stability analysis for minimizing the interpolation and quadrature errors. Independent meshes are created for the direct and adjoint solutions, as well as for the base flow extracted via selective frequency damping. The spectrum of the linearized Navier-Stokes operator reveals a pair of complex conjugate eigenvalues, with frequency f≈0.233. Therefore, the transition is attributed to a Hopf bifurcation that takes place at Reb,cr=2528. A structural sensitivity analysis is performed by extracting the wavemaker. We identify the primary source of instability located on the outer wall, θ≈15 downstream of the bend inlet. This region corresponds to the separation bubble on the outer wall. We thus conclude that the instability is caused by the strong shear resulting from the backflow, similar to the 90-bend pipe flow. We believe that understanding the stability mechanism and characterizing the base flow in bent pipes is crucial for studying various biological flows, like blood vessels. Hence, this paper aims to close the knowledge gap between a 90-bend and toroidal pipes by investigating the transition nature in a 18-bend pipe flow.
|
|
| 21. |
- Massaro, Daniele, et al.
(författare)
-
Interface Discontinuities in Spectral-Element Simulations with Adaptive Mesh Refinement
- 2023
-
Ingår i: Spectral and High Order Methods for Partial Differential Equations ICOSAHOM 2020+1 - Selected Papers from the ICOSAHOM Conference 2021. - : Springer Nature. ; , s. 375-386
-
Konferensbidrag (refereegranskat)abstract
- We investigate the discontinuities arising at non-conforming (or non-conformal) interfaces in spectral element method (SEM) simulations. The derivate terms are by definition discontinuous and interface instabilities are usually not visible with a conformal mesh and sufficient resolution. Using the adaptive mesh refinement (AMR) technique the initial coarse mesh is progressively refined according to an error indicator or estimator. In our case, the spectral error indicator (SEI) is adopted. This leads to non-conformal interfaces, where hanging nodes are introduced through h-refinement implemented in the SEM code Nek5000. We consider the flow in a three-dimensional periodic straight pipe and use the turbulent kinetic energy budget as an indicator for assessing discontinuities (wiggles). They involve first and second-order derivatives and represent a fixed point in the statistical analysis of this canonical flow. Looking at the results, we observe that our AMR implementation does not affect the interface discontinuities. The jumps in derivatives are uniquely related to an inadequately resolved mesh. Relying on an error-driven approach, the SEI produces a mesh that allows computing the TKE budgets in excellent agreement with the literature and ensures saving in grid points by a factor of 2.
|
|
| 22. |
|
|
| 23. |
- Massaro, Daniele, et al.
(författare)
-
The flow around a stepped cylinder with turbulent wake and stable shear layer
- 2023
-
Ingår i: Journal of Fluid Mechanics. - : Cambridge University Press (CUP). - 0022-1120 .- 1469-7645. ; 977
-
Tidskriftsartikel (refereegranskat)abstract
- The turbulent external flow around a three-dimensional stepped cylinder is studied by means of direct numerical simulations with the adaptive mesh refinement technique. We give a broad perspective of the flow regimes from laminar to turbulent wake at, which is the highest ever considered for this flow case. In particular, we focus on the intermediate Reynolds number that reveals a turbulent wake coupled with a stable cylinder shear layer (subcritical regime). This flow shows a junction dynamics similar to the laminar, where no hairpin vortex appears around the edges, and just two horseshoe vortices are visible. A new stable vortex in the form of a ring, which coils around the rear area, is also identified. In the turbulent wake, the presence of three wake cells is pointed out: the large and small cylinder cells together with the modulation region. However, the modulation dynamics varies between the subcritical and turbulent regimes. A time-averaged, three-dimensional set of statistics is computed, and spatially coherent structures are extracted via proper orthogonal decomposition (POD). The POD identifies the (long-debated) connection between the N-cell and the downwash behind the junction. Furthermore, as the Reynolds number increases, the downwash phenomenon becomes less prominent. Eventually, a reduced-order reconstruction with the most energetically relevant modes is defined to explain the wake vortex interactions. This also serves as a valuable starting point for simulating the stepped cylinder wake behaviour within complex frameworks, e.g. fluid-structure interaction.
|
|
| 24. |
- Noorani, Azad, et al.
(författare)
-
Informal introduction to program structure of spectral interpolation in nek5000
-
Annan publikation (övrigt vetenskapligt/konstnärligt)abstract
- The algorithm of the interpolation implementation in the spectral element codenek5000is documented informally. The original code is written by James Lottes at Argonne National Laboratories. The various steps of the operations are generally described and visualised for a typical deformed mesh. The corresponding routines and their argument lists for each stage of the interpolation are also explained. The memory structure of the implementation is briefly discussed. Finally, the code overview of the routines is presented.
|
|
| 25. |
- Offermans, Nicolas, 1991-, et al.
(författare)
-
Adaptive mesh refinement for steady flows in Nek5000
-
Annan publikation (övrigt vetenskapligt/konstnärligt)abstract
- Adaptive mesh refinement is performed in the framework of the spectral element method augmented by approaches to error estimation and control. The h-refinement technique is used for adapting the mesh, where selected grid elements are split by a quadtree (2D) or octree (3D) structure. Continuity between parent–child elements is enforced by high-order interpolation of the solution across the common faces. Parallel mesh partitioning and grid management respectively, are taken care of by the external libraries ParMETIS and p4est. Two methods are considered for estimating and controling the error of the solution. The first error estimate is local and based on the spectral properties of the solutionon each element. This method gives a local measure of the L2-norm of the solution over the entire computational domain. The second error estimate uses the dual-weighted residuals method — it is based on and takes into account both the local properties of the solution and the global dependence of the error in the solution via an adjoint problem. The objective of this second approach is to optimize the computation of a given functional of physical interest. The simulations are performed by using the code Nek5000 and three steady-state test cases are studied: a two-dimensional lid-driven cavity at Re = 7, 500, a two-dimensional flow past a cylinder at Re = 40, and a three-dimensional lid-driven cavity at Re = 2, 000 with a moving lid tilted by an angle of 30 degrees. The efficiency of both error estimators is compared in terms of refinement patterns and accuracy on the functional of interest. In the case of the adjoint error estimators, the trend on the error of the functional is shown to be correctly represented up to a multiplicative constant.
|
|
