Terascale direct numerical simulations of turbulent combustion using S3D

J. H. Chen, A. Choudhary, B. De Supinski, M. Devries, E. R. Hawkes, S. Klasky, W. K. Liao, Kwan-Liu Ma, J. Mellor-Crummey, N. Podhorszki, R. Sankaran, S. Shende, C. S. Yoo

Research output: Contribution to journalArticle

353 Citations (Scopus)

Abstract

Computational science is paramount to the understanding of underlying processes in internal combustion engines of the future that will utilize non-petroleum-based alternative fuels, including carbon-neutral biofuels, and burn in new combustion regimes that will attain high efficiency while minimizing emissions of particulates and nitrogen oxides. Next-generation engines will likely operate at higher pressures, with greater amounts of dilution and utilize alternative fuels that exhibit a wide range of chemical and physical properties. Therefore, there is a significant role for high-fidelity simulations, direct numerical simulations (DNS), specifically designed to capture key turbulence-chemistry interactions in these relatively uncharted combustion regimes, and in particular, that can discriminate the effects of differences in fuel properties. In DNS, all of the relevant turbulence and flame scales are resolved numerically using high-order accurate numerical algorithms. As a consequence terascale DNS are computationally intensive, require massive amounts of computing power and generate tens of terabytes of data. Recent results from terascale DNS of turbulent flames are presented here, illustrating its role in elucidating flame stabilization mechanisms in a lifted turbulent hydrogen/air jet flame in a hot air coflow, and the flame structure of a fuel-lean turbulent premixed jet flame. Computing at this scale requires close collaborations between computer and combustion scientists to provide optimized scaleable algorithms and software for terascale simulations, efficient collective parallel I/O, tools for volume visualization of multiscale, multivariate data and automating the combustion workflow. The enabling computer science, applied to combustion science, is also required in many other terascale physics and engineering simulations. In particular, performance monitoring is used to identify the performance of key kernels in the DNS code, S3D and especially memory intensive loops in the code. Through the careful application of loop transformations, data reuse in cache is exploited thereby reducing memory bandwidth needs, and hence, improving S3D's nodal performance. To enhance collective parallel I/O in S3D, an MPI-I/O caching design is used to construct a two-stage write-behind method for improving the performance of write-only operations. The simulations generate tens of terabytes of data requiring analysis. Interactive exploration of the simulation data is enabled by multivariate time-varying volume visualization. The visualization highlights spatial and temporal correlations between multiple reactive scalar fields using an intuitive user interface based on parallel coordinates and time histogram. Finally, an automated combustion workflow is designed using Kepler to manage large-scale data movement, data morphing, and archival and to provide a graphical display of run-time diagnostics.

Original languageEnglish (US)
Article number015001
JournalComputational Science and Discovery
Volume2
Issue number1
DOIs
StatePublished - Jul 23 2009

Fingerprint

Turbulent Combustion
turbulent combustion
Direct numerical simulation
Flame
direct numerical simulation
Combustion
Volume Visualization
Parallel I/O
flames
Visualization
Alternative fuels
Simulation
Work Flow
Turbulence
simulation
turbulence
Loop Transformations
Data Reuse
Burn-in
burn-in

ASJC Scopus subject areas

  • Numerical Analysis
  • Physics and Astronomy(all)
  • Computational Mathematics

Cite this

Chen, J. H., Choudhary, A., De Supinski, B., Devries, M., Hawkes, E. R., Klasky, S., ... Yoo, C. S. (2009). Terascale direct numerical simulations of turbulent combustion using S3D. Computational Science and Discovery, 2(1), [015001]. https://doi.org/10.1088/1749-4699/2/1/015001

Terascale direct numerical simulations of turbulent combustion using S3D. / Chen, J. H.; Choudhary, A.; De Supinski, B.; Devries, M.; Hawkes, E. R.; Klasky, S.; Liao, W. K.; Ma, Kwan-Liu; Mellor-Crummey, J.; Podhorszki, N.; Sankaran, R.; Shende, S.; Yoo, C. S.

In: Computational Science and Discovery, Vol. 2, No. 1, 015001, 23.07.2009.

Research output: Contribution to journalArticle

Chen, JH, Choudhary, A, De Supinski, B, Devries, M, Hawkes, ER, Klasky, S, Liao, WK, Ma, K-L, Mellor-Crummey, J, Podhorszki, N, Sankaran, R, Shende, S & Yoo, CS 2009, 'Terascale direct numerical simulations of turbulent combustion using S3D', Computational Science and Discovery, vol. 2, no. 1, 015001. https://doi.org/10.1088/1749-4699/2/1/015001
Chen JH, Choudhary A, De Supinski B, Devries M, Hawkes ER, Klasky S et al. Terascale direct numerical simulations of turbulent combustion using S3D. Computational Science and Discovery. 2009 Jul 23;2(1). 015001. https://doi.org/10.1088/1749-4699/2/1/015001
Chen, J. H. ; Choudhary, A. ; De Supinski, B. ; Devries, M. ; Hawkes, E. R. ; Klasky, S. ; Liao, W. K. ; Ma, Kwan-Liu ; Mellor-Crummey, J. ; Podhorszki, N. ; Sankaran, R. ; Shende, S. ; Yoo, C. S. / Terascale direct numerical simulations of turbulent combustion using S3D. In: Computational Science and Discovery. 2009 ; Vol. 2, No. 1.
@article{4e58191ad3624b16a2484d3c83fee8c4,
title = "Terascale direct numerical simulations of turbulent combustion using S3D",
abstract = "Computational science is paramount to the understanding of underlying processes in internal combustion engines of the future that will utilize non-petroleum-based alternative fuels, including carbon-neutral biofuels, and burn in new combustion regimes that will attain high efficiency while minimizing emissions of particulates and nitrogen oxides. Next-generation engines will likely operate at higher pressures, with greater amounts of dilution and utilize alternative fuels that exhibit a wide range of chemical and physical properties. Therefore, there is a significant role for high-fidelity simulations, direct numerical simulations (DNS), specifically designed to capture key turbulence-chemistry interactions in these relatively uncharted combustion regimes, and in particular, that can discriminate the effects of differences in fuel properties. In DNS, all of the relevant turbulence and flame scales are resolved numerically using high-order accurate numerical algorithms. As a consequence terascale DNS are computationally intensive, require massive amounts of computing power and generate tens of terabytes of data. Recent results from terascale DNS of turbulent flames are presented here, illustrating its role in elucidating flame stabilization mechanisms in a lifted turbulent hydrogen/air jet flame in a hot air coflow, and the flame structure of a fuel-lean turbulent premixed jet flame. Computing at this scale requires close collaborations between computer and combustion scientists to provide optimized scaleable algorithms and software for terascale simulations, efficient collective parallel I/O, tools for volume visualization of multiscale, multivariate data and automating the combustion workflow. The enabling computer science, applied to combustion science, is also required in many other terascale physics and engineering simulations. In particular, performance monitoring is used to identify the performance of key kernels in the DNS code, S3D and especially memory intensive loops in the code. Through the careful application of loop transformations, data reuse in cache is exploited thereby reducing memory bandwidth needs, and hence, improving S3D's nodal performance. To enhance collective parallel I/O in S3D, an MPI-I/O caching design is used to construct a two-stage write-behind method for improving the performance of write-only operations. The simulations generate tens of terabytes of data requiring analysis. Interactive exploration of the simulation data is enabled by multivariate time-varying volume visualization. The visualization highlights spatial and temporal correlations between multiple reactive scalar fields using an intuitive user interface based on parallel coordinates and time histogram. Finally, an automated combustion workflow is designed using Kepler to manage large-scale data movement, data morphing, and archival and to provide a graphical display of run-time diagnostics.",
author = "Chen, {J. H.} and A. Choudhary and {De Supinski}, B. and M. Devries and Hawkes, {E. R.} and S. Klasky and Liao, {W. K.} and Kwan-Liu Ma and J. Mellor-Crummey and N. Podhorszki and R. Sankaran and S. Shende and Yoo, {C. S.}",
year = "2009",
month = "7",
day = "23",
doi = "10.1088/1749-4699/2/1/015001",
language = "English (US)",
volume = "2",
journal = "Computational Science and Discovery",
issn = "1749-4680",
publisher = "IOP Publishing Ltd.",
number = "1",

}

TY - JOUR

T1 - Terascale direct numerical simulations of turbulent combustion using S3D

AU - Chen, J. H.

AU - Choudhary, A.

AU - De Supinski, B.

AU - Devries, M.

AU - Hawkes, E. R.

AU - Klasky, S.

AU - Liao, W. K.

AU - Ma, Kwan-Liu

AU - Mellor-Crummey, J.

AU - Podhorszki, N.

AU - Sankaran, R.

AU - Shende, S.

AU - Yoo, C. S.

PY - 2009/7/23

Y1 - 2009/7/23

N2 - Computational science is paramount to the understanding of underlying processes in internal combustion engines of the future that will utilize non-petroleum-based alternative fuels, including carbon-neutral biofuels, and burn in new combustion regimes that will attain high efficiency while minimizing emissions of particulates and nitrogen oxides. Next-generation engines will likely operate at higher pressures, with greater amounts of dilution and utilize alternative fuels that exhibit a wide range of chemical and physical properties. Therefore, there is a significant role for high-fidelity simulations, direct numerical simulations (DNS), specifically designed to capture key turbulence-chemistry interactions in these relatively uncharted combustion regimes, and in particular, that can discriminate the effects of differences in fuel properties. In DNS, all of the relevant turbulence and flame scales are resolved numerically using high-order accurate numerical algorithms. As a consequence terascale DNS are computationally intensive, require massive amounts of computing power and generate tens of terabytes of data. Recent results from terascale DNS of turbulent flames are presented here, illustrating its role in elucidating flame stabilization mechanisms in a lifted turbulent hydrogen/air jet flame in a hot air coflow, and the flame structure of a fuel-lean turbulent premixed jet flame. Computing at this scale requires close collaborations between computer and combustion scientists to provide optimized scaleable algorithms and software for terascale simulations, efficient collective parallel I/O, tools for volume visualization of multiscale, multivariate data and automating the combustion workflow. The enabling computer science, applied to combustion science, is also required in many other terascale physics and engineering simulations. In particular, performance monitoring is used to identify the performance of key kernels in the DNS code, S3D and especially memory intensive loops in the code. Through the careful application of loop transformations, data reuse in cache is exploited thereby reducing memory bandwidth needs, and hence, improving S3D's nodal performance. To enhance collective parallel I/O in S3D, an MPI-I/O caching design is used to construct a two-stage write-behind method for improving the performance of write-only operations. The simulations generate tens of terabytes of data requiring analysis. Interactive exploration of the simulation data is enabled by multivariate time-varying volume visualization. The visualization highlights spatial and temporal correlations between multiple reactive scalar fields using an intuitive user interface based on parallel coordinates and time histogram. Finally, an automated combustion workflow is designed using Kepler to manage large-scale data movement, data morphing, and archival and to provide a graphical display of run-time diagnostics.

AB - Computational science is paramount to the understanding of underlying processes in internal combustion engines of the future that will utilize non-petroleum-based alternative fuels, including carbon-neutral biofuels, and burn in new combustion regimes that will attain high efficiency while minimizing emissions of particulates and nitrogen oxides. Next-generation engines will likely operate at higher pressures, with greater amounts of dilution and utilize alternative fuels that exhibit a wide range of chemical and physical properties. Therefore, there is a significant role for high-fidelity simulations, direct numerical simulations (DNS), specifically designed to capture key turbulence-chemistry interactions in these relatively uncharted combustion regimes, and in particular, that can discriminate the effects of differences in fuel properties. In DNS, all of the relevant turbulence and flame scales are resolved numerically using high-order accurate numerical algorithms. As a consequence terascale DNS are computationally intensive, require massive amounts of computing power and generate tens of terabytes of data. Recent results from terascale DNS of turbulent flames are presented here, illustrating its role in elucidating flame stabilization mechanisms in a lifted turbulent hydrogen/air jet flame in a hot air coflow, and the flame structure of a fuel-lean turbulent premixed jet flame. Computing at this scale requires close collaborations between computer and combustion scientists to provide optimized scaleable algorithms and software for terascale simulations, efficient collective parallel I/O, tools for volume visualization of multiscale, multivariate data and automating the combustion workflow. The enabling computer science, applied to combustion science, is also required in many other terascale physics and engineering simulations. In particular, performance monitoring is used to identify the performance of key kernels in the DNS code, S3D and especially memory intensive loops in the code. Through the careful application of loop transformations, data reuse in cache is exploited thereby reducing memory bandwidth needs, and hence, improving S3D's nodal performance. To enhance collective parallel I/O in S3D, an MPI-I/O caching design is used to construct a two-stage write-behind method for improving the performance of write-only operations. The simulations generate tens of terabytes of data requiring analysis. Interactive exploration of the simulation data is enabled by multivariate time-varying volume visualization. The visualization highlights spatial and temporal correlations between multiple reactive scalar fields using an intuitive user interface based on parallel coordinates and time histogram. Finally, an automated combustion workflow is designed using Kepler to manage large-scale data movement, data morphing, and archival and to provide a graphical display of run-time diagnostics.

UR - http://www.scopus.com/inward/record.url?scp=67650695660&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=67650695660&partnerID=8YFLogxK

U2 - 10.1088/1749-4699/2/1/015001

DO - 10.1088/1749-4699/2/1/015001

M3 - Article

AN - SCOPUS:67650695660

VL - 2

JO - Computational Science and Discovery

JF - Computational Science and Discovery

SN - 1749-4680

IS - 1

M1 - 015001

ER -