In-situ processing and visualization for ultrascale simulations

Kwan-Liu Ma, Chaoli Wang, Hongfeng Yu, Anna Tikhonova

Research output: Contribution to journalArticle

58 Citations (Scopus)

Abstract

The growing power of parallel supercomputers gives scientists the ability to simulate more complex problems at higher fidelity, leading to many high-impact scientific advances. To maximize the utilization of the vast amount of data generated by these simulations, scientists also need scalable solutions for studying their data to different extents and at different abstraction levels. As we move into peta- and exa-scale computing, simply dumping as much raw simulation data as the storage capacity allows for post-processing analysis and visualization is no longer a viable approach. A common practice is to use a separate parallel computer to prepare data for subsequent analysis and visualization. A naive realization of this strategy not only limits the amount of data that can be saved, but also turns I/O into a performance bottleneck when using a large parallel system. We conjecture that the most plausible solution for the peta- and exa-scale data problem is to reduce or transform the data in-situ as it is being generated, so the amount of data that must be transferred over the network is kept to a minimum. In this paper, we discuss different approaches to in-situ processing and visualization as well as the results of our preliminary study using large-scale simulation codes on massively parallel supercomputers.

Original languageEnglish (US)
Article number012043
JournalJournal of Physics: Conference Series
Volume78
Issue number1
DOIs
StatePublished - Jul 1 2007
Externally publishedYes

Fingerprint

supercomputers
dumping
parallel computers
simulation
data simulation

ASJC Scopus subject areas

  • Physics and Astronomy(all)

Cite this

In-situ processing and visualization for ultrascale simulations. / Ma, Kwan-Liu; Wang, Chaoli; Yu, Hongfeng; Tikhonova, Anna.

In: Journal of Physics: Conference Series, Vol. 78, No. 1, 012043, 01.07.2007.

Research output: Contribution to journalArticle

Ma, Kwan-Liu ; Wang, Chaoli ; Yu, Hongfeng ; Tikhonova, Anna. / In-situ processing and visualization for ultrascale simulations. In: Journal of Physics: Conference Series. 2007 ; Vol. 78, No. 1.
@article{7f7f4333285b417391bd6a2de3e6dbd9,
title = "In-situ processing and visualization for ultrascale simulations",
abstract = "The growing power of parallel supercomputers gives scientists the ability to simulate more complex problems at higher fidelity, leading to many high-impact scientific advances. To maximize the utilization of the vast amount of data generated by these simulations, scientists also need scalable solutions for studying their data to different extents and at different abstraction levels. As we move into peta- and exa-scale computing, simply dumping as much raw simulation data as the storage capacity allows for post-processing analysis and visualization is no longer a viable approach. A common practice is to use a separate parallel computer to prepare data for subsequent analysis and visualization. A naive realization of this strategy not only limits the amount of data that can be saved, but also turns I/O into a performance bottleneck when using a large parallel system. We conjecture that the most plausible solution for the peta- and exa-scale data problem is to reduce or transform the data in-situ as it is being generated, so the amount of data that must be transferred over the network is kept to a minimum. In this paper, we discuss different approaches to in-situ processing and visualization as well as the results of our preliminary study using large-scale simulation codes on massively parallel supercomputers.",
author = "Kwan-Liu Ma and Chaoli Wang and Hongfeng Yu and Anna Tikhonova",
year = "2007",
month = "7",
day = "1",
doi = "10.1088/1742-6596/78/1/012043",
language = "English (US)",
volume = "78",
journal = "Journal of Physics: Conference Series",
issn = "1742-6588",
publisher = "IOP Publishing Ltd.",
number = "1",

}

TY - JOUR

T1 - In-situ processing and visualization for ultrascale simulations

AU - Ma, Kwan-Liu

AU - Wang, Chaoli

AU - Yu, Hongfeng

AU - Tikhonova, Anna

PY - 2007/7/1

Y1 - 2007/7/1

N2 - The growing power of parallel supercomputers gives scientists the ability to simulate more complex problems at higher fidelity, leading to many high-impact scientific advances. To maximize the utilization of the vast amount of data generated by these simulations, scientists also need scalable solutions for studying their data to different extents and at different abstraction levels. As we move into peta- and exa-scale computing, simply dumping as much raw simulation data as the storage capacity allows for post-processing analysis and visualization is no longer a viable approach. A common practice is to use a separate parallel computer to prepare data for subsequent analysis and visualization. A naive realization of this strategy not only limits the amount of data that can be saved, but also turns I/O into a performance bottleneck when using a large parallel system. We conjecture that the most plausible solution for the peta- and exa-scale data problem is to reduce or transform the data in-situ as it is being generated, so the amount of data that must be transferred over the network is kept to a minimum. In this paper, we discuss different approaches to in-situ processing and visualization as well as the results of our preliminary study using large-scale simulation codes on massively parallel supercomputers.

AB - The growing power of parallel supercomputers gives scientists the ability to simulate more complex problems at higher fidelity, leading to many high-impact scientific advances. To maximize the utilization of the vast amount of data generated by these simulations, scientists also need scalable solutions for studying their data to different extents and at different abstraction levels. As we move into peta- and exa-scale computing, simply dumping as much raw simulation data as the storage capacity allows for post-processing analysis and visualization is no longer a viable approach. A common practice is to use a separate parallel computer to prepare data for subsequent analysis and visualization. A naive realization of this strategy not only limits the amount of data that can be saved, but also turns I/O into a performance bottleneck when using a large parallel system. We conjecture that the most plausible solution for the peta- and exa-scale data problem is to reduce or transform the data in-situ as it is being generated, so the amount of data that must be transferred over the network is kept to a minimum. In this paper, we discuss different approaches to in-situ processing and visualization as well as the results of our preliminary study using large-scale simulation codes on massively parallel supercomputers.

UR - http://www.scopus.com/inward/record.url?scp=36048993712&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=36048993712&partnerID=8YFLogxK

U2 - 10.1088/1742-6596/78/1/012043

DO - 10.1088/1742-6596/78/1/012043

M3 - Article

AN - SCOPUS:36048993712

VL - 78

JO - Journal of Physics: Conference Series

JF - Journal of Physics: Conference Series

SN - 1742-6588

IS - 1

M1 - 012043

ER -