Streamlining data-intensive biology with workflow systems

Taylor Reiter, Phillip T. Brooks†, Luiz Irber†, Shannon E.K. Joslin†, Charles M. Reid†, Camille Scott†, C. Titus Brown, N. Tessa Pierce-Ward

Research output: Contribution to journalReview articlepeer-review

Abstract

As the scale of biological data generation has increased, the bottleneck of research has shifted from data generation to analysis. Researchers commonly need to build computational workflows that include multiple analytic tools and require incremental development as experimental insights demand tool and parameter modifications. These workflows can produce hundreds to thousands of intermediate files and results that must be integrated for biological insight. Data-centric workflow systems that internally manage computational resources, software, and conditional execution of analysis steps are reshaping the landscape of biological data analysis and empowering researchers to conduct reproducible analyses at scale. Adoption of these tools can facilitate and expedite robust data analysis, but knowledge of these techniques is still lacking. Here, we provide a series of strategies for leveraging workflow systems with structured project, data, and resource management to streamline large-scale biological analysis. We present these practices in the context of high-throughput sequencing data analysis, but the principles are broadly applicable to biologists working beyond this field.

Original languageEnglish (US)
JournalGigaScience
Volume10
Issue number1
DOIs
StatePublished - Jan 1 2021

Keywords

  • automation
  • data-intensive biology
  • repeatability
  • workflows

ASJC Scopus subject areas

  • Computer Science Applications
  • Health Informatics

Fingerprint Dive into the research topics of 'Streamlining data-intensive biology with workflow systems'. Together they form a unique fingerprint.

Cite this