Systematic evaluation of variability in ChIP-chip experiments using predefined DNA targets

David S. Johnson, Wei Li, D. Benjamin Gordon, Arindam Bhattacharjee, Bo Curry, Jayati Ghosh, Leonardo Brizuela, Jason S. Carroll, Myles Brown, Paul Flicek, Christoph M. Koch, Ian Dunham, Mark Bieda, Xiaoqin Xu, Peggy J. Farnham, Philipp Kapranov, David A. Nix, Thomas R. Gingeras, Xinmin Zhang, Heather HolsterNan Jiang, Roland D. Green, Jun S. Song, Scott A. McCuine, Elizabeth Anton, Loan Nguyen, Nathan D. Trinklein, Zhen Ye, Keith Ching, David Hawkins, Bing Ren, Peter C. Scacheri, Joel Rozowsky, Alexander Karpikov, Ghia Euskirchen, Sherman Weissman, Mark Gerstein, Michael Snyder, Annie Yang, Zarmik Moqtaderi, Heather Hirsch, Hennady P. Shulha, Yutao Fu, Zhiping Weng, Kevin Struhl, Richard M. Myers, Jason D. Lieb, X. Shirley Liu

Research output: Contribution to journalArticle

107 Citations (Scopus)

Abstract

The most widely used method for detecting genome-wide protein-DNA interactions is chromatin immunoprecipitation on tiling microarrays, commonly known as ChIP-chip. Here, we conducted the first objective analysis of tiling array platforms, amplification procedures, and signal detection algorithms in a simulated ChIP-chip experiment. Mixtures of human genomic DNA and "spike-ins" comprised of nearly 100 human sequences at various concentrations were hybridized to four tiling array platforms by eight independent groups. Blind to the number of spike-ins, their locations, and the range of concentrations, each group made predictions of the spike-in locations. We found that microarray platform choice is not the primary determinant of overall performance. In fact, variation in performance between labs, protocols, and algorithms within the same array platform was greater than the variation in performance between array platforms. However, each array platform had unique performance characteristics that varied with tiling resolution and the number of replicates, which have implications for cost versus detection power. Long oligonucleotide arrays were slightly more sensitive at detecting very low enrichment. On all platforms, simple sequence repeats and genome redundancy tended to result in false positives. LM-PCR and WGA, the most popular sample amplification techniques, reproduced relative enrichment levels with high fidelity. Performance among signal detection algorithms was heavily dependent on array platform. The spike-in DNA samples and the data presented here provide a stable benchmark against which future ChIP platforms, protocol improvements, and analysis methods can be evaluated.

Original languageEnglish (US)
Pages (from-to)393-403
Number of pages11
JournalGenome Research
Volume18
Issue number3
DOIs
StatePublished - Mar 2008

Fingerprint

DNA
Genome
Benchmarking
Chromatin Immunoprecipitation
Oligonucleotide Array Sequence Analysis
Microsatellite Repeats
Costs and Cost Analysis
Polymerase Chain Reaction
Proteins

ASJC Scopus subject areas

  • Genetics

Cite this

Johnson, D. S., Li, W., Gordon, D. B., Bhattacharjee, A., Curry, B., Ghosh, J., ... Liu, X. S. (2008). Systematic evaluation of variability in ChIP-chip experiments using predefined DNA targets. Genome Research, 18(3), 393-403. https://doi.org/10.1101/gr.7080508

Systematic evaluation of variability in ChIP-chip experiments using predefined DNA targets. / Johnson, David S.; Li, Wei; Gordon, D. Benjamin; Bhattacharjee, Arindam; Curry, Bo; Ghosh, Jayati; Brizuela, Leonardo; Carroll, Jason S.; Brown, Myles; Flicek, Paul; Koch, Christoph M.; Dunham, Ian; Bieda, Mark; Xu, Xiaoqin; Farnham, Peggy J.; Kapranov, Philipp; Nix, David A.; Gingeras, Thomas R.; Zhang, Xinmin; Holster, Heather; Jiang, Nan; Green, Roland D.; Song, Jun S.; McCuine, Scott A.; Anton, Elizabeth; Nguyen, Loan; Trinklein, Nathan D.; Ye, Zhen; Ching, Keith; Hawkins, David; Ren, Bing; Scacheri, Peter C.; Rozowsky, Joel; Karpikov, Alexander; Euskirchen, Ghia; Weissman, Sherman; Gerstein, Mark; Snyder, Michael; Yang, Annie; Moqtaderi, Zarmik; Hirsch, Heather; Shulha, Hennady P.; Fu, Yutao; Weng, Zhiping; Struhl, Kevin; Myers, Richard M.; Lieb, Jason D.; Liu, X. Shirley.

In: Genome Research, Vol. 18, No. 3, 03.2008, p. 393-403.

Research output: Contribution to journalArticle

Johnson, DS, Li, W, Gordon, DB, Bhattacharjee, A, Curry, B, Ghosh, J, Brizuela, L, Carroll, JS, Brown, M, Flicek, P, Koch, CM, Dunham, I, Bieda, M, Xu, X, Farnham, PJ, Kapranov, P, Nix, DA, Gingeras, TR, Zhang, X, Holster, H, Jiang, N, Green, RD, Song, JS, McCuine, SA, Anton, E, Nguyen, L, Trinklein, ND, Ye, Z, Ching, K, Hawkins, D, Ren, B, Scacheri, PC, Rozowsky, J, Karpikov, A, Euskirchen, G, Weissman, S, Gerstein, M, Snyder, M, Yang, A, Moqtaderi, Z, Hirsch, H, Shulha, HP, Fu, Y, Weng, Z, Struhl, K, Myers, RM, Lieb, JD & Liu, XS 2008, 'Systematic evaluation of variability in ChIP-chip experiments using predefined DNA targets', Genome Research, vol. 18, no. 3, pp. 393-403. https://doi.org/10.1101/gr.7080508
Johnson DS, Li W, Gordon DB, Bhattacharjee A, Curry B, Ghosh J et al. Systematic evaluation of variability in ChIP-chip experiments using predefined DNA targets. Genome Research. 2008 Mar;18(3):393-403. https://doi.org/10.1101/gr.7080508
Johnson, David S. ; Li, Wei ; Gordon, D. Benjamin ; Bhattacharjee, Arindam ; Curry, Bo ; Ghosh, Jayati ; Brizuela, Leonardo ; Carroll, Jason S. ; Brown, Myles ; Flicek, Paul ; Koch, Christoph M. ; Dunham, Ian ; Bieda, Mark ; Xu, Xiaoqin ; Farnham, Peggy J. ; Kapranov, Philipp ; Nix, David A. ; Gingeras, Thomas R. ; Zhang, Xinmin ; Holster, Heather ; Jiang, Nan ; Green, Roland D. ; Song, Jun S. ; McCuine, Scott A. ; Anton, Elizabeth ; Nguyen, Loan ; Trinklein, Nathan D. ; Ye, Zhen ; Ching, Keith ; Hawkins, David ; Ren, Bing ; Scacheri, Peter C. ; Rozowsky, Joel ; Karpikov, Alexander ; Euskirchen, Ghia ; Weissman, Sherman ; Gerstein, Mark ; Snyder, Michael ; Yang, Annie ; Moqtaderi, Zarmik ; Hirsch, Heather ; Shulha, Hennady P. ; Fu, Yutao ; Weng, Zhiping ; Struhl, Kevin ; Myers, Richard M. ; Lieb, Jason D. ; Liu, X. Shirley. / Systematic evaluation of variability in ChIP-chip experiments using predefined DNA targets. In: Genome Research. 2008 ; Vol. 18, No. 3. pp. 393-403.
@article{03a8c517f3c641328b5fdc646e21340e,
title = "Systematic evaluation of variability in ChIP-chip experiments using predefined DNA targets",
abstract = "The most widely used method for detecting genome-wide protein-DNA interactions is chromatin immunoprecipitation on tiling microarrays, commonly known as ChIP-chip. Here, we conducted the first objective analysis of tiling array platforms, amplification procedures, and signal detection algorithms in a simulated ChIP-chip experiment. Mixtures of human genomic DNA and {"}spike-ins{"} comprised of nearly 100 human sequences at various concentrations were hybridized to four tiling array platforms by eight independent groups. Blind to the number of spike-ins, their locations, and the range of concentrations, each group made predictions of the spike-in locations. We found that microarray platform choice is not the primary determinant of overall performance. In fact, variation in performance between labs, protocols, and algorithms within the same array platform was greater than the variation in performance between array platforms. However, each array platform had unique performance characteristics that varied with tiling resolution and the number of replicates, which have implications for cost versus detection power. Long oligonucleotide arrays were slightly more sensitive at detecting very low enrichment. On all platforms, simple sequence repeats and genome redundancy tended to result in false positives. LM-PCR and WGA, the most popular sample amplification techniques, reproduced relative enrichment levels with high fidelity. Performance among signal detection algorithms was heavily dependent on array platform. The spike-in DNA samples and the data presented here provide a stable benchmark against which future ChIP platforms, protocol improvements, and analysis methods can be evaluated.",
author = "Johnson, {David S.} and Wei Li and Gordon, {D. Benjamin} and Arindam Bhattacharjee and Bo Curry and Jayati Ghosh and Leonardo Brizuela and Carroll, {Jason S.} and Myles Brown and Paul Flicek and Koch, {Christoph M.} and Ian Dunham and Mark Bieda and Xiaoqin Xu and Farnham, {Peggy J.} and Philipp Kapranov and Nix, {David A.} and Gingeras, {Thomas R.} and Xinmin Zhang and Heather Holster and Nan Jiang and Green, {Roland D.} and Song, {Jun S.} and McCuine, {Scott A.} and Elizabeth Anton and Loan Nguyen and Trinklein, {Nathan D.} and Zhen Ye and Keith Ching and David Hawkins and Bing Ren and Scacheri, {Peter C.} and Joel Rozowsky and Alexander Karpikov and Ghia Euskirchen and Sherman Weissman and Mark Gerstein and Michael Snyder and Annie Yang and Zarmik Moqtaderi and Heather Hirsch and Shulha, {Hennady P.} and Yutao Fu and Zhiping Weng and Kevin Struhl and Myers, {Richard M.} and Lieb, {Jason D.} and Liu, {X. Shirley}",
year = "2008",
month = "3",
doi = "10.1101/gr.7080508",
language = "English (US)",
volume = "18",
pages = "393--403",
journal = "Genome Research",
issn = "1088-9051",
publisher = "Cold Spring Harbor Laboratory Press",
number = "3",

}

TY - JOUR

T1 - Systematic evaluation of variability in ChIP-chip experiments using predefined DNA targets

AU - Johnson, David S.

AU - Li, Wei

AU - Gordon, D. Benjamin

AU - Bhattacharjee, Arindam

AU - Curry, Bo

AU - Ghosh, Jayati

AU - Brizuela, Leonardo

AU - Carroll, Jason S.

AU - Brown, Myles

AU - Flicek, Paul

AU - Koch, Christoph M.

AU - Dunham, Ian

AU - Bieda, Mark

AU - Xu, Xiaoqin

AU - Farnham, Peggy J.

AU - Kapranov, Philipp

AU - Nix, David A.

AU - Gingeras, Thomas R.

AU - Zhang, Xinmin

AU - Holster, Heather

AU - Jiang, Nan

AU - Green, Roland D.

AU - Song, Jun S.

AU - McCuine, Scott A.

AU - Anton, Elizabeth

AU - Nguyen, Loan

AU - Trinklein, Nathan D.

AU - Ye, Zhen

AU - Ching, Keith

AU - Hawkins, David

AU - Ren, Bing

AU - Scacheri, Peter C.

AU - Rozowsky, Joel

AU - Karpikov, Alexander

AU - Euskirchen, Ghia

AU - Weissman, Sherman

AU - Gerstein, Mark

AU - Snyder, Michael

AU - Yang, Annie

AU - Moqtaderi, Zarmik

AU - Hirsch, Heather

AU - Shulha, Hennady P.

AU - Fu, Yutao

AU - Weng, Zhiping

AU - Struhl, Kevin

AU - Myers, Richard M.

AU - Lieb, Jason D.

AU - Liu, X. Shirley

PY - 2008/3

Y1 - 2008/3

N2 - The most widely used method for detecting genome-wide protein-DNA interactions is chromatin immunoprecipitation on tiling microarrays, commonly known as ChIP-chip. Here, we conducted the first objective analysis of tiling array platforms, amplification procedures, and signal detection algorithms in a simulated ChIP-chip experiment. Mixtures of human genomic DNA and "spike-ins" comprised of nearly 100 human sequences at various concentrations were hybridized to four tiling array platforms by eight independent groups. Blind to the number of spike-ins, their locations, and the range of concentrations, each group made predictions of the spike-in locations. We found that microarray platform choice is not the primary determinant of overall performance. In fact, variation in performance between labs, protocols, and algorithms within the same array platform was greater than the variation in performance between array platforms. However, each array platform had unique performance characteristics that varied with tiling resolution and the number of replicates, which have implications for cost versus detection power. Long oligonucleotide arrays were slightly more sensitive at detecting very low enrichment. On all platforms, simple sequence repeats and genome redundancy tended to result in false positives. LM-PCR and WGA, the most popular sample amplification techniques, reproduced relative enrichment levels with high fidelity. Performance among signal detection algorithms was heavily dependent on array platform. The spike-in DNA samples and the data presented here provide a stable benchmark against which future ChIP platforms, protocol improvements, and analysis methods can be evaluated.

AB - The most widely used method for detecting genome-wide protein-DNA interactions is chromatin immunoprecipitation on tiling microarrays, commonly known as ChIP-chip. Here, we conducted the first objective analysis of tiling array platforms, amplification procedures, and signal detection algorithms in a simulated ChIP-chip experiment. Mixtures of human genomic DNA and "spike-ins" comprised of nearly 100 human sequences at various concentrations were hybridized to four tiling array platforms by eight independent groups. Blind to the number of spike-ins, their locations, and the range of concentrations, each group made predictions of the spike-in locations. We found that microarray platform choice is not the primary determinant of overall performance. In fact, variation in performance between labs, protocols, and algorithms within the same array platform was greater than the variation in performance between array platforms. However, each array platform had unique performance characteristics that varied with tiling resolution and the number of replicates, which have implications for cost versus detection power. Long oligonucleotide arrays were slightly more sensitive at detecting very low enrichment. On all platforms, simple sequence repeats and genome redundancy tended to result in false positives. LM-PCR and WGA, the most popular sample amplification techniques, reproduced relative enrichment levels with high fidelity. Performance among signal detection algorithms was heavily dependent on array platform. The spike-in DNA samples and the data presented here provide a stable benchmark against which future ChIP platforms, protocol improvements, and analysis methods can be evaluated.

UR - http://www.scopus.com/inward/record.url?scp=40449126726&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=40449126726&partnerID=8YFLogxK

U2 - 10.1101/gr.7080508

DO - 10.1101/gr.7080508

M3 - Article

C2 - 18258921

AN - SCOPUS:40449126726

VL - 18

SP - 393

EP - 403

JO - Genome Research

JF - Genome Research

SN - 1088-9051

IS - 3

ER -