Impact of pre-analytical variables on deep learning accuracy in histopathology

Andrew D. Jones, John Paul Graff, Morgan Darrow, Alexander D Borowsky, Kristin A Olson, Regina F Gandour-Edwards, Ananya Datta Mitra, Dongguang Wei, Guofeng Gao, Blythe Durbin-Johnson, Hooman Rashidi

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

Aims: Machine learning (ML) binary classification in diagnostic histopathology is an area of intense investigation. Several assumptions, including training image quality/format and the number of training images required, appear to be similar in many studies irrespective of the paucity of supporting evidence. We empirically compared training image file type, training set size, and two common convolutional neural networks (CNNs) using transfer learning (ResNet50 and SqueezeNet). Methods and results: Thirty haematoxylin and eosin (H&E)-stained slides with carcinoma or normal tissue from three tissue types (breast, colon, and prostate) were photographed, generating 3000 partially overlapping images (1000 per tissue type). These lossless Portable Networks Graphics (PNGs) images were converted to lossy Joint Photographic Experts Group (JPG) images. Tissue type-specific binary classification ML models were developed by the use of all PNG or JPG images, and repeated with a subset of 500, 200, 100, 50, 30 and 10 images. Eleven models were generated for each tissue type, at each quantity of training images, for each file type, and for each CNN, resulting in 924 models. Internal accuracies and generalisation accuracies were compared. There was no meaningful significant difference in accuracies between PNG and JPG models. Models trained with more images did not invariably perform better. ResNet50 typically outperformed SqueezeNet. Models were generalisable within a tissue type but not across tissue types. Conclusions: Lossy JPG images were not inferior to lossless PNG images in our models. Large numbers of unique H&E-stained slides were not required for training optimal ML models. This reinforces the need for an evidence-based approach to best practices for histopathological ML.

Original languageEnglish (US)
JournalHistopathology
DOIs
StatePublished - Jan 1 2019

Fingerprint

Learning
Hematoxylin
Eosine Yellowish-(YS)
Practice Guidelines
Prostate
Colon
Breast
Carcinoma
Machine Learning

Keywords

  • anatomic pathology
  • artificial intelligence
  • breast
  • colon
  • deep learning
  • histopathology
  • machine learning
  • prostate
  • whole slide imaging
  • WSI

ASJC Scopus subject areas

  • Pathology and Forensic Medicine
  • Histology

Cite this

Impact of pre-analytical variables on deep learning accuracy in histopathology. / Jones, Andrew D.; Graff, John Paul; Darrow, Morgan; Borowsky, Alexander D; Olson, Kristin A; Gandour-Edwards, Regina F; Datta Mitra, Ananya; Wei, Dongguang; Gao, Guofeng; Durbin-Johnson, Blythe; Rashidi, Hooman.

In: Histopathology, 01.01.2019.

Research output: Contribution to journalArticle

@article{13202b97f5b44bb8bd776d01874792b6,
title = "Impact of pre-analytical variables on deep learning accuracy in histopathology",
abstract = "Aims: Machine learning (ML) binary classification in diagnostic histopathology is an area of intense investigation. Several assumptions, including training image quality/format and the number of training images required, appear to be similar in many studies irrespective of the paucity of supporting evidence. We empirically compared training image file type, training set size, and two common convolutional neural networks (CNNs) using transfer learning (ResNet50 and SqueezeNet). Methods and results: Thirty haematoxylin and eosin (H&E)-stained slides with carcinoma or normal tissue from three tissue types (breast, colon, and prostate) were photographed, generating 3000 partially overlapping images (1000 per tissue type). These lossless Portable Networks Graphics (PNGs) images were converted to lossy Joint Photographic Experts Group (JPG) images. Tissue type-specific binary classification ML models were developed by the use of all PNG or JPG images, and repeated with a subset of 500, 200, 100, 50, 30 and 10 images. Eleven models were generated for each tissue type, at each quantity of training images, for each file type, and for each CNN, resulting in 924 models. Internal accuracies and generalisation accuracies were compared. There was no meaningful significant difference in accuracies between PNG and JPG models. Models trained with more images did not invariably perform better. ResNet50 typically outperformed SqueezeNet. Models were generalisable within a tissue type but not across tissue types. Conclusions: Lossy JPG images were not inferior to lossless PNG images in our models. Large numbers of unique H&E-stained slides were not required for training optimal ML models. This reinforces the need for an evidence-based approach to best practices for histopathological ML.",
keywords = "anatomic pathology, artificial intelligence, breast, colon, deep learning, histopathology, machine learning, prostate, whole slide imaging, WSI",
author = "Jones, {Andrew D.} and Graff, {John Paul} and Morgan Darrow and Borowsky, {Alexander D} and Olson, {Kristin A} and Gandour-Edwards, {Regina F} and {Datta Mitra}, Ananya and Dongguang Wei and Guofeng Gao and Blythe Durbin-Johnson and Hooman Rashidi",
year = "2019",
month = "1",
day = "1",
doi = "10.1111/his.13844",
language = "English (US)",
journal = "Histopathology",
issn = "0309-0167",
publisher = "Wiley-Blackwell",

}

TY - JOUR

T1 - Impact of pre-analytical variables on deep learning accuracy in histopathology

AU - Jones, Andrew D.

AU - Graff, John Paul

AU - Darrow, Morgan

AU - Borowsky, Alexander D

AU - Olson, Kristin A

AU - Gandour-Edwards, Regina F

AU - Datta Mitra, Ananya

AU - Wei, Dongguang

AU - Gao, Guofeng

AU - Durbin-Johnson, Blythe

AU - Rashidi, Hooman

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Aims: Machine learning (ML) binary classification in diagnostic histopathology is an area of intense investigation. Several assumptions, including training image quality/format and the number of training images required, appear to be similar in many studies irrespective of the paucity of supporting evidence. We empirically compared training image file type, training set size, and two common convolutional neural networks (CNNs) using transfer learning (ResNet50 and SqueezeNet). Methods and results: Thirty haematoxylin and eosin (H&E)-stained slides with carcinoma or normal tissue from three tissue types (breast, colon, and prostate) were photographed, generating 3000 partially overlapping images (1000 per tissue type). These lossless Portable Networks Graphics (PNGs) images were converted to lossy Joint Photographic Experts Group (JPG) images. Tissue type-specific binary classification ML models were developed by the use of all PNG or JPG images, and repeated with a subset of 500, 200, 100, 50, 30 and 10 images. Eleven models were generated for each tissue type, at each quantity of training images, for each file type, and for each CNN, resulting in 924 models. Internal accuracies and generalisation accuracies were compared. There was no meaningful significant difference in accuracies between PNG and JPG models. Models trained with more images did not invariably perform better. ResNet50 typically outperformed SqueezeNet. Models were generalisable within a tissue type but not across tissue types. Conclusions: Lossy JPG images were not inferior to lossless PNG images in our models. Large numbers of unique H&E-stained slides were not required for training optimal ML models. This reinforces the need for an evidence-based approach to best practices for histopathological ML.

AB - Aims: Machine learning (ML) binary classification in diagnostic histopathology is an area of intense investigation. Several assumptions, including training image quality/format and the number of training images required, appear to be similar in many studies irrespective of the paucity of supporting evidence. We empirically compared training image file type, training set size, and two common convolutional neural networks (CNNs) using transfer learning (ResNet50 and SqueezeNet). Methods and results: Thirty haematoxylin and eosin (H&E)-stained slides with carcinoma or normal tissue from three tissue types (breast, colon, and prostate) were photographed, generating 3000 partially overlapping images (1000 per tissue type). These lossless Portable Networks Graphics (PNGs) images were converted to lossy Joint Photographic Experts Group (JPG) images. Tissue type-specific binary classification ML models were developed by the use of all PNG or JPG images, and repeated with a subset of 500, 200, 100, 50, 30 and 10 images. Eleven models were generated for each tissue type, at each quantity of training images, for each file type, and for each CNN, resulting in 924 models. Internal accuracies and generalisation accuracies were compared. There was no meaningful significant difference in accuracies between PNG and JPG models. Models trained with more images did not invariably perform better. ResNet50 typically outperformed SqueezeNet. Models were generalisable within a tissue type but not across tissue types. Conclusions: Lossy JPG images were not inferior to lossless PNG images in our models. Large numbers of unique H&E-stained slides were not required for training optimal ML models. This reinforces the need for an evidence-based approach to best practices for histopathological ML.

KW - anatomic pathology

KW - artificial intelligence

KW - breast

KW - colon

KW - deep learning

KW - histopathology

KW - machine learning

KW - prostate

KW - whole slide imaging

KW - WSI

UR - http://www.scopus.com/inward/record.url?scp=85066046540&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85066046540&partnerID=8YFLogxK

U2 - 10.1111/his.13844

DO - 10.1111/his.13844

M3 - Article

C2 - 30801768

AN - SCOPUS:85066046540

JO - Histopathology

JF - Histopathology

SN - 0309-0167

ER -