Mapping point-of-care performance using locally-smoothed median and maximum absolute difference curves

Gerald J Kost, Nam Tran, Harpreet Singh

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

Background: The goal is to introduce visual performance mapping efficient for establishing acceptance criteria and facilitating decisions regarding the utility of hospital point-of-care devices. This approach uniquely reveals the quality of performance locally, as opposed to globally. Methods: After presenting theoretical foundations, this study illustrates the approach by applying it to six hospital glucose meter systems (GMSs) using clinical multi-center (n=2767) and multi-system (n=613, n=100) observations. Results: LS MAD curves identified breakouts, that is, points where the locally-smoothed median absolute difference (LS MAD) curve exceeds the recommended error tolerance limit of 5 mg/dL (0.28 mmol/L). LS maximum absolute difference (MaxAD) breakthroughs, which occur where the LS MaxAD curve exceeds the 99th percentile of MaxADs from x=30-200 mg/dL (1.67-11.10 mmol/L), showed extreme error locations. A multi-sensor interference- and hematocrirt-correcting GMS displayed a flat LS MAD curve until it reached a breakout of 179 mg/dL (9.94 mmol/L) and generated breakthroughs that could affect bedside decision-making, but less erratically than other systems with inadequate performance for hospital critical care. We discovered Class I (meter high, reference low) and Class II (converse) discrepant values in some systems. Class I errors could lead to inappropriate insulin dosing and hypoglycemic episodes in tight glucose control. Conclusions: LS MAD-MaxAD curves help assess the performance of point-of-care testing. Visual mapping of systematic and random errors locally over the entire analyte measurement range in a single integrated display is an advantage when considering the adverse impact of zones of poor quantitative performance on specific clinical applications, threshold-driven bedside decisions and the care of critically ill patients.

Original languageEnglish (US)
Pages (from-to)1637-1646
Number of pages10
JournalClinical Chemistry and Laboratory Medicine
Volume49
Issue number10
DOIs
StatePublished - Oct 1 2011

Fingerprint

Point-of-Care Systems
Glucose
Random errors
Systematic errors
Critical Care
Hypoglycemic Agents
Critical Illness
Decision Making
Theoretical Models
Decision making
Display devices
Insulin
Equipment and Supplies
Sensors
Testing

Keywords

  • bandwidth
  • breakout
  • breakthrough
  • discrepant value
  • erroneous result
  • glucose meter
  • International Standards Organization (ISO) 15197
  • tight glucose control (TGC)
  • tolerance limit
  • visual logistics

ASJC Scopus subject areas

  • Clinical Biochemistry
  • Biochemistry, medical

Cite this

Mapping point-of-care performance using locally-smoothed median and maximum absolute difference curves. / Kost, Gerald J; Tran, Nam; Singh, Harpreet.

In: Clinical Chemistry and Laboratory Medicine, Vol. 49, No. 10, 01.10.2011, p. 1637-1646.

Research output: Contribution to journalArticle

@article{0e39844c772f40b0bf9b80a535acaa9c,
title = "Mapping point-of-care performance using locally-smoothed median and maximum absolute difference curves",
abstract = "Background: The goal is to introduce visual performance mapping efficient for establishing acceptance criteria and facilitating decisions regarding the utility of hospital point-of-care devices. This approach uniquely reveals the quality of performance locally, as opposed to globally. Methods: After presenting theoretical foundations, this study illustrates the approach by applying it to six hospital glucose meter systems (GMSs) using clinical multi-center (n=2767) and multi-system (n=613, n=100) observations. Results: LS MAD curves identified breakouts, that is, points where the locally-smoothed median absolute difference (LS MAD) curve exceeds the recommended error tolerance limit of 5 mg/dL (0.28 mmol/L). LS maximum absolute difference (MaxAD) breakthroughs, which occur where the LS MaxAD curve exceeds the 99th percentile of MaxADs from x=30-200 mg/dL (1.67-11.10 mmol/L), showed extreme error locations. A multi-sensor interference- and hematocrirt-correcting GMS displayed a flat LS MAD curve until it reached a breakout of 179 mg/dL (9.94 mmol/L) and generated breakthroughs that could affect bedside decision-making, but less erratically than other systems with inadequate performance for hospital critical care. We discovered Class I (meter high, reference low) and Class II (converse) discrepant values in some systems. Class I errors could lead to inappropriate insulin dosing and hypoglycemic episodes in tight glucose control. Conclusions: LS MAD-MaxAD curves help assess the performance of point-of-care testing. Visual mapping of systematic and random errors locally over the entire analyte measurement range in a single integrated display is an advantage when considering the adverse impact of zones of poor quantitative performance on specific clinical applications, threshold-driven bedside decisions and the care of critically ill patients.",
keywords = "bandwidth, breakout, breakthrough, discrepant value, erroneous result, glucose meter, International Standards Organization (ISO) 15197, tight glucose control (TGC), tolerance limit, visual logistics",
author = "Kost, {Gerald J} and Nam Tran and Harpreet Singh",
year = "2011",
month = "10",
day = "1",
doi = "10.1515/CCLM.2011.655",
language = "English (US)",
volume = "49",
pages = "1637--1646",
journal = "Clinical Chemistry and Laboratory Medicine",
issn = "1434-6621",
publisher = "Walter de Gruyter GmbH & Co. KG",
number = "10",

}

TY - JOUR

T1 - Mapping point-of-care performance using locally-smoothed median and maximum absolute difference curves

AU - Kost, Gerald J

AU - Tran, Nam

AU - Singh, Harpreet

PY - 2011/10/1

Y1 - 2011/10/1

N2 - Background: The goal is to introduce visual performance mapping efficient for establishing acceptance criteria and facilitating decisions regarding the utility of hospital point-of-care devices. This approach uniquely reveals the quality of performance locally, as opposed to globally. Methods: After presenting theoretical foundations, this study illustrates the approach by applying it to six hospital glucose meter systems (GMSs) using clinical multi-center (n=2767) and multi-system (n=613, n=100) observations. Results: LS MAD curves identified breakouts, that is, points where the locally-smoothed median absolute difference (LS MAD) curve exceeds the recommended error tolerance limit of 5 mg/dL (0.28 mmol/L). LS maximum absolute difference (MaxAD) breakthroughs, which occur where the LS MaxAD curve exceeds the 99th percentile of MaxADs from x=30-200 mg/dL (1.67-11.10 mmol/L), showed extreme error locations. A multi-sensor interference- and hematocrirt-correcting GMS displayed a flat LS MAD curve until it reached a breakout of 179 mg/dL (9.94 mmol/L) and generated breakthroughs that could affect bedside decision-making, but less erratically than other systems with inadequate performance for hospital critical care. We discovered Class I (meter high, reference low) and Class II (converse) discrepant values in some systems. Class I errors could lead to inappropriate insulin dosing and hypoglycemic episodes in tight glucose control. Conclusions: LS MAD-MaxAD curves help assess the performance of point-of-care testing. Visual mapping of systematic and random errors locally over the entire analyte measurement range in a single integrated display is an advantage when considering the adverse impact of zones of poor quantitative performance on specific clinical applications, threshold-driven bedside decisions and the care of critically ill patients.

AB - Background: The goal is to introduce visual performance mapping efficient for establishing acceptance criteria and facilitating decisions regarding the utility of hospital point-of-care devices. This approach uniquely reveals the quality of performance locally, as opposed to globally. Methods: After presenting theoretical foundations, this study illustrates the approach by applying it to six hospital glucose meter systems (GMSs) using clinical multi-center (n=2767) and multi-system (n=613, n=100) observations. Results: LS MAD curves identified breakouts, that is, points where the locally-smoothed median absolute difference (LS MAD) curve exceeds the recommended error tolerance limit of 5 mg/dL (0.28 mmol/L). LS maximum absolute difference (MaxAD) breakthroughs, which occur where the LS MaxAD curve exceeds the 99th percentile of MaxADs from x=30-200 mg/dL (1.67-11.10 mmol/L), showed extreme error locations. A multi-sensor interference- and hematocrirt-correcting GMS displayed a flat LS MAD curve until it reached a breakout of 179 mg/dL (9.94 mmol/L) and generated breakthroughs that could affect bedside decision-making, but less erratically than other systems with inadequate performance for hospital critical care. We discovered Class I (meter high, reference low) and Class II (converse) discrepant values in some systems. Class I errors could lead to inappropriate insulin dosing and hypoglycemic episodes in tight glucose control. Conclusions: LS MAD-MaxAD curves help assess the performance of point-of-care testing. Visual mapping of systematic and random errors locally over the entire analyte measurement range in a single integrated display is an advantage when considering the adverse impact of zones of poor quantitative performance on specific clinical applications, threshold-driven bedside decisions and the care of critically ill patients.

KW - bandwidth

KW - breakout

KW - breakthrough

KW - discrepant value

KW - erroneous result

KW - glucose meter

KW - International Standards Organization (ISO) 15197

KW - tight glucose control (TGC)

KW - tolerance limit

KW - visual logistics

UR - http://www.scopus.com/inward/record.url?scp=80054947865&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=80054947865&partnerID=8YFLogxK

U2 - 10.1515/CCLM.2011.655

DO - 10.1515/CCLM.2011.655

M3 - Article

VL - 49

SP - 1637

EP - 1646

JO - Clinical Chemistry and Laboratory Medicine

JF - Clinical Chemistry and Laboratory Medicine

SN - 1434-6621

IS - 10

ER -