Decision support issues using a physiology based score

James P Marcin, M. M. Pollack, K. M. Patel, U. E. Ruttimann

Research output: Contribution to journalArticle

25 Citations (Scopus)

Abstract

Objective: As physiology based assessments of mortality risk become more accurate, their potential utility in clinical decision support and resource rationing decisions increases. Before these prediction models can be used, however, their performance must be statistically evaluated and interpreted in a clinical context. We examine the issues of confidence intervals (as estimates of survival ranges) and confidence levels (as estimates of clinical certainty) by applying Pediatric Risk of Mortality III (PRISM III) in two scenarios: (1) survival prediction for individual patients and (2) resource rationing. Design: A non-concurrent cohort study. Setting: 32 pediatric intensive care units (PICUs). Patients: 10,608 consecutive patients (571 deaths). Interventions: None. Measurements and results: For the individual patient application, we investigated the observed survival rates for patients with low survival predictions and the confidence intervals associated with these predictions. For the resource rationing application, we investigated the maximum error rate of a policy which would limit therapy for patients with scores exceeding a very high threshold. For both applications, we also investigated how the confidence intervals change as the confidence levels change. The observed survival in the PRISM III groups > 28, > 35, and > 42 were 6.3, 5.3, and 0%, with 95% upper confidence interval bounds of 10.5, 13.0, and 13.3%, respectively. Changing the confidence level altered the survival range by more than 300% in the highest risk group, indicating the importance of clinical certainty provisions in prognostic estimates. The maximum error rates for resource allocation decisions were low (e. g., 29 per 100,000 at a 95% certainty level), equivalent to many of the risks of daily living. Changes in confidence level had relatively little effect on this result. Conclusions: Predictions for an individual patient's risk of death with a high PRISM score are statistically not precise by virtue of the small number of patients in these groups and the resulting wide confidence intervals. Clinical certainty (confidence level) issues substantially influence outcome ranges for individual patients, directly affecting the utility of scores for individual patient use. However, sample sizes are sufficient for rationing decisions for many groups with higher certainty levels. Before there can be wide spread acceptance of this type of decision support, physicians and families must confront what they believe is adequate certainty.

Original languageEnglish (US)
Pages (from-to)1299-1304
Number of pages6
JournalIntensive Care Medicine
Volume24
Issue number12
DOIs
StatePublished - 1998
Externally publishedYes

Fingerprint

Confidence Intervals
Survival
Mortality
Clinical Decision Support Systems
Pediatrics
Pediatric Intensive Care Units
Resource Allocation
Family Physicians
Sample Size
Cohort Studies
Survival Rate
Therapeutics

Keywords

  • Certainty
  • Intensive care units
  • Pediatric intensive care
  • Prediction
  • PRISM
  • Severity of illness

ASJC Scopus subject areas

  • Critical Care and Intensive Care Medicine

Cite this

Decision support issues using a physiology based score. / Marcin, James P; Pollack, M. M.; Patel, K. M.; Ruttimann, U. E.

In: Intensive Care Medicine, Vol. 24, No. 12, 1998, p. 1299-1304.

Research output: Contribution to journalArticle

Marcin, JP, Pollack, MM, Patel, KM & Ruttimann, UE 1998, 'Decision support issues using a physiology based score', Intensive Care Medicine, vol. 24, no. 12, pp. 1299-1304. https://doi.org/10.1007/s001340050766
Marcin, James P ; Pollack, M. M. ; Patel, K. M. ; Ruttimann, U. E. / Decision support issues using a physiology based score. In: Intensive Care Medicine. 1998 ; Vol. 24, No. 12. pp. 1299-1304.
@article{5b8a7ed006194865b27305f72b6b664f,
title = "Decision support issues using a physiology based score",
abstract = "Objective: As physiology based assessments of mortality risk become more accurate, their potential utility in clinical decision support and resource rationing decisions increases. Before these prediction models can be used, however, their performance must be statistically evaluated and interpreted in a clinical context. We examine the issues of confidence intervals (as estimates of survival ranges) and confidence levels (as estimates of clinical certainty) by applying Pediatric Risk of Mortality III (PRISM III) in two scenarios: (1) survival prediction for individual patients and (2) resource rationing. Design: A non-concurrent cohort study. Setting: 32 pediatric intensive care units (PICUs). Patients: 10,608 consecutive patients (571 deaths). Interventions: None. Measurements and results: For the individual patient application, we investigated the observed survival rates for patients with low survival predictions and the confidence intervals associated with these predictions. For the resource rationing application, we investigated the maximum error rate of a policy which would limit therapy for patients with scores exceeding a very high threshold. For both applications, we also investigated how the confidence intervals change as the confidence levels change. The observed survival in the PRISM III groups > 28, > 35, and > 42 were 6.3, 5.3, and 0{\%}, with 95{\%} upper confidence interval bounds of 10.5, 13.0, and 13.3{\%}, respectively. Changing the confidence level altered the survival range by more than 300{\%} in the highest risk group, indicating the importance of clinical certainty provisions in prognostic estimates. The maximum error rates for resource allocation decisions were low (e. g., 29 per 100,000 at a 95{\%} certainty level), equivalent to many of the risks of daily living. Changes in confidence level had relatively little effect on this result. Conclusions: Predictions for an individual patient's risk of death with a high PRISM score are statistically not precise by virtue of the small number of patients in these groups and the resulting wide confidence intervals. Clinical certainty (confidence level) issues substantially influence outcome ranges for individual patients, directly affecting the utility of scores for individual patient use. However, sample sizes are sufficient for rationing decisions for many groups with higher certainty levels. Before there can be wide spread acceptance of this type of decision support, physicians and families must confront what they believe is adequate certainty.",
keywords = "Certainty, Intensive care units, Pediatric intensive care, Prediction, PRISM, Severity of illness",
author = "Marcin, {James P} and Pollack, {M. M.} and Patel, {K. M.} and Ruttimann, {U. E.}",
year = "1998",
doi = "10.1007/s001340050766",
language = "English (US)",
volume = "24",
pages = "1299--1304",
journal = "Intensive Care Medicine",
issn = "0342-4642",
publisher = "Springer Verlag",
number = "12",

}

TY - JOUR

T1 - Decision support issues using a physiology based score

AU - Marcin, James P

AU - Pollack, M. M.

AU - Patel, K. M.

AU - Ruttimann, U. E.

PY - 1998

Y1 - 1998

N2 - Objective: As physiology based assessments of mortality risk become more accurate, their potential utility in clinical decision support and resource rationing decisions increases. Before these prediction models can be used, however, their performance must be statistically evaluated and interpreted in a clinical context. We examine the issues of confidence intervals (as estimates of survival ranges) and confidence levels (as estimates of clinical certainty) by applying Pediatric Risk of Mortality III (PRISM III) in two scenarios: (1) survival prediction for individual patients and (2) resource rationing. Design: A non-concurrent cohort study. Setting: 32 pediatric intensive care units (PICUs). Patients: 10,608 consecutive patients (571 deaths). Interventions: None. Measurements and results: For the individual patient application, we investigated the observed survival rates for patients with low survival predictions and the confidence intervals associated with these predictions. For the resource rationing application, we investigated the maximum error rate of a policy which would limit therapy for patients with scores exceeding a very high threshold. For both applications, we also investigated how the confidence intervals change as the confidence levels change. The observed survival in the PRISM III groups > 28, > 35, and > 42 were 6.3, 5.3, and 0%, with 95% upper confidence interval bounds of 10.5, 13.0, and 13.3%, respectively. Changing the confidence level altered the survival range by more than 300% in the highest risk group, indicating the importance of clinical certainty provisions in prognostic estimates. The maximum error rates for resource allocation decisions were low (e. g., 29 per 100,000 at a 95% certainty level), equivalent to many of the risks of daily living. Changes in confidence level had relatively little effect on this result. Conclusions: Predictions for an individual patient's risk of death with a high PRISM score are statistically not precise by virtue of the small number of patients in these groups and the resulting wide confidence intervals. Clinical certainty (confidence level) issues substantially influence outcome ranges for individual patients, directly affecting the utility of scores for individual patient use. However, sample sizes are sufficient for rationing decisions for many groups with higher certainty levels. Before there can be wide spread acceptance of this type of decision support, physicians and families must confront what they believe is adequate certainty.

AB - Objective: As physiology based assessments of mortality risk become more accurate, their potential utility in clinical decision support and resource rationing decisions increases. Before these prediction models can be used, however, their performance must be statistically evaluated and interpreted in a clinical context. We examine the issues of confidence intervals (as estimates of survival ranges) and confidence levels (as estimates of clinical certainty) by applying Pediatric Risk of Mortality III (PRISM III) in two scenarios: (1) survival prediction for individual patients and (2) resource rationing. Design: A non-concurrent cohort study. Setting: 32 pediatric intensive care units (PICUs). Patients: 10,608 consecutive patients (571 deaths). Interventions: None. Measurements and results: For the individual patient application, we investigated the observed survival rates for patients with low survival predictions and the confidence intervals associated with these predictions. For the resource rationing application, we investigated the maximum error rate of a policy which would limit therapy for patients with scores exceeding a very high threshold. For both applications, we also investigated how the confidence intervals change as the confidence levels change. The observed survival in the PRISM III groups > 28, > 35, and > 42 were 6.3, 5.3, and 0%, with 95% upper confidence interval bounds of 10.5, 13.0, and 13.3%, respectively. Changing the confidence level altered the survival range by more than 300% in the highest risk group, indicating the importance of clinical certainty provisions in prognostic estimates. The maximum error rates for resource allocation decisions were low (e. g., 29 per 100,000 at a 95% certainty level), equivalent to many of the risks of daily living. Changes in confidence level had relatively little effect on this result. Conclusions: Predictions for an individual patient's risk of death with a high PRISM score are statistically not precise by virtue of the small number of patients in these groups and the resulting wide confidence intervals. Clinical certainty (confidence level) issues substantially influence outcome ranges for individual patients, directly affecting the utility of scores for individual patient use. However, sample sizes are sufficient for rationing decisions for many groups with higher certainty levels. Before there can be wide spread acceptance of this type of decision support, physicians and families must confront what they believe is adequate certainty.

KW - Certainty

KW - Intensive care units

KW - Pediatric intensive care

KW - Prediction

KW - PRISM

KW - Severity of illness

UR - http://www.scopus.com/inward/record.url?scp=0031702711&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0031702711&partnerID=8YFLogxK

U2 - 10.1007/s001340050766

DO - 10.1007/s001340050766

M3 - Article

C2 - 9885884

AN - SCOPUS:0031702711

VL - 24

SP - 1299

EP - 1304

JO - Intensive Care Medicine

JF - Intensive Care Medicine

SN - 0342-4642

IS - 12

ER -