Korean Journal of Radiology

The Korean Society of Radiology (대한영상의학회)
권호 표지
DOI QR코드

DOI QR Code

Statistical Methods for Comparing Predictive Values in Medical Diagnosis

  • Chanrim Park (Biomedical Research Institute, Seoul National University Hospital) ;
  • Seo Young Park (Department of Statistics and Data Science, Korea National Open University) ;
  • Hwa Jung Kim (Department of Preventive Medicine, University of Ulsan College of Medicine) ;
  • Hee Jung Shin (Department of Radiology and Research Institute of Radiology, Asan Medical Center)
  • Received : 2024.01.16
  • Accepted : 2024.05.04
  • Published : 2024.07.01
Download PDF
⟨ Previous Next ⟩

Abstract

Evaluating the performance of a binary diagnostic test, including artificial intelligence classification algorithms, involves measuring sensitivity, specificity, positive predictive value, and negative predictive value. Particularly when comparing the performance of two diagnostic tests applied on the same set of patients, these metrics are crucial for identifying the more accurate test. However, comparing predictive values presents statistical challenges because their denominators depend on the test outcomes, unlike the comparison of sensitivities and specificities. This paper reviews existing methods for comparing predictive values and proposes using the permutation test. The permutation test is an intuitive, non-parametric method suitable for datasets with small sample sizes. We demonstrate each method using a dataset from MRI and combined modality of mammography and ultrasound in diagnosing breast cancer.

Keywords

References

  1. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ 2015;351:h5527 
  2. Celentano DD, Szklo M. Gordis epidemiology. 6th ed. Philadelphia: Elsevier, 2019 
  3. Mercaldo ND, Lau KF, Zhou XH. Confidence intervals for predictive values with an emphasis to case-control studies. Stat Med 2007;26:2170-2183 
  4. Leisenring W, Alonzo T, Pepe MS. Comparisons of predictive values of binary medical diagnostic tests for paired designs. Biometrics 2000;56:345-351 
  5. Moskowitz CS, Pepe MS. Comparing the predictive values of diagnostic tests: sample size and analysis for paired study designs. Clin Trials 2006;3:272-279 
  6. Wang W, Davis CS, Soong SJ. Comparison of predictive values of two diagnostic tests from the same sample of subjects using weighted least squares. Stat Med 2006;25:2215-2229 
  7. Kosinski AS. A weighted generalized score statistic for comparison of predictive values of diagnostic tests. Stat Med 2013;32:964-977 
  8. Bakeman R, Robinson BF, Quera V. Testing sequential association: estimating exact p values using sampled permutations. Psychol Methods 1996;1:4-15 
  9. Ludbrook J. Advantages of permutation (randomization) tests in clinical and experimental pharmacology and physiology. Clin Exp Pharmacol Physiol 1994;21:673-686 
  10. Takahashi K, Yamamoto K. An exact test for comparing two predictive values in small-size clinical trials. Pharm Stat 2020;19:31-43 
  11. Stock C, Hielscher T, Discacciati A. DTComPair: comparison of binary diagnostic tests in a paired study design [accessed on January 3, 2024]. Available at: http://CRAN.R-project.org/package=DTComPair