The Korean Journal of Applied Statistics (응용통계연구)

The Korean Statistical Society (한국통계학회)
권호 표지
DOI QR코드

DOI QR Code

AUC and VUS using truncated distributions

절단함수를 이용한 AUC와 VUS

  • Received : 2019.05.16
  • Accepted : 2019.06.28
  • Published : 2019.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

Significant literature exists on the area under the ROC curve (AUC) and the volume under the ROC surface (VUS) which are statistical measures of the discriminant power of classification models. Whereas the partial AUC is restricted on the false positive rate, the two-way partial AUC is restricted on both the false positive rate and true positive rate, which could be more efficient and accurate than partial AUC. The two-way partial AUC was suggested as more efficient and accurate than the partial AUC. Partial VUS as well as the three-way partial VUS were also developed for the ROC surface. A proposed AUC is expressed in this paper with probability and integration using two truncated distribution functions restricted on both the false positive rate and true positive rate. It is also found that this AUC has a relation with the two-way partial AUC. The three-way partial VUS for the ROC surface is also related to the VUS using truncated distribution functions. These AUC and VUS are represented and estimated in terms of Mann-Whitney statistics. Their parametric and non-parametric estimation methods are explored based on normal distributions and random samples.

ROC 곡선 아래 면적과 ROC 곡면 아래 부피를 이용하여 분류모형의 판별력을 측정하는 통계량인 AUC와 VUS에 관한 많은 연구가 있다. ROC 곡선을 구성하는 FPR과 TPR 모두에 제한을 두는 양방향 부분 AUC는 부분 AUC보다 더 효과적이고 정확하게 제안되었다. ROC 곡면에서도 부분 VUS 뿐만 아니라 세 방향 부분 VUS 통계량이 개발되었다. 본 연구에서는 ROC 곡선의 FPR과 TPR 모두에 제한된 두 개의 절단함수를 이용하여 확률 개념과 적분 표현으로 대안적인 AUC를 제안한다. 또한 이 AUC는 양방향 부분 AUC와 관계가 있음을 알 수 있다. ROC 곡면에서의 세 방향 부분 VUS도 절단함수를 이용하는 VUS와 관련되어 있음을 발견하였다. 그리고 이러한 대안적인 AUC와 VUS는 맨-휘트니 통계량으로 표현되고 추정된다. 정규분포와 확률표본을 기반으로 이들의 모수적인 추정 방법과 비모수적인 추정 방법을 탐색한다.

Keywords

References

  1. Bradley, A. P. (1997). The use of the area under the ROC curve in the evaluation of machine learning algorithms, Pattern Recognition, 30, 1145-1159. https://doi.org/10.1016/S0031-3203(96)00142-2
  2. Dreiseitl, S., Ohno-Machado, L., and Binder, M. (2000). Comparing three-class diagnostic tests by threeway ROC analysis, Medical Decision Making, 20, 323-331. https://doi.org/10.1177/0272989X0002000309
  3. Fawcett, T. (2003). ROC graphs: notes and practical considerations for data mining researchers, HP Laboratories, California.
  4. Greiner, M., Pfeiffer, D., and Smith, R. D. (2000). Principles and practical application of the receiver operating characteristic analysis for diagnostic tests, Preventive Veterinary Medicine, 45, 23-41. https://doi.org/10.1016/S0167-5877(00)00115-X
  5. Heckerling, P. S. (2001). Parametric three-way receiver operating characteristic surface analysis using mathematica, Medical Decision Making, 21, 409-417. https://doi.org/10.1177/02729890122062703
  6. Hong, C. S. and Cho, H. S. (2019). Partial AUC and optimal thresholds, The Korean Journal of Applied Statistics, 32, 187-198. https://doi.org/10.5351/KJAS.2019.32.2.187
  7. Hong, C. S. and Cho, M. H. (2015). VUS and HUM represented with Mann-Whitney statistic, Communications for Statistical Applications and Methods, 22, 223-232. https://doi.org/10.5351/CSAM.2015.22.3.223
  8. Hong, C. S., Jung, M. S., and Shin, H. S. (2019). Three-way partial VUS, Journal of The Korean Data and Information Science Society, 30, 445-454. https://doi.org/10.7465/jkdi.2019.30.2.445
  9. Jiang, Y., Metz, C., and Nishikawa, R. (1996). A receiver operating characteristic partial area index for highly sensitive diagnostic tests, Radiology, 201, 745-750. https://doi.org/10.1148/radiology.201.3.8939225
  10. Krzanowski, W. J. and Hand, D. J. (2009). ROC Curves for Continuous Data (1st ed.), CRC Press, New York.
  11. McClish, D. (1989). Analyzing a portion of the ROC curve, Medical Decision Making, 9, 190-195. https://doi.org/10.1177/0272989X8900900307
  12. Metz, C. E. (1978). Basic principles of ROC analysis, Seminars in Nuclear Medicine, 8, 283-298. https://doi.org/10.1016/S0001-2998(78)80014-2
  13. Mossman, D. (1999). Three-way ROCs, Medical Decision Making, 19, 78-89. https://doi.org/10.1177/0272989X9901900110
  14. Nakas, C. T. and Yiannoutsos, C. T. (2004). Ordered multiple-class ROC analysis with continuous measurements, Statistics in Medicine, 23, 3437-3449. https://doi.org/10.1002/sim.1917
  15. Nakas, C. T., Alonzo, T. A., and Yiannoutsos, C. T. (2010). Accuracy and cut-off point selection in threeclass classification problems using a generalization of the Youden index, Statistics in Medicine, 23, 3437-3449. https://doi.org/10.1002/sim.1917
  16. Scurfield, B. K. (1996). Multiple-event forced-choice tasks in the theory of signal detectability, Journal of Mathematical Psychology, 40, 253-269. https://doi.org/10.1006/jmps.1996.0024
  17. Tasche, D. (2006). Validation of internal rating systems and PD estimates, arXiv.org, eprint arXiv: physics/0606071
  18. Thompson, M. L. and Zucchini, W. (1989). On the statistical analysis of ROC curves, Statistics in Medicine, 8, 1277-1290. https://doi.org/10.1002/sim.4780081011
  19. Wandishin, M. S. and Mullen, S. J. (2009). Multiclass ROC analysis, Weather and Forecasting, 24, 530-547. https://doi.org/10.1175/2008WAF2222119.1
  20. Xiong, C., Van Belle, G., Miller, J. P., and Morris, J. C. (2006). Measuring and estimating diagnostic accuracy when there are three ordinal diagnostic groups, Statistics in Medicine, 25, 1251-1273. https://doi.org/10.1002/sim.2433
  21. Yang, H., Lu, K., Lyu, X., and Hu, F. (2019). Two-way partial AUC and its properties, Statistical Methods in Medical Research, 28, 184-195. https://doi.org/10.1177/0962280217718866
  22. Zweig, M. H. and Campbell, G. (1993). Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine, Clinical Chemistry, 39, 561-577. https://doi.org/10.1093/clinchem/39.4.561