Accuracy Evaluation and Alert Level Setting for Real-time Cyanobacteria Measurement Using Receiver Operating Characteristic Curve Analysis

ROC 분석을 이용한 수질자동측정소 실시간 남조류 측정의 정확성 평가 및 경보기준 설정

  • Song, Sanghwan (Yeongsan River Environment Research Center, National Institute of Environmental Research) ;
  • Park, Jong-hwan (Yeongsan River Environment Research Center, National Institute of Environmental Research) ;
  • Kang, Tae-Woo (Yeongsan River Environment Research Center, National Institute of Environmental Research) ;
  • Kim, Young-Suk (Yeongsan River Environment Research Center, National Institute of Environmental Research) ;
  • Kim, Jihyun (Yeongsan River Environment Research Center, National Institute of Environmental Research) ;
  • Kang, Taegu (Yeongsan River Environment Research Center, National Institute of Environmental Research)
  • 송상환 (국립환경과학원 영산강물환경연구소) ;
  • 박종환 (국립환경과학원 영산강물환경연구소) ;
  • 강태우 (국립환경과학원 영산강물환경연구소) ;
  • 김영석 (국립환경과학원 영산강물환경연구소) ;
  • 김지현 (국립환경과학원 영산강물환경연구소) ;
  • 강태구 (국립환경과학원 영산강물환경연구소)
  • Received : 2016.11.15
  • Accepted : 2017.02.22
  • Published : 2017.03.30


With the need to evaluate accuracy of real-time measurement of cyanobacterial fluorescence to determine cyanobacterial blooms, this research examined 357 paired data (2013-2016) comprising both microscopic toxic cyanobacterial cell counts and concurrent real-time cyanobacterial concentrations at 2 sites (YS1 and YS2) in Yeongsan river. The increase in real-time cyanobacterial concentration was closely associated with the exceedance of 5,000 cyanobacterial cells/ml (odds ratio [OR] 1.07, 95% confidence interval [CI] 1.03-1.12) and 10,000 cells/ml (OR 1.08, 95% CI 1.04-1.12) at YS2 site. The area under the receiver operating characteristic (ROC) curve for the real-time cyanobacterial measurement at the YS2 site was 0.93, which indicates the measurement provides a high accurate detection of cyanobacterial blooms. On the ROC curve, the early alert levels of real-time cyanobacteria ranging $16-23{\mu}g$ chl-a/L would produce acceptable sensitivity of 79% and specificities greater than 90%. The real-time fluorescence measurement was found to be an accurate indicator of cyanobacteria and can serve as a tool for detecting toxic cyanobacterial bloom events in Youngsan river.


  1. Ahn, C. Y., Joung, S. H., Yoon, S. K., and Oh, H. M. (2007). Alternative Alert System for Cyanobacterial Bloom, Using Phycocyanin as a Level Determinant, The Journal of Microbiology, 45(2), 98-104.
  2. Agha, R., Cires, S., Wormer, L., Dominguez, J. A., and Quesada, A. (2012). Multi-scale Strategies for the Monitoring of Freshwater Cyanobacteria: Reducing the Sources of Uncertainty, Water Research, 46(9), 3043-3053.
  3. Beutler, M., Wiltshire, K. H., Meyer, B., Moldaenke, C., Luring, C., Meyerhofer, M., Hansen, U. P., and Dau, H. (2002). A Fluorometric Method for the Differentiation of Algal Populations in vivo and in situ, Photosynthesis Research, 72(1), 39-53.
  4. Bowling, L., Ryan, D., Holliday, J., and Honeyman, G. (2013). Evaluation of in situ Fluorometry to Determine Cyanobacterial Abundance in the Murray and Lower Daring Rivers, Australia, River Research and Applications, 29(8), 1059-1071.
  5. Brient, L., Lengronne, M., Bertrand, E., Rolland, D., Sipel, A., Steinmann, D., Baudin, I., Legeas, M., Le Rouzic, B., and Bormans, M. (2008). A Phycocyanin Probe as a Tool for Monitoring Cyanobacteria in Freshwater Bodies, Journal of Environmental Monitoring, 10(2), 248-255.
  6. Catherine, A., Escoffier, N., Belhocine, A., Nasri, A. B., Hamlaoui, S., Yepremian, C., Bernard, C., and Troussellier, M. (2012). On the Use of the $FluoroProbe^{(R)}$, a Phytoplankton Quantification Method Based on Fluorescence Excitation Spectra for Large-scale Surveys of Lakes and Reservoirs, Water Research, 46(6), 1771-1784.
  7. Chorus, I. and Bartram, J. (1999). Toxic Cyanobacteria in Water : A Guide to Their Public Health Consequences, Monitoring, and Management, London ; New York, E & FN Spon on behalf of World Health Organization.
  8. Falconer, I. R. and Humpage, A. R. (2005). Health Risk Assessment of Cyanobacterial (Blue-green Algal) Toxins in Drinking Water, International Journal of Environmental Research and Public Health, 2(1), 43-50.
  9. Gregor, J. and Marsalek, B. (2004). Freshwater Phytoplankton Quantification by Chlorophyll a: A Comparative Study of in vitro, in vivo and in situ Methods, Water Research, 38(3), 517-522.
  10. Gregor, J., Marsalek, B., and Sipkova, H. (2007). Detection and Estimation of Potentially Toxic Cyanobacteria in Raw Water at the Drinking Water Treatment Plant by in vivo Fluorescence Method, Water Research, 41(1), 228-234.
  11. Greiner, M., Pfeiffer, D., and Smith, R. D. (2000). Principles and Practical Application of the Receiver-operating Characteristic Analysis for Diagnostic Tests, Preventive Veterinary Medicine, 45(1-2), 23-41.
  12. Izydorczyk, K., Carpentier, C., Mrowczynski, J., Wagenvoort, A., Jurczak, T., and Tarczynska, M. (2009). Establishment of an Alert Level Framework for Cyanobacteria in Drinking Water Resources by Using the Algae Online Analyser for Monitoring Cyanobacterial Chlorophyll a, Water Research, 43(4), 989-996.
  13. Leboulanger, C., Dorigo, U., Jacquet, S., Le Berre, B., Paolini, G., and Humbert, J. F. (2002). Application of a Submersible Spectrofluorometer for Rapid Monitoring Freshwater Cyanobacterial Bloom: A Case Study, Aquatic Microbial Ecology, 30, 83-89.
  14. Linden, A. (2006). Measuring Diagnostic and Predictive Accuracy in Disease Management: An Introduction to Receiver Operating Characteristic (ROC) Analysis, Journal of Evaluation in Clinical Practice, 12(2), 132-139.
  15. Lorenzen, C. J. (1966). A Method for the Continuous Measurement of in vivo Chlorophyll Concentration, Deep-Sea Research, 13, 223-227.
  16. Marion, J. W., Lee, J., Wilkins, III, J. R., Lemeshow, S., Lee, C., Waletzko, E. J., and Buckley, T. J. (2012). In vivo Phycocyanin Flourometry as a Potential Rapid Screening Tool for Predicting Elevated Microcystin Concentrations at Eutrophic Lakes, Environmental Science and Technology, 46(8), 4523-4531.
  17. McLaughlin, D. B. (2012). Assessing the Predictive Performance of Risk-Based Water Quality Criteria Using Decision Error Estimates from Receiver Operating Characteristics (ROC) Analysis, Integrated Environmental Assessment and Management, 8(4), 674-684.
  18. Metz, C. E. (1978). Basic Principles of ROC Analysis, Seminars in Nuclear Medicine, 8(4), 283-298.
  19. Ministry of Environment (MOE). (2015). Standard Method for the Examination of Water Pollution, Ministry of Environment. [Korean Literature]
  20. Ministry of Environment (MOE). (2016). Water Information System (WIS), (accessed Sept. 2016).
  21. Morrison, A. M., Coughlin, K., Shine, J. P., Coull, B. A., and Rex, A. C. (2003). Receiver Operating Characteristic Curve Analysis of Beach Water Quality Indicator Variables, Applied Environmental Microbiology, 69(11), 6405-6411.
  22. Noh, S. Y., Park, H. K., Choi, H. L., and Lee, J. A. (2014). Effect of Climate Change for Cyanobacteria Growth Pattern in Chudong Station of Lake Daechung, Journal of Korean Society on Water Environment, 30(4), 377-385. [Korean Literature]
  23. Paerl, H. W., Hall, N. S., and Calandrino, E. S. (2011). Controlling Harmful Cyanobacterial Blooms in a World Experiencing Anthropogenic and Climatic-induced Change, Science of Total Environment, 409(10), 1739-1745.
  24. Paerl, H. W. and Paul, V. J. (2012). Climate Change: Links to Global Expansion of Harmful Cyanobacteria, Water Research, 46(5), 1349-1363.
  25. Park, H. K., Kim, H., Lee, J. J., Lee, J. A., Lee, H., Park, J. H., Seo, J., Youn, S. J., and Moon, J. (2011). Investigation of Criterion on Harmful Algae Alert System using Correlation between Cell Numbers and Cellular Microcystins Content of Korean Toxic Cyanobacteria, Journal of Korean Society on Water Environment, 27(4), 491-498. [Korean Literature]
  26. Park, H. K., Shin, R. Y., Lee, H., Lee, K. L., and Cheon, S. U. (2015). Spatio-temporal Characteristics of Cyanobacterial Communities in the Middle-downstream of Nakdong River and Lake Dukdong, Journal of Korean Society on Water Environment, 31(3), 286-294. [Korean Literature]
  27. Pinto, A. M., Sperling, E. V., and Moreira, R. M. (2001). Chlorophyll-a Determination via Continuous Measurement of Plankton Fluorescence: Methodology Development, Water Research, 35(16), 3977-3981.
  28. Rolland, A., Rimet, F., and Jacquet, S. (2010). A 2-year Survey of Phytoplankton in the Marine Reservoir (France): A Case Study to Validate the Use of an in situ Spectrofluorometer by Comparison with Algal Taxonomy and Chlorophyll a Measurements, Knowledge and Management of Aquatic Ecosystems, 398(02), 1-19.
  29. World Health Organization (WHO). (2003). Guidelines for Safe Recreational Water Environments, Geneva, World Health Organization.