The Journal of the Korea institute of electronic communication sciences (한국전자통신학회논문지)

Korea Institute of Electronic Communication Science (한국전자통신학회)
권호 표지
DOI QR코드

DOI QR Code

Species-level Zooplankton Classifier and Visualization using a Convolutional Neural Network

합성곱 신경망을 이용한 종 수준의 동물플랑크톤 분류기 및 시각화

  • Man-Ki Jeong ;
  • Ho Young Soh ;
  • Hyi-Thaek Ceong (Dept. of Multimedia, Chonnam National University)
  • 정만기 (전남대학교 스마트자원관리학과 ) ;
  • 서호영 (전남대학교 해양융합과학과 ) ;
  • 정희택 (전남대학교 멀티미디어전공)
  • Received : 2024.06.30
  • Accepted : 2024.07.25
  • Published : 2024.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

Species identification of zooplankton is the most basic process in understanding the marine ecosystem and studying global warming. In this study, we propose an convolutional neural network model that can classify females and males of three zooplankton at the species level. First, training data including morphological features is constructed based on microscopic images acquired by researchers. In constructing training data, a data argumentation method that preserves morphological feature information of the target species is applied. Next, we propose a convolutional neural network model in which features can be learned from the constructed learning data. The proposed model minimized the information loss of training image in consideration of high resolution and minimized the number of learning parameters by using the global average polling layer instead of the fully connected layer. In addition, in order to present the generality of the proposed model, the performance was presented based on newly acquired data. Finally, through the visualization of the features extracted from the model, the key features of the classification model were presented.

동물플랑크톤의 종 동종은 해양 생태계의 이해 및 지구온난화를 연구하는데 가장 기본이다. 본 연구에서는 3종의 동물플랑크톤을 종 수준에서 암컷과 수컷을 분류할 수 있는 합성곱 신경망 모델을 제안한다. 첫째 연구자들이 획득하는 현미경 이미지를 기반으로 형태적 특징을 포함하는 학습데이터를 구축한다. 학습데이터의 구축에 있어 대상 종의 형태적 특징 정보를 보존하는 데이터 확대 방법을 적용한다. 둘째 구축된 학습데이터로부터 종 특징들이 학습될 수 있는 합성곱 신경망 모델을 제안한다. 제안한 모델은 높은 해상도를 고려하여 학습 이미지 정보 손실을 최소화하였고 완전 연결 층 대신에 전역 평균 폴링 층을 사용하여 학습 매개 변수 개수를 최소화하였다. 제안한 모델의 일반성을 제시하기 위해 새로이 획득한 데이터를 기반으로 성능을 제시하였다. 마지막으로 개발된 모델에서 추출된 특징들의 시각화를 통해, 분류 모델의 중요 특징을 제시하였다.

Keywords

Acknowledgement

이 논문은 2024년 해양수산부 재원으로 해양수산과학기술진흥원의 지원을 받아 수행된 연구임(RS-2018-KS181192, 수산전문인력양성).

References

  1. A. Sournia, M.-J. Chrdtiennot-Dinet, and M. Ricard, "Marine phytoplankton: How many species in the world ocean?," J. Plankton Res, vol. 13, no. 5, 1991, pp. 1093-1099. https://doi.org/10.1093/plankt/13.5.1093 
  2. G. Park, C. Hwang, L. Ryung, and H. Jang, "CNN-LSTM based Autonomous Driving Technology," J. of The Korea Institute of Electronic Communication Sciences, vol. 18, no. 6, 2023, pp. 1259-1268. http://dx.doi.org/10.13067/JKIECS.2023.18.6.1259 
  3. K. Shokhrukh and J. H. Yoo, "Lightweight Residual Layer Based Convolutional Neural Networks for Traffic Sign Recognition," J. of The Korea Institute of Electronic Communication Sciences, vol. 17, no. 1, 2022, pp. 105-110. http://dx.doi.org/10.13067/JKIECS.2022.17.1.105 
  4. E. Orenstein, O. Beijbom, E. E. Peacock, and H. Sosik, "Whoi-plankton-a large scale fine grained visual recognition benchmark dataset for plankton classification," ArXiv, 2015, https://doi.org/10.48550/arXiv.1510.00745 
  5. H. Lee, M. Park, and J. Kim, "Plankton classification on imbalanced large scale database via convolutional neural networks with transfer learning," In Proc. IEEE Int. Conf. on Image Processing, Phoenix, USA, 2016, pp. 3713-3717. 
  6. J. Cui, B. Wei, C. Wang, Z. Yu, H. Zheng, B. Zheng, and H. Yang, "Texture and shape information fusion of convolutional neural network for plankton image classification," In Proc. OCEANS-MTS/IEEE Kobe Techno-Oceans (OTO), Kobe, Japan, 2018, pp. 1-5. 
  7. S. S. Rawat, A. Bisht, and R. Nijhawan, "A Deep Learning based CNN framework approach for Plankton Classification," In Proc. Fifth Int. Conf. on Image Information Processing (ICIIP), Shimla, India, 2019, pp. 268-273. 
  8. A. Lumini and L. Nanni, "Deep learning and transfer learning features for plankton classification," Ecological informatics, vol. 51, May 2019, pp. 33-43. https://doi.org/10.1016/j.ecoinf.2019.02.007 
  9. X. Li, R. Long, J. Yan, K. Jin, and J. Lee, "TANet: a tiny plankton classification network for mobile devices," Mobile Information Systems, no. 1, Apr. 2019, pp. 6536925. https://doi.org/10.1155/2019/6536925 
  10. V. P. Pastore, T. G. Zimmerman, S. K. Biswas, and S. Bianco, "Annotation-free learning of plankton for classification and anomaly detection," Scientific reports, vol. 10, no. 1, 2020, pp. 1-15. https://doi.org/10.1038/s41598-020-68662-3 
  11. A. Lumini, L. Nanni, and G. Maguolo, "Deep learning for plankton and coral classification," Applied Computing and Informatics, vol. 19, no. 3/4, June 2023, pp. 265-283. https://doi.org/10.1016/j.aci.2019.11.004 
  12. L. K. Leow, L. L. Chew, V. C. Chong, and S. K. Dhillon, "Automated identification of copepods using digital image processing and artificial neural network," BMC bioinformatics, vol. 16, no. 18, 2015, pp. 1-12. https://doi.org/10.1186/1471-2105-16-S18-S4 
  13. R. Wang, J. Dai, H. Zheng, G. Ji, and X. Qiao, "Multi features combination for automated zooplankton classification," In OCEANS 2016-Shanghai, Shanghai, China, 2016, pp. 1-5. 
  14. M. S. Schmid, C. Aubry, J. Grigor, and L. Fortier, "The LOKI underwater imaging system and an automatic identification model for the detection of zooplankton taxa in the Arctic Ocean," Methods in Oceanography, vol. 15, 2016, pp. 129-160. https://doi.org/10.1016/j.mio.2016.03.003 
  15. J. Dai, R. Wang, H. Zheng, G. Ji, and X. Qiao, "ZooplanktoNet: Deep convolutional network for zooplankton classification," In OCEANS 2016-Shanghai, Shanghai, China, 2016, pp. 1-6. 
  16. I. Correa, P. Drews, S. Botelho, M. S. de Souza, and V. M. Tavano, "Deep learning for microalgae classification," In 16th IEEE Int. Conf. on Machine Learning and Applications (ICMLA), Cancun, Mexico, 2017, pp. 20-25. 
  17. A. Y. Hsiang, A. Brombacher, M. C. Rillo, M. J. Mleneck Vautravers, S. Conn, S. Lordsmith, and P. M. Hull, "Endless Forams:> 34,000 modern planktonic foraminiferal images for taxonomic training and automated species recognition using convolutional neural networks," Paleoceanography and Paleoclimatology, vol. 34, no. 7, June, 2019, pp. 1157-1177. https://doi.org/10.1029/2019PA003612 
  18. K. Cheng, X. Cheng, Y. Wang, H. Bi, and M. C. Benfield, "Enhanced convolutional neural network for plankton identification and enumeration," PLoS One, vol. 14, no. 7, July, 2019, e0219570. https://doi.org/10.1371/journal.pone.0219570 
  19. J. S. Ellen, C. A. Graff, and M. D. Ohman, "Improving plankton image classification using context metadata," Limnology and Oceanography: Methods, vol. 17, no. 8, 2019, pp. 439-461. https://doi.org/10.1002/lom3.10324 
  20. O. Hwang, S. Hwang, K. Choi, H. Park, J. Kown, and G. Lee, "Application of Artificial Intelligence to Classify Zooplankton (>50 µm) in Ballast Water," J. Korean Soc. Mar. Environ. Energy, vol. 22, no. 2, 2019, pp.114-124. 
  21. B. Guo, L. Nyman, A. R. Nayak, D. Milmore, M. McFarland, M. S. Twardowski, and J. Hong, "Automated plankton classification from holographic imagery with deep convolutional neural networks," Limnology and Oceanography: Methods, vol. 19, no. 1, 2021, pp. 21-36. https://doi.org/10.1002/lom3.10402 
  22. X. Cheng, Y. Ren, K. Cheng, J. Cao, and Q. Hao, "Method for training convolutional neural networks for in situ plankton image recognition and classification based on the mechanisms of the human eye," Sensors, vol. 20, no. 9, 2020, pp. 2592. https://doi.org/10.3390/s20092592 
  23. B. Krawczyk, "Learning from imbalanced data: open challenges and future directions," Progress in Artificial Intelligence, vol. 5, 2016, pp. 221-232. https://doi.org/10.1007/s13748-016-0094-0 
  24. K. He, X. Zhang, S. Ren, and J. Sun, "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification," In Proc. IEEE Int. Conf. on computer vision, Santiago, Chile, 2015, pp. 1026-1034. 
  25. R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, "Grad-cam: Visual explanations from deep networks via gradient-based localization," In Proc. of the IEEE Int. Conf. on computer vision, Venics, Italy, 2017, pp. 618-626.