Korean Journal of Agricultural and Forest Meteorology (한국농림기상학회지)

Korean Society of Agricultural and Forest Meteorology (한국농림기상학회)
권호 표지
DOI QR코드

DOI QR Code

Estimation of Rice Heading Date of Paddy Rice from Slanted and Top-view Images Using Deep Learning Classification Model

딥 러닝 분류 모델을 이용한 직하방과 경사각 영상 기반의 벼 출수기 판별

  • Hyeok-jin Bak (National Institute of Crop Science, Rural Development Administration) ;
  • Wan-Gyu Sang (National Institute of Crop Science, Rural Development Administration) ;
  • Sungyul Chang (National Institute of Crop Science, Rural Development Administration) ;
  • Dongwon Kwon (National Institute of Crop Science, Rural Development Administration) ;
  • Woo-jin Im (National Institute of Crop Science, Rural Development Administration) ;
  • Ji-hyeon Lee (National Institute of Crop Science, Rural Development Administration) ;
  • Nam-jin Chung (Jeonbuk National University, Department of Crop Science and Biotechnology) ;
  • Jung-Il Cho (National Institute of Crop Science, Rural Development Administration)
  • 박혁진 (국립식량과학원 작물재배생리과) ;
  • 상완규 (국립식량과학원 작물재배생리과) ;
  • 장성율 (국립식량과학원 작물재배생리과) ;
  • 권동원 (국립식량과학원 작물재배생리과) ;
  • 임우진 (국립식량과학원 작물재배생리과) ;
  • 이지현 (국립식량과학원 작물재배생리과) ;
  • 정남진 (전북대학교 농학과) ;
  • 조정일 (국립식량과학원 작물재배생리과)
  • Received : 2023.10.17
  • Accepted : 2023.11.30
  • Published : 2023.12.30
Download PDF
⟨ Previous Next ⟩

Abstract

Estimating the rice heading date is one of the most crucial agricultural tasks related to productivity. However, due to abnormal climates around the world, it is becoming increasingly challenging to estimate the rice heading date. Therefore, a more objective classification method for estimating the rice heading date is needed than the existing methods. This study, we aimed to classify the rice heading stage from various images using a CNN classification model. We collected top-view images taken from a drone and a phenotyping tower, as well as slanted-view images captured with a RGB camera. The collected images underwent preprocessing to prepare them as input data for the CNN model. The CNN architectures employed were ResNet50, InceptionV3, and VGG19, which are commonly used in image classification models. The accuracy of the models all showed an accuracy of 0.98 or higher regardless of each architecture and type of image. We also used Grad-CAM to visually check which features of the image the model looked at and classified. Then verified our model accurately measure the rice heading date in paddy fields. The rice heading date was estimated to be approximately one day apart on average in the four paddy fields. This method suggests that the water head can be estimated automatically and quantitatively when estimating the rice heading date from various paddy field monitoring images.

벼의 출수기를 추정하는 것은 농업생산성과 관련된 중요한 과정 중 하나이지만 세계적인 이상기후의 증가로 벼의 출수기를 추정하는 것이 어려워지고 있다. 본 연구에서는 CNN 분류모델을 사용하여 다양한 영상데이터에서 벼의 출수기를 추정하려고 시도하였다. 드론과 타워형 영상관측장치 그리고 일반 RGB 카메라로 촬영된 직하방과 경사각 영상을 수집하였다. 수집한 영상은 CNN 모델의 입력데이터로 사용하기 위해서 전처리를 진행하였고, 사용된 CNN 아키텍처는 이미지 분류 모델에서 일반적으로 사용되는 ResNet50, InceptionV3 그리고 VGG19 를 사용하였다. 각각의 아키텍처는 모델의 종류, 영상의 유형과 관계없이 0.98 이상의 정확도를 나타내었다. 또한 CNN 분류 모델이 영상의 어떤 특징을 보고 분류하였는지 시각적으로 확인하기 위해서 Grad-CAM 을 사용하였다. Grad-CAM 결과 CNN 분류 모델은 벼의 출수를 이삭의 형태에 높은 가중치를 두어 분류 하는 것을 확인하였다. 다음으로 작성된 모델이 실제 논 포장 모니터링 이미지에서 벼의 출수기를 정확하게 추정하는지 확인하였다. 각각 다른 지역 4 개의 벼 포장에서 벼의 출수기를 약 하루정도의 차이로 추정하는 것을 확인하였다. 이 방법을 통해서 다양한 논 포장의 모니터링 이미지를 활용하여 자동적이고 정량적으로 벼의 출수기를 추정 할 수 있다고 판단된다.

Keywords

Acknowledgement

본 논문은 농촌진흥청 연구개발사업(과제번호: PJ016759032023)의 지원에 의해 이루어진 것임.

References

  1. Ban, H. Y., J. K. Beak, W. G. Sang, J. H. Kim and M. C. Seo. 2021: Estimation of the Lodging Area in Rice Using Deep Learning. Korea Journal of Crop Science 66(2), 105-111. (in Korean with English abstract)
  2. Bernier, G. and C. Perilleux. 2005: A physiological overview of the genetics of flowering time control. Plant Biotechnol. Journal 3, 3-16. doi: 10.1111/j.1467-7652.2004.00114.x
  3. Corbesier, L., C. Vincent, and S. Jang. 2007: FT protein movement contributes to long-distance signaling in floral induction of Arabidopsis. Science 316(5827), 1030-1033. doi: 10.1126/science.1141752
  4. Desai, S. V., V. N. Balasubramanian, T. Fukatsu, S. Ninomiya and W. Guo: 2019. Automatic estimation of heading date of paddy rice using deep learning. Plant Methods 15, 76. https://doi.org/10.1186/s13007-019-0457-1
  5. Fukushima, K: 1980: Neocognitron: A Self-organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position. Biological Cybernetics 36, 193-202 https://doi.org/10.1007/BF00344251
  6. Gulzar, Y., Y. Hamid, A. B. Alwan and L. Journaux. 2020: A Convolution Neural Network-Based Seed Classification. System Symmetry 12(12). doi:10.3390/sym12122018
  7. Hama, A., K. Tanaka, B. Chen and A. Kondoh. 2021. Examination of appropriate observation time and correction of vegetation index for drone-based crop monitoring. Journal of Agricultural Meteorology 77(3) DOI: 10.2480/agrmet.D-20-00047
  8. Hoshikawa, K. 1989: The growing rice plant. An anatomical monograph. Nobunkyo Tokyo. 238-239.
  9. Jang, H. and S. Cho: 2016 Automatic Tagging for Social images using Convolution Neural Networks. Korean Institute of Information Scientists and Engineers 43(1), 47-53. (in Korean with English abstract) https://doi.org/10.5626/JOK.2016.43.1.47
  10. Kim, M. H., K.D. Bu, and B. W. Lee. 2006: Yield Response to Nitrogen Topdress Rate at Panicle Initiation Stage under Different Growth and Nitrogen Nutrition Status of Rice Plnat. Korea Journal of Crop Science 51(7), 571-583. (in Korean with English abstract)
  11. Kim, S. S., J. H. Lee, J. K. Nam, W. Y. Choi, N. H. Beak, H. G. Park, M. G. Choi, J. G. Kim and G. Y. Jung. 2005: Proper Harvesting Time for Imporving the Rice Quality in Honam Plain Area. Korea Journal of Crop Science 50(S), 62-68. (in Korean with English abstract)
  12. Lee, J. G., S. H. Jun, Y. W. Cho, H. N. Lee, G. B. Kim, J. B. Seo and N. K. Kim. 2017: Deep Learning in Medical Imaging: General Overview. Korean Journal of Radiolgy 18(4), 570-584. https://doi.org/10.3348/kjr.2017.18.4.570
  13. Leonardo, M. M., T. J. Carvalho, E. Rezende, R. Zucchi and F. A. Faria. 2019: Deep Feature-based Classifiers for Fruit Fly Identification (Diptera: Tephritidae). Conference: 2018 31st SIBGRAPI Conference on Graphics, Patterns and Images. DOI:10.1109/SIBGRAPI.2018.00012
  14. Loomis, R. S. and D. J. Connor. 1992: Crop ecology: productivity and management in agricultural systems. Cambridge University Press Cambridge, 104-128.
  15. Luis, P and W. Jason: 2017: The Effectiveness of Data Augmentation in Image Classification using Deep Learning. Computer Vision and Pattern Recognition, arXiv:1712.04621
  16. Oh, Y. J. and K. H. Park, 1993. In : Low input sustainable crop production systems in Asia. Korean Journal of Crop Science symposium in Seoul National University, 125-139.
  17. Schmidhuber, J. 2014: Deep Learning in Neural Networks: An Overview. arXiv:1404.7828
  18. Selvaraju, R. R., M. Cogswell, A. Das, R. Vedantam. D. Parikh and D. Batra 2017: Grad-CAM: Visual Explantations from Deep Networks via Gradient-Based Localization. International Conference on Computer Vision, 22-29. doi:10.1109/ICCV.2017.74
  19. Sritarapipat, T., P. Rakwatin and T. Kasetkasem. 2014: Automatic Rice Crop Height Measurement Using a Field Server and Digital Image Processing. Sensors 14, 900-926. doi:10.3390/s140100900
  20. Soontranon, N., P. Srestasathiern and P. Rakwatin. 2015: Rice Crop Calendar Based on Phenology Analysis from Time-series Images. Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology 12, 1-5. doi: 10.1109/ECTICon.2015.7207029
  21. Vishal, M. K., R. Saluja, D. Aggrawal, B. Banerjee, D. Raju, S. Kumar, V. Chinnusamy, R. N. Sahoo and J, Adinarayana: 2022. Leaf Count Aided Novel Framework for Rice (Oryza sativa L.) Genotypes Discrimination in Phenomics: Leveraging Computer Vision and Deep Learning Applications. Plant 11(19), 2663. https://doi.org/10.3390/plants11192663
  22. Yoshida, S. 1981: Fundamentals of rice crop science. IRRI, Los Banos, Laguna, Philippines. pp 269.
  23. Zhang, Y., Z. Su, W. Shen, R. Jia, J. Luan. 2016: Remote Monitoring of Heading Rice Growing and Nitrogen Content Based on UAV Images. International Journal of Smart Home 10(7), 103-114. https://doi.org/10.14257/ijsh.2016.10.7.11