Korean Journal of Agricultural Science (농업과학연구)

Institute of Agricultural Science, CNU (충남대학교 농업과학연구소)
권호 표지
DOI QR코드

DOI QR Code

Estimation of tomato maturity as a continuous index using deep neural networks

  • Taehyeong Kim (Artificial Intelligence Laboratory, Chief Technology Officer Division, LG Electronics) ;
  • Dae-Hyun Lee (Department of Biosystems Mechanical Engineering, Chungnam National University) ;
  • Seung-Woo Kang (Department of Biosystems Mechanical Engineering, Chungnam National University) ;
  • Soo-Hyun Cho (Department of Biosystems Mechanical Engineering, Chungnam National University) ;
  • Kyoung-Chul Kim (Department of Agricultural Engineering, National Institute of Agricultural Sciences)
  • Received : 2022.09.07
  • Accepted : 2022.11.03
  • Published : 2022.12.01
Download PDF
⟨ Previous Next ⟩

Abstract

In this study, tomato maturity was estimated based on deep learning for a harvesting robot. Tomato images were obtained using a RGB camera installed on a monitoring robot, which was developed previously, and the samples were cropped to 128 × 128 size images to generate a dataset for training the classification model. The classification model was constructed based on convolutional neural networks, and the mean-variance loss was used to learn implicitly the distribution of the data features by class. In the test stage, the tomato maturity was estimated as a continuous index, which has a range of 0 to 1, by calculating the expected class value. The results show that the F1-score of the classification was approximately 0.94, and the performance was similar to that of a deep learning-based classification task in the agriculture field. In addition, it was possible to estimate the distribution in each maturity stage. From the results, it was found that our approach can not only classify the discrete maturation stages of the tomatoes but also can estimate the continuous maturity.

Keywords

Acknowledgement

본 결과물은 농림축산식품부 및 과학기술정보통신부, 농촌진흥청의 재원으로 농림식품기술기획평가원과 재단법인 스마트팜연구개발사업단의 스마트팜다부처 패키지혁신기술개발사업의 지원을 받아 연구되었으며(421031-04), 2022년도 교육부의 재원으로 한국연구재단의 지원을 받아 수행된 지자체-대학 협력기반 지역혁신 사업의 결과임(2021RIS-004).

References

  1. Baek SM, Kim WS, Kim YJ, Chung SO, Nam KC, Lee DH. 2020. Development of a real-time crop recognition system using a stereo camera. Korean Journal of Agricultural Science 47:315-326. [in Korean] https://doi.org/10.7744/KJOAS.20200022
  2. Begum N, Hazarika MK. 2022. Maturity detection of tomatoes using transfer learning. Measurement: Food 7:100038.
  3. Benavides M, Canton-Garbin M, Sanchez-Molina JA, Rodriguez F. 2020. Automatic tomato and peduncle location system based on computer vision for use in robotized harvesting. Applied Sciences 10:5887.
  4. Chun CJ, Shim SB, Kang SM, Ryu SK. 2018. Development of evaluation of automatic pothole detection using fully convolutional neural networks. The Journal of the Korea Institute of Intelligent Transport Systems 17:55-64. [in Korean]
  5. Fu L, Majeed Y, Zhang X, Karkee M, Zhang Q. 2020. Faster R-CNN-based apple detection in dense-foliage fruiting-wall trees using RGB and depth features for robotic harvesting. Biosystems Engineering 197:245-256. https://doi.org/10.1016/j.biosystemseng.2020.07.007
  6. Kamilaris A, Kartakoullis A, Prenafeta-Boldu FX. 2017. A review on the practice of big data analysis in agriculture. Computers and Electronics in Agriculture 143:23-37. https://doi.org/10.1016/j.compag.2017.09.037
  7. Kang H, Zhou H, Wang X, Chen C. 2020. Real-time fruit recognition and grasping estimation for robotic apple harvesting. Sensors 20:5670.
  8. Kim G, Seo D, Kim KC, Hong Y, Lee M, Lee S, Kim H, Ryu H, Kim YJ, Chung SO, et al. 2020. Tillage boundary detection based on RGB imagery classification for an autonomous tractor. Korean Journal of Agricultural Science 47:205-217. [in Korean] https://doi.org/10.7744/KJOAS.20200006
  9. Kim WS, Lee DH, Kim T, Kim G, Kim H, Sim T, Kim YJ. 2021. One-shot classification-based tilled soil region segmentation for boundary guidance in autonomous tillage. Computers and Electronics in Agriculture 189:106371.
  10. Li H, Zhu Q, Huang M, Gui Y, Qin J. 2018. Pose estimation of sweet pepper through symmetry axis detection. Sensors 18:3083.
  11. Li J, Tang Y, Zou X, Lin G, Wang H. 2020. Detection of fruit-bearing branches and localization of litchi clusters for visionbased harvesting robots. IEEE Access 8:117746-117758. https://doi.org/10.1109/ACCESS.2020.3005386
  12. Lin G, Tang Y, Zou X, Xiong J, Li J. 2019. Guava detection and pose estimation using a low-cost RGB-D sensor in the field. Sensors 19:428.
  13. Pan H, Han H, Shan S, Chen X. 2018. Mean-variance loss for deep age estimation from a face. pp. 5285-5294. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  14. Rong J, Dai G, Wang P. 2021. A peduncle detection method of tomato for autonomous harvesting. Complex & Intelligent Systems 8:2955-2969. https://doi.org/10.1007/s40747-021-00522-7
  15. Seo D, Cho B, Kim KC. 2021. Development of monitoring robot system for tomato fruits in hydroponic greenhouses. Agronomy 11:2211.
  16. Stefas N, Bayram H, Isler V. 2019. Vision-based monitoring of orchards with UAVs. Computers and Electronics in Agriculture 163:104814.
  17. Zhang F, Gao J, Zhou H, Zhang J, Zou K, Yuan T. 2022. Three-dimensional pose detection method based on keypoints detection network for tomato bunch. Computers and Electronics in Agriculture 195:106824.