The Journal of Korea Robotics Society (로봇학회논문지)

Korea Robotics Society (한국로봇학회)
권호 표지
DOI QR코드

DOI QR Code

Real-Time Tomato Instance Tracking Algorithm by using Deep Learning and Probability Model

딥러닝과 확률모델을 이용한 실시간 토마토 개체 추적 알고리즘

  • Ko, KwangEun (Robot Applied R&D Group, Korea Institute of Industrial Technology) ;
  • Park, Hyun Ji (Robot Applied R&D Group, Korea Institute of Industrial Technology) ;
  • Jang, In Hoon (Robot Applied R&D Group, Korea Institute of Industrial Technology)
  • Received : 2020.02.23
  • Accepted : 2020.11.03
  • Published : 2021.02.26
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, a smart farm technology is drawing attention as an alternative to the decline of farm labor population problems due to the aging society. Especially, there is an increasing demand for automatic harvesting system that can be commercialized in the market. Pre-harvest crop detection is the most important issue for the harvesting robot system in a real-world environment. In this paper, we proposed a real-time tomato instance tracking algorithm by using deep learning and probability models. In general, It is hard to keep track of the same tomato instance between successive frames, because the tomato growing environment is disturbed by the change of lighting condition and a background clutter without a stochastic approach. Therefore, this work suggests that individual tomato object detection for each frame is conducted by YOLOv3 model, and the continuous instance tracking between frames is performed by Kalman filter and probability model. We have verified the performance of the proposed method, an experiment was shown a good result in real-world test data.

Keywords

References

  1. O. O. Arjenaki, P. A. Moghaddam, and A. M. Motlagh, "Online tomato sorting based on shape, maturity, size, and surface defects using machine vision," Turkish J. Agric. For., vol. 37, pp. 62-68, 2013, DOI: 10.3906/tar-1201-10.
  2. M. P. Arakeri and Lakshmana, "Computer Vision Based Fruit Grading System for Quality Evaluation of Tomato in Agriculture industry," Procedia Comput. Sci., vol. 79, pp. 426-433, 2016, DOI: 10.1016/j.procs.2016.03.055.
  3. C. Li, Q. Cao, and F. Guo, "A method for color classification of fruits based on machine vision," WSEAS Trans. Syst., vol. 8, no. 2, 2009, [Online], https://dl.acm.org/doi/10.5555/1558836.1558849.
  4. N. El-Bendary, E. El Hariri, A. E. Hassanien, and A. Badr, "Using machine learning techniques for evaluating tomato ripeness," Expert Syst. Appl., vol. 42, no. 4, pp. 1892-1905, 2015, DOI: 10.1016/j.eswa.2014.09.057.
  5. Y.-Y. Zheng, J.-L. Kong, X.-B. Jin, X.-Y. Wang, T.-L. Su, and M. Zuo, "CropDeep: The Crop Vision Dataset for Deep-Learning-Based Classification and Detection in Precision Agriculture," Sensors, vol. 19, no. 5, p. 1058, Mar. 2019, DOI: 10.3390/s19051058.
  6. K. E. Ko, S.-H. Yang, and I. Jang, "Real-time tomato ripeness classification system based on deep learning model for object detection," J. Inst. Control. Robot. Syst., vol. 24, no. 11, pp. 999-1004, 2018, DOI: 10.5302/J.ICROS.2018.18.0166.
  7. N. Goel and P. Sehgal, "Fuzzy classification of pre-harvest tomatoes for ripeness estimation {\textendash} An approach based on automatic rule learning using decision tree," Appl. Soft Comput. J., vol. 36, pp. 45-56, 2015, DOI: 10.1016/j.asoc.2015.07.009.
  8. G. Liu, S. Mao, and J. H. Kim, "A mature-tomato detection algorithm using machine learning and color analysis," Sensors (Switzerland), vol. 19, no. 9, May, 2019, DOI: 10.3390/s19092023.
  9. A. Singh, B. Ganapathysubramanian, A. K. Singh, and S. Sarkar, "Machine Learning for High-Throughput Stress Phenotyping in Plants," Trends Plant Sci., vol. 21, no. 2, pp. 110-124, 2016, DOI: 10.1016/j.tplants.2015.10.015.
  10. G. L. Grinblat, L. C. Uzal, M. G. Larese, and P. M. Granitto, "Deep learning for plant identification using vein morphological patterns," Comput. Electron. Agric., vol. 127, no. C, pp. 418-424, 2016, DOI: 10.1016/j.compag.2016.07.003.
  11. J. Redmon and A. Farhadi, "YOLOv3: An Incremental Improvement," arXiv:1804.02767, 2018, [Online], https://arxiv.org/abs/1804.02767.
  12. S. Ren, K. He, R. Girshick, and J. Sun, "Faster R-CNN: towards real-time object detection with region proposal networks," IEEE Trans. Pattern Anal. Mach. Intell., no. 6, pp. 1137-1149, 2017, DOI: 10.1109/TPAMI.2016.2577031.
  13. J. Redmon and A. Farhadi, "YOLO9000: better, faster, stronger," 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, pp. 7263-7271, 2017, DOI: 10.1109/CVPR.2017.690.
  14. K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, pp. 770-778, DOI: 10.1109/CVPR.2016.90.
  15. G. Welch and G. Bishop, "The Discrete Kalman Filter," An Introduction to the Kalman Filter, University of North Carolina at Chapel Hill, 1995, pp. 2-7, [Online], https://courses.cs.washington.edu/courses/cse571/03wi/notes/welch-bishop-tutorial.pdf
  16. S. Thrun, "Probabilistic robotics," Commun. ACM, vol. 45, no. 3, pp. 52-57, Mar., 2002, DOI: 10.1145/504729.504754.
  17. J. Redmon, Darknet: Open source neural networks in c, 2013, [Online], https://pjreddie.com/darknet.