Journal of Bio-Environment Control (생물환경조절학회지)

The Korean Society for Bio-Environment Control (한국생물환경조절학회)
권호 표지
DOI QR코드

DOI QR Code

Grading of Harvested 'Mihwang' Peach Maturity with Convolutional Neural Network

합성곱 신경망을 이용한 '미황' 복숭아 과실의 성숙도 분류

  • Shin, Mi Hee (Institute of Agriculture and Life Sciences, Gyeongsang National University) ;
  • Jang, Kyeong Eun (Division of Applied Life Science, Graduate School of Gyeongsang National University) ;
  • Lee, Seul Ki (Fruit Research Division, National Institute of Horticultural and Herbal Science) ;
  • Cho, Jung Gun (Fruit Research Division, National Institute of Horticultural and Herbal Science) ;
  • Song, Sang Jun (Farm&Farm Soft) ;
  • Kim, Jin Gook (Department of Horticulture, College of Agriculture and Life Science, Gyeongsang National University)
  • 신미희 (경상국립대학교 농업생명과학연구원) ;
  • 장경은 (경상국립대학교 대학원 응용생명과학부) ;
  • 이슬기 (국립원예특작과학원 과수과) ;
  • 조정건 (국립원예특작과학원 과수과) ;
  • 송상준 (팜앤팜소프트) ;
  • 김진국 (경상국립대학교 원예학과)
  • Received : 2022.07.04
  • Accepted : 2022.09.10
  • Published : 2022.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

This study was conducted using deep learning technology to classify for 'Mihwang' peach maturity with RGB images and fruit quality attributes during fruit development and maturation periods. The 730 images of peach were used in the training data set and validation data set at a ratio of 8:2. The remains of 170 images were used to test the deep learning models. In this study, among the fruit quality attributes, firmness, Hue value, and a* value were adapted to the index with maturity classification, such as immature, mature, and over mature fruit. This study used the CNN (Convolutional Neural Networks) models for image classification; VGG16 and InceptionV3 of GoogLeNet. The performance results show 87.1% and 83.6% with Hue left value in VGG16 and InceptionV3, respectively. In contrast, the performance results show 72.2% and 76.9% with firmness in VGG16 and InceptionV3, respectively. The loss rate shows 54.3% and 62.1% with firmness in VGG16 and InceptionV3, respectively. It considers increasing for adapting a field utilization with firmness index in peach.

본 연구는 무대재배 복숭아 '미황'을 대상으로 성숙기간 중 RGB 영상을 취득한 후 다양한 품질 지표를 측정하고 이를 딥러닝 기술에 적용하여 복숭아 과실 숙도 분류의 가능성을 탐색하고자 실시하였다. 취득 영상 730개의 데이터를 training과 validation에 사용하였고, 170개는 최종테스트 이미지로 사용하였다. 본 연구에서는 딥러닝을 활용한 성숙도 자동 분류를 위하여 조사된 품질 지표 중 경도, Hue 값, a*값을 최종 선발하여 이미지를 수동으로 미성숙(immature), 성숙(mature), 과숙(over mature)으로 분류하였다. 이미지 자동 분류는 CNN(Convolutional Neural Networks, 컨볼루션 신경망) 모델 중에서 이미지 분류 및 탐지에서 우수한 성능을 보이고 있는 VGG16, GoogLeNet의 InceptionV3 두종류의 모델을 사용하여 복숭아 품질 지표 값의 분류 이미지별 성능을 측정하였다. 딥러닝을 통한 성숙도 이미지 분석 결과, VGG16과 InceptionV3 모델에서 Hue_left 특성이 각각 87.1%, 83.6%의 성능(F1 기준)을 나타냈고, 그에 비해 Firmness 특성이 각각 72.2%, 76.9%를 나타냈고, Loss율이 각각 54.3%, 62.1%로 Firmness를 기준으로 한 성숙도 분류는 적용성이 낮음을 확인하였다. 추후에 더 많은 종류의 이미지와 다양한 품질 지표를 가지고 학습이 진행된다면 이전 연구보다 향상된 정확도와 세밀한 성숙도 판별이 가능할 것으로 판단되었다.

Keywords

Acknowledgement

이 연구는 2021년도 경상국립대학교 연구년제연구교수 연구지원비에 의하여 수행되었음. 본 논문은 농촌진흥청 공동연구사업(과제번호: PJ015646032022)의 지원에 의해 이루어진 것임.

References

  1. Bang J.H., J. Park, S.W. Park, J.Y. Kim, S.H. Jung, and C.B. Sim 2022, A system for determining the growth stage of fruit tree using a deep learning-based object detection model. Smart Media J 11:9-18. (in Korean) doi:10.30693/SMJ.2022.11.4.9
  2. Behera S.K., A.K. Rath, and P.K. Sethy 2021, Maturity status classification of papaya fruits based on machine learning and transfer learning approach. Inf Process Agric 8:244-250. doi:10.1016/j.inpa.2020.05.003
  3. Bhole V., and A. Kumar 2020, Mango quality grading using deep learning technique: Perspectives from agriculture and food industry. Proceedings of the 21st Annual Conference on Information Technology Education, pp 180-186. doi:10.1145/3368308.3415370
  4. Chun J.P., and J.G. Kim 2013, Changes of quality indices and cell wall components during ripening and softening in peach fruit (Prunus persica L. Batsch). J Agric Life Sci 47:35-42. (in Korean)
  5. Gong L., W. Wang, T. Wang, and C. Liu 2022, Robotic harvesting of the occluded fruits with a precise shape and position reconstruction approach. J Field Robotics 39:69-84. doi:10.1002/rob.22041
  6. Ham H.S., and H.C. Cho 2020, A development of rose leaf disease classification system using convolutional neural network. Trans Korean Inst Electr Eng 69:1040-1045. (in Korean) doi:10.5370/KIEE.2020.69.7.1040
  7. Jeong S.H., M.H. Lee, and H. Yoe 2018, Fruit classification system using deep learning. J Knowl Inf Technol Syst 13:589-595. (in Korean) doi:10.34163/jkits.2018.13.5.009
  8. Kim J.K., W. Cho, M.H. Na, and M. Chun 2019, Development of automatic classification system of vegetables by image processing and deep learning. J Korean Data Anal Soc 21: 63-73. (in Korean) doi:10.37727/jkdas.2019.21.1.63
  9. Kim M., and C. Choi 2020, Superpixel-based apple leaf disease classification using convolutional neural network. J Broadcast Eng 25:208-217. (in Korean) doi:10.5909/JBE.2020.25.2.208
  10. Kingma D.P., and J. Ba 2015, Adam: A method for stochastic optimization. 3rd International Conference on Learning Representations. doi:10.48550/arXiv.1412.6980
  11. LeCun Y., B. Boser, J.S. Denker, D. Henderson, R.E. Howard, W. Hubbard, and L.D. Jackel 1989, Backpropagation applied to handwritten zip code recognition. Neural Comput 1:541-551. doi:10.1162/neco.1989.1.4.541
  12. Li X., C. Cai, H. Zheng, and H. Zhu 2022, Recognizing strawberry appearance quality using different combinations of deep feature and classifiers. J Food Process Eng 45:e13982. doi:10.1111/jfpe.13982
  13. Lim S.C., S.H. Kim, Y.H. Kim, and D.Y. Kim 2017, Training network design based on convolution neural network for object classification in few class problem. J Korea Inst Inf Commun Eng 21:144-150. (in Korean) doi:10.6109/jkiice.2017.21.1.144
  14. Mazen F.M.A., and A.A. Nashat 2019, Ripeness classification of bananas using an artificial neural network. Arab J Sci Eng 44:6901-6910. doi:10.1007/s13369-018-03695-5
  15. Moon T., J. Park, and J.E. Son 2020, Estimation of sweet pepper crop fresh weight with convolutional neural network. Protected Hort Plant Fac 29:381-387. (in Korean) doi:10.12791/KSBEC.2020.29.4.381
  16. Park J.Y., H.J. Kim, and K. Kim 2020, Accessing impact of DCGAN image data augmentation for CNN based tomato disease classification. J Digital Cont Soc 21:959-967. (in Korean) doi:10.9728/dcs.2020.21.5.959
  17. Saleem M.H., J. Potgieter, and K.M. Arif 2021, Automation in agriculture by machine and deep learning techniques: A review of recent developments. Precis Agric 22:2053-2091 doi:10.1007/s11119-021-09806-x
  18. Seo D., K.C. Kim, M. Lee, K.D. Kwon, and G. Kim 2021, Research on tomato flowers and fruits object detection model in greenhouse environment using deep learning. J Korean Inst Commun Inf Sci 46:2072-2077. (in Korean) doi:10.7840/kics.2021.46.11.2072
  19. Silalahi D.D., C.E. Reano, F.P. Lansigan, R.G. Panopio, and N.C. Bantayan 2016, Using genetic algorithm neural network on near infrared spectral data for ripeness grading of oil palm fresh fruit. Inf Process Agric 3:252-261. doi:10.1016/j.inpa.2016.10.001
  20. Stasenko N., E. Chernova, D. Shadrin, G. Ovchinnikov, L. Krivolapov, and M. Pukalchik 2021, Deep learning for improving the storage process: Accurate and automatic segmentation of spoiled areas on apples. 2021 IEEE Instrumentation and Measurement Technology Conference 1-6. doi:10.1109/I2MTC50364.2021.9460071
  21. Tan K., W.S. Lee, H. Gan, and S. Wang 2018, Recognising blueberry fruit of different maturity using histogram oriented gradients and colour features in outdoor scenes. Biosyst Eng 176:59-72. doi:10.1016/j.biosystemseng.2018.08.011
  22. Wang D., W. Cao, F. Zhang, Z. Li, S. Xu, and X. Wu 2022, A review of deep learning in multiscale agricultural sensing. Remote Sens 14:559. doi:10.3390/rs14030559
  23. Xu M., J. Sun, K. Yao, Q. Cai, J. Shen, Y. Tian, and X Zhou 2022, Developing deep learning based regression approaches for prediction of firmness and pH in Kyoho grape using Vis/NIR hyperspectral imaging. Infrared Phys Technol 120:104003. doi:10.1016/j.infrared.2021.104003
  24. Yu X., H. Lu, and D. Wu 2018, Development of deep learning method for predicting firmness and soluble solid content of postharvest Korla fragrant pear using Vis/NIR hyperspectral reflectance imaging. Postharvest Biol Technol 141:39-49. doi:10.1016/j.postharvbio.2018.02.013