The Journal of Society for e-Business Studies (한국전자거래학회지)

Society for e-Business Studies (한국전자거래학회)
권호 표지
DOI QR코드

DOI QR Code

Automatic Collection of Production Performance Data Based on Multi-Object Tracking Algorithms

다중 객체 추적 알고리즘을 이용한 가공품 흐름 정보 기반 생산 실적 데이터 자동 수집

  • Lim, Hyuna (Department of Industrial and Systems Engineering, Kyonggi University) ;
  • Oh, Seojeong (Department of Industrial and Systems Engineering, Kyonggi University) ;
  • Son, Hyeongjun (Department of Industrial and Systems Engineering, Kyonggi University) ;
  • Oh, Yosep (Department of Industrial and Systems Engineering, Kyonggi University)
  • Received : 2022.05.06
  • Accepted : 2022.05.19
  • Published : 2022.05.31
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, digital transformation in manufacturing has been accelerating. It results in that the data collection technologies from the shop-floor is becoming important. These approaches focus primarily on obtaining specific manufacturing data using various sensors and communication technologies. In order to expand the channel of field data collection, this study proposes a method to automatically collect manufacturing data based on vision-based artificial intelligence. This is to analyze real-time image information with the object detection and tracking technologies and to obtain manufacturing data. The research team collects object motion information for each frame by applying YOLO (You Only Look Once) and DeepSORT as object detection and tracking algorithms. Thereafter, the motion information is converted into two pieces of manufacturing data (production performance and time) through post-processing. A dynamically moving factory model is created to obtain training data for deep learning. In addition, operating scenarios are proposed to reproduce the shop-floor situation in the real world. The operating scenario assumes a flow-shop consisting of six facilities. As a result of collecting manufacturing data according to the operating scenarios, the accuracy was 96.3%.

최근 제조업에서의 디지털 전환이 가속화되고 있다. 이에 따라 사물인터넷(internet of things: IoT) 기반으로 현장 데이터를 수집하는 기술의 중요성이 증대되고 있다. 이러한 접근법들은 주로 각종 센서와 통신 기술을 활용하여 특정 제조 데이터를 확보하는 것에 초점을 맞춘다. 현장 데이터 수집의 채널을 확장하기 위해 본 연구는 비전(vision) 인공지능 기반으로 제조 데이터를 자동 수집하는 방법을 제안한다. 이는 실시간 영상 정보를 객체 탐지 및 추적 기술로 분석하고, 필요한 제조 데이터를 확보하는 것이다. 연구진은 객체 탐지 및 추적 알고리즘으로 YOLO(You Only Look Once)와 딥소트(DeepSORT)를 적용하여 프레임별 객체의 움직임 정보를 수집한다. 이후, 움직임 정보는 후보정을 통해 두 가지 제조 데이터(생산 실적, 생산 시간)로 변환된다. 딥러닝을 위한 학습 데이터를 확보하기 위해 동적으로 움직이는 공장 모형이 제작되었다. 또한, 실시간 영상 정보가 제조 데이터로 자동 변환되어 데이터베이스에 저장되는 상황을 재현하기 위해 운영 시나리오를 수립하였다. 운영 시나리오는 6개의 설비로 구성된 흐름 생산 공정(flow-shop)을 가정한다. 운영 시나리오에 따른 제조 데이터를 수집한 결과 96.3%의 정확도를 보였다.

Keywords

References

  1. Bae, G., Bae, S., Jeong, T., Heo, J., Moon, S., Song, S., Lee, S., Lee, J., Bae, Y., Jung, J., Na, H., Park, Y., Shin, G., Wang, J., and Kim,. B., "Smart Factory Management & Technology," Dreamdesign, 2019.
  2. Bewley, A., Ge, Z., Ott, L., Famos, F., and Upcroft, B., "Simple Online and Realtime Tracking," arXiv:1602.00763, 2016.
  3. Choi, W., and Cho, S., "Development of Mask Inspection Systems Based on Deep Learning," IE Magazine, Vol. 29, No. 1, pp. 42-44, 2022.
  4. Jeong, J., Kim, J., Chi, S., Roh, M., and Biggs, H., "Solitary Work Detection of Heavy Equipment Using Computer Vision," KSCE Journal of Civil and Environmental Engineering Research, Vol. 41, No. 4, pp. 441-447, 2021. https://doi.org/10.12652/KSCE.2021.41.4.0441
  5. Cho, C., Kim, J., Lee, S., Kim, B., Seo, Y., Park, Y., and Kwak, H.., "Vision-based Inspection Systems for Defect Detection," Proceedings of The Korean Society of Manufacturing Technology Engineers Conference, No. 12, pp 34-34, 2021.
  6. Jung, B., Seo, S., Park, B., and Bae, S., "Microscopic Traffic Parameters Estimation from UAV Video Using Multiple Object Tracking of Deep Learning-based," The Journal of The Korea Institute of Intelligent Transportation Systems, Vol. 20, No. 5, pp. 83-99, 2021. https://doi.org/10.12815/kits.2021.20.5.83
  7. Kim, J., and Jeong, J., "Smart Warehouse Management System Utilizing IoT-based Autonomous Mobile Robot for SME Manufacturing Factory," The Journal of the Institute of Internet, Broadcasting and Communication, Vol. 18, No. 5, pp. 237-244, 2018. https://doi.org/10.7236/JIIBC.2018.18.5.237
  8. KSX9101-1, Manufacturing Business Data Interchange Between Manufacturing Enterprise Business Systems - Part 1: Data Model, 2021.
  9. Lee, B., Liew, L., Cheach, W., and Wang, Y., "Occlusion handling in videos object tracking: A survey," IOP Conference Series Earth and Environmental Science, Vol. 18, 2014.
  10. Moon, S., Lee, J., Nam, D., and Kim, H., "A comparative study on multi object tracking methods for sports video," Proceedings of the Korea Information Processing Society Conference, Vol. 23, No. 2, pp. 653-654, 2016.
  11. Park, J., Hong, J., and Kim, W., "A Study on Intuitive IoT Interface System using 3D Depth Camera," The Journal of Society for e-Business Studies, Vol. 22, No. 2, pp. 137-152, 2017.
  12. Park, J., Park, D., Hyun, D., Na, Y., and Lee, S., "Deep-Learning Based Real-time Fire Detection Using Object Tracking Algorithm," Journal of the Korea Society of Computer and Information, Vol. 27, No. 1, pp. 1-8, 2022.
  13. Park, E., "Multi-Object Tracking Using Joint Detection and Identification Network based MLFPN," Hanyang University, MS Thesis, 2021.
  14. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A., "You Only Look Once: Unified, Real-Time Object Detection," arXiv:1506. 02640, 2016.
  15. Wojke, N., Bewley, A., and Paulus, D., "Simple Online and Realtime Tracking with a Deep Association Metric," arXiv:1703.07402, 2017.
  16. Yang, S., Jung, I., Kang, D., and Baek, H., "Real-Time Multi-Object Tracking using Mixture of SORT and DeepSORT," The Journal of Korean Institute of Information Technology, Vol. 19, No. 10, pp. 1-9, 2021.
  17. Yun, J., An, H., and Choi, Y., "A Machine Learning Based Facility Error Pattern Extraction Framework for Smart Manufacturing," The Journal of Society for e-Business Studies, Vol. 23, No. 2, pp. 97-110, 2018.