The Journal of Korea Robotics Society (로봇학회논문지)

Korea Robotics Society (한국로봇학회)
권호 표지
DOI QR코드

DOI QR Code

A Study on the Training Methodology of Combining Infrared Image Data for Improving Place Classification Accuracy of Military Robots

군 로봇의 장소 분류 정확도 향상을 위한 적외선 이미지 데이터 결합 학습 방법 연구

  • Received : 2023.05.31
  • Accepted : 2023.06.28
  • Published : 2023.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

The military is facing a continuous decrease in personnel, and in order to cope with potential accidents and challenges in operations, efforts are being made to reduce the direct involvement of personnel by utilizing the latest technologies. Recently, the use of various sensors related to Manned-Unmanned Teaming and artificial intelligence technologies has gained attention, emphasizing the need for flexible utilization methods. In this paper, we propose four dataset construction methods that can be used for effective training of robots that can be deployed in military operations, utilizing not only RGB image data but also data acquired from IR image sensors. Since there is no publicly available dataset that combines RGB and IR image data, we directly acquired the dataset within buildings. The input values were constructed by combining RGB and IR image sensor data, taking into account the field of view, resolution, and channel values of both sensors. We compared the proposed method with conventional RGB image data classification training using the same learning model. By employing the proposed image data fusion method, we observed improved stability in training loss and approximately 3% higher accuracy.

Keywords

Acknowledgement

This work was supported by Korea Research Institute for Defense Technology planning and advancement (KRIT) grant funded by Korea government DAPA (Defense Acquisition Program Administration) (No. KRIT-CT-22-006-002, Development of the situation/environment recognition technology for micro-swarm robot)

References

  1. Ministry of National Defense, 2018 Defense white paper, 2019, [Online], https://www.mnd.go.kr/cop/pblictn/selectPublicationUser.do?siteId=mndEN&componentId=51&categoryId=0&publicationSeq=846&pageIndex=1&id=mndEN_031300000000, Accessed: May. 9, 2023.
  2. "Military innovation 4.0," Ministry of National Defense, [Online], https://www.mnd.go.kr/mbshome/mbs/mnd/subview.jsp?id=mnd_010302010000, Accessed: Mar. 3, 2023.
  3. K. S. Jang and Y. K. Cheung, "A study on the method acquiring NCOs according to the decrease in school age population-Activation of the department of navy noncommissioned officers," Journal of Information and Security, vol. 21, no. 1, pp. 159-168, Mar., 2021, DOI: 10.33778/kcsa.2021.21.1.159.
  4. C. Grandou, L. Wallace, H. H. K. Fullagar, R. Duffield, and S. Burley, "The Effects of Sleep Loss on Military Physical Performance," Sports Med, vol. 49, pp. 1159-1172, May., 2019, DOI: 10.1007/s40279-019-01123-8.
  5. C. H. Good, A. J. Brager, V. F. Capaldi, and V. Mysliwiec, "Sleep in the United States Military," Neuropsychopharmacology, vol. 45, pp. 176-191, Jun., 2020, DOI: 10.1038/s41386-019-0431-7.
  6. E. S. Kim, "Application of AI in Defense, U.S. Case Studies, and Key Considerations to Know," KIDA Defense Issues & Analysis, vol. 20, no. 18, pp. 1-9, May., 2020, [Online], https://kida.re.kr/cmm/viewBoardImageFile.do?idx=28058.
  7. Boannews, "CCTV Cameras for Military Border Security, Available Products," [Online], https://www.boannews.com/media/view.asp?idx=101175, Accessed: Oct. 5, 2021.
  8. B. Zhou, A. Lapedriza, A. Khosla, A. Oliva, and A. Torralba, "Places: A 10 million image database for scene recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 40, no. 6, pp. 1452-1464, Jul., 2017, DOI: 10.1109/TPAMI.2017.2723009.
  9. AXIS Communications, IR used for surveillance (White paper), Jun., 2018.
  10. "How do thermal camera work?," FLIR Korea, [Online], https://www.flirkorea.com/discover/rd-science/how-do-thermal-cameras-work/, Accessed: Jun. 16, 2020.
  11. F. Farahnakian, J. Poikonen, M. Laurinen, and J. Heikkonen, "Deep Convolutional Neural Network-based Fusion of RGB and IR Images in Marine Environment," International Conference on Intelligent Transportation, Auckland, New Zealand, 2019, DOI: 10.1109/ITSC.2019.8917332.
  12. S. Speth, A. Goncalves, B. Rigault, S. Suzuki, M. Bouazizi, Y. Matsuo, and H. Predinger, "Deep learning with RGB and thermal images onboard a drone for monitoring operations," Journal of Field Robotics, vol. 39, no. 6, pp. 840-868, May., 2022, DOI: 10.1002/rob.22082.
  13. S. Dey, Hands-On Image Processing with Python, Sandipan Dey, 2018, [Online], https://www.oreilly.com/library/view/hands-onimage-processing/9781789343731/