Journal of Korea Multimedia Society (한국멀티미디어학회논문지)

Korea Multimedia Society (한국멀티미디어학회)
권호 표지
DOI QR코드

DOI QR Code

Multi-modal Sensor System and Database for Human Detection and Activity Learning of Robot in Outdoor

실외에서 로봇의 인간 탐지 및 행위 학습을 위한 멀티모달센서 시스템 및 데이터베이스 구축

  • Uhm, Taeyoung (Field Robotics R&D Division, Korean Institute of Robot and Convergence) ;
  • Park, Jeong-Woo (Field Robotics R&D Division, Korean Institute of Robot and Convergence) ;
  • Lee, Jong-Deuk (Field Robotics R&D Division, Korean Institute of Robot and Convergence) ;
  • Bae, Gi-Deok (Field Robotics R&D Division, Korean Institute of Robot and Convergence) ;
  • Choi, Young-Ho (Field Robotics R&D Division, Korean Institute of Robot and Convergence)
  • Received : 2018.10.30
  • Accepted : 2018.12.10
  • Published : 2018.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

Robots which detect human and recognize action are important factors for human interaction, and many researches have been conducted. Recently, deep learning technology has developed and learning based robot's technology is a major research area. These studies require a database to learn and evaluate for intelligent human perception. In this paper, we propose a multi-modal sensor-based image database condition considering the security task by analyzing the image database to detect the person in the outdoor environment and to recognize the behavior during the running of the robot.

Keywords

MTMDCW_2018_v21n12_1459_f0001.png 이미지

Fig. 1. Multi-modal sensor system configuration.

MTMDCW_2018_v21n12_1459_f0002.png 이미지

Fig. 2. Multi-modal sensor module interface.

MTMDCW_2018_v21n12_1459_f0003.png 이미지

Fig. 3. Multi-modal sensor module Head design.

MTMDCW_2018_v21n12_1459_f0004.png 이미지

Fig. 4. Wire based vibration damper.

MTMDCW_2018_v21n12_1459_f0005.png 이미지

Fig. 5. Wire based vibration damper.

MTMDCW_2018_v21n12_1459_f0006.png 이미지

Fig. 6. The muliti-modal sensor module mounted on robot platform.

MTMDCW_2018_v21n12_1459_f0007.png 이미지

Fig. 7. An example of mulit-modal sensor data.

MTMDCW_2018_v21n12_1459_f0008.png 이미지

Fig. 8. Result of vibration damping. (a) Original, (b) Proposed method, (c) Experimental Platform

MTMDCW_2018_v21n12_1459_f0009.png 이미지

Fig. 9. Take 1: Day and Night time database in the lawn playground.

MTMDCW_2018_v21n12_1459_f0010.png 이미지

Fig. 10. Take 2: Lying database in the driveway.

MTMDCW_2018_v21n12_1459_f0011.png 이미지

Fig. 11. Take 3: Heavy rain and thunder in the driveway.

MTMDCW_2018_v21n12_1459_f0012.png 이미지

Fig. 12. Take 4: Daytime in the park.

Table 1. The behavior definition in normal and abnormal situations

MTMDCW_2018_v21n12_1459_t0001.png 이미지

Table 2. The database configuration

MTMDCW_2018_v21n12_1459_t0002.png 이미지

References

  1. PETS, https://motchallenge.net/workshops/bmtt-pets2017/ (accessed June, 12, 2018).
  2. BEHAVE, http://groups.inf.ed.ac.uk/vision/BEHAVEDATA/INTERACTIONS/ (accessed June, 22, 2018).
  3. i-Lids, www.ilids.co.uk. (accessed July, 2, 2018).
  4. ViSOR, http://imagelab.ing.unimore.it/visor/ (accessed July, 9, 2018).
  5. G. Moon and J. Rue, "Remote Person Recognition Test Database for Intelligent Video Surveillance," Journal of Information Security, Vol. 22, No. 4, pp. 38-45, 2012.
  6. R. Fenrich and J.J. Hull, "Concern in Creation of Image Database," Proceedings of Third International Workshop on Frontiers in Handwriting Recognition, pp. 112-121, 1993.
  7. B. Zhou, A. Lapedriza, A. Khosla, A. Oliva, and A. Torralba, “Places: A 10 Million Image Database for Scene Recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 40, No. 6, pp. 1452-1464, 2018. https://doi.org/10.1109/TPAMI.2017.2723009
  8. S. Bae, H. Lee, and D. Cho, "Design and Implementation of a Web Crawler System for Collection of Structured and Unstructured Data," Journal of Korea Multimedia Society, Vol. 21, No. 2, pp. 199-209. 2018. https://doi.org/10.9717/KMMS.2018.21.2.199
  9. T. Uhm, G. Bae, J. Lee, and Y. Choi, "Multi-modal Sensor Calibration Method for Intelligent Unmanned Outdoor Security Robot," Proceedings of the Sixth Intercational Conference on Green and Human Information Technology, pp. 215-220, 2018.