The Journal of Korea Robotics Society (로봇학회논문지)

Korea Robotics Society (한국로봇학회)
권호 표지
DOI QR코드

DOI QR Code

Robust Vehicle Occupant Detection based on RGB-Depth-Thermal Camera

다양한 환경에서 강건한 RGB-Depth-Thermal 카메라 기반의 차량 탑승자 점유 검출

  • Song, Changho (Intelligent Robotics Research Center, Korea Electronics Technology Institute(KETI)) ;
  • Kim, Seung-Hun (Intelligent Robotics Research Center, Korea Electronics Technology Institute(KETI))
  • Received : 2018.01.18
  • Accepted : 2018.02.09
  • Published : 2018.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, the safety in vehicle also has become a hot topic as self-driving car is developed. In passive safety systems such as airbags and seat belts, the system is being changed into an active system that actively grasps the status and behavior of the passengers including the driver to mitigate the risk. Furthermore, it is expected that it will be possible to provide customized services such as seat deformation, air conditioning operation and D.W.D (Distraction While Driving) warning suitable for the passenger by using occupant information. In this paper, we propose robust vehicle occupant detection algorithm based on RGB-Depth-Thermal camera for obtaining the passengers information. The RGB-Depth-Thermal camera sensor system was configured to be robust against various environment. Also, one of the deep learning algorithms, OpenPose, was used for occupant detection. This algorithm is advantageous not only for RGB image but also for thermal image even using existing learned model. The algorithm will be supplemented to acquire high level information such as passenger attitude detection and face recognition mentioned in the introduction and provide customized active convenience service.

Keywords

References

  1. N. Kim, Y Chio, S Hwang, K Park, J. S. Yoon, and I. S. Kweon, "Geometrical calibration of multispectral calibration," 2015 12th International Conference on Ubiquitous Robots and Ambient Intelligence, Goyang, South Korea, pp. 384-385, 2015.
  2. C. Palmero, A. Clapes, C. Bahnsen, A. Mogelmose, T. B. Moeslund, and S. Escalera, "Multi-modal RGB-depth-thermal human body segmentation," International Journal of Computer Vision, vol.118, no. 2, pp. 217-239, June, 2016. https://doi.org/10.1007/s11263-016-0901-x
  3. K. He, G. Gkioxari, P. Dollár, and R. Girshick, "Mask R-CNN," 2017 IEEE International Conference on Computer Vision, Venice, Italy, pp. 2980-2988, 2017.
  4. Z. Cao, T. Simon, S. Wei and Y. Sheikh, "Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields," 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 2017, doi: 10.1109/CVPR.2017.143.
  5. C. Song and S. Kim, "A study of an effective camera calibration and image registration method for RGB-Depth-Thermal image fusion," KSPE Autumn Conference, Gwangju, Korea, pp. 236-237, 2017.
  6. S. Vidas, R. Lakemond, S. Denman, C. Fookes, S. Sridharan, and T. Wakr, "A mask-based approach for the geometric calibration of thermal-infrared cameras," IEEE Transactions on Instrumentation and Measurement, vol. 61, no. 6, pp. 1625-1635, 2012. https://doi.org/10.1109/TIM.2012.2182851
  7. W. Usrine, F. Calado, G. Teixeira, H. Diniz, S. Silvino, and R. de Andrade, "Thermal/visible autonomous stereo visio system calibration methodology for non-controlled environments," 11th International Conference on Quantitative InfraRed Thermography, Naples, Italy, 2012, doi: 10.21611/qirt.2012.261.
  8. R. Girshick, F. Iandola, T. Darrell, and J. Malik, "Deformable part models are convolutional neural networks," 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, pp. 437-446, 2015.
  9. A. Newell, K. Yang, and J. Deng, "Stacked hourglass networks for human pose estimation," European Conference on Computer Vision, Amsterdam, The Netherlands, pp. 483-499, 2016.
  10. L. Pishchulin, E. Insafutdinov, S. Tang, B. Andres, M. Andriluka, P. Gehler, and B. Schiele, "Deepcut: Joint subset partition and labeling for multi person pose estimation," 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 2016, doi: 10.1109/CVPR.2016.533.

Cited by

  1. SPAD과 CNN의 특성을 반영한 ToF 센서와 스테레오 카메라 융합 시스템 vol.13, pp.4, 2018, https://doi.org/10.7746/jkros.2018.13.4.230