그림 1. 카메라와 라이다 부착 위치 Fig. 1. Camera and LiDAR sensor mount position
그림 2. CNN과 헝가리안 알고리즘을 이용한 객체 추적 프로세스[14] Fig. 2. Object tracking process using CNN with Hungarian data association
그림 3. 다양한 센서 융합 방법 Fig. 3. Architectures of different sensor fusion schemes
그림 4. 제안하는 센서 융합 시스템의 개요 Fig. 4. Overview of the proposed fusion system
그림 5. 행렬 계산에 사용되는 센서 실측값 측정 예시 Fig. 5. Example of measuring sensor values used in matrix calculations
그림 6. 이미지상의 거리 추정과 보정 Fig. 6. Distance estimation and correction on images
그림 7. 라이다를 이용한 객체 검출 과정 Fig. 7. Object detection process using LiDAR
그림 8. 차량 위치에 따른 point cloud의 배열 Fig. 8. Arrangement of point clouds according to vehicle location
그림 9. 월드 좌표에서의 거리와 각도 정의 Fig. 9. Define distance and angle in world coordinates
그림 10. 라이다를 이용하여 검출된 객체를 이미지 평면에 옮기는 방법 Fig. 10. Projection detected object to the image plane using LiDAR
그림 11. 로지스틱 회귀에 사용된 layers Fig. 11. Layers used for logistic regression
그림 12. 카메라 객체 검출 (좌). 라이다 객체 검출 (중), 센서 융합 결과 (우) Fig. 12. Camera object detection (left), LiDAR object detection (middle), sensor fusion result (right)
그림 13. 객체 검출이 잘 되지 않은 경우 Fig. 13. Case of miss detection
표 1. 주변 환경별 센서의 성능 비교 Table 1. Comparison of sensor performance by environment
표 2. 카메라와 라이다 객체 검출과 융합의 성능 분석 Table 2. Performance analysis of camera and LiDAR detection and fusion
References
- Robert J. Wang, Xiang Li and Charles X. Ling,"Pelee: A Real-Time Object Detection System on Mobile Devices", Advances in neural information processing system, Montreal, Canada, pp.1963-1972, 2018.
- Tsung-Yi Lin, Priya Goyal and Ross Girshick, et al.,"Focal Loss for Dense Object Detection", Proceedings of the IEEE international conference on computer vision, Venice, Italy, pp.2980-2988, 2017.
- Joseph Redmon and Ali Farhadi, "YOLOv3: An Incremental Improvement", arXiv preprint arXiv:1804.02767, 2018.
- Mark Sandler, Andrew Howard and Menglong Zhu, et al., "Mobile NetV2: Inverted Residuals and Linear Bottlenecks", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Utah, pp.4510-4520, 2018.
- Yuan Wang, Tianyue Shi and Peng Yun, et al.,"PointSeg: Real-Time Semantic Segmentation Based on 3D LiDAR Point Cloud", arXiv preprint arXiv:1807.06288, 2018.
- Bin Yang, Wenjie Luo and Raquel Urtasum,"PIXOR: Real-time 3D Object Detection from Point Clouds", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Utah, pp.7652-7660, 2018.
- Ming Liang, Bin Yang and Shenlong Wang, et al.,"Deep Continuous Fusion for Multi-Sensor 3D Object Detection", Proceedings of the European Conference on Computer Vision, Munich, Germany, pp.641-656, 2018.
- Xiaozhi Chen, Huimin Ma, Ji Wan, et al.,"Multi-View 3D Object Detection Network for Autonomous Driving", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Hawaii, pp.1907-1915, 2017
- Pan Wei, Lucas Cagle and Tasmia Reza, et al.,"LiDAR and Camera Detection Fusion in a Real-Time Industrial Multi-Sensor Collision Avoidance System", Electronics, Vol.7, No.84, 2018
- Wei Liu, Dragomir Anguelov and Dumiru Erhan, et al.,"SSD: Single Shot MultiBox Detector", Proceedings of the European Conference on Computer Vision, Amsterdam, Netherlands, pp.21-37, 2016.
- Ross Girshick, Jeff Donahue, Trevor Darrell, et al.,"Rich feature hierarchies for accurate object detection and semantic segmentation", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, Ohio, pp.580-587, 2014.
- Ross Girshick, "Fast R-CNN", Proceedings of the IEEE international conference on computer vision, Santiago, Chile, pp.1440-1448, 2015.
- Shaoqing Ren, Kaiming He, Ross Girshick, et al.,"Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks", Advances in neural information processing system, Montreal, Canada, pp.91-99, 2015.
- Liu Mingjie, Cheng Bin Jin and Xuenan Cui, et al.,"Online multiple object tracking using confidence score-based appearance model learning and hierarchical data association", IET Computer Vision, Vol.13, No.3, pp.312-318, April 2019. https://doi.org/10.1049/iet-cvi.2018.5499
- Morgan Quigley, Brian Gerkey and Ken Conley, et al.,"ROS: an open-source Robot Operating System", ICRA workshop on open source software, Vol.3, No.3.2, p.5, 2009.