방송공학회논문지 (Journal of Broadcast Engineering)

  • 제26권3호
  • /
  • Pages.283-295
  • /
  • 2021
  • /
  • 1226-7953(pISSN)
  • /
  • 2287-9137(eISSN)
한국방송∙미디어공학회 (The Korean Institute of Broadcast and Media Engineers)
권호 표지
DOI QR코드

DOI QR Code

라이다 깊이 맵과 이미지를 사용한 자기 조직화 지도 기반의 고밀도 깊이 맵 생성 방법

Dense-Depth Map Estimation with LiDAR Depth Map and Optical Images based on Self-Organizing Map

  • Choi, Hansol (Dept. of Computer Engineering, Kwangwoon University) ;
  • Lee, Jongseok (Dept. of Computer Engineering, Kwangwoon University) ;
  • Sim, Donggyu (Dept. of Computer Engineering, Kwangwoon University)
  • 투고 : 2021.04.26
  • 심사 : 2021.05.14
  • 발행 : 2021.05.30
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

본 논문은 자기 조직화 지도 기법을 기반으로 라이다 기반으로 생성된 깊이 맵과 컬러 이미지의 정보를 기반으로 고밀도 깊이 맵을 생성하는 방법을 제안한다. 제안하는 깊이 맵 업샘플링 방법은 라이다에서 취득되지 않은 공간에 대한 초기 깊이 예측 단계와 초기 깊이 필터링 단계로 구성된다. 초기 깊이 예측 단계에서는 두 장의 컬러 이미지에 대해 스테레오 매칭을 수행하여 초기 깊이 값을 예측한다. 깊이 맵 필터링 단계에서는 예측된 초기 깊이 값의 오차를 감소시키고자 예측 깊이 픽셀에 대하여 주변의 실측 깊이 값을 이용하여 자기 조직화 지도 기법을 수행한다. 자기 조직화 기법 수행 시 예측 깊이 픽셀과 실측 깊이 픽셀의 거리와, 각 픽셀에 대응되는 컬러 값의 차이에 따라 가중치를 결정한다. 본 논문에서는 성능 비교를 위하여 깊이 맵 업샘플링 방법으로 널리 사용되고 있는 양방향 필터 및 k-최근접 이웃 알고리즘과 비교를 진행하였다. 제안하는 방법은 양방향 필터 방법 및 k-최근접 이웃 알고리즘 대비 MAE 관점에서 각각 약 6.4%, 8.6%이 감소하였고 RMSE 관점에서 각각 약 10.8%, 14.3%이 감소하였다.

This paper proposes a method for generating dense depth map using information of color images and depth map generated based on lidar based on self-organizing map. The proposed depth map upsampling method consists of an initial depth prediction step for an area that has not been acquired from LiDAR and an initial depth filtering step. In the initial depth prediction step, stereo matching is performed on two color images to predict an initial depth value. In the depth map filtering step, in order to reduce the error of the predicted initial depth value, a self-organizing map technique is performed on the predicted depth pixel by using the measured depth pixel around the predicted depth pixel. In the process of self-organization map, a weight is determined according to a difference between a distance between a predicted depth pixel and an measured depth pixel and a color value corresponding to each pixel. In this paper, we compared the proposed method with the bilateral filter and k-nearest neighbor widely used as a depth map upsampling method for performance comparison. Compared to the bilateral filter and the k-nearest neighbor, the proposed method reduced by about 6.4% and 8.6% in terms of MAE, and about 10.8% and 14.3% in terms of RMSE.

키워드

과제정보

이 기술은 과학기술정보통신부 및 정보통신기획평가원의 대학ICT연구센터육성지원사업(IITP-2021-2016-0-00288) 및 2021년도 광운대학교 우수연구자 지원 사업에 의한 연구결과로 개발한 기술입니다.

참고문헌

  1. L. Wang, M. Liao, M. Gong, R. Yang, and D. Nister, "High-quality real-time stereo using adaptive cost aggregation and dynamic programming," Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06), Chapel Hill, NC, USA, 14-16 June 2006.
  2. Q. Yang, L. Wang, R. Yang, H. Stewe'nius, and D. Niste'r, "Stereo matching with color-weighted correlation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no.3, pp. 492 - 504, Apr. 2008. https://doi.org/10.1109/TPAMI.2008.99
  3. K.-J. Yoon and I. S. Kweon, "Adaptive support-weight approach for correspondence search," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 4, pp. 650 - 656, Apr. 2006. https://doi.org/10.1109/TPAMI.2006.70
  4. J. L. Schonberger and J.-M. Frahm, "Structure-from-motion revisited," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27-30 June. 2016.
  5. S. B. Imandoust and M. Bolandraftar, "Application of K-Nearest Neighbor (KNN) Approach for Predicting Economic Events: Theoretical Background," International Journal of Engineering Research and Applications, vol. 3, no. 5, pp. 605-610, Sep. 2013.
  6. WG 07 MPEG 3D Graphics coding, "Text of ISO/IEC FDIS 23090-9 Geometry-based Point Cloud Compression," ISO/IEC JTC1/SC29/WG07 N00004, Online, Oct. 2020
  7. C. Premebida, L. Garrote, A. Asvadi, A. P. Ribeiro, and U. Nunes, "High-resolution LIDAR-based Depth Mapping using Bilateral Filter," 2016 IEEE 19th international conference on intelligent transportation systems (ITSC). IEEE, Rio de Janeiro, Brazil, 1-4 Nov. 2016.
  8. A. Hirata, R. Ishikawa, M. Roxas, and T. Oishi, "Real-Time Dense Depth Estimation Using Semantically-Guided LIDAR Data Propagation and Motion Stereo," IEEE Robotics and Automation Letters, vol. 4, no. 4, pp. 3806-3811, Oct. 2019. https://doi.org/10.1109/lra.2019.2927126
  9. N. Schneider, L. Schneider, P. Pinggera, U. Franke, M. Pollefeys, and C. Stiller, "Semantically Guided Depth Upsampling," German Conference on Pattern Recognition 2016, Hannover, Germany, 12-15 Sep. 2016.
  10. C. Tomasi and R. Manduchi, "Bilateral filtering for gray and color images," Sixth International Conference on Computer Vision, Bombay, India, India, 7-7 Jan. 1998.
  11. T. Kohonen, "The Self-organizing Map," Proceedings of the IEEE, vol. 78, no. 9, pp. 1464-1480, Sep. 2009. https://doi.org/10.1109/5.58325
  12. G. Bradski and A. Kaehler, Learning OpenCV: Computer vision with the OpenCV library, O'Reilly Media, Inc., 2008.
  13. H. Hirschmuller, "Stereo Processing by Semi-Global Matching and Mutual Information," IEEE Transactions on pattern analysis and machine intelligence, vol. 30, no. 2, pp. 328-341, Feb. 2008. https://doi.org/10.1109/TPAMI.2007.1166
  14. A. Geiger, P. Lenz, C. Stiller, and R. Urtasun, "Vision meets robotics:The kitti dataset," The International Journal of Robotics Research, vol. 32, no. 11, pp. 1231-1237, Aug. 2013. https://doi.org/10.1177/0278364913491297
  15. J. Jung, Y. S. Ho, "Depth Map Up-sampling Using Maximum Gradient of Color Image", The Korean Society Of Broad Engineers, pp. 29-30, Nov. 2012.
  16. S. Hong, Y. S. Ho, "Depth Upsampling Method Using Total Generalized Variation," JBE Vol. 21, No. 6, Nov. 2016.
  17. Y. Ko, H. C. Moon, H. H. Kim, J. G. Kim, "Single-Image Depth Estimation Based on CNN Using Edge Map," The Korean Society Of Broad Engineers, pp. 573-574, Jul. 2020.
  18. W. Lee, J. Lee, D. Sim, S. Oh, "A Deep Learning based Inter-Layer Reference Picture Generation Method for Improving SHVC Coding Performance," pp. 401-410, JBE Vol. 24, No. 3, May. 2019.
  19. J. Park, J. Lee, S. Park, D. Sim, "Super Resolution Using Gradient-SR," The Korean Society Of Broad Engineers, pp. 198-199, Jul. 2018
  20. T. Hui, C. Loy, and X. Tang. "Depth map super-resolution by deep multi-scale guidance," European conference on computer vision, pp. 353-369, Oct. 2016.