The Journal of Korean Institute of Communications and Information Sciences (한국통신학회논문지)

The Korean Institute of Commucations and Information Sciences (한국통신학회)
권호 표지
DOI QR코드

DOI QR Code

Stereoscopic Video Compositing with a DSLR and Depth Information by Kinect

키넥트 깊이 정보와 DSLR을 이용한 스테레오스코픽 비디오 합성

  • 권순철 (광운대학교 정보콘텐츠대학원 디지털3D연구실) ;
  • 강원영 (광운대학교 정보콘텐츠대학원 디지털3D연구실) ;
  • 정영후 (광운대학교 정보콘텐츠대학원 디지털3D연구실) ;
  • 이승현 (광운대학교 정보콘텐츠대학원 디지털3D연구실)
  • Received : 2013.07.31
  • Accepted : 2013.10.08
  • Published : 2013.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

Chroma key technique which composes images by separating an object from its background in specific color has restrictions on color and space. Especially, unlike general chroma key technique, image composition for stereo 3D display requires natural image composition method in 3D space. The thesis attempted to compose images in 3D space using depth keying method which uses high resolution depth information. High resolution depth map was obtained through camera calibration between the DSLR and Kinect sensor. 3D mesh model was created by the high resolution depth information and mapped with RGB color value. Object was converted into point cloud type in 3D space after separating it from its background according to depth information. The image in which 3D virtual background and object are composed obtained and played stereo 3D images using a virtual camera.

크로마키 방식에 의한 영상 합성은 색상 정보에 의해 전경 후경을 분리하기 때문에 객체 색상의 제약과 특정 스크린이 위치해 있어야 하는 공간의 제약이 있다. 특히 스테레오스코픽 3D 디스플레이를 위한 영상 합성은 크로마키 방식과는 달리 3D 공간에서의 자연스러운 영상 합성이 요구된다. 본 논문에서는 고해상도의 깊이 정보를 이용하여 깊이 키잉(depth keying) 방식에 의한 3D 공간에서의 스테레오스코픽 영상 합성을 제안하였다. 이를 위해 DSLR과 마이크로소프트사 키넥트 센서간의 카메라 캘리브레이션을 통해 고화질의 깊이 정보 획득 후 RGB 정보와의 정합 과정을 통해 3차원 데이터를 획득하였다. 깊이 정보에 의해 배경과 분리 된 객체는 3차원 공간에서의 포인트 클라우드 형태로 표현되어 가상 배경과 합성하였다. 이후 가상의 스테레오 카메라에 의해 Full HD 스테레오스코픽 비디오 합성 영상 획득 결과를 보였다.

Keywords

References

  1. B. Julesz, "Binocular depth perception of computer generated images," Bell Syst. Tech. J., vol. 39, no. 5, pp. 1125-1163, Sep. 1960. https://doi.org/10.1002/j.1538-7305.1960.tb03954.x
  2. A. Smolic, P. Kauff, S. Knorr, A. Hornung, M. Kunter, M. Mueller, and M. Lang, "Three-dimensional video postproduction and processing," Proc. IEEE, vol. 99, no. 4, pp. 607-625, Apr. 2011. https://doi.org/10.1109/JPROC.2010.2098350
  3. K. Fukui, M. Hayashi, and Y. Yamanouchi, "A virtual studio system for TV program production," Soc. Motion Picture, Television Eng. (SMPTE) J., vol. 103, no. 6, pp. 386-390, June 1994.
  4. G. J. Iddan and G. Yahav, "3D imaging in the studio," Proc. SPIE, vol. 4298, pp. 48-55, Apr. 2001.
  5. S. C. Kwon, S. J. Lee, K. C. Son, Y. H. Jeong, and S. H. Lee, "High resolution 3D object generation with a DSLR and depth information by Kinect," J. Korean Soc. Computer Game, vol. 26, no. 1, pp. 221-227, Mar. 2013.
  6. J. Park, H. Kim, Y.-W. Tai, M. S. Brown, and I. Kweon, "High quality depth map upsampling for 3D-TOF cameras," in Proc. IEEE Int. Conf. Computer Vision (ICCV), pp. 1623-1630, Barcelona, Spain, Nov. 2011.
  7. K. Khoshelham and S. O. Elberink, "Accuracy and resolution of kinect depth data for indoor mapping applications," Sensor, vol. 12, no. 2, pp. 1437-1454, Feb. 2012. https://doi.org/10.3390/s120201437
  8. R. A. Hamzah and S. I. Salim, "Software calibration for stereo camera on stereo vision mobile robot using Tsai's method," Int. J. Computer Theory Eng. (IJCTE), vol. 2, no. 3, pp. 390-394, June 2010.
  9. H. Shin, S. Kim, and K. Sohn, "Hybrid stereoscopic camera system," J. Korean Soc. Broadcast Eng. (KOSBE), vol. 16, no. 4, pp. 602-613, July 2011. https://doi.org/10.5909/JEB.2011.16.4.602
  10. C. Lee, H. Song, B. Choi, and Y.-S. Ho, "Multi-view generation using high resolution stereoscopic cameras and a low resolution Time-of-Flight camera," J. Korea Inst. Commun. Inform Sci. (KICS), vol. 37A, no. 4, pp. 239-249, Apr. 2012. https://doi.org/10.7840/KICS.2012.37A.4.239
  11. J. Zhang, L.-H. Wang, D.-X. Li, and M. Zhang, "High quality depth maps from stereo matching and ToF camera," in Proc. Int. Conf. Soft Comput. Pattern Recognition (SoCPaR), pp. 68-72, Dalian, China, Oct. 2011.
  12. J. Kopf, M. Cohen, D. Lischinski, and M. Uyttendaele, "Joint bilateral upsampling," ACM Trans. Graphics, vol. 26, no. 3, Article no. 96, July 2007.
  13. G.-C. Lee and J. Yoo, "Real-time virtual-view image synthesis algorithm using Kinect camera," J. Korean Inst. Commun. Inform. Sci. (KICS), vol. 38c, no. 5, pp. 409-419, May 2013. https://doi.org/10.7840/kics.2013.38C.5.409
  14. B. Mediburu, 3D Movie Making, Focal Press, 2009.
  15. A. R. Smith and J. F. Blinn, "Blue screen matting," in Proc. SIGGRAPH '96, pp. 259-268, New Orleans, U.S.A., Aug. 1996.
  16. M. Ben-Ezra, "Segmentation with invisible keying signal," in Proc. IEEE Conf. Computer Vision Pattern Recognition, vol. 1, pp. 32-37, Hilton Head Island, U.S.A., June 2000.
  17. I. Schiller and R. Koch, "Improved video segmentation by adaptive combination of depth keying and Mixture-of-Gaussians," Lecture Notes in Computer Sci., Image Analysis, vol. 6688, pp. 59-68, May 2011.
  18. S.-C. Kwon, J.-H. Kim, K.-C. Son, and A. Hamacher, "Acquisition of stereo composite images by depth keying in three dimensional space," J. Korean Soc. Computer Game, vol. 26, no. 2, pp. 139-145, June 2013.