The Journal of Korean Institute of Communications and Information Sciences (한국통신학회논문지)

The Korean Institute of Commucations and Information Sciences (한국통신학회)
권호 표지
DOI QR코드

DOI QR Code

An Illumination-Insensitive Stereo Matching Scheme Based on Weighted Mutual Information

조명 변화에 강인한 상호 정보량 기반 스테레오 정합 기법

  • Heo, Yong Seok (Department of Electrical and Computer Engineering, Ajou University)
  • Received : 2015.07.31
  • Accepted : 2015.11.02
  • Published : 2015.11.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose a method which infers an accurate disparity map for radiometrically varying stereo images. For this end, firstly, we transform the input color images to the log-chromaticity color space from which a linear relationship can be established during constructing a joint pdf between input stereo images. Based on this linear property, we present a new stereo matching cost by combining weighted mutual information and the SIFT (Scale Invariant Feature Transform) descriptor with segment-based plane-fitting constraints to robustly find correspondences for stereo image pairs which undergo radiometric variations. Experimental results show that our method outperforms previous methods and produces accurate disparity maps even for stereo images with severe radiometric differences.

본 논문에서는 조명 변화에 강인한 상호 정보량 기반의 스테레오 정합 기법을 제안한다. 일반적으로 다양한 조명 조건에서 취득한 스테레오 영상은 좌우 영상 간에 컬러의 변화가 발생하기 때문에 정확한 스테레오 정합점을 찾는 것이 쉽지 않다. 이 경우, 컬러를 보정하는 작업을 우선 수행하는 것이 일반적이다. 그러나, 좌우 스테레오 영상에 대해서는 컬러 값을 동일하게 보정하는 작업도 시점 차이로 인한 좌우 영상의 변화로 인해서 좌우 영상에 대한 정합 정보가 요구되므로 쉽지 않다. 본 논문에서는 다양한 조명 조건에서 취득한 영상에 강인한 스테레오 정합 기법을 제안한다. 이를 위해서 선형적인 관계를 갖는 로그-색도 (log-chromaticity) 컬러 공간으로 변형을 수행하였고, 이 컬러 공간에서 상호 정보량에 기반한 새로운 스테레오 정합 비용 (cost)을 제안하였다. 제안하는 비용은 가중치가 적용된 상호 정보량과 SIFT (Scale Invariant Feature Transform) 묘사 벡터의 정보를 화소 (pixel)마다 적응적으로 결합한다. 또한, 보다 정확한 변위 지도 예측을 위해서 세그먼트 기반의 평면 제한 조건도 제안하는 비용에 포함되었다. 다양한 실험 데이터에 대해서 테스트한 결과, 제안하는 방법이 기존의 방법들에 비해서 보다 정확한 변위 지도 결과를 얻는 것을 확인하였다.

Keywords

References

  1. W.-S. Jang, C. Lee, and Y.-S. Ho, "Efficient depth map generation for various stereo camera arrangements," J. KICS, vol. 37, no. 6, pp. 458-463, 2012. https://doi.org/10.7840/KICS.2012.37.6A.458
  2. C. Lee, H. Song, B. Choi, and Y.-S. Ho, "Multi-view generation using high resolution stereoscopic cameras and a low resolution time-of-flight camera," J. KICS, vol. 37, no. 4, pp. 239-249, 2012. https://doi.org/10.7840/KICS.2012.37A.4.239
  3. C. Song and J. Lee, "Detection of illegal u-turn vehicles by optical flow analysis," J. KICS, vol. 39, no. 10, pp. 948-956, 2014.
  4. http://vision.middlebury.edu/stereo/, 2015.
  5. http://www.photomodeler.com/, 2015.
  6. S. Birchfield and C. Tomasi, "A pixel dissimilarity measure that is insensitive to image sampling," IEEE Trans. Pattern Anal. and Machine Intell., vol. 20, no. 4, pp. 401-406, 1998. https://doi.org/10.1109/34.677269
  7. Y. Boykov, O. Veksler, and R. Zabih, "Fast approximate energy minimization via graph cuts," IEEE Trans. Pattern Anal. and Machine Intell., vol. 23, no. 11, pp. 1222-1239, 2001. https://doi.org/10.1109/34.969114
  8. D. Comaniciu and P. Meer, "Mean shift: A robust approach toward feature space analysis," IEEE Trans. Pattern Anal. and Machine Intell., vol. 24, no. 5, pp. 603-619, 2001.
  9. A. Chakrabarti, D. Scharstein, and T. Zickler, "An empirical camera model for internet color vision," in Proc. British Machine Vision Conf., 2009.
  10. M. Ebner, Color constancy, ser. Wiley-IS&T Series in Imaging Sci. and Technol., John Wiley & Sons, 2007.
  11. G. Egnal, Mutual Information as a Stereo Correspondence Measure, in Technical Report MS-CIS-00-20, Computer and Information Science, Univ. of Pennsylvania, 2000.
  12. G. D. Finlayson, S. D. Hordley, G. Schaefer, and G. Y. Tian, "Illuminant and device invariant colour using histogram equalisation," Pattern Recognition, vol. 38, no. 2, pp. 179-190, 2005. https://doi.org/10.1016/j.patcog.2004.04.010
  13. Y. S. Heo, K. M. Lee, and S. U. Lee, "Robust stereo matching using adaptive normalized cross correlation," IEEE Trans. Pattern Anal. and Machine Intell., vol. 33, no. 4, pp. 807-822, 2011. https://doi.org/10.1109/TPAMI.2010.136
  14. Y. S. Heo, K. M. Lee, and S. U. Lee, "Mutual information-based stereo matching combined with SIFT descriptor in log-chromaticity color space," in Proc. IEEE CVPR 2009, pp. 445-452, Miami, FL, Jun. 2009.
  15. Y. S. Heo, K. M. Lee, and S. U. Lee, "Joint depth map and color consistency estimation for stereo images with different illuminations and cameras," IEEE Trans. Pattern Anal. and Machine Intell., vol. 35, no. 5, pp. 1094-1106, 2013. https://doi.org/10.1109/TPAMI.2012.167
  16. H. Hirschmuller, "Stereo processing by semiglobal matching and mutual information," IEEE Trans. Pattern Anal. and Machine Intell., vol. 30, no. 2, pp. 328-341, 2008. https://doi.org/10.1109/TPAMI.2007.1166
  17. H. Hirschmuller and D. Scharstein, "Evaluation of stereo matching costs on images with radiometric differences," IEEE Trans. Pattern Anal. and Machine Intell., vol. 31, no. 9, pp. 1582-1599, 2009. https://doi.org/10.1109/TPAMI.2008.221
  18. L. Hong and G. Chen, "Segment-based stereo matching using graph cuts," in Proc. IEEE CVPR, vol. 1, pp. 74-81, 2004.
  19. X. Hu and P. Mordohai, "Evaluation of stereo confidence indoors and outdoors," in Proc. IEEE CVPR, pp. 1466-1473, San Francisco, Jun. 2010.
  20. H. Jeon, A. Basso, and P. F. Driessen, "A global correspondence for scale invariant matching using mutual information and the graph search," in Proc. Int. Conf. Multimedia and Expo, pp. 1745-1748, Toronto, Ont, Jul. 2006.
  21. S. Kagarlitsky, Y. Moses, and Y. Hel-Or, "Piecewise-consistent color mappings of images acquired under various conditions," in Proc. IEEE Int. Conf. Computer Vision, pp. 2311-2318, Kyoto, Sept.-Oct. 2009.
  22. J. Kim, V. Kolmogorov, and R. Zabih, "Visual correspondence using energy minimization and mutual information," in Proc. IEEE Int'l Conf. Computer Vision, vol. 2, pp. 1033-1040, Nice, France, Oct. 2003.
  23. S. J. Kim and M. Pollefeys, "Robust radiometric calibration and vignetting correction," IEEE Trans. Pattern Anal. and Machine Intell., vol. 30, no. 4, pp. 562-576, Apr. 2008. https://doi.org/10.1109/TPAMI.2007.70732
  24. S. Lin, Y. Li, S. B. Kang, X. Tong, and H. Y. Shum, "Diffuse-specular separation and depth recovery from image sequences," in Proc. Eur. Conf. Computer Vision, 2002.
  25. C. Liu, J. Yuen, A. Torralba, J. Sivic, and W. T. Freeman, "SIFT flow: Dense correspondence across different scenes," in Proc. Eur. Conf. Computer Vision, vol. 5034, pp. 28-42, 2008.
  26. D. G. Lowe, "Distinctive image features from scale-invariant keypoints," Int'l J. Computer Vision, vol. 60, no. 2, pp. 91-110, 2004. https://doi.org/10.1023/B:VISI.0000029664.99615.94
  27. P. Montesinos, V. Gouet, R. Deriche, and D. Pele, "Matching color uncalibrated images using differential invariants," Image and Vision Computing, vol. 18, no. 1, pp. 659-671, 2000. https://doi.org/10.1016/S0262-8856(99)00070-0
  28. A. S. Ogale and Y. Aloimonos, "Robust contrast invariant stereo correspondence," in Proc. IEEE Int. Conf. Robot. and Automat., pp. 819-824, Apr. 2004.
  29. J. P. W. Pluim, J. B. A. Maintz, and M. A. Viergever, "Image registration by maximization of combined mutual information and gradient information," IEEE Trans. Med. Imaging, vol. 19, no. 8, pp. 809-814, 2000. https://doi.org/10.1109/42.876307
  30. J. P. W. Pluim, J. B. A. Maintz and M. A. Viergever, "Mutual-information-based registration of medical images : A survey," IEEE Trans. Med. Imaging, vol. 22, no. 8, pp. 986-1004, 2003. https://doi.org/10.1109/TMI.2003.815867
  31. D. B. Russakoff, C. Tomasi, T. Rohlfing, and C. R. Maurer Jr., "Image similarity using mutual information of regions," in Proc. Eur. Conf. Computer Vision, vol. 3023, pp. 596-607, 2004.
  32. D. Scharstein and R. Szeliski, "A taxonomy and evaluation of dense two-frame stereo correspondence algorithms," Int. J. Computer Vision, vol. 47, no. 1, pp. 7-42, 2002. https://doi.org/10.1023/A:1014573219977
  33. N. Snavely, S. M. Seitz, and R. Szeliski, "Modeling the world from internet photo collections," Int. J. Computer Vision, vol. 80, no. 2, pp. 189-210, 2008. https://doi.org/10.1007/s11263-007-0107-3
  34. S. M. Seitz and S. Baker, "Filter flow," in Proc. IEEE Int. Conf. Computer Vision, 2009.
  35. J. Sun, Y. Li, S. B. Kang, and H. Y. Shum, "Symmetric stereo matching for occlusion handling," in Proc. CVPR, vol. 2, pp. 399-406, Jun. 2005.
  36. E. Tola, V. Lepetit, and P. Fua, "DAISY: An efficient dense descriptor applied to wide-baseline stereo," IEEE Trans. Pattern Anal. and Machine Intell., vol. 32, no. 5, pp. 815-830, 2010. https://doi.org/10.1109/TPAMI.2009.77
  37. P. Viola and W. M. Wells III, "Alignment by maximization of mutual information," Int. J. Computer Vision, vol. 24, no. 2, pp. 137-154, 1997. https://doi.org/10.1023/A:1007958904918
  38. Y. Weiss, "Deriving intrinsic images from image sequences," in Proc. IEEE Int. Conf. Computer Vision, vol. 2, pp. 68-75, Vancouver, 2001.
  39. W. Xiong and B. Funt, "Stereo retinex," in Proc. Canadian Conf. Computer and Robot Vision, Jun. 2006.
  40. J. N. Yang and S. K. Shevell, "Stereo disparity improves color constancy," Vision Res., vol. 42, no. 1, pp. 1979-1989, 2002. https://doi.org/10.1016/S0042-6989(02)00098-6
  41. R. Zabih and J. Woodfill, "Non-parametric local transforms for computing visual correspondence," in Proc. Eur. Conf. Computer Vision, vol. 801, pp. 151-158, Jun. 1994.
  42. C. L. Zitnick, S. B. Kang, M. Uyttendaele, S. Winder, and R. Szeliski, "High-quality video view interpolation using a layered representation," in Proc. SIGGRAPH, pp. 600-608, Aug. 2004.
  43. V. Kolmogorov, "Convergent tree-reweighted message passing for energy minimization," IEEE Trans. Pattern Anal. and Machine Intell., vol. 28, no. 10, pp. 1568-1583, 2006. https://doi.org/10.1109/TPAMI.2006.200
  44. R. Szeliski, R. Zabih, D. Scharstein, O. Veksler, V. Kolmogorov, A. Agarwala, M. Tappen, and C. Rother, "A comparative study of energy minimization methods for markov random fields with smoothness-based priors," IEEE Trans. Pattern Anal. and Machine Intell., vol. 30, no. 6, pp. 1068-1080, Jun. 2008. https://doi.org/10.1109/TPAMI.2007.70844