IEIE Transactions on Smart Processing and Computing

The Institute of Electronics and Information Engineers (대한전자공학회)
권호 표지
DOI QR코드

DOI QR Code

Depth-adaptive Sharpness Adjustments for Stereoscopic Perception Improvement and Hardware Implementation

  • Kim, Hak Gu (Department of Electronic Engineering, Inha University) ;
  • Kang, Jin Ku (Department of Electronic Engineering, Inha University) ;
  • Song, Byung Cheol (Department of Electronic Engineering, Inha University)
  • Received : 2013.11.22
  • Accepted : 2014.03.20
  • Published : 2014.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

This paper reports a depth-adaptive sharpness adjustment algorithm for stereoscopic perception improvement, and presents its field-programmable gate array (FPGA) implementation results. The first step of the proposed algorithm was to estimate the depth information of an input stereo video on a block basis. Second, the objects in the input video were segmented according to their depths. Third, the sharpness of the foreground objects was enhanced and that of the background was maintained or weakened. This paper proposes a new sharpness enhancement algorithm to suppress visually annoying artifacts, such as jagging and halos. The simulation results show that the proposed algorithm can improve stereoscopic perception without intentional depth adjustments. In addition, the hardware architecture of the proposed algorithm was designed and implemented on a general-purpose FPGA board. Real-time processing for full high-definition stereo videos was accomplished using 30,278 look-up tables, 24,553 registers, and 1,794,297 bits of memory at an operating frequency of 200MHz.

Keywords

References

  1. R.Hartley andA. Zisserman, Multiple View Geometry in Computer Vision, Cambridge University Press, 2000.
  2. D. Kim and W. Sohn, "Depth adjustment for stereoscopic image using visual fatigue prediction and depth-based view synthesis," Proc. Of IEEE International Conference on Multimedia and Expo, pp. 956-961, Jul. 2010.
  3. S. Kishi, S. H. Kim, T. Shibata, T.Kawai, J. Hakkinen, J. Takatalo, and G. Nyman, "Scalable 3D image conversion and ergonomic evaluation," Proc. SPIE, vol. 6803, 2008.
  4. J. Konrad, B. Lacotte and E. Dubois, "Cancellation of image crosstalk in time sequential displays of stereoscopic video," IEEE Trans. Image Processing, vol. 9, no. 5, pp. 897-908, 2000. https://doi.org/10.1109/83.841535
  5. J. Park, G. Um, C. Ahn, and C.-T. Ahn, "Virtual control of optical axis of the 3DTV camera for reducing visual fatigue in stereoscopic 3DTV," ETRI Journal, pp. 597-604, 2004. https://doi.org/10.4218/etrij.04.0603.0024
  6. N. Holliman, "Mapping perceived depth to regions of interest in stereoscopic images," in Proc. SPIE, Stereoscopic Disp. Virtual Reality Syst. XI, 2004, vol. 5291, pp. 1-12.
  7. G. Sun and N. S. Holliman, "Evaluating methods for controlling depth perception in stereoscopic cinematography," in Proc. Stereoscopic Displays Virtual Reality Syst. XX, 2009, vol. 7237, pp.72370I-1-72370I-12.
  8. S. J. Daly, R. T. Held, and D. M. Hoffman, "Perceptual issues in stereoscopic signal processing," IEEE Trans. Broadcast., vol. 57, no. 2, pp. 347-361, Jun. 2011. https://doi.org/10.1109/TBC.2011.2127630
  9. A. Criminisi, P. Perez, and K. Toyama, "Object removal by exemplar-based inpainting," in Proc. Conf. Computer Vision and Pattern Recognition, Madison, WI, Jun. 2003.
  10. H. Kim, D. B. Min, S. Choi, and K. Sohn, "Real-time disparity estimation using foreground segmentation for stereo sequences," Optical Engineering, vol. 45, no. 3, pp. 037402(10pages) Mar. 2006. https://doi.org/10.1117/1.2183667
  11. S. H. Lee and S. Sharma, "Real-time Disparity estimation algorithm for Stereo Camera Systems," IEEE Transaction on Consumer Electronics, vol. 57, pp. 1018-1026, 2011. https://doi.org/10.1109/TCE.2011.6018850
  12. D. Tzovaras, M. G. Strintzis, and H. Sahinoglou, "Evaluation of multi resolution block matching techniques for motion and disparity estimation," Signal Processing: Image Communication, vol. 6, no. 1, pp. 59-67, Mar. 1994. https://doi.org/10.1016/0923-5965(94)90046-9
  13. L. Zhang, "Hierarchical block-based disparity estimation using mean absolute difference and dynamic programming," in Proc. Int. Workshop VeryLow Bit rate Video Coding, 2001, pp. 114-118.
  14. E. Lee and Y. Ho, "Generation of multi-view video using a fusion camera system for 3D displays," IEEE Trans. On Consumer Electronics, vol. 56, no. 4, pp. 2797-2805, Dec. 2010. https://doi.org/10.1109/TCE.2010.5681171
  15. M. EbroulIzuierdo, "Stereo image analysis for multiviewpoint telepresence applications," Signal Processing: Image Communication, vol. 11, pp. 231-254, 1998. https://doi.org/10.1016/S0923-5965(97)00031-3
  16. L. Shapiro, and G. Stockman, Computer Vision, Prentice Hall, pp. 69-73. 2002.
  17. A. Polesel, G. Ramponi, and V. Mathews, "Image enhancement via adaptive unsharp masking," IEEE Trans. Image Process., vol. 9, no. 3, pp. 505-510, Mar. 2000. https://doi.org/10.1109/83.826787
  18. Z. Wang, E. P. Simoncelli, and A. C Bovik, "Multiscale structural similarity for image quality assessment," presented at the IEEE Asilomar Conf. Signals, Systems, and Computers, Nov. 2003.
  19. G. Deng, "A generalized unsharp masking algorithm," IEEE Trans. Image Process., vol. 20, no. 5, pp. 1249-1261, May 2011. https://doi.org/10.1109/TIP.2010.2092441

Cited by

  1. Depth map generation using a single image sensor with phase masks vol.24, pp.12, 2016, https://doi.org/10.1364/OE.24.012868
  2. Optical range-finding system using a single-image sensor with liquid crystal display aperture vol.41, pp.22, 2016, https://doi.org/10.1364/OL.41.005154