KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

A Multi-Level Accumulation-Based Rectification Method and Its Circuit Implementation

  • Son, Hyeon-Sik (School of Electronics Engineering, Kyungpook National University) ;
  • Moon, Byungin (School of Electronics Engineering, Kyungpook National University)
  • Received : 2016.09.19
  • Accepted : 2017.04.05
  • Published : 2017.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

Rectification is an essential procedure for simplifying the disparity extraction of stereo matching algorithms by removing vertical mismatches between left and right images. To support real-time stereo matching, studies have introduced several look-up table (LUT)- and computational logic (CL)-based rectification approaches. However, to support high-resolution images, the LUT-based approach requires considerable memory resources, and the CL-based approach requires numerous hardware resources for its circuit implementation. Thus, this paper proposes a multi-level accumulation-based rectification method as a simple CL-based method and its circuit implementation. The proposed method, which includes distortion correction, reduces addition operations by 29%, and removes multiplication operations by replacing the complex matrix computations and high-degree polynomial calculations of the conventional rectification with simple multi-level accumulations. The proposed rectification circuit can rectify $1,280{\times}720$ stereo images at a frame rate of 135 fps at a clock frequency of 125 MHz. Because the circuit is fully pipelined, it continuously generates a pair of left and right rectified pixels every cycle after 13-cycle latency plus initial image buffering time. Experimental results show that the proposed method requires significantly fewer hardware resources than the conventional method while the differences between the results of the proposed and conventional full rectifications are negligible.

Keywords

References

  1. Kohtaro Sabe, Masaki Fukuchi, Jens-Steffen Gutmann, Takeshi Ohashi, Kenta Kawamoto and Takayuki Yoshigahara, "Obstacle avoidance and path planning for humanoid robots using stereo vision," in Proc. of IEEE Int. Conf. on Robotics and Automation, pp. 592-597, April 26-May 1, 2004.
  2. Kunsoo Huh, Jaehak Park, Junyeon Hwang and Daegun Hong, "A stereo vision-based obstacle detection system in vehicles," Optics and Laser in Engineering, vol. 46, no. 2, pp. 168-178, February, 2008. https://doi.org/10.1016/j.optlaseng.2007.08.002
  3. Annett Chilian and Heiko Hirschmuller, "Stereo camera based navigation of mobile robots on rough terrain," in Proc. of IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pp. 4571-4576, October 11-15, 2009.
  4. Martin Humenberger, Tobias Engelke and Wilfried Kubinger, "A census-based stereo vision algorithm using modified semi-global matching and plane fitting to improve matching quality," in Proc. of the 2010 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition Workshops, pp. 77-84, June 13-18, 2010.
  5. Wannes van der Mark and Dariu M. Gavrila, "Real-time dense stereo for intelligent vehicles," IEEE Transactions on Intelligent Transportation Systems, vol. 7, no. 1, pp. 38-50, March, 2006. https://doi.org/10.1109/TITS.2006.869625
  6. Charles Loop and Zhengyou Zhang, "Computing rectifying homographies for stereo vision," in Proc. of IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, pp. 125-131, June 23-25, 1999.
  7. Joshua Gluckman and Shree K. Nayar, "Rectified catadioptric stereo sensors," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 2, pp.224-236, February, 2002. https://doi.org/10.1109/34.982902
  8. Khurram Jawed, John Morris, Tariq Khan and Georgy Gimel'farb, "Real time rectification for stereo correspondence," in Proc. of IEEE Int. Conf. on Computational Science and Engineering, pp. 277-284, August 29-31, 2009.
  9. Abdulkadir Akin, Ipek Baz, Luis Manuel Gaemperle, Alexandre Schmid and Yusuf Leblebici, "Compressed look-up-table based real-time rectification hardware," in Proc. of 2013 IFIP/IEEE 21st Int. Conf. on Very Large Scale Integration, pp. 272-277, October 7-9, 2013.
  10. S. Jin, J. Cho, X. D. Pham, K. M. Lee, S.-K. Park, M. Kim and J. W. Jeon, "FPGA design and implementation of a real-time stereo vision system," IEEE Transactions on Circuits and Systems for Video Technology, vol. 20, no. 1, pp.15-26, January, 2010. https://doi.org/10.1109/TCSVT.2009.2026831
  11. Sang-Kyo Han, SeongHoon Woo, Mun-Ho Jeong and Bum-Jae You, "Improved-quality real-time stereo vision processor," in Proc. of 22nd Int. Conf. on VLSI Design, pp. 287-292, January 5-9, 2009.
  12. Hongru Zhang, Zaifeng Shi, Ke Pang, Qingjie Cao, Tao Luo and Suying Yao, "Seam-based variable-step Bresenham blending method for real-time video mosaicking," Journal of Electronic Imaging, vol. 25, no. 5, pp. 053008, September, 2016. https://doi.org/10.1117/1.JEI.25.5.053008
  13. Jean-Yves Bouguet, "Camera calibration toolbox for Matlab," http://www.vision.caltech.edu/bouguetj/calib_doc/.
  14. Andrea Fusiello, Emanuele Trucco and Alessandro Verri, "A compact algorithm for rectification of stereo pairs," Machine Vision and Applications, vol. 12, no. 1, pp. 16-22, July, 2000. https://doi.org/10.1007/s001380050120
  15. Cristian Vancea and Sergiu Nedevschi, "LUT-based image rectification module implemented in FPGA," in Proc. of IEEE Int. Conf. on Intelligent Computer Communication and Processing, pp. 147-154, September 6-8, 2007.
  16. Deuk Hyun Park, Hyoung Seok Ko, Jae Gon Kim and Jun Dong Cho, "Real time rectification using differentially encoded lookup table," in Proc. of Int. Conf. on Ubiquitous Information Management and Communication, February 21-23, 2011.
  17. Paolo Zicari, "Efficient and high performance FPGA-based rectification architecture for stereo vision," Microprocessors and Microsystems, vol. 37, no. 8, pp. 1144-1154, November, 2013. https://doi.org/10.1016/j.micpro.2013.09.007
  18. Jongkil Hyun and Byungin Moon, "A simplified rectification method and its hardware architecture for embedded multimedia systems," Multimedia Tools and Applications, published online, April, 2016.
  19. Hyeon-Sik Son and Byungin Moon, "An accumulation-based rectification and distortion correction method," in Proc. of Int. Conf. on Electronics, Electrical Engineering, Computer Science, pp.1-6, January 20-22, 2016.
  20. Roger Y. Tsai, "A versatile camera calibration technique for high-accuracy 3d machine vision metrology using off-the-shelf TV cameras and Lenses," IEEE Journal of Robotics and Automation, vol. RA-3, no. 4, pp. 323-343, August, 1987.
  21. Matlab documentation, "What is camera calibration?," http://www.mathworks.com/help/vision/ug/camera-calibration.html.
  22. Peter Abeles. "Boofcv," http://boofcv.org/.
  23. Matlab documentation, "Stereo calibration app," http://www.mathworks.com/help/vision/ug/stereo-camera-calibrator-app.html.
  24. Zhengyou Zhang, "A flexible new technique for camera calibration," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 11, pp. 1330-1334, November, 2000. https://doi.org/10.1109/34.888718
  25. Janne Heikkila and Olli Silven, "A four-step camera calibration procedure with implicit image correction," in Proc. of IEEE Int. Conf. on Computer Vision and Pattern Recognition. pp. 1106-1112, June 17-19, 1997.
  26. Heiko Hirschmuller, "Accurate and efficient stereo processing by semi-global matching and mutual information," in Proc. of IEEE Int. Conf. on Computer Vision and Pattern Recognition. pp. 807-814, June 20-25, 2005.