• The Journal of Korean Institute of Communications and Information Sciences
• /
• v.37 no.1A
• /
• pp.57-63
• /
• 2012

In this paper, we propose a real-time view interpolation method using virtual viewpoint disparity estimation and convergence check. For the real-time process, we estimate a disparity map at the virtual viewpoint from stereo images using the belief propagation method. This method needs only one disparity map, compared to the conventional methods that need two disparity maps. In the view synthesis part, we warp pixels from the reference images to the virtual viewpoint image using the disparity map at the virtual viewpoint. For real-time acceleration, we utilize a high speed GPU parallel programming, called CUDA. As a result, we can interpolate virtual viewpoint images in real-time.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.22 no.3
• /
• pp.425-433
• /
• 2018

In this paper, we propose a super-resolution reconstruction algorithm for plenoptic images based on the reliability of disparity. The subperture image generated by the Flanoptic camera image is used for disparity estimation and reconstruction of super-resolution image based on TV_L1 algorithm. In particular, the proposed image reconstruction method is effective in the boundary region where disparity may be relatively inaccurate. The determination of reliability of disparity vector is based on the upper, lower, left and right positional relationship of the sub-aperture image. In our method, the unreliable vectors are excluded in reconstruction. The performance of the proposed method was evaluated by comparing to a bicubic interpolation method, a conventional disparity based method and dictionary based method. The experimental results show that the proposed method provides the best performance in terms of PSNR(Peak Signal to noise ratio), SSIM(Structural Similarity).

• Journal of the Institute of Electronics Engineers of Korea TE
• /
• v.39 no.4
• /
• pp.389-394
• /
• 2002

In this paper, we propose a robust stereo matching algorithm using nonlinear extended Kalman filter. The proposed algorithm estimates disparity using nonlinear extended Kalman filter and compares left image to right image for obtained disparity. As this process is run iteratively, we get disparity only with a few search. And, we can get robust stereo matching results by comparing left image to right image using bilinear interpolation to consider influence of neighborhood pixel. We compared SSD algorithm which is widely used, in stereo matching method, to result of the proposed algorithm. As the result, the proposed algorithm has an outstanding matching performance.

• Journal of KIISE
• /
• v.44 no.9
• /
• pp.932-945
• /
• 2017

In this paper, a new hardware and software architecture for a stereo vision processing system including rectification, disparity estimation, and visualization was developed. The developed method, named LArge scale stereo matching method using Support Point Interpolation (LASPI), shows excellence in real-time processing for obtaining dense disparity maps from high quality image regions that contain high density support points. In the real-time processing of high definition (HD) images, LASPI does not degrade the quality level of disparity maps compared to existing stereo-matching methods such as Efficient LArge-scale Stereo matching (ELAS). LASPI has been designed to meet a high frame-rate, accurate distance resolution performance, and a low resource usage even in a limited resource environment. These characteristics enable LASPI to be deployed to safety-critical applications such as an obstacle recognition system and distance detection system for autonomous vehicles. A Field Programmable Gate Array (FPGA) for the LASPI algorithm has been implemented in order to support parallel processing and 4-stage pipelining. From various experiments, it was verified that the developed FPGA system (Xilinx Virtex-7 FPGA, 148.5MHz Clock) is capable of processing 30 HD ($1280{\times}720pixels$) frames per second in real-time while it generates disparity maps that are applicable to real vehicles.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2009.01a
• /
• pp.233-237
• /
• 2009

This paper represents a new approach which addresses quality degradation of a synthesized view, when a virtual camera moves forward. Generally, interpolation technique using only two neighboring views is used when a virtual view is synthesized. Because a size of the object increases when the virtual camera moves forward, most methods solved this by interpolation in order to synthesize a virtual view. However, as it generates a degraded view such as blurred images, we prevent a synthesized view from being blurred by using more cameras in multiview camera configuration. That is, we solve this by applying super-resolution concept which reconstructs a high resolution image from several low resolution images. Therefore, data fusion is executed by geometric warping using a disparity of the multiple images followed by deblur operation. Experimental results show that the image quality can further be improved by reducing blur in comparison with interpolation method.

• Journal of Broadcast Engineering
• /
• v.4 no.1
• /
• pp.59-67
• /
• 1999

In this paper, we study two important topics in stereoscopic image communication system. One is a disparity estimation (DE) method to obtain the depth information of a scene at the transmitter and the other is an intermediate view reconstruction(IVR) method at the receiver. We propose a new DE method using an edge-preserving directional regularization technique. The proposed DE method smooths disparity vectors in smooth regions and preserves edges without over-smoothing problem. It provides better reconstructed stereoscopic images and improved coding efficiency than the existing regularization techniques. In addition. we propose a new IVR method using interpolation and extrapolation techniques. The proposed IVR method preserves edge regions as well as occlusion regions well. Thus. it gives better intermediate views than the existing IVR methods.

• Journal of the Institute of Electronics Engineers of Korea SC
• /
• v.45 no.6
• /
• pp.108-116
• /
• 2008

In this paper, we propose an algorithm that can estimate the distance using disparity based on stereo vision system, even though the obstacle is located in long ranges as well as short ranges. We use sub-pixel interpolation to minimize quantization errors which deteriorate the distance accuracy when calculating the distance with integer disparity, and also we use enhanced adaptive fuzzy strong tracking Kalman filter(EAFSTKF) to improve the distance accuracy and track the path optimally. The proposed method can solve the divergence problem caused by nonlinear dynamics such as various vehicle movements in the conventional Kalman filter(CKF), and also enhance the distance accuracy and reliability. Our simulation results show that the performance of our method improves by about 13.5% compared to other methods in point of root mean square error rate(RMSER).

• Journal of Korea Multimedia Society
• /
• v.20 no.12
• /
• pp.1865-1873
• /
• 2017

The feature-based visual SLAM requires 3D positions for the extracted feature points to perform 3D-2D motion estimation. LiDAR can provide reliable and accurate 3D position information with low computational burden, while stereo camera has the problem of the impossibility of stereo matching in simple texture image region, the inaccuracy in depth value due to error contained in intrinsic and extrinsic camera parameter, and the limited number of depth value restricted by permissible stereo disparity. However, the sparsity of LiDAR data may increase the inaccuracy of motion estimation and can even lead to the result of motion estimation failure. Therefore, in this paper, we propose three interpolation methods which can be applied to interpolate sparse LiDAR data. Simulation results obtained by applying these three methods to a visual odometry algorithm demonstrates that the selective bilinear interpolation shows better performance in the view point of computation speed and accuracy.

• Journal of Broadcast Engineering
• /
• v.19 no.5
• /
• pp.616-626
• /
• 2014

In this paper, we introduce a real-time stereo-to-multiview conversion system using FPGA and GPU. The system is based on two different devices so that it consists of two major blocks. The first block is a disparity estimation block that is implemented on FPGA. In this block, each disparity map of stereoscopic video is estimated by DP(dynamic programming)-based stereo matching. And then the estimated disparity maps are refined by post-processing. The refined disparity map is transferred to the GPU device through USB 3.0 and PCI-express interfaces. Stereoscopic video is also transferred to the GPU device. These data are used to render arbitrary number of virtual views in next block. In the second block, disparity-based view interpolation is performed to generate virtual multi-view video. As a final step, all generated views have to be re-arranged into a single image at full resolution for presenting on the target autostereoscopic 3D display. All these steps of the second block are performed in parallel on the GPU device.

• Journal of the Institute of Electronics Engineers of Korea SP
• /
• v.40 no.6
• /
• pp.141-148
• /
• 2003

This paper proposes a new algorithm that generates the smooth and realistic transition views from one viewpoint to another on the multi-panorama based navigation system. The proposed algorithm is composed of two steps. One is prewarping that aligns the viewing directions of two panoramic images, and the other is the bidirectional disparity morphing(BDM) that generates the intermediate scene from the aligned panoramic images. For prewarping, we compute the phase correlation between two images in order to obtain the information, such as translation, rotation, and scaling. Then we align the viewing directions of two original images using these information. Afterprewarping, we compute the block based disparity vector(DV) and smooth them using two occluding patterns. As we apply these DVs to the BDM, we can generate the elaborate intermediate scene. We make an experiment on the proposed algorithm with some real panoramic images and obtain good quality intermediate scenes.