• ETRI Journal
• /
• v.31 no.4
• /
• pp.429-437
• /
• 2009

This paper proposes a novel technique for 3D scene relighting with interactive viewpoint changes. The proposed technique is based on a deep framebuffer framework for fast relighting computation which adopts image-based techniques to provide arbitrary view-changing. In the preprocessing stage, the shading parameters required for the surface shaders, such as surface color, normal, depth, ambient/diffuse/specular coefficients, and roughness, are cached into multiple deep framebuffers generated by several caching cameras which are created in an automatic manner. When the user designs the lighting setup, the relighting renderer builds a map to connect a screen pixel for the current rendering camera to the corresponding deep framebuffer pixel and then computes illumination at each pixel with the cache values taken from the deep framebuffers. All the relighting computations except the deep framebuffer pre-computation are carried out at interactive rates by the GPU.

• Journal of the HCI Society of Korea
• /
• v.2 no.1
• /
• pp.25-31
• /
• 2007

We develop an interactive relighting renderer allowing camera view changes based on a deep-frame buffer approach. The renderer first caches the rendering parameters for a given 3D scene in an auxiliary buffer with the same size of the output image. The rendering parameters independent from light changes are selected from the shading models used for shading pixels. Next, as the user interactively edits one light at one time, the relighting renderer instantly re-shades each pixel by updating the contribution of the changed light with the shading parameters cached in the deep-frame buffer. When the camera moves, the cache values should be re-computed because the currently cached values become obsolete. We present a novel method to synthesize them quickly from the cache images of the user specified cameras by using an image-based technique. This computations are all performed on GPU to achieve real-time performance.

• International Journal of CAD/CAM
• /
• v.12 no.1
• /
• pp.38-47
• /
• 2012

The paper proposes a method for generating a sequence of images with smooth transition of illumination from two input images with different lighting conditions. Our relighting approach is image-based, such as the light field rendering. We store the luminances (pixel RGB values) into "lighting functions" consisting of a couple of parameters related to normal vectors. Images with different light positions are rendered by interpolating the luminances retrieved from the lighting functions. The proposed method is a promising technique for many applications requiring a scene with variety of lighting effects, such as movies, TV games, and so on.

• 한국HCI학회:학술대회논문집
• /
• 2007.02a
• /
• pp.38-43
• /
• 2007

재조명(relighting) 렌더링은 장면 내에 새로운 광원의 추가 또는 기존 광원 속성의 변경으로 인한 영상의 변화를 효율적으로 계산하는 과정을 말한다. 본 논문에서는 쉐이딩(shading) 계산에서 광원에 독립적인 파라메터를 미리 텍스쳐 이미지 형태로 캐시화하여 재조명 렌더링 과정에서의 계산량을 줄이는 방법을 사용하였다. 이러한 쉐이딩 파라메터들의 캐시 이미지들은 사용자가 카메라 시점을 바꾸고자 할 경우 새로 생성을 하여야 하는데, 이 계산에 많은 시간이 소요된다. 본 논문에서는 새로운 시점에서의 캐시 이미지들를 영상 기반 렌더링(image-based rendering) 기법을 이용하여 실시간에 구하는 방법을 제시한다. 먼저 여러 개의 지정된 카메라 시점에 대한 캐시 이미지들을 미리 생성해 둔다. 다음 원하는 시점의 캐시 이미지는 각 픽셀에 투영되는 3차원 표면점을 역시점변환(inverse viewing transform)을 통해 구하고, 이 점을 지정된 카메라 시점으로 다시 투영하여 캐시 이미지에서의 대응 픽셀을 찾는다. 대응 픽셀의 파라메터 값들을 평균하여 새 캐시 이미지에 써준다. 이 과정들은 하드웨어 그래픽 가속기의 단편 쉐이더(fragment shader)를 이용하여 실시간으로 수행된다.

• Journal of Broadcast Engineering
• /
• v.14 no.6
• /
• pp.807-810
• /
• 2009

This paper deals with a real-time image based relighting system for tangible video teleconference. The proposed image based relighting system renders the extracted human object using the virtual environmental images. The proposed system can homogenize virtually the lighting environments of remote users on the video teleconference, or render the humans like they are in the virtual places. To realize the video teleconference, the paper obtains the 3D object models of users in real-time using the controlled lighting system. In this paper, we use single color camera and synchronized two directional flash lights. Proposed system generates pure shading images using on and off flash images subtraction. One pure shading reflectance map generates a directional normal map from multiplication of each reflectance map and basic normal vector map. Each directional basic normal map is generated by inner vector calculation of incident light vector and camera viewing vector. And the basic normal vector means a basis component of real surface normal vector. The proposed system enables the users to immerse video teleconference just as they are in the virtual environments.

• Journal of Broadcast Engineering
• /
• v.13 no.3
• /
• pp.339-349
• /
• 2008

We present a novel method to estimate the light source direction and relight a face texture image of a single 3D model under arbitrary unknown illumination conditions. We create a virtual irradiance sphere to detect the light source direction from a given illuminated texture image using both normal vector mapping and weighted bilinear interpolation. We then induce a relighting equation with estimated ambient and diffuse coefficients. We provide the result of a series of experiments on light source estimation, relighting and face recognition to show the efficiency and accuracy of the proposed method in restoring the shading and shadows areas of a face texture image. Our approach for face relighting can be used for not only illuminant invariant face recognition applications but also reducing visual load and Improving visual performance in tasks using 3D displays.

• Journal of the Korea Computer Graphics Society
• /
• v.26 no.3
• /
• pp.87-97
• /
• 2020

In this paper, we propose a novel neural relighting that infers a relighted rendering image based on the user-guided specular highlight map. The proposed network utilizes a pre-trained neural renderer as a backbone network learned from the rendered image of a 3D scene with various lighting conditions. We jointly optimize a 3D light position and its associated relighted image by back-propagation, so that the difference between the base image and the relighted image is similar to the user-guided specular highlight map. The proposed method has the advantage of being able to explicitly infer the 3D lighting position, while providing the artists' preferred 2D screen-space interface. The performance of the proposed network was measured under the conditions that can establish ground truths, and the average error rate of light position estimations is 0.11, with the normalized 3D scene size.

• Journal of Korea Game Society
• /
• v.7 no.4
• /
• pp.53-62
• /
• 2007

Although radiosity can represent diffuse reflections of the object surfaces by modeling energy exchange in 3D space, there are some restrictions for real-time applications because of its computation loads. Therefore, GPU(Graphics Processing Unit) based radiosity algorithms have been presented actively to improve its rendering performance. We implement the progressive refinement radiosity on GPU by G. Coombe in 3D scene that is constructed with HDR(High Dynamic Range) radiance map. This radiosity method can generate a photo-realistic rendering image in 3D space, where the synthetic objects were illuminated by the environmental light sources. In the simulation results, the rendering performance is analyzed according to the resolution of the texel in the environmental map and mipmaping. In addition, we compare the rendering results by our method with those by the incremental radiosity.

• 한국HCI학회:학술대회논문집
• /
• 2007.02a
• /
• pp.988-993
• /
• 2007

전역조명기법(global illumination)중에서 난반사(diffuse reflection) 객체들 사이의 관계를 효과적으로 표현하는 래디오시티(radiosity)방법은 객체들 사이의 에너지 교환에 에너지 평형 상태를 모델링 한다. 그러나 래디오시티는 많은 계산량으로 인해 실시간 활용에는 적합하지 않았다. 최근 장면생성에 걸리는 소요시간을 크게 단축시킬 수 있는 비용대비 고성능의 그래픽스 하드웨어(GPU)를 이용한 방법들이 제안되고 있다. 객체들 사이에서 교환되는 에너지는 래디언스(radiance)로 표현이 가능하며, 이러한 래디언스는 대상 장면에서 취득한 HDR(High Dynamic Range) 영상으로부터 래디언스 맵을 구성해서 얻을 수 있다. 이를 기반으로 대상장면의 조명환경을 구성하면 대상장면의 복잡도와는 별개로 빠르고 사실적인 합성장면을 생성할 수 있다. 본 논문에서는 G. Coombe 등이 제안한 점진적 세분(progressive refinement) 알고리즘을 수정하여 래디언스 맵을 이용할 수 있도록 하였으며, 각 텍셀(texel)설정 및 보간(interpolation) 적용 등에 따른 실험 결과를 얻고 분석하였다. 구현된 방법은 이후 영상기반 재조명과 그래픽스 하드웨어를 이용한 영상합성 기술로 영화, 애니메이션, 가상현실, 게임 등에 다양하게 활용될 예정이다.

• Journal of the Institute of Electronics Engineers of Korea CI
• /
• v.43 no.2 s.308
• /
• pp.63-71
• /
• 2006

Denoising and reconstruction of color images are extensively studied in the field of computer vision and image processing. Especially, denoising and reconstruction of color face images are more difficult than those of natural images because of the structural characteristics of human faces as well as the subtleties of color interactions. In this paper, we propose a denoising method based on PCA reconstruction for removing complex color noise on human faces, which is not easy to remove by using vectorial color filters. The proposed method is composed of the following five steps: training of canonical eigenface space using PCA, automatic extraction of facial features using active appearance model, relishing of reconstructed color image using bilateral filter, extraction of noise regions using the variance of training data, and reconstruction using partial information of input images (except the noise regions) and blending of the reconstructed image with the original image. Experimental results show that the proposed denoising method maintains the structural characteristics of input faces, while efficiently removing complex color noise.