• Journal of Broadcast Engineering
• /
• v.27 no.1
• /
• pp.20-30
• /
• 2022

Recently, as VR (Virtual Reality) technology has been in the spotlight, 360° panoramic images that can view lively VR contents are attracting a lot of attention. Image stitching technology is a major technology for producing 360° panorama images, and many studies are being actively conducted. Typical stitching algorithms are based on feature point-based image stitching. However, conventional feature point-based image stitching methods have a problem that stitching results are intensely affected by feature points. To solve this problem, deep learning-based image stitching technologies have recently been studied, but there are still many problems when there are few overlapping areas between images or large parallax. In addition, there is a limit to complete supervised learning because labeled ground-truth panorama images cannot be obtained in a real environment. Therefore, we produced three fisheye images with different camera centers and corresponding ground truth image through carla simulator that is widely used in the autonomous driving field. We propose image stitching model that creates a 360° panorama image with the produced fisheye image. The final experimental results are virtual datasets configured similar to the actual environment, verifying stitching results that are strong against various environments and large parallax.

• Journal of Broadcast Engineering
• /
• v.27 no.4
• /
• pp.487-498
• /
• 2022

Depth map is an image that contains distance information in 3D space on a 2D plane and is used in various 3D vision tasks. Many existing depth estimation studies mainly use narrow FoV images, in which a significant portion of the entire scene is lost. In this paper, we propose a technique for generating 360° omnidirectional RGBD images from a sparse set of narrow FoV images. The proposed generative adversarial network based image generation model estimates the relative FoV for the entire panoramic image from a small number of non-overlapping images and produces a 360° RGB and depth image simultaneously. In addition, it shows improved performance by configuring a network reflecting the spherical characteristics of the 360° image.

• Journal of Broadcast Engineering
• /
• v.24 no.1
• /
• pp.3-24
• /
• 2019

Techniques related to 360 VR service have been developed to improve the quality of the stitched image and video, where illumination compensation scheme is one of the important tools. Among the conventional illumination compensation algorithms, Gain-based compensation and Block Gain-based compensation algorithms have shown the outstanding performances in the process of making panorama picture. However, those are ineffective in the 360 VR service, because the disparity between illuminations of the multiple pictures in 360 VR is much more than that in making the panorama picture. In addition, the number of the pictures to be stitched in 360 VR system is more than that in the conventional panorama image system. Thus, we propose a preprocessing tool to enhance the illumination compensation algorithm so that the method reduces the degradation in the stitched picture of 360 VR systems. The proposed algorithm consists of 'color compensation' and 'illumination compensation'. The simulation results show that the proposed technique improve the conventional techniques without additional complexity.

• Journal of IKEEE
• /
• v.21 no.3
• /
• pp.268-271
• /
• 2017

In this paper, we propose the development of high resolution $360^{\circ}$ omnidirectional IP camera with 12 million pixels. The proposed 12-megapixel high-resolution $360^{\circ}$ omnidirectional IP camera consists of a lens unit with $360^{\circ}$ omnidirectional viewing angle and a 12-megapixel high-resolution IP camera unit. The lens section of $360^{\circ}$ omnidirectional viewing angle adopts the isochronous lens design method and the catadioptric facet production method to obtain the image without peripheral distortion which is inevitably generated in the fisheye lens. The 12 megapixel high-resolution IP camera unit consists of a CMOS sensor & ISP unit, a DSP unit, and an I / O unit, and converts the image input to the camera into a digital image to perform image distortion correction, image correction and image compression And then transmits it to the NVR (Network Video Recorder). In order to evaluate the performance of the proposed 12-megapixel high-resolution $360^{\circ}$ omnidirectional IP camera, 12.3 million pixel image efficiency, $360^{\circ}$ omnidirectional lens angle of view, and electromagnetic certification standard were measured.

• Journal of the Semiconductor & Display Technology
• /
• v.15 no.2
• /
• pp.66-69
• /
• 2016

Recently, augmented reality and virtual reality in the ICT sector have been highlighted. So also interested in related HMD areas to facilitate contact with the VR content is being attend. This paper proposes a method for implementing to the virtual reality through the mobile HMD device with the real time 360 image. This system is required the real time 360 image streaming server configuration and image processing for augmented reality and virtual reality. The configuration of the streaming server is configured the DB server to store images and the relay server that can relay images to other devices. Augmented image processing module is composed based on markerless tracking, and there are four modules that are recognition, tracking, detecting and learning module. Also, the purpose of this paper is shown the augmented 360 image processing through the Mobile HMD.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2021.06a
• /
• pp.212-215
• /
• 2021

본 논문에서는 6 Fisheye lens 원본 영상에 대하여 Insta360 stitcher, AutoStitch[4], As-Projective-AsPossible(APAP)[5] 스티칭 방법으로 360 도 파노라마 영상을 생성하고 기하학적 왜곡과 컬러 왜곡을 비교 평가한다. 360 도 파노라마 Image Quality Assessment(IQA) 메트릭으로 Natural Image Quality Evaluator(NIQE)[6], Blind/Referenceless Image Spatial Quality Evaluator (BRISQUE)[7], Perception based Image Quality Evaluator(PIQE)[8], Feature Similarity(FSIM)[9] 그리고 high frequency feature 에 대한 Structural Similarity(SSIM)[10]을 측정하여 정량적 평가를 하며 정성적인 비교를 통하여 파노라마 영상의 품질과 평가 메트릭에 대한 벤치마크를 제공한다.

• Journal of Broadcast Engineering
• /
• v.23 no.6
• /
• pp.866-875
• /
• 2018

With the popularity of VR content, actual image based VR 360-degree video production using various shooting devices has attracted attention in recent years. Requiring a lot of work, the existing VR content production was possible only at the level of the research institute, but the recent miniaturization of cameras has made the VR actual image shooting more common to the content creators. In this paper, we compare and study various VR contents distribute in today's context and analyze the strengths and weaknesses of robust actual image-based 360 video production in terms of cost. We are to point out the disadvantages that make it difficult for existing video shooting experts to use and suggest a solution for this. The experiment shooting proved that it can be operated more manually than action cam or integrated camera and 360-degree shooting on a raw format support camera is more suitable for film type post-production.

• Journal of IKEEE
• /
• v.21 no.4
• /
• pp.412-415
• /
• 2017

In this paper, we propose a multi license plate recognition system using high resolution $360^{\circ}$ omnidirectional IP camera. The proposed system consists of a planar division part of $360^{\circ}$ circular image and a multi license plate recognition part. The planar division part of the $360^{\circ}$ circular image are divided into a planar image with enhanced image quality through processes such as circular image acquisition, circular image segmentation, conversion to plane image, pixel correction using color interpolation, color correction and edge correction in a high resolution $360^{\circ}$ omnidirectional IP Camera. Multi license plate recognition part is through the multi-plate extraction candidate region, a multi-plate candidate area normalized and restore, multiple license plate number, character recognition using a neural network in the process of recognizing a multi-planar imaging plates. In order to evaluate the multi license plate recognition system using the proposed high resolution $360^{\circ}$ omnidirectional IP camera, we experimented with a specialist in the operation of intelligent parking control system, and 97.8% of high plate recognition rate was confirmed.

• Journal of the Korea Society of Computer and Information
• /
• v.27 no.10
• /
• pp.147-153
• /
• 2022

Most of the metaverse service is a gamification type of metaverse solution composed by placing 3D objects on a 360 panoramic world base. However, the metaverse webtoon service is lacking in implementation and research. In this study, a method for realizing a 2D flat image form in a 3D space and a realistic 360-degree panoramic webtoon metaverse service were studied. The research process consisted of basic storytelling and design work for webtoon production, panoramic image creation to convert the produced image into 360-degree form, and content creation process for viewing 360-degree directions. Finally, we implemented shading and material work with game engine tools so that you can enjoy virtual reality-based webtoons. The webtoon-based metaverse study studied the process of creating a 360-degree panoramic webtoon from content that can only be viewed in 2D format. Accordingly, it is thought that webtoons will also be presented as a standard for the production and implementation method for the metaverse service.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• fall
• /
• pp.41-44
• /
• 2021

In this paper, we propose an efficient reference-based compression artifact reduction network for 360-degree images in an equi-rectangular projection (ERP) domain. In our insight, conventional image restoration methods cannot be applied straightforwardly to 360-degree images due to the spherical distortion. To address this problem, we propose an adaptive disparity estimator using a deformable convolution to exploit correlation among 360-degree images. With the help of the proposed convolution, the disparity estimator establishes the spatial correspondence successfully between the ERPs and extract matched textures to be used for image restoration. The experimental results demonstrate that the proposed algorithm provides reliable high-quality textures from the reference and improves the quality of the restored image as compared to the state-of-the-art single image restoration methods.