• Journal of Internet Computing and Services
• /
• v.22 no.3
• /
• pp.1-8
• /
• 2021

Ultra high-quality and ultra high-resolution omnidirectional 360-degree video streaming is needed to provide immersive media through head-mounted display(HMD) in virtual reality environment, which requires high bandwidth and computational complexity. One of the approaches avoiding these problems is to apply viewport-dependent selective streaming using tile-based segmentation method. This paper presents a performance analysis of viewport-dependent tiled streaming on 16K ultra high-quality 360-degree videos and 4K 360-degree videos which are widely used. Experimental results showed 42.47% of bjotegaard delta rate(BD-rate) saving on 16K ultra high-quality 360-degree video tiled streaming compared to viewport-independent streaming while 4K 360-degree video showed 26.41% of BD-rate saving. Therefore, this paper verified that tiled streaming is more efficient on ultra-high quality video.

• Journal of Digital Convergence
• /
• v.17 no.4
• /
• pp.255-260
• /
• 2019

This study is about an efficient synthesis of $360^{\circ}$ video and 3D graphics. First, the video image filmed by a binocular integral type $360^{\circ}$ camera was stitched, and location values of the camera and objects were extracted. And the data of extracted location values were moved to the 3D program to create 3D objects, and the methods for natural compositing was researched. As a result, as the method for natural compositing of $360^{\circ}$ video image and 3D graphics, rendering factors and rendering method were derived. First, as for rendering factors, there were 3D objects' location and quality of material, lighting and shadow. Second, as for rendering method, actual video based rendering method's necessity was found. Providing the method for natural compositing of $360^{\circ}$ video image and 3D graphics through this study process and results is expected to be helpful for research and production of $360^{\circ}$ video image and VR video contents.

• Proceedings of the Korean Society of Computer Information Conference
• /
• 2019.07a
• /
• pp.317-318
• /
• 2019

가상/증강현실로 대표되는 공간정보 기반 실감형 콘텐츠에 대한 관심이 증대되면서 객체인식 등의 지능형 공간인지 기술에 대한 연구가 활발히 진행되고 있다. 특히 HMD등의 영상 시각화 장치의 발달 및 5G 통신기술의 출현으로 인해 실시간 대용량 영상정보의 송, 수신 및 가시화 처리 기술의 기반이 구축됨에 따라, $360^{\circ}$ 스트리밍 영상정보 처리와 같은 고자유도 콘텐츠를 위한 관련 연구의 필요성이 증대되고 있다. 하지만 지능형 영상정보 처리의 대표적 연구인 딥 러닝(Deep Learning) 기반 객체 인식 기술의 경우 대부분 일반적인 평면 영상(Planar Image)에 대한 처리를 다루고 있고, 파노라마 영상(Panorama Image) 특히, $360^{\circ}$ 스트리밍 영상 처리를 위한 연구는 미비한 상황이다. 본 논문에서는 딥 러닝을 이용하여 $360^{\circ}$ 스트리밍 영상에서의 객체인식 연구 방법에 대해 서술한다. 이를 위해 $360^{\circ}$ 카메라 영상에서 딥 러닝을 위한 학습 데이터를 획득하고, 실시간 객체 인식이 가능한 YOLO(You Only Look Once)기법을 이용하여 학습을 한다. 실험 결과에서는 학습 데이터를 이용하여 $360^{\circ}$영상에서 객체 인식 결과와, 학습 횟수에 따른 객체 인식에 대한 결과를 보여준다.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2017.10a
• /
• pp.665-667
• /
• 2017

가상현실은 4차 산업 혁명의 키워드 중에 하나이다. 가상현실은 인간이 시간, 공간의 제약으로 인해 경험하지 못하는 것들을 새로운 플랫폼으로 경험하게 해주어 시간, 공간의 제약과 환경의 제약을 극복하게 해준다. 현재 게임으로 한정된 VR 콘텐츠를 사용자들이 촬영한 360도 동영상으로 보완할 수 있기 때문에 국내외 대기업에서도 360도 VR 동영상에 공을 들이고 있다. 특히, 360도 VR동영상은 방송, 쇼핑, 관광 등에 활용하기에 적합하여 관련 분야의 기업에서 360도 동영상을 활용하면 효율적인 비즈니스가 가능하다. 그러나 아직 국내에서는 본격적으로 360도 동영상 콘텐츠를 활용한 사례가 거의 없다. 본 연구에서는 360도 VR동영상에 대해 고찰하고 호텔기업에 활용 할 것을 제안하고 분석한다.

• Journal of Broadcast Engineering
• /
• v.24 no.1
• /
• pp.3-24
• /
• 2019

Techniques related to 360 VR service have been developed to improve the quality of the stitched image and video, where illumination compensation scheme is one of the important tools. Among the conventional illumination compensation algorithms, Gain-based compensation and Block Gain-based compensation algorithms have shown the outstanding performances in the process of making panorama picture. However, those are ineffective in the 360 VR service, because the disparity between illuminations of the multiple pictures in 360 VR is much more than that in making the panorama picture. In addition, the number of the pictures to be stitched in 360 VR system is more than that in the conventional panorama image system. Thus, we propose a preprocessing tool to enhance the illumination compensation algorithm so that the method reduces the degradation in the stitched picture of 360 VR systems. The proposed algorithm consists of 'color compensation' and 'illumination compensation'. The simulation results show that the proposed technique improve the conventional techniques without additional complexity.

• Journal of Broadcast Engineering
• /
• v.25 no.5
• /
• pp.770-775
• /
• 2020

Since 360-degree ERP frame structure has location-dependent distortion, existing video surveillance algorithms cannot be applied to 360-degree video. In this paper, an activated viewport based event detection method is proposed for 360-degree video. After extracting activated viewports enclosing object candidates, objects are finally detected in the viewports. These objects are tracked in 360-degree video space for region-based event detection. The proposed method is shown to improve the recall and the false negative rate more than 30% compared to the conventional method without activated viewports.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2018.06a
• /
• pp.11-14
• /
• 2018

본 논문에서는 실시간 고화질 360 영상 전송을 위해 사용자 시점에 기반한 타일 분할 기법을 적용하고 360 영상을 가상현실 기기 화면에 표현하여 주관적 화질 평가를 위한 플랫폼을 구현한다. 사용자 시점에 기반한 고화질 360 영상 전송을 위한 방안으로, 분할된 영상에서 전송되지 않은 영상으로의 움직임 참조 문제를 해결하기 위해 적용된 MCTS (motion constrained tile sets) 기술과 실시간으로 사용자 시점에 위치한 타일들만 추출할 수 있도록 미리 구성된 타일 정보들을 포함하는 EIS (extraction information sets) SEI, 사용자 시점에 위치한 타일 정보만 추출해내고 영상을 분할 및 추출해주는 Extractor, 실제 추출된 영상 정보를 이용해 가상현실 기기 화면에 표현하는 방법에 대한 구현 내용을 설명한다. 따라서 제안된 구현물을 기반으로 고화질 360 영상 전송을 수행하면, 사용자 시점 영역의 영상만 전송하여 불필요한 영상 전송을 하지 않게 되어 화질 대비 낮은 대역폭의 향상된 실시간 전송 영상을 얻을 수 있다.

• Journal of Broadcast Engineering
• /
• v.24 no.6
• /
• pp.1096-1112
• /
• 2019

The cameras generating 360 VR image are too expensive to be used publically. To overcome this problem, we propose a way to use smart phones instead of VR camera, where more than 100 pictures are taken by smart phone and are stitched into a 360 VR image. In this scenario, when moving objects are in some of the pictures, the stitched 360 VR image has various degradations, for example, ghost effect and mis-aligning. In this paper, we proposed an algorithm to modify the seam finding algorithms, where the saliency map in ROI is generated to check whether the pixel belongs to visually salient objects or not. Various simulation results show that the proposed algorithm is effective to increase the quality of the generated 360 VR image.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.18 no.6
• /
• pp.63-68
• /
• 2018

Offloading is a method of communicating, processing, and receiving results from some of the applications performed on local computers to overcome the limitations of computing resources and computational speed.Recently, it has been applied in mobile games, multimedia data, 360-degree video processing, and image processing for Internet broadcasting to speed up processing and reduce battery consumption in the mobile computing sector. This paper implements a viewer that enables users to convert various flat-panel images and view contents in a wireless Internet environment and presents actual results of an experiment so that users can easily understand the images. The 360 degree spherical image is successfully converted to a plane image with Double Panorama, Quad, Single Rectangle, 360 Overview + 3 Rectangle depending on the image acquisition position of the 360 degree camera through the interface. During the experiment, more than 100 360 degree spherical images were successfully converted into plane images through the interface below.

• Journal of the Korea Computer Graphics Society
• /
• v.23 no.3
• /
• pp.65-75
• /
• 2017

As a 360-degree image carries information of all directions, it often has too much information. Moreover, in order to investigate a 360-degree image on a 2D display, a user has to either click and drag the image with a mouse, or project it to a 2D panorama image, which inevitably introduces severe distortions. In consequence, investigating a 360-degree image and finding an object of interest in such a 360-degree image could be a tedious task. To resolve this issue, this paper proposes a method to find a region of interest and produces a 2D naturally looking image from a given 360-degree image that best matches a description given by a user in a natural language sentence. Our method also considers photo composition so that the resulting image is aesthetically pleasing. Our method first converts a 360-degree image to a 2D cubemap. As objects in a 360-degree image may appear distorted or split into multiple pieces in a typical cubemap, leading to failure of detection of such objects, we introduce a modified cubemap. Then our method applies a Long Short Term Memory (LSTM) network based object detection method to find a region of interest with a given natural language sentence. Finally, our method produces an image that contains the detected region, and also has aesthetically pleasing composition.