• 전자통신동향분석
• /
• 제37권3호
• /
• pp.52-63
• /
• 2022

Immersive media, also known as spatial media, has become essential with the decrease in face-to-face activities in the COVID-19 pandemic era. Teleconference, metaverse, and digital twin have been developed with high expectations as immersive media services, and the demand for hyper-realistic media is increasing. Under these circumstances, MPEG-I Immersive Media is being standardized as a technologies of navigable virtual reality, which is expected to be launched in the first half of 2024, and the Audio Group is working to standardize the immersive audio technology. Following this trend, this article introduces the trend in MPEG-I immersive audio standardization. Further, it describes the features of the immersive audio rendering technology, focusing on the structure and function of the RM0 base technology, which was chosen after evaluating all the technologies proposed in the January 2022 "MPEG Audio Meeting."

• 방송공학회논문지
• /
• 제25권5호
• /
• pp.723-733
• /
• 2020

본 고에서는 현재 탐색단계의 표준화가 진행 중인 MPEG-I Immersive Audio 표준화 동향을 소개한다. 이 표준은 5G/6G와 같은 초연결 환경에서 킬러 어플리케이션으로 기대되는 가상현실(Virtual Reality; VR) 및 증강현실(Augmemted Reality; AR)에서, 이용자가 가상환경과 상호작용을 통해 6 자유도(Degrees of freedom; DoF)로 소리를 실감나게 느끼고 실제 환경에서 경험하는 것과 같은 공간음향 체험을 제공하는 것을 목표로 한다. 이를 위하여, MPEG Audio Working Group에서는 가상현실 및 증강현실에서 공간음향 체험을 위한 시스템 구조 및 요구사항을 정의하였다. 이를 기반으로 요구사항에 대한 제안 기술 선정을 위한 오디오 평가 플랫폼(Audio evaluation platform; AEP), 인코더 입력 포맷(Encoder input format; EIF) 및 평가 절차 등에 대한 논의를 진행하고 있으며, 본 고에서는 그 주요 내용을 요약 기술한다.

• 전자통신동향분석
• /
• 제34권3호
• /
• pp.13-22
• /
• 2019

Although virtual reality technology may not be deemed as having a satisfactory quality for all users, it tends to incite interest because of the expectation that the technology can allow one to experience something that they may never experience in real life. The most important aspect of this indirect experience is the provision of immersive 3D audio and video, which interacts naturally with every action of the user. The immersive audio faithfully reproduces an acoustic scene in a space corresponding to the position and movement of the listener, and this technology is also called spatial audio. In this paper, we briefly introduce the trend of spatial audio technology in view of acquisition, analysis, reproduction, and the concept of MPEG-I audio standard technology, which is being promoted for spatial audio services.

• 방송공학회논문지
• /
• 제27권7호
• /
• pp.974-984
• /
• 2022

In this paper, we introduce the minimum and maximum distance setting methods used in geometric distance attenuation processing, which is one of spatial sound reproduction methods. In general, sound attenuation by distance is inversely proportional to distance, that is 1/r law, but when the relative distance between the user and the audio object is very short or long, exceptional processing might be performed by setting the minimum distance or the maximum distance. While MPEG-I Immersive Audio's RM0 uses fixed values for the minimum and maximum distances, this study proposes effective methods for setting the distances considering the signal gain of an audio object. Proposed methods were verified through simulation of the proposed methods and experiments using RM0 renderer.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제14권3호
• /
• pp.1121-1141
• /
• 2020

We propose a robust visual object tracking algorithm fusing a convolutional neural network tracker trained offline from a large number of video repositories and a color histogram based tracker to track objects for mixing immersive audio. Our algorithm addresses the problem of occlusion and large movements of the CNN based GOTURN generic object tracker. The key idea is the offline training of a binary classifier with the color histogram similarity values estimated via both trackers used in this method to opt appropriate tracker for target tracking and update both trackers with the predicted bounding box position of the target to continue tracking. Furthermore, a histogram similarity constraint is applied before updating the trackers to maximize the tracking accuracy. Finally, we compute the depth(z) of the target object by one of the prominent unsupervised monocular depth estimation algorithms to ensure the necessary 3D position of the tracked object to mix the immersive audio into that object. Our proposed algorithm demonstrates about 2% improved accuracy over the outperforming GOTURN algorithm in the existing VOT2014 tracking benchmark. Additionally, our tracker also works well to track multiple objects utilizing the concept of single object tracker but no demonstrations on any MOT benchmark.

• 한국전자통신학회논문지
• /
• 제10권2호
• /
• pp.233-238
• /
• 2015

본 논문에서는 FTV(Free-viewpoint TV) 서비스에서, 몰입도를 향상시킬 수 있는 유효 잡음 이용 입체 음향효과 방법을 제안한다. 농구장에서 초지향성 마이크 및 무선 마이크를 사용하여 선수와 심판의 연속적인 음향 정보를 획득함으로써 주파수 스펙트럼을 관찰하였으며, 스펙트럼을 분석하여 시청자가 Zoom-in을 할 경우, 유효 주파수 여부를 판단하였다. 따라서 FTV 서비스에서 시청자가 피사체를 향해 Zoom-in 시, 제거대상이었던 잡음을 활용할 필요가 있음을 제시하였다. 본 연구는 향후 FTV의 입체 음향 연구에 활용될 것으로 기대된다.

• International Journal of Internet, Broadcasting and Communication
• /
• 제13권1호
• /
• pp.161-167
• /
• 2021

This paper proposes a new personalized HRTF estimation method which is based on a deep neural network (DNN) model and improved elevation reproduction using a notch filter. In the previous study, a DNN model was proposed that estimates the magnitude of HRTF by using anthropometric measurements [1]. However, since this method uses zero-phase without estimating the phase, it causes the internalization (i.e., the inside-the-head localization) of sound when listening the spatial sound. We devise a method to estimate both the magnitude and phase of HRTF based on the DNN model. Personalized HRIR was estimated using the anthropometric measurements including detailed data of the head, torso, shoulders and ears as inputs for the DNN model. After that, the estimated HRIR was filtered with an appropriate notch filter to improve elevation reproduction. In order to evaluate the performance, both of the objective and subjective evaluations are conducted. For the objective evaluation, the root mean square error (RMSE) and the log spectral distance (LSD) between the reference HRTF and the estimated HRTF are measured. For subjective evaluation, the MUSHRA test and preference test are conducted. As a result, the proposed method can make listeners experience more immersive audio than the previous methods.

• 대한전자공학회논문지SP
• /
• 제47권5호
• /
• pp.45-55
• /
• 2010

객체 기반 오디오 부호화 기술은 다양한 응용 분야를 기대할 수 있는 차세대 오디오 기술로써 관심이 높다. 최근 MPEG에서는 SAOC (Spatial Audio Object Coding)라는 압축 효율이 우수한 Parametric 객체 부호화 방법을 표준화하였다. 본 논문에서는 SAOC를 중심으로 Parametric 객체 오디오 부호화의 기술을 소개하고, 이를 실제 적용하기 위한 고려사항들에 대해 다룬다.

• ETRI Journal
• /
• 제38권5호
• /
• pp.829-837
• /
• 2016

In this paper, we introduce extension tools for MPEG Surround, which were recently adopted as MPEG-H 3D Audio tools by the ISO/MPEG standardization group. MPEG-H 3D Audio is a next-generation technology for representing spatial audio in an immersive manner. However, considerably large numbers of input signals can degrade the compression performance during a low bitrate operation. The proposed extension of MPEG Surround was basically designed based on the original MPEG Surround technology, where the limitations of MPEG Surround were revised by adopting a new coding structure. The proposed MPEG-H 3D Audio technologies will play a pivotal role in dramatically improving the sound quality during a lower bitrate operation.

• International journal of advanced smart convergence
• /
• 제11권4호
• /
• pp.1-9
• /
• 2022

The recent development of information and communication technology is also affecting audio/video systems used in industry. The audio/video device configuration system changes from analog to digital, and the network-based audio/video system control has the advantage of reducing costs in accordance with system operation. However, audio/video systems released on the market have limitations in that they can only control their own products or can only be performed on specific platforms (Windows, Mac, Linux). This paper is a study on a device (Network Audio Video Integrated Control: NAVICS) that can integrate and control multiple audio / video devices with different functions, and can control digitalized audio / video devices through network and serial communication. As a result of the study, it was confirmed that individual control and integrated control were possible through the protocol provided by each audio/video device by NAVICS, and that even non-experts could easily control the audio/video system. In the future, it is expected that network-based audio/video integrated control technology will become the technical standard for complex audio/video system control.