• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2012.07a
• /
• pp.257-259
• /
• 2012

이 논문에서는 실험을 통해서, 기존에 제안하였던 쿼리-바이-싱잉/허밍 (Query-by-singing/humming, QbSH) 시스템의 매칭 엔진의 강인성을 검증하고 그 결과를 제시한다. QbSH 시스템은 디지털 음악의 사용이 보편화되면서 음악 검색의 방법으로 많은 연구가 진행되어 오고 있다. QbSH 시스템은 입력으로부터 멜로디의 특징을 추출하는 부분과 추출된 특징을 매칭하는 부분으로 나눌 수 있는데, 매칭 단계에서 특징이 추출된 두 개의 멜로디 사이의 유사도 또는 거리를 계산하여 가장 유사한 멜로디를 데이터베이스에서 찾게 된다. 이 논문에서는 이 중, 기존에 제안하였던 매칭 엔진 부분의 강인성을 알아보기 위해서 입력으로부터 멜로디의 피치 시퀀스를 추출하는 과정을 간략히 하여 그 결과를 살펴보았다. 즉, 기존에 제안한 매칭 엔진이 특정한 피치 시퀀스 추출 과정에서만 좋은 성능을 보이는 게 아님을 실험을 통해서 살펴보았다. 실험 결과, 피치 시퀀스를 추출하는 과정이 극도로 간략해지더라도, 매칭 엔진을 좋은 성능을 보여주었다.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2013.11a
• /
• pp.1390-1393
• /
• 2013

인터페이스가 제한된 단말에서 정보 검색 서비스를 제공하는 경우, 검색 재현율보다는 정확도가 중요하다. 데이터를 쉽게 구조화할 수 있고 검색 정확도가 중요한 한정된 도메인에서는 시맨틱 검색 기술을 통해 강력한 정보 검색 서비스를 제공할 수 있지만, 사용자 키워드 질의를 시스템 질의로 변환하는 과정에서 다양한 해석들이 존재할 수 있기에 개선의 여지도 많다. 본 논문에서는 해석 정확도와 확장성을 동시에 향상시키기 위한 새로운 모델을 제안한다. 제안 모델은 공간의 구조와 요소들의 해석을 제한함으로써 중간 탐색 공간의 크기를 점진적으로 줄이면서 사용자의 검색 의도는 가능한 보존할 수 있다. 실제 데이터로 이루어진 대용량 지식을 이용해 다른 최신 기술과 비교하여 실험적 평가를 제시하였다.

• Journal of Information Processing Systems
• /
• v.17 no.5
• /
• pp.1020-1033
• /
• 2021

Nowadays, online or mobile social network services (SNS) are very popular and widely spread in our society and daily lives to instantly share, disseminate, and search information. In particular, SNS such as YouTube, Flickr, Facebook, and Amazon allow users to upload billions of images or videos and also provide a number of multimedia information to users. Information retrieval in multimedia-rich SNS is very useful but challenging task. Content-based media retrieval (CBMR) is the process of obtaining the relevant image or video objects for a given query from a collection of information sources. However, CBMR suffers from the dimensionality curse due to inherent high dimensionality features of media data. This paper investigates the effectiveness of the kernel trick in CBMR, specifically, the kernel principal component analysis (KPCA) for dimensionality reduction. KPCA is a nonlinear extension of linear principal component analysis (LPCA) to discovering nonlinear embeddings using the kernel trick. The fundamental idea of KPCA is mapping the input data into a highdimensional feature space through a nonlinear kernel function and then computing the principal components on that mapped space. This paper investigates the potential of KPCA in CBMR for feature extraction or dimensionality reduction. Using the Gaussian kernel in our experiments, we compute the principal components of an image dataset in the transformed space and then we use them as new feature dimensions for the image dataset. Moreover, KPCA can be applied to other many domains including CBMR, where LPCA has been used to extract features and where the nonlinear extension would be effective. Our results from extensive experiments demonstrate that the potential of KPCA is very encouraging compared with LPCA in CBMR.

• Journal of Intelligence and Information Systems
• /
• v.20 no.3
• /
• pp.59-76
• /
• 2014

The revolution of smart media such as smart phone, smart TV and tablets has brought easiness for people to get contents and related information anywhere and anytime. The characteristics of the smart media have changed user behavior for watching the contents from passive attitude into active one. Video is a kind of multimedia resources and widely used to provide information effectively. People not only watch video contents, but also search for related information to specific objects appeared in the contents. However, people have to use extra views or devices to find the information because the existing video contents provide no information through the contents. Therefore, the interaction between user and media is becoming a major concern. The demand for direct interaction and instant information is much increasing. Digital media environment is no longer expected to serve as a one-way information service, which requires user to search manually on the internet finding information they need. To solve the current inconvenience, an interactive service is needed to provide the information exchange function between people and video contents, or between people themselves. Recently, many researchers have recognized the importance of the requirements for interactive services, but only few services provide interactive video within restricted functionality. Only cooking domain is chosen for an interactive cooking video query service in this research. Cooking is receiving lots of people attention continuously. By using smart media devices, user can easily watch a cooking video. One-way information nature of cooking video does not allow to interactively getting more information about the certain contents, although due to the characteristics of videos, cooking videos provide various information such as cooking scenes and explanation for each recipe step. Cooking video indeed attracts academic researches to study and solve several problems related to cooking. However, just few studies focused on interactive services in cooking video and they still not sufficient to provide the interaction with users. In this paper, an interactive cooking video query service system with linked data to provide the interaction functionalities to users. A linked recipe schema is used to handle the linked data. The linked data approach is applied to construct queries in systematic manner when user interacts with cooking videos. We add some classes, data properties, and relations to the linked recipe schema because the current version of the schema is not enough to serve user interaction. A web crawler extracts recipe information from allrecipes.com. All extracted recipe information is transformed into ontology instances by using developed instance generator. To provide a query function, hundreds of questions in cooking video web sites such as BBC food, Foodista, Fine cooking are investigated and analyzed. After the analysis of the investigated questions, we summary the questions into four categories by question generalization. For the question generalization, the questions are clustered in eleven questions. The proposed system provides an environment associating UI (User Interface) and UX (User Experience) that allow user to watch cooking videos while obtaining the necessary additional information using extra information layer. User can use the proposed interactive cooking video system at both PC and mobile environments because responsive web design is applied for the proposed system. In addition, the proposed system enables the interaction between user and video in various smart media devices by employing linked data to provide information matching with the current context. Two methods are used to evaluate the proposed system. First, through a questionnaire-based method, computer system usability is measured by comparing the proposed system with the existing web site. Second, the answer accuracy for user interaction is measured to inspect to-be-offered information. The experimental results show that the proposed system receives a favorable evaluation and provides accurate answers for user interaction.

• ETRI Journal
• /
• v.24 no.1
• /
• pp.1-11
• /
• 2002

A video summary abstracts the gist from an entire video and also enables efficient access to the desired content. In this paper, we propose a novel method for summarizing news video based on multimodal analysis of the content. The proposed method exploits the closed caption data to locate semantically meaningful highlights in a news video and speech signals in an audio stream to align the closed caption data with the video in a time-line. Then, the detected highlights are described using MPEG-7 Summarization Description Scheme, which allows efficient browsing of the content through such functionalities as multi-level abstracts and navigation guidance. Multimodal search and retrieval are also within the proposed framework. By indexing synchronized closed caption data, the video clips are searchable by inputting a text query. Intensive experiments with prototypical systems are presented to demonstrate the validity and reliability of the proposed method in real applications.

• Journal of Korea Multimedia Society
• /
• v.10 no.6
• /
• pp.707-715
• /
• 2007

The need for content-based retrieval from visual media, such as image and video data, is ever increasing rapidly in many applications, such as electronic art museums, internet shopping malls, internet search engines, and medical information systems. In our previous research, we proposed an architecture, called the HERMES, which is a Web Service-enabled visual media retrieval framework. In this paper, we propose the Web Service workflows that are employed in the HERMES. We describe how we designed the workflows for service registration and query processing in the framework. We especially explain how metadata and ontology can be utilized to realize more intelligent content-based retrieval on visual media data.

• Journal of KIISE:Software and Applications
• /
• v.35 no.1
• /
• pp.22-32
• /
• 2008

To overcome the weakness of the image retrieval system using the existing Ontology and the distributed image based on the database having a simple structure, HERMES was suggested to ensure the self-control of various image suppliers and support the image retrieval based on semantic, the mentioned framework could not solve the problems which are not considered the deterioration in the capacity and scalability when many users connect to broker server simultaneously. In this paper the tables are written which in the case numerous users connect at the same time to the supply analogous level of services without the deterioration in the capacity installs Broker servers and then measures the performance time of each inner Broker Component through Monitoring System and saved and decides the ranking in saved data. As many Query performances are dispersed into several Servers User inputted from the users Interface with reference to Broker Ranking Table, Load Balancing system improving reliability in capacity is proposed. Through the experiment, the scheduling technique has proved that this schedule is faster than existing techniques.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2012.07a
• /
• pp.164-166
• /
• 2012

In this paper, we proposed a method for extracting predominant melody of polyphonic music based on harmonic structure. Harmonic structure is an important feature parameter of monophonic signal that has spectral peaks at the integer multiples of its fundamental frequency. We extract all fundamental frequency candidates contained in the polyphonic signal by verifying the required condition of harmonic structure. Then, we combine those harmonic peaks corresponding to each extracted fundamental frequency and assign a rank to each after calculating its harmonic average energy. We run pitch tracking based on the rank of extracted fundamental frequency and continuity of fundamental frequency, and determine the predominant melody. For the query by singing/humming (QbSH) task, we proposed Dynamic Time Warping (DTW) based matching engine. Our system reduces false alarm by combining the distances of multiple DTW processes. To improve the performance, we introduced the asymmetric sense, pitch level compensation, and distance intransitiveness to DTW algorithm.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2012.11a
• /
• pp.637-640
• /
• 2012

본 논문에서는 다성 음원 기반의 다양한 응용 분야에서 적용이 가능한 실용 Query by Singing/Humming (QbSH) 시스템을 구현한다. 이를 위해서 검색 서버을 위한 소프트웨어 스택을 개발하고 다양한 형태의 클라이언트와 함께 임베디드 시스템과 같은 저성능 클라이언트를 위한 허밍검색 에이전트를 개발하여 기존의 클라이언트 시스템에 플러그인 할 수 있는 모듈을 개발한다. 실용 QbSH 시스템을 개발하기 위하여 각각의 핵심 알고리즘들의 최적의 조합을 이루어 통합된 전체시스템의 성능을 평가한다. 학술적인 연구에만 그쳤던 허밍기반 오디오 검색 서비스를 상업적으로 이용하기 위해 다양한 응용분야의 프로토타입을 구현한다.

• Journal of the Korea Society of Computer and Information
• /
• v.19 no.9
• /
• pp.45-53
• /
• 2014

This paper suggests methods to chronicle and track the history of dialogues exchanged among characters to prevent logical errors of a story. As for stories that are long with many characters, especially in full-length novels and co-written stories, cognitive burden is imposed on a writer. If the writer has confused understanding of a character, then a logical error would enter the story. This would compromise completeness and integrity of writing. Against the backdrop, this paper shows how dialogues among characters are chronicled and tracked by using the aforementioned tracking methods through design of a writer support system that relieves a writer's cognitive burden while supporting the writing and through an analysis of existing novels. In addition, we showed the accuracy results of average 68.5% through the performance evaluation of the query used in the dialogue history tracking.