• Journal of Korea Multimedia Society
• /
• v.20 no.2
• /
• pp.170-178
• /
• 2017

Biometric recognition is one of the major challenging topics which needs high performance of recognition accuracy. Most of existing methods rely on a single source of biometric to achieve recognition. The recognition accuracy in biometrics is affected by the variability of effects, including illumination and appearance variations. In this paper, we propose a new multimodal biometrics recognition using convolutional neural network. We focus on multimodal biometrics from face and periocular regions. Through experiments, we have demonstrated that facial multimodal biometrics features deep learning framework is helpful for achieving high recognition performance.

• Journal of Korea Society of Industrial Information Systems
• /
• v.11 no.5
• /
• pp.170-180
• /
• 2006

The increase in international logistics market and various customer demands emphasize the importance of Multimodal Transportation, and that market is continuously keeping growing. In order to ensure competitive superiority in a market of such infinite competition, service that can satisfy each individual customer by considering various customer characteristics, has become an issue. Thus, through the aspect of service, in order to improve customer satisfaction, growing factors of Multimodal Transportation Route was studied in this research. For this research, first of all main service factors that affect the growth of Multimodal Transportation were seized by literature survey and positive research. Then, by using these factors a methodology that enables individual customers to assess Multimodal Transportation Route was studied. Through this research, individual customers can acquire objective assessment data and Multimodal Transportation companies can seize what factors are considered as important by their customers.

• Proceedings of the Korean Institute of Navigation and Port Research Conference
• /
• v.1
• /
• pp.251-259
• /
• 2006

The increase in international logistics market and various customer demands emphasize the importance of Multimodal Transportation, and that market is continuously keep growing. In order to ensure competitive superiority in a market of such infinite competition, service that can satisfy each individual customer by considering various customer characteristics, has become an issue. Thus, through the aspect of service, in order to improve customer satisfaction, growing factors of Multimodal Transportation Route on was studied in this research. For this research, first of all main service factors that affect the growth of Multimodal Transportation were seized by literature survey and positive research. The, by using these factors a methodology that enables individual customers to assess Multimodal Transportation Route was studied. Through this research, individual customers can acquire objective assessment data and Multimodal Transportation companies can seize what factors are considered as important by their customers.

• Proceedings of the Korean Information Science Society Conference
• /
• 2011.06b
• /
• pp.231-234
• /
• 2011

Audiovisual-based human emotion recognition can be considered a good approach for multimodal humancomputer interaction. However, the optimal multimodal information fusion remains challenges. In order to overcome the limitations and bring robustness to the interface, we propose a framework of automatic human emotion recognition system from speech and face display. In this paper, we develop a new approach for fusing information in model-level based on the relationship between speech and face expression to detect automatic temporal segments and perform multimodal information fusion.

• International Journal of Contents
• /
• v.18 no.3
• /
• pp.11-20
• /
• 2022

Human Emotion Recognition is an exciting topic that has been attracting many researchers for a lengthy time. In recent years, there has been an increasing interest in exploiting contextual information on emotion recognition. Some previous explorations in psychology show that emotional perception is impacted by facial expressions, as well as contextual information from the scene, such as human activities, interactions, and body poses. Those explorations initialize a trend in computer vision in exploring the critical role of contexts, by considering them as modalities to infer predicted emotion along with facial expressions. However, the contextual information has not been fully exploited. The scene emotion created by the surrounding environment, can shape how people perceive emotion. Besides, additive fusion in multimodal training fashion is not practical, because the contributions of each modality are not equal to the final prediction. The purpose of this paper was to contribute to this growing area of research, by exploring the effectiveness of the emotional scene gist in the input image, to infer the emotional state of the primary target. The emotional scene gist includes emotion, emotional feelings, and actions or events that directly trigger emotional reactions in the input image. We also present an attention-based fusion network, to combine multimodal features based on their impacts on the target emotional state. We demonstrate the effectiveness of the method, through a significant improvement on the EMOTIC dataset.

• International Journal of Fuzzy Logic and Intelligent Systems
• /
• v.6 no.4
• /
• pp.304-313
• /
• 2006

This paper describes a novel way of applying capacitive sensing technology to a mobile user interface. The key idea is to use grip-pattern, which is naturally produced when a user tries to use the mobile device, as a clue to determine an application to be launched. To this end, a capacitive touch sensing system is carefully designed and installed underneath the housing of the mobile device to capture the information of the user's grip-pattern. The captured data is then recognized by dedicated recognition algorithms. The feasibility of the proposed user interface system is thoroughly evaluated with various recognition tests.

• Journal of Convergence for Information Technology
• /
• v.9 no.5
• /
• pp.1-6
• /
• 2019

As the mobile market expands, a variety of platforms are available to provide multimodal media content. Multimodal media content contains heterogeneous data, accordingly, user requires much time and effort to select preferred content. Therefore, in this paper we propose multimodal media content classification using keyword weighting for recommendation. The proposed method extracts keyword that best represent contents through keyword weighting in text data of multimodal media contents. Based on the extracted data, genre class with subclass are generated and classify appropriate multimodal media contents. In addition, the user's preference evaluation is performed for personalized recommendation, and multimodal content is recommended based on the result of the user's content preference analysis. The performance evaluation verifies that it is superiority of recommendation results through the accuracy and satisfaction. The recommendation accuracy is 74.62% and the satisfaction rate is 69.1%, because it is recommended considering the user's favorite the keyword as well as the genre.

• International Journal of Computer Science & Network Security
• /
• v.22 no.7
• /
• pp.268-282
• /
• 2022

One of the most important technologies today is augmented reality technology, it allows users to experience the real world using virtual objects that are combined with the real world. This technology is interesting and has become applied in many sectors such as the shopping and medicine, also it has been included in the sector of education. In the field of education, AR technology has become widely used due to its effectiveness. It has many benefits, such as arousing students' interest in learning imaginative concepts that are difficult to understand. On the other hand, studies have proven that collaborative between students increases learning opportunities by exchanging information, and this is known as Collaborative Learning. The use of multimodal creates a distinctive and interesting experience, especially for students, as it increases the interaction of users with the technologies. The research aims at developing collaborative framework for developing achievement of 6th graders through designing a framework that integrated a collaborative framework with a multimodal input "hand-gesture and touch", considering the development of an effective, fun and easy to use framework with a multimodal interaction in AR technology that was applied to reformulate the genetics and traits lesson from the science textbook for the 6th grade, the first semester, the second lesson, in an interactive manner by creating a video based on the science teachers' consultations and a puzzle game in which the game images were inserted. As well, the framework adopted the cooperative between students to solve the questions. The finding showed a significant difference between post-test and pre-test of the experimental group on the mean scores of the science course at the level of remembering, understanding, and applying. Which indicates the success of the framework, in addition to the fact that 43 students preferred to use the framework over traditional education.

• Proceedings of the KSPS conference
• /
• 2007.05a
• /
• pp.29-32
• /
• 2007

This paper describes a multimodal dialog system that uses Hidden Information State (HIS) method to manage the human-machine dialog. HIS dialog manager is a variation of classic partially observable Markov decision process (POMDP), which provides one of the stochastic dialog modeling frameworks. Because dialog modeling using conventional POMDP requires very large size of state space, it has been hard to apply POMDP to the real domain of dialog system. In HIS dialog manager, system groups the belief states to reduce the size of state space, so that HIS dialog manager can be used in real world domain of dialog system. We adapted this HIS method to Smart-home domain multimodal dialog system.

• Journal of Information Processing Systems
• /
• v.13 no.1
• /
• pp.8-25
• /
• 2017

This survey paper explores the application of multimodal feedback in automated systems for motor learning. In this paper, we review the findings shown in recent studies in this field using rehabilitation and various motor training scenarios as context. We discuss popular feedback delivery and sensing mechanisms for motion capture and processing in terms of requirements, benefits, and limitations. The selection of modalities is presented via our having reviewed the best-practice approaches for each modality relative to motor task complexity with example implementations in recent work. We summarize the advantages and disadvantages of several approaches for integrating modalities in terms of fusion and frequency of feedback during motor tasks. Finally, we review the limitations of perceptual bandwidth and provide an evaluation of the information transfer for each modality.