• Proceedings of the IEEK Conference
• /
• 2006.06a
• /
• pp.875-876
• /
• 2006

In this paper, we propose a multimodal biometric embedded system. It is designed to support face, iris, fingerprint and vascular pattern recognition. We use a S3C2440A based on ARM926T core processor that is made in Samsung. The system has support various external device interfaces for multi biometric sensors, and RFID/Smart Card reader/writer. Additionally, it has a 6" LCD panel and numeric keypad for easy GUI. The embedded system offers useful environments to develop better biometric algorithms for stand alone biometric system and accelerator hardware modules for real time operation.

• The Journal of Korean Institute of Communications and Information Sciences
• /
• v.40 no.1
• /
• pp.132-141
• /
• 2015

There might have been weakness in securing user authentication or verification with real time service approach, while existing unimodal biometric authentication has been used mainly for user identification and recognition. Accordingly, it is essential to research and develop ways that upgrade security performance with multi biometric based real time authentication and verification technology. This paper focused to suggest binding assignment and strategy for developing multi biometric authentication technology through investigation of advanced study and patents. Description includes introduction, technology outline, technology trend, patent analysis, and conclusion.

• Journal of KIISE:Computing Practices and Letters
• /
• v.9 no.4
• /
• pp.420-430
• /
• 2003

This paper presents an immersive multimodal Gumdo simulation game that allows a user to experience the whole body interaction with an intelligent cyber fencer. The proposed system consists of three modules: (i) a nondistracting multimodal interface with 3D vision and speech (ii) an intelligent cyber fencer and (iii) an immersive feedback by a big screen and sound. First, the multimodal Interface with 3D vision and speech allows a user to move around and to shout without distracting the user. Second, an intelligent cyber fencer provides the user with intelligent interactions by perception and reaction modules that are created by the analysis of real Gumdo game. Finally, an immersive audio-visual feedback by a big screen and sound effects helps a user experience an immersive interaction. The proposed system thus provides the user with an immersive Gumdo experience with the whole body movement. The suggested system can be applied to various applications such as education, exercise, art performance, etc.

• Journal of Intelligence and Information Systems
• /
• v.25 no.1
• /
• pp.163-177
• /
• 2019

As smartphones are getting widely used, human activity recognition (HAR) tasks for recognizing personal activities of smartphone users with multimodal data have been actively studied recently. The research area is expanding from the recognition of the simple body movement of an individual user to the recognition of low-level behavior and high-level behavior. However, HAR tasks for recognizing interaction behavior with other people, such as whether the user is accompanying or communicating with someone else, have gotten less attention so far. And previous research for recognizing interaction behavior has usually depended on audio, Bluetooth, and Wi-Fi sensors, which are vulnerable to privacy issues and require much time to collect enough data. Whereas physical sensors including accelerometer, magnetic field and gyroscope sensors are less vulnerable to privacy issues and can collect a large amount of data within a short time. In this paper, a method for detecting accompanying status based on deep learning model by only using multimodal physical sensor data, such as an accelerometer, magnetic field and gyroscope, was proposed. The accompanying status was defined as a redefinition of a part of the user interaction behavior, including whether the user is accompanying with an acquaintance at a close distance and the user is actively communicating with the acquaintance. A framework based on convolutional neural networks (CNN) and long short-term memory (LSTM) recurrent networks for classifying accompanying and conversation was proposed. First, a data preprocessing method which consists of time synchronization of multimodal data from different physical sensors, data normalization and sequence data generation was introduced. We applied the nearest interpolation to synchronize the time of collected data from different sensors. Normalization was performed for each x, y, z axis value of the sensor data, and the sequence data was generated according to the sliding window method. Then, the sequence data became the input for CNN, where feature maps representing local dependencies of the original sequence are extracted. The CNN consisted of 3 convolutional layers and did not have a pooling layer to maintain the temporal information of the sequence data. Next, LSTM recurrent networks received the feature maps, learned long-term dependencies from them and extracted features. The LSTM recurrent networks consisted of two layers, each with 128 cells. Finally, the extracted features were used for classification by softmax classifier. The loss function of the model was cross entropy function and the weights of the model were randomly initialized on a normal distribution with an average of 0 and a standard deviation of 0.1. The model was trained using adaptive moment estimation (ADAM) optimization algorithm and the mini batch size was set to 128. We applied dropout to input values of the LSTM recurrent networks to prevent overfitting. The initial learning rate was set to 0.001, and it decreased exponentially by 0.99 at the end of each epoch training. An Android smartphone application was developed and released to collect data. We collected smartphone data for a total of 18 subjects. Using the data, the model classified accompanying and conversation by 98.74% and 98.83% accuracy each. Both the F1 score and accuracy of the model were higher than the F1 score and accuracy of the majority vote classifier, support vector machine, and deep recurrent neural network. In the future research, we will focus on more rigorous multimodal sensor data synchronization methods that minimize the time stamp differences. In addition, we will further study transfer learning method that enables transfer of trained models tailored to the training data to the evaluation data that follows a different distribution. It is expected that a model capable of exhibiting robust recognition performance against changes in data that is not considered in the model learning stage will be obtained.

• Journal of Electrical Engineering and Technology
• /
• v.9 no.6
• /
• pp.2118-2125
• /
• 2014

Gait Recognition is a new technique to identify the people by the way they walk. Human gait is a spatio-temporal phenomenon that typifies the motion characteristics of an individual. The proposed method makes a simple but efficient attempt to gait recognition. For each video file, spatial silhouettes of a walker are extracted by an improved background subtraction procedure using Gaussian Mixture Model (GMM). Here GMM is used as a parametric probability density function represented as a weighted sum of Gaussian component densities. Then, the relevant features are extracted from the silhouette tracked from the given video file using the Principal Component Analysis (PCA) method. The Fisher Linear Discriminant Analysis (FLDA) classifier is used in the classification of dimensional reduced image derived by the PCA method for gait recognition. Although gait images can be easily acquired, the gait recognition is affected by clothes, shoes, carrying status and specific physical condition of an individual. To overcome this problem, it is combined with footprint as a multimodal biometric system. The minutiae is extracted from the footprint and then fused with silhouette image using the Discrete Stationary Wavelet Transform (DSWT). The experimental result shows that the efficiency of proposed fusion algorithm works well and attains better result while comparing with other fusion schemes.

• Journal of The Korean Association For Science Education
• /
• v.34 no.5
• /
• pp.479-490
• /
• 2014

The purpose of this study is to develop an argument-based modeling strategy, utilizing writing and argumentation for communication in science education. We need to support students and teachers who have difficulty in modeling in science education, this strategy focuses on development of four kinds of factors as follows: First, awareness of problems, recognizing in association with problems by observing several problematic situations. Second is science concept structuralization suggesting enough science concepts by organization for scientific explanation. The third is claim-evidence appropriateness that suggests appropriate representation as evidence for assertions. Last, the use of various representations and multimodal representations that converts and integrates these representations in evidence suggestion. For the development of these four factors, this study organized three stages. 'Recognition process' for understanding of multimodal representations, and 'Interpretation process' for understanding of activity according to multimodal representations, 'Application process' for understanding of modeling through argumentation. This application process has been done with eight stages of 'Asking questions or problems - Planning experiment - Investigation through observation on experiment - Analyzing and interpreting data - Constructing pre-model - Presenting model - Expressing model using multimodal representations - Evaluating model - Revising model'. After this application process, students could have opportunity to form scientific knowledge by making their own model as scientific explanation system for the phenomenon of the natural world they observed during a series of courses of modeling.

• 한국HCI학회:학술대회논문집
• /
• 2009.02a
• /
• pp.512-515
• /
• 2009

In this paper, we propose layout based multimodal contents authoring tool for Digilog Book. In authoring step, users create a virtual area using mouse or pen-type device and select property of the area repetitively. After finishing authoring step, system recognizes printed page number and generate page layout including areas and property information. Page layout is represented as a scene graph and stored as XML format. Digilog Book viewer loads stored page layout and analyze properties then augment virtual contents or execute functions based on area. Users can author visual and auditory contents easily by using hybrid interface. In AR environment, system provides area templates in order to help creating area. In addition, proposed authoring tool separates page recognition module from page tracking module. So, it is possible to author many pages using only single marker. As a result of experiment, we showed proposed authoring tool has reasonable performance time in AR environment. We expect that proposed authoring tool would be applicable to many fields such as education and publication.

• The Journal of the Korea Contents Association
• /
• v.22 no.1
• /
• pp.138-147
• /
• 2022

Recently, chatbots have been applied in various fields and have shown good results, and many attempts to use chatbots in shopping mall product recommendation services are being conducted on e-commerce platforms. In this paper, for a conversation system that recommends a fashion that a user wants based on conversation between the user and the system and fashion image information, a transformer model that is currently performing well in various AI fields such as natural language processing, voice recognition, and image recognition. We propose a multimodal-based improved transformer model that is improved to increase the accuracy of recommendation by using dialogue (text) and fashion (image) information together for data preprocessing and data representation. We also propose a method to improve accuracy through data improvement by analyzing the data. The proposed system has a recommendation accuracy score of 0.6563 WKT (Weighted Kendall's tau), which significantly improved the existing system's 0.3372 WKT by 0.3191 WKT or more.

• 한국HCI학회:학술대회논문집
• /
• 2008.02a
• /
• pp.348-353
• /
• 2008

This paper proposes a novel touch interface for recognizing user's touch pattern and understanding emotional information by eliciting natural user interaction. To classify physical touches, we represent the similarity between touches by analyzing touches based on its dictionary meaning and design the algorithm to recognize various touch patterns in real time. Finally we suggest the methodology to estimate user's emotional state based on touch.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2001.10a
• /
• pp.729-732
• /
• 2001

Individual authentication system that take advantage of multimodal information is very efficient method that can take advantage of method of speech recognition, face recognition, electron signature etc. and protect important information from much dangers that exits on communication network whole as skill that construct security system. This paper deal product connected with hardware from internet space based on public key sign and electron signature description embodied system. Maintenance of public security is explaining that commercial transaction system implementation that is considered is possible as applying individual authentication.