• 제어로봇시스템학회:학술대회논문집
• /
• 1998.10a
• /
• pp.250-253
• /
• 1998

This paper presents a multi-modal neural network composed of a preprocessing module and a multi-layer neural network module in order to enhance the nonlinear characteristics of neural network. The former module is based on spectral method using Chebyschev polynomials and transforms input data into spectra. The latter module identifies the system using the spectra generated by the preprocessing module. The omnibus numerical experiments show that the method is applicable to many a nonlinear dynamic system in the real world, and that preprocessing using Chebyschev polynomials reduces the number of neurons required for the multi-layer neural network.

• 제어로봇시스템학회:학술대회논문집
• /
• 1998.10a
• /
• pp.201-204
• /
• 1998

This paper presents a new construction method of candidate controllers using Multi-modal Neural Network(MNN). To improve a control performance of multiple controller, we construct, candidate controllers which consist of MNN. MNN can learn more complicated function than multilayer neural network. MNN consists of preprocessing module and neural network module. The preprocessing module transforms input signals into spectra which are used as input of the following neural network module. We apply the proposed method to multiple control system which controls the cart-pole balancing system and show the effectiveness of the proposed method.

• KIISE Transactions on Computing Practices
• /
• v.23 no.2
• /
• pp.104-109
• /
• 2017

We propose a new machine learning algorithm which analyzes daily activity patterns of users from multi-modal wearable sensor data. The proposed model learns and extracts activity patterns using input from wearable devices in real-time. Inspired by cue integration of human's property, we constructed gated multi-modal neural networks which integrate wearable sensor input data selectively by using gate modules. For the experiments, sensory data were collected by using multiple wearable devices in restaurant situations. As an experimental result, we first show that the proposed model performs well in terms of prediction accuracy. Then, the possibility to construct a knowledge schema automatically by analyzing the activation patterns in the middle layer of our proposed model is explained.

• Journal of Korea Multimedia Society
• /
• v.23 no.2
• /
• pp.146-154
• /
• 2020

Negative emotion causes stress and lack of attention concentration. The classification of negative emotion is important to recognize risk factors. To classify emotion status, various methods such as questionnaires and interview are used and it could be changed by personal thinking. To solve the problem, we acquire multi modal bio-signals such as electrocardiogram (ECG), skin temperature (ST), galvanic skin response (GSR) and extract features. The neural network (NN), the deep neural network (DNN), and the deep belief network (DBN) is designed using the multi modal bio-signals to analyze emotion status. As a result, the DBN based on features extracted from ECG, ST and GSR shows the highest accuracy (93.8%). It is 5.7% higher than compared to the NN and 1.4% higher than compared to the DNN. It shows 12.2% higher accuracy than using only single bio-signal (GSR). The multi modal bio-signal acquisition and the deep learning classifier play an important role to classify emotion.

• KIPS Transactions on Software and Data Engineering
• /
• v.6 no.4
• /
• pp.203-210
• /
• 2017

In this paper, we propose an effective neural network model for image caption generation and model transfer. This model is a kind of multi-modal recurrent neural network models. It consists of five distinct layers: a convolution neural network layer for extracting visual information from images, an embedding layer for converting each word into a low dimensional feature, a recurrent neural network layer for learning caption sentence structure, and a multi-modal layer for combining visual and language information. In this model, the recurrent neural network layer is constructed by LSTM units, which are well known to be effective for learning and transferring sequence patterns. Moreover, this model has a unique structure in which the output of the convolution neural network layer is linked not only to the input of the initial state of the recurrent neural network layer but also to the input of the multimodal layer, in order to make use of visual information extracted from the image at each recurrent step for generating the corresponding textual caption. Through various comparative experiments using open data sets such as Flickr8k, Flickr30k, and MSCOCO, we demonstrated the proposed multimodal recurrent neural network model has high performance in terms of caption accuracy and model transfer effect.

• Journal of Advanced Navigation Technology
• /
• v.23 no.6
• /
• pp.526-534
• /
• 2019

Localization methods using radio signals should obtain range measurements from three or more anchors. However, a typical building consists of narrow, long hallways and corners, making it difficult to secure more than three light of sight anchors. The result is a multi-modal solution that makes it difficult to estimate the user's location. In order to overcome this problem, this paper proposes a method for estimating the location using artificial neural networks. Using the artificial neural network, even if a multi-modal solution occurs, the position can be estimated by acquiring user movement pattern information based on accumulated range measurements. The method does not require any additional equipment or sensors, and only anchor-based range measurements can estimate the user's location. In order to verify the proposed method, location estimation tests were performed in situations where the multi-modal solution occurred by installing an insufficient number of anchors in a building. As a result, it was confirmed that the location can be estimated even when the number of anchors is insufficient.

• Transactions of the Korean Society for Noise and Vibration Engineering
• /
• v.22 no.7
• /
• pp.667-675
• /
• 2012

A damage estimation method for support structure of offshore wind turbine using modal parameters is presented for effective structural health monitoring. Natural frequencies and mode shapes for a support structure with monopile of an offshore wind turbine were calculated considering soil condition and added mass. A neural network was learned based on training patterns generated by the changes of natural frequency and mode shape due to various damages. Natural frequencies and mode shapes for 10 prospective damage cases were input to the trained neural network for damage estimation. The identified damage locations and severities agreed reasonably well with the accurate damages. Multi-damage cases could also be successfully estimated. Enhancement of estimation result using another parameters as input to neural network will be carried out by further study. Proposed method could be applied to other type of support structure of offshore wind turbine for structural health monitoring.

• Journal of Broadcast Engineering
• /
• v.23 no.3
• /
• pp.351-360
• /
• 2018

Human emotion recognition is a research topic that is receiving continuous attention in computer vision and artificial intelligence domains. This paper proposes a method for classifying human emotions through multiple neural networks based on multi-modal signals which consist of image, landmark, and audio in a wild environment. The proposed method has the following features. First, the learning performance of the image-based network is greatly improved by employing both multi-task learning and semi-supervised learning using the spatio-temporal characteristic of videos. Second, a model for converting 1-dimensional (1D) landmark information of face into two-dimensional (2D) images, is newly proposed, and a CNN-LSTM network based on the model is proposed for better emotion recognition. Third, based on an observation that audio signals are often very effective for specific emotions, we propose an audio deep learning mechanism robust to the specific emotions. Finally, so-called emotion adaptive fusion is applied to enable synergy of multiple networks. The proposed network improves emotion classification performance by appropriately integrating existing supervised learning and semi-supervised learning networks. In the fifth attempt on the given test set in the EmotiW2017 challenge, the proposed method achieved a classification accuracy of 57.12%.

• Proceedings of the IEEK Conference
• /
• 1998.10a
• /
• pp.1289-1292
• /
• 1998

HCI(human computer interface) technologies have been often implemented using mouse, keyboard and joystick. Because mouse and keyboard are used only in limited situation, More natural HCI methods such as speech based method and gesture based method recently attract wide attention. In this paper, we present multi-modal input system to control Windows system for practical use of multi-media computer. Our multi-modal input system consists of three parts. First one is virtual-hand mouse part. This part is to replace mouse control with a set of gestures. Second one is Windows control system using speech recognition. Third one is Windows control system using gesture recognition. We introduce neural network and HMM methods to recognize speeches and gestures. The results of three parts interface directly to CPU and through Windows.

• International Journal of Precision Engineering and Manufacturing
• /
• v.3 no.1
• /
• pp.43-48
• /
• 2002

This paper describes an automated computer-aided engineering (CAE) system for three-dimensional structures. An automatic finite element mesh-generation technique, which is based on fuzzy knowledge processing and computational geometry techniques, is incorporated into the system, together with a commercial FE analysis code, and a commercial solid modeler. The system allows a geometry model of interest to be automatically converted to different FE models, depending on the physical phenomena of the structures to be analyzed, i.e., electrostatic analysis, stress analysis, modal analysis, and so on. Also, with the aid of multilayer neural networks, the present system allows us to obtain automatically a design window in which a number of satisfactory design solutions exist in a multi-dimensional design parameter space. The developed CAE system is successfully applied to evaluate an electrostatic micromachines.