• KIISE Transactions on Computing Practices
• /
• v.23 no.2
• /
• pp.104-109
• /
• 2017

We propose a new machine learning algorithm which analyzes daily activity patterns of users from multi-modal wearable sensor data. The proposed model learns and extracts activity patterns using input from wearable devices in real-time. Inspired by cue integration of human's property, we constructed gated multi-modal neural networks which integrate wearable sensor input data selectively by using gate modules. For the experiments, sensory data were collected by using multiple wearable devices in restaurant situations. As an experimental result, we first show that the proposed model performs well in terms of prediction accuracy. Then, the possibility to construct a knowledge schema automatically by analyzing the activation patterns in the middle layer of our proposed model is explained.

• Journal of Korea Multimedia Society
• /
• v.23 no.8
• /
• pp.1006-1018
• /
• 2020

Dataset is getting more import to develop a learning based algorithm. Quality of the algorithm definitely depends on dataset. So we introduce new dataset over 200 thousands images which are fully labeled multi modal sensor data. Proposed dataset was designed and constructed for researchers who want to develop detection, tracking, and action classification in outdoor environment for surveillance scenarios. The dataset includes various images and multi modal sensor data under different weather and lighting condition. Therefor, we hope it will be very helpful to develop more robust algorithm for systems equipped with difference kinds of sensors in outdoor application. Case studies with the proposed dataset are also discussed in this paper.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.3 no.4
• /
• pp.344-365
• /
• 2009

The performance of a tracking system is greatly increased if multiple types of sensors are combined to achieve the objective of the tracking instead of relying on single type of sensor. To conduct the multi-modal tracking, we have previously developed a multi-modal sensor-based tracking model where acoustic sensors mainly track the objects and visual sensors compensate the tracking errors [1]. In this paper, we find a network synchronization problem appearing in the developed tracking system. The problem is caused by the different location and traffic characteristics of multi-modal sensors and non-synchronized arrival of the captured sensor data at a processing server. To effectively deliver the sensor data, we propose a time-based packet aggregation algorithm where the acoustic sensor data are aggregated based on the sampling time and sent to the server. The delivered acoustic sensor data is then compensated by visual images to correct the tracking errors and such a compensation process improves the tracking accuracy in ideal case. However, in real situations, the tracking improvement from visual compensation can be severely degraded due to the aforementioned network synchronization problem, the impact of which is analyzed by simulations in this paper. To resolve the network synchronization problem, we differentiate the service level of sensor traffic based on Weight Round Robin (WRR) scheduling at the routers. The weighting factor allocated to each queue is calculated by a proposed Delay-based Weight Allocation (DWA) algorithm. From the simulations, we show the traffic differentiation model can mitigate the non-synchronization of sensor data. Finally, we analyze expected traffic behaviors of the tracking system in terms of acoustic sampling interval and visual image size.

• Smart Structures and Systems
• /
• v.14 no.1
• /
• pp.55-70
• /
• 2014

The paper presents a multi-objective optimization strategy for a multi-type sensor placement for Structural Health Monitoring (SHM) of long span bridges. The problem is formulated for simultaneous placement of strain sensors and accelerometers (heterogeneous network) based on application demands for SHM system. Modal Identification (MI) and Accurate Mode Shape Expansion (AMSE) were chosen as the application demands for SHM. The optimization problem is solved through the use of integer Genetic Algorithm (GA) to maximize a common metric to ensure adequate MI and AMSE. The performance of the joint optimization problem solved by GA is compared with other established methods for homogenous sensor placement. The results indicate that the use of a multi-type sensor system can improve the quality of SHM. It has also been demonstrated that use of GA improves the overall quality of the sensor placement compared to other methods for optimization of sensor placement.

• Journal of Institute of Control, Robotics and Systems
• /
• v.18 no.8
• /
• pp.780-786
• /
• 2012

Surveillance has become one of promising application areas of wireless sensor networks which allow for pervasive monitoring of concerned environmental phenomena by facilitating context awareness through sensor fusion. Existing systems that depend on a postmortem context analysis of sensor data on a centralized server expose several shortcomings, including a single point of failure, wasteful energy consumption due to unnecessary data transfer as well as deficiency of scalability. As an opposite direction, this paper proposes an energy-efficient distributed context-aware surveillance in which sensor nodes in the wireless sensor network collaborate with neighbors in a distributed manner to analyze and aware surrounding context. We design and implement multi-modal sensor stations for use as sensor nodes in our wireless sensor network implementing our distributed context awareness. This paper presents an initial experimental performance result of our proposed system. Results show that multi-modal sensor performance of our sensor station, a key enabling factor for distributed context awareness, is comparable to each independent sensor setting. They also show that its initial performance of context-awareness is satisfactory for a set of introductory surveillance scenarios in the current interim stage of our ongoing research.

• Journal of KSNVE
• /
• v.8 no.4
• /
• pp.632-642
• /
• 1998

A method based on finite element discretization is developed for optimizing the polarization profile of PVDF film to create the modal transducer for specific modes. Using this concept, one can design the modal transducer in two-dimensional structure having arbitrary geometry and boundary conditions. As a practical means for implementing this polarization profile without repoling the PVDF film the polarization profile is approximated by optimizing electrode patterns, lamination angles, and poling directions of the multi-layered PVDF transducer. This corresponds to the approximation of a continuous function using discrete values. The electrode pattern of each PVDF layer is optimized by deciding the electrode of each finite element to be used or not. Genetic algorithm, suitable for discrete problems, is used as an optimization scheme. For the optimization of each layers lamination angle, the continuous lamination angle is encoded into discrete value using binary 5 bit string. For the experimental demonstration, a modal sensor for first and second modes of cantilevered composite plate is designed using two layers of PVDF films. The actuator is designed based on the criterion of minimizing the system energy in the control modes under a given initial condition. Experimental results show that the signals from residual modes are successfully reduced using the optimized multi-layered PVDF sensor. Using discrete LQG control law, the modal peaks of first and second modes are reduced in the amount of 12 dB and 4 dB, resepctively.

• Journal of the Ergonomics Society of Korea
• /
• v.31 no.4
• /
• pp.499-505
• /
• 2012

Objective: Research on the transformation of interaction between mobile machines and users through analysis on current gesture interface technology development trend. Background: For smooth interaction between machines and users, interface technology has evolved from "command line" to "mouse", and now "touch" and "gesture recognition" have been researched and being used. In the future, the technology is destined to evolve into "multi-modal", the fusion of the visual and auditory senses and "3D multi-modal", where three dimensional virtual world and brain waves are being used. Method: Within the development of computer interface, which follows the evolution of mobile machines, actively researching gesture interface and related technologies' trend and development will be studied comprehensively. Through investigation based on gesture based information gathering techniques, they will be separated in four categories: sensor, touch, visual, and multi-modal gesture interfaces. Each category will be researched through technology trend and existing actual examples. Through this methods, the transformation of mobile machine and human interaction will be studied. Conclusion: Gesture based interface technology realizes intelligent communication skill on interaction relation ship between existing static machines and users. Thus, this technology is important element technology that will transform the interaction between a man and a machine more dynamic. Application: The result of this study may help to develop gesture interface design currently in use.

• Structural Engineering and Mechanics
• /
• v.37 no.6
• /
• pp.671-684
• /
• 2011

Deciding on an optimal sensor placement (OSP) is a common problem encountered in many engineering applications and is also a critical issue in the construction and implementation of an effective structural health monitoring (SHM) system. The present study focuses with techniques for selecting optimal sensor locations in a sensor network designed to monitor the health condition of Dalian World Trade Building which is the tallest in the northeast of China. Since the number of degree-of-freedom (DOF) of the building structure is too large, multi-modes should be selected to describe the dynamic behavior of a structural system with sufficient accuracy to allow its health state to be determined effectively. However, it's difficult to accurately distinguish the translational and rotational modes for the flexible structures with closely spaced modes by the modal participation mass ratios. In this paper, a new method of the OSP that computing the mode shape matrix in the weak axis of structure by the simplified multi-DOF system was presented based on the equivalent rigidity parameter identification method. The initial sensor assignment was obtained by the QR-factorization of the structural mode shape matrix. Taking the maximum off-diagonal element of the modal assurance criterion (MAC) matrix as a target function, one more sensor was added each time until the maximum off-diagonal element of the MAC reaches the threshold. Considering the economic factors, the final plan of sensor placement was determined. The numerical example demonstrated the feasibility and effectiveness of the proposed scheme.

• Transactions of the Korean Society of Mechanical Engineers A
• /
• v.20 no.10
• /
• pp.3173-3185
• /
• 1996

Multi-model vibration control of laminated composites plates for various fiver orientations has been carried out by making use of piezolectric materials(PZT) as sensors and actuators. Cantilever plate is used as a specimen to test multi-modal vibration supression under random exitation. Impulse technique is applied to determine the natural frequency, the damping ratio(.zeta.) and the modal damping(2.zeta..omega.) of the first bending and the trosion modes. Two independent controllers are implemented to control the two modes simultaneously and established digitally on the basis of the direct negative velocity feedback control with collocated sensor/actuator. Experimental results for various fiber orientations and feedback gains are compared with finite element analysis considering stiffnesses and dampings of piezoeletiric sensors, actuators and bonding layer.

• Journal of the Institute of Electronics Engineers of Korea SP
• /
• v.46 no.4
• /
• pp.76-81
• /
• 2009

We propose an effective emotion recognition algorithm based on the minimum classification error (MCE) incorporating multi-modal system The emotion recognition is performed based on a Gaussian mixture model (GMM) based on MCE method employing on log-likelihood. In particular, the reposed technique is based on the fusion of feature vectors based on voice signal and galvanic skin response (GSR) from the body sensor. The experimental results indicate that performance of the proposal approach based on MCE incorporating the multi-modal system outperforms the conventional approach.