• ETRI Journal
• /
• v.38 no.6
• /
• pp.1085-1094
• /
• 2016

In this work, we present interactive automultiscopic content with mobile surface haptics for multimodal interaction. Our system consists of a 40-view automultiscopic display and a tablet supporting surface haptics in an immersive room. Animated graphics are projected onto the walls of the room. The 40-view automultiscopic display is placed at the center of the front wall. The haptic tablet is installed at the mobile station to enable the user to interact with the tablet. The 40-view real-time rendering and multiplexing technology is applied by establishing virtual cameras in the convergence layout. Surface haptics rendering is synchronized with three-dimensional (3D) objects on the display for real-time haptic interaction. We conduct an experiment to evaluate user experiences of the proposed system. The results demonstrate that the system's multimodal interaction provides positive user experiences of immersion, control, user interface intuitiveness, and 3D effects.

• Proceedings of the Korean Institute of Navigation and Port Research Conference
• /
• 2006.06b
• /
• pp.265-271
• /
• 2006

According to globalization of world economy by extension of production, sales and distribution all around the world and international transportation system changed into a transport system that puts great importance on speediness and value-added services, international transport system is changing into a Multimodal Transport Routing centered system that systematically connects marine, air and rail transports. Due to such changes production, sales and distribution must be provided in time and Multimodal Transport Routing, which can provide multi-dimensional logistics services to customers of global network, is needed but information connection for international transport and connection system between transport modes are insufficient and can not be activated. Especially in Korea, selection standard of 3rd party logistics companies and transport companies is presented, but logistics exclusive companies, which plan and execute the transportation, can't present a systematic evaluation standard for international Multimodal Transport Routing, selection. Thus, this research surveys important previous studies about Multimodal Transport Routing, selection, derives an evaluation standard for Multimodal Transport Routing, selection through interview with company officials, and presents a theoretical basis for Multimodal Transport Routing, selection through systematic analysis of Multimodal Transport Routing, selection using Analytical Hierarchy Process (AHP).

• KIPS Transactions on Software and Data Engineering
• /
• v.12 no.12
• /
• pp.525-534
• /
• 2023

Gas leak detection system is a key to minimize the loss of life due to the explosiveness and toxicity of gas. Most of the leak detection systems detect by gas sensors or thermal imaging cameras. To improve the performance of gas leak detection system using single-modal methods, the paper propose multimodal approach to gas sensor data and thermal camera data in developing a gas type identification model. MultimodalGasData, a multimodal open-dataset, is used to compare the performance of the four models developed through multimodal approach to gas sensors and thermal cameras with existing models. As a result, 1D CNN and GasNet models show the highest performance of 96.3% and 96.4%. The performance of the combined early fusion model of 1D CNN and GasNet reached 99.3%, 3.3% higher than the existing model. We hoped that further damage caused by gas leaks can be minimized through the gas leak detection system proposed in the study.

• Proceedings of the Korean Society for Emotion and Sensibility Conference
• /
• 2003.05a
• /
• pp.94-100
• /
• 2003

Patients with schizophrenia have thinking disorders such as delusion or hallucination, because they have a deficit in the ability which to systematize and integrate information. Therefore, they cannot integrate or systemize visual, auditory and tactile stimuli. In this study we suggest a virtual reality system for the assessment of cognitive ability of schizophrenia patients, based on the brain multimodal integration model. The virtual reality system provides multimodal stimuli, such as visual and auditory stimuli, to the patient, and can evaluate the patient's multimodal integration and working memory integration abilities by making the patient interpret and react to multimodal stimuli, which must be remembered for a given period of time. The clinical study showed that the virtual reality program developed is comparable to those of the WCST and the SPM.

• Korean System Dynamics Review
• /
• v.11 no.2
• /
• pp.45-75
• /
• 2010

Today, the logistics industry has played a critical role in national economy activities. The low cost and high efficiency of the logistics industry are meaningful in the improvement of national competitiveness and the logistics industry. However, efficiency of logistics is lower than that of the United States and Japan since most quantities are processed in road transportation in Korea. With regard to this, existing studies expected a saving of social and environmental costs due to a decrease of road transportation as well as improvement of logistics productivity due to bulk transportation through activation of rail and costal transport. For the expectation, the existing multimodal transport systems should be improved first. Therefore it aimed to develop scenario-based simulation models of multimodal transport systems for decision makers in charge of improvement in the logistics area. For model development, this study utilized Thinking Process and System Dynamics(TP-SD) methodology.

• Proceedings of the IEEK Conference
• /
• 2000.07a
• /
• pp.249-252
• /
• 2000

An experimental multimodal system combining natural input modes such as speech, lip movement, and gaze is proposed in this paper. It benefits from novel human-compute. interaction (HCI) modalities and from multimodal integration for tackling the problem of the HCI bottleneck. This system allows the user to select menu items on the screen by employing speech recognition, lip reading, and gaze tracking components in parallel. Face tracking is a supplementary component to gaze tracking and lip movement analysis. These key components are reviewed and preliminary results are shown with multimodal integration and user testing on the prototype system. It is noteworthy that the system equipped with gaze tracking and lip reading is very effective in noisy environment, where the speech recognition rate is low, moreover, not stable. Our long term interest is to build a user interface embedded in a commercial car navigation system (CNS).

• Journal of Korea Port Economic Association
• /
• v.22 no.1
• /
• pp.175-198
• /
• 2006

This paper presents a network capacity model that can be used as an analytical tool for strategic planning and resource allocation for multimodal transportation systems. In the context of freight transportation, the multimodal network capacity problem (MNCP) is formulated as a mathematical model of nonlinear bi-level optimization problem. Given network configuration and freight demand for multiple origin-destination pairs, the MNCP model is designed to determine the maximum flow that the network can accommodate. To solve the MNCP, a heuristic solution algorithm is developed on the basis of a linear approximation method. A hypothetical exercise shows that the MNCP model and solution algorithm can be successfully implemented and applied to not only estimate the capacity of multimodal network, but also to identify the capacity gaps over all individual facilities in the network, including intermodal facilities. Transportation agencies and planners would benefit from the MNCP model in identifying investment priorities and thus developing sustainable transportation systems in a manner that considers all feasible modes as well as low-cost capacity improvements.

• Proceedings of the KSPS conference
• /
• 2007.05a
• /
• pp.29-32
• /
• 2007

This paper describes a multimodal dialog system that uses Hidden Information State (HIS) method to manage the human-machine dialog. HIS dialog manager is a variation of classic partially observable Markov decision process (POMDP), which provides one of the stochastic dialog modeling frameworks. Because dialog modeling using conventional POMDP requires very large size of state space, it has been hard to apply POMDP to the real domain of dialog system. In HIS dialog manager, system groups the belief states to reduce the size of state space, so that HIS dialog manager can be used in real world domain of dialog system. We adapted this HIS method to Smart-home domain multimodal dialog system.

• Science of Emotion and Sensibility
• /
• v.6 no.3
• /
• pp.55-62
• /
• 2003

Patients with schizophrenia have thinking disorders such as delusion or hallucination, because they have a deficit in the ability which to systematize and integrate information. therefore, they cannot integrate or systematize visual, auditory and tactile stimuli. In this study, we suggest a virtual reality system for the assessment of cognitive ability of schizophrenia patients, based on the brain multimodal integration model. The virtual reality system provides multimodal stimuli, such as visual and auditory stimuli, to the patient, and can evaluate the patient's multimodal integration and working memory integration abilities by making the patient interpret and react to multimodal stimuli, which must be remembered for a given period of time. the clinical study showed that the virtual reality program developed is comparable to those of the WCST and the SPM.

• Journal of Korea Multimedia Society
• /
• v.16 no.3
• /
• pp.302-308
• /
• 2013

In this research, we have explored personal authentication system through multimodal biometrics for mobile computing environment. We have selected face and speaker recognition for the implementation of multimodal biometrics system. For face recognition part, we detect the face with Modified Census Transform (MCT). Detected face is pre-processed through eye detection module based on k-means algorithm. Then we recognize the face with Principal Component Analysis (PCA) algorithm. For speaker recognition part, we extract features using the end-point of voice and the Mel Frequency Cepstral Coefficient (MFCC). Then we verify the speaker through Dynamic Time Warping (DTW) algorithm. Our proposed multimodal biometrics system shows improved verification rate through combining two different biometrics described above. We implement our proposed system based on Android environment using Galaxy S hoppin. Proposed system presents reduced false acceptance ratio (FAR) of 1.8% which shows improvement from single biometrics system using the face and the voice (presents 4.6% and 6.7% respectively).