• Title/Summary/Keyword: multimodal system

Search Result 225, Processing Time 0.019 seconds

A Bio-Edutainment System to Virus-Vaccine Discovery based on Collaborative Molecular in Real-Time with VR

  • Park, Sung-Jun
    • Journal of the Korea Society of Computer and Information
    • /
    • v.25 no.6
    • /
    • pp.109-117
    • /
    • 2020
  • An edutainment system aims to help learners to recognize problems effectively, grasp and classify important information needed to solve the problems and convey the contents of what they have learned. Edutainment contents can be usefully applied to education and training in the both scientific and industrial areas. Our present work proposes an edutainment system that can be applied to a drug discovery process including virtual screening by using intuitive multi-modal interfaces. In this system, a stereoscopic monitor is used to make three-dimensional (3D) macro-molecular images, with supporting multi-modal interfaces to manipulate 3D models of molecular structures effectively. In this paper, our system can easily solve a docking simulation function, which is one of important virtual drug screening methods, by applying gaming factors. The level-up concept is implemented to realize a bio-game approach, in which the gaming factor depends on number of objects and users. The quality of the proposed system is evaluated with performance comparison in terms of a finishing time of a drug docking process to screen new inhibitors against target proteins of human immunodeficiency virus (HIV) in an e-drug discovery process.

The Interactive Learning Experience by Integrating Educational Robots into the Augmented Reality (교육용 로봇과 증강 현실 결합을 통한 인터랙티브 학습 경험)

  • Yu, Jeong Su
    • Journal of The Korean Association of Information Education
    • /
    • v.16 no.4
    • /
    • pp.419-427
    • /
    • 2012
  • This paper presents the effect of a interactive learning experience and student's response to technological components We develop the interactive learning environment and learning model in lessons relying on educational robots and augmented reality in the school classroom. The developed learning model is based on the problem-based learning model. The experiments of the study conduct with 18 students, the $5^{th}$ and $6^{th}$ graders of an elementary school for 8 weeks using developed system. We find out the interactive learning experiences have an influence on the creative ability of children. We know that students who scored lower on the school exam scored higher on the score of creativity compared to top students through educational robots and augmented reality.

  • PDF

Expression Analysis System of Game Player based on Multi-modal Interface (멀티 모달 인터페이스 기반 플레이어 얼굴 표정 분석 시스템 개발)

  • Jung, Jang-Young;Kim, Young-Bin;Lee, Sang-Hyeok;Kang, Shin-Jin
    • Journal of Korea Game Society
    • /
    • v.16 no.2
    • /
    • pp.7-16
    • /
    • 2016
  • In this paper, we propose a method for effectively detecting specific behavior. The proposed method detects outlying behavior based on the game players' characteristics. These characteristics are captured non-invasively in a general game environment and add keystroke based on repeated pattern. In this paper, cameras were used to analyze observed data such as facial expressions and player movements. Moreover, multimodal data from the game players was used to analyze high-dimensional game-player data for a detection effect of repeated behaviour pattern. A support vector machine was used to efficiently detect outlying behaviors. We verified the effectiveness of the proposed method using games from several genres. The recall rate of the outlying behavior pre-identified by industry experts was approximately 70%. In addition, Repeated behaviour pattern can be analysed possible. The proposed method can also be used for feedback and quantification about analysis of various interactive content provided in PC environments.

Structural health monitoring of high-speed railway tracks using diffuse ultrasonic wave-based condition contrast: theory and validation

  • Wang, Kai;Cao, Wuxiong;Su, Zhongqing;Wang, Pengxiang;Zhang, Xiongjie;Chen, Lijun;Guan, Ruiqi;Lu, Ye
    • Smart Structures and Systems
    • /
    • v.26 no.2
    • /
    • pp.227-239
    • /
    • 2020
  • Despite proven effectiveness and accuracy in laboratories, the existing damage assessment based on guided ultrasonic waves (GUWs) or acoustic emission (AE) confronts challenges when extended to real-world structural health monitoring (SHM) for railway tracks. Central to the concerns are the extremely complex signal appearance due to highly dispersive and multimodal wave features, restriction on transducer installations, and severe contaminations of ambient noise. It remains a critical yet unsolved problem along with recent attempts to implement SHM in bourgeoning high-speed railway (HSR). By leveraging authors' continued endeavours, an SHM framework, based on actively generated diffuse ultrasonic waves (DUWs) and a benchmark-free condition contrast algorithm, has been developed and deployed via an all-in-one SHM system. Miniaturized lead zirconate titanate (PZT) wafers are utilized to generate and acquire DUWs in long-range railway tracks. Fatigue cracks in the tracks show unique contact behaviours under different conditions of external loads and further disturb DUW propagation. By contrast DUW propagation traits, fatigue cracks in railway tracks can be characterised quantitatively and the holistic health status of the tracks can be evaluated in a real-time manner. Compared with GUW- or AE-based methods, the DUW-driven inspection philosophy exhibits immunity to ambient noise and measurement uncertainty, less dependence on baseline signals, use of significantly reduced number of transducers, and high robustness in atrocious engineering conditions. Conformance tests are performed on HSR tracks, in which the evolution of fatigue damage is monitored continuously and quantitatively, demonstrating effectiveness, adaptability, reliability and robustness of DUW-driven SHM towards HSR applications.

The Effects of Virtual Reality-based Continuous Slow Exercise on Factors for Falls in the Elderly (가상현실에서 연속적 느린 운동이 노인의 낙상 요인에 미치는 영향)

  • Kim, Jung-Jin;Gu, Seul;Lee, Jin-Ju;Kim, Yu-Shin;Yoon, Bum-Chul
    • The Journal of Korean Physical Therapy
    • /
    • v.24 no.2
    • /
    • pp.90-97
    • /
    • 2012
  • Purpose: The purpose of this study was to assess the effects of virtual reality-based continuous slow exercise on muscle strength and dynamic balance capacity, in older adults over 65 years of age. Methods: Twenty-six volunteers were randomly divided into two groups; a Virtual Reality (VR) exercise-group ($67.8{\pm}4.1$ yrs) and a Control group ($65.5{\pm}5.2$ yrs). The VR group participated in eight weeks of virtual reality exercise, utilizing modified Tai-Chi provided by a motion capture system, and the Control group had no intervention. The hip muscle strength and dynamic balance of the members of both the VR group and the Control group were measured at pre- and post-intervention, using a multimodal dynamometer, and backward stepping test, respectively. Results: 1. After the 8-week VR-based exercise, the VR group showed significant improvement of hip strength, compared to the control group: hip extension (p=0.00), flexion (p=0.00), abduction (p=0.00), and adduction (p=0.00). 2. After the 8-week VR-based exercise, the VR group showed significant improvement of dynamic balance capacity as ground reaction force, compared to the control group. Eyes opened backward stepping test: Fx (+) (p=0.00), Fy (-) (p=0.02), Ver (+) (p=0.02) direction. Eyes closed backward stepping test: Fx (+) (p=0.04), Fy (-) (p=0.04), Ver (+) (p=0.03) direction. Conclusion: The VR group showed improvement of their hip muscle strength, and dynamic balance capacity. Therefore VR-based continuous slow exercise would contribute to reducing the risk of falls in the elderly.

Sentinel lymph node mapping using tri-modal human serum albumin conjugated with visible dye, near infrared fluorescent dye and radioisotope

  • Kang, Se Hun;Kim, Seo-il;Jung, So-Youn;Lee, Seeyoun;Kim, Seok Won;Kim, Seok-ki
    • Journal of Radiopharmaceuticals and Molecular Probes
    • /
    • v.1 no.1
    • /
    • pp.62-73
    • /
    • 2015
  • We developed an evans blue-indocyanine green-$^{99m}Tc$-human serum albumin conjugate for sentinel lymph node mapping and we describe its unique potential usage for clinical implications. This conjugate has combined the strengths of visible blue dye, near-infrared fluorescence and radioisotope into one single conjugate without any additional weakness/disadvantage. All the components of evans blue-indocyanine green-$^{99m}Tc$-human serum albumin are safe and of low cost, and they have already been clinically used. This conjugate was stable in the serum, it showed a long retention time in the lymphatic system and the lymph nodes showed a much higher signal-to-noise ratio after the conjugate was injected intradermally into the paw of mice. Both the single-photon emission computed tomography and near-infrared fluorescent images of the mice were successfully obtained at the same time as the excised sentinel lymph nodes showed blue color. The visual color, near-infrared fluorescence and gamma ray from this agent could be complementary for each other in all the steps of sentinel lymph node sampling: exploring and planning sentinel lymph node before excision with visualization of the exact sentinel lymph node location during an operation. Therefore, the triple modal agent will possibly be very ideal for sentinel lymph node mapping because of the high signal-to-noise ratio for non-invasive imaging and its complementary multimodal nature, easy preparation and safety. It is promising for clinical applications and it may have great advantages over the traditional single modal methods.

Intelligent Hybrid Fusion Algorithm with Vision Patterns for Generation of Precise Digital Road Maps in Self-driving Vehicles

  • Jung, Juho;Park, Manbok;Cho, Kuk;Mun, Cheol;Ahn, Junho
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.14 no.10
    • /
    • pp.3955-3971
    • /
    • 2020
  • Due to the significant increase in the use of autonomous car technology, it is essential to integrate this technology with high-precision digital map data containing more precise and accurate roadway information, as compared to existing conventional map resources, to ensure the safety of self-driving operations. While existing map technologies may assist vehicles in identifying their locations via Global Positioning System, it is however difficult to update the environmental changes of roadways in these maps. Roadway vision algorithms can be useful for building autonomous vehicles that can avoid accidents and detect real-time location changes. We incorporate a hybrid architectural design that combines unsupervised classification of vision data with supervised joint fusion classification to achieve a better noise-resistant algorithm. We identify, via a deep learning approach, an intelligent hybrid fusion algorithm for fusing multimodal vision feature data for roadway classifications and characterize its improvement in accuracy over unsupervised identifications using image processing and supervised vision classifiers. We analyzed over 93,000 vision frame data collected from a test vehicle in real roadways. The performance indicators of the proposed hybrid fusion algorithm are successfully evaluated for the generation of roadway digital maps for autonomous vehicles, with a recall of 0.94, precision of 0.96, and accuracy of 0.92.

Improvement of Lipreading Performance Using Gabor Filter for Ship Environment (선박 환경에서 Gabor 여파기를 적용한 입술 읽기 성능향상)

  • Shin, Do-Sung;Lee, Seong-Ro;Kwon, Jang-Woo
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.35 no.7C
    • /
    • pp.598-603
    • /
    • 2010
  • In this paper, we work for Lipreading using visual information for ship environment. Lipreading is studied for using image information including lips of a speaker at the existing speech recognition system. This technique is a compensation method to increase recognition rate decreasing remarkably in noisy circumstances. Proposed way improved the rate of recognition improving methode of preprocessing using the Gabor Filter for Ship Environment. The experiment were carried out under changing of light with time in the ship environment with lip image. For Comparing with recognition, make a compare with between method of lip region of interest (ROI) before Gabor filtering and after Gabor filtering. In the case of using method of lip ROI before Gabor filtering, the result of the experiments applying to the proposed ways recognition resulting in 44% of recognition.

DIAGNOSIS OF MESIODENS BY $SCANORA^{(R)}$ ($SCANORA^{(R)}$를 이용한 Mesiodens의 진단)

  • Jeon, Hyung-Joon;Kim, Jong-Soo;Kwon, Soon-Won
    • Journal of the korean academy of Pediatric Dentistry
    • /
    • v.27 no.4
    • /
    • pp.490-493
    • /
    • 2000
  • Mesiodens are supernumerary teeth in the region of the maxillary central incisors, a condition which can lead to disorders of the dentition. Their presence may lead to disorders, such as delay in eruption of permanent teeth, development of dentigerous cysts, resorption of adjacent roots and eruption of a supernumerary tooth into nasal cavity. The optimal time for surgical intervention is controversial. Early diagnosis is important so as to enable good prognosis. Diagnosis primarily depends on x-ray films. Panorama film, occlusal film, periapical film have been used for detection of mesiodens. But, all of them have disadvantages. $SCANORA^{(R)}$ is a multimodal radiology system which utilizes the principles of narrow beam radiology and spiral tomography. Pre-programmed imaging procedure are provided for many dental situations An optional personal computer can be connected into the unit to help design the examination. We report two cases diagnosed by $SCANORA^{(R)}$. When compared with tube shift technique, it is simple and exact method of detecting mesiodens.

  • PDF

Leveraging Social Media for Enriching Disaster related Location Trustiness (재난 관련 위치 신뢰도 향상을 위한 소셜 미디어 활용)

  • Nguyen, Van-Quyet;Nguyen, Giang-Truong;Nguyen, Sinh-Ngoc;Kim, Kyungbaek
    • Journal of Digital Contents Society
    • /
    • v.18 no.3
    • /
    • pp.567-575
    • /
    • 2017
  • Location-based services play an important role in many applications such as disaster warning systems and recommendation systems. These applications often require not only location information (e.g., name, latitude, longitude, etc.) but also the impact of events (e.g., earthquake, typhoon, etc.) on locations. Recently, to provide the impact of an event on a location, how to calculate location trustiness by using multimodal information such as earthquake information and disaster sensor data is researched. In the previous approach, the linear decrement of impact value of an event is applied to obtain the location trustiness of a specific location. In this paper, we propose a new approach to enrich location trustiness, that is, the impact of an event on a location, by using social media information additionally. Firstly, we design a collecting system for earthquake information and social media data. Secondly, we present an approach of location trustiness calculation based on earthquake information. Finally, we propose a new approach to enrich location trustiness by augmenting the trustiness in spatially distributed manner based on social media.