• Title/Summary/Keyword: Multimodal model

Search Result 142, Processing Time 0.028 seconds

Deep Learning-Based Companion Animal Abnormal Behavior Detection Service Using Image and Sensor Data

  • Lee, JI-Hoon;Shin, Min-Chan;Park, Jun-Hee;Moon, Nam-Mee
    • Journal of the Korea Society of Computer and Information
    • /
    • v.27 no.10
    • /
    • pp.1-9
    • /
    • 2022
  • In this paper, we propose the Deep Learning-Based Companion Animal Abnormal Behavior Detection Service, which using video and sensor data. Due to the recent increase in households with companion animals, the pet tech industry with artificial intelligence is growing in the existing food and medical-oriented companion animal market. In this study, companion animal behavior was classified and abnormal behavior was detected based on a deep learning model using various data for health management of companion animals through artificial intelligence. Video data and sensor data of companion animals are collected using CCTV and the manufactured pet wearable device, and used as input data for the model. Image data was processed by combining the YOLO(You Only Look Once) model and DeepLabCut for extracting joint coordinates to detect companion animal objects for behavior classification. Also, in order to process sensor data, GAT(Graph Attention Network), which can identify the correlation and characteristics of each sensor, was used.

Interaction Intent Analysis of Multiple Persons using Nonverbal Behavior Features (인간의 비언어적 행동 특징을 이용한 다중 사용자의 상호작용 의도 분석)

  • Yun, Sang-Seok;Kim, Munsang;Choi, Mun-Taek;Song, Jae-Bok
    • Journal of Institute of Control, Robotics and Systems
    • /
    • v.19 no.8
    • /
    • pp.738-744
    • /
    • 2013
  • According to the cognitive science research, the interaction intent of humans can be estimated through an analysis of the representing behaviors. This paper proposes a novel methodology for reliable intention analysis of humans by applying this approach. To identify the intention, 8 behavioral features are extracted from the 4 characteristics in human-human interaction and we outline a set of core components for nonverbal behavior of humans. These nonverbal behaviors are associated with various recognition modules including multimodal sensors which have each modality with localizing sound source of the speaker in the audition part, recognizing frontal face and facial expression in the vision part, and estimating human trajectories, body pose and leaning, and hand gesture in the spatial part. As a post-processing step, temporal confidential reasoning is utilized to improve the recognition performance and integrated human model is utilized to quantitatively classify the intention from multi-dimensional cues by applying the weight factor. Thus, interactive robots can make informed engagement decision to effectively interact with multiple persons. Experimental results show that the proposed scheme works successfully between human users and a robot in human-robot interaction.

Crack Identification Using Evolutionary Algorithms in Parallel Computing Environment (병렬 환경하의 진화 이론을 이용한 결함인식)

  • Sim, Mun-Bo;Seo, Myeong-Won
    • Transactions of the Korean Society of Mechanical Engineers A
    • /
    • v.26 no.9
    • /
    • pp.1806-1813
    • /
    • 2002
  • It is well known that a crack has an important effect on the dynamic behavior of a structure. This effect depends mainly on the location and depth of the crack. To identify the location and depth of a crack in a structure, a classical optimization technique was adopted by previous researchers. That technique overcame the difficulty of finding the intersection point of the superposed contours that correspond to the eigenfrequency caused by the crack presence. However, it is hard to select a trial solution initially for optimization because the defined objective function is heavily multimodal. A method is presented in this paper, which uses continuous evolutionary algorithms(CEAs). CEAs are effective for solving inverse problems and implemented on PC clusters to shorten calculation time. With finite element model of the structure to calculate eigenfrequencies, it is possible to formulate the inverse problem in optimization format. CEAs are used to identify the crack location and depth minimizing the difference from the measured frequencies. We have tried this new idea on a simple beam structure and the results are promising with high parallel efficiency over about 94%.

Feasibility Study of Determining the Healing Phase of Achilles Tendon Rupture in Rats Using Optical Coherence Tomography

  • Kim, Young-Sik;Chae, Yu-Gyeong;Jeon, Min Yong;Kim, Dong Kyu;Ahn, Yeh-Chan
    • Journal of the Optical Society of Korea
    • /
    • v.19 no.2
    • /
    • pp.175-181
    • /
    • 2015
  • Optical coherence tomography (OCT) is a noninvasive technique for microscopic investigation of tissue. We thought that the OCT method could be a potential tool for monitoring the healing process of a tendon. In this study we used two rat models, denervated and non-denervated groups, to observe a variety of healing phases of Achilles tendon (AT) injury. We made samples of AT injury lesions, to take OCT images and to make histopathological samples of serial sectional tissue. In an OCT image the denervated rat showed no specific finding, but the non-denervated rat showed a large defect lesion that was scaffolding tissue. OCT findings combined with pathologic findings showed advantages in visualization of tendon microstructure over other imaging modalities such as MRI and US, and OCT is beneficial to making a treatment plan, especially the timing and intensity of rehabilitation. Therefore a multimodal platform using OCT for evaluation of tendon injury may be potentially useful for many applications.

The Busan Port Throughput Routing analysis (부산항 물동량 경로 분석)

  • Jo, Min-Ji;Ganbat, Enkhtsetseg;Kim, Hwan-Seong
    • Proceedings of the Korean Institute of Navigation and Port Research Conference
    • /
    • 2013.06a
    • /
    • pp.91-92
    • /
    • 2013
  • With development of port industry, inland transportation was also the developed. Connecting port with inland becomes more and more important. So studies about cargo flow from ports to regions are actively in progress. But freight statistics from regional to national has a problem that do not comprehend exactly with freight flow. Also these statistics don't reflect characteristics of multimodal transportation system. The objective of this paper is to analyze freight flow of container with the introduction of P/C and rebuilding freight statistics from regional to national scale.

  • PDF

Combining Multi-Criteria Analysis with CBR for Medical Decision Support

  • Abdelhak, Mansoul;Baghdad, Atmani
    • Journal of Information Processing Systems
    • /
    • v.13 no.6
    • /
    • pp.1496-1515
    • /
    • 2017
  • One of the most visible developments in Decision Support Systems (DSS) was the emergence of rule-based expert systems. Hence, despite their success in many sectors, developers of Medical Rule-Based Systems have met several critical problems. Firstly, the rules are related to a clearly stated subject. Secondly, a rule-based system can only learn by updating of its rule-base, since it requires explicit knowledge of the used domain. Solutions to these problems have been sought through improved techniques and tools, improved development paradigms, knowledge modeling languages and ontology, as well as advanced reasoning techniques such as case-based reasoning (CBR) which is well suited to provide decision support in the healthcare setting. However, using CBR reveals some drawbacks, mainly in its interrelated tasks: the retrieval and the adaptation. For the retrieval task, a major drawback raises when several similar cases are found and consequently several solutions. Hence, a choice for the best solution must be done. To overcome these limitations, numerous useful works related to the retrieval task were conducted with simple and convenient procedures or by combining CBR with other techniques. Through this paper, we provide a combining approach using the multi-criteria analysis (MCA) to help, the traditional retrieval task of CBR, in choosing the best solution. Afterwards, we integrate this approach in a decision model to support medical decision. We present, also, some preliminary results and suggestions to extend our approach.

Resolution of Anaphoric Noun Phrases using a Centering Algorithm with a Dual Cache Model in a Multimodal Dialogue System (다중모드 대화 시스템에서 이중 캐시 모델의 센터링 알고리즘을 이용한 명사 대용어구 처리)

  • Kim, Hak-Su;Seo, Jeong-Yeon
    • Journal of KIISE:Software and Applications
    • /
    • v.27 no.11
    • /
    • pp.1133-1140
    • /
    • 2000
  • 다중모드 대화에서 나타나는 대용어는 언어만을 사용하는 대화에서 나타나는 것과 비교하여 매우 다른 형태와 특징을 가진다. 그것은 행위나 시각이 대용 행위로 사용될 수 있기 때문이다. 본 논문에서는 터치스크린 인터페이스를 이용한 홈쇼핑 가구점 영역의 다중모드 대화 시스템에서 나타나는 다양한 대용어의 처리 방법을 알아본다. 먼저, 화면 대용어와 참조 대용어를 정의하여 다양한 형태의 대용어를 분류한다. 그리고 각 대용어를 처리할 수 있는 두 가지의 일반적인 방법을 제안한다. 하나는 지시 행위를 수반하거나 생략한 채 발화되어 현재 화면에 나타나 있는 아이템을 참조하는 대용어를 처리하는 단순한 매핑 알고리즘이다. 다른 하나는 다중 모드 대화 시스템을 위해 워커(Walker)의 센터링 알고리즘을 확장한 이중 캐시 구조의 센터링 알고리즘이다. 확장된 센터링 알고리즘은 발화와시각 정보 그리고 화면 전환 시간을 유지할 수 있기 때문에 다중모드 대화에서 발생하는 다양한 대용어를 처리하기에 적합하다. 실험에서 제안된 시스템은 40개의 대화에서 나타난 402개의 대용어(발화당 0.54)중에서 387개를 처리하여 96.3%의 정확도를 보였다.

  • PDF

Relationship between Postural Balance Training and Fall Risks for Elderly: a Systematic Review of Randomized Controlled Trials

  • Kim, Heesuk;Hwang, Sujin
    • Physical Therapy Rehabilitation Science
    • /
    • v.10 no.2
    • /
    • pp.185-196
    • /
    • 2021
  • Objective: Falling is one of main accident to facilitate the physical injuries in order adults. The purpose of the systematic review was to determine the effects of postural balance training whether the recovery of falls in elderly with normal physical function or not throughout summing the selected studies quantitatively. Design: A systematic review Methods: MEDLINE and other four databases were searched up to April 20, 2021 and randomized controlled trials (RCTs) evaluating postural balance approaches on fall risks in elderly. The researched studies excluded the double studies, titles and abstract, and finally full-reported study. The selected RCTs studies were extracted characteristics of the studies and summary of results based on PICOS-SD (population, intervention, comparison, outcomes, and setting- study design) model to synthesize the papers qualitatively. Results: The review involved 22 RCT reports with 4,847 community older adults aged 65 years or over. Nineteen of the selected RCT studies reported dual or multimodal exercises show the beneficial effect for older adults compared to one-type treatment or no intervention. All of selected showed low risk in the selection, attrition, and reporting bias. However, detection bias showed low risk at 75% records of the involved RCTs and performance bias was low risk at only three records. Conclusions: The results of the systematic review propose that a standardized therapeutic approach and the intensity are needed for improving risk of falls in older adults.

Attention based multimodal model for Korean speech recognition post-editing (한국어 음성인식 후처리를 위한 주의집중 기반의 멀티모달 모델)

  • Jeong, Yeong-Seok;Oh, Byoung-Doo;Heo, Tak-Sung;Choi, Jeong-Myeong;Kim, Yu-Seop
    • Annual Conference on Human and Language Technology
    • /
    • 2020.10a
    • /
    • pp.145-150
    • /
    • 2020
  • 최근 음성인식 분야에서 신경망 기반의 종단간 모델이 제안되고 있다. 해당 모델들은 음성을 직접 입력받아 전사된 문장을 생성한다. 음성을 직접 입력받는 모델의 특성상 데이터의 품질이 모델의 성능에 많은 영향을 준다. 본 논문에서는 이러한 종단간 모델의 문제점을 해결하고자 음성인식 결과를 후처리하기 위한 멀티모달 기반 모델을 제안한다. 제안 모델은 음성과 전사된 문장을 입력 받는다. 입력된 각각의 데이터는 Encoder를 통해 자질을 추출하고 주의집중 메커니즘을 통해 Decoder로 추출된 정보를 전달한다. Decoder에서는 전달받은 주의집중 메커니즘의 결과를 바탕으로 후처리된 토큰을 생성한다. 본 논문에서는 후처리 모델의 성능을 평가하기 위해 word error rate를 사용했으며, 실험결과 Google cloud speech to text모델에 비해 word error rate가 8% 감소한 것을 확인했다.

  • PDF

Developing the Design Guideline of Auditory User Interface for Digital Appliances (가전제품의 청각 사용자 인터페이스(AUI) 디자인을 위한 가이드라인 개발 사례)

  • Lee, Ju-Hwan;Jeon, Myoung-Hoon;Han, Kwang-Hee
    • Science of Emotion and Sensibility
    • /
    • v.10 no.3
    • /
    • pp.307-320
    • /
    • 2007
  • In this study, we attempted to provide a distinctive cognitive, emotional 'Auditory User Interface (AUI) Design Guideline' according to home appliance groups and their functions. It is an effort to apply a new design method to practical affairs to overcome the limit of GUI centered appliance design and reflect user multimodal properties by presenting a guideline possible to generate auditory signals intuitively associable with the operational functions. The reason why this study is required is because of frequent instances given rise to annoyance as not systematic application of AUI, but arbitrary mapping. This study tried to provide a useful guideline of AUI in home appliances by extracting the relations with cognitive, emotional properties of a certain device or function induced by several properties of auditory signal and showing the empirical data on the basic mechanism of such relations.

  • PDF