• 한국HCI학회:학술대회논문집
• /
• 2006.02b
• /
• pp.362-369
• /
• 2006

3 차원 그래픽스에 적용하는 한국어 립씽크 합성 체계를 연구하여, 말소리에 대응하는 자연스러운 립씽크를 자동적으로 생성하도록 하는 디자인 시스템을 연구 개발하였다. 페이셜애니메이션은 크게 나누어 감정 표현, 즉 표정의 애니메이션과 대화 시 입술 모양의 변화를 중심으로 하는 대화 애니메이션 부분으로 구분할 수 있다. 표정 애니메이션의 경우 약간의 문화적 차이를 제외한다면 거의 세계 공통의 보편적인 요소들로 이루어지는 반면 대화 애니메이션의 경우는 언어에 따른 차이를 고려해야 한다. 이와 같은 문제로 인해 영어권 및 일본어 권에서 제안되는 음성에 따른 립싱크 합성방법을 한국어에 그대로 적용하면 청각 정보와 시각 정보의 부조화로 인해 지각의 왜곡을 일으킬 수 있다. 본 연구에서는 이와 같은 문제점을 해결하기 위해 표기된 텍스트를 한국어 발음열로 변환, HMM 알고리듬을 이용한 입력 음성의 시분할, 한국어 음소에 따른 얼굴특징점의 3 차원 움직임을 정의하는 과정을 거쳐 텍스트와 음성를 통해 3 차원 대화 애니메이션을 생성하는 한국어 립싱크합성 시스템을 개발 실제 캐릭터 디자인과정에 적용하도록 하였다. 또한 본 연구는 즉시 적용이 가능한 3 차원 캐릭터 애니메이션뿐만 아니라 아바타를 활용한 동적 인터페이스의 요소기술로서 사용될 수 있는 선행연구이기도 하다. 즉 3 차원 그래픽스 기술을 활용하는 영상디자인 분야와 HCI 에 적용할 수 있는 양면적 특성을 지니고 있다. 휴먼 커뮤니케이션은 언어적 대화 커뮤니케이션과 시각적 표정 커뮤니케이션으로 이루어진다. 즉 페이셜애니메이션의 적용은 보다 인간적인 휴먼 커뮤니케이션의 양상을 지니고 있다. 결국 인간적인 상호작용성이 강조되고, 보다 편한 인간적 대화 방식의 휴먼 인터페이스로 그 미래적 양상이 변화할 것으로 예측되는 아바타를 활용한 인터페이스 디자인과 가상현실 분야에 보다 폭넓게 활용될 수 있다.

• Journal of the Korea Computer Graphics Society
• /
• v.9 no.2
• /
• pp.11-18
• /
• 2003

For extracting geometric features in 3D meshes according to user-steering with effective interactions. this paper generalizes the 2D algorithms of snapping and wrapping that. respectively. moves a cursor to a nearby feature and constructs feature boundaries. First. we define approximate curvatures and move cost functions that are the numerical values measuring the geometric characteristics of the meshes, By exploiting the measuring values. the algorithms of geometric snapping and geometric wrapping are developed and implemented. We also visualize the results from applying the algorithms to extracting geometric features of general 3D mesh models such as a face model and a tooth model.

• Journal of the Institute of Electronics and Information Engineers
• /
• v.49 no.9
• /
• pp.347-352
• /
• 2012

In this paper, an 2D spatial-map construction for workers identification and avoidance of AGV using the detection scheme of the spatial coordinates based on stereo camera is proposed. In the proposed system, face area of a moving person is detected from a left image among the stereo image pairs by using the YCbCr color model and its center coordinates are computed by using the centroid method and then using these data, the stereo camera embedded on the mobile robot can be controlled for tracking the moving target in real-time. Moreover, using the disparity map obtained from the left and right images captured by the tracking-controlled stereo camera system and the perspective transformation between a 3-D scene and an image plane, depth map can be detected. From some experiments on AGV driving with 240 frames of the stereo images, it is analyzed that error ratio between the calculated and measured values of the worker's width is found to be very low value of 2.19% and 1.52% on average.

• The korean journal of orthodontics
• /
• v.41 no.2
• /
• pp.76-86
• /
• 2011

Objective: To investigate whether the accuracy of 3D laser scanning is influenced by the angles and number of scans. Methods: Using a 3D laser scanner, 10 manikins with facial markers were scanned at 7 horizontal angles (front view and at $20^{\circ}$, $45^{\circ}$, and $60^{\circ}$ angles on the right and left sides). Three-dimensional facial images were reconstructed by 6 methods differing in the number and angles of scans, and measurements of these images were compared to the physical measurements from the manikins. Results: The laser scan images were magnified by 0.14 - 0.26%. For images reconstructed by merging 2 scans, excluding the front view; and by merging 3 scans, including the front view and scans obtained at $20^{\circ}$ on both sides; several measurements were significantly different than the physical measurements. However, for images reconstructed by merging 3 scans, including the front view; and 5 scans, including the front view and scans obtained at $20^{\circ}$ and $60^{\circ}$ on both sides; only 1 measurement was significantly different. Conclusions: These results suggest that the number and angle of scans influence the accuracy of 3D laser scanning. A minimum of 3 scans, including the front view and scans obtained at more than $45^{\circ}$ on both sides, should be integrated to obtain accurate 3D facial images.

• Journal of the Institute of Electronics Engineers of Korea SP
• /
• v.45 no.2
• /
• pp.49-64
• /
• 2008

Recently, many researches about making more comfortable input device based on gaze detection technology have been done in human computer interface. However, the system cost becomes high due to the complicated hardware and there is difficulty to use the gaze detection system due to the complicated user calibration procedure. In this paper, we propose a new gaze detection method based on the 2D analysis and a simple user calibration. Our method used a small USB (Universal Serial Bus) camera attached on a HMD (Head-Mounted Display), hot-mirror and IR (Infra-Red) light illuminator. Because the HMD is moved according to user's facial movement, we can implement the gaze detection system of which performance is not affected by facial movement. In addition, we apply our gaze detection system to 3D first person shooting game. From that, the gaze direction of game character is controlled by our gaze detection method and it can target the enemy character and shoot, which can increase the immersion and interest of game. Experimental results showed that the game and gaze detection system could be operated at real-time speed in one desktop computer and we could obtain the gaze detection accuracy of 0.88 degrees. In addition, we could know our gaze detection technology could replace the conventional mouse in the 3D first person shooting game.

• The Journal of Korean Institute of Communications and Information Sciences
• /
• v.29 no.4C
• /
• pp.514-523
• /
• 2004

In this paper, a new intelligent moving target tracking and surveillance system basing on the pan/tilt-embedded stereo camera system is suggested and implemented. In the proposed system, once the face area of a target is detected from the input stereo image by using a YCbCr color model and then, using this data as well as the geometric information of the tracking system, the distance and 3D information of the target are effectively extracted in real-time. Basing on these extracted data the pan/tilted-embedded stereo camera system is adaptively controlled and as a result, the proposed system can track the target adaptively under the various circumstance of the target. From some experiments using 80 frames of the test input stereo image, it is analyzed that standard deviation of the position displacement of the target in the horizontal and vertical directions after tracking is kept to be very low value of 1.82, 1.11, and error ratio between the measured and computed 3D coordinate values of the target is also kept to be very low value of 0.5% on average. From these good experimental results a possibility of implementing a new real-time intelligent stereo target tracking and surveillance system using the proposed scheme is finally suggested.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.9 no.5
• /
• pp.213-224
• /
• 2009

This paper presents a new algorithm for estimating a driver's physical condition using real-time vision system and performs experimentation for real facial image data. The system relies on a face recognition to robustly track the center points and sizes of person's two pupils, and two side edge points of the mouth. The face recognition constitutes the color statistics by YUV color space together with geometrical model of a typical face. The system can classify the rotation in all viewing directions, to detect eye/mouth occlusion, eye blinking and eye closure, and to recover the three dimensional gaze of the eyes. These are utilized to determine the carelessness and drowsiness of the driver. Finally, experimental results have demonstrated the validity and the applicability of the proposed method for the estimation of a driver's physical condition.

• Journal of Korea Multimedia Society
• /
• v.17 no.3
• /
• pp.277-286
• /
• 2014

In this paper, we propose a texture feature-based language identification by fusion of Gabor, MDLC (multi-lag directional local correlation), and co-occurrence features. In the proposed method, for a test image, Gabor magnitude images are first obtained by Gabor transform followed by magnitude operator. Moments for the Gabor magniude images are then computed and vectorized. MDLC images are then obtained by MDLC operator and their moments are computed and vectorized. GLCM (gray-level co-occurrence matrix) is next calculated from the test image and co-occurrence features are computed using the GLCM, and the features are also vectorized. The three vectors of the Gabor, MDLC, and co-occurrence features are fused into a feature vector. In classification, the WPCA (whitened principal component analysis) classifier, which is usually adopted in the face identification, searches the training feature vector most similar to the test feature vector. We evaluate the performance of our method by examining averaged identification rates for a test document image DB obtained by scanning of documents with 15 languages. Experimental results show that the proposed method yields excellent language identification with rather low feature dimension for the test DB.

• Journal of Broadcast Engineering
• /
• v.23 no.3
• /
• pp.351-360
• /
• 2018

Human emotion recognition is a research topic that is receiving continuous attention in computer vision and artificial intelligence domains. This paper proposes a method for classifying human emotions through multiple neural networks based on multi-modal signals which consist of image, landmark, and audio in a wild environment. The proposed method has the following features. First, the learning performance of the image-based network is greatly improved by employing both multi-task learning and semi-supervised learning using the spatio-temporal characteristic of videos. Second, a model for converting 1-dimensional (1D) landmark information of face into two-dimensional (2D) images, is newly proposed, and a CNN-LSTM network based on the model is proposed for better emotion recognition. Third, based on an observation that audio signals are often very effective for specific emotions, we propose an audio deep learning mechanism robust to the specific emotions. Finally, so-called emotion adaptive fusion is applied to enable synergy of multiple networks. The proposed network improves emotion classification performance by appropriately integrating existing supervised learning and semi-supervised learning networks. In the fifth attempt on the given test set in the EmotiW2017 challenge, the proposed method achieved a classification accuracy of 57.12%.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.16 no.3
• /
• pp.115-122
• /
• 2016

This paper proposes a system to detect driver's cognitive state by internal and external information of vehicle. The proposed system can measure driver's eye gaze. This is done by concept of information delivery and mutual information measure. For this study, we set up two web-cameras at vehicles to obtain visual information of the driver and front of the vehicle. We propose Gestalt principle based selective attention model to define information quantity of road scene. The saliency map based on gestalt principle is prominently represented by stimulus such as traffic signals. The proposed system assumes driver's cognitive resource allocation on the front scene by gaze analysis and head pose direction information. Then we use several feature algorithms for detecting driver's characteristics in real time. Modified census transform (MCT) based Adaboost is used to detect driver's face and its component whereas POSIT algorithms are used for eye detection and 3D head pose estimation. Experimental results show that the proposed system works well in real environment and confirm its usability.