• Journal of the Institute of Electronics Engineers of Korea SP
• /
• v.41 no.3
• /
• pp.155-162
• /
• 2004

This paper concentrated on lip synchronization (lip sync) test for DTV with respect to audio and video signals using a finite digital bitstream In this paper, we propose a new lip sync test method which does not effect on the current program by use of the transient effect area test signals (TATS) and audio-video time index lip sync test signals (TILS).the experimental result shows that the time difference between audio and video signal can be easily measured by captured oscilloscope waveform at any time.

• Cartoon and Animation Studies
• /
• s.13
• /
• pp.97-114
• /
• 2008

This study aims to study mouth shapes suitable to the shapes of Korean consonants and vowels for Korean animations by analyzing the process of English-speaking animation lip-sync based on pre-recording in the United States. A research was conducted to help character animators understand the concept of Korean lip-sync which is done after recording and to introduce minimum, basic mouth shapes required for Korean expressions which can be applied to various characters. In the introduction, this study mentioned the necessity of Korean lip-sync in local animations and introduced the research methods of Korean lip-sync data based on English lip-sync data by laking an American production as an example. In the main subject, this study demonstrated the characteristics and roles of 8 basic mouth shapes required for English pronunciation expressions, left out mouth shapes that are required for English expressions but not for Korean expressions, and in contrast, added mouth shapes required for Korean expressions but not for English expressions. Based on these results, this study made a diagram for the mouth shapes of Korean expressions by laking various examples and made a research on how mouth shapes vary when they are used as consonants, vowels and batchim. In audition, the case study proposed a method to transfer lines to the exposure sheet and a method to arrange mouth shapes according to lip-sync for practical animation production. However, lines from a Korean movie were inevitably used as an example because there has not been any precedents in Korea about animation production with systematic Korean lip-sync data.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.18 no.3
• /
• pp.127-134
• /
• 2018

In this paper, we have developed the Auto Lip-Sync Maya plug-in for extracting Korean phonemes from voice data and text information based on Korean and produce high quality 3D lip-sync animation using divided phonemes. In the developed system, phoneme separation was classified into 8 vowels and 13 consonants used in Korean, referring to 49 phonemes provided by Microsoft Speech API engine SAPI. In addition, the pronunciation of vowels and consonants has variety Mouth Shapes, but the same Viseme can be applied to some identical ones. Based on this, we have developed Auto Lip-sync Maya Plug-in based on Python to enable lip-sync animation to be implemented automatically at once.

• The Transactions of the Korea Information Processing Society
• /
• v.13 no.4
• /
• pp.166-173
• /
• 2024

This study explores lip-sync detection, focusing on the synchronization between lip movements and voices in videos. Typically, lip-sync detection techniques involve cropping the facial area of a given video, utilizing the lower half of the cropped box as input for the visual encoder to extract visual features. To enhance the emphasis on the articulatory region of lips for more accurate lip-sync detection, we propose utilizing a pre-trained visual attention-based encoder. The Visual Transformer Pooling (VTP) module is employed as the visual encoder, originally designed for the lip-reading task, predicting the script based solely on visual information without audio. Our experimental results demonstrate that, despite having fewer learning parameters, our proposed method outperforms the latest model, VocaList, on the LRS2 dataset, achieving a lip-sync detection accuracy of 94.5% based on five context frames. Moreover, our approach exhibits an approximately 8% superiority over VocaList in lip-sync detection accuracy, even on an untrained dataset, Acappella.

• Journal of Digital Convergence
• /
• v.10 no.7
• /
• pp.253-259
• /
• 2012

In this study, the dialog syntax-based video content production flow for the character set, 'Form Noah' chart using the mouse, lip-sync Animation been making 3D characters were applying. Vertex Animation Morphing techniques by expressing the natural shape of the mouth for the characters engaging and the transmission of visual information for the viewers to be able to get a high intelligibility is considered.

• Annual Conference of KIPS
• /
• 2024.10a
• /
• pp.523-526
• /
• 2024

Recent progress in audio-driven talking face generation has focused on achieving more realistic and emotionally expressive lip movements, enhancing the quality of virtual avatars and animated characters for applications in entertainment, education, healthcare, and more. Despite these advances, challenges remain in creating natural and emotionally nuanced lip synchronization efficiently and accurately. To address these issues, we introduce a novel method for audio-driven lip-sync that offers precise control over emotional expressions, outperforming current techniques. Our method utilizes Conditional Deep Variational Autoencoder to produce lifelike lip movements that align seamlessly with audio inputs while dynamically adjusting for various emotional states. Experimental results highlight the advantages of our approach, showing significant improvements in emotional accuracy and the overall quality of the generated facial animations, video sequences on the Crema-D dataset [1].

• The Journal of Korea Institute of Information, Electronics, and Communication Technology
• /
• v.6 no.4
• /
• pp.201-207
• /
• 2013

Voice Recognition technique has been developed and it has been actively applied to various information devices such as smart phones and car navigation system. But the basic research technique related the speech recognition is based on research results in English. Since the lip sync producing generally requires tedious hand work of animators and it serious affects the animation producing cost and development period to get a high quality lip animation. In this research, a real time processed automatic lip sync algorithm for virtual characters in digital contents is studied by considering Korean vocal sound system. This suggested algorithm contributes to produce a natural lip animation with the lower producing cost and the shorter development period.

• Journal of the Korea Convergence Society
• /
• v.1 no.1
• /
• pp.69-75
• /
• 2010

The importance of a using the visual media in English education has been increased. By an importance of Characters in English language content, the more effort is needed for a learner to show the English pronunciation and a realistic implementation. In this paper, we tried to review the Syntax-based Educational Contents Authoring System. For the more realistic lip-sync character, 3D character to enhance the efficiency of the education was constructed. We used a chart of the association structure analysis of mouth's shape. we produced an optimized 3D character through a process of a concept, a modeling, a mapping and an animating design. For more effective educational content for 3D character creation, the next research will be continuously a 3d Character added to a hand motion and body motion in order to show an effective communication example.

• The Journal of the Convergence on Culture Technology
• /
• v.9 no.4
• /
• pp.409-414
• /
• 2023

We described how to implement character expressions and animations that play an important role in expressing emotions and personalities in low-polygon character animation. With the development of the video industry, character expressions and mouth-shaped lip-syncing in animation can realize natural movements at a level close to real life. However, for non-experts, it is difficult to use expert-level advanced technology. Therefore, We aimed to present a guide for low-budget low-polygon character animators or non-experts to create mouth-shaped lip-syncing more naturally using accessible and highly usable features. A total of 8 mouth shapes were developed for mouth shape lip-sync animation: 'ㅏ', 'ㅔ', 'ㅣ', 'ㅗ', 'ㅜ', 'ㅡ', 'ㅓ' and a mouth shape that expresses a labial consonant. In the case of facial expression animation, a total of nine animations were produced by adding highly utilized interest, boredom, and pain to the six basic human emotions classified by Paul Ekman: surprise, fear, disgust, anger, happiness, and sadness. This study is meaningful in that it makes it easy to produce natural animation using the features built into the modeling program without using complex technologies or programs.

• The Journal of the Acoustical Society of Korea
• /
• v.20 no.3
• /
• pp.63-69
• /
• 2001

A corpus-based lip sync algorithm for synthesizing natural face animation is proposed in this paper. To get the lip parameters, some marks were attached some marks to the speaker's face, and the marks' positions were extracted with some Image processing methods. Also, the spoken utterances were labeled with HTK and prosodic information (duration, pitch and intensity) were analyzed. An audio-visual corpus was constructed by combining the speech and image information. The basic unit used in our approach is syllable unit. Based on this Audio-visual corpus, lip information represented by mark's positions was synthesized. That is. the best syllable units are selected from the audio-visual corpus and each visual information of selected syllable units are concatenated. There are two processes to obtain the best units. One is to select the N-best candidates for each syllable. The other is to select the best smooth unit sequences, which is done by Viterbi decoding algorithm. For these process, the two distance proposed between syllable units. They are a phonetic environment distance measure and a prosody distance measure. Computer simulation results showed that our proposed algorithm had good performances. Especially, it was shown that pitch and intensity information is also important as like duration information in lip sync.