• Title/Summary/Keyword: Lip-reading

Search Result 36, Processing Time 0.028 seconds

Real-time Lip Region Detection for Lipreadingin Mobile Device (모바일 장치에서의 립리딩을 위한 실시간 입술 영역 검출)

  • Kim, Young-Un;Kang, Sun-Kyung;Jung, Sung-Tae
    • Journal of the Korea Society of Computer and Information
    • /
    • v.14 no.4
    • /
    • pp.39-46
    • /
    • 2009
  • Many lip region detection methods have been developed in PC environment. But the existing methods are difficult to run on real-time in resource limited mobile devices. To solve the problem, this paper proposes a real-time lip region detection method for lipreading in Mobile device. It detects face region by using adaptive face color information. After that, it detects lip region by using geometrical relation between eyes and lips. The proposed method is implemented in a smart phone with Intel PXA 270 embedded processor and 386MB memory. Experimental results show that the proposed method runs at the speed 9.5 frame/see and the correct detection rate was 98.8% for 574 images.

A Study on Lip Detection based on Eye Localization for Visual Speech Recognition in Mobile Environment (모바일 환경에서의 시각 음성인식을 위한 눈 정위 기반 입술 탐지에 대한 연구)

  • Gyu, Song-Min;Pham, Thanh Trung;Kim, Jin-Young;Taek, Hwang-Sung
    • Journal of the Korean Institute of Intelligent Systems
    • /
    • v.19 no.4
    • /
    • pp.478-484
    • /
    • 2009
  • Automatic speech recognition(ASR) is attractive technique in trend these day that seek convenient life. Although many approaches have been proposed for ASR but the performance is still not good in noisy environment. Now-a-days in the state of art in speech recognition, ASR uses not only the audio information but also the visual information. In this paper, We present a novel lip detection method for visual speech recognition in mobile environment. In order to apply visual information to speech recognition, we need to extract exact lip regions. Because eye-detection is more easy than lip-detection, we firstly detect positions of left and right eyes, then locate lip region roughly. After that we apply K-means clustering technique to devide that region into groups, than two lip corners and lip center are detected by choosing biggest one among clustered groups. Finally, we have shown the effectiveness of the proposed method through the experiments based on samsung AVSR database.

Development of Smart Mirror System for Hearing Deaf's Pronunciation Training (청각 장애인을 위한 발음 교정 학습용 스마트 미러 시스템 개발)

  • Jung, Ha-Yoon;Jeong, Da-Mi;Lee, Jong-Hyeok;Kim, Byung-Gyu
    • Journal of Digital Contents Society
    • /
    • v.18 no.2
    • /
    • pp.267-274
    • /
    • 2017
  • Recently, there is a new trend about internet of things (IoT) such as shops with smart mirror around the fashion and beauty industry. Since smart mirror can display a content through a monitor which is attached to back of mirror system while looking through a mirror, it can be applied to various industries such as fashion, beauty and health care. This paper proposes an efficient learning system requiring no assistance from others for the hearing deaf who atrophy verbal skill and are inaccurate in pronunciation by using features of smart mirror. Also, this system proposes an efficient and simple lip reading method which can be applied to an embedded system and improves a learning efficiency by employing previously verified pronunciation training data.

Cloning, Expression, and Characterization of a Cold-Adapted Lipase Gene from an Antarctic Deep-Sea Psychrotrophic Bacterium, Psychrobacter sp. 7195

  • Zhang, Jinwei;Lin, Shu;Zeng, Runying
    • Journal of Microbiology and Biotechnology
    • /
    • v.17 no.4
    • /
    • pp.604-610
    • /
    • 2007
  • A psychrotrophic strain 7195 showing extracellular lipolytic activity towards tributyrin was isolated from deep-sea sediment of Prydz Bay and identified as a Psychrobacter species. By screening a genomic DNA library of Psychrobacter sp. 7195, an open reading frame of 954 bp coding for a lipase gene, lipA1, was identified, cloned, and sequenced. The deduced LipA1 consisted of 317 amino acids with a molecular mass of 35,210 kDa. It had one consensus motif, G-N-S-M-G (GXSXG), containing the putative active-site serine, which was conserved in other cold-adapted lipolytic enzymes. The recombinant LipA1 was purified by column chromatography with DEAE Sepharose CL-4B, and Sephadex G-75, and preparative polyacrylamide gel electrophoresis, in sequence. The purified enzyme showed highest activity at $30^{\circ}C$, and was unstable at temperatures higher than $30^{\circ}C$, indicating that it was a typical cold-adapted enzyme. The optimal pH for activity was 9.0, and the enzyme was stable between pH 7.0-10.0 after 24h incubation at $4^{\circ}C$. The addition of $Ca^{2+}\;and\;Mg^{2+}$ enhanced the enzyme activity of LipA1, whereas the $Cd^{2+},\;Zn^{2+},\;CO^{2+},\;Fe^{3+},\;Hg^{2+},\;Fe^{2+},\;Rb^{2+}$, and EDTA strongly inhibited the activity. The LipA1 was activated by various detergents, such as Triton X-100, Tween 80, Tween 40, Span 60, Span 40, CHAPS, and SDS, and showed better resistance towards them. Substrate specificity analysis showed that there was a preference for trimyristin and p-nitrophenyl myristate $(C_{14}\;acyl\; groups)$.

Isolation and Expression Analysis of a GDSL-like Lipase Gene from Brassica napus L.

  • Ling, Hua;Zhao, Jingya;Zuo, Kaijing;Qiu, Chengxiang;Yao, Hongyan;Qin, Jie;Sun, Xiaofen;Tang, Kexuan
    • BMB Reports
    • /
    • v.39 no.3
    • /
    • pp.297-303
    • /
    • 2006
  • As lipolytic enzymes, GDSL lipases play an important role in plant growth and development. In order to identify their functions and roles, the full-length cDNA of a GDSL lipase gene, designated BnLIP2, was isolated from Brassica napus L. BnLIP2 was 1,300 bp long, with 1,122 bp open reading frame (ORF) encoding 373 amino acid residues. Sequence analysis indicated that BnLIP2 belonged to GDSL family. Southern blot analysis indicated that BnLIP2 belonged to a small gene family in rapeseed genome. RT-PCR analysis revealed that BnLIP2 was a tissue-specific expressing gene during reproductive growth and strongly expressed during seed germination. BnLIP2 expression could not be detected until three days after germination, and it subsequently became stronger. The transcript of this gene was deficient in root of seedlings growing at different stages. When juvenile seedlings were treated by methyl jasmonate (MeJ), salicylic acid (SA) and naphthalene acetic acid (NAA), BnLIP2 expression could not be induced in root. Our study implicates that BnLIP2 probably plays an important role in rapeseed germination, morphogenesis, flowering, but independent of root growth and development.

Lip and Voice Synchronization with SMS Messages for Mobile 3D Avatar (SMS 메시지에 따른 모바일 3D 아바타의 입술 모양과 음성 동기화)

  • Youn, Jae-Hong;Song, Yong-Gyu;Kim, Eun-Seok;Hur, Gi-Taek
    • Proceedings of the Korea Contents Association Conference
    • /
    • 2006.11a
    • /
    • pp.682-686
    • /
    • 2006
  • There have been increasing interests in 3D mobile content service with emergence of a terminal equipping with a mobile 3D engine and growth of mobile content market. Mobile 3D Avatar is the most effective product displaying the character of a personalized mobile device user. However, previous studies on the method of expressing 3D Avatar have been mainly focused on natural and realistic expressions according to the change in facial expressions and lip shape of a character in PC based virtual environments. In this paper, we propose a method of synchronizing the lip shape with voice by applying a SMS message received in mobile environments to 3D mobile Avatar. The proposed method enables to realize a natural and effective SMS message reading service of mobile Avatar by disassembling a received message sentence into units of a syllable and then synchronizing the lip shape of 3D Avatar with the corresponding voice.

  • PDF

Improvement of Lipreading Performance Using Gabor Filter for Ship Environment (선박 환경에서 Gabor 여파기를 적용한 입술 읽기 성능향상)

  • Shin, Do-Sung;Lee, Seong-Ro;Kwon, Jang-Woo
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.35 no.7C
    • /
    • pp.598-603
    • /
    • 2010
  • In this paper, we work for Lipreading using visual information for ship environment. Lipreading is studied for using image information including lips of a speaker at the existing speech recognition system. This technique is a compensation method to increase recognition rate decreasing remarkably in noisy circumstances. Proposed way improved the rate of recognition improving methode of preprocessing using the Gabor Filter for Ship Environment. The experiment were carried out under changing of light with time in the ship environment with lip image. For Comparing with recognition, make a compare with between method of lip region of interest (ROI) before Gabor filtering and after Gabor filtering. In the case of using method of lip ROI before Gabor filtering, the result of the experiments applying to the proposed ways recognition resulting in 44% of recognition.

Monosyllable Speech Recognition through Facial Movement Analysis (안면 움직임 분석을 통한 단음절 음성인식)

  • Kang, Dong-Won;Seo, Jeong-Woo;Choi, Jin-Seung;Choi, Jae-Bong;Tack, Gye-Rae
    • The Transactions of The Korean Institute of Electrical Engineers
    • /
    • v.63 no.6
    • /
    • pp.813-819
    • /
    • 2014
  • The purpose of this study was to extract accurate parameters of facial movement features using 3-D motion capture system in speech recognition technology through lip-reading. Instead of using the features obtained through traditional camera image, the 3-D motion system was used to obtain quantitative data for actual facial movements, and to analyze 11 variables that exhibit particular patterns such as nose, lip, jaw and cheek movements in monosyllable vocalizations. Fourteen subjects, all in 20s of age, were asked to vocalize 11 types of Korean vowel monosyllables for three times with 36 reflective markers on their faces. The obtained facial movement data were then calculated into 11 parameters and presented as patterns for each monosyllable vocalization. The parameter patterns were performed through learning and recognizing process for each monosyllable with speech recognition algorithms with Hidden Markov Model (HMM) and Viterbi algorithm. The accuracy rate of 11 monosyllables recognition was 97.2%, which suggests the possibility of voice recognition of Korean language through quantitative facial movement analysis.

Time domain Filtering of Image for Lip-reading Enhancement (시간영역 이미지 필터링에 의한 립리딩 성능 향상)

  • Lee Jeeeun;Kim Jinyoung;Lee Joohun
    • Proceedings of the Acoustical Society of Korea Conference
    • /
    • autumn
    • /
    • pp.45-48
    • /
    • 2001
  • 립리딩은 잡음 환경 하에서 음성 인식 성능을 향상을 위해 영상정보를 이용한 바이모달(bimodal)음성인식으로 연구되었다[1][2]. 그 일환으로 이미 영상정보를 이용한 립리딩은 구현되었다. 그러나 현재까지의 시스템들은 환경의 변화에 강인하지 못하다. 본 논문에서는 이미지 기반 립리딩 방법을 적용하여 입술 영역을 보다 안정적으로 찾아 성능을 향상 시켰다. 그러나 이 방법은 많은 데이터량을 처리해야 하므로 전처리 과정이 필요하다. 전처리로 입력영상을 그레이 레벨로 변환하는 방법과, 입술을 반으로 접는 방법, 그리고 주성분 분석(PCA: Principal Component Analysis)을 사용하였다. 또한 인식성능 향상을 위해 음성에서 잡음 제거나 분석$\cdot$합성에 효과적인 성능을 보이는 RASTA(Relative Spectral)필터를 적용하여 시간 영역에서의 변화가 적은 성분이나 급변하는 성분, 그 밖의 잡음 등을 제거하였다. 그 결과 $72.7\%$의 높은 인식 성능을 보였다.

  • PDF

Lipreading과 음성인식에 의한 향상된 화자 인증 시스템

  • 지승남;이종수
    • 제어로봇시스템학회:학술대회논문집
    • /
    • 2000.10a
    • /
    • pp.274-274
    • /
    • 2000
  • In the future, the convenient speech command system will become an widely-using interface in automation systems. But the previous research in speech recognition didn't give satisfactory recognition results for the practical realization in the noise environment. The purpose of this research is the development of a practical system, which reliably recognizes the speech command of the registered users, by complementing an existing research which used the image information with the speech signal. For the lip-reading feature extraction from a image, we used the DWT(Discrete Wavelet Transform), which reduces the size and gives useful characteristics of the original image. And to enhance the robustness to the environmental changes of speakers, we acquired the speech signal by stereo method. We designed an economic stand-alone system, which adopted a Bt829 and an AD1819B with a TMS320C31 DSP based add-on board.

  • PDF