BULLETIN of the

POLISH ACADEMY of SCIENCES

TECHNICAL SCIENCES

 BULLETIN of the POLISH ACADEMY of SCIENCES: TECHNICAL SCIENCES
Volume 60, Issue 2, June 2012

Issue Index Authors Index Scope Index  Web Info
Aims&Scope, Subscription Editors Authors' guide to read PDF files mirror: http://fluid.ippt.gov.pl/~bulletin/
pp 307 - 316

PDF -  1,1 MB

Characteristics of the use of coupled hidden Markov models for audio-visual Polish speech recognition

M. KUBANEK, J. BOBULSKI, and L. ADRJANOWICZ
This paper focuses on combining audio-visual signals for Polish speech recognition in conditions of the highly disturbed audio speech signal. Recognition of audio-visual speech was based on combined hidden Markov models (CHMM). The described methods were developed for a single isolated command, nevertheless their effectiveness indicated that they would also work similarly in continuous audiovisual speech recognition. The problem of a visual speech analysis is very difficult and computationally demanding, mostly because of an extreme amount of data that needs to be processed. Therefore, the method of audio-video speech recognition is used only while the audiospeech signal is exposed to a considerable level of distortion. There are proposed the authors' own methods of the lip edges detection and a visual characteristic extraction in this paper. Moreover, the method of fusing speech characteristics for an audio-video signal was proposed and tested. A significant increase of recognition effectiveness and processing speed were noted during tests - for properly selected CHMM parameters and an adequate codebook size, besides the use of the appropriate fusion of audio-visual characteristics. The experimental results were very promising and close to those achieved by leading scientists in the field of audio-visual speech recognition.
Key words:

coupled hidden Markov models, audio-visual speech recognition, lip reading

Issue Index Authors Index Scope Index  Web Info
Aims&Scope, Subscription Editors Authors' guide to read PDF files
Copyright ® Bulletin of the Polish Academy of Sciences: Technical Sciences