Isolated spanish digit recognition based on audio-visual features

Abstract

The performance of classical speech recognition techniques based on audio features is degraded in noisy environments. The inclu-sion of visual features related to mouth movements into the recogni-tion process improves the performance of the system. This paper proposes an isolated word speech recognition system based on audio-visual features. The proposed system combines three classifiers based on au-dio, visual and audio-visual information, respectively. An audio-visual database composed by the utterances of the digits (in Spanish language) is employed to test the proposed system. The experimental results show a significant improvement on the recognition rates through a wide range of signal-to-noise ratios.