S.Argyropoulos, D.Tzovaras, and M.G.Strintzis: "Multimodal Fusion for Cued Speech Language Recognition", European Signal Processing Conference (EUSIPCO2007), Poznan, Poland, September 2007.
Abstract:
In this paper, a novel method for Cued Speech language recognition is proposed. A multimodal processing framework is developed for the efficient fusion of the modalities under consideration. The robust feature extraction from the audio, lip shape and gesture modalities is also discussed. Moreover, the inherent correlation among modalities is modelled using a modified Coupled Hidden Markov Model. The contribution of each modality to the overall fused decision is modified by assessing its reliability and assigning a weighting factor in order to improve the accuracy of the recognized words. The method is experimentally evaluated and is shown to outperform methods that rely only on the perceivable information to infer the transmitted messages.
Informatics & Telematics InstituteHome Activities Projects Publications
