, ), and in subjects with intact hearing abilities, substantially facilitates speech perception under noisy environmental conditions, ,. “Lip-reading” or “speech-reading” allows hearing-impaired individuals to understand speech (e.g.

Watching a speaker's facial movements, especially lip movements, provides input that can supplement the information from the speaker's voice. Vision often plays a crucial role in understanding speech. We conclude that auditory-visual speech perception obeys the same notion of optimality previously observed only for simple multisensory stimuli. We confirm this prediction in a behavioral experiment. When the auditory and visual stimuli differ slightly in high noise, the model makes a counterintuitive prediction: as sound quality increases, the proportion of reported words corresponding to the visual stimulus should first increase and then decrease. When the dimensionality of the feature space is low, the Bayesian model predicts inverse effectiveness when the dimensionality is high, the enhancement is maximal at intermediate auditory noise levels. In this model, words are regarded as points in a multidimensional space and word recognition is a probabilistic inference process. We present a Bayesian model of optimal cue integration that can explain these conflicts. In contrast, we find, in accord with a recent paper, that visual information improves performance more at intermediate levels of auditory noise than at the highest levels, and we show that a novel visual stimulus containing only temporal information does the same. From a general doctrine of combining information from different sensory modalities (the principle of inverse effectiveness), one would expect that the visual signals would be most effective at the highest levels of auditory noise. Watching a speaker's facial movements can dramatically enhance our ability to comprehend words, especially in noisy environments.

Lips sound reference