Abstract

The current study investigated the relation between the bottom-up mechanism of audiovisual speech perception and top-down mechanism of semantic integration, with specific attention paid to working memory (WM). Event-related potentials (ERPs) were recorded from 28 younger adult participants to determine if the neurophysiological reaction to semantic language information (or the absence of it) could be modified by presenting speech in an audiovisual modality or in the auditory modality alone, with the prediction being that audiovisual speech would provide a benefit to processing sentences with an absence of semantic information. The N400 ERP component, a neural indicator of the effortful processing of semantic content, was observed to determine this reaction. Initial analyses did not reveal an interaction between speech modality and degree of semantic content, but a subsequent grouping of participants based upon individual WM capacity yielded significant results. In the auditory modality it was found that while participants with a high WM capacity were able to utilize semantic content to reduce the N400 amplitude, low WM participants had a higher amplitude N400 for both low-constraint for acceptance (LC) and high constraint for acceptance (HC) sentences, indicating significant processing demands. Conversely in the audiovisual modality, low WM participants displayed a reduction in N400 amplitude similar to high WM participants for HC sentences, indicating that the addition of visual speech cues assisted in maintaining the semantic content. The results are discussed with regards to implications for maintaining face-to-face communication, particularly for those individuals with lower WM capacities.