Abstract

These studies examined the role of sentence and visual context in the access to verb-complement information, using a new eye tracker and change blindness paradigm. Participants' eye movements were monitored as they viewed pictures of objects (Experiment 1) or dynamic scenes (Experiment 2), and listened to related sentences. In Experiment 1, two sets of sentences were contrasted, a highly constraining causative construction in which there was a close conceptual relation between the verb and its direct object (e.g., "The woman burned the candle ") and a neutral construction with a transitive perception verb (e.g., "The woman admired the candle "). Starting at three different points within the presentation of the verb (onset, middle, offset) and noun (onset, offset, offset+200 ms), participants' task was to look and indicate whether the target objects mentioned in the sentences (e.g., " candle ") were present in the visual displays. Results indicate that semantic information extracted at the verb can be used to constrain the domain of reference in the scene and in some cases predict the referent of the grammatical complement of the verb, depending on tasks demands, conceptual consolidation, of the scene, and the presence of competitor objects. In Experiment 2, two different classes of verbs were contrasted, denominal and non-denominal verbs, which either implicitly (e.g., "The woman will iron the shirt ") or explicitly (e.g., "The woman will chop the vegetables with the knife ") named the instrument nouns. These movies were edited, unbeknownst to the participants, so that the real referents of the verb's grammatical object (e.g., " shirt/vegetables ") or instrument (e.g., " iron/knife ") gradually dissolved. Participants' task was to provide a detailed description of each scene, and later perform a recognition task. Although results indicate that information extracted from the sentence helped identify, describe and remember scene details, visual context seems to take precedence over linguistic input properties in guiding eye movements. In conclusion, the processor appears to incrementally integrate all available knowledge, linguistic and non-linguistic, with the aim to rapidly interpret the linguistic description of what we see in the world and how we may interact with it.