The Role of Cinematic Visual Context in Supporting Viewers’ Language Processing

Abstract

Previous research has demonstrated that basic forms of visual context such as object identification and gaze support language comprehension. However, complex forms of narrative context may structure visual supports for language in ways research has yet to reflect. I investigated how cinematic cues such as edit timing and shot coverages (such as depicting actors in close-ups as opposed to wider views) support language comprehension. Participants were shown scenes that either maintain or disrupt the timing of cuts, shot coverage, and other elements of visual context. Participants were tested for their memory for conversation and theory of mind accuracy, as well as reported their perception of continuity from each scene. The experimental conditions had a significant effect on memory for conversation performance but not on theory of mind inference. Memory performance was significantly decreased in the slideshow and reordered conditions, and perceived continuity was significantly decreased for all three conditions compromising the original scene’s visual context.