The Combined Role of Representational Redundant Gesture and Filled Pauses on Narrative Recall

Abstract

Real-world language is multimodal, incorporating a variety of rich communicative cues such as gesture and disfluency into the speech stream. These cues serve a range of important communicative and cognitive functions that offer benefits for both the speakers and the listeners in everyday discourse. For example, both gesture and disfluency are linked to enhancing the recall of spoken information. Gestures can help listeners better understand and remember spoken information through increasing language comprehension. Similarly, disfluencies, such as filled pauses like “um” or “uh,” often signal speaker hesitation or planning and have been linked to enhanced attention and improved recall in listeners. Although these two cues frequently co-occur during natural speech and are temporally intertwined, much of the current literature has examined their effects separately. As a result, there lacks a combined account for how they interact and jointly shape memory for what was said. To address this gap, we analyzed the combined impact of both gesture and disfluency on the recall of short narratives. Participants viewed a series of videos of a narrator telling short stories and were immediately asked to recount each story. We manipulated whether the narrator produced filler disfluencies (i.e., UM), representational redundant gestures, or both. We hypothesized that both gesture and disfluency would independently enhance overall passage recall, particularly for portions of the narratives where these features were present. Furthermore, we predicted that the simultaneous presence of both cues would have an additive or superadditive effect, resulting in better recall for the combined condition compared to either cue alone. Results from a mixed-effect logistic regression model indicated no significant difference between conditions, or their interactions, providing no evidence for the hypothesis that gesture or disfluency improve memory for spoken narrative.