Media release

From: The Royal Society

When communicating with others, we typically use not only speech but also visual signals, such as hand gestures. We know that gestures can support speech when auditory information is disrupted, but it is unclear whether speech is used to compensate for gestures when visual information is disrupted. We asked pairs of individuals have unscripted conversations over a video-calling set-up, while degrading the visual quality over several steps. We found that not only are gesture characteristics adapted to the visual quality, but people speak louder to compensate for when gestures visibility is impoverished. Our findings suggest that speech and gesture are two channels of communication that should be seen as being on par with one another. 

• “YOU’VE FROZEN!” – Dodgy video on a call causes us to raise our voices, but not for the reason you might think. This study explored whether speech is used to compensate for gestures when visual information is disrupted. As researchers reduced the image quality of a video call, participants would speak louder when it reached the point that gestures were obscured, demonstrating that speech and gestures play equally important roles in communication.