News release

From: PLOS

A new deep-learning based tool allows researchers to delve deeper into monkey social behavior

A new tool that allows researchers to create realistic full-body animations of monkeys has provided the first evidence that non-human primates experience the “uncanny valley” phenomenon for body avatars, according to a study by Lucas Maximilian Martini at the University Clinic Tubingen in Germany and colleagues at KU Leuven in Belgium, publishing July 14^th in the open access journal PLOS Biology.

Macaques are highly social creatures that use both facial and body signals to communicate. Their visual system is also very similar to that of humans. For these reasons, macaques are commonly used in neuroscience, psychology and behavioral research to understand social behavior and perception.

To develop a deeper understanding of how primate brains process social signals, researchers need tools to precisely manipulate the pose of monkey bodies, which is only possible by generating realistic animated 3D avatars. However, standard methods used to create such avatars for humans — such as using body markers for motion tracking — are not feasible for monkeys. To address these challenges, researchers developed a new tool, called “MacAction”, which uses deep learning to generate a realistic avatar from multi-camera video footage of monkey movements. To test the realism of the generated avatars, they showed eight male rhesus macaques (Macaca mulatta) videos of real monkeys and the animated avatar performing the same movements and measured  where, how often, and for how long the monkeys looked at each video. The monkeys’ responses to the real video and the animation were indistinguishable, indicating that they perceive the animated monkey as highly realistic.

Next, the researchers investigated whether macaques experience a phenomenon known as the uncanny valley, first identified in humans, where avatars that are highly realistic, but not quite perfect, are disliked more than unrealistic ones. They showed macaques variations of the 3D avatar with different levels of realism — sequentially removing the fur, color and texture from the animation — and measured their response. They found that monkeys fixated less often at avatars with intermediate levels of realism compared to both very unrealistic avatars and the highly realistic 3D animation. This U-shaped relationship between attention and realism is characteristic of the uncanny valley effect.

Using MacAction, scientists can produce realistic animations of macaque bodies with limited resources, enabling new types of neurological and behavioral research, the authors say. The study also provides the first evidence that non-human primates experience an uncanny valley when viewing artificially generated macaque bodies.

The authors add, “Monkeys, just like humans, respond to realistic body avatars similarly as to real videos. We developed methods to generate such highly realistic avatars and show that monkeys, similar to humans, show an 'uncanny valley', where almost realistic stimuli are liked less than ones that a highly realistic or very unrealistic. The social perception of monkeys
and humans thus exhibit quite similar properties.”

“Since monkeys cannot be motion-captured by using reflecting markers as common in the film industry, we had to develop a new marker-less tracking method with sufficient accuracy for computer animation.”