PRINCETON (US) — To figure out how someone is really feeling, don’t just read their lips, watch their body language.
In a recent study, researchers asked participants to determine from photographs if people were experiencing feelings such as loss, victory, or pain from facial expressions or body language alone, or from both. In some cases, a facial expression associated with one emotion was paired with a body experiencing the opposite emotion.
In four separate experiments, participants more accurately guessed the pictured emotion based on body language—alone or combined with facial expressions—than on facial context alone. These results, reported in the journal Science, challenge the clinical—and conventional—presumption that the face best communicates feeling, says Alexander Todorov, senior researcher and professor of psychology at Princeton University.
Indeed, despite the findings, a majority of the study’s participants sided with the face when asked how they gauge feelings, a misconception the researchers referred to as “illusory facial affect.”
“We find that extremely positive and extremely negative emotions are maximally indistinctive,” says Todorov, who worked with first author Hillel Aviezer, a psychology professor at Hebrew University of Jerusalem and Yaacov Trope, a psychology professor at New York University.
“People can’t tell the difference, although they think they can,” Todorov says. “Subjectively people think they can tell the difference, but objectively they are totally at (random) chance of determining correctly. The message of this research is that there is a lot of information in body language people aren’t necessarily aware of.”
The paper counters popular theories holding that facial expressions are universally consistent indicators of emotion. The most prominent, Todorov says, have been developed by psychologist and University of California-San Francisco professor emeritus Paul Ekman, whose work was fictionalized in the television series Lie to Me.
Instead, facial movements may be “much blurrier” than those theories account for. In particular, when emotions reach a certain intensity, the intricacies of facial expressions become lost, similar to “increasing the volume on stereo speakers to the point that it becomes completely distorted,” Todorov says.
“There’s much more ambiguity in the face than we assume there is,” he explains. “We assume that the face conveys whatever is in the person’s mind, that we can recognize their emotions. But that’s not necessarily true. If we remove all the other contextual clues, we might not be so good at picking out emotional cues.”
The work demonstrates in a new way that the physical cues of emotion are more varied and dependent on the emotion felt than predominating theories suggest, says Jamin Halberstadt, a psychology professor at the University of Otago in New Zealand, who is familiar with the research but had no role it.
Based on theories of facial expression, one would intuit that intense emotions would be even easier to interpret from the face than subtle emotions, says Halberstadt, who studies cognitive-emotion interactions. Yet the new research demonstrates that facial movements at some point become secondary to the body.
“Before I read this paper, I would have thought that the body only provides contextual clues,” Halberstadt says. “This is not saying that bodily context helps interpret an expression of emotion—it is saying that bodily context is the expression of emotion. And the face reveals a general intensity of feeling but doesn’t communicate what the person is feeling exactly. The body is where the valid information comes from during intense feelings.”
The new research introduces an additional element to interpreting emotions that scientists have to account for. In particular, interrogation and security-screening techniques—such as the US Transportation Security Administration’s Screening of Passengers by Observation Techniques (SPOT) program—have been developed based on facial expression research. The new work suggests that a crucial bodily element may have been overlooked.
“This study really questions the primacy of the face in emotion,” Halberstadt says. “Real emotional expressions are much more ambiguous, subtle and malleable than you would think from the research. Any application of emotion theory that relies on or assumes that emotional expressions reside primarily in the face should be under reconsideration from this kind of study.”
Emotional peaks and valleys
For the new study, researchers used stock photos of people at six emotional “peaks”: pain, pleasure, victory, defeat, grief, and joy.
In the first experiment, three groups of 15 people were shown only the facial expression, the body position, or the face and body together, respectively. Participants who saw the face only had a 50-50 chance of being correct. Those who only saw a body or the face and body together were far more accurate.
Yet, these respondents also exhibited a high degree of illusory facial affect: 53 percent of people who saw the body-and-face photos said they relied on the face. Of a group for whom the pictures were described but not shown, 80 percent said they would rely solely on the face when determining the emotion pictured, while 20 percent said they would look to the face and the body together. No one indicated they would judge by body language alone.
In the second experiment, photos were manipulated so that faces from one emotional peak such as victory were spliced onto a body from an opposing peak such as defeat. In those cases, participants more often determined the emotion to be that associated with the body.
For the third experiment, participants rated a variety of faces that fell within the six emotional categories with ambiguous results. In fact, the authors report, respondents interpreted the positive faces as negative more than they did the negative faces. Those faces were then randomly put upon bodies in a situation of victory or pain, and victory or defeat. Again, study participants typically guessed the situation in accordance with what they gleaned from the body rather than the face.
The final experiment asked participants to mimic the facial expressions in the photos for victory and defeat. Those images were put onto corresponding or opposing body images of victory or defeat. A separate group of people then had to determine the feeling being shown in each image. As in the previous experiments, the body language more often influenced respondents, who labeled a feeling negative when a winning face was on a body of defeat, and vice versa.
If anything, Todorov says, the findings promote a more holistic view of understanding how people physically communicate feelings.
“This research involved very clear cases of positive and negative experiences, and yet people cannot tell them apart from the face. There are lots of cues that help us in the social environment, but we often think the face has this special status, that we can tell so much from it. In reality it tells us much less than we think.”
The National Science Foundation supported the research.
Source: Princeton University