• The Listener
  • North & South
  • Noted
  • RNZ
Photo/Getty Images

How facial expressions can fool us into misreading emotions

Think you can read people’s expressions like a book? Forget it, says psychologist Lisa Feldman Barrett, author of How Emotions Are Made, ahead of an appearance at the NZ Festival of the Arts. 

The classical view of how emotions work is straightforward: we’re born with a set of circuits in our brain, one for anger, one for sadness, one for fear and so on. When we see a threat, our fear circuit triggers, we make a scared face, our heart speeds up and we run away. Right? Completely wrong, says Lisa Feldman Barrett. The professor of psychology at Northeastern University in Boston and Harvard Medical School psychiatry lecturer talks about how viewing emotions as constructed by our brain may help us better deal with anxiety and depression, how the modern world seems set up to disrupt the human nervous system, and why for the sake of justice we need to jettison the idea of universal emotional expressions.

The subtitle of your book How Emotions Are Made is “The Secret Life of the Brain”. Can you explain constructed emotions?

There’s not a lot of evidence for the classical view of emotions. The actual evidence suggests something more complex. Imagine your brain is running a budget for your body, budgeting glucose and salt and water and all of the nutrients your body needs to stay alive and well. Your brain is receiving sensory inputs from the body and from the world – sights and sounds and so on – that are the effects of some set of causes. If you have an ache in your gut or you experience a flash of light or a change in air pressure, the brain doesn’t have access to the causes; it has access only to the effects. So it has to guess at the causes. The only thing it has available is the past experiences that it can reconstitute in its neurons. Your brain is basically asking itself, “Well, the last time I was in this kind of situation, what caused these sensations?” That’s why your heart racing or your hands sweating can be determination or it can be fear. An ache in your gut can be anxiety, or it can be longing for someone that you miss or a gut feeling that someone is guilty of a crime. You certainly can take control of that process when you need to, but most of the time it’s happening without your awareness.

Lisa Feldman Barrett. Photo/Supplied

If we can change our perceptions of emotions, such as feeling excitement in place of anxiety in public speaking, how real are they?

If your heart is racing, that’s real. If you are experiencing anxiety, that’s real. The question is, could you take that racing heart and transform it, by giving it a different meaning, into a different experience that would also be real? You can. It’s not that you have anxiety lurking somewhere in your brain and body and you’re just deceiving yourself by turning it into determination.

It’s like chronic pain. In chronic pain, the brain believes that there’s tissue damage in the body. Probably there was at one point, and the body was sending sense data back to the brain to tell it, “Hey, there’s damage here.” The brain learns that, but as the body healed, it didn’t change its beliefs about what was going on in the body. So, the person continues to feel pain. Well, is that pain real? Yes, it’s real, it’s happening in your brain. But you can learn to change the activity of your brain to give it a different meaning and construct a different feeling, a completely different emotion.

Stress, pain, anxiety – your book suggests that all these are constructed in some sense by your brain.

When I say that something is constructed by your brain, I don’t mean it’s imaginary. Everything you experience from when you’re born to when you die is constructed in your brain by neurons. Your brain is doing this predictively. It doesn’t wait around to receive sensory inputs and then ask itself, gee, what are these similar to? It predicts what’s going to happen next by changing the firing of its own neurons. It starts to construct what you’re going to see and hear, to anticipate what bodily sensations you’re going to feel, and what the cause of them is. It makes these neural guesses, then it waits for information to come from the world and the body. And that information either confirms the brain’s guesses or it changes them.

And depression, too?

What happens when your body budget is really running a deficit? What are the two most expensive things from a metabolic standpoint a brain can do? Move your body and learn something new. Those are also two major symptoms of depression: fatigue and what’s called context insensitivity, meaning your brain is making all these predictions, but it’s not checking them against the world. It’s just going with its own beliefs. So, if you are depressed, what happened to you probably is that, basically, you have a metabolic illness, your body budget is out of whack, you might be running a serious deficit. You feel really crappy and fatigued, and probably you have a lot of negative things going on inside your brain and it isn’t checking them against the world; it’s just assuming that they’re right. That’s depression. And there are lots of ways to get to those states; there are lots of ways to get out of those states. I’m not trivialising depression. I think part of the reason people have such a difficult time with depression is because they don’t realise the number of things in the day that actually impact their body budgets.

To improve your emotional life, you suggest old-fashioned remedies such as eight hours of sleep a night, eating well, exercising, managing social-media usage, yoga, reading novels and spending time with friends. Simple, yet hard in modern life, right?

It’s extremely hard. If I had to design an environment that was deliberately going to disrupt the human nervous system, it would be the one that we live in. We’re social animals; we influence each other’s nervous systems. What’s really hard on a human nervous system is ambiguous social feedback where someone might be evaluating you badly. That’s like all of social media. Basically, a lot of modern life is taxing on our body budget and human nervous system. If you don’t sufficiently replenish them, you’re going to become sick.

Working on our emotional granularity can help?

Emotional granularity is basically just feeding your brain to provide it with experiences so that you can make a wider range of emotions out of the sense data that come from your body and the world. For example, if you don’t know what schadenfreude is, your brain can still make that emotion. Because your brain can do what’s called conceptual combination: take different experiences from the past and combine them in novel ways. If you have a word for it, it actually speeds your brain’s ability to make that experience.

It’s like if you do public speaking and the first time try to take anxiety and transform it into determination, you have to work really hard at it, the next time it’s really hard, the third time maybe it’s a little less hard. Eventually, your brain will pretty automatically conjure up past experiences of determination to make sense of the present racing heart and sweating palms. But you have to work at it like any kind of skill.

Serena Williams. Photo/Getty Images

In your book you have an image of a woman ostensibly screaming in terror – it’s actually Serena Williams winning a tennis match. It’s tricky to identify emotion from people’s faces, isn’t it?

The idea that faces display emotion or that we have “body language” are metaphors that don’t work well in the real world. Your attention may be on someone’s face, but your brain is taking in all sorts of information about the surrounding situation: that person’s body posture, what their voice sounds like, what’s going on in your own body, what just happened a moment ago, what it expects to happen a moment from now and so on.

Also, [four senior authors and I] just published a big paper reviewing more than 1000 published papers showing that people, for example, don’t always scowl when they’re angry – that happens only about 30% of the time. People can cry in anger, smile in anger, laugh in anger, widen their eyes in anger. They can scowl when they’re confused or curious or concentrating, or when they have gas. The idea that expressions are universal and that they’re hard-coded into the brain is just not true. Particularly in this political moment in time, in issues of sexual consent and the MeToo movement, it’s important to understand that your brain is guessing about what a body movement means. And it can guess wrong – very easily.

You write about criminal trials such as that of the Boston Bomber. Given that memory is constructed, eyewitness testimony is flawed, facial expressions are guesses, impartiality is an illusion and neuroscience is misunderstood, you suggest educating jurists about constructed emotions and even of doing away with jury trials.

I don’t really see it as my job to claim what the solution should be, but I do see it as my job to raise questions about potential problems. One problem is that in the Western world, we have a system of laws founded on the principle that a person’s intent matters to how culpable they are if they cause harm. We make a distinction between harm that’s caused deliberately, with intent, and harm that’s caused by accident. That requires a jury or a judge to make an inference about intent. And that is always a guess. Is that stony-faced stare remorse? Is it lack of remorse? Is it guilt? Is it lack of guilt? That wrong guess can cost him his liberty, or his life. In the [Dzhokhar] Tsarnaev case – the Boston bombing took place a mile from my office and he was caught less than two miles away from my house – I’m not in any way saying that this guy isn’t guilty for what he did. And I believe he should be punished with the full force of the law – he harmed a lot of people and it was a very scary time.

But in the US, according to the Supreme Court, someone gets a fair trial only when jurors can know the heart and mind of the defendant. There’s certainly evidence that suggests that well before his trial Tsarnaev wrote letters full of remorse and apology to his victims and those were not entered into the record. It’s my place as a scientist to point out that, in Chechen culture, the thing that you do basically in defeat to honour your enemy is be stoic. But he was given the death penalty. If you listen to what the jurors reported after they were interviewed, it is very much in line with what the research shows, which is that they believed he didn’t feel remorse. It’s an issue with judges, too. We all use our past experiences to make sense of what’s going on in the present moment. And if our experiences are very different from another person’s experiences, then what we honestly experience in that moment will be very different.

Lisa Feldman Barrett will be speaking at the NZ Festival of the Arts in Wellington on Friday, March 6, at 8pm.

This article was first published in the March 7, 2020 issue of the New Zealand Listener.

For more on the political, cultural and literary life of the country, follow us on Facebook, Twitter and Instagram and sign up to our weekly newsletter.