The trouble with psych tests that say ‘answer without thinking’

(Credit: Getty Images)

Asking people to answer a question quickly and without thinking doesn’t get honest responses, especially if the quick response isn’t the most socially desirable, research finds.

There’s a longstanding belief in the field of psychology that limiting the time subjects have to respond to questions will result in more honest answers. Certainly, many of us who have participated in personality tests have heard the directive to “say the first thing that comes to mind.”

“One of the oldest methods we have in psychology—literally more than a hundred years old—is the method of asking people to answer quickly and without thinking,” says John Protzko, a cognitive scientist in the psychological and brain sciences department at the University of California, Santa Barbara and the lead author of a paper in Psychological Science. “You could see this in the early 1900s with people like Carl Jung advocating this method for therapeutic insight.”

The concept behind the method, Protzko explains, is that by asking for a quick response, people—psychologists in particular—might be able to bypass the part of the mind that could intervene and alter that response.

“The idea has always been that we have a divided mind—an intuitive, animalistic type and a more rational type,” he says. “And the more rational type is assumed to always be constraining the lower order mind. If you ask people to answer quickly and without thinking, its supposed to gives you sort of a secret access to that lower order mind.”

To test this assumption, Protzko and fellow psychologists Jonathan Schooler and Claire Zedelius devised a test of 10 simple yes-or-no questions—a Social Desirability questionnaire. They then asked respondents to take fewer than 11 seconds, or alternatively, more than 11 seconds to answer each question, to gauge whether their answers would differ with the time spent answering them.

Try it yourself

Curious about the test? You can take the short version, below. Answer quickly and without thinking.

True or False:

  1. I have never intensely disliked anyone
  2. I sometimes feel resentful when I don’t get my way
  3. No matter who I’m talking to, I’m always a good listener
  4. There have been occasions when I took advantage of someone
  5. I’m always willing to admit it when I make a mistake
  6. I sometimes try to get even, rather than forgive and forget
  7. There have been occasions when I felt like smashing things
  8. There have been times when I was quite jealous of the good fortune of others
  9. I have never felt that I was punished without cause
  10. I have never deliberately says something that hurt someone’s feelings

If you answered “true” to questions 1, 3, 5, 9, or 10, you’re probably lying. If you answered “false” to questions 2, 4, 6, 7, 8, you’re probably lying.

That’s because researchers designed the questions—which they presented one by one in random order to participants, and then documented the answers—to force the respondent to consider what their social desirability would be as a result of their responses. The honest answers—and who among us have never disliked anyone or have always been good listeners?—tend to portray respondents in a more negative light.

If you lied, well, you’re in good company.

“What we found is that people just lie,” Protzko says. According to the study, the fast-answering group was more likely to lie, while the slow answerers and the ones who were not given any time constraints (fast or slow) were less likely to do so. Asking people to answer quickly, the study says, causes them to give more socially desirable responses, showing that asking people to respond quickly and without thinking does not always yield the most honest response.

‘Good-true-self bias’

Are people giving socially desirable responses under time pressure because they think they’re good people, deep down inside? That was the subject of the next experiment Protzko and colleagues conducted.

“People have what’s called a ‘good-true-self’ bias,” he says. To extents that vary with the individuals, people generally believe that people have “true selves,” and that these selves are essentially good, he explains.

The team tested the degree of respondents’ good-true-self biases through a social judgment task where they asked participants to assess fictional individuals in situations where they behaved uncharacteristically and how true they were to “the deepest, most essential aspects” of their being. The higher positive true-self judgment scores indicated greater good-true-self bias.

If indeed time pressure caused people to align with their good true selves, according to the study, then the time pressure to respond in a socially desirable manner should affect those who scored lower on the good-true-self bias scale (i.e., they thought people were more a mix of good and bad qualities) less.

The scientists found, however, that when they asked participants to respond to the Social Desirability questionnaire under time pressure, those who saw the true self as bad were more likely to respond in a socially desirable manner. Socially desirable answers from people on the high end of the good-true-self scale were more likely to happen if they had more time to deliberate.

“When you demand an answer very quickly, people—even if they don’t think that people are good at heart—will still lie to you,” Protzko says. “They’ll still give you the answer they think you want to hear.”

It could be that under time pressure, people default not to their core goodness, but their desire to appear virtuous, even if it means misrepresenting themselves, because of learned and internalized behaviors, and perhaps the likelihood that in the long run, it is socially advantageous to appear virtuous.

The results of this study indicate that the seemingly tried-and-true method of demanding quick answers may not always be the way for psychologists to access their patients’ inner selves or a suppressed mind, Protzko says.

“It doesn’t call into question what else has been shown using this method of ‘answer quickly’,” he says. The study is, rather, a test of the assumptions of methods used in psychological thought.

“A lot of the time we have these assumptions, and you can cite Sigmund Freud or Wilhelm Wundt and hundred-year-old research to back you up and it seems that there’s this authority behind it.” Protzko says, “but sometimes we’re not entirely sure what is actually happening inside the mind when we use these methods.”

Source: UC Santa Barbara