The Science of Asking
The Science of Asking
“Our studies show that despite a strong reputation for being over-optimistic, people appear to have a clear capacity to put optimistic biases aside and to be quite accurate when making predictions.”
- David Armor, Psychologist & Professor Emeritus, Yale University
“Our ‘subjects’ are much smarter than many of us thought they were. Hence, if we don’t get in their way by asking the wrong questions, and if we provide appropriate structure, they often can tell us much about themselves.“
- Walter Mischel, developmental psychologist famous for his “Marshmallow” studies on impulse control
“If we want to know how people feel: what they experience and what they remember, what their emotions and motives are like, and the reasons for acting as they do—we need to ask them.“
- Richard H. Dana, clinical psychologist and author of Multicultural Assessment Perspectives for Professional Psychology
“Language allows mere words to substitute for direct experience, and mere thoughts to simulate states of the world. The brain’s capacity to accept such propositions as valid guides to behavior and emotion, and predictive of future outcomes, explains the power of words. There’s a term we use for this acceptance. We call it belief."
- Sam Harris, neuroscientist & moral philosopher, from the Moral Landscape
In the insights business, most stakeholders don’t pay much attention to the questions we are asking until the data come back with something we can’t explain or something that seems wrong. Once that happens, everybody starts to criticize the research. And in fact, over time, we have seen a lot of fundamental cynicism build up in the world of Primary Market Research (PMR). We synthesize this into three Key Criticisms of PMR that we hear a lot.
3 Deep Criticisms of PMR
“People can’t tell us what they do or what they think. Either they lie, they don’t take the exercise seriously, or they don’t remember.”
“People can’t tell us what they will do in the future. It’s just too complicated – there are too many factors to consider.”
“People can’t say why they do things, so why bother asking?”
Any veteran in our space has encountered one or all of these at some point or another.
The big question is: Are these criticisms fair?
There are empirical answers to that question. Happily, the accumulated science tells us these challenges can be substantially addressed by creating exercises and questions that are appropriate to the way humans think - a process we call The Science of Asking.
The Science of Asking
People Can’t Accurately Tell Us What They Do or Think – What Science Tells Us
Popular books on business psychology have trained us to think that people lack self-insight and are unable to self-assess in a coherent way. This is mostly a by-product of the industry obsession with Behavioral Economics. But this criticism turns out to be mostly wrong.
Perceived PMR Problem
What Do Scientists Have To Say?
- Overall, humans are good at self-assessment (and often better than outside observers).
- In general, the more fluent and informed people are on a topic area, the more accurate their self assessments tend to be.
- Humans are really excellent at frequency estimation in most circumstances. So asking people how often they do things makes sense in most circumstances.
- People use different strategies to answer questions about their behavior – and those strategies vary based on how often the behavior occurs.
- Some topics simply can’t be asked directly, and we need to use alternative techniques to get good answers.
The Science of Asking
People Can’t Really Tell Us Why – What Science Tells Us
Psychologists have also trained us to think that customers have minimal insight into their own motivations. A common metaphor you here with human motivation research involves an iceberg – we think that people can only see a tiny percentage of their own motives and behavioral drivers. But ask yourself some simple questions:
Perceived PMR Problem
- Do you know why you save money in your 401(K)?
- Do you know why you are getting surgery next week?
- Do you know why you are getting on a plane tomorrow?
Do you think YOU could provide high-quality answers to these questions?
In light of this, can a physician say why he/she will use a new medicine?
The Science of Asking
People Can’t Really Tell Us Why – What Science Tells Us
Rather than assuming that asking about motivations is a waste of time, let’s look at what the science can tell us.
Perceived PMR Problem
What Do Scientists Have To Say?
- The more complex and important the decision, the more self-insight people have about the driving beliefs and motivations
- When seemingly irrational behaviors persist, it is usually due to external causes or a failure of understanding.
- Example: A physician may persist in using a suboptimal drug or regimen because she works in a department or institution that has a structural preference for a different therapy.
- Routine and habitual behavior is ordinarily initiated without deliberation, BUT before they become habits, these behaviors are preceded by a period of behavioral trial that is driven by significant thought and evaluation.
- Example: If you have the habit of going to the gym every day, did you know why you were going when you started?
The Science of Asking
People Can’t Predict - What Science Tells Us
And finally, what might be the most common critique of primary research – the problem of predicting the future. We run into this most often with demand studies, forecasts, segmentations and opportunity assessments. The science surrounding human judgment about future behaviors and events is very complex and robust. It also has some big surprises that run counter to many people's expectations.
Perceived PMR Problem
What Do Scientists Have To Say?
- Most people in most instances are very motivated to predict their own futures accurately and seek out information to help themselves do it well.
- Guiding people to think through multiple steps, barriers and competing demands improves prediction accuracy substantially.
- The more similar the future options for behavior, the more accurate people are in their predictions.
- People do much better predicting outcomes that THEY can influence directly (for example, career choice vs. next week’s weather) compared with outcomes that are out of their control.
- There’s a big difference between predicting specific behaviors and predicting the likelihood of world events, and these tasks should not be confused.
- Again, some topics are simply taboo, and we need to use alternate types of prediction tasks to get at these subjects.
