Daniel Kahneman talks extensively about how we make reasoning errors because we tend to use mental shortcuts. One mental shortcut is ‘substitution‘. Substitution is what we do when we (often unconsciously) answer an easier question than the one being asked. I find that I sometimes do this in my own research. For instance, when I set out to answer the question, “How can X be rational?” I sometimes end up answering easier questions like, “How does X work?”. In an effort to avoid such mistakes, I will (1) explain the question substitution error, (2) give an example of how we can distinguish between questions, (3) give a personal example of the substitution error, and (4) say what we can do about it.
1. Substitution
In case you’re not familiar with Kahnemen’s notion of ‘substitution’, here is some clarification. In short, substitution is this: responding to a difficult question by (often unintentionally) answering a different, easier question. People use this mental shortcut all the time. Here are some everyday instances:
| Difficult Question | Easier Question |
|---|---|
| How satisfied are you with your life? | What is my mood right now? |
| Should I believe what my parents believe? | Can I believe what my parents believe? |
| What are the merits/demerits of that woman who is running for president? | What do I remember people in my community saying about that woman? |
For further discussion of mental shortcuts and substitution, see Part 1 of Kahneman’s Thinking Fast and Slow (2012).
Now, how does this mental shortcut apply to research?
2. Some Research Questions
Regular readers will know that I am interested in — among other things — the differences between reflective and unreflective reasoning (Pennycook et al 2015). Philosophers ask lots of questions that involve reflective reasoning:
Normative Questions
- Must I do this? (Korsgaard 1996)
- Should I believe this? (Sosa 2009)
Explanatory questions
- Why do I believe that? (Kornblith 2012)
- Why did I do that? (Doris 2015; Vargas 2005)
Self-knowledge questions
- What do we know about our own minds? (Cassam 2015)
- How is my relationship to my own mind different than my relationship to other minds? (Moran 2001).
Obviously there are other questions that involve reflective reasoning. These are just a few. And, obviously, there are also hybrid questions that involve reflective reasoning. For instance, there are normative-explanatory questions like “Why is it wrong to X?” And there are normative-explanatory-self-knowledge questions like, “Why do I believe that it is wrong to X?
The point of this section is just to point out that we can — when we are careful — discern clear differences between similar questions.
3. Falling Prey To Substitution
Like any researcher, I don’t want to make an erroneous mental shortcut in my research. Alas, I sometimes find myself doing exactly that. For instance, I will substitute an answer to an easier question for an answer to my more difficult, research question.
| Research Question | Easier Question |
|---|---|
| How can reflective reasoning be rational? | How is reflective reasoning related to our behavior? |
| How does reflective reasoning work? | |
| How does reflective reasoning fail? |
I sometimes mistake answers to these easier questions for answers to the target question. (Based on my reading of the literature, I do not think that I am the only one who does this. But that’s a much longer conversation.)
Don’t get me wrong. We might have to answer the easier questions as a means to an answer to the target question. But we shouldn’t mistake the means for the end. That is, we can’t think that the answers to the easier questions just are answers to the target question.
4. General Take-aways
Consuming research. When a researcher tells us that they plan to answer a certain question, we should take note of that question. And then we should pay attention to see whether their research, argument, etc. can and/or actually does answer their target question. If the research can’t or doesn’t answer the original question, then we should take note of it. Heck, we might want to do more — e.g., tell the researcher.
Doing research. Once we pick a research question, it is important to regularly remind ourselves of that question throughout our enquiry. Otherwise we might lose sight of the target question. And when we do that, we might end up substituting the target question with an easier question. And, worse, we might then erroneously think that our answer to the easier question — fruitful, though it may be — is an answer to the original, target question.
Sometimes though the ‘hard question’ is ill-formed or otherwise doesn’t admit of an answer, and it would be a mistake not to shift to the ‘easier’ questions.
The obvious example of this as a rhetorical move is the opening paragraphs of Turing’s Computing Machinery and Intelligence, but I think similar issues come up with e.g. ‘the meaning of life’ and ‘the hard problem of consciousness,’ which are just less fruitful than the questions ‘what makes life worth living’ and ‘how does cognition work,’ respectively.
Great point. As long as that kind of substitution is made explicit and argued for by the researcher(s), that seems like an acceptable/good move. After all, that’d be less of a careless shortcut and more of a wise detour.
I never heard of Substitution questions, but I know as a lit person we do very different research. What your talking about reminds me of critical thinking, however, in reverse. You ask the easier question and answer it, but then you change the language of the question to create a deeper thought so it becomes more of critical thinking. As lit teachers like to say “dig deeper” I like to joke back that you can only dig so deep before you hit lava.
I have question that does not pertain to the topic of this post, but do you have any post or ever thought about writing a blog about the pros and cons of getting a PHD or if someone is thinking about getting a PHD?
-Francesca
Interesting. I’m not familiar with that kind of substitution. Thanks for sharing it.
Regarding your question about getting a PhD, I have a series on that which starts here: https://byrdnick.com/archives/5746/grad-school-1. I hope that’s helpful.
Peace!