Research Questions & Mental Shortcuts: A Warning


Daniel Kahneman talks extensively about how we make reasoning errors because we tend to use mental shortcuts. One mental shortcut is ‘substitution‘. Substitution is what we do when we (often unconsciously) answer an easier question than the one being asked. I find that I sometimes do this in my own research. For instance, when I set out to answer the question, “How can X be rational?” I sometimes end up answering easier questions like, “How does X work?”. In an effort to avoid such mistakes, I will (1) explain the question substitution error, (2) give an example of how we can distinguish between questions, (3) give a personal example of the substitution error, and (4) say what we can do about it.

1.  Substitution

In case you’re not familiar with Kahnemen’s notion of ‘substitution’, here is some clarification. In short, substitution is this: responding to a difficult question by (often unintentionally) answering a different, easier question. People use this mental shortcut all the time. Here are some everyday instances:

Difficult QuestionEasier Question
How satisfied are you with your life?What is my mood right now?
Should I believe what my parents believe?Can I believe what my parents believe?
What are the merits/demerits of that woman who is running for president?What do I remember people in my community saying about that woman?

For further discussion of mental shortcuts and substitution, see Part 1 of Kahneman’s Thinking Fast and Slow (2012).

Now, how does this mental shortcut apply to research?  Continue reading Research Questions & Mental Shortcuts: A Warning

Grading Shorthand: Quick, Consistent, and Constructive Feedback


Grading with shorthand allows me to grade papers quickly. This is great for me, of course, but —more importantly — it’s great for students. Using grading shorthand means that students get prompt, consistent, and constructive feedback.

I’ve included the key to my grading shorthand below. I’ve also included the printer-friendly, PDF version of the key that I give to students. You are welcome to use and adapt the shorthand, including the PDF, however you see fit — it’s in the public domain. And you are more than welcome to share your own shorthand with me in the comments, on social media, or by contacting me directly. Happy grading!  Continue reading Grading Shorthand: Quick, Consistent, and Constructive Feedback

Visual Brainstorming on Whiteboards for Posters, Slides, and More


A whiteboard is pretty versatile. It can be used many times for many purposes. I use it during meetings and while working alone. In this post, I’ll explain how I use a whiteboard for creating visual aids.

1. Visual Brainstorming

I am very committed to the digital workspace. My library, papers, notes, handouts, etc. are in the cloud (more about that in this post). I do all of my reading and writing on a computer or a smartphone. But very occasionally a physical workspace trumps my digital workspace.

Visual brainstorming is one task for which a physical workspace outshines the digital counterpart. By ‘visual brainstorming’ I just mean  Continue reading Visual Brainstorming on Whiteboards for Posters, Slides, and More

Politicians Defunding Based on Political Bias? Sounds Biased

“I would use the Department of Education … to monitor our institutions of higher education for extreme political bias and deny federal funding if it exists.” –Ben Carson

1. Everyone has biases — political and otherwise.

So denying funding on the basis of any political bias would be tantamount to denying all federal education funding. That’d be problematic. So — if we assume a charitable interpretation of Carson — that’s surely not the Republican plan (…or is it?). So let’s assume that Carson is not out to defund any educational institution that exhibits just any political bias.

Instead, maybe Carson’s plan is to monitor for particular biases. The idea here would be that only institutions with certain biases should be defunded. But even that would be problematic. After all, Carson is a human. And humans are more likely to notice and take issue with others’ biases (Corner et al 2012; Lord et al 1979) or biases that merely seem like others’ biases (Trouche et al 2015, 2018). So Carson might be more attuned to and dismissive of others’ biases than his own. And that itself is a political bias.

To overcome that bias, we would need to make sure that Continue reading Politicians Defunding Based on Political Bias? Sounds Biased

Peer-review: should we get rid of it?


There are way more manuscripts than opportunities for respected peer-reviewed publications (Sinhababu 2016). So many good manuscripts might never be properly reviewed (or published). This would be bad. In this post, I’ll mention a few potential solutions. Then I’ll briefly evaluate one: eliminating compulsory peer-review altogether. 

1.  Peer Review Is New

I learned from Kate Norlock that peer-review is a relatively recent thing.†

… the surprisingly short history of what we now think of as peer-review [Times Higher Ed.] … the Google ngram on peer-review: [Google ngram article] …. suggests that academics have only been so fixated on it as the measure of our worth since the 1970s.

2.  The Current Form of Peer Review Isn’t Obviously Optimal

One reason for peer-review might be that it inhibits bias. And there is some evidence that anonymous peer-review reduces bias (Budden et al 2008). However, a review of 17 studies Continue reading Peer-review: should we get rid of it?

Interview with ACI Scholarly Blog Index


I recently answered some questions from Traci Hector at ACI Scholarly Blog Index. More about the interview and about ACI below.

What We Talked About

  • My experience of studying religion.
  • Why philosophy led me to ‪cognitive science.
  • How intuition is related to beliefs about god and science.
  • Computational corpus linguistics and how philosophers use it.
  • How I use blogging and ‎social media‬ for my research.
  • My thoughts on podcasts.
  • About blogging as an academic.

The full interview is here: http://aci.info/2016/04/27/aci-interview-with-scholarly-blogger-phd-candidate-nick-byrd/

About ACI Scholarly Blog Index

…an editorially created and curated index of scholarly social media. Authors are selected for inclusion based on their academic credentials as well as the scope and quality of their writing. Metadata, taxonomies, and proprietary Author Profile Cards are appended to each publication. An elegantly sophisticated search interface easily surfaces highly relevant articles. Post-search filtering allows researchers to further hone in on appropriate articles. ACI Scholarly Blog Index is free to use.

Check out ACI Scholarly Blog Index at acindex.com

Where to follow ACI:

Twitter @aciblogindex

Facebook facebook.com/aci.info

LinkedIn linkedin.com/company/4859067

Google+ plus.google.com/+AciInfogroup

Peer-review: on what basis should we reject papers?


When you peer-review a paper, you can make one of a few basic recommendations to the editor. One option is this: do not publish the paper.

So what criteria should you use to make such a recommendation? In this post, I argue that some criteria are better than others.

1. Is the paper convincing?

A friend of mine mentioned this criterion the other day: “…[philosophy] papers ought to be convincing.” Call this the Convince Me standard or CM.

Maybe you think that CM sounds like a reasonable standard for peer-review. I don’t.  Continue reading Peer-review: on what basis should we reject papers?