Table 1 of Cassidy et al 2019. Categories of fallacious descriptions of 'statistical significance'

The meaning of ‘statistical significance’ and of p-values

A 2019 paper in the Advances in Methods and Practices in Psychological Science found that most psychology textbooks, instructors, and students misinterpret ‘statistical significance’ and p values. Talk about a headline! More important than the headline, however, are the right interpretations and what we can do to correct widespread misinterpretations. In this post, I explain the authors’ findings and the three solutions they propose.

I decided to explain all of this on Twitter. I’ll include the entire thread below. (Don’t worry. It’s not a Tweetstorm. It’s just 5 tweets. And there’s no moral outrage involved.

So that’s it. It seems like a pretty big and complicated problem. But Cassidy and colleagues do a pretty good job of sorting out the problem, quantifying it, and offering some reasonable solutions. Now for the hard part: implementing the solutions. It’s going to take more than an academic paper and a Twitter thread to course correct our discussions of ‘statistical significance’ and p values. Indeed, it will take lots of action from lots of people—textbook authors, methodological instructors, etc. Solutions that require widespread action are often difficult to achieve—but collective action problems are a topic of another post for another day.

Related Posts

Published by

Nick Byrd

Nick is a cognitive scientist at Florida State University studying reasoning, wellbeing, and willpower. Check out his blog at