Mar

28

The Garden of Forking Paths

March 28, 2021 |

The garden of forking paths: Why multiple comparisons can be a
problem, even when there is no “fishing expedition” or “p-hacking” and
the research hypothesis was posited ahead of time
 Andrew Gelman† and Eric Loken‡ 14 Nov 2013

Abstract
Researcher degrees of freedom can lead to a multiple comparisons
problem, even in settings where researchers perform only a single
analysis on their data. The problem is there can be a large number of
potential comparisons when the details of data analysis are highly
contingent on data, without the researcher having to perform any
conscious procedure of fishing or examining multiple p-values. We
discuss in the context of several examples of published papers where
data-analysis decisions were theoretically-motivated based on previous
literature, but where the details of data selection and analysis were
not pre-specified and, as a result, were contingent on data.

http://www.stat.columbia.edu/~gelman/research/unpublished/forking.pdf


Comments

WordPress database error: [Table './dailyspeculations_com_@002d_dailywordpress/wp_comments' is marked as crashed and last (automatic?) repair failed]
SELECT * FROM wp_comments WHERE comment_post_ID = '13260' AND comment_approved = '1' ORDER BY comment_date

Name

Email

Website

Speak your mind

Archives

Resources & Links

Search