Targeting next generations to change the common practice of underpowered research

Abstract

Underpowered studies persist in the scientific literature, which leads to biased conclusions. Moreover, participants are a scarce resource. Using up this scarce resource for an underpowered study means that other (adequately powered) studies will have a harder time recruiting participants. In the current paper, we argue that practices that are ubiquitous in our curricula have the unintended effect of promoting underpowered research, and we recommend easy-to-implement solutions. For example, evidence from underpowered studies is presented as sufficiently high-quality evidence to draw conclusions in textbooks and when students collect data (e.g., as part of a student project or to fulfil requirements for a Bachelor’s or Master’s degree), they are often permitted to collect datasets lacking the power to draw conclusions. These practices disseminate a norm of a typical study consisting of dozens, rather than hundreds or thousands, of participants. The first venue through which this dysfunctional norm of underpowered research can be remedied is by updating course materials. The second venue is to target the next generations by means of relatively small changes in the curricula of undergraduate and graduate degrees. This is a fairly simple, but achievable structural change – in comparison with, for example, changing funding policies – that has a large impact as the students of today are the researchers and policymakers of tomorrow. We believe that correcting what we teach students to reflect best practices can contribute to eliminating underpowered studies.