- Title
- Do p values lose their meaning in exploratory analyses? It depends how you define the familywise error rate
- Creator
- Rubin, Mark
- Relation
- Review of General Psychology Vol. 21, Issue 3, p. 269-275
- Publisher Link
- http://dx.doi.org/10.1037/gpr0000123
- Publisher
- American Psychological Association
- Resource Type
- journal article
- Date
- 2017
- Description
- Several researchers have recently argued that p values lose their meaning in exploratory analyses due to an unknown inflation of the alpha level (e.g., Nosek & Lakens, 2014; Wagenmakers, 2016). For this argument to be tenable, the familywise error rate must be defined in relation to the number of hypotheses that are tested in the same study or article. Under this conceptualization, the familywise error rate is usually unknowable in exploratory analyses because it is usually unclear how many hypotheses have been tested on a spontaneous basis and then omitted from the final research report. In the present article, I argue that it is inappropriate to conceptualize the familywise error rate in relation to the number of hypotheses that are tested. Instead, it is more appropriate to conceptualize familywise error in relation to the number of different tests that are conducted on the same null hypothesis in the same study. Under this conceptualization, alpha-level adjustments in exploratory analyses are (a) less necessary and (b) objectively verifiable. As a result, p values do not lose their meaning in exploratory analyses.
- Subject
- confirmatory research; exploratory research; familywise error rate; p values; type I errors
- Identifier
- http://hdl.handle.net/1959.13/1386725
- Identifier
- uon:32453
- Identifier
- ISSN:1089-2680
- Rights
- © American Psychological Association, 2017. This paper is not the copy of record and may not exactly replicate the authoritative document published in the APA journal. Please do not copy or cite without author's permission. The final article is available, upon publication, at: http://dx.doi.org/10.1037/gpr0000123.
- Language
- eng
- Full Text
- Reviewed
- Hits: 1148
- Visitors: 1419
- Downloads: 283
| Thumbnail | File | Description | Size | Format | |||
|---|---|---|---|---|---|---|---|
| View Details Download | ATTACHMENT02 | Author final version | 396 KB | Adobe Acrobat PDF | View Details Download |