…our outcomes counsel that the [instrumental variables] and, to a lesser extent, [difference-in-difference] analysis our bodies have considerably extra p-hacking and/or selective publication than these based mostly on [randomized controlled trials] and [regression-discontinuity]… (p.3)
We discover no proof that: (1) Papers printed within the ‘High 5’ journals are completely different to others; (2) The journal ‘revise and resubmit’ course of mitigates the issue; (3) Issues are enhancing by time.
That’s from this forthcoming AER paper by Brodeur, Cook dinner, and Hayes.
In distinction, this blog post argues that:
I’ve proposed right here that we should always not infer that literatures with extra bunching simply previous .05 are much less reliable, and that visually putting comparisons of ‘anticipated’ and noticed take a look at outcomes may be fairly deceptive resulting from incorrect assumptions in regards to the anticipated line.
The authors reply here. I don’t but have an opinion on this dispute, however everyone seems to be speaking about it proper now, so I believed I’d a minimum of ship alongside the essential paperwork to you all.