Raising the value of research studies in psychological science by increasing the credibility of research reports : the transparent Psi project

Kekecs, ZoltanLund University,Lunds universitet,Institutionen för psykologi,Samhällsvetenskapliga institutioner och centrumbildningar,Samhällsvetenskapliga fakulteten,Department of Psychology,Departments of Administrative, Economic and Social Sciences,Faculty of Social Sciences,Eötvös Loránd University (author)

The low reproducibility rate in social sciences has produced hesitation among researchers in accepting published findings at their face value. Despite the advent of initiatives to increase transparency in research reporting, the field is still lacking tools to verify the credibility of research reports. In the present paper, we describe methodologies that let researchers craft highly credible research and allow their peers to verify this credibility. We demonstrate the application of these methods in a multi-laboratory replication of Bem's Experiment 1 (Bem 2011 J. Pers. Soc. Psychol. 100, 407-425. (doi:10.1037/a0021524)) on extrasensory perception (ESP), which was co-designed by a consensus panel including both proponents and opponents of Bem's original hypothesis. In the study we applied direct data deposition in combination with born-open data and real-time research reports to extend transparency to protocol delivery and data collection. We also used piloting, checklists, laboratory logs and video-documented trial sessions to ascertain as-intended protocol delivery, and external research auditors to monitor research integrity. We found 49.89% successful guesses, while Bem reported 53.07% success rate, with the chance level being 50%. Thus, Bem's findings were not replicated in our study. In the paper, we discuss the implementation, feasibility and perceived usefulness of the credibility-enhancing methodologies used throughout the project.

Palfi, BenceImperial College London (author)
Szaszi, BarnabasEötvös Loránd University (author)
Szecsi, PeterEötvös Loránd University (author)
Zrubka, MarkUniversity of Amsterdam (author)
Kovacs, MartonEötvös Loránd University (author)
Bakos, Bence E.Eötvös Loránd University (author)
Cousineau, DenisUniversity of Ottawa (author)
Tressoldi, PatrizioUniversity of Padova (author)
Schmidt, KathleenAshland University,Southern Illinois University, Carbondale (author)
Grassi, MassimoUniversity of Padova (author)
Evans, Thomas RhysUniversity of Greenwich (author)
Yamada, YukiKyushu University (author)
Miller, Jeremy K. (author)
Liu, HuanxuKyushu University (author)
Yonemitsu, Fumiya (author)
Dubrov, DmitriiNational Research University Higher School of Economics, Moscow (author)
Röer, Jan PhilippWitten/Herdecke University (author)
Becker, MarvinWitten/Herdecke University (author)
Schnepper, RoxaneWitten/Herdecke University (author)
Ariga, Atsunori (author)
Arriaga, Patrícia (author)
Oliveira, Raquel (author)
Põldver, NeleUniversity of Tartu (author)
Kreegipuu, KairiUniversity of Tartu (author)
Hall, BraedenSouthern Illinois University, Carbondale (author)
Wiechert, SeraHebrew University of Jerusalem (author)
Verschuere, Bruno (author)
Girán, KyraEötvös Loránd University (author)
Aczel, BalazsEötvös Loránd University (author)
Institutionen för psykologiSamhällsvetenskapliga institutioner och centrumbildningar (creator_code:org_t)

About the subject

Kungliga biblioteket hanterar dina personuppgifter i enlighet med EU:s dataskyddsförordning (2018), GDPR. Läs mer om hur det funkar här.
Så här hanterar KB dina uppgifter vid användning av denna tjänst.