1. |
- Menkveld, Albert J., et al.
(author)
-
Nonstandard Errors
- 2024
-
In: JOURNAL OF FINANCE. - : Wiley-Blackwell. - 0022-1082 .- 1540-6261. ; 79:3, s. 2339-2390
-
Journal article (peer-reviewed)abstract
- In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty-nonstandard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for more reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.
|
|
2. |
|
|
3. |
- Perignon, Christophe, et al.
(author)
-
Computational Reproducibility in Finance : Evidence from 1,000 Tests
- 2024
-
In: Review of Financial Studies. - : Oxford Univ Press. - 1465-7368 .- 0893-9454.
-
Journal article (peer-reviewed)abstract
- We analyze the computational reproducibility of more than 1,000 empirical answers to 6 research questions in finance provided by 168 research teams. Running the researchers' code on the same raw data regenerates exactly the same results only 52% of the time. Reproducibility is higher for researchers with better coding skills and those exerting more effort. It is lower for more technical research questions, more complex code, and results lying in the tails of the distribution. Researchers exhibit overconfidence when assessing the reproducibility of their own research. We provide guidelines for finance researchers and discuss implementable reproducibility policies for academic journals.
|
|
4. |
- Pérignon, Christophe, et al.
(author)
-
Reproducibility of Empirical Results : Evidence from 1,000 Tests in Finance
- 2022
-
In: SSRN Electronic Journal. - Paris : HEC Paris. - 1556-5068.
-
Other publication (other academic/artistic)abstract
- We analyze the computational reproducibility of more than 1,000 empirical answers to six research questions in finance provided by 168 international research teams. Surprisingly, neither researcher seniority, nor the quality of the research paper seem related to the level of reproducibility. Moreover, researchers exhibit strong overconfidence when assessing the reproducibility of their own research and underestimate the difficulty faced by their peers when attempting to reproduce their results. We further find that reproducibility is higher for researchers with better coding skills and for those exerting more effort. It is lower for more technical research questions and more complex code
|
|