| 1. |
- Grossmann, Igor, et al.
(författare)
-
Insights into the accuracy of social scientists' forecasts of societal change
- 2023
-
Ingår i: Nature Human Behaviour. - : Springer Nature. - 2397-3374. ; 7, s. 484-501
-
Tidskriftsartikel (refereegranskat)abstract
- How well can social scientists predict societal change, and what processes underlie their predictions? To answer these questions, we ran two forecasting tournaments testing the accuracy of predictions of societal change in domains commonly studied in the social sciences: ideological preferences, political polarization, life satisfaction, sentiment on social media, and gender-career and racial bias. After we provided them with historical trend data on the relevant domain, social scientists submitted pre-registered monthly forecasts for a year (Tournament 1; N = 86 teams and 359 forecasts), with an opportunity to update forecasts on the basis of new data six months later (Tournament 2; N = 120 teams and 546 forecasts). Benchmarking forecasting accuracy revealed that social scientists' forecasts were on average no more accurate than those of simple statistical models (historical means, random walks or linear regressions) or the aggregate forecasts of a sample from the general public (N = 802). However, scientists were more accurate if they had scientific expertise in a prediction domain, were interdisciplinary, used simpler models and based predictions on prior data. How accurate are social scientists in predicting societal change, and what processes underlie their predictions? Grossmann et al. report the findings of two forecasting tournaments. Social scientists' forecasts were on average no more accurate than those of simple statistical models.
|
|
| 2. |
- Lakens, Daniel, et al.
(författare)
-
Justify your alpha
- 2018
-
Ingår i: Nature Human Behaviour. - : Nature Publishing Group. - 2397-3374. ; 2:3, s. 168-171
-
Tidskriftsartikel (refereegranskat)abstract
- In response to recommendations to redefine statistical significance to P ≤ 0.005, we propose that researchers should transparently report and justify all choices they make when designing a study, including the alpha level.
|
|
| 3. |
- Schweinsberg, Martin, et al.
(författare)
-
Same data, different conclusions : Radical dispersion in empirical results when independent analysts operationalize and test the same hypothesis
- 2021
-
Ingår i: Organizational Behavior and Human Decision Processes. - : Elsevier BV. - 0749-5978 .- 1095-9920. ; 165, s. 228-249
-
Tidskriftsartikel (refereegranskat)abstract
- In this crowdsourced initiative, independent analysts used the same dataset to test two hypotheses regarding the effects of scientists' gender and professional status on verbosity during group meetings. Not only the analytic approach but also the operationalizations of key variables were left unconstrained and up to individual analysts. For instance, analysts could choose to operationalize status as job title, institutional ranking, citation counts, or some combination. To maximize transparency regarding the process by which analytic choices are made, the analysts used a platform we developed called DataExplained to justify both preferred and rejected analytic paths in real time. Analyses lacking sufficient detail, reproducible code, or with statistical errors were excluded, resulting in 29 analyses in the final sample. Researchers reported radically different analyses and dispersed empirical outcomes, in a number of cases obtaining significant effects in opposite directions for the same research question. A Boba multiverse analysis demonstrates that decisions about how to operationalize variables explain variability in outcomes above and beyond statistical choices (e.g., covariates). Subjective researcher decisions play a critical role in driving the reported empirical results, underscoring the need for open data, systematic robustness checks, and transparency regarding both analytic paths taken and not taken. Implications for orga-nizations and leaders, whose decision making relies in part on scientific findings, consulting reports, and internal analyses by data scientists, are discussed.
|
|
| 4. |
- Ademuyiwa, Adesoji O., et al.
(författare)
-
Determinants of morbidity and mortality following emergency abdominal surgery in children in low-income and middle-income countries
- 2016
-
Ingår i: BMJ Global Health. - : BMJ Publishing Group Ltd. - 2059-7908. ; 1:4
-
Tidskriftsartikel (refereegranskat)abstract
- Background: Child health is a key priority on the global health agenda, yet the provision of essential and emergency surgery in children is patchy in resource-poor regions. This study was aimed to determine the mortality risk for emergency abdominal paediatric surgery in low-income countries globally.Methods: Multicentre, international, prospective, cohort study. Self-selected surgical units performing emergency abdominal surgery submitted prespecified data for consecutive children aged <16 years during a 2-week period between July and December 2014. The United Nation's Human Development Index (HDI) was used to stratify countries. The main outcome measure was 30-day postoperative mortality, analysed by multilevel logistic regression.Results: This study included 1409 patients from 253 centres in 43 countries; 282 children were under 2 years of age. Among them, 265 (18.8%) were from low-HDI, 450 (31.9%) from middle-HDI and 694 (49.3%) from high-HDI countries. The most common operations performed were appendectomy, small bowel resection, pyloromyotomy and correction of intussusception. After adjustment for patient and hospital risk factors, child mortality at 30 days was significantly higher in low-HDI (adjusted OR 7.14 (95% CI 2.52 to 20.23), p<0.001) and middle-HDI (4.42 (1.44 to 13.56), p=0.009) countries compared with high-HDI countries, translating to 40 excess deaths per 1000 procedures performed.Conclusions: Adjusted mortality in children following emergency abdominal surgery may be as high as 7 times greater in low-HDI and middle-HDI countries compared with high-HDI countries. Effective provision of emergency essential surgery should be a key priority for global child health agendas.
|
|
| 5. |
- Graham, Jesse R., et al.
(författare)
-
The pipeline project: Pre-publication independent replications of a single laboratory's research pipeline
- 2016
-
Ingår i: Journal of Experimental Social Psychology. - : Elsevier. - 1096-0465 .- 0022-1031. ; 66, s. 55-67
-
Tidskriftsartikel (refereegranskat)abstract
- This crowdsourced project introduces a collaborative approach to improving the reproducibility of scientific research, in which findings are replicated in qualified independent laboratories before (rather than after) they are published. Our goal is to establish a non-adversarial replication process with highly informative final results. To illustrate the Pre-Publication Independent Replication (PPIR) approach, 25 research groups conducted replications of all ten moral judgment effects which the last author and his collaborators had “in the pipeline” as of August 2014. Six findings replicated according to all replication criteria, one finding replicated but with a significantly smaller effect size than the original, one finding replicated consistently in the original culture but not outside of it, and two findings failed to find support. In total, 40% of the original findings failed at least one major replication criterion. Potential ways to implement and incentivize pre-publication independent replication on a large scale are discussed.
|
|
| 6. |
- Washburn, Anthony N., et al.
(författare)
-
Data from a pre-publication independent replication initiative examining ten moral judgement effects
- 2016
-
Ingår i: Scientific Data. - : Nature Research (part of Springer Nature): Fully open access journals / Nature Publishing Group. - 2052-4463. ; 3
-
Tidskriftsartikel (refereegranskat)abstract
- We present the data from a crowdsourced project seeking to replicate findings in independent laboratories before (rather than after) they are published. In this Pre-Publication Independent Replication (PPIR) initiative, 25 research groups attempted to replicate 10 moral judgment effects from a single laboratory's research pipeline of unpublished findings. The 10 effects were investigated using online/lab surveys containing psychological manipulations (vignettes) followed by questionnaires.
|
|
