| 1. |
- Camerer, C. F., et al.
(författare)
-
Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015
- 2018
-
Ingår i: Nature Human Behaviour. - : Springer Science and Business Media LLC. - 2397-3374. ; 2:9, s. 637-644
-
Tidskriftsartikel (refereegranskat)abstract
- Being able to replicate scientific findings is crucial for scientific progress1-15. We replicate 21 systematically selected experimental studies in the social sciences published in Nature and Science between 2010 and 201516-36. The replications follow analysis plans reviewed by the original authors and pre-registered prior to the replications. The replications are high powered, with sample sizes on average about five times higher than in the original studies. We find a significant effect in the same direction as the original study for 13 (62%) studies, and the effect size of the replications is on average about 50% of the original effect size. Replicability varies between 12 (57%) and 14 (67%) studies for complementary replicability indicators. Consistent with these results, the estimated truepositive rate is 67% in a Bayesian analysis. The relative effect size of true positives is estimated to be 71%, suggesting that both false positives and inflated effect sizes of true positives contribute to imperfect reproducibility. Furthermore, we find that peer beliefs of replicability are strongly related to replicability, suggesting that the research community could predict which results would replicate and that failures to replicate were not the result of chance alone.
|
|
| 2. |
- Altmejd, Adam
(författare)
-
Education & replication : essays on the determinants of college choice and the predictability of lab replications
- 2018
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- This Ph.D. thesis is a collection of three research articles.In "Sibling Influence on College Choice" the author studies how preferences for university schooling are affected by the education experience of siblings. An individual is considerably more likely to apply to a specific program if their sibling studies it. The effect is driven by siblings having a preference for going to the same institution. It is twice as strong when both siblings are male, but does not change with neither parental education nor the popularity of the program. A possible explanation is that siblings follow each other out of convenience."Relative Returns to Swedish College Fields" is a paper about the economic returns to different college orientations. Even though Sweden has the lowest average college premium in the developed world, differences between fields can be large. Medicine and engineering have returns of over $10,000 per year compared to other fields. Humanities, on the other hand, has a large enough negative payoff that the degree holder likely earns less than those who do not go to university at all. Interestingly, applicants do not seem to care much about low returns when chosing what to study, but instead prioritize non-pucuniary benefits of college education."Predicting Replication" covers a completely different topic. In the paper, the author team design a simple machine learning algorithmic that predicts the outcomes of laboratory experiment replications in Psychology and Economics. The model is very accurate, on par with the forecasts of experts. It could be used to make better decisions about which studies to replicate, and thus increase cost-effectiveness of replication efforts.
|
|
| 3. |
- Altmejd, Adam, et al.
(författare)
-
Inequality and COVID-19 in Sweden : Relative risks of nine bad life events, by four social gradients, in pandemic vs. prepandemic years
- 2023
-
Ingår i: Proceedings of the National Academy of Sciences - PNAS. - : National Academy of Sciences. - 1091-6490 .- 0027-8424. ; 120:46
-
Tidskriftsartikel (refereegranskat)abstract
- The COVID-19 pandemic struck societies directly and indirectly, not just challenging population health but disrupting many aspects of life. Different effects of the spreading virus—and the measures to fight it—are reported and discussed in different scientific fora, with hard-to-compare methods and metrics from different traditions. While the pandemic struck some groups more than others, it is difficult to assess the comprehensive impact on social inequalities. This paper gauges social inequalities using individual-level administrative data for Sweden’s entire population. We describe and analyze the relative risks for different social groups in four dimensions—gender, education, income, and world region of birth—to experience three types of COVID-19 incidence, as well as six additional negative life outcomes that reflect general health, access to medical care, and economic strain. During the pandemic, the overall population faced severe morbidity and mortality from COVID-19 and saw higher all-cause mortality, income losses and unemployment risks, as well as reduced access to medical care. These burdens fell more heavily on individuals with low income or education and on immigrants. Although these vulnerable groups experienced larger absolute risks of suffering the direct and indirect consequences of the pandemic, the relative risks in pandemic years (2020 and 2021) were conspicuously similar to those in prepandemic years (2016 to 2019).
|
|
| 4. |
- Altmejd, Adam, et al.
(författare)
-
Nowcasting COVID-19 statistics reported with delay : A case-study of Sweden and the UK
- 2023
-
Ingår i: International Journal of Environmental Research and Public Health. - : MDPI. - 1661-7827 .- 1660-4601. ; 20:4
-
Tidskriftsartikel (refereegranskat)abstract
- The COVID-19 pandemic has demonstrated the importance of unbiased, real-time statistics of trends in disease events in order to achieve an effective response. Because of reporting delays, real-time statistics frequently underestimate the total number of infections, hospitalizations and deaths. When studied by event date, such delays also risk creating an illusion of a downward trend. Here, we describe a statistical methodology for predicting true daily quantities and their uncertainty, estimated using historical reporting delays. The methodology takes into account the observed distribution pattern of the lag. It is derived from the "removal method"-a well-established estimation framework in the field of ecology.
|
|
| 5. |
- Altmejd, Adam, et al.
(författare)
-
O Brother, Where Start Thou? Sibling Spillovers on College and Major Choice in Four Countries
- 2021
-
Ingår i: Quarterly Journal of Economics. - : Oxford University Press. - 0033-5533 .- 1531-4650. ; 136:3, s. 1831-1886
-
Tidskriftsartikel (refereegranskat)abstract
- Family and social networks are widely believed to influence important life decisions, but causal identification of those effects is notoriously challenging. Using data from Chile, Croatia, Sweden, and the United States, we study within-family spillovers in college and major choice across a variety of national contexts. Exploiting college-specific admissions thresholds that directly affect older but not younger siblings’ college options, we show that in all four countries a meaningful portion of younger siblings follow their older sibling to the same college or college-major combination. Older siblings are followed regardless of whether their target and counterfactual options have large, small, or even negative differences in quality. Spillover effects disappear, however, if the older sibling drops out of college, suggesting that older siblings’ college experiences matter. That siblings influence important human capital investment decisions across such varied contexts suggests that our findings are not an artifact of particular institutional detail but a more generalizable description of human behavior. Causal links between the postsecondary paths of close peers may partly explain persistent college enrollment inequalities between social groups, and this suggests that interventions to improve college access may have multiplier effects.
|
|
| 6. |
- Altmejd, Adam, et al.
(författare)
-
Predicting the replicability of social science lab experiments
- 2019
-
Ingår i: PLOS ONE. - : Public Library of Science (PLoS). - 1932-6203. ; 14:12
-
Tidskriftsartikel (refereegranskat)abstract
- We measure how accurately replication of experimental results can be predicted by black-box statistical models. With data from four large-scale replication projects in experimental psychology and economics, and techniques from machine learning, we train predictive models and study which variables drive predictable replication. The models predicts binary replication with a cross-validated accuracy rate of 70% (AUC of 0.77) and estimates of relative effect sizes with a Spearman ρ of 0.38. The accuracy level is similar to market-aggregated beliefs of peer scientists [1, 2]. The predictive power is validated in a pre-registered out of sample test of the outcome of [3], where 71% (AUC of 0.73) of replications are predicted correctly and effect size correlations amount to ρ = 0.25. Basic features such as the sample and effect sizes in original papers, and whether reported effects are single-variable main effects or two-variable interactions, are predictive of successful replication. The models presented in this paper are simple tools to produce cheap, prognostic replicability metrics. These models could be useful in institutionalizing the process of evaluation of new findings and guiding resources to those direct replications that are likely to be most informative.
|
|
| 7. |
|
|
| 8. |
- Camerer, C. F., et al.
(författare)
-
Evaluating replicability of laboratory experiments in economics
- 2016
-
Ingår i: Science. - : American Association for the Advancement of Science (AAAS). - 0036-8075 .- 1095-9203. ; 351:6280, s. 1433-1436
-
Tidskriftsartikel (refereegranskat)abstract
- The replicability of some scientific findings has recently been called into question. To contribute data about replicability in economics, we replicated 18 studies published in the American Economic Review and the Quarterly Journal of Economics between 2011 and 2014. All of these replications followed predefined analysis plans that weremade publicly available beforehand, and they all have a statistical power of at least 90% to detect the original effect size at the 5% significance level. We found a significant effect in the same direction as in the original study for 11 replications (61%); on average, the replicated effect size is 66% of the original. The replicability rate varies between 67% and 78% for four additional replicability indicators, including a prediction market measure of peer beliefs.
|
|
| 9. |
|
|
| 10. |
- Dreber Almenberg, Anna, et al.
(författare)
-
Using prediction markets to forecast research evaluations
- 2015
-
Ingår i: Royal Society Open Science. - : Royal Society, The: Open Access / Royal Society. - 2054-5703. ; 2:10
-
Tidskriftsartikel (refereegranskat)abstract
- The 2014 Research Excellence Framework (REF2014) was conducted to assess the quality of research carried out at higher education institutions in the UK over a 6 year period. However, the process was criticized for being expensive and bureaucratic, and it was argued that similar information could be obtained more simply from various existing metrics. We were interested in whether a prediction market on the outcome of REF2014 for 33 chemistry departments in the UK would provide information similar to that obtained during the REF2014 process. Prediction markets have become increasingly popular as a means of capturing what is colloquially known as the 'wisdom of crowds', and enable individuals to trade 'bets' on whether a specific outcome will occur or not. These have been shown to be successful at predicting various outcomes in a number of domains (e.g. sport, entertainment and politics), but have rarely been tested against outcomes based on expert judgements such as those that formed the basis of REF2014.
|
|
