| 1. |
- Ebersole, Charles R., et al.
(författare)
-
Many Labs 5: Testing Pre-Data-Collection Peer Review as an Intervention to Increase Replicability
- 2020
-
Ingår i: Advances in Methods and Practices in Psychological Science. - : Sage. - 2515-2467 .- 2515-2459. ; 3:3, s. 309-331
-
Tidskriftsartikel (refereegranskat)abstract
- Replication studies in psychological science sometimes fail to reproduce prior findings. If these studies use methods that are unfaithful to the original study or ineffective in eliciting the phenomenon of interest, then a failure to replicate may be a failure of the protocol rather than a challenge to the original finding. Formal pre-data-collection peer review by experts may address shortcomings and increase replicability rates. We selected 10 replication studies from the Reproducibility Project: Psychology (RP:P; Open Science Collaboration, 2015) for which the original authors had expressed concerns about the replication designs before data collection; only one of these studies had yielded a statistically significant effect (p < .05). Commenters suggested that lack of adherence to expert review and low-powered tests were the reasons that most of these RP:P studies failed to replicate the original effects. We revised the replication protocols and received formal peer review prior to conducting new replication studies. We administered the RP:P and revised protocols in multiple laboratories (median number of laboratories per original study = 6.5, range = 3-9; median total sample = 1,279.5, range = 276-3,512) for high-powered tests of each original finding with both protocols. Overall, following the preregistered analysis plan, we found that the revised protocols produced effect sizes similar to those of the RP:P protocols (Delta r = .002 or .014, depending on analytic approach). The median effect size for the revised protocols (r = .05) was similar to that of the RP:P protocols (r = .04) and the original RP:P replications (r = .11), and smaller than that of the original studies (r = .37). Analysis of the cumulative evidence across the original studies and the corresponding three replication attempts provided very precise estimates of the 10 tested effects and indicated that their effect sizes (median r = .07, range = .00-.15) were 78% smaller, on average, than the original effect sizes (median r = .37, range = .19-.50).
|
|
| 2. |
- Hardwicke, Tom E., et al.
(författare)
-
Data availability, reusability, and analytic reproducibility : evaluating the impact of a mandatory open data policy at the journal Cognition
- 2018
-
Ingår i: Royal Society Open Science. - : The Royal Society. - 2054-5703. ; 5:8
-
Tidskriftsartikel (refereegranskat)abstract
- Access to data is a critical feature of an efficient, progressive and ultimately self-correcting scientific ecosystem. But the extent to which in-principle benefits of data sharing are realized in practice is unclear. Crucially, it is largely unknown whether published findings can be reproduced by repeating reported analyses upon shared data ('analytic reproducibility'). To investigate this, we conducted an observational evaluation of a mandatory open data policy introduced at the journal Cognition. Interrupted time-series analyses indicated a substantial post-policy increase in data available statements (104/417, 25% pre-policy to 136/ 174, 78% post-policy), although not all data appeared reusable (23/ 104, 22% pre-policy to 85/136, 62%, post-policy). For 35 of the articles determined to have reusable data, we attempted to reproduce 1324 target values. Ultimately, 64 values could not be reproduced within a 10% margin of error. For 22 articles all target values were reproduced, but 11 of these required author assistance. For 13 articles at least one value could not be reproduced despite author assistance. Importantly, there were no clear indications that original conclusions were seriously impacted. Mandatory open data policies can increase the frequency and quality of data sharing. However, suboptimal data curation, unclear analysis specification and reporting errors can impede analytic reproducibility, undermining the utility of data sharing and the credibility of scientific findings.
|
|
| 3. |
- Zhang, Guo-Qiang, et al.
(författare)
-
Exogenous female sex steroid hormones and new-onset asthma in women: a matched case-control study
- 2023
-
Ingår i: BMC medicine. - 1741-7015. ; 21:1
-
Tidskriftsartikel (refereegranskat)abstract
- Evidence on the role of exogenous female sex steroid hormones in asthma development in women remains conflicting. We sought to quantify the potential causal role of hormonal contraceptives and menopausal hormone therapy (MHT) in the development of asthma in women.We conducted a matched case-control study based on the West Sweden Asthma Study, nested in a representative cohort of 15,003 women aged 16-75years, with 8-year follow-up (2008-2016). Data were analyzed using Frequentist and Bayesian conditional logistic regression models.We included 114 cases and 717 controls. In Frequentist analysis, the odds ratio (OR) for new-onset asthma with ever use of hormonal contraceptives was 2.13 (95% confidence interval [CI] 1.03-4.38). Subgroup analyses showed that the OR increased consistently with older baseline age. The OR for new-onset asthma with ever MHT use among menopausal women was 1.17 (95% CI 0.49-2.82). In Bayesian analysis, the ORs for ever use of hormonal contraceptives and MHT were, respectively, 1.11 (95% posterior interval [PI] 0.79-1.55) and 1.18 (95% PI 0.92-1.52). The respective probability of each OR being larger than 1 was 72.3% and 90.6%.Although use of hormonal contraceptives was associated with an increased risk of asthma, this may be explained by selection of women by baseline asthma status, given the upward trend in the effect estimate with older age. This indicates that use of hormonal contraceptives may in fact decrease asthma risk in women. Use of MHT may increase asthma risk in menopausal women.
|
|
