| 1. |
- Anderson, Christopher J., et al.
(författare)
-
Response to Comment on "Estimating the reproducibility of psychological science"
- 2016
-
Ingår i: Science. - : American Association for the Advancement of Science (AAAS). - 0036-8075 .- 1095-9203. ; 351:6277
-
Tidskriftsartikel (övrigt vetenskapligt/konstnärligt)abstract
- Gilbert et al. conclude that evidence from the Open Science Collaboration's Reproducibility Project: Psychology indicates high reproducibility, given the study methodology. Their very optimistic assessment is limited by statistical misconceptions and by causal inferences from selectively interpreted, correlational data. Using the Reproducibility Project: Psychology data, both optimistic and pessimistic conclusions about reproducibility are possible, and neither are yet warranted.
|
|
| 2. |
|
|
| 3. |
- Hardwicke, Tom E., et al.
(författare)
-
Data availability, reusability, and analytic reproducibility : evaluating the impact of a mandatory open data policy at the journal Cognition
- 2018
-
Ingår i: Royal Society Open Science. - : The Royal Society. - 2054-5703. ; 5:8
-
Tidskriftsartikel (refereegranskat)abstract
- Access to data is a critical feature of an efficient, progressive and ultimately self-correcting scientific ecosystem. But the extent to which in-principle benefits of data sharing are realized in practice is unclear. Crucially, it is largely unknown whether published findings can be reproduced by repeating reported analyses upon shared data ('analytic reproducibility'). To investigate this, we conducted an observational evaluation of a mandatory open data policy introduced at the journal Cognition. Interrupted time-series analyses indicated a substantial post-policy increase in data available statements (104/417, 25% pre-policy to 136/ 174, 78% post-policy), although not all data appeared reusable (23/ 104, 22% pre-policy to 85/136, 62%, post-policy). For 35 of the articles determined to have reusable data, we attempted to reproduce 1324 target values. Ultimately, 64 values could not be reproduced within a 10% margin of error. For 22 articles all target values were reproduced, but 11 of these required author assistance. For 13 articles at least one value could not be reproduced despite author assistance. Importantly, there were no clear indications that original conclusions were seriously impacted. Mandatory open data policies can increase the frequency and quality of data sharing. However, suboptimal data curation, unclear analysis specification and reporting errors can impede analytic reproducibility, undermining the utility of data sharing and the credibility of scientific findings.
|
|
| 4. |
- Ingre, Michael, et al.
(författare)
-
Estimating statistical power, posterior probability and publication bias of psychological research using the observed replication rate
- 2018
-
Ingår i: Royal Society Open Science. - : The Royal Society. - 2054-5703. ; 5:9
-
Tidskriftsartikel (refereegranskat)abstract
- In this paper, we show how Bayes' theorem can be used to better understand the implications of the 36% reproducibility rate of published psychological findings reported by the Open Science Collaboration. We demonstrate a method to assess publication bias and show that the observed reproducibility rate was not consistent with an unbiased literature. We estimate a plausible range for the prior probability of this body of research, suggesting expected statistical power in the original studies of 48-75%, producing (positive) findings that were expected to be true 41-62% of the time. Publication bias was large, assuming a literature with 90% positive findings, indicating that negative evidence was expected to have been observed 55-98 times before one negative result was published. These findings imply that even when studied associations are truly NULL, we expect the literature to be dominated by statistically significant findings.
|
|
| 5. |
- Ingre, Michael, 1965-
(författare)
-
P-hacking in academic research : a critical review of the job strain model and of the association between night work and breast cancer in women
- 2017
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- P-hacking can be described as a more or less deliberate, explorative approach to data analysis with a flexible/opportunistic search space and the reporting of primarily statistically significant findings. This leads to inflated type-1 error rates and to bias in reported estimates in the scientific literature.This thesis aims to describe how p-hacking can be manifested in academic research and to illustrate how bias due to p-hacking is expected to affect the veracity of published findings using two specific examples from the literature. This thesis also argues that when evaluating published findings in the current academic environment, we should assume a priori that biases due to p-hacking and publication bias are present.The thesis used Monte Carlo simulations and systematic reviews of the literature in two specific fields: the proposed associations between exposure to night work and breast cancer in women, and between job strain and coronary heart disease.A general model and mathematical framework to predict expected bias from p-hacking was developed, and can be used for a priori defined protected inferences of any published finding, under explicit assumptions of various levels of p-hacking. The model indicated a close to 100% chance of demonstrating a false positive association in larger studies, but also showed that even minimal p-hacking results in substantial bias in estimates.The literature review identified large flexibility in the analytical process, allowing for the final model to be picked from a large pool of available models, with an implied search space of thousands of estimates. Some of the specific observations made here could be used to argue evidence for high risk of p-hacking and publication bias in the reviewed literature:None of the 17 reviewed studies on job strain and coronary heart disease reported the proper estimate of the job strain interaction (chapter 6) and our analysis showed that the proper estimate would not have been statistically significant in any of the studies (chapter 7).One study described a data driven approach with an implied search space of at least 502 models, where adjusting for confounding did not reduce the strength of the association, as would be expected, but instead increased its strength so it fell above the threshold for statistical significance (chapter 5).One study was based on a speculative and marginally significant estimate after arbitrarily restricting the analysis to a subgroup, when estimates on the full group were available and indicated a non-significant association (chapter 5).Statistical power analyses on research into night work and breast cancer indicated that statistically significant findings were over-represented in the literature (p≈.001) suggesting the presence of bias from p-hacking or selective publishing of significant findings (chapter 5).The findings also suggest that previously reported estimates in meta-analyses was likely to represent prevailing bias in the two fields reviewed here. A bias-adjusted meta-analysis on the job strain model and coronary heart disease with a total of 462,220 subjects and 6,836 CHD events indicated no support for the job strain interaction (RR=1.00; 95% CI: 0.88--1.14). In addition, it did not show an increased risk due to high job demand (RR=1.03; 95% CI: 0.97--1.11) but it did confirm previously reported risks due to low job control (RR=1.11; 95% CI: 1.03--1.20).
|
|
| 6. |
- Klein, Olivier, et al.
(författare)
-
A Practical Guide for Transparency in Psychological Science
- 2018
-
Ingår i: Collabra: Psychology. - : University of California Press. - 2474-7394. ; 4:1
-
Forskningsöversikt (refereegranskat)abstract
- The credibility of scientific claims depends upon the transparency of the research products upon which they are based (e.g., study protocols, data, materials, and analysis scripts). As psychology navigates a period of unprecedented introspection, user-friendly tools and services that support open science have flourished. However, the plethora of decisions and choices involved can be bewildering. Here we provide a practical guide to help researchers navigate the process of preparing and sharing the products of their research (e.g., choosing a repository, preparing their research products for sharing, structuring folders, etc.). Being an open scientist means adopting a few straightforward research management practices, which lead to less error prone, reproducible research workflows. Further, this adoption can be piecemeal – each incremental step towards complete transparency adds positive value. Transparent research practices not only improve the efficiency of individual researchers, they enhance the credibility of the knowledge generated by the scientific community.
|
|
| 7. |
- Lakens, Daniel, et al.
(författare)
-
Justify your alpha
- 2018
-
Ingår i: Nature Human Behaviour. - : Nature Publishing Group. - 2397-3374. ; 2:3, s. 168-171
-
Tidskriftsartikel (refereegranskat)abstract
- In response to recommendations to redefine statistical significance to P ≤ 0.005, we propose that researchers should transparently report and justify all choices they make when designing a study, including the alpha level.
|
|
| 8. |
- Lekander, Mats, et al.
(författare)
-
Intrinsic functional connectivity of insular cortex and symptoms of sickness during acute experimental inflammation
- 2016
-
Ingår i: Brain, behavior, and immunity. - : Elsevier BV. - 0889-1591 .- 1090-2139. ; 56, s. 34-41
-
Tidskriftsartikel (refereegranskat)abstract
- Task-based fMRI has been used to study the effects of experimental inflammation on the human brain, but it remains unknown whether intrinsic connectivity in the brain at rest changes during a sickness response. Here, we investigated the effect of experimental inflammation on connectivity between areas relevant for monitoring of bodily states, motivation, and subjective symptoms of sickness. In a double blind randomized controlled trial, 52 healthy volunteers were injected with 0.6 ng/kg LPS (lipopolysaccharide) or placebo, and participated in a resting state fMRI experiment after approximately 2h 45 minutes. Resting state fMRI data were available from 48 participants, of which 28 received LPS and 20 received placebo. Bilateral anterior and bilateral posterior insula sections were used as seed regions and connectivity with bilateral orbitofrontal and cingulate (anterior and middle) cortices was investigated. Back pain, headache and global sickness increased significantly after as compared to before LPS, while a non-significant trend was shown for increased nausea. Compared to placebo, LPS was followed by increased connectivity between left anterior insula and left midcingulate cortex. This connectivity was significantly correlated to increase in back pain after LPS and tended to be related to increased global sickness, but was not related to increased headache or nausea. LPS did not affect the connectivity from other insular seeds. In conclusion, the finding of increased functional connectivity between left anterior insula and middle cingulate cortex suggests a potential neurophysiological mechanism that can be further tested to understand the subjective feeling of malaise and discomfort during a sickness response.
|
|
| 9. |
- Lodin, Karin, et al.
(författare)
-
Cross-sectional associations between inflammation, sickness behaviour, health anxiety and self-rated health in a Swedish primary care population
- 2019
-
Ingår i: European journal of inflammation. - : SAGE Publications. - 2058-7392 .- 1721-727X. ; 17
-
Tidskriftsartikel (refereegranskat)abstract
- This study investigated associations between inflammatory markers, sickness behaviour, health anxiety and self-rated health in 311 consecutive primary care patients. Poor self-rated health was associated with high sickness behaviour (rho = 0.28, P < 0.001; rho = 0.42, P = 0.003) and high health anxiety (rho = 0.31, P < 0.001; rho = -0.32, P = 0.003). High levels of interleukin 6 were associated with poor self-rated health in men (rho = 0.26, P = 0.009). Low levels of interleukin-6 were associated with poor self-rated health in women (rho = -0.15, P = 0.04), but this association was non-significant when adjusted for health anxiety (rho = -0.08, P = 0.31). These results are consistent with the theory that interoceptive processes draw on both inflammatory mediators and the state of sickness behaviour in inferring health state.
|
|
| 10. |
- McGrath, Cormac, et al.
(författare)
-
Data sharing in qualitative research : opportunities and concerns
- 2018
-
Ingår i: MedEdPublish. - : F1000 Research Ltd. - 2312-7996. ; 7:4
-
Tidskriftsartikel (refereegranskat)abstract
- Data sharing is increasingly practiced by researchers and mandated by research funders as well as scientific journals. However, data sharing within qualitative research paradigms is less common, and sharing interview data has particular challenges. Earlier debate has pointed to the value of data sharing for discouraging research fraud and permitting critical scrutiny. We elaborate on this discussion by highlighting the value of data sharing for cumulative science, for re-use, and to maximise the value of the participants’ contribution. We review methods and possibilities for sharing interview data, and give concrete recommendations for mitigating risks to the participants. In conclusion, we find that sharing of interview data is possible, valuable, and ethical, and serves a purpose for both journals and researchers.
|
|
