| 1. |
- Botvinik-Nezer, Rotem, et al.
(författare)
-
Variability in the analysis of a single neuroimaging dataset by many teams
- 2020
-
Ingår i: Nature. - : Springer Science and Business Media LLC. - 0028-0836 .- 1476-4687. ; 582, s. 84-88
-
Tidskriftsartikel (refereegranskat)abstract
- Data analysis workflows in many scientific domains have become increasingly complex and flexible. Here we assess the effect of this flexibility on the results of functional magnetic resonance imaging by asking 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses(1). The flexibility of analytical approaches is exemplified by the fact that no two teams chose identical workflows to analyse the data. This flexibility resulted in sizeable variation in the results of hypothesis tests, even for teams whose statistical maps were highly correlated at intermediate stages of the analysis pipeline. Variation in reported results was related to several aspects of analysis methodology. Notably, a meta-analytical approach that aggregated information across teams yielded a significant consensus in activated regions. Furthermore, prediction markets of researchers in the field revealed an overestimation of the likelihood of significant findings, even by researchers with direct knowledge of the dataset(2-5). Our findings show that analytical flexibility can have substantial effects on scientific conclusions, and identify factors that may be related to variability in the analysis of functional magnetic resonance imaging. The results emphasize the importance of validating and sharing complex analysis workflows, and demonstrate the need for performing and reporting multiple analyses of the same data. Potential approaches that could be used to mitigate issues related to analytical variability are discussed. The results obtained by seventy different teams analysing the same functional magnetic resonance imaging dataset show substantial variation, highlighting the influence of analytical choices and the importance of sharing workflows publicly and performing multiple analyses.
|
|
| 2. |
- Aczel, Balazs, et al.
(författare)
-
Consensus-based guidance for conducting and reporting multi-analyst studies
- 2021
-
Ingår i: eLIFE. - : eLife Sciences Publications. - 2050-084X. ; 10
-
Tidskriftsartikel (refereegranskat)abstract
- Any large dataset can be analyzed in a number of ways, and it is possible that the use of different analysis strategies will lead to different results and conclusions. One way to assess whether the results obtained depend on the analysis strategy chosen is to employ multiple analysts and leave each of them free to follow their own approach. Here, we present consensus-based guidance for conducting and reporting such multi-analyst studies, and we discuss how broader adoption of the multi-analyst approach has the potential to strengthen the robustness of results and conclusions obtained from analyses of datasets in basic and applied research.
|
|
| 3. |
- Albrecht, Eva, et al.
(författare)
-
Telomere length in circulating leukocytes is associated with lung function and disease
- 2014
-
Ingår i: European Respiratory Journal. - : European Respiratory Society (ERS). - 0903-1936 .- 1399-3003. ; 43:4, s. 983-992
-
Tidskriftsartikel (refereegranskat)abstract
- Several clinical studies suggest the involvement of premature ageing processes in chronic obstructive pulmonary disease (COPD). Using an epidemiological approach, we studied whether accelerated ageing indicated by telomere length, a marker of biological age, is associated with COPD and asthma, and whether intrinsic age-related processes contribute to the interindividual variability of lung function. Our meta-analysis of 14 studies included 934 COPD cases with 15 846 controls defined according to the Global Lungs Initiative (GLI) criteria (or 1189 COPD cases according to the Global Initiative for Chronic Obstructive Lung Disease (GOLD) criteria), 2834 asthma cases with 28 195 controls, and spirometric parameters (forced expiratory volume in is (FEV1), forced vital capacity (PVC) and FEV1/FVC) of 12 595 individuals. Associations with telomere length were tested by linear regression, adjusting for age, sex and smoking status. We observed negative associations between telomere length and asthma (beta= -0.0452, p= 0.024) as well as COPD (beta= -0.0982, p=0.001), with associations being stronger and more significant when using GLI criteria than those of GOLD. In both diseases, effects were stronger in females than males. The investigation of spirometric indices showed positive associations between telomere length and FEV1 (p=1.07 x 10(-7)), FVC (p=2.07 x 10(-5)), and FEV1/FVC (p =5.27 x 10(-3)). The effect was somewhat weaker in apparently healthy subjects than in COPD or asthma patients. Our results provide indirect evidence for the hypothesis that cellular senescence may contribute to the pathogenesis of COPD and asthma, and that lung function may reflect biological ageing primarily due to intrinsic processes, which are likely to be aggravated in lung diseases.
|
|
| 4. |
- Altmejd, Adam, et al.
(författare)
-
Predicting the replicability of social science lab experiments
- 2019
-
Ingår i: PLOS ONE. - : Public Library of Science (PLoS). - 1932-6203. ; 14:12
-
Tidskriftsartikel (refereegranskat)abstract
- We measure how accurately replication of experimental results can be predicted by black-box statistical models. With data from four large-scale replication projects in experimental psychology and economics, and techniques from machine learning, we train predictive models and study which variables drive predictable replication. The models predicts binary replication with a cross-validated accuracy rate of 70% (AUC of 0.77) and estimates of relative effect sizes with a Spearman ρ of 0.38. The accuracy level is similar to market-aggregated beliefs of peer scientists [1, 2]. The predictive power is validated in a pre-registered out of sample test of the outcome of [3], where 71% (AUC of 0.73) of replications are predicted correctly and effect size correlations amount to ρ = 0.25. Basic features such as the sample and effect sizes in original papers, and whether reported effects are single-variable main effects or two-variable interactions, are predictive of successful replication. The models presented in this paper are simple tools to produce cheap, prognostic replicability metrics. These models could be useful in institutionalizing the process of evaluation of new findings and guiding resources to those direct replications that are likely to be most informative.
|
|
| 5. |
- Keller, Ulrich, et al.
(författare)
-
Myc suppression of Nfkb2 accelerates lymphomagenesis
- 2010
-
Ingår i: BMC Cancer. - : Springer Science and Business Media LLC. - 1471-2407 .- 1471-2407. ; 10
-
Tidskriftsartikel (refereegranskat)abstract
- Background: Deregulated c-Myc expression is a hallmark of several human cancers where it promotes proliferation and an aggressive tumour phenotype. Myc overexpression is associated with reduced activity of Rel/NF-kappa B, transcription factors that control the immune response, cell survival, and transformation, and that are frequently altered in cancer. The Rel/NF-kappa B family member NFKB2 is altered by chromosomal translocations or deletions in lymphoid malignancies and deletion of the C-terminal ankyrin domain of NF-kappa B2 augments lymphocyte proliferation. Methods: Precancerous E mu-Myc-transgenic B cells, E mu-Myc lymphomas and human Burkitt lymphoma samples were assessed for Nfkb2 expression. The contribution of Nfkb2 to Myc-driven apoptosis, proliferation, and lymphomagenesis was tested genetically in vivo. Results: Here we report that the Myc oncoprotein suppresses Nfkb2 expression in vitro in primary mouse fibroblasts and B cells, and in vivo in the E mu-Myc transgenic mouse model of human Burkitt lymphoma (BL). NFKB2 suppression by Myc was also confirmed in primary human BL. Promoter-reporter assays indicate that Myc-mediated suppression of Nfkb2 occurs at the level of transcription. The contribution of Nfkb2 to Myc-driven lymphomagenesis was tested in vivo, where Nfkb2 loss was shown to accelerate lymphoma development in E mu-Myc transgenic mice, by impairing Myc's apoptotic response. Conclusions: Nfkb2 is suppressed by c-Myc and harnesses Myc-driven lymphomagenesis. These data thus link Myc-driven lymphomagenesis to the non-canonical NF-kappa B pathway.
|
|
| 6. |
- Klionsky, Daniel J., et al.
(författare)
-
Guidelines for the use and interpretation of assays for monitoring autophagy
- 2012
-
Ingår i: Autophagy. - : Informa UK Limited. - 1554-8635 .- 1554-8627. ; 8:4, s. 445-544
-
Forskningsöversikt (refereegranskat)abstract
- In 2008 we published the first set of guidelines for standardizing research in autophagy. Since then, research on this topic has continued to accelerate, and many new scientists have entered the field. Our knowledge base and relevant new technologies have also been expanding. Accordingly, it is important to update these guidelines for monitoring autophagy in different organisms. Various reviews have described the range of assays that have been used for this purpose. Nevertheless, there continues to be confusion regarding acceptable methods to measure autophagy, especially in multicellular eukaryotes. A key point that needs to be emphasized is that there is a difference between measurements that monitor the numbers or volume of autophagic elements (e.g., autophagosomes or autolysosomes) at any stage of the autophagic process vs. those that measure flux through the autophagy pathway (i.e., the complete process); thus, a block in macroautophagy that results in autophagosome accumulation needs to be differentiated from stimuli that result in increased autophagic activity, defined as increased autophagy induction coupled with increased delivery to, and degradation within, lysosomes (in most higher eukaryotes and some protists such as Dictyostelium) or the vacuole (in plants and fungi). In other words, it is especially important that investigators new to the field understand that the appearance of more autophagosomes does not necessarily equate with more autophagy. In fact, in many cases, autophagosomes accumulate because of a block in trafficking to lysosomes without a concomitant change in autophagosome biogenesis, whereas an increase in autolysosomes may reflect a reduction in degradative activity. Here, we present a set of guidelines for the selection and interpretation of methods for use by investigators who aim to examine macroautophagy and related processes, as well as for reviewers who need to provide realistic and reasonable critiques of papers that are focused on these processes. These guidelines are not meant to be a formulaic set of rules, because the appropriate assays depend in part on the question being asked and the system being used. In addition, we emphasize that no individual assay is guaranteed to be the most appropriate one in every situation, and we strongly recommend the use of multiple assays to monitor autophagy. In these guidelines, we consider these various methods of assessing autophagy and what information can, or cannot, be obtained from them. Finally, by discussing the merits and limits of particular autophagy assays, we hope to encourage technical innovation in the field.
|
|
| 7. |
- McCarthy, Randy J., et al.
(författare)
-
Registered Replication Report on Srull and Wyer (1979)
- 2018
-
Ingår i: Advances in Methods and Practices in Psychological Science. - : SAGE Publications Inc. - 2515-2459 .- 2515-2467. ; 1:3, s. 321-336
-
Tidskriftsartikel (refereegranskat)abstract
- Srull and Wyer (1979) demonstrated that exposing participants to more hostility-related stimuli caused them subsequently to interpret ambiguous behaviors as more hostile. In their Experiment 1, participants descrambled sets of words to form sentences. In one condition, 80% of the descrambled sentences described hostile behaviors, and in another condition, 20% described hostile behaviors. Following the descrambling task, all participants read a vignette about a man named Donald who behaved in an ambiguously hostile manner and then rated him on a set of personality traits. Next, participants rated the hostility of various ambiguously hostile behaviors (all ratings on scales from 0 to 10). Participants who descrambled mostly hostile sentences rated Donald and the ambiguous behaviors as approximately 3 scale points more hostile than did those who descrambled mostly neutral sentences. This Registered Replication Report describes the results of 26 independent replications (N = 7,373 in the total sample; k = 22 labs and N = 5,610 in the primary analyses) of Srull and Wyer?s Experiment 1, each of which followed a preregistered and vetted protocol. A random-effects meta-analysis showed that the protagonist was seen as 0.08 scale points more hostile when participants were primed with 80% hostile sentences than when they were primed with 20% hostile sentences (95% confidence interval, CI = [0.004, 0.16]). The ambiguously hostile behaviors were seen as 0.08 points less hostile when participants were primed with 80% hostile sentences than when they were primed with 20% hostile sentences (95% CI = [?0.18, 0.01]). Although the confidence interval for one outcome excluded zero and the observed effect was in the predicted direction, these results suggest that the currently used methods do not produce an assimilative priming effect that is practically and routinely detectable.
|
|
| 8. |
- Menkveld, Albert J., et al.
(författare)
-
Non-Standard Errors
- 2021
-
Annan publikation (övrigt vetenskapligt/konstnärligt)abstract
- In statistics, samples are drawn from a population in a data generating process (DGP). Standard errors measure the uncertainty in sample estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence generating process (EGP). We claim that EGP variation across researchers adds uncertainty: non-standard errors. To study them, we let 164 teams test six hypotheses on the same sample. We find that non-standard errors are sizeable, on par with standard errors. Their size (i) co-varies only weakly with team merits, reproducibility, or peer rating, (ii) declines significantly after peer-feedback, and (iii) is underestimated by participants.
|
|
| 9. |
- Menkveld, Albert J., et al.
(författare)
-
Non-Standard Errors
- 2024
-
Ingår i: Journal of Finance. - 0022-1082. ; 79:3, s. 2339-2390
-
Tidskriftsartikel (refereegranskat)abstract
- In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty—nonstandard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for more reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.
|
|
| 10. |
- Menkveld, Albert J., et al.
(författare)
-
Nonstandard Errors
- 2024
-
Ingår i: JOURNAL OF FINANCE. - : Wiley-Blackwell. - 0022-1082 .- 1540-6261. ; 79:3, s. 2339-2390
-
Tidskriftsartikel (refereegranskat)abstract
- In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty-nonstandard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for more reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.
|
|
