| 1. |
- Botvinik-Nezer, Rotem, et al.
(författare)
-
Variability in the analysis of a single neuroimaging dataset by many teams
- 2020
-
Ingår i: Nature. - : Springer Science and Business Media LLC. - 0028-0836 .- 1476-4687. ; 582, s. 84-88
-
Tidskriftsartikel (refereegranskat)abstract
- Data analysis workflows in many scientific domains have become increasingly complex and flexible. Here we assess the effect of this flexibility on the results of functional magnetic resonance imaging by asking 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses(1). The flexibility of analytical approaches is exemplified by the fact that no two teams chose identical workflows to analyse the data. This flexibility resulted in sizeable variation in the results of hypothesis tests, even for teams whose statistical maps were highly correlated at intermediate stages of the analysis pipeline. Variation in reported results was related to several aspects of analysis methodology. Notably, a meta-analytical approach that aggregated information across teams yielded a significant consensus in activated regions. Furthermore, prediction markets of researchers in the field revealed an overestimation of the likelihood of significant findings, even by researchers with direct knowledge of the dataset(2-5). Our findings show that analytical flexibility can have substantial effects on scientific conclusions, and identify factors that may be related to variability in the analysis of functional magnetic resonance imaging. The results emphasize the importance of validating and sharing complex analysis workflows, and demonstrate the need for performing and reporting multiple analyses of the same data. Potential approaches that could be used to mitigate issues related to analytical variability are discussed. The results obtained by seventy different teams analysing the same functional magnetic resonance imaging dataset show substantial variation, highlighting the influence of analytical choices and the importance of sharing workflows publicly and performing multiple analyses.
|
|
| 2. |
- Ebersole, Charles R., et al.
(författare)
-
Many Labs 5: Testing Pre-Data-Collection Peer Review as an Intervention to Increase Replicability
- 2020
-
Ingår i: Advances in Methods and Practices in Psychological Science. - : Sage. - 2515-2467 .- 2515-2459. ; 3:3, s. 309-331
-
Tidskriftsartikel (refereegranskat)abstract
- Replication studies in psychological science sometimes fail to reproduce prior findings. If these studies use methods that are unfaithful to the original study or ineffective in eliciting the phenomenon of interest, then a failure to replicate may be a failure of the protocol rather than a challenge to the original finding. Formal pre-data-collection peer review by experts may address shortcomings and increase replicability rates. We selected 10 replication studies from the Reproducibility Project: Psychology (RP:P; Open Science Collaboration, 2015) for which the original authors had expressed concerns about the replication designs before data collection; only one of these studies had yielded a statistically significant effect (p < .05). Commenters suggested that lack of adherence to expert review and low-powered tests were the reasons that most of these RP:P studies failed to replicate the original effects. We revised the replication protocols and received formal peer review prior to conducting new replication studies. We administered the RP:P and revised protocols in multiple laboratories (median number of laboratories per original study = 6.5, range = 3-9; median total sample = 1,279.5, range = 276-3,512) for high-powered tests of each original finding with both protocols. Overall, following the preregistered analysis plan, we found that the revised protocols produced effect sizes similar to those of the RP:P protocols (Delta r = .002 or .014, depending on analytic approach). The median effect size for the revised protocols (r = .05) was similar to that of the RP:P protocols (r = .04) and the original RP:P replications (r = .11), and smaller than that of the original studies (r = .37). Analysis of the cumulative evidence across the original studies and the corresponding three replication attempts provided very precise estimates of the 10 tested effects and indicated that their effect sizes (median r = .07, range = .00-.15) were 78% smaller, on average, than the original effect sizes (median r = .37, range = .19-.50).
|
|
| 3. |
- Martens, Marvin, et al.
(författare)
-
ELIXIR and Toxicology : a community in development
- 2021
-
Ingår i: F1000 Research. - : F1000 Research Ltd. - 2046-1402. ; 10, s. 1129-1129
-
Tidskriftsartikel (refereegranskat)abstract
- Toxicology has been an active research field for many decades, with academic, industrial and government involvement. Modern omics and computational approaches are changing the field, from merely disease-specific observational models into target-specific predictive models. Traditionally, toxicology has strong links with other fields such as biology, chemistry, pharmacology and medicine. With the rise of synthetic and new engineered materials, alongside ongoing prioritisation needs in chemical risk assessment for existing chemicals, early predictive evaluations are becoming of utmost importance to both scientific and regulatory purposes. ELIXIR is an intergovernmental organisation that brings together life science resources from across Europe. To coordinate the linkage of various life science efforts around modern predictive toxicology, the establishment of a new ELIXIR Community is seen as instrumental. In the past few years, joint efforts, building on incidental overlap, have been piloted in the context of ELIXIR. For example, the EU-ToxRisk, diXa, HeCaToS, transQST, and the nanotoxicology community have worked with the ELIXIR TeSS, Bioschemas, and Compute Platforms and activities. In 2018, a core group of interested parties wrote a proposal, outlining a sketch of what this new ELIXIR Toxicology Community would look like. A recent workshop (held September 30th to October 1st, 2020) extended this into an ELIXIR Toxicology roadmap and a shortlist of limited investment-high gain collaborations to give body to this new community. This Whitepaper outlines the results of these efforts and defines our vision of the ELIXIR Toxicology Community and how it complements other ELIXIR activities.
|
|
| 4. |
- Bonkhoff, Anna K, et al.
(författare)
-
The relevance of rich club regions for functional outcome post-stroke is enhanced in women.
- 2023
-
Ingår i: Human brain mapping. - : Wiley. - 1097-0193 .- 1065-9471. ; 44:4, s. 1579-1592
-
Tidskriftsartikel (refereegranskat)abstract
- This study aimed to investigate the influence of stroke lesions in predefined highly interconnected (rich-club) brain regions on functional outcome post-stroke, determine their spatial specificity and explore the effects of biological sex on their relevance. We analyzed MRI data recorded at index stroke and ~3-months modified Rankin Scale (mRS) data from patients with acute ischemic stroke enrolled in the multisite MRI-GENIE study. Spatially normalized structural stroke lesions were parcellated into 108 atlas-defined bilateral (sub)cortical brain regions. Unfavorable outcome (mRS>2) was modeled in a Bayesian logistic regression framework. Effects of individual brain regions were captured as two compound effects for (i) six bilateral rich club and (ii) all further non-rich club regions. In spatial specificity analyses, we randomized the split into "rich club" and "non-rich club" regions and compared the effect of the actual rich club regions to the distribution of effects from 1000 combinations of six random regions. In sex-specific analyses, we introduced an additional hierarchical level in our model structure to compare male and female-specific rich club effects. A total of 822 patients (age: 64.7[15.0], 39% women) were analyzed. Rich club regions had substantial relevance in explaining unfavorable functional outcome (mean of posterior distribution: 0.08, area under the curve: 0.8). In particular, the rich club-combination had a higher relevance than 98.4% of random constellations. Rich club regions were substantially more important in explaining long-term outcome in women than in men. All in all, lesions in rich club regions were associated with increased odds of unfavorable outcome. These effects were spatially specific and more pronounced in women.
|
|
| 5. |
- Duncanson, Laura, et al.
(författare)
-
Aboveground biomass density models for NASA's Global Ecosystem Dynamics Investigation (GEDI) lidar mission
- 2022
-
Ingår i: Remote Sensing of Environment. - : Elsevier BV. - 0034-4257 .- 1879-0704. ; 270
-
Tidskriftsartikel (refereegranskat)abstract
- NASA's Global Ecosystem Dynamics Investigation (GEDI) is collecting spaceborne full waveform lidar data with a primary science goal of producing accurate estimates of forest aboveground biomass density (AGBD). This paper presents the development of the models used to create GEDI's footprint-level (~25 m) AGBD (GEDI04_A) product, including a description of the datasets used and the procedure for final model selection. The data used to fit our models are from a compilation of globally distributed spatially and temporally coincident field and airborne lidar datasets, whereby we simulated GEDI-like waveforms from airborne lidar to build a calibration database. We used this database to expand the geographic extent of past waveform lidar studies, and divided the globe into four broad strata by Plant Functional Type (PFT) and six geographic regions. GEDI's waveform-to-biomass models take the form of parametric Ordinary Least Squares (OLS) models with simulated Relative Height (RH) metrics as predictor variables. From an exhaustive set of candidate models, we selected the best input predictor variables, and data transformations for each geographic stratum in the GEDI domain to produce a set of comprehensive predictive footprint-level models. We found that model selection frequently favored combinations of RH metrics at the 98th, 90th, 50th, and 10th height above ground-level percentiles (RH98, RH90, RH50, and RH10, respectively), but that inclusion of lower RH metrics (e.g. RH10) did not markedly improve model performance. Second, forced inclusion of RH98 in all models was important and did not degrade model performance, and the best performing models were parsimonious, typically having only 1-3 predictors. Third, stratification by geographic domain (PFT, geographic region) improved model performance in comparison to global models without stratification. Fourth, for the vast majority of strata, the best performing models were fit using square root transformation of field AGBD and/or height metrics. There was considerable variability in model performance across geographic strata, and areas with sparse training data and/or high AGBD values had the poorest performance. These models are used to produce global predictions of AGBD, but will be improved in the future as more and better training data become available.
|
|
| 6. |
- Enshaei, Amir, et al.
(författare)
-
A validated novel continuous prognostic index to deliver stratified medicine in pediatric acute lymphoblastic leukemia
- 2020
-
Ingår i: Blood. - : American Society of Hematology. - 0006-4971 .- 1528-0020. ; 135:17, s. 1438-1446
-
Tidskriftsartikel (refereegranskat)abstract
- Risk stratification is essential for the delivery of optimal treatment in childhood acute lymphoblastic leukemia. However, current risk stratification algorithms dichotomize variables and apply risk factors independently, which may incorrectly assume identical associations across biologically heterogeneous subsets and reduce statistical power. Accordingly, we developed and validated a prognostic index (PIUKALL) that integrates multiple risk factors and uses continuous data. We created discovery (n = 2405) and validation (n = 2313) cohorts using data from 4 recent trials (UKALL2003, COALL-03, DCOG-ALL10, and NOPHO-ALL2008). Using the discovery cohort, multivariate Cox regression modeling defined a minimal model including white cell count at diagnosis, pretreatment cytogenetics, and end-of-induction minimal residual disease. Using this model, we defined PIUKALL as a continuous variable that assigns personalized risk scores. PIUKALL correlated with risk of relapse and was validated in an independent cohort. Using PIUKALL to risk stratify patients improved the concordance index for all end points compared with traditional algorithms. We used PIUKALL to define 4 clinically relevant risk groups that had differential relapse rates at 5 years and were similar between the 2 cohorts (discovery: low, 3% [95% confidence interval (CI), 2%-4%]; standard, 8% [95% CI, 6%-10%]; intermediate, 17% [95% CI, 14%-21%]; and high, 48% [95% CI, 36%-60%; validation: low, 4% [95% CI, 3%-6%]; standard, 9% [95% CI, 6%-12%]; intermediate, 17% [95% CI, 14%-21%]; and high, 35% [95% CI, 24%-48%]). Analysis of the area under the curve confirmed the PIUKALL groups were significantly better at predicting outcome than algorithms employed in each trial. PIUKALL provides an accurate method for predicting outcome and more flexible method for defining risk groups in future studies.
|
|
| 7. |
- Martins Dos Santos, Vitor, et al.
(författare)
-
Systems Biology in ELIXIR: modelling in the spotlight
- 2022
-
Ingår i: F1000Research. - : F1000 Research Ltd. - 1759-796X .- 2046-1402. ; 11
-
Tidskriftsartikel (refereegranskat)abstract
- In this white paper, we describe the founding of a new ELIXIR Community - the Systems Biology Community - and its proposed future contributions to both ELIXIR and the broader community of systems biologists in Europe and worldwide. The Community believes that the infrastructure aspects of systems biology - databases, (modelling) tools and standards development, as well as training and access to cloud infrastructure - are not only appropriate components of the ELIXIR infrastructure, but will prove key components of ELIXIR's future support of advanced biological applications and personalised medicine. By way of a series of meetings, the Community identified seven key areas for its future activities, reflecting both future needs and previous and current activities within ELIXIR Platforms and Communities. These are: overcoming barriers to the wider uptake of systems biology; linking new and existing data to systems biology models; interoperability of systems biology resources; further development and embedding of systems medicine; provisioning of modelling as a service; building and coordinating capacity building and training resources; and supporting industrial embedding of systems biology. A set of objectives for the Community has been identified under four main headline areas: Standardisation and Interoperability, Technology, Capacity Building and Training, and Industrial Embedding. These are grouped into short-term (3-year), mid-term (6-year) and long-term (10-year) objectives.
|
|
