| 1. |
- Botvinik-Nezer, Rotem, et al.
(författare)
-
Variability in the analysis of a single neuroimaging dataset by many teams
- 2020
-
Ingår i: Nature. - : Springer Science and Business Media LLC. - 0028-0836 .- 1476-4687. ; 582, s. 84-88
-
Tidskriftsartikel (refereegranskat)abstract
- Data analysis workflows in many scientific domains have become increasingly complex and flexible. Here we assess the effect of this flexibility on the results of functional magnetic resonance imaging by asking 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses(1). The flexibility of analytical approaches is exemplified by the fact that no two teams chose identical workflows to analyse the data. This flexibility resulted in sizeable variation in the results of hypothesis tests, even for teams whose statistical maps were highly correlated at intermediate stages of the analysis pipeline. Variation in reported results was related to several aspects of analysis methodology. Notably, a meta-analytical approach that aggregated information across teams yielded a significant consensus in activated regions. Furthermore, prediction markets of researchers in the field revealed an overestimation of the likelihood of significant findings, even by researchers with direct knowledge of the dataset(2-5). Our findings show that analytical flexibility can have substantial effects on scientific conclusions, and identify factors that may be related to variability in the analysis of functional magnetic resonance imaging. The results emphasize the importance of validating and sharing complex analysis workflows, and demonstrate the need for performing and reporting multiple analyses of the same data. Potential approaches that could be used to mitigate issues related to analytical variability are discussed. The results obtained by seventy different teams analysing the same functional magnetic resonance imaging dataset show substantial variation, highlighting the influence of analytical choices and the importance of sharing workflows publicly and performing multiple analyses.
|
|
| 2. |
- Aczel, Balazs, et al.
(författare)
-
Consensus-based guidance for conducting and reporting multi-analyst studies
- 2021
-
Ingår i: eLIFE. - : eLife Sciences Publications. - 2050-084X. ; 10
-
Tidskriftsartikel (refereegranskat)abstract
- Any large dataset can be analyzed in a number of ways, and it is possible that the use of different analysis strategies will lead to different results and conclusions. One way to assess whether the results obtained depend on the analysis strategy chosen is to employ multiple analysts and leave each of them free to follow their own approach. Here, we present consensus-based guidance for conducting and reporting such multi-analyst studies, and we discuss how broader adoption of the multi-analyst approach has the potential to strengthen the robustness of results and conclusions obtained from analyses of datasets in basic and applied research.
|
|
| 3. |
- Anderson, Christopher J., et al.
(författare)
-
Response to Comment on "Estimating the reproducibility of psychological science"
- 2016
-
Ingår i: Science. - : American Association for the Advancement of Science (AAAS). - 0036-8075 .- 1095-9203. ; 351:6277
-
Tidskriftsartikel (övrigt vetenskapligt/konstnärligt)abstract
- Gilbert et al. conclude that evidence from the Open Science Collaboration's Reproducibility Project: Psychology indicates high reproducibility, given the study methodology. Their very optimistic assessment is limited by statistical misconceptions and by causal inferences from selectively interpreted, correlational data. Using the Reproducibility Project: Psychology data, both optimistic and pessimistic conclusions about reproducibility are possible, and neither are yet warranted.
|
|
| 4. |
|
|
| 5. |
- Bourget, M. H., et al.
(författare)
-
Microscopy-BIDS: An Extension to the Brain Imaging Data Structure for Microscopy Data
- 2022
-
Ingår i: Frontiers in Neuroscience. - : Frontiers Media SA. - 1662-4548 .- 1662-453X. ; 16
-
Tidskriftsartikel (refereegranskat)abstract
- The Brain Imaging Data Structure (BIDS) is a specification for organizing, sharing, and archiving neuroimaging data and metadata in a reusable way. First developed for magnetic resonance imaging (MRI) datasets, the community-led specification evolved rapidly to include other modalities such as magnetoencephalography, positron emission tomography, and quantitative MRI (qMRI). In this work, we present an extension to BIDS for microscopy imaging data, along with example datasets. Microscopy-BIDS supports common imaging methods, including 2D/3D, ex/in vivo, micro-CT, and optical and electron microscopy. Microscopy-BIDS also includes comprehensible metadata definitions for hardware, image acquisition, and sample properties. This extension will facilitate future harmonization efforts in the context of multi-modal, multi-scale imaging such as the characterization of tissue microstructure with qMRI.
|
|
| 6. |
- Buchanan, E. M., et al.
(författare)
-
The Psychological Science Accelerator's COVID-19 rapid-response dataset
- 2023
-
Ingår i: Scientific Data. - : Springer Science and Business Media LLC. - 2052-4463. ; 10:1
-
Tidskriftsartikel (refereegranskat)abstract
- In response to the COVID-19 pandemic, the Psychological Science Accelerator coordinated three large-scale psychological studies to examine the effects of loss-gain framing, cognitive reappraisals, and autonomy framing manipulations on behavioral intentions and affective measures. The data collected (April to October 2020) included specific measures for each experimental study, a general questionnaire examining health prevention behaviors and COVID-19 experience, geographical and cultural context characterization, and demographic information for each participant. Each participant started the study with the same general questions and then was randomized to complete either one longer experiment or two shorter experiments. Data were provided by 73,223 participants with varying completion rates. Participants completed the survey from 111 geopolitical regions in 44 unique languages/dialects. The anonymized dataset described here is provided in both raw and processed formats to facilitate re-use and further analyses. The dataset offers secondary analytic opportunities to explore coping, framing, and self-determination across a diverse, global sample obtained at the onset of the COVID-19 pandemic, which can be merged with other time-sampled or geographic data.
|
|
| 7. |
- Czeszumski, Artur, et al.
(författare)
-
#EEGManyLabs: Investigating the Replicability of Influential EEG Experiments
- 2024
-
Annan publikation (övrigt vetenskapligt/konstnärligt)abstract
- There is growing awareness across the neuroscience community that the replicability of findings on the relationship between brain activity and cognitive phenomena can be improved by conducting studies with high statistical power that adhere to well-defined and standardized analysis pipelines. Inspired by efforts from the psychological sciences, and with the desire to examine some of the foundational findings using electroencephalography (EEG), we have launched #EEGManyLabs, a large-scale international collaborative replication effort. Since its discovery in the early 20th century, EEG has had a profound influence on our understanding of human cognition, but there is limited evidence on the replicability of some of the most highly cited discoveries. After a systematic search and selection process, we have identified 27 of the most influential and continually cited studies in the field. We plan to directly test the replicability of key findings from 20 of these studies in teams of at least three independent laboratories. The design and protocol of each replication effort will be submitted as a Registered Report and peer-reviewed prior to data collection. Prediction markets, open to all EEG researchers, will be used as a forecasting tool to examine which findings the community expects to replicate. This project will update our confidence in some of the most influential EEG findings and generate a large open access database that can be used to inform future research practices. Finally, through this international effort, we hope to create a cultural shift towards inclusive, high-powered multi-laboratory collaborations.
|
|
| 8. |
- Darai-Ramqvist, Eva, et al.
(författare)
-
Microenvironment-dependent phenotypic changes in a SCID mouse model for malignant mesothelioma
- 2013
-
Ingår i: Frontiers in Oncology. - : Frontiers Media SA. - 2234-943X. ; 3
-
Tidskriftsartikel (refereegranskat)abstract
- Background and Aims: Malignant mesothelioma is an aggressive, therapy-resistant tumor. Mesothelioma cells may assume an epithelioid or a sarcomatoid phenotype, and presence of sarcomatoid cells predicts poor prognosis. In this study, we investigated differentiation of mesothelioma cells in a xenograft model, where mesothelioma cells of both phenotypes were induced to form tumors in severe combined immunodeficiency mice.Methods: Xenografts were established and thoroughly characterized using a comprehensive immunohistochemical panel, array comparative genomic hybridization (aCGH) of chromosome 3, fluorescent in situ hybridization, and electron microscopy.Results: Epithelioid and sarcomatoid cells gave rise to xenografts of similar epithelioid morphology. While sarcomatoid-derived xenografts had higher growth rates, the morphology and expression of differentiation-related markers was similar between xenografts derived from both phenotypes. aCGH showed a convergent genotype for both xenografts, resembling the original aggressive sarcomatoid cell sub-line.Conclusion: Human mesothelioma xenografts from sarcomatoid and epithelioid phenotypes converged to a similar differentiation state, and genetic analyses suggested that clonal selection in the mouse microenvironment was a major contributing factor. This thoroughly characterized animal model can be used for further studies of molecular events underlying tumor cell differentiation.
|
|
| 9. |
|
|
| 10. |
- Dreber, Anna, et al.
(författare)
-
Olika slutsatser från samma data : Den analytiska flexibiliteten i medicinsk forskning är ofta stor
- 2023
-
Ingår i: Läkartidningen. - : Läkartidningen Förlag AB. - 1652-7518 .- 0023-7205. ; 120
-
Tidskriftsartikel (refereegranskat)abstract
- Analysis of research data entails many choices. As a result, a space of different analytical strategies is open to researchers. Different justifiable analyses may not give similar results. The method of multiple analysts is a way to study the analytical flexibility and behaviour of researchers under naturalistic conditions, as part of the field known as metascience. Analytical flexibility and risks of bias can be counteracted by open data sharing, pre-registration of analysis plans, and registration of clinical trials in trial registers. These measures are particularly important for retrospective studies where analytical flexibility can be greatest, although pre-registration is less useful in this context. Synthetic datasets can be an alternative to pre-registration when used to decide what analyses should be conducted on real datasets by independent parties. All these strategies help build trustworthiness in scientific reports, and improve the reliability of research findings.
|
|
