| 1. |
- Botvinik-Nezer, Rotem, et al.
(författare)
-
Variability in the analysis of a single neuroimaging dataset by many teams
- 2020
-
Ingår i: Nature. - : Springer Science and Business Media LLC. - 0028-0836 .- 1476-4687. ; 582, s. 84-88
-
Tidskriftsartikel (refereegranskat)abstract
- Data analysis workflows in many scientific domains have become increasingly complex and flexible. Here we assess the effect of this flexibility on the results of functional magnetic resonance imaging by asking 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses(1). The flexibility of analytical approaches is exemplified by the fact that no two teams chose identical workflows to analyse the data. This flexibility resulted in sizeable variation in the results of hypothesis tests, even for teams whose statistical maps were highly correlated at intermediate stages of the analysis pipeline. Variation in reported results was related to several aspects of analysis methodology. Notably, a meta-analytical approach that aggregated information across teams yielded a significant consensus in activated regions. Furthermore, prediction markets of researchers in the field revealed an overestimation of the likelihood of significant findings, even by researchers with direct knowledge of the dataset(2-5). Our findings show that analytical flexibility can have substantial effects on scientific conclusions, and identify factors that may be related to variability in the analysis of functional magnetic resonance imaging. The results emphasize the importance of validating and sharing complex analysis workflows, and demonstrate the need for performing and reporting multiple analyses of the same data. Potential approaches that could be used to mitigate issues related to analytical variability are discussed. The results obtained by seventy different teams analysing the same functional magnetic resonance imaging dataset show substantial variation, highlighting the influence of analytical choices and the importance of sharing workflows publicly and performing multiple analyses.
|
|
| 2. |
- Aczel, Balazs, et al.
(författare)
-
Consensus-based guidance for conducting and reporting multi-analyst studies
- 2021
-
Ingår i: eLIFE. - : eLife Sciences Publications. - 2050-084X. ; 10
-
Tidskriftsartikel (refereegranskat)abstract
- Any large dataset can be analyzed in a number of ways, and it is possible that the use of different analysis strategies will lead to different results and conclusions. One way to assess whether the results obtained depend on the analysis strategy chosen is to employ multiple analysts and leave each of them free to follow their own approach. Here, we present consensus-based guidance for conducting and reporting such multi-analyst studies, and we discuss how broader adoption of the multi-analyst approach has the potential to strengthen the robustness of results and conclusions obtained from analyses of datasets in basic and applied research.
|
|
| 3. |
|
|
| 4. |
- Ben-Avraham, Dan, et al.
(författare)
-
The complex genetics of gait speed : Genome-wide meta-analysis approach
- 2017
-
Ingår i: Aging. - : Impact Journals, LLC. - 1945-4589. ; 9:1, s. 209-246
-
Tidskriftsartikel (refereegranskat)abstract
- Emerging evidence suggests that the basis for variation in late-life mobility is attributable, in part, to genetic factors, which may become increasingly important with age. Our objective was to systematically assess the contribution of genetic variation to gait speed in older individuals. We conducted a meta-analysis of gait speed GWASs in 31,478 older adults from 17 cohorts of the CHARGE consortium, and validated our results in 2,588 older adults from 4 independent studies. We followed our initial discoveries with network and eQTL analysis of candidate signals in tissues. The meta-analysis resulted in a list of 536 suggestive genome wide significant SNPs in or near 69 genes. Further interrogation with Pathway Analysis placed gait speed as a polygenic complex trait in five major networks. Subsequent eQTL analysis revealed several SNPs significantly associated with the expression of PRSS16, WDSUB1 and PTPRT, which in addition to the meta-analysis and pathway suggested that genetic effects on gait speed may occur through synaptic function and neuronal development pathways. No genome-wide significant signals for gait speed were identified from this moderately large sample of older adults, suggesting that more refined physical function phenotypes will be needed to identify the genetic basis of gait speed in aging.
|
|
| 5. |
|
|
| 6. |
|
|
| 7. |
- Hoffman, Joel, et al.
(författare)
-
Mapping the availability of translated versions of posttraumatic stress disorder screening questionnaires for adults : A scoping review
- 2022
-
Ingår i: European Journal of Psychotraumatology. - : Informa UK Limited. - 2000-8198 .- 2000-8066. ; 13:2
-
Forskningsöversikt (refereegranskat)abstract
- Background: The most used questionnaires for PTSD screening in adults were developed in English. Although many of these questionnaires were translated into other languages, the procedures used to translate them and to evaluate their reliability and validity have not been consistently documented. This comprehensive scoping review aimed to compile the currently available translated and evaluated questionnaires used for PTSD screening, and highlight important gaps in the literature.Objective: This review aimed to map the availability of translated and evaluated screening questionnaires for posttraumatic stress disorder (PTSD) for adults.Methods: All peer-reviewed studies in which a PTSD screening questionnaire for adults was translated, and which reported at least one result of a qualitative and /or quantitative evaluation procedure were included. The literature was searched using Embase, MEDLINE, and APA PsycInfo, citation searches and contributions from study team members. There were no restrictions regarding the target languages of the translations. Data on the translation procedure, the qualitative evaluation, the quantitative evaluation (dimensionality of the questionnaire, reliability, and performance), and open access were extracted.Results: A total of 866 studies were screened, of which 126 were included. Collectively, 128 translations of 12 different questionnaires were found. Out of these, 105 (83.3%) studies used a forward and backward translation procedure, 120 (95.2%) assessed the reliability of the translated questionnaire, 60 (47.6%) the dimensionality, 49 (38.9%) the performance, and 42 (33.3%) used qualitative evaluation procedures. Thirty-four questionnaires (27.0%) were either freely available or accessible on request.Conclusions: The analyses conducted and the description of the methods and results varied substantially, making a quality assessment impractical. Translations into languages spoken in middle- or low-income countries were underrepresented. In addition, only a small proportion of all translated questionnaires were available. Given the need for freely accessible translations, an online repository was developed.
|
|
| 8. |
- Uhlmann, Eric, L., et al.
(författare)
-
Subjective Evidence Evaluation Survey For Multi-Analyst Studies
- 2024
-
Annan publikation (övrigt vetenskapligt/konstnärligt)abstract
- Multi-analyst studies explore how well an empirical claim withstands plausible alternative analyses of the same data set by multiple, independent analysis teams. Conclusions from these studies typically rely on a single outcome metric (e.g., effect size) provided by each analysis team. Although informative about the range of plausible effects in a data set, a single effect size from each team does not provide a complete, nuanced understanding of how analysis choices are related to the outcome. We used the Delphi consensus technique with input from 37 experts to develop an 18-item Subjective Evidence Evaluation Survey (SEES) to evaluate how each analysis team views the methodological appropriateness of the research design and the strength of evidence for the hypothesis. We illustrate the usefulness of the SEES in providing richer evidence assessment with pilot data from a previous multi-analyst study.
|
|
