| 1. |
- Botvinik-Nezer, Rotem, et al.
(author)
-
Variability in the analysis of a single neuroimaging dataset by many teams
- 2020
-
In: Nature. - : Springer Science and Business Media LLC. - 0028-0836 .- 1476-4687. ; 582, s. 84-88
-
Journal article (peer-reviewed)abstract
- Data analysis workflows in many scientific domains have become increasingly complex and flexible. Here we assess the effect of this flexibility on the results of functional magnetic resonance imaging by asking 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses(1). The flexibility of analytical approaches is exemplified by the fact that no two teams chose identical workflows to analyse the data. This flexibility resulted in sizeable variation in the results of hypothesis tests, even for teams whose statistical maps were highly correlated at intermediate stages of the analysis pipeline. Variation in reported results was related to several aspects of analysis methodology. Notably, a meta-analytical approach that aggregated information across teams yielded a significant consensus in activated regions. Furthermore, prediction markets of researchers in the field revealed an overestimation of the likelihood of significant findings, even by researchers with direct knowledge of the dataset(2-5). Our findings show that analytical flexibility can have substantial effects on scientific conclusions, and identify factors that may be related to variability in the analysis of functional magnetic resonance imaging. The results emphasize the importance of validating and sharing complex analysis workflows, and demonstrate the need for performing and reporting multiple analyses of the same data. Potential approaches that could be used to mitigate issues related to analytical variability are discussed. The results obtained by seventy different teams analysing the same functional magnetic resonance imaging dataset show substantial variation, highlighting the influence of analytical choices and the importance of sharing workflows publicly and performing multiple analyses.
|
|
| 2. |
- Hulme, Oliver, et al.
(author)
-
Reply to "The Limitations of Growth-Optimal Approaches to Decision Making Under Uncertainty"
- 2023
-
In: Econ Journal Watch. - : Institute of Spontaneous Order Economics. - 1933-527X. ; 20:2, s. 335-348
-
Journal article (other academic/artistic)abstract
- In an article appearing concurrently with the present one, Matthew Ford and John Kay put forward their understanding of a decision theory which emerges in ergodicity economics. Their understanding leads them to believe that ergodicity economics evades the core problem of decisions under uncertainty and operates solely in a regime where there is no measurable uncertainty. If this were the case, then the authors' critical stance would be justified and, as the authors point out, the decision theory would yield only trivial results, identical to a flavor of expected-utility theory. Here we clarify that the critique is based on a theoretical misunderstanding, and that uncertainty-quantified in any reasonable way-is large in the regime where the model operates. Our resolution explains the success of recent laboratory experiments, where ergodicity economics makes predictions different from expected-utility theory, contrary to the claim of equivalence by Ford and Kay. Also, a state of the world is identified where ergodicity economics outperforms expected-utility theory empirically.
|
|
| 3. |
- Schewe, Jacob, et al.
(author)
-
State-of-the-art global models underestimate impacts from climate extremes
- 2019
-
In: Nature Communications. - : Springer Science and Business Media LLC. - 2041-1723. ; 10
-
Journal article (peer-reviewed)abstract
- Global impact models represent process-level understanding of how natural and human systems may be affected by climate change. Their projections are used in integrated assessments of climate change. Here we test, for the first time, systematically across many important systems, how well such impact models capture the impacts of extreme climate conditions. Using the 2003 European heat wave and drought as a historical analogue for comparable events in the future, we find that a majority of models underestimate the extremeness of impacts in important sectors such as agriculture, terrestrial ecosystems, and heat-related human mortality, while impacts on water resources and hydropower are overestimated in some river basins; and the spread across models is often large. This has important implications for economic assessments of climate change impacts that rely on these models. It also means that societal risks from future extreme events may be greater than previously thought.
|
|
| 4. |
- Wartenburger, Richard, et al.
(author)
-
Evapotranspiration simulations in ISIMIP2a-Evaluation of spatio-temporal characteristics with a comprehensive ensemble of independent datasets
- 2018
-
In: Environmental Research Letters. - : IOP Publishing. - 1748-9326. ; 13:7
-
Journal article (peer-reviewed)abstract
- Actual land evapotranspiration (ET) is a key component of the global hydrological cycle and an essential variable determining the evolution of hydrological extreme events under different climate change scenarios. However, recently available ET products show persistent uncertainties that are impeding a precise attribution of human-induced climate change. Here, we aim at comparing a range of independent global monthly land ET estimates with historical model simulations from the global water, agriculture, and biomes sectors participating in the second phase of the Inter-Sectoral Impact Model Intercomparison Project (ISIMIP2a). Among the independent estimates, we use the EartH2Observe Tier-1 dataset (E2O), two commonly used reanalyses, a pre-compiled ensemble product (LandFlux-EVAL), and an updated collection of recently published datasets that algorithmically derive ET from observations or observations-based estimates (diagnostic datasets). A cluster analysis is applied in order to identify spatio-temporal differences among all datasets and to thus identify factors that dominate overall uncertainties. The clustering is controlled by several factors including the model choice, the meteorological forcing used to drive the assessed models, the data category (models participating in the different sectors of ISIMIP2a, E2O models, diagnostic estimates, reanalysis-based estimates or composite products), the ET scheme, and the number of soil layers in the models. By using these factors to explain spatial and spatio-temporal variabilities in ET, we find that the model choice mostly dominates (24%-40% of variance explained), except for spatio-temporal patterns of total ET, where the forcing explains the largest fraction of the variance (29%). The most dominant clusters of datasets are further compared with individual diagnostic and reanalysis-based estimates to assess their representation of selected heat waves and droughts in the Great Plains, Central Europe and western Russia. Although most of the ET estimates capture these extreme events, the generally large spread among the entire ensemble indicates substantial uncertainties.
|
|
