| 1. |
- Schweinsberg, Martin, et al.
(author)
-
Same data, different conclusions : Radical dispersion in empirical results when independent analysts operationalize and test the same hypothesis
- 2021
-
In: Organizational Behavior and Human Decision Processes. - : Elsevier BV. - 0749-5978 .- 1095-9920. ; 165, s. 228-249
-
Journal article (peer-reviewed)abstract
- In this crowdsourced initiative, independent analysts used the same dataset to test two hypotheses regarding the effects of scientists' gender and professional status on verbosity during group meetings. Not only the analytic approach but also the operationalizations of key variables were left unconstrained and up to individual analysts. For instance, analysts could choose to operationalize status as job title, institutional ranking, citation counts, or some combination. To maximize transparency regarding the process by which analytic choices are made, the analysts used a platform we developed called DataExplained to justify both preferred and rejected analytic paths in real time. Analyses lacking sufficient detail, reproducible code, or with statistical errors were excluded, resulting in 29 analyses in the final sample. Researchers reported radically different analyses and dispersed empirical outcomes, in a number of cases obtaining significant effects in opposite directions for the same research question. A Boba multiverse analysis demonstrates that decisions about how to operationalize variables explain variability in outcomes above and beyond statistical choices (e.g., covariates). Subjective researcher decisions play a critical role in driving the reported empirical results, underscoring the need for open data, systematic robustness checks, and transparency regarding both analytic paths taken and not taken. Implications for orga-nizations and leaders, whose decision making relies in part on scientific findings, consulting reports, and internal analyses by data scientists, are discussed.
|
|
| 2. |
- Abolfathi, Bela, et al.
(author)
-
The LSST DESC DC2 Simulated Sky Survey
- 2021
-
In: Astrophysical Journal Supplement Series. - : American Astronomical Society. - 0067-0049 .- 1538-4365. ; 253:31
-
Journal article (peer-reviewed)abstract
- We describe the simulated sky survey underlying the second data challenge (DC2) carried out in preparation for analysis of the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) by the LSST Dark Energy Science Collaboration (LSST DESC). Significant connections across multiple science domains will be a hallmark of LSST; the DC2 program represents a unique modeling effort that stresses this interconnectivity in a way that has not been attempted before. This effort encompasses a full end-to-end approach: starting from a large N-body simulation, through setting up LSST-like observations including realistic cadences, through image simulations, and finally processing with Rubin's LSST Science Pipelines. This last step ensures that we generate data products resembling those to be delivered by the Rubin Observatory as closely as is currently possible. The simulated DC2 sky survey covers six optical bands in a wide-fast-deep area of approximately 300 deg2, as well as a deep drilling field of approximately 1 deg2. We simulate 5 yr of the planned 10 yr survey. The DC2 sky survey has multiple purposes. First, the LSST DESC working groups can use the data set to develop a range of DESC analysis pipelines to prepare for the advent of actual data. Second, it serves as a realistic test bed for the image processing software under development for LSST by the Rubin Observatory. In particular, simulated data provide a controlled way to investigate certain image-level systematic effects. Finally, the DC2 sky survey enables the exploration of new scientific ideas in both static and time domain cosmology.
|
|
| 3. |
|
|
| 4. |
|
|
| 5. |
- Hammond, A., et al.
(author)
-
The reliability and validity of the English version of the Evaluation of Daily Activity Questionnaire for people with rheumatoid arthritis
- 2015
-
In: Rheumatology. - : Oxford University Press (OUP). - 1462-0324 .- 1462-0332. ; 54:9, s. 1605-1615
-
Journal article (peer-reviewed)abstract
- Objectives. The Evaluation of Daily Activity Questionnaire (EDAQ) includes 138 items in 14 domains identified as important by people with RA. The aim of this study was to test the validity and reliability of the English EDAQ. Methods. A total of 502 participants completed two questionnaires 3 weeks apart. The first consisted of the EDAQ, HAQ, RA Quality of Life (RAQoL) and the Medical Outcomes Scale (MOS) 36-item Short-Form Health Survey (SF-36v2), and the second consisted of the EDAQ only. The 14 EDAQ domains were tested for: unidimensionality-using confirmatory factor analysis; fit, response dependency, invariance across groups (differential item functioning)-using Rasch analysis; internal consistency [Person Separation Index (PSI)]; concurrent validity-by correlations with the HAQ, SF-36v2 and RAQoL; and test-retest reliability (Spearman's correlations). Results. Confirmatory factor analysis of the 14 EDAQ domains indicated unidimensionality, after adjustment for local dependency in each domain. All domains achieved a root mean square error of approximation <0.10 and satisfied Rasch model expectations for local dependency. DIF by age, gender and employment status was largely absent. The PSI was consistent with individual use (PSI = 0.94 for all 14 domains). For all domains, except Caring, concurrent validity was good: HAQ (r(s) = 0.72-0.91), RAQoL (r(s) = 0.67-0.82) and SF36v2 Physical Function scale (r(s) = -0.60 to -0.84) and test-retest reliability was good (r(s) = 0.70-0.89). Conclusion. Analysis supported a 14-domain, two-component structure (Self care and Mobility) of the EDAQ, where each domain, and both components, satisfied Rasch model requirements, and have robust reliability and validity.
|
|
