| 1. |
- Schweinsberg, Martin, et al.
(författare)
-
Same data, different conclusions : Radical dispersion in empirical results when independent analysts operationalize and test the same hypothesis
- 2021
-
Ingår i: Organizational Behavior and Human Decision Processes. - : Elsevier BV. - 0749-5978 .- 1095-9920. ; 165, s. 228-249
-
Tidskriftsartikel (refereegranskat)abstract
- In this crowdsourced initiative, independent analysts used the same dataset to test two hypotheses regarding the effects of scientists' gender and professional status on verbosity during group meetings. Not only the analytic approach but also the operationalizations of key variables were left unconstrained and up to individual analysts. For instance, analysts could choose to operationalize status as job title, institutional ranking, citation counts, or some combination. To maximize transparency regarding the process by which analytic choices are made, the analysts used a platform we developed called DataExplained to justify both preferred and rejected analytic paths in real time. Analyses lacking sufficient detail, reproducible code, or with statistical errors were excluded, resulting in 29 analyses in the final sample. Researchers reported radically different analyses and dispersed empirical outcomes, in a number of cases obtaining significant effects in opposite directions for the same research question. A Boba multiverse analysis demonstrates that decisions about how to operationalize variables explain variability in outcomes above and beyond statistical choices (e.g., covariates). Subjective researcher decisions play a critical role in driving the reported empirical results, underscoring the need for open data, systematic robustness checks, and transparency regarding both analytic paths taken and not taken. Implications for orga-nizations and leaders, whose decision making relies in part on scientific findings, consulting reports, and internal analyses by data scientists, are discussed.
|
|
| 2. |
- Graham, Jesse R., et al.
(författare)
-
The pipeline project: Pre-publication independent replications of a single laboratory's research pipeline
- 2016
-
Ingår i: Journal of Experimental Social Psychology. - : Elsevier. - 1096-0465 .- 0022-1031. ; 66, s. 55-67
-
Tidskriftsartikel (refereegranskat)abstract
- This crowdsourced project introduces a collaborative approach to improving the reproducibility of scientific research, in which findings are replicated in qualified independent laboratories before (rather than after) they are published. Our goal is to establish a non-adversarial replication process with highly informative final results. To illustrate the Pre-Publication Independent Replication (PPIR) approach, 25 research groups conducted replications of all ten moral judgment effects which the last author and his collaborators had “in the pipeline” as of August 2014. Six findings replicated according to all replication criteria, one finding replicated but with a significantly smaller effect size than the original, one finding replicated consistently in the original culture but not outside of it, and two findings failed to find support. In total, 40% of the original findings failed at least one major replication criterion. Potential ways to implement and incentivize pre-publication independent replication on a large scale are discussed.
|
|
| 3. |
- Washburn, Anthony N., et al.
(författare)
-
Data from a pre-publication independent replication initiative examining ten moral judgement effects
- 2016
-
Ingår i: Scientific Data. - : Nature Research (part of Springer Nature): Fully open access journals / Nature Publishing Group. - 2052-4463. ; 3
-
Tidskriftsartikel (refereegranskat)abstract
- We present the data from a crowdsourced project seeking to replicate findings in independent laboratories before (rather than after) they are published. In this Pre-Publication Independent Replication (PPIR) initiative, 25 research groups attempted to replicate 10 moral judgment effects from a single laboratory's research pipeline of unpublished findings. The 10 effects were investigated using online/lab surveys containing psychological manipulations (vignettes) followed by questionnaires.
|
|
| 4. |
- Silberzahn, Raphael, et al.
(författare)
-
Many analysts, one dataset : Making transparent how variations in analytical choices affect results
- 2018
-
Ingår i: Advances in Methods and Practices in Psychological Science. - : Sage Publications. - 2515-2459 .- 2515-2467. ; 1:3, s. 337-356
-
Tidskriftsartikel (refereegranskat)abstract
- Twenty-nine teams involving 61 analysts used the same dataset to address the same research question: whether soccer referees are more likely to give red cards to dark skin toned players than light skin toned players. Analytic approaches varied widely across teams, and estimated effect sizes ranged from 0.89 to 2.93 in odds ratio units, with a median of 1.31. Twenty teams (69%) found a statistically significant positive effect and nine teams (31%) observed a non-significant relationship. Overall 29 differentanalyses used 21 unique combinations of covariates. We found that neither analysts' prior beliefs about the effect, nor their level of expertise, nor peer-reviewed quality of analysis readily explained variation in analysis outcomes. This suggests that significant variation in the results of analyses of complex data may be difficult to avoid, even by experts with honest intentions. Crowdsourcing data analysis, a strategy by which numerous research teams are recruited to simultaneously investigate the same research question, makes transparent how defensible, yet subjective analytic choices influence research results.
|
|
| 5. |
- Warrier, Indira, et al.
(författare)
-
Factor IX inhibitors and anaphylaxis in hemophilia B
- 1997
-
Ingår i: Journal of Pediatric Hematology/Oncology. - : Ovid Technologies (Wolters Kluwer Health). - 1077-4114 .- 0192-8562. ; 19:1, s. 23-27
-
Tidskriftsartikel (refereegranskat)abstract
- Purpose: We present clinical and laboratory data on 18 children from 12 hemophilia treatment centers in the United States, Canada, and Europe with the purpose of disseminating information regarding a recently recognized, potentially life-threatening complication of treatment in very young children with hemophilia B. Patients and Methods: Twelve hemophilia centers from the United States, Canada, and Europe provided clinical information and laboratory data concerning 18 children who had severe allergic reactions to infused factor (F) IX in close association with the development of an inhibitor to FIX. Laboratory testing for establishment of the diagnosis of hemophilia B and inhibitor to FIX was done locally at the centers treating these patients. FIX gene analysis was performed at one of six molecular genetics institutes. Results: All 18 children had severe hemophilia B, and in each an inhibitor antibody to FIX developed. The median age at the time of anaphylaxis (or anaphylactoid reaction) was 16 months, and the median number of exposure days to FIX was 11. The FIX inhibitor was detected almost simultaneously with the first occurrence of anaphylaxis in 12 of 18 patients. Maximum inhibitor liters were 4.5-600 Bethesda units (BU), with a median titer of 48 BU. FIX gene analysis, performed in 17 of 18 patients, demonstrated complete deletion of the FIX gene in 10 and major derangements in seven. Immune tolerance induction (ITI) regimens have been attempted in 12 patients, with generally poor responses. Two of the 12 experienced nephrotic syndrome while on ITI. Recombinant FVIIa has been successfully used to treat bleeding episodes in 11 of these children. Conclusion: Physicians treating young children with hemophilia B should be aware of the potentially life- threatening complication of anaphylaxis. Children with complete gene deletions or major derangements of the FIX gene appear to be at greater risk. Those identified by genotype as being at greater risk may need to receive their first 10-20 treatments in a medical facility equipped for handling such emergencies. Recombinant FVIIa, although not licensed for use in the United States, appears to be the most suitable treatment option for bleeding episodes in such patients.
|
|
