1. |
- Tierney, W., et al.
(author)
-
A creative destruction approach to replication : Implicit work and sex morality across cultures
- 2021
-
In: Journal of Experimental Social Psychology. - : Elsevier BV. - 0022-1031 .- 1096-0465. ; 93
-
Journal article (peer-reviewed)abstract
- How can we maximize what is learned from a replication study? In the creative destruction approach to replication, the original hypothesis is compared not only to the null hypothesis, but also to predictions derived from multiple alternative theoretical accounts of the phenomenon. To this end, new populations and measures are included in the design in addition to the original ones, to help determine which theory best accounts for the results across multiple key outcomes and contexts. The present pre-registered empirical project compared the Implicit Puritanism account of intuitive work and sex morality to theories positing regional, religious, and social class differences; explicit rather than implicit cultural differences in values; self-expression vs. survival values as a key cultural fault line; the general moralization of work; and false positive effects. Contradicting Implicit Puritanism's core theoretical claim of a distinct American work morality, a number of targeted findings replicated across multiple comparison cultures, whereas several failed to replicate in all samples and were identified as likely false positives. No support emerged for theories predicting regional variability and specific individual-differences moderators (religious affiliation, religiosity, and education level). Overall, the results provide evidence that work is intuitively moralized across cultures.
|
|
2. |
- Delios, A., et al.
(author)
-
Examining the generalizability of research findings from archival data
- 2022
-
In: Proceedings of the National Academy of Sciences of the United States of America. - : Proceedings of the National Academy of Sciences. - 0027-8424 .- 1091-6490. ; 119:30
-
Journal article (peer-reviewed)abstract
- This initiative examined systematically the extent to which a large set of archival research findings generalizes across contexts. We repeated the key analyses for 29 original strategic management effects in the same context (direct reproduction) as well as in 52 novel time periods and geographies; 45% of the reproductions returned results matching the original reports together with 55% of tests in different spans of years and 40% of tests in novel geographies. Some original findings were associated with multiple new tests. Reproducibility was the best predictor of generalizability-for the findings that proved directly reproducible, 84% emerged in other available time periods and 57% emerged in other geographies. Overall, only limited empirical evidence emerged for context sensitivity. In a forecasting survey, independent scientists were able to anticipate which effects would find support in tests in new samples.
|
|
3. |
- Buchanan, E. M., et al.
(author)
-
The Psychological Science Accelerator's COVID-19 rapid-response dataset
- 2023
-
In: Scientific Data. - : Springer Science and Business Media LLC. - 2052-4463. ; 10:1
-
Journal article (peer-reviewed)abstract
- In response to the COVID-19 pandemic, the Psychological Science Accelerator coordinated three large-scale psychological studies to examine the effects of loss-gain framing, cognitive reappraisals, and autonomy framing manipulations on behavioral intentions and affective measures. The data collected (April to October 2020) included specific measures for each experimental study, a general questionnaire examining health prevention behaviors and COVID-19 experience, geographical and cultural context characterization, and demographic information for each participant. Each participant started the study with the same general questions and then was randomized to complete either one longer experiment or two shorter experiments. Data were provided by 73,223 participants with varying completion rates. Participants completed the survey from 111 geopolitical regions in 44 unique languages/dialects. The anonymized dataset described here is provided in both raw and processed formats to facilitate re-use and further analyses. The dataset offers secondary analytic opportunities to explore coping, framing, and self-determination across a diverse, global sample obtained at the onset of the COVID-19 pandemic, which can be merged with other time-sampled or geographic data.
|
|
4. |
|
|
5. |
|
|
6. |
|
|
7. |
|
|
8. |
|
|
9. |
- Ebersole, Charles R., et al.
(author)
-
Many Labs 5: Testing Pre-Data-Collection Peer Review as an Intervention to Increase Replicability
- 2020
-
In: Advances in Methods and Practices in Psychological Science. - : Sage. - 2515-2467 .- 2515-2459. ; 3:3, s. 309-331
-
Journal article (peer-reviewed)abstract
- Replication studies in psychological science sometimes fail to reproduce prior findings. If these studies use methods that are unfaithful to the original study or ineffective in eliciting the phenomenon of interest, then a failure to replicate may be a failure of the protocol rather than a challenge to the original finding. Formal pre-data-collection peer review by experts may address shortcomings and increase replicability rates. We selected 10 replication studies from the Reproducibility Project: Psychology (RP:P; Open Science Collaboration, 2015) for which the original authors had expressed concerns about the replication designs before data collection; only one of these studies had yielded a statistically significant effect (p < .05). Commenters suggested that lack of adherence to expert review and low-powered tests were the reasons that most of these RP:P studies failed to replicate the original effects. We revised the replication protocols and received formal peer review prior to conducting new replication studies. We administered the RP:P and revised protocols in multiple laboratories (median number of laboratories per original study = 6.5, range = 3-9; median total sample = 1,279.5, range = 276-3,512) for high-powered tests of each original finding with both protocols. Overall, following the preregistered analysis plan, we found that the revised protocols produced effect sizes similar to those of the RP:P protocols (Delta r = .002 or .014, depending on analytic approach). The median effect size for the revised protocols (r = .05) was similar to that of the RP:P protocols (r = .04) and the original RP:P replications (r = .11), and smaller than that of the original studies (r = .37). Analysis of the cumulative evidence across the original studies and the corresponding three replication attempts provided very precise estimates of the 10 tested effects and indicated that their effect sizes (median r = .07, range = .00-.15) were 78% smaller, on average, than the original effect sizes (median r = .37, range = .19-.50).
|
|
10. |
|
|
11. |
- Grossmann, Igor, et al.
(author)
-
Insights into the accuracy of social scientists' forecasts of societal change
- 2023
-
In: Nature Human Behaviour. - : Springer Nature. - 2397-3374. ; 7, s. 484-501
-
Journal article (peer-reviewed)abstract
- How well can social scientists predict societal change, and what processes underlie their predictions? To answer these questions, we ran two forecasting tournaments testing the accuracy of predictions of societal change in domains commonly studied in the social sciences: ideological preferences, political polarization, life satisfaction, sentiment on social media, and gender-career and racial bias. After we provided them with historical trend data on the relevant domain, social scientists submitted pre-registered monthly forecasts for a year (Tournament 1; N = 86 teams and 359 forecasts), with an opportunity to update forecasts on the basis of new data six months later (Tournament 2; N = 120 teams and 546 forecasts). Benchmarking forecasting accuracy revealed that social scientists' forecasts were on average no more accurate than those of simple statistical models (historical means, random walks or linear regressions) or the aggregate forecasts of a sample from the general public (N = 802). However, scientists were more accurate if they had scientific expertise in a prediction domain, were interdisciplinary, used simpler models and based predictions on prior data. How accurate are social scientists in predicting societal change, and what processes underlie their predictions? Grossmann et al. report the findings of two forecasting tournaments. Social scientists' forecasts were on average no more accurate than those of simple statistical models.
|
|
12. |
- Ombrello, MJ, et al.
(author)
-
Genetic architecture distinguishes systemic juvenile idiopathic arthritis from other forms of juvenile idiopathic arthritis: clinical and therapeutic implications
- 2017
-
In: Annals of the rheumatic diseases. - : BMJ. - 1468-2060 .- 0003-4967. ; 76:5, s. 906-913
-
Journal article (peer-reviewed)abstract
- Juvenile idiopathic arthritis (JIA) is a heterogeneous group of conditions unified by the presence of chronic childhood arthritis without an identifiable cause. Systemic JIA (sJIA) is a rare form of JIA characterised by systemic inflammation. sJIA is distinguished from other forms of JIA by unique clinical features and treatment responses that are similar to autoinflammatory diseases. However, approximately half of children with sJIA develop destructive, long-standing arthritis that appears similar to other forms of JIA. Using genomic approaches, we sought to gain novel insights into the pathophysiology of sJIA and its relationship with other forms of JIA.MethodsWe performed a genome-wide association study of 770 children with sJIA collected in nine countries by the International Childhood Arthritis Genetics Consortium. Single nucleotide polymorphisms were tested for association with sJIA. Weighted genetic risk scores were used to compare the genetic architecture of sJIA with other JIA subtypes.ResultsThe major histocompatibility complex locus and a locus on chromosome 1 each showed association with sJIA exceeding the threshold for genome-wide significance, while 23 other novel loci were suggestive of association with sJIA. Using a combination of genetic and statistical approaches, we found no evidence of shared genetic architecture between sJIA and other common JIA subtypes.ConclusionsThe lack of shared genetic risk factors between sJIA and other JIA subtypes supports the hypothesis that sJIA is a unique disease process and argues for a different classification framework. Research to improve sJIA therapy should target its unique genetics and specific pathophysiological pathways.
|
|
13. |
|
|
14. |
- ODonnell, Michael, et al.
(author)
-
Registered Replication Report: Dijksterhuis and van Knippenberg (1998)
- 2018
-
In: Perspectives on Psychological Science. - : SAGE PUBLICATIONS LTD. - 1745-6916 .- 1745-6924. ; 13:2, s. 268-294
-
Journal article (peer-reviewed)abstract
- Dijksterhuis and van Knippenberg (1998) reported that participants primed with a category associated with intelligence (professor) subsequently performed 13% better on a trivia test than participants primed with a category associated with a lack of intelligence (soccer hooligans). In two unpublished replications of this study designed to verify the appropriate testing procedures, Dijksterhuis, van Knippenberg, and Holland observed a smaller difference between conditions (2%-3%) as well as a gender difference: Men showed the effect (9.3% and 7.6%), but women did not (0.3% and -0.3%). The procedure used in those replications served as the basis for this multilab Registered Replication Report. A total of 40 laboratories collected data for this project, and 23 of these laboratories met all inclusion criteria. Here we report the meta-analytic results for those 23 direct replications (total N = 4,493), which tested whether performance on a 30-item general-knowledge trivia task differed between these two priming conditions (results of supplementary analyses of the data from all 40 labs, N = 6,454, are also reported). We observed no overall difference in trivia performance between participants primed with the professor category and those primed with the hooligan category (0.14%) and no moderation by gender.
|
|
15. |
- Thompson, PM, et al.
(author)
-
ENIGMA and global neuroscience: A decade of large-scale studies of the brain in health and disease across more than 40 countries
- 2020
-
In: Translational psychiatry. - : Springer Science and Business Media LLC. - 2158-3188. ; 10:1, s. 100-
-
Journal article (peer-reviewed)abstract
- This review summarizes the last decade of work by the ENIGMA (Enhancing NeuroImaging Genetics through Meta Analysis) Consortium, a global alliance of over 1400 scientists across 43 countries, studying the human brain in health and disease. Building on large-scale genetic studies that discovered the first robustly replicated genetic loci associated with brain metrics, ENIGMA has diversified into over 50 working groups (WGs), pooling worldwide data and expertise to answer fundamental questions in neuroscience, psychiatry, neurology, and genetics. Most ENIGMA WGs focus on specific psychiatric and neurological conditions, other WGs study normal variation due to sex and gender differences, or development and aging; still other WGs develop methodological pipelines and tools to facilitate harmonized analyses of “big data” (i.e., genetic and epigenetic data, multimodal MRI, and electroencephalography data). These international efforts have yielded the largest neuroimaging studies to date in schizophrenia, bipolar disorder, major depressive disorder, post-traumatic stress disorder, substance use disorders, obsessive-compulsive disorder, attention-deficit/hyperactivity disorder, autism spectrum disorders, epilepsy, and 22q11.2 deletion syndrome. More recent ENIGMA WGs have formed to study anxiety disorders, suicidal thoughts and behavior, sleep and insomnia, eating disorders, irritability, brain injury, antisocial personality and conduct disorder, and dissociative identity disorder. Here, we summarize the first decade of ENIGMA’s activities and ongoing projects, and describe the successes and challenges encountered along the way. We highlight the advantages of collaborative large-scale coordinated data analyses for testing reproducibility and robustness of findings, offering the opportunity to identify brain systems involved in clinical syndromes across diverse samples and associated genetic, environmental, demographic, cognitive, and psychosocial factors.
|
|
16. |
- Jones, Benedict C, et al.
(author)
-
To which world regions does the valence-dominance model of social perception apply?
- 2021
-
In: Nature Human Behaviour. - : Springer Science and Business Media LLC. - 2397-3374. ; 5:1, s. 159-169
-
Journal article (peer-reviewed)abstract
- Over the past 10 years, Oosterhof and Todorov's valence-dominance model has emerged as the most prominent account of how people evaluate faces on social dimensions. In this model, two dimensions (valence and dominance) underpin social judgements of faces. Because this model has primarily been developed and tested in Western regions, it is unclear whether these findings apply to other regions. We addressed this question by replicating Oosterhof and Todorov's methodology across 11 world regions, 41 countries and 11,570 participants. When we used Oosterhof and Todorov's original analysis strategy, the valence-dominance model generalized across regions. When we used an alternative methodology to allow for correlated dimensions, we observed much less generalization. Collectively, these results suggest that, while the valence-dominance model generalizes very well across regions when dimensions are forced to be orthogonal, regional differences are revealed when we use different extraction methods and correlate and rotate the dimension reduction solution. PROTOCOL REGISTRATION: The stage 1 protocol for this Registered Report was accepted in principle on 5 November 2018. The protocol, as accepted by the journal, can be found at https://doi.org/10.6084/m9.figshare.7611443.v1 .
|
|
17. |
- Pfeiffer, Thomas, et al.
(author)
-
Predicting the replicability of social and behavioural science claims in a crisis: The COVID-19 Preprint Replication Project
- 2023
-
Other publication (other academic/artistic)abstract
- Replications are important for assessing the reliability of published findings. However, they are costly, and it is infeasible to replicate everything. Accurate, fast, lower-cost alternatives such as eliciting predictions could accelerate assessment for rapid policy implementation in a crisis. We elicited judgments from participants on 100 claims from preprints about an emerging area of research (COVID-19 pandemic) using an interactive structured elicitation protocol, and we conducted 29 new high-powered replications. After interacting with their peers, participant groups with lower task expertise (‘beginners’) updated their estimates and confidence in their judgements significantly more than groups with greater task expertise (‘experienced’). For experienced individuals, the average accuracy was 0.57 (95% CI: [0.53, 0.61]) after interaction, and they correctly classified 61% of claims; beginners’ average accuracy was 0.58 (95% CI: [0.54, 0.62]), correctly classifying 69% of claims. The difference in accuracy between groups was not statistically significant, and their judgments on the full set of claims were correlated (r=.48). These results suggest that both beginners and more experienced participants using a structured process have some ability to make better-than-chance predictions about the reliability of ‘fast science’ under conditions of high uncertainty. However, given the importance of such assessments for making evidence-based critical decisions in a crisis, more research is required to understand who the right experts in forecasting replicability are and how their judgements ought to be elicited.
|
|
18. |
- Torchia, Jonathon, et al.
(author)
-
Molecular subgroups of atypical teratoid rhabdoid tumours in children : an integrated genomic and clinicopathological analysis
- 2015
-
In: The Lancet Oncology. - 1470-2045 .- 1474-5488. ; 16:5, s. 569-582
-
Journal article (peer-reviewed)abstract
- Background Rhabdoid brain tumours, also called atypical teratoid rhabdoid tumours, are lethal childhood cancers with characteristic genetic alterations of SMARCB1/hSNF5. Lack of biological understanding of the substantial clinical heterogeneity of these tumours restricts therapeutic advances. We integrated genomic and clinicopathological analyses of a cohort of patients with atypical teratoid rhabdoid tumours to find out the molecular basis for clinical heterogeneity in these tumours. Methods We obtained 259 rhabdoid tumours from 37 international institutions and assessed transcriptional profiles in 43 primary tumours and copy number profiles in 38 primary tumours to discover molecular subgroups of atypical teratoid rhabdoid tumours. We used gene and pathway enrichment analyses to discover group-specific molecular markers and did immunohistochemical analyses on 125 primary tumours to evaluate clinicopathological significance of molecular subgroup and ASCL1-NOTCH signalling. Findings Transcriptional analyses identified two atypical teratoid rhabdoid tumour subgroups with differential enrichment of genetic pathways, and distinct clinicopathological and survival features. Expression of ASCL1, a regulator of NOTCH signalling, correlated with supratentorial location (p=0.004) and superior 5-year overall survival (35%, 95% CI 13-57, and 20%, 6-34, for ASCL1-positive and ASCL1-negative tumours, respectively; p=0.033) in 70 patients who received multimodal treatment. ASCL1 expression also correlated with superior 5-year overall survival (34%, 7-61, and 9%, 0-21, for ASCL1-positive and ASCL1-negative tumours, respectively; p=0.001) in 39 patients who received only chemotherapy without radiation. Cox hazard ratios for overall survival in patients with differential ASCL1 enrichment treated with chemotherapy with or without radiation were 2.02 (95% CI 1.04-3.85; p=0.038) and 3.98 (1.71-9.26; p=0.001). Integrated analyses of molecular subgroupings with clinical prognostic factors showed three distinct clinical risk groups of tumours with different therapeutic outcomes. Interpretation An integration of clinical risk factors and tumour molecular groups can be used to identify patients who are likely to have improved long-term radiation-free survival and might help therapeutic stratification of patients with atypical teratoid rhabdoid tumours.
|
|
19. |
|
|
20. |
- McCarthy, Randy J., et al.
(author)
-
Registered Replication Report on Srull and Wyer (1979)
- 2018
-
In: Advances in Methods and Practices in Psychological Science. - : SAGE Publications Inc. - 2515-2459 .- 2515-2467. ; 1:3, s. 321-336
-
Journal article (peer-reviewed)abstract
- Srull and Wyer (1979) demonstrated that exposing participants to more hostility-related stimuli caused them subsequently to interpret ambiguous behaviors as more hostile. In their Experiment 1, participants descrambled sets of words to form sentences. In one condition, 80% of the descrambled sentences described hostile behaviors, and in another condition, 20% described hostile behaviors. Following the descrambling task, all participants read a vignette about a man named Donald who behaved in an ambiguously hostile manner and then rated him on a set of personality traits. Next, participants rated the hostility of various ambiguously hostile behaviors (all ratings on scales from 0 to 10). Participants who descrambled mostly hostile sentences rated Donald and the ambiguous behaviors as approximately 3 scale points more hostile than did those who descrambled mostly neutral sentences. This Registered Replication Report describes the results of 26 independent replications (N = 7,373 in the total sample; k = 22 labs and N = 5,610 in the primary analyses) of Srull and Wyer?s Experiment 1, each of which followed a preregistered and vetted protocol. A random-effects meta-analysis showed that the protagonist was seen as 0.08 scale points more hostile when participants were primed with 80% hostile sentences than when they were primed with 20% hostile sentences (95% confidence interval, CI = [0.004, 0.16]). The ambiguously hostile behaviors were seen as 0.08 points less hostile when participants were primed with 80% hostile sentences than when they were primed with 20% hostile sentences (95% CI = [?0.18, 0.01]). Although the confidence interval for one outcome excluded zero and the observed effect was in the predicted direction, these results suggest that the currently used methods do not produce an assimilative priming effect that is practically and routinely detectable.
|
|
21. |
- Molinski, Steven V, et al.
(author)
-
Genetic, cell biological, and clinical interrogation of the CFTR mutation c.3700 A>G (p.Ile1234Val) informs strategies for future medical intervention
- 2014
-
In: Genetics in Medicine. - : Nature Publishing Group. - 1098-3600 .- 1530-0366. ; 16:8, s. 625-632
-
Journal article (peer-reviewed)abstract
- Purpose:The purpose of this study was to determine the molecular consequences of the variant c.3700 A>G in the cystic fibrosis transmembrane conductance regulator (CFTR) gene, a variant that has been predicted to cause a missense mutation in the CFTR protein (p.Ile1234Val).Methods:Clinical assays of CFTR function were performed, and genomic DNA from patients homozygous for c.3700 A>G and their family members was sequenced. Total RNA was extracted from epithelial cells of the patients, transcribed into complementary DNA, and sequenced. CFTR complementary DNA clones containing the missense mutation p.Ile1234Val or a truncated exon 19 (p.Ile1234_Arg1239del) were constructed and heterologously expressed to test CFTR protein synthesis and processing.ResultsIn vivo functional measurements revealed that the individuals homozygous for the variant c.3700 A>G exhibited defective CFTR function. We show that this mutation in exon 19 activates a cryptic donor splice site 18 bp upstream of the original donor splice site, resulting in deletion of six amino acids (r.3700_3717del; p.Ile1234_Arg1239del). This deletion, similar to p.Phe508del, causes a primary defect in folding and processing. Importantly, Lumacaftor (VX-809), currently in clinical trial for cystic fibrosis patients with the major cystic fibrosis-causing mutation, p.Phe508del, partially ameliorated the processing defect caused by p.Ile1234_Arg1239del.Conclusion:These studies highlight the need to verify molecular and clinical consequences of CFTR variants to define possible therapeutic strategies
|
|
22. |
- Moshontz, Hannah, et al.
(author)
-
The Psychological Science Accelerator: Advancing Psychology Through a Distributed Collaborative Network
- 2018
-
In: Advances in Methods and Practices in Psychological Science. - : SAGE Publications. - 2515-2459 .- 2515-2467. ; 1:4, s. 501-515
-
Journal article (peer-reviewed)abstract
- Concerns about the veracity of psychological research have been growing. Many findings in psychological science are based on studies with insufficient statistical power and nonrepresentative samples, or may otherwise be limited to specific, ungeneralizable settings or populations. Crowdsourced research, a type of large-scale collaboration in which one or more research projects are conducted across multiple lab sites, offers a pragmatic solution to these and other current methodological challenges. The Psychological Science Accelerator (PSA) is a distributed network of laboratories designed to enable and support crowdsourced research projects. These projects can focus on novel research questions or replicate prior research in large, diverse samples. The PSA’s mission is to accelerate the accumulation of reliable and generalizable evidence in psychological science. Here, we describe the background, structure, principles, procedures, benefits, and challenges of the PSA. In contrast to other crowdsourced research networks, the PSA is ongoing (as opposed to time limited), efficient (in that structures and principles are reused for different projects), decentralized, diverse (in both subjects and researchers), and inclusive (of proposals, contributions, and other relevant input from anyone inside or outside the network). The PSA and other approaches to crowdsourced psychological science will advance understanding of mental processes and behaviors by enabling rigorous research and systematic examination of its generalizability.
|
|
23. |
- Muratov, E. N., et al.
(author)
-
QSAR without borders
- 2020
-
In: Chemical Society Reviews. - : Royal Society of Chemistry (RSC). - 0306-0012 .- 1460-4744. ; 49:11, s. 3525-3564
-
Journal article (peer-reviewed)abstract
- Prediction of chemical bioactivity and physical properties has been one of the most important applications of statistical and more recently, machine learning and artificial intelligence methods in chemical sciences. This field of research, broadly known as quantitative structure-activity relationships (QSAR) modeling, has developed many important algorithms and has found a broad range of applications in physical organic and medicinal chemistry in the past 55+ years. This Perspective summarizes recent technological advances in QSAR modeling but it also highlights the applicability of algorithms, modeling methods, and validation practices developed in QSAR to a wide range of research areas outside of traditional QSAR boundaries including synthesis planning, nanotechnology, materials science, biomaterials, and clinical informatics. As modern research methods generate rapidly increasing amounts of data, the knowledge of robust data-driven modelling methods professed within the QSAR field can become essential for scientists working both within and outside of chemical research. We hope that this contribution highlighting the generalizable components of QSAR modeling will serve to address this challenge.
|
|
24. |
|
|
25. |
|
|