| 1. |
- Tierney, W., et al.
(författare)
-
A creative destruction approach to replication : Implicit work and sex morality across cultures
- 2021
-
Ingår i: Journal of Experimental Social Psychology. - : Elsevier BV. - 0022-1031 .- 1096-0465. ; 93
-
Tidskriftsartikel (refereegranskat)abstract
- How can we maximize what is learned from a replication study? In the creative destruction approach to replication, the original hypothesis is compared not only to the null hypothesis, but also to predictions derived from multiple alternative theoretical accounts of the phenomenon. To this end, new populations and measures are included in the design in addition to the original ones, to help determine which theory best accounts for the results across multiple key outcomes and contexts. The present pre-registered empirical project compared the Implicit Puritanism account of intuitive work and sex morality to theories positing regional, religious, and social class differences; explicit rather than implicit cultural differences in values; self-expression vs. survival values as a key cultural fault line; the general moralization of work; and false positive effects. Contradicting Implicit Puritanism's core theoretical claim of a distinct American work morality, a number of targeted findings replicated across multiple comparison cultures, whereas several failed to replicate in all samples and were identified as likely false positives. No support emerged for theories predicting regional variability and specific individual-differences moderators (religious affiliation, religiosity, and education level). Overall, the results provide evidence that work is intuitively moralized across cultures.
|
|
| 2. |
- Delios, A., et al.
(författare)
-
Examining the generalizability of research findings from archival data
- 2022
-
Ingår i: Proceedings of the National Academy of Sciences of the United States of America. - : Proceedings of the National Academy of Sciences. - 0027-8424 .- 1091-6490. ; 119:30
-
Tidskriftsartikel (refereegranskat)abstract
- This initiative examined systematically the extent to which a large set of archival research findings generalizes across contexts. We repeated the key analyses for 29 original strategic management effects in the same context (direct reproduction) as well as in 52 novel time periods and geographies; 45% of the reproductions returned results matching the original reports together with 55% of tests in different spans of years and 40% of tests in novel geographies. Some original findings were associated with multiple new tests. Reproducibility was the best predictor of generalizability-for the findings that proved directly reproducible, 84% emerged in other available time periods and 57% emerged in other geographies. Overall, only limited empirical evidence emerged for context sensitivity. In a forecasting survey, independent scientists were able to anticipate which effects would find support in tests in new samples.
|
|
| 3. |
|
|
| 4. |
|
|
| 5. |
|
|
| 6. |
- Jones, Benedict C, et al.
(författare)
-
To which world regions does the valence-dominance model of social perception apply?
- 2021
-
Ingår i: Nature Human Behaviour. - : Springer Science and Business Media LLC. - 2397-3374. ; 5:1, s. 159-169
-
Tidskriftsartikel (refereegranskat)abstract
- Over the past 10 years, Oosterhof and Todorov's valence-dominance model has emerged as the most prominent account of how people evaluate faces on social dimensions. In this model, two dimensions (valence and dominance) underpin social judgements of faces. Because this model has primarily been developed and tested in Western regions, it is unclear whether these findings apply to other regions. We addressed this question by replicating Oosterhof and Todorov's methodology across 11 world regions, 41 countries and 11,570 participants. When we used Oosterhof and Todorov's original analysis strategy, the valence-dominance model generalized across regions. When we used an alternative methodology to allow for correlated dimensions, we observed much less generalization. Collectively, these results suggest that, while the valence-dominance model generalizes very well across regions when dimensions are forced to be orthogonal, regional differences are revealed when we use different extraction methods and correlate and rotate the dimension reduction solution. PROTOCOL REGISTRATION: The stage 1 protocol for this Registered Report was accepted in principle on 5 November 2018. The protocol, as accepted by the journal, can be found at https://doi.org/10.6084/m9.figshare.7611443.v1 .
|
|
| 7. |
- Grossmann, Igor, et al.
(författare)
-
Insights into the accuracy of social scientists' forecasts of societal change
- 2023
-
Ingår i: Nature Human Behaviour. - : Springer Nature. - 2397-3374. ; 7, s. 484-501
-
Tidskriftsartikel (refereegranskat)abstract
- How well can social scientists predict societal change, and what processes underlie their predictions? To answer these questions, we ran two forecasting tournaments testing the accuracy of predictions of societal change in domains commonly studied in the social sciences: ideological preferences, political polarization, life satisfaction, sentiment on social media, and gender-career and racial bias. After we provided them with historical trend data on the relevant domain, social scientists submitted pre-registered monthly forecasts for a year (Tournament 1; N = 86 teams and 359 forecasts), with an opportunity to update forecasts on the basis of new data six months later (Tournament 2; N = 120 teams and 546 forecasts). Benchmarking forecasting accuracy revealed that social scientists' forecasts were on average no more accurate than those of simple statistical models (historical means, random walks or linear regressions) or the aggregate forecasts of a sample from the general public (N = 802). However, scientists were more accurate if they had scientific expertise in a prediction domain, were interdisciplinary, used simpler models and based predictions on prior data. How accurate are social scientists in predicting societal change, and what processes underlie their predictions? Grossmann et al. report the findings of two forecasting tournaments. Social scientists' forecasts were on average no more accurate than those of simple statistical models.
|
|
| 8. |
- Moshontz, Hannah, et al.
(författare)
-
The Psychological Science Accelerator: Advancing Psychology Through a Distributed Collaborative Network
- 2018
-
Ingår i: Advances in Methods and Practices in Psychological Science. - : SAGE Publications. - 2515-2459 .- 2515-2467. ; 1:4, s. 501-515
-
Tidskriftsartikel (refereegranskat)abstract
- Concerns about the veracity of psychological research have been growing. Many findings in psychological science are based on studies with insufficient statistical power and nonrepresentative samples, or may otherwise be limited to specific, ungeneralizable settings or populations. Crowdsourced research, a type of large-scale collaboration in which one or more research projects are conducted across multiple lab sites, offers a pragmatic solution to these and other current methodological challenges. The Psychological Science Accelerator (PSA) is a distributed network of laboratories designed to enable and support crowdsourced research projects. These projects can focus on novel research questions or replicate prior research in large, diverse samples. The PSA’s mission is to accelerate the accumulation of reliable and generalizable evidence in psychological science. Here, we describe the background, structure, principles, procedures, benefits, and challenges of the PSA. In contrast to other crowdsourced research networks, the PSA is ongoing (as opposed to time limited), efficient (in that structures and principles are reused for different projects), decentralized, diverse (in both subjects and researchers), and inclusive (of proposals, contributions, and other relevant input from anyone inside or outside the network). The PSA and other approaches to crowdsourced psychological science will advance understanding of mental processes and behaviors by enabling rigorous research and systematic examination of its generalizability.
|
|
| 9. |
- Protzko, J., et al.
(författare)
-
Nonlinear Effect Amplification: Differential Susceptibility of Verbal Overshadowing as a Function of Time to Interference
- 2022
-
Ingår i: Journal of Applied Research in Memory and Cognition. - : American Psychological Association (APA). - 2211-3681 .- 2211-369X.
-
Tidskriftsartikel (refereegranskat)abstract
- Verbalizing visual memories can interfere with later accurate recall. Whereas changes in the magnitude of this verbal overshadowing effect (VOE) as a function of delay have been reported, no study has systematically investigated multiple shorter nonimmediate delays. Does VOE happen when verbalization occurs 5-min postencoding? 10 min? 15 min? We show in a preregistered study involving 4,501 American adults randomly assigned to different timing paradigms, that the effect size of VOE at 5 or 10 min is nearly zero, with a stable and significant inhibitory effect from 15 to 20 min. We further investigate this nonlinearity in a second study of 3,174 individuals showing a distinct nonlinear "effect amplification " sometime between 12 and 14 min. This apparent critical period after stimulus onset where susceptibility to verbal interference dramatically increases may help explain potential difficulties replicating VOE. More importantly, it suggests the possibility that the 12-14 min period may represent a critical window for other interference paradigms as well. General Audience Summary Eyewitness memory after seeing a crime committed can become corrupted by numerous sources. One source of corruption is describing the memory using words, called verbal overshadowing. Here, we show in two large preregistered studies that the effect of verbalization after seeing a crime occur nonlinearly amplifies from a null effect at short time intervals to a stable effect around 14 min after seeing the crime. This nonlinear effect occurs somewhere in the 12-14 min window. Future studies should investigate whether this 12-14 min window where effect sizes nonlinearly amplify is seen around other memory paradigms.
|
|
| 10. |
- Lundmark, Sebastian, 1986, et al.
(författare)
-
The need for public opinion and survey methodology research to embrace preregistration and replication, exemplified by a team's failure to replicate their own findings on visual cues in grid-type questions
- 2023
-
Ingår i: International journal of public opinion research. - : Oxford University Press (OUP). - 0954-2892 .- 1471-6909. ; 35:1
-
Tidskriftsartikel (refereegranskat)abstract
- Survey researchers take great care to measure respondents' answers in an unbiased way; but, how successful are we as a field at remedying unintended and intended biases in our research? The validity of inferences drawn from studies has been found to be improved by the implementation of preregistration practices. Despite this, only 3 of the 83 published articles in POQ and IJPOR in 2020 feature explicitly stated preregistered hypotheses or analyses. This manuscript aims to show survey methodologists how preregistration and replication (where possible) are in service to the broader mission of survey methodology. To that end, we present a practical example of how unknown biases in analysis strategies without preregistration or replication inflate type I errors. In an initial data collection, our analysis showed that the visual layout of battery-type questions significantly decreased data quality. But after committing to replicating and preregistering the hypotheses and analysis plans, none of the results replicated successfully, despite keeping the procedure, sample provider, and analyses identical. This manuscript illustrates how preregistration and replication practices might, in the long term, likely help unburden the academic literature from follow-up publications relying on type I errors.
|
|
