| 1. |
- Breznau, Nate, et al.
(författare)
-
Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty
- 2022
-
Ingår i: Proceedings of the National Academy of Sciences of the United States of America. - : National Academy of Sciences. - 0027-8424 .- 1091-6490. ; 119:44
-
Tidskriftsartikel (refereegranskat)abstract
- This study explores how researchers analytical choices affect the reliability of scientific findings. Most discussions of reliability problems in science focus on systematic biases. We broaden the lens to emphasize the idiosyncrasy of conscious and unconscious decisions that researchers make during data analysis. We coordinated 161 researchers in 73 research teams and observed their research decisions as they used the same data to independently test the same prominent social science hypothesis: that greater immigration reduces support for social policies among the public. In this typical case of social science research, research teams reported both widely diverging numerical findings and substantive conclusions despite identical start conditions. Researchers expertise, prior beliefs, and expectations barely predict the wide variation in research outcomes. More than 95% of the total variance in numerical results remains unexplained even after qualitative coding of all identifiable decisions in each teams workflow. This reveals a universe of uncertainty that remains hidden when considering a single study in isolation. The idiosyncratic nature of how researchers results and conclusions varied is a previously underappreciated explanation for why many scientific hypotheses remain contested. These results call for greater epistemic humility and clarity in reporting scientific findings.
|
|
| 2. |
- Leitman, David I., et al.
(författare)
-
“It’s not what you say, but how you say it” : A reciprocal temporo-frontal network for affective prosody
- 2010
-
Ingår i: Frontiers in Human Neuroscience. - Lausanne, Switzerland : Frontiers Research Foundation. - 1662-5161. ; 4:Article 19
-
Tidskriftsartikel (refereegranskat)abstract
- Humans communicate emotion vocally by modulating acoustic cues such as pitch, intensity and voice quality. Research has documented how the relative presence or absence of such cues alters the likelihood of perceiving an emotion, but the neural underpinnings of acoustic cue-dependent emotion perception remain obscure. Using functional magnetic resonance imaging in 20 subjects we examined a reciprocal circuit consisting of superior temporal cortex, amygdala and inferior frontal gyrus that may underlie affective prosodic comprehension. Results showed that increased saliency of emotion-specific acoustic cues was associated with increased activation in superior temporal cortex (planum temporale (PT), posterior superior temporal gyrus (pSTG), and posterior superior middle gyrus (pMTG)) and amygdala, whereas decreased saliency of acoustic cues was associated with increased inferior frontal activity and temporo-frontal connectivity. These results suggest that sensory-integrative processing is facilitated when the acoustic signal is rich in affective information, yielding increased activation in temporal cortex and amygdala. Conversely, when the acoustic signal is ambiguous, greater evaluative processes are recruited, increasing activation in inferior frontal gyrus (IFG) and IFG STG connectivity. Auditory regions may thus integrate acoustic information with amygdala input to form emotion-specific representations, which are evaluated within inferior frontal regions.
|
|
| 3. |
- Ebersole, Charles R., et al.
(författare)
-
Many Labs 5: Testing Pre-Data-Collection Peer Review as an Intervention to Increase Replicability
- 2020
-
Ingår i: Advances in Methods and Practices in Psychological Science. - : Sage. - 2515-2467 .- 2515-2459. ; 3:3, s. 309-331
-
Tidskriftsartikel (refereegranskat)abstract
- Replication studies in psychological science sometimes fail to reproduce prior findings. If these studies use methods that are unfaithful to the original study or ineffective in eliciting the phenomenon of interest, then a failure to replicate may be a failure of the protocol rather than a challenge to the original finding. Formal pre-data-collection peer review by experts may address shortcomings and increase replicability rates. We selected 10 replication studies from the Reproducibility Project: Psychology (RP:P; Open Science Collaboration, 2015) for which the original authors had expressed concerns about the replication designs before data collection; only one of these studies had yielded a statistically significant effect (p < .05). Commenters suggested that lack of adherence to expert review and low-powered tests were the reasons that most of these RP:P studies failed to replicate the original effects. We revised the replication protocols and received formal peer review prior to conducting new replication studies. We administered the RP:P and revised protocols in multiple laboratories (median number of laboratories per original study = 6.5, range = 3-9; median total sample = 1,279.5, range = 276-3,512) for high-powered tests of each original finding with both protocols. Overall, following the preregistered analysis plan, we found that the revised protocols produced effect sizes similar to those of the RP:P protocols (Delta r = .002 or .014, depending on analytic approach). The median effect size for the revised protocols (r = .05) was similar to that of the RP:P protocols (r = .04) and the original RP:P replications (r = .11), and smaller than that of the original studies (r = .37). Analysis of the cumulative evidence across the original studies and the corresponding three replication attempts provided very precise estimates of the 10 tested effects and indicated that their effect sizes (median r = .07, range = .00-.15) were 78% smaller, on average, than the original effect sizes (median r = .37, range = .19-.50).
|
|
| 4. |
- Leitman, David I., et al.
(författare)
-
Not pitch perfect : Sensory contributions to affective communication impairment in schizophrenia
- 2011
-
Ingår i: Biological Psychiatry. - : Elsevier. - 0006-3223 .- 1873-2402. ; 70:7, s. 611-618
-
Tidskriftsartikel (refereegranskat)abstract
- Background: Schizophrenia patients have vocal affect (prosody) deficits that are treatment resistant and associated with negativesymptoms and poor outcome. The neural correlates of this dysfunction are unclear. Prior study has suggested that schizophrenia vocal affectperception deficits stem from an inability to use acoustic cues, notably pitch, in decoding emotion. Methods: Functional magnetic resonance imaging was performed in 24 schizophrenia patients and 28 healthy control subjects, during theperformance of a four-choice (happiness, fear, anger, neutral) vocal affect identification task in which items for each emotion variedparametrically in affective salient acoustic cue levels. Results: We observed that parametric increases in cue levels in schizophrenia failed to produce the same identification rate increases as incontrol subjects. These deficits correlated with diminished reciprocal activation changes in superior temporal and inferior frontal gyri andreduced temporo-frontal connectivity. Task activation also correlated with independent measures of pitch perception and negativesymptom severity. Conclusions: These findings illustrate the interplay between sensory and higher-order cognitive dysfunction in schizophrenia. Sensorycontributions to vocal affect deficits also suggest that this neurobehavioral marker could be targeted by pharmacological or behavioralremediation of acoustic feature discrimination.
|
|
| 5. |
- Moshontz, Hannah, et al.
(författare)
-
The Psychological Science Accelerator: Advancing Psychology Through a Distributed Collaborative Network
- 2018
-
Ingår i: Advances in Methods and Practices in Psychological Science. - : SAGE Publications. - 2515-2459 .- 2515-2467. ; 1:4, s. 501-515
-
Tidskriftsartikel (refereegranskat)abstract
- Concerns about the veracity of psychological research have been growing. Many findings in psychological science are based on studies with insufficient statistical power and nonrepresentative samples, or may otherwise be limited to specific, ungeneralizable settings or populations. Crowdsourced research, a type of large-scale collaboration in which one or more research projects are conducted across multiple lab sites, offers a pragmatic solution to these and other current methodological challenges. The Psychological Science Accelerator (PSA) is a distributed network of laboratories designed to enable and support crowdsourced research projects. These projects can focus on novel research questions or replicate prior research in large, diverse samples. The PSA’s mission is to accelerate the accumulation of reliable and generalizable evidence in psychological science. Here, we describe the background, structure, principles, procedures, benefits, and challenges of the PSA. In contrast to other crowdsourced research networks, the PSA is ongoing (as opposed to time limited), efficient (in that structures and principles are reused for different projects), decentralized, diverse (in both subjects and researchers), and inclusive (of proposals, contributions, and other relevant input from anyone inside or outside the network). The PSA and other approaches to crowdsourced psychological science will advance understanding of mental processes and behaviors by enabling rigorous research and systematic examination of its generalizability.
|
|
| 6. |
- Beery, T. H., et al.
(författare)
-
Nature to place : Rethinking the environmental connectedness perspective
- 2014
-
Ingår i: Journal of Environmental Psychology. - Amsterdam : Elsevier BV. - 0272-4944 .- 1522-9610. ; 40, s. 198-205
-
Tidskriftsartikel (refereegranskat)abstract
- The environmental connectedness perspective posits that direct encounter with generalized, or non-specific "nature," leads to environmental connectedness and subsequent pro-environmental behavior. This article examines this perspective and proposes a place-based application of the nature encounter-environmental behavior relation. An empirical study using data from a national survey on outdoor recreation and nature-based tourism is presented. Results show a minimal relationship between measures of environmental connectedness and self-reports of environmental behavior. The following examination of the environmental connectedness perspective reveals that environmental connectedness is rooted in a material/objective perspective, neglecting the human domain of perceptions, values, and representations. The environment as "nature" is portrayed as a geographically undefined agent with the inherent power to change human attitudes and behavior. Based on this, the article concludes with a proposed replacement of the elusive concept of nature for the relational concept of place. © 2014 Elsevier Ltd.
|
|
| 7. |
- Jones, Benedict C, et al.
(författare)
-
To which world regions does the valence-dominance model of social perception apply?
- 2021
-
Ingår i: Nature Human Behaviour. - : Springer Science and Business Media LLC. - 2397-3374. ; 5:1, s. 159-169
-
Tidskriftsartikel (refereegranskat)abstract
- Over the past 10 years, Oosterhof and Todorov's valence-dominance model has emerged as the most prominent account of how people evaluate faces on social dimensions. In this model, two dimensions (valence and dominance) underpin social judgements of faces. Because this model has primarily been developed and tested in Western regions, it is unclear whether these findings apply to other regions. We addressed this question by replicating Oosterhof and Todorov's methodology across 11 world regions, 41 countries and 11,570 participants. When we used Oosterhof and Todorov's original analysis strategy, the valence-dominance model generalized across regions. When we used an alternative methodology to allow for correlated dimensions, we observed much less generalization. Collectively, these results suggest that, while the valence-dominance model generalizes very well across regions when dimensions are forced to be orthogonal, regional differences are revealed when we use different extraction methods and correlate and rotate the dimension reduction solution. PROTOCOL REGISTRATION: The stage 1 protocol for this Registered Report was accepted in principle on 5 November 2018. The protocol, as accepted by the journal, can be found at https://doi.org/10.6084/m9.figshare.7611443.v1 .
|
|
| 8. |
- McCarthy, Randy J., et al.
(författare)
-
Registered Replication Report on Srull and Wyer (1979)
- 2018
-
Ingår i: Advances in Methods and Practices in Psychological Science. - : SAGE Publications Inc. - 2515-2459 .- 2515-2467. ; 1:3, s. 321-336
-
Tidskriftsartikel (refereegranskat)abstract
- Srull and Wyer (1979) demonstrated that exposing participants to more hostility-related stimuli caused them subsequently to interpret ambiguous behaviors as more hostile. In their Experiment 1, participants descrambled sets of words to form sentences. In one condition, 80% of the descrambled sentences described hostile behaviors, and in another condition, 20% described hostile behaviors. Following the descrambling task, all participants read a vignette about a man named Donald who behaved in an ambiguously hostile manner and then rated him on a set of personality traits. Next, participants rated the hostility of various ambiguously hostile behaviors (all ratings on scales from 0 to 10). Participants who descrambled mostly hostile sentences rated Donald and the ambiguous behaviors as approximately 3 scale points more hostile than did those who descrambled mostly neutral sentences. This Registered Replication Report describes the results of 26 independent replications (N = 7,373 in the total sample; k = 22 labs and N = 5,610 in the primary analyses) of Srull and Wyer?s Experiment 1, each of which followed a preregistered and vetted protocol. A random-effects meta-analysis showed that the protagonist was seen as 0.08 scale points more hostile when participants were primed with 80% hostile sentences than when they were primed with 20% hostile sentences (95% confidence interval, CI = [0.004, 0.16]). The ambiguously hostile behaviors were seen as 0.08 points less hostile when participants were primed with 80% hostile sentences than when they were primed with 20% hostile sentences (95% CI = [?0.18, 0.01]). Although the confidence interval for one outcome excluded zero and the observed effect was in the predicted direction, these results suggest that the currently used methods do not produce an assimilative priming effect that is practically and routinely detectable.
|
|
| 9. |
- Verschuere, Bruno, et al.
(författare)
-
Registered Replication Report on Mazar, Amir, and Ariely (2008)
- 2018
-
Ingår i: Advances in Methods and Practices in Psychological Science. - : SAGE Publications. - 2515-2459 .- 2515-2467. ; 1:3, s. 299-317
-
Tidskriftsartikel (refereegranskat)abstract
- The self-concept maintenance theory holds that many people will cheat in order to maximize self-profit, but only to the extent that they can do so while maintaining a positive self-concept. Mazar, Amir, and Ariely (2008, Experiment 1) gave participants an opportunity and incentive to cheat on a problem-solving task. Prior to that task, participants either recalled the Ten Commandments (a moral reminder) or recalled 10 books they had read in high school (a neutral task). Results were consistent with the self-concept maintenance theory. When given the opportunity to cheat, participants given the moral-reminder priming task reported solving 1.45 fewer matrices than did those given a neutral prime (Cohen’s d = 0.48); moral reminders reduced cheating. Mazar et al.’s article is among the most cited in deception research, but their Experiment 1 has not been replicated directly. This Registered Replication Report describes the aggregated result of 25 direct replications (total N = 5,786), all of which followed the same preregistered protocol. In the primary meta-analysis (19 replications, total n = 4,674), participants who were given an opportunity to cheat reported solving 0.11 more matrices if they were given a moral reminder than if they were given a neutral reminder (95% confidence interval = [−0.09, 0.31]). This small effect was numerically in the opposite direction of the effect observed in the original study (Cohen’s d = −0.04).
|
|
| 10. |
- Huntington-Klein, Nick, et al.
(författare)
-
Subjective evidence evaluation survey for many-analysts studies
- 2024
-
Ingår i: Royal Society Open Science. - : The Royal Society. - 2054-5703 .- 2054-5703. ; 11:7
-
Tidskriftsartikel (refereegranskat)abstract
- Many-analysts studies explore how well an empirical claim withstands plausible alternative analyses of the same dataset by multiple, independent analysis teams. Conclusions from these studies typically rely on a single outcome metric (e.g. effect size) provided by each analysis team. Although informative about the range of plausible effects in a dataset, a single effect size from each team does not provide a complete, nuanced understanding of how analysis choices are related to the outcome. We used the Delphi consensus technique with input from 37 experts to develop an 18-item subjective evidence evaluation survey (SEES) to evaluate how each analysis team views the methodological appropriateness of the research design and the strength of evidence for the hypothesis. We illustrate the usefulness of the SEES in providing richer evidence assessment with pilot data from a previous many-analysts study.
|
|
