| 1. |
- Adams, Rick A., et al.
(författare)
-
Variability in Action Selection Relates to Striatal Dopamine 2/3 Receptor Availability in Humans : A PET Neuroimaging Study Using Reinforcement Learning and Active Inference Models
- 2020
-
Ingår i: Cerebral Cortex. - : Oxford University Press (OUP). - 1047-3211 .- 1460-2199. ; 30:6, s. 3573-3589
-
Tidskriftsartikel (refereegranskat)abstract
- Choosing actions that result in advantageous outcomes is a fundamental function of nervous systems. All computational decision-making models contain a mechanism that controls the variability of (or confidence in) action selection, but its neural implementation is unclear-especially in humans. We investigated this mechanism using two influential decision-making frameworks: active inference (AI) and reinforcement learning (RL). In AI, the precision (inverse variance) of beliefs about policies controls action selection variability-similar to decision 'noise' parameters in RL-and is thought to be encoded by striatal dopamine signaling. We tested this hypothesis by administering a 'go/no-go' task to 75 healthy participants, and measuring striatal dopamine 2/3 receptor (D2/3R) availability in a subset (n = 25) using [C-11]-(+)-PHNO positron emission tomography. In behavioral model comparison, RL performed best across the whole group but AI performed best in participants performing above chance levels. Limbic striatal D2/3R availability had linear relationships with AI policy precision (P = 0.029) as well as with RL irreducible decision 'noise' (P = 0.020), and this relationship with D2/3R availability was confirmed with a 'decision stochasticity' factor that aggregated across both models (P = 0.0006). These findings are consistent with occupancy of inhibitory striatal D(2/3)Rs decreasing the variability of action selection in humans.
|
|
| 2. |
- Bach, Dominik R., et al.
(författare)
-
Human Hippocampus Arbitrates Approach-Avoidance Conflict
- 2014
-
Ingår i: Current Biology. - : Elsevier BV. - 0960-9822 .- 1879-0445. ; 24:5, s. 541-547
-
Tidskriftsartikel (refereegranskat)abstract
- Animal models of human anxiety often invoke a conflict between approach and avoidance [1, 2]. In these, a key behavioral assay comprises passive avoidance of potential threat and inhibition, both thought to be controlled by ventral hippocampus [2-6]. Efforts to translate these approaches to clinical contexts [7, 8] are hampered by the fact that it is not known whether humans manifest analogous approach-avoidance dispositions and, if so, whether they share a homologous neurobiological substrate [9]. Here, we developed a paradigm to investigate the role of human hippocampus in arbitrating an approach-avoidance conflict under varying levels of potential threat. Across four experiments, subjects showed analogous behavior by adapting both passive avoidance behavior and behavioral inhibition to threat level. Using functional magnetic resonance imaging (fMRI), we observe that threat level engages the anterior hippocampus, the human homolog of rodent ventral hippocampus [10]. Testing patients with selective hippocampal lesions, we demonstrate a causal role for the hippocampus with patients showing reduced passive avoidance behavior and inhibition across all threat levels. Our data provide the first human assay for approach-avoidance conflict akin to that of animal anxiety models. The findings bridge rodent and human research on passive avoidance and behavioral inhibition and furnish a framework for addressing the neuronal underpinnings of human anxiety disorders, where our data indicate a major role for the hippocampus.
|
|
| 3. |
- Betts, Matthew J., et al.
(författare)
-
Learning in anticipation of reward and punishment : perspectives across the human lifespan
- 2020
-
Ingår i: Neurobiology of Aging. - : Elsevier BV. - 0197-4580 .- 1558-1497. ; 96, s. 49-57
-
Tidskriftsartikel (refereegranskat)abstract
- Learning to act to receive reward and to withhold to avoid punishment has been found to be easier than learning the opposite contingencies in young adults. To what extent this type of behavioral adaptation might develop during childhood and adolescence and differ during aging remains unclear. We therefore tested 247 healthy individuals across the human life span (7–80 years) with an orthogonalized valenced go/no-go learning task. Computational modeling revealed that peak performance in young adults was attributable to greater sensitivity to both reward and punishment. However, in children and adolescents, we observed an increased bias toward action but not reward sensitivity. By contrast, reduced learning in midlife and older adults was accompanied by decreased reward sensitivity and especially punishment sensitivity along with an age-related increase in the Pavlovian bias. These findings reveal distinct motivation-dependent learning capabilities across the human life span, which cannot be probed using conventional go/reward no-go/punishment style paradigms that have important implications in lifelong education.
|
|
| 4. |
- Chowdhury, Rumana, et al.
(författare)
-
Dopamine restores reward prediction errors in old age
- 2013
-
Ingår i: Nature Neuroscience. - : Springer Science and Business Media LLC. - 1097-6256 .- 1546-1726. ; 16:5, s. 648-653
-
Tidskriftsartikel (refereegranskat)abstract
- Senescence affects the ability to utilize information about the likelihood of rewards for optimal decision-making. Using functional magnetic resonance imaging in humans, we found that healthy older adults had an abnormal signature of expected value, resulting in an incomplete reward prediction error (RPE) signal in the nucleus accumbens, a brain region that receives rich input projections from substantia nigra/ventral tegmental area (SN/VTA) dopaminergic neurons. Structural connectivity between SN/VTA and striatum, measured by diffusion tensor imaging, was tightly coupled to inter-individual differences in the expression of this expected reward value signal. The dopamine precursor levodopa (L-DOPA) increased the task-based learning rate and task performance in some older adults to the level of young adults. This drug effect was linked to restoration of a canonical neural RPE. Our results identify a neurochemical signature underlying abnormal reward processing in older adults and indicate that this can be modulated by L-DOPA.
|
|
| 5. |
- de Boer, Lieke, et al.
(författare)
-
Attenuation of dopamine-modulated prefrontal value signals underlies probabilistic reward learning deficits in old age
- 2017
-
Ingår i: eLIFE. - : ELIFE SCIENCES PUBLICATIONS LTD. - 2050-084X. ; 6
-
Tidskriftsartikel (refereegranskat)abstract
- Probabilistic reward learning is characterised by individual differences that become acute in aging. This may be due to age-related dopamine (DA) decline affecting neural processing in striatum, prefrontal cortex, or both. We examined this by administering a probabilistic reward learning task to younger and older adults, and combining computational modelling of behaviour, fMRI and PET measurements of DA D1 availability. We found that anticipatory value signals in ventromedial prefrontal cortex (vmPFC) were attenuated in older adults. The strength of this signal predicted performance beyond age and was modulated by D1 availability in nucleus accumbens. These results uncover that a value-anticipation mechanism in vmPFC declines in aging, and that this mechanism is associated with DA D1 receptor availability.
|
|
| 6. |
- de Boer, Lieke, et al.
(författare)
-
Corticostriatal White Matter Integrity and Dopamine D1 Receptor Availability Predict Age Differences in Prefrontal Value Signaling during Reward Learning
- 2020
-
Ingår i: Cerebral Cortex. - : Oxford University Press (OUP). - 1047-3211 .- 1460-2199. ; 30:10, s. 5270-5280
-
Tidskriftsartikel (refereegranskat)abstract
- Probabilistic reward learning reflects the ability to adapt choices based on probabilistic feedback. The dopaminergically innervated corticostriatal circuit in the brain plays an important role in supporting successful probabilistic reward learning. Several components of the corticostriatal circuit deteriorate with age, as it does probabilistic reward learning. We showed previously that D1 receptor availability in NAcc predicts the strength of anticipatory value signaling in vmPFC, a neural correlate of probabilistic learning that is attenuated in older participants and predicts probabilistic reward learning performance. We investigated how white matter integrity in the pathway between nucleus accumbens (NAcc) and ventromedial prefrontal cortex (vmPFC) relates to the strength of anticipatory value signaling in vmPFC in younger and older participants. We found that in a sample of 22 old and 23 young participants, fractional anisotropy in the pathway between NAcc and vmPFC predicted the strength of value signaling in vmPFC independently from D1 receptor availability in NAcc. These findings provide tentative evidence that integrity in the dopaminergic and white matter pathways of corticostriatal circuitry supports the expression of value signaling in vmPFC which supports reward learning, however, the limited sample size calls for independent replication. These and future findings could add to the improved understanding of how corticostriatal integrity contributes to reward learning ability.
|
|
| 7. |
- de Boer, Lieke, et al.
(författare)
-
Dorsal striatal dopamine D1 receptor availability predicts an instrumental bias in action learning
- 2019
-
Ingår i: Proceedings of the National Academy of Sciences of the United States of America. - : National Academy of Sciences. - 0027-8424 .- 1091-6490. ; 116:1, s. 261-270
-
Tidskriftsartikel (refereegranskat)abstract
- Learning to act to obtain reward and inhibit to avoid punishment is easier compared with learning the opposite contingencies. This coupling of action and valence is often thought of as a Pavlovian bias, although recent research has shown it may also emerge through instrumental mechanisms. We measured this learning bias with a rewarded go/no-go task in 60 adults of different ages. Using computational modeling, we characterized the bias as being instrumental. To assess the role of endogenous dopamine (DA) in the expression of this bias, we quantified DA D1 receptor availability using positron emission tomography (PET) with the radioligand [11C]SCH23390. Using principal-component analysis on the binding potentials in a number of cortical and striatal regions of interest, we demonstrated that cortical, dorsal striatal, and ventral striatal areas provide independent sources of variance in DA D1 receptor availability. Interindividual variation in the dorsal striatal component was related to the strength of the instrumental bias during learning. These data suggest at least three anatomical sources of variance in DA D1 receptor availability separable using PET in humans, and we provide evidence that human dorsal striatal DA D1 receptors are involved in the modulation of instrumental learning biases.
|
|
| 8. |
- Economides, Marcos, et al.
(författare)
-
Anterior Cingulate Cortex Instigates Adaptive Switches in Choice by Integrating Immediate and Delayed Components of Value in Ventromedial Prefrontal Cortex
- 2014
-
Ingår i: Journal of Neuroscience. - 0270-6474 .- 1529-2401. ; 34:9, s. 3340-3349
-
Tidskriftsartikel (refereegranskat)abstract
- Actions can lead to an immediate reward or punishment and a complex set of delayed outcomes. Adaptive choice necessitates the brain track and integrate both of these potential consequences. Here, we designed a sequential task whereby the decision to exploit or forego an available offer was contingent on comparing immediate value and a state-dependent future cost of expending a limited resource. Crucially, the dynamics of the task demanded frequent switches in policy based on an online computation of changing delayed consequences. We found that human subjects choose on the basis of a near-optimal integration of immediate reward and delayed consequences, with the latter computed in a prefrontal network. Within this network, anterior cingulate cortex (ACC) was dynamically coupled to ventromedial prefrontal cortex (vmPFC) when adaptive switches in choice were required. Our results suggest a choice architecture whereby interactions between ACC and vmPFC underpin an integration of immediate and delayed components of value to support flexible policy switching that accommodates the potential delayed consequences of an action.
|
|
| 9. |
- Ereira, Sam, et al.
(författare)
-
Overcoming Pavlovian bias in semantic space
- 2021
-
Ingår i: Scientific Reports. - : Springer Science and Business Media LLC. - 2045-2322. ; 11:1
-
Tidskriftsartikel (refereegranskat)abstract
- Action is invigorated in the presence of reward-predicting stimuli and inhibited in the presence of punishment-predicting stimuli. Although valuable as a heuristic, this Pavlovian bias can also lead to maladaptive behaviour and is implicated in addiction. Here we explore whether Pavlovian bias can be overcome through training. Across five experiments, we find that Pavlovian bias is resistant to unlearning under most task configurations. However, we demonstrate that when subjects engage in instrumental learning in a verbal semantic space, as opposed to a motoric space, not only do they exhibit the typical Pavlovian bias, but this Pavlovian bias diminishes with training. Our results suggest that learning within the semantic space is necessary, but not sufficient, for subjects to unlearn their Pavlovian bias, and that other task features, such as gamification and spaced stimulus presentation may also be necessary. In summary, we show that Pavlovian bias, whilst robust, is susceptible to change with experience, but only under specific environmental conditions.
|
|
| 10. |
- Garzón, Benjamín, et al.
(författare)
-
Investigating associations of delay discounting with brain structure, working memory, and episodic memory
- 2022
-
Ingår i: Cerebral Cortex. - : Oxford University Press (OUP). - 1047-3211 .- 1460-2199.
-
Tidskriftsartikel (refereegranskat)abstract
- Introduction: Delay discounting (DD), the preference for smaller and sooner rewards over larger and later ones, is an important behavioural phenomenon for daily functioning of increasing interest within psychopathology. The neurobiological mechanisms behind DD are not well understood and the literature on structural correlates of DD shows inconsistencies.Methods: Here we leveraged a large openly available dataset (n = 1196) to investigate associations with memory performance and gray and white matter correlates of DD using linked independent component analysis.Results: Greater DD was related to smaller anterior temporal gray matter volume. Associations of DD with total cortical volume, subcortical volumes, markers of white matter microscopic organization, working memory, and episodic memory scores were not significant after controlling for education and income.Conclusion: Effects of size comparable to the one we identified would be unlikely to be replicated with sample sizes common in many previous studies in this domain, which may explain the incongruities in the literature. The paucity and small size of the effects detected in our data underscore the importance of using large samples together with methods that accommodate their statistical structure and appropriate control for confounders, as well as the need to devise paradigms with improved task parameter reliability in studies relating brain structure and cognitive abilities with DD.
|
|
