| 1. |
- Weglage, Moritz, et al.
(författare)
-
Complete representation of action space and value in all dorsal striatal pathways
- 2021
-
Ingår i: Cell Reports. - : Elsevier BV. - 2211-1247. ; 36:4
-
Tidskriftsartikel (refereegranskat)abstract
- The dorsal striatum plays a central role in the selection, execution, and evaluation of actions. An emerging model attributes action selection to the matrix and evaluation to the striosome compartment. Here, we use large-scale cell-type-specific calcium imaging to determine the activity of striatal projection neurons (SPNs) during motor and decision behaviors in the three major outputs of the dorsomedial striatum: Oprm1+ striosome versus D1+ direct and A2A+ indirect pathway SPNs. We find that Oprm1+ SPNs show complex tunings to simple movements and value-guided actions, which are conserved across many sessions in a single task but remap between contexts. During decision making, the SPN tuning profiles form a complete representation in which sequential SPN activity jointly encodes task progress and value. We propose that the three major output pathways in the dorsomedial striatum share a similarly complete representation of the entire action space, including task- and phase-specific signals of action value and choice.
|
|
| 2. |
- Wärnberg, Emil, et al.
(författare)
-
Feasibility of dopamine as a vector-valued feedback signal in the basal ganglia.
- 2023
-
Ingår i: Proceedings of the National Academy of Sciences of the United States of America. - : Proceedings of the National Academy of Sciences. - 0027-8424 .- 1091-6490. ; 120:32
-
Tidskriftsartikel (refereegranskat)abstract
- It is well established that midbrain dopaminergic neurons support reinforcement learning (RL) in the basal ganglia by transmitting a reward prediction error (RPE) to the striatum. In particular, different computational models and experiments have shown that a striatum-wide RPE signal can support RL over a small discrete set of actions (e.g., no/no-go, choose left/right). However, there is accumulating evidence that the basal ganglia functions not as a selector between predefined actions but rather as a dynamical system with graded, continuous outputs. To reconcile this view with RL, there is a need to explain how dopamine could support learning of continuous outputs, rather than discrete action values. Inspired by the recent observations that besides RPE, the firing rates of midbrain dopaminergic neurons correlate with motor and cognitive variables, we propose a model in which dopamine signal in the striatum carries a vector-valued error feedback signal (a loss gradient) instead of a homogeneous scalar error (a loss). We implement a local, "three-factor" corticostriatal plasticity rule involving the presynaptic firing rate, a postsynaptic factor, and the unique dopamine concentration perceived by each striatal neuron. With this learning rule, we show that such a vector-valued feedback signal results in an increased capacity to learn a multidimensional series of real-valued outputs. Crucially, we demonstrate that this plasticity rule does not require precise nigrostriatal synapses but remains compatible with experimental observations of random placement of varicosities and diffuse volume transmission of dopamine.
|
|
| 3. |
- Wärnberg, Emil
(författare)
-
On learning in mice and machines : continuous population codes in natural and artificial neural networks
- 2023
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Neural networks, whether artificial in a computer or natural in the brain, could represent information either using discrete symbols or continuous vector spaces. In this thesis, I explore how neural networks can represent continuous vector spaces, using both simulated neural networks and analysis of real neural population data recorded from mice. A special focus is on the networks of the basal ganglia circuit and on reinforcement learning, i.e., learning from rewards and punishments.The thesis includes four scientific papers: two theoretical/computational (Papers I and IV) and two with analysis of real data (Papers II and III).In Paper I, we explore methods for implementing continuous vector spaces in networks of spiking neurons using multidimensional attractors, and propose an explanation for why it is hard to escape the neural manifolds created by such attractors.In Paper II, we analyze experimental data from dorsomedial striatum collected using 1-photon calcium imaging of transgenic mice with celltype-specific markers for the striatal direct, indirect and patch pathways, as the mice were gathering rewards in a 2-choice task. In line with extensive previous results, our data analysis revealed a number of neural signatures of reinforcement learning, but no apparent difference between the pathways.In Paper III, we present a new software tool for tracking neurons across weeks of 1-photon calcium imaging, and employ it to follow patch-specific striatal projection neurons from the dorsomedial striatum across two weeks of daily recordings.In Paper IV, we propose a model for how the nigrostriatal dopaminergic projection could, in a biologically plausible way, convey a vector-valued error gradient to the dorsal striatum, as required for backpropagation.Based on the results of the papers and a review of existing literature, I argue that while the basal ganglia indeed make up a circuit for reinforcement learning as previously thought, this circuit represents reinforcement learning states, actions and policies using a continuous population code and not using discrete symbols.
|
|
| 4. |
- Wärnberg, Emil, et al.
(författare)
-
Perturbing low dimensional activity manifolds in spiking neuronal networks
- 2019
-
Ingår i: PloS Computational Biology. - : Public Library of Science. - 1553-734X .- 1553-7358. ; 15:5
-
Tidskriftsartikel (refereegranskat)abstract
- Several recent studies have shown that neural activity in vivo tends to be constrained to a low-dimensional manifold. Such activity does not arise in simulated neural networks with homogeneous connectivity and it has been suggested that it is indicative of some other connectivity pattern in neuronal networks. In particular, this connectivity pattern appears to be constraining learning so that only neural activity patterns falling within the intrinsic manifold can be learned and elicited. Here, we use three different models of spiking neural networks (echo-state networks, the Neural Engineering Framework and Efficient Coding) to demonstrate how the intrinsic manifold can be made a direct consequence of the circuit connectivity. Using this relationship between the circuit connectivity and the intrinsic manifold, we show that learning of patterns outside the intrinsic manifold corresponds to much larger changes in synaptic weights than learning of patterns within the intrinsic manifold. Assuming larger changes to synaptic weights requires extensive learning, this observation provides an explanation of why learning is easier when it does not require the neural activity to leave its intrinsic manifold.
|
|
| 5. |
- Wärnberg, Emil, et al.
(författare)
-
Tracking activity of neurons across weeks from 1-photon calcium imaging
-
Annan publikation (övrigt vetenskapligt/konstnärligt)abstract
- One-photon calcium imaging using head-mounted miniaturized endoscopes ("miniscopes") is a powerful method for imaging neuron activity of genetically-labeled neurons with single-cellresolution from awake and freely moving animals. A major challenge in analysis of imagingdata is the recording of single neurons over multiple days. To improve speed and accuracy,we created a software package for calcium extraction with a highly optimized GPU-basedimplementation of the Constrained Non-negative Matrix Factorization (CNMF) algorithmwith OASIS deconvolution of the traces. Additionally, we introduce negentropy as a robust statistical measure to rapidly initialize spatial footprints as well as to automatically find the best regularization and time constant of the OASIS deconvolution. We found that our implementation is 50x-100x faster compared to a state-of-the-art calcium processing tool. The significant speed advantage enables processing of a large number of calcium videos from separate recording days as a single concatenated video, making it feasible to follow individual cells across multiple days in a consistent manner. We demonstrate how our approach can capture the neural dynamics across more than two weeks of daily recordings with 1-photon miniscope of Oprm1-Cre+ striatal projection neurons as mice develop a task strategy in a maze.
|
|
