| 1. |
- Wärnberg, Emil
(författare)
-
On learning in mice and machines : continuous population codes in natural and artificial neural networks
- 2023
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Neural networks, whether artificial in a computer or natural in the brain, could represent information either using discrete symbols or continuous vector spaces. In this thesis, I explore how neural networks can represent continuous vector spaces, using both simulated neural networks and analysis of real neural population data recorded from mice. A special focus is on the networks of the basal ganglia circuit and on reinforcement learning, i.e., learning from rewards and punishments.The thesis includes four scientific papers: two theoretical/computational (Papers I and IV) and two with analysis of real data (Papers II and III).In Paper I, we explore methods for implementing continuous vector spaces in networks of spiking neurons using multidimensional attractors, and propose an explanation for why it is hard to escape the neural manifolds created by such attractors.In Paper II, we analyze experimental data from dorsomedial striatum collected using 1-photon calcium imaging of transgenic mice with celltype-specific markers for the striatal direct, indirect and patch pathways, as the mice were gathering rewards in a 2-choice task. In line with extensive previous results, our data analysis revealed a number of neural signatures of reinforcement learning, but no apparent difference between the pathways.In Paper III, we present a new software tool for tracking neurons across weeks of 1-photon calcium imaging, and employ it to follow patch-specific striatal projection neurons from the dorsomedial striatum across two weeks of daily recordings.In Paper IV, we propose a model for how the nigrostriatal dopaminergic projection could, in a biologically plausible way, convey a vector-valued error gradient to the dorsal striatum, as required for backpropagation.Based on the results of the papers and a review of existing literature, I argue that while the basal ganglia indeed make up a circuit for reinforcement learning as previously thought, this circuit represents reinforcement learning states, actions and policies using a continuous population code and not using discrete symbols.
|
|
| 2. |
- Lindahl, Mikael
(författare)
-
Computational Dissection of the Basal Ganglia : functions and dynamics in health and disease
- 2016
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- The basal ganglia (BG), a group of nuclei in the forebrain of all vertebrates, are important for behavioral selection. BG receive contextual input from most cortical areas as well as from parts of the thalamus, and provide output to brain systems that are involved in the generation of behavior, i.e. the thalamus and the brain stem. Many neurological disorders such as Parkinson’s disease and Huntington’s disease, and several neuropsychiatric disorders, are related to BG. Studying BG enhances the understanding as to how behaviors are learned and modified. These insights can be used to improve treatments for several BG disorders, and to develop brain-inspired algorithms for solving special information-processing tasks. In this thesis modeling and simulations have been used to investigate function and dynamics of BG. In the first project a model was developed to explore a new hypothesis about how conflicts between competing actions are resolved in BG. It was proposed that a subsystem named the arbitration system, composed of the subthalamic nucleus (STN), pedunculopontine nucleus (PPN), the brain stem, central medial nucleus of thalamus (CM), globus pallidus interna (GPi) and globus pallidus externa (GPe), resolve basic conflicts between alternative motor programs. On top of the arbitration system there is a second subsystem named the extension systems, which involves the direct and indirect pathway of the striatum. This system can modify the output of the arbitration system to bias action selection towards outcomes dependent on contextual information. In the second project a model framework was developed in two steps, with the aim to gain a deeper understanding of how synapse dynamics, connectivity and neural excitability in the BG relate to function and dynamics in health and disease. First a spiking model of STN, GPe and substantia nigra pars reticulata (SNr), with emulated inputs from striatal medium spiny neurons (MSNs) and the cortex, was built and used to study how synaptic short-term plasticity affected action selection signaling in the direct-, hyperdirect- and indirect pathways. It was found that the functional consequences of facilitatory synapses onto SNr neurons are substantial, and only a few presynaptic MSNs can suppress postsynaptic SNr neurons. The model also predicted that STN signaling in SNr is mainly effective in a transient manner. The model was later extended with a striatal network, containing MSNs and fast spiking interneurons (FSNs), and modified to represent GPe with two types of neurons: type I, which projects downstream in BG, and type A, which have a back-projection to striatum. Furthermore, dopamine depletion dependent modification of connectivity and neuron excitability were added to the model. Using this extended BG model, it was found that FSNs and GPe type A neurons controlled excitability of striatal neurons during low cortical drive, whereas MSN collaterals have a greater impact at higher cortical drive. The indirect pathway increased the dynamical range over which two possible action commands were competing, while removing intrastriatal inhibition deteriorated action selection capabilities. Dopamine-depletion induced effects on spike synchronization and oscillations in the BG were also investigated here. For the final project, an abstract spiking BG model which included a hypothesized control of the reward signaling dopamine system was developed. This model incorporated dopamine-dependent synaptic plasticity, and used a plasticity rule based on probabilistic inference called Bayesian Confidence Propagation Neural Network (BCPNN). In this paradigm synaptic connections were controlled by gathering statistics about neural input and output activity. Synaptic weights were inferred using Bayes’ rule to estimate the confidence of future observations from the input. The model exhibits successful performance, measured as a moving average of correct selected actions, in a multiple-choice learning task with a changeable reward schedule. Furthermore, the model predicts a decreased performance upon dopamine lesioning, and suggests that removing the indirect pathway may disrupt learning in profound ways.
|
|
