| 1. |
- Saifullah, Mohammad, 1973-
(författare)
-
Biologically-Based Interactive Neural Network Models for Visual Attention and Object Recognition
- 2012
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- The main focus of this thesis is to develop biologically-based computational models for object recognition. A series of models for attention and object recognition were developed in the order of increasing functionality and complexity. These models are based on information processing in the primate brain, and specially inspired from the theory of visual information processing along the two parallel processing pathways of the primate visual cortex. To capture the true essence of incremental, constraint satisfaction style processing in the visual system, interactive neural networks were used for implementing our models. Results from eye-tracking studies on the relevant visual tasks, as well as our hypothesis regarding the information processing in the primate visual system, were implemented in the models and tested with simulations.As a first step, a model based on the ventral pathway was developed to recognize single objects. Through systematic testing, structural and algorithmic parameters of these models were fine tuned for performing their task optimally. In the second step, the model was extended by considering the dorsal pathway, which enables simulation of visual attention as an emergent phenomenon. The extended model was then investigated for visual search tasks. In the last step, we focussed on occluded and overlapped object recognition. A couple of eye-tracking studies were conducted in this regard and on the basis of the results we made some hypotheses regarding information processing in the primate visual system. The models were further advanced on the lines of the presented hypothesis, and simulated on the tasks of occluded and overlapped object recognition.On the basis of the results and analysis of our simulations we have further found that the generalization performance of interactive hierarchical networks improves with the addition of a small amount of Hebbian learning to an otherwise pure error-driven learning. We also concluded that the size of the receptive fields in our networks is an important parameter for the generalization task and depends on the object of interest in the image. Our results show that networks using hard coded feature extraction perform better than the networks that use Hebbian learning for developing feature detectors. We have successfully demonstrated the emergence of visual attention within an interactive network and also the role of context in the search task. Simulation results with occluded and overlapped objects support our extended interactive processing approach, which is a combination of the interactive and top-down approach, to the segmentation-recognition issue. Furthermore, the simulation behavior of our models is in line with known human behavior for similar tasks.In general, the work in this thesis will improve the understanding and performance of biologically-based interactive networks for object recognition and provide a biologically-plausible solution to recognition of occluded and overlapped objects. Moreover, our models provide some suggestions for the underlying neural mechanism and strategies behind biological object recognition.
|
|
| 2. |
- Stathis, Dimitrios, 1989-
(författare)
-
Synchoros VLSI Design Style
- 2022
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Computers have become essential to everyday life as much as electricity, communications and transport. That is evident from the amount of electricity we spend to power our computing systems. According to some reports it is estimated to be ≈ 7% of the total consumption worldwide. This trend is very worrisome, and the development of computing systems with lower power consumption is essential. This is even more important for battery-powered computers deployed in the field. The industry and the scientific community have realised that general-purpose computing platforms cannot offer that level of computational efficiency and that customisation is the solution to this problem. Application-Specific Integrated Circuits (ASICs) provide the highest efficiency in the mainstream implementation styles. ASICs have been shown to provide 100 to 1000× better computational efficiency than general-purpose computing platforms. However, the design cost of ASICs restricts it to products that have a large volume or large profit. In essence, to achieve ASIC-like computational efficiency, the design efficiency becomes the bottleneck. SynchorosVLSI design has been proposed to non-incrementally lower the design cost of custom ASIC-like solutions. The synchoros VLSI design is a novel concept that can reduce the design cost of ASICs and their manufacturing. Insynchoros design, the space is discretised, and the final design emerges by the abutment of synchoros micro-architecture level design objects called SiLago(Silicon Lego) blocks. The SiLago framework has the potential to reduce the design cost of ASICs and their manufacturing. This thesis makes three research areas of contributions toward synchoros VLSI design. The first area concerns composition by abutment. In this contribution, a design has been proposed to show how a clock tree can be created by abutting fragments inside the SiLago blocks. Additionally, the clock tree created by abutment was validated by the EDA tools and its cost metrics compared to the functionally equivalent clock tree created by the conventional EDA flows. The second area is to enhance the micro-architectural framework. These contributions include SiLago blocks tailored for neural network computation and architectural enhancements to improve the efficiency of executing streaming applications in the SiLago framework. Furthermore, a novel genome recognition application based on a self-organising map (SOM) was also mapped to the SiLago framework. The third area of contribution is implementing a model of cortex as a tiled ASIC design using custom 3D DRAM vaults for synaptic storage. This work is preparatory work to identify the SiLago blocks needed to support the implementation of spiking neuromorphic structures and in general applications of ordinary differential equations.
|
|
| 3. |
- Steinert, Rebecca
(författare)
-
Probabilistic Fault Management in Networked Systems
- 2014
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Technical advances in network communication systems (e.g. radio access networks) combined with evolving concepts based on virtualization (e.g. clouds), require new management algorithms in order to handle the increasing complexity in the network behavior and variability in the network environment. Current network management operations are primarily centralized and deterministic, and are carried out via automated scripts and manual interventions, which work for mid-sized and fairly static networks. The next generation of communication networks and systems will be of significantly larger size and complexity, and will require scalable and autonomous management algorithms in order to meet operational requirements on reliability, failure resilience, and resource-efficiency.A promising approach to address these challenges includes the development of probabilistic management algorithms, following three main design goals. The first goal relates to all aspects of scalability, ranging from efficient usage of network resources to computational efficiency. The second goal relates to adaptability in maintaining the models up-to-date for the purpose of accurately reflecting the network state. The third goal relates to reliability in the algorithm performance in the sense of improved performance predictability and simplified algorithm control.This thesis is about probabilistic approaches to fault management that follow the concepts of probabilistic network management (PNM). An overview of existing network management algorithms and methods in relation to PNM is provided. The concepts of PNM and the implications of employing PNM-algorithms are presented and discussed. Moreover, some of the practical differences of using a probabilistic fault detection algorithm compared to a deterministic method are investigated. Further, six probabilistic fault management algorithms that implement different aspects of PNM are presented.The algorithms are highly decentralized, adaptive and autonomous, and cover several problem areas, such as probabilistic fault detection and controllable detection performance; distributed and decentralized change detection in modeled link metrics; root-cause analysis in virtual overlays; event-correlation and pattern mining in data logs; and, probabilistic failure diagnosis. The probabilistic models (for a large part based on Bayesian parameter estimation) are memory-efficient and can be used and re-used for multiple purposes, such as performance monitoring, detection, and self-adjustment of the algorithm behavior.
|
|
| 4. |
- Auffarth, Benjamin, 1979-
(författare)
-
Machine Learning Techniques with Specific Application to the Early Olfactory System
- 2012
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- This thesis deals with machine learning techniques for the extraction of structure and the analysis of the vertebrate olfactory pathway based on related methods. Some of its main contributions are summarized below.We have performed a systematic investigation for classification in biomedical images with the goal of recognizing a material in these images by its texture. This investigation included (i) different measures for evaluating the importance of image descriptors (features), (ii) methods to select a feature set based on these evaluations, and (iii) classification algorithms. Image features were evaluated according to their estimated relevance for the classification task and their redundancy with other features. For this purpose, we proposed a framework for relevance and redundancy measures and, within this framework, we proposed two new measures. These were the value difference metric and the fit criterion. Both measures performed well in comparison with other previously used ones for evaluating features. We also proposed a Hopfield network as a method for feature selection, which in experiments gave one of the best results relative to other previously used approaches.We proposed a genetic algorithm for clustering and tested it on several realworld datasets. This genetic algorithm was novel in several ways, including (i) the use of intra-cluster distance as additional optimization criterion, (ii) an annealing procedure, and (iii) adaptation of mutation rates. As opposed to many conventional clustering algorithms, our optimization framework allowed us to use different cluster validation measures including those which do not rely on cluster centroids. We demonstrated the use of the clustering algorithm experimentally with several cluster validity measures as optimization criteria. We compared the performance of our clustering algorithm to that of the often-used fuzzy c-means algorithm on several standard machine learning datasets from the University of California/Urvine (UCI) and obtained good results.The organization of representations in the brain has been observed at several stages of processing to spatially decompose input from the environment into features that are somehow relevant from a behavioral or perceptual standpoint. For the perception of smells, the analysis of such an organization, however, is not as straightforward because of the missing metric. Some studies report spatial clusters for several combinations of physico-chemical properties in the olfactory bulb at the level of the glomeruli. We performed a systematic study of representations based on a dataset of activity-related images comprising more than 350 odorants and covering the whole spatial array of the first synaptic level in the olfactory system. We found clustered representations for several physico-chemical properties. We compared the relevance of these properties to activations and estimated the size of the coding zones. The results confirmed and extended previous studies on olfactory coding for physico-chemical properties. Particularly of interest was the spatial progression by carbon chain that we found. We discussed our estimates of relevance and coding size in the context of processing strategies. We think that the results obtained in this study could guide the search into olfactory coding primitives and the understanding of the stimulus space.In a second study on representations in the olfactory bulb, we grouped odorants together by perceptual categories, such as floral and fruity. By the application of the same statistical methods as in the previous study, we found clustered zones for these categories. Furthermore, we found that distances between spatial representations were related to perceptual differences in humans as reported in the literature. This was possibly the first time that such an analysis had been done. Apart from pointing towards a spatial decomposition by perceptual dimensions, results indicate that distance relationships between representations could be perceptually meaningful.In a third study, we modeled axon convergence from olfactory receptor neurons to the olfactory bulb. Sensory neurons were stimulated by a set of biologically-relevant odors, which were described by a set of physico-chemical properties that covaried with the neural and glomerular population activity in the olfactory bulb. Convergence was mediated by the covariance between olfactory neurons. In our model, we could replicate the formation of glomeruli and concentration coding as reported in the literature, and further, we found that the spatial relationships between representational zones resulting from our model correlated with reported perceptual differences between odor categories. This shows that natural statistics, including similarity of physico-chemical structure of odorants, can give rise to an ordered arrangement of representations at the olfactory bulb level where the distances between representations are perceptually relevant.
|
|
| 5. |
- Berthet, Pierre, 1982-
(författare)
-
Computational Modeling of the Basal Ganglia : Functional Pathways and Reinforcement Learning
- 2015
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- We perceive the environment via sensor arrays and interact with it through motor outputs. The work of this thesis concerns how the brain selects actions given the information about the perceived state of the world and how it learns and adapts these selections to changes in this environment. Reinforcement learning theories suggest that an action will be more or less likely to be selected if the outcome has been better or worse than expected. A group of subcortical structures, the basal ganglia (BG), is critically involved in both the selection and the reward prediction.We developed and investigated a computational model of the BG. We implemented a Bayesian-Hebbian learning rule, which computes the weights between two units based on the probability of their activations. We were able test how various configurations of the represented pathways impacted the performance in several reinforcement learning and conditioning tasks. Then, following the development of a more biologically plausible version with spiking neurons, we simulated lesions in the different pathways and assessed how they affected learning and selection.We observed that the evolution of the weights and the performance of the models resembled qualitatively experimental data. The absence of an unique best way to configure the model over all the learning paradigms tested indicates that an agent could dynamically configure its action selection mode, mainly by including or not the reward prediction values in the selection process. We present hypotheses on possible biological substrates for the reward prediction pathway. We base these on the functional requirements for successful learning and on an analysis of the experimental data. We further simulate a loss of dopaminergic neurons similar to that reported in Parkinson’s disease. We suggest that the associated motor symptoms are mostly causedby an impairment of the pathway promoting actions, while the pathway suppressing them seems to remain functional.
|
|
| 6. |
|
|
| 7. |
- Fiebig, Florian
(författare)
-
Active Memory Processing on Multiple Time-scales in Simulated Cortical Networks with Hebbian Plasticity
- 2018
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- This thesis examines declarative memory function, and its underlying neural activity and mechanisms in simulated cortical networks. The included simulation models utilize and synthesize proposed universal computational principles of the brain, such as the modularity of cortical circuit organization, attractor network theory, and Hebbian synaptic plasticity, along with selected biophysical detail from the involved brain areas to implement functional models of known cortical memory systems. The models hypothesize relations between neural activity, brain area interactions, and cognitive memory functions such as sleep-dependent memory consolidation, or specific working memory tasks. In particular, this work addresses the acutely relevant research question if recently described fast forms of Hebbian synaptic plasticity are a possible mechanism behind working memory. The proposed models specifically challenge the “persistent activity hypothesis of working memory”, an established but increasingly questioned paradigm in working memory theory. The proposed alternative is a novel synaptic working memory model that is arguably more defensible than the existing paradigm as it can better explain memory function and important aspects of working memory-linked activity (such as the role of long-term memory in working memory tasks), while simultaneously matching experimental data from behavioral memory testing and important evidence from electrode recordings.
|
|
| 8. |
- Kaplan, Bernhard, 1984-
(författare)
-
Modeling prediction and pattern recognition in the early visual and olfactory systems
- 2015
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Our senses are our mind's window to the outside world and determine how we perceive our environment.Sensory systems are complex multi-level systems that have to solve a multitude of tasks that allow us to understand our surroundings.However, questions on various levels and scales remain to be answered ranging from low-level neural responses to behavioral functions on the highest level.Modeling can connect different scales and contribute towards tackling these questions by giving insights into perceptual processes and interactions between processing stages.In this thesis, numerical simulations of spiking neural networks are used to deal with two essential functions that sensory systems have to solve: pattern recognition and prediction.The focus of this thesis lies on the question as to how neural network connectivity can be used in order to achieve these crucial functions.The guiding ideas of the models presented here are grounded in the probabilistic interpretation of neural signals, Hebbian learning principles and connectionist ideas.The main results are divided into four parts.The first part deals with the problem of pattern recognition in a multi-layer network inspired by the early mammalian olfactory system with biophysically detailed neural components.Learning based on Hebbian-Bayesian principles is used to organize the connectivity between and within areas and is demonstrated in behaviorally relevant tasks.Besides recognition of artificial odor patterns, phenomena like concentration invariance, noise robustness, pattern completion and pattern rivalry are investigated.It is demonstrated that learned recurrent cortical connections play a crucial role in achieving pattern recognition and completion.The second part is concerned with the prediction of moving stimuli in the visual system.The problem of motion-extrapolation is studied using different recurrent connectivity patterns.The main result shows that connectivity patterns taking the tuning properties of cells into account can be advantageous for solving the motion-extrapolation problem.The third part focuses on the predictive or anticipatory response to an approaching stimulus.Inspired by experimental observations, particle filtering and spiking neural network frameworks are used to address the question as to how stimulus information is transported within a motion sensitive network.In particular, the question if speed information is required to build up a trajectory dependent anticipatory response is studied by comparing different network connectivities.Our results suggest that in order to achieve a dependency of the anticipatory response to the trajectory length, a connectivity that uses both position and speed information seems necessary.The fourth part combines the self-organization ideas from the first part with motion perception as studied in the second and third parts.There, the learning principles used in the olfactory system model are applied to the problem of motion anticipation in visual perception.Similarly to the third part, different connectivities are studied with respect to their contribution to anticipate an approaching stimulus.The contribution of this thesis lies in the development and simulation of large-scale computational models of spiking neural networks solving prediction and pattern recognition tasks in biophysically plausible frameworks.
|
|
| 9. |
- Liu, Lizheng, et al.
(författare)
-
A FPGA-based Hardware Accelerator for Bayesian Confidence Propagation Neural Network
- 2020
-
Ingår i: 2020 IEEE Nordic Circuits and Systems Conference, NORCAS 2020 - Proceedings. - : Institute of Electrical and Electronics Engineers (IEEE).
-
Konferensbidrag (refereegranskat)abstract
- The Bayesian Confidence Propagation Neural Network (BCPNN) has been applied in higher level of cognitive intelligence (e.g. working memory, associative memory). However, in the spike-based version of this learning rule the pre-, postsynaptic and coincident activity is traced in three low-passfiltering stages, the calculation processes of weight update are very computationally intensive. In this paper, a hardware architecture of the updating process for lazy update mode is proposed for updating 8 local synaptic state variables. The parallelism by decomposing the calculation steps of formulas based on the inherent data dependencies is optimized. The FPGA-based hardware accelerator of BCPNN is designed and implemented. The experimental results show the updating process on FPGA can be accomplished within 110 ns with a clock frequency of 200 MHz, the updating speed is greatly enhanced compared with the CPU test. The trade-off between performance, accuracy and resources on dedicated hardware is evaluated, and the impact of the module reuse on resource consumption and computing performance is evaluated.
|
|
| 10. |
- Tully, Philip, 1988-
(författare)
-
Spike-Based Bayesian-Hebbian Learning in Cortical and Subcortical Microcircuits
- 2017
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Cortical and subcortical microcircuits are continuously modified throughout life. Despite ongoing changes these networks stubbornly maintain their functions, which persist although destabilizing synaptic and nonsynaptic mechanisms should ostensibly propel them towards runaway excitation or quiescence. What dynamical phenomena exist to act together to balance such learning with information processing? What types of activity patternsdo they underpin, and how do these patterns relate to our perceptual experiences? What enables learning and memory operations to occur despite such massive and constant neural reorganization? Progress towards answering many of these questions can be pursued through large-scale neuronal simulations. In this thesis, a Hebbian learning rule for spiking neurons inspired by statistical inference is introduced. The spike-based version of the Bayesian Confidence Propagation Neural Network (BCPNN) learning rule involves changes in both synaptic strengths and intrinsic neuronal currents. The model is motivated by molecular cascades whose functional outcomes are mapped onto biological mechanisms such as Hebbian and homeostatic plasticity, neuromodulation, and intrinsic excitability. Temporally interacting memory traces enable spike-timing dependence, a stable learning regime that remains competitive, postsynaptic activity regulation, spike-based reinforcement learning and intrinsic graded persistent firing levels. The thesis seeks to demonstrate how multiple interacting plasticity mechanisms can coordinate reinforcement, auto- and hetero-associative learning within large-scale, spiking, plastic neuronal networks. Spiking neural networks can represent information in the form of probability distributions, and a biophysical realization of Bayesian computation can help reconcile disparate experimental observations.
|
|
