| 1. |
- Bagheri, Zahra M., et al.
(författare)
-
An autonomous robot inspired by insect neurophysiology pursues moving features in natural environments
- 2017
-
Ingår i: Journal of Neural Engineering. - : IOP Publishing. - 1741-2560 .- 1741-2552. ; 14:4
-
Tidskriftsartikel (refereegranskat)abstract
- Objective. Many computer vision and robotic applications require the implementation of robust and efficient target-tracking algorithms on a moving platform. However, deployment of a real-time system is challenging, even with the computational power of modern hardware. Lightweight and low-powered flying insects, such as dragonflies, track prey or conspecifics within cluttered natural environments, illustrating an efficient biological solution to the target-tracking problem. Approach. We used our recent recordings from 'small target motion detector' neurons in the dragonfly brain to inspire the development of a closed-loop target detection and tracking algorithm. This model exploits facilitation, a slow build-up of response to targets which move along long, continuous trajectories, as seen in our electrophysiological data. To test performance in real-world conditions, we implemented this model on a robotic platform that uses active pursuit strategies based on insect behaviour. Main results. Our robot performs robustly in closed-loop pursuit of targets, despite a range of challenging conditions used in our experiments; low contrast targets, heavily cluttered environments and the presence of distracters. We show that the facilitation stage boosts responses to targets moving along continuous trajectories, improving contrast sensitivity and detection of small moving targets against textured backgrounds. Moreover, the temporal properties of facilitation play a useful role in handling vibration of the robotic platform. We also show that the adoption of feed-forward models which predict the sensory consequences of self-movement can significantly improve target detection during saccadic movements. Significance. Our results provide insight into the neuronal mechanisms that underlie biological target detection and selection (from a moving platform), as well as highlight the effectiveness of our bio-inspired algorithm in an artificial visual system.
|
|
| 2. |
- Bagheri, Zahra M, et al.
(författare)
-
Performance of an insect-inspired target tracker in natural conditions
- 2017
-
Ingår i: Bioinspiration and Biomimetics. - : IOP Publishing. - 1748-3182 .- 1748-3190. ; 12:2
-
Tidskriftsartikel (refereegranskat)abstract
- Robust and efficient target-tracking algorithms embedded on moving platforms, are a requirement for many computer vision and robotic applications. However, deployment of a real-time system is challenging, even with the computational power of modern hardware. As inspiration, we look to biological lightweight solutions-lightweight and low-powered flying insects. For example, dragonflies pursue prey and mates within cluttered, natural environments, deftly selecting their target amidst swarms. In our laboratory, we study the physiology and morphology of dragonfly 'small target motion detector' neurons likely to underlie this pursuit behaviour. Here we describe our insect-inspired tracking model derived from these data and compare its efficacy and efficiency with state-of-the-art engineering models. For model inputs, we use both publicly available video sequences, as well as our own task-specific dataset (small targets embedded within natural scenes). In the context of the tracking problem, we describe differences in object statistics within the video sequences. For the general dataset, our model often locks on to small components of larger objects, tracking these moving features. When input imagery includes small moving targets, for which our highly nonlinear filtering is matched, the robustness outperforms state-of-the-art trackers. In all scenarios, our insect-inspired tracker runs at least twice the speed of the comparison algorithms.
|
|
| 3. |
- Bagheri, Zahra M, et al.
(författare)
-
Properties of neuronal facilitation that improve target tracking in natural pursuit simulations.
- 2015
-
Ingår i: Journal of the Royal Society Interface. - : The Royal Society. - 1742-5662 .- 1742-5689. ; 12:108
-
Tidskriftsartikel (refereegranskat)abstract
- Although flying insects have limited visual acuity (approx. 1°) and relatively small brains, many species pursue tiny targets against cluttered backgrounds with high success. Our previous computational model, inspired by electrophysiological recordings from insect 'small target motion detector' (STMD) neurons, did not account for several key properties described from the biological system. These include the recent observations of response 'facilitation' (a slow build-up of response to targets that move on long, continuous trajectories) and 'selective attention', a competitive mechanism that selects one target from alternatives. Here, we present an elaborated STMD-inspired model, implemented in a closed loop target-tracking system that uses an active saccadic gaze fixation strategy inspired by insect pursuit. We test this system against heavily cluttered natural scenes. Inclusion of facilitation not only substantially improves success for even short-duration pursuits, but it also enhances the ability to 'attend' to one target in the presence of distracters. Our model predicts optimal facilitation parameters that are static in space and dynamic in time, changing with respect to the amount of background clutter and the intended purpose of the pursuit. Our results provide insights into insect neurophysiology and show the potential of this algorithm for implementation in artificial visual systems and robotic applications.
|
|
| 4. |
- Bagheri, Zahra, et al.
(författare)
-
Performance assessment of an insect-inspired target tracking model in background clutter
- 2014
-
Ingår i: 2014 13th International Conference on Control Automation Robotics and Vision, ICARCV 2014. - 9781479951994 ; , s. 822-826
-
Konferensbidrag (refereegranskat)abstract
- Biological visual systems provide excellent examples of robust target detection and tracking mechanisms capable of performing in a wide range of environments. Consequently, they have been sources of inspiration for many artificial vision algorithms. However, testing the robustness of target detection and tracking algorithms is a challenging task due to the diversity of environments for applications of these algorithms. Correlation between image quality metrics and model performance is one way to deal with this problem. Previously we developed a target detection model inspired by physiology of insects and implemented it in a closed loop target tracking algorithm. In the current paper we vary the kinetics of a salience-enhancing element of our algorithm and test its effect on the robustness of our model against different natural images to find the relationship between model performance and background clutter.
|
|
| 5. |
- Bagheri, Zahra, et al.
(författare)
-
Robustness and real-time performance of an insect inspired target tracking algorithm under natural conditions
- 2015
-
Ingår i: Proceedings - 2015 IEEE Symposium Series on Computational Intelligence, SSCI 2015. - 9781479975600 ; , s. 97-102
-
Konferensbidrag (refereegranskat)abstract
- Many computer vision tasks require the implementation of robust and efficient target tracking algorithms. Furthermore, in robotic applications these algorithms must perform whilst on a moving platform (ego motion). Despite the increase in computational processing power, many engineering algorithms are still challenged by real-Time applications. In contrast, lightweight and low-power flying insects, such as dragonflies, can readily chase prey and mates within cluttered natural environments, deftly selecting their target amidst distractors (swarms). In our laboratory, we record from 'target-detecting' neurons in the dragonfly brain that underlie this pursuit behavior. We recently developed a closed-loop target detection and tracking algorithm based on key properties of these neurons. Here we test our insect-inspired tracking model in open-loop against a set of naturalistic sequences and compare its efficacy and efficiency with other state-of-The-Art engineering models. In terms of tracking robustness, our model performs similarly to many of these trackers, yet is at least 3 times more efficient in terms of processing speed.
|
|
| 6. |
- Bekkouche, Bo M.B., et al.
(författare)
-
Modeling Nonlinear Dendritic Processing of Facilitation in a Dragonfly Target-Tracking Neuron
- 2021
-
Ingår i: Frontiers in Neural Circuits. - : Frontiers Media SA. - 1662-5110. ; 15
-
Tidskriftsartikel (refereegranskat)abstract
- Dragonflies are highly skilled and successful aerial predators that are even capable of selectively attending to one target within a swarm. Detection and tracking of prey is likely to be driven by small target motion detector (STMD) neurons identified from several insect groups. Prior work has shown that dragonfly STMD responses are facilitated by targets moving on a continuous path, enhancing the response gain at the present and predicted future location of targets. In this study, we combined detailed morphological data with computational modeling to test whether a combination of dendritic morphology and nonlinear properties of NMDA receptors could explain these observations. We developed a hybrid computational model of neurons within the dragonfly optic lobe, which integrates numerical and morphological components. The model was able to generate potent facilitation for targets moving on continuous trajectories, including a localized spotlight of maximal sensitivity close to the last seen target location, as also measured during in vivo recordings. The model did not, however, include a mechanism capable of producing a traveling or spreading wave of facilitation. Our data support a strong role for the high dendritic density seen in the dragonfly neuron in enhancing non-linear facilitation. An alternative model based on the morphology of an unrelated type of motion processing neuron from a dipteran fly required more than three times higher synaptic gain in order to elicit similar levels of facilitation, despite having only 20% fewer synapses. Our data support a potential role for NMDA receptors in target tracking and also demonstrate the feasibility of combining biologically plausible dendritic computations with more abstract computational models for basic processing as used in earlier studies.
|
|
| 7. |
- Bekkouche, Bo, et al.
(författare)
-
Multicompartment simulations of NMDA receptor based facilitation in an insect target tracking neuron
- 2017
-
Ingår i: Artificial Neural Networks and Machine Learning – ICANN 2017 - 26th International Conference on Artificial Neural Networks, Proceedings. - Cham : Springer International Publishing. - 0302-9743 .- 1611-3349. - 9783319685991 ; 10613 LNCS, s. 397-404
-
Konferensbidrag (refereegranskat)abstract
- Computational modelling of neurons on different scales provides not only methods to explore mechanisms observed in vivo but also for testing hypotheses that would be impossible physiologically. In this paper we present initial computational analysis of insect lobula small target motion detector (STMD) neurons. We simulate a multicompartment model in combination with a bioinspired model for front-end processing. This combination of different simulation environments enables a combination of scale and detail not possible otherwise. The addressed hypothesis is that facilitation involves N-methyl-D-aspartate (NMDA) synapses which map retinotopically onto the dendritic tree of the STMD neuron. Our results show that a stronger response (facilitation) is generated when using continuous visual stimuli as opposed to random jumps. We observe two levels of facilitation which may be involved in selective attention.
|
|
| 8. |
- Evans, Bernard J.E., et al.
(författare)
-
Differential Tuning to Visual Motion Allows Robust Encoding of Optic Flow in the Dragonfly
- 2019
-
Ingår i: The Journal of Neuroscience : the official journal of the Society for Neuroscience. - 1529-2401. ; 39:41, s. 8051-8063
-
Tidskriftsartikel (refereegranskat)abstract
- Visual cues provide an important means for aerial creatures to ascertain their self-motion through the environment. In many insects, including flies, moths, and bees, wide-field motion-sensitive neurons in the third optic ganglion are thought to underlie such motion encoding; however, these neurons can only respond robustly over limited speed ranges. The task is more complicated for some species of dragonflies that switch between extended periods of hovering flight and fast-moving pursuit of prey and conspecifics, requiring motion detection over a broad range of velocities. Since little is known about motion processing in these insects, we performed intracellular recordings from hawking, emerald dragonflies (Hemicordulia spp.) and identified a diverse group of motion-sensitive neurons that we named lobula tangential cells (LTCs). Following prolonged visual stimulation with drifting gratings, we observed significant differences in both temporal and spatial tuning of LTCs. Cluster analysis of these changes confirmed several groups of LTCs with distinctive spatiotemporal tuning. These differences were associated with variation in velocity tuning in response to translated, natural scenes. LTCs with differences in velocity tuning ranges and optima may underlie how a broad range of motion velocities are encoded. In the hawking dragonfly, changes in LTC tuning over time are therefore likely to support their extensive range of behaviors, from hovering to fast-speed pursuits.SIGNIFICANCE STATEMENT Understanding how animals navigate the world is an inherently difficult and interesting problem. Insects are useful models for understanding neuronal mechanisms underlying these activities, with neurons that encode wide-field motion previously identified in insects, such as flies, hawkmoths, and butterflies. Like some Dipteran flies, dragonflies exhibit complex aerobatic behaviors, such as hovering, patrolling, and aerial combat. However, dragonflies lack halteres that support such diverse behavior in flies. To understand how dragonflies might address this problem using only visual cues, we recorded from their wide-field motion-sensitive neurons. We found these differ strongly in the ways they respond to sustained motion, allowing them collectively to encode the very broad range of velocities experienced during diverse behavior.
|
|
| 9. |
- Evans, Bernard John Essex, et al.
(författare)
-
Dragonfly Neurons Selectively Attend to Targets Within Natural Scenes
- 2022
-
Ingår i: Frontiers in Cellular Neuroscience. - : Frontiers Media SA. - 1662-5102. ; 16
-
Tidskriftsartikel (refereegranskat)abstract
- Aerial predators, such as the dragonfly, determine the position and movement of their prey even when both are moving through complex, natural scenes. This task is likely supported by a group of neurons in the optic lobe which respond to moving targets that subtend less than a few degrees. These Small Target Motion Detector (STMD) neurons are tuned to both target size and velocity, whilst also exhibiting facilitated responses to targets traveling along continuous trajectories. When presented with a pair of targets, some STMDs generate spiking activity that represent a competitive selection of one target, as if the alternative does not exist (i.e., selective attention). Here, we describe intracellular responses of CSTMD1 (an identified STMD) to the visual presentation of targets embedded within cluttered, natural scenes. We examine CSTMD1 response changes to target contrast, as well as a range of target and background velocities. We find that background motion affects CSTMD1 responses via the competitive selection between features within the natural scene. Here, robust discrimination of our artificially embedded “target” is limited to scenarios when its velocity is matched to, or greater than, the background velocity. Additionally, the background’s direction of motion affects discriminability, though not in the manner observed in STMDs of other flying insects. Our results highlight that CSTMD1’s competitive responses are to those features best matched to the neuron’s underlying spatiotemporal tuning, whether from the embedded target or other features in the background clutter. In many scenarios, CSTMD1 responds robustly to targets moving through cluttered scenes. However, whether this neuronal system could underlie the task of competitively selecting slow moving prey against fast-moving backgrounds remains an open question.
|
|
| 10. |
- Evans, Bernard J E, et al.
(författare)
-
Salience invariance with divisive normalization in higher-order insect neurons
- 2016
-
Ingår i: Proceedings of the 2016 6th European Workshop on Visual Information Processing, EUVIP 2016. - 9781509027811
-
Konferensbidrag (refereegranskat)abstract
- We present a biologically inspired model for estimating the position of a moving target that is invariant to the target's contrast. Our model produces a monotonic relationship between position and output activity using a divisive normalization between the 'receptive fields' of two overlapping, wide-field, small-target motion detector (STMD) neurons. These visual neurons found in flying insects, likely underlie the impressive ability to pursue prey within cluttered environments. Individual STMD responses confound the properties of target contrast, size, velocity and position. Inspired by results from STMD recordings we developed a model using a division operation to overcome the inherent positional ambiguities of integrative neurons. We used genetic algorithms to determine the plausibility of such an operation arising and existing over multiple generations. This method allows the lost information to be recovered without needing additional neuronal pathways.
|
|
