| 1. |
- Li, Cai, et al.
(author)
-
A novel approach to locomotion learning: Actor-Critic architecture using central pattern generators and dynamic motor primitives
- 2014
-
In: Frontiers in Neurorobotics. - : Frontiers. - 1662-5218. ; 8
-
Journal article (peer-reviewed)abstract
- In this article, we propose an architecture of a bio-inspired controller that addresses the problem of learning different locomotion gaits for different robot morphologies. The modeling objective is split into two: baseline motion modeling and dynamics adaptation. Baseline motion modeling aims to achieve fundamental functions of a certain type of locomotion and dynamics adaptation provides a "reshaping" function for adapting the baseline motion to desired motion. Based on this assumption, a three-layer architecture is developed using central pattern generators (CPGs, a bio-inspired locomotor center for the baseline motion) and dynamic motor primitives (DMPs, a model with universal "reshaping" functions). In this article, we use this architecture with the actor-critic algorithms for finding a good "reshaping" function. In order to demonstrate the learning power of the actor-critic based architecture, we tested it on two experiments: (1) learning to crawl on a humanoid and, (2) learning to gallop on a puppy robot. Two types of actor-critic algorithms (policy search and policy gradient) are compared in order to evaluate the advantages and disadvantages of different actor-critic based learning algorithms for different morphologies. Finally, based on the analysis of the experimental results, a generic view/architecture for locomotion learning is discussed in the conclusion.
|
|
| 2. |
- Li, Cai, et al.
(author)
-
Humanoids learning to walk : a natural CPG-actor-critic architecture
- 2013
-
In: Frontiers in Neurorobotics. - : Frontiers Media S.A.. - 1662-5218. ; 7:5
-
Journal article (peer-reviewed)abstract
- The identification of learning mechanisms for locomotion has been the subject of much research for some time but many challenges remain. Dynamic systems theory (DST) offers a novel approach to humanoid learning through environmental interaction. Reinforcement learning (RL) has offered a promising method to adaptively link the dynamic system to the environment it interacts with via a reward-based value system. In this paper, we propose a model that integrates the above perspectives and applies it to the case of a humanoid (NAO) robot learning to walk the ability of which emerges from its value-based interaction with the environment. In the model, a simplified central pattern generator (CPG) architecture inspired by neuroscientific research and DST is integrated with an actor-critic approach to RL (cpg-actor-critic). In the cpg-actor-critic architecture, least-square-temporal-difference based learning converges to the optimal solution quickly by using natural gradient learning and balancing exploration and exploitation. Futhermore, rather than using a traditional (designer-specified) reward it uses a dynamic value function as a stability indicator that adapts to the environment. The results obtained are analyzed using a novel DST-based embodied cognition approach. Learning to walk, from this perspective, is a process of integrating levels of sensorimotor activity and value.
|
|
| 3. |
- Li, Cai, et al.
(author)
-
Humanoids that crawl : Comparing gait performance of iCub and NAO using a CPG architecture
- 2011
-
In: Proceedings - 2011 IEEE International Conference on Computer Science and Automation Engineering, CSAE 2011. - : IEEE conference proceedings. - 9781424487271 - 9781424487288 - 9781424487264 - 9781424487257 ; , s. 577-582
-
Conference paper (peer-reviewed)abstract
- In this article, a generic CPG architecture is used to model infant crawling gaits and is implemented on the NAO robot platform. The CPG architecture is chosen via a systematic approach to designing CPG networks on the basis of group theory and dynamic systems theory. The NAO robot performance is compared to the iCub robot which has a different anatomical structure. Finally, the comparison of performance and NAO whole-body stability are assessed to show the adaptive property of the CPG architecture and the extent of its ability to transfer to different robot morphologies. © 2011 IEEE.
|
|
| 4. |
- Li, Cai, et al.
(author)
-
Modelling Walking Behaviors Based on CPGs : A Simplified Bio-inspired Architecture
- 2012
-
In: From Animals to Animats 12. - Berlin; Heidelberg : Springer Berlin/Heidelberg. - 9783642330926 - 9783642330933 ; , s. 156-166
-
Conference paper (peer-reviewed)abstract
- In this article, we use a recurrent neural network including four-cell core architecture to model the walking gait and implement it with the simulated and physical NAO robot. Meanwhile, inspired by the biological CPG models, we propose a simplified CPG model which comprises motorneurons, interneurons, sensor neurons and the simplified spinal cord. Within this model, the CPGs do not directly output trajectories to the servo motors. Instead, they only work to maintain the phase relation among ipsilateral and contralateral limbs. The final output is dependent on the integration of CPG signals, outputs of interneurons, motor neurons and sensor neurons (sensory feedback).
|
|
