Simulating Human Walking: A Model-BasedReinforcement Learning Approach with musculoskeletal Modelling

• Recent advancements in reinforcement learning algorithms have accelerated the development of control models with high-dimensional inputs and outputs that can reproduce human movement. However, the produced motion tends to be less human-like if algorithms do not involve a biomechanical human model that accounts for skeletal and muscle-tendon properties and geometry. In this study, we have integrated a reinforcement learning algorithm and a musculoskeletal model including trunk, pelvis, and leg segments to develop control modes that drive the model to walk. We simulated human walking first without imposing target walking speed, in which the model was allowed to settle on a stable walking speed itself. A range of other speeds were imposed for the simulation based on the previous self-developed walking speed. All simulations were generated by solving the Markov decision process problem with covariance matrix adaptation evolution strategy, without any reference motion data. We finally demonstrated that our reinforcement learning framework also has the potential to model and predict pathological gait that can result from muscle weakness. 

Ämnesord

TEKNIK OCH TEKNOLOGIER  -- Maskinteknik -- Annan maskinteknik (hsv//swe)

ENGINEERING AND TECHNOLOGY  -- Mechanical Engineering -- Other Mechanical Engineering (hsv//eng)

Nyckelord

Reinforcement learning

human and humanoid motion analysis and synthesis

optimization and optimal control

kinematics.

Publikations- och innehållstyp

vet (ämneskategori)

ovr (ämneskategori)