| 1. |
- Su, Binbin, et al.
(författare)
-
Gait Phase Recognition Using Deep Convolutional Neural Network with Inertial Measurement Units
- 2020
-
Ingår i: Biosensors. - : MDPI AG. - 2079-6374. ; 10:9, s. 109-109
-
Tidskriftsartikel (refereegranskat)abstract
- Gait phase recognition is of great importance in the development of assistance-as-needed robotic devices, such as exoskeletons. In order for a powered exoskeleton with phase-based control to determine and provide proper assistance to the wearer during gait, the user’s current gait phase must first be identified accurately. Gait phase recognition can potentially be achieved through input from wearable sensors. Deep convolutional neural networks (DCNN) is a machine learning approach that is widely used in image recognition. User kinematics, measured from inertial measurement unit(IMU) output, can be considered as an ‘image’ since it exhibits some local ‘spatial’ pattern when the sensor data is arranged in sequence. We propose a specialized DCNN to distinguish five phases in a gait cycle, based on IMU data and classified with foot switch information. The DCNN showed approximately 97% accuracy during an offline evaluation of gait phase recognition. Accuracy was highest in the swing phase and lowest in terminal stance.
|
|
| 2. |
- Su, Binbin, et al.
(författare)
-
Gait Trajectory and Gait Phase Prediction Based on an LSTM Network
- 2020
-
Ingår i: Sensors. - : MDPI AG. - 1424-8220. ; 20:24, s. 7127-7127
-
Tidskriftsartikel (refereegranskat)abstract
- Lower body segment trajectory and gait phase prediction is crucial for the control of assistance-as-needed robotic devices, such as exoskeletons. In order for a powered exoskeleton with phase-based control to determine and provide proper assistance to the wearer during gait, we propose an approach to predict segment trajectories up to 200 ms ahead (angular velocity of the thigh, shank, and foot segments) and five gait phases (loading response, mid-stance, terminal stance, preswing, and swing), based on collected data from inertial measurement units placed on the thighs, shanks, and feet. The approach we propose is a long-short term memory (LSTM)-based network, a modified version of recurrent neural networks, which can learn order dependence in sequence prediction problems. The algorithm proposed has a weighted discount loss function that places more weight in predicting the next three to five time frames but also contributes to an overall prediction performance for up to 10 time frames. The LSTM model was designed to learn lower limb segment trajectories using training samples and was tested for generalization across participants. All predicted trajectories were strongly correlated with the measured trajectories, with correlation coefficients greater than 0.98. The proposed LSTM approach can also accurately predict the five gait phases, particularly the swing phase with 95% accuracy in inter-subject implementation. The ability of the LSTMnetwork to predict future gait trajectories and gait phases can be applied in designing exoskeleton controllers that can better compensate for system delays to smooth the transition between gait phases.
|
|
| 3. |
- Su, Binbin
(författare)
-
Human motion prediction using wearable sensors and machine Learning
- 2021
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Accurately measuring and predicting human movement is important in many contexts, such as in rehabilitation and the design of assistive devices. Thanks to the development and availability of a wide variety of sensors, scientists study human movement in many settings and capture characteristic properties unique to individuals as well as to larger study populations. Inertial measurement units (IMU), which contain accelerometers and gyroscopes, measure segment accelerations and angular velocities, and electromyography (EMG) sensors measure muscle excitation. These types of wearable sensors can be donned at the same time and can record data at a high frequency, potentially resulting in a large amount of data. Machine learning (ML) is an effective tool to extract the prominent features and make statistical inferences from data and has the potential to enhance human motion analyses through data-driven prediction. The overall aim of this thesis was to predict human motion through data-driven approaches and musculoskeletal simulations using wearable sensors and ML. A deterministic machine learning approach using a convolutional neural network (CNN) was first proposed to segment gait cycles into five phases based on experimental IMU data in subjects at different walking speeds. The proposed CNN was able to capture kinematic characteristics in raw IMU data, such as linear acceleration, rotational velocity, and magnetic field, and distinguish different gait phases. In recognizing all gait phases, it achieved an overall accuracy of 97.5% on a well-trained model, with up to 99.6% accuracy in detecting the swing phase. Our results also showed walking speed did not have a major influence on the overall gait phase recognition accuracy for people with typical gait patterns. However, while the swing was most accurately recognized, the terminal stance was least accurately recognized, and even more so at lower walking speeds.We then developed a long short-term (LSTM) network to predict both gait phase (loading response, midstance, terminal stance, pre-swing, and swing) and gait trajectory (angular velocities of thigh, shank, and foot segments) in up to the subsequent 200ms, based on immediately prior data. The overall accuracy of gait phase prediction was up to 94%, with the swing phase the most accurately predicted (97%). Our results also showed a high correlation between predicted and true values of the angular velocity of the thigh, shank, and foot segments. People walk on different terrains daily, for instance, level-ground walking, ramp/stairs ascent/descent, and stepping over obstacles. Movements patterns change as people move from one terrain to another, i.e. transition from one locomotion mode to another. Locomotion modes are typically labeled between two gait events, foot contact (FC) and toe off (TO). Since there is no exact instance for discriminating the transition between two locomotion modes, we identified TO as the critical gait event. We integrated locomotion mode prediction and gait event identification into one machine learning framework comprising two multilayer perceptrons (MLP), using fused data from two types of wearable sensors, namely EMG sensors and IMUs. The first MLP successfully identified FC and TO; FC events were identified accurately, and a small number of misclassifications only occurred near TO events. A small time difference (2.5 and -5.3 ms for FC and TO respectively) was found between predicted and true gait events. The second MLP correctly identified walking, ramp ascent, and ramp descent transitions with the best aggregate accuracy of 96.3%, 90.1%, and 90.6%, respectively, with sufficient prediction time before the critical events, using EMG and IMU signals as input features.Data-driven approaches using wearable sensors are incapable of modeling the mechanism between neuromuscular control and wearable sensor outputs. Musculoskeletal simulation can, on the other hand, explain the interactions between muscular control, kinematics, and kinetics in human motion. Thus, we integrated a reinforcement learning algorithm, a reflex-based controller, and a musculoskeletal model including trunk, pelvis, and leg segments to simulate reasonably realistic human walking at different speeds. We further generated pathological gaits that may result from ankle plantarflexor weakness using the same approach. The simulated hip and knee angles correlate reasonably well with reported experimental data, though less so for ankle kinematics. The computed muscle excitations in major low limb muscles largely correspond to the expected on-off timing of these muscles during walking. In summary, the studies in this thesis describe and predict human movement with wearable sensors and machine learning algorithms. We detected and predicted gait phases and events, predicted segment movements and identified intended transitions between walking modes during the stance phase of the previous gait cycle on the same side, before the step into the new mode, all based on data from wearable sensors. This has important potential implications in continuous monitoring and analysis of a person's movements outside a lab environment. The musculoskeletal simulation provided insight into the relationship between neuromuscular control and sensory feedback, which could also be applied to better understand and predict likely changes in gait when changes occur in neuromuscular control. Our approaches combining wearable sensors and machine learning could be ultimately applied to facilitate the design of exoskeletons that can provide seamless assistance for people with motor disorders.
|
|
| 4. |
- Su, Binbin, et al.
(författare)
-
Locomotion mode transition prediction based on gait event identification using wearable sensors and multilayer perceptrons
-
Annan publikation (övrigt vetenskapligt/konstnärligt)abstract
- People walk on different terrains daily, for instance, level-ground walking, ramp and stair ascent and descent, and stepping over obstacles are common activities in daily life. Movementspatterns change as people move from one terrain to another. Prediction of transitions between locomotion modes is important for developing assistive devices such as exoskeletons, as optimal assistive strategies may differ for different locomotion modes. Prediction of locomotion mode transitions is often accompanied by gait event detection that provides important information during locomotion about critical events, such as foot contact (FC) and toe off (TO). In this study, we introduce a method to integrate locomotion mode prediction and gait event identification into one machine learning framework, comprised of two multilayer perceptrons (MLP). Input features to the framework were from fused data from wearable sensors, specifically, electromyography sensors and inertial measurement units. The first MLP successfully identified FC and TO; FC events were identified accurately, and a small number of misclassifications only occurred near TO events. A small time difference (2.5 ms and -5.3 ms for FC and TO respectively) was found between predicted and true gait events. The second MLP correctly identified walking, ramp ascent, and ramp descent transitions with the best aggregate accuracy of 96.3%, 90.1%, and 90.6%, respectively, with sufficient prediction time prior to the critical events. The models in this study demonstrate high accuracy in predicting transitions between different locomotion modes in the same side’s mid-to late stance of the stride prior to the step into the new model using data from EMG and IMUs sensors. Our results may help assistive devices achieve smooth and seamless transitions in different locomotion modes for the person with motor disorders.
|
|
| 5. |
- Su, Binbin, et al.
(författare)
-
Locomotion Mode Transition Prediction Based on Gait-Event Identification Using Wearable Sensors and Multilayer Perceptrons
- 2021
-
Ingår i: Sensors. - : MDPI AG. - 1424-8220. ; 21:22, s. 7473-
-
Tidskriftsartikel (refereegranskat)abstract
- People walk on different types of terrain daily; for instance, level-ground walking, ramp and stair ascent and descent, and stepping over obstacles are common activities in daily life. Movement patterns change as people move from one terrain to another. The prediction of transitions between locomotion modes is important for developing assistive devices, such as exoskeletons, as the optimal assistive strategies may differ for different locomotion modes. The prediction of locomotion mode transitions is often accompanied by gait-event detection that provides important information during locomotion about critical events, such as foot contact (FC) and toe off (TO). In this study, we introduce a method to integrate locomotion mode prediction and gait-event identification into one machine learning framework, comprised of two multilayer perceptrons (MLP). Input features to the framework were from fused data from wearable sensors-specifically, electromyography sensors and inertial measurement units. The first MLP successfully identified FC and TO, FC events were identified accurately, and a small number of misclassifications only occurred near TO events. A small time difference (2.5 ms and -5.3 ms for FC and TO, respectively) was found between predicted and true gait events. The second MLP correctly identified walking, ramp ascent, and ramp descent transitions with the best aggregate accuracy of 96.3%, 90.1%, and 90.6%, respectively, with sufficient prediction time prior to the critical events. The models in this study demonstrate high accuracy in predicting transitions between different locomotion modes in the same side's mid- to late stance of the stride prior to the step into the new mode using data from EMG and IMU sensors. Our results may help assistive devices achieve smooth and seamless transitions in different locomotion modes for those with motor disorders.
|
|
| 6. |
- Su, Binbin, et al.
(författare)
-
Simulating human walking : a model-based reinforcement learning approach with musculoskeletal modeling
- 2023
-
Ingår i: Frontiers in Neurorobotics. - : Frontiers Media SA. - 1662-5218. ; 17
-
Tidskriftsartikel (refereegranskat)abstract
- IntroductionRecent advancements in reinforcement learning algorithms have accelerated the development of control models with high-dimensional inputs and outputs that can reproduce human movement. However, the produced motion tends to be less human-like if algorithms do not involve a biomechanical human model that accounts for skeletal and muscle-tendon properties and geometry. In this study, we have integrated a reinforcement learning algorithm and a musculoskeletal model including trunk, pelvis, and leg segments to develop control modes that drive the model to walk.MethodsWe simulated human walking first without imposing target walking speed, in which the model was allowed to settle on a stable walking speed itself, which was 1.45 m/s. A range of other speeds were imposed for the simulation based on the previous self-developed walking speed. All simulations were generated by solving the Markov decision process problem with covariance matrix adaptation evolution strategy, without any reference motion data.ResultsSimulated hip and knee kinematics agreed well with those in experimental observations, but ankle kinematics were less well-predicted.DiscussionWe finally demonstrated that our reinforcement learning framework also has the potential to model and predict pathological gait that can result from muscle weakness.
|
|
| 7. |
- Su, Binbin, et al.
(författare)
-
Simulating Human Walking: A Model-BasedReinforcement Learning Approach with musculoskeletal Modelling
-
Annan publikation (övrigt vetenskapligt/konstnärligt)abstract
- Recent advancements in reinforcement learning algorithms have accelerated the development of control models with high-dimensional inputs and outputs that can reproduce human movement. However, the produced motion tends to be less human-like if algorithms do not involve a biomechanical human model that accounts for skeletal and muscle-tendon properties and geometry. In this study, we have integrated a reinforcement learning algorithm and a musculoskeletal model including trunk, pelvis, and leg segments to develop control modes that drive the model to walk. We simulated human walking first without imposing target walking speed, in which the model was allowed to settle on a stable walking speed itself. A range of other speeds were imposed for the simulation based on the previous self-developed walking speed. All simulations were generated by solving the Markov decision process problem with covariance matrix adaptation evolution strategy, without any reference motion data. We finally demonstrated that our reinforcement learning framework also has the potential to model and predict pathological gait that can result from muscle weakness.
|
|
