| 1. |
- Bai, Ting, et al.
(författare)
-
Rollout-Based Charging Strategy for Electric Trucks With Hours-of-Service Regulations
- 2023
-
Ingår i: IEEE Control Systems Letters. - : Institute of Electrical and Electronics Engineers (IEEE). - 2475-1456. ; 7, s. 2167-2172
-
Tidskriftsartikel (refereegranskat)abstract
- Freight drivers of electric trucks need to design charging strategies for where and how long to recharge the truck in order to complete delivery missions on time. Moreover, the charging strategies should be aligned with drivers' driving and rest time regulations, known as hours-of-service (HoS) regulations. This letter studies the optimal charging problems of electric trucks with delivery deadlines under HoS constraints. We assume that a collection of charging and rest stations is given along a pre-planned route with known detours and that the problem data are deterministic. The goal is to minimize the total cost associated with the charging and rest decisions during the entire trip. This problem is formulated as a mixed integer program with bilinear constraints, resulting in a high computational load when applying exact solution approaches. To obtain real-time solutions, we develop a rollout-based approximate scheme, which scales linearly with the number of stations while offering solid performance guarantees. We perform simulation studies over the Swedish road network based on realistic truck data. The results show that our rollout-based approach provides near-optimal solutions to the problem in various conditions while cutting the computational time drastically.
|
|
| 2. |
- Emanuelsson, William, et al.
(författare)
-
Multiagent Rollout with Reshuffling for Warehouse Robots Path Planning
- 2023
-
Ingår i: IFAC-PapersOnLine. - : Elsevier B.V.. ; , s. 3027-3032
-
Konferensbidrag (refereegranskat)abstract
- Efficiently solving path planning problems for a large number of robots is critical to the successful operation of modern warehouses. The existing approaches adopt classical shortest path algorithms to plan in environments whose cells are associated with both space and time in order to avoid collision between robots. In this work, we achieve the same goal by means of simulation in a smaller static environment. Built upon the new framework introduced in (Bertsekas, 2021a), we propose multiagent rollout with reshuffling algorithm, and apply it to address the warehouse robots path planning problem. The proposed scheme has a solid theoretical guarantee and exhibits consistent performance in our numerical studies. Moreover, it inherits from the generic rollout methods the ability to adapt to a changing environment by online replanning, which we demonstrate through examples where some robots malfunction.
|
|
| 3. |
- Krook, Jonas, 1986, et al.
(författare)
-
Design and Formal Verification of a Safe Stop Supervisor for an Automated Vehicle
- 2019
-
Ingår i: Proceedings - IEEE International Conference on Robotics and Automation. - : Institute of Electrical and Electronics Engineers (IEEE). - 1050-4729. - 9781538660263 ; , s. 5607-5613
-
Konferensbidrag (refereegranskat)abstract
- Autonomous vehicles apply pertinent planning and control algorithms under different driving conditions. The mode switch between these algorithms should also be autonomous. On top of the nominal planners, a safe fallback routine is needed to stop the vehicle at a safe position if nominal operational conditions are violated, such as for a system failure. This paper describes the design and formal verification of a supervisor to manage all requirements for mode switching between nominal planners, and additional requirements for switching to a safe stop trajectory planner that acts as the fallback routine. The supervisor is designed via a model-based approach and its abstraction is formally verified by model checking. The supervisor is implemented and integrated with the Research Concept Vehicle, an experimental research and demonstration vehicle developed at the KTH Royal Institute of Technology. Simulations and experiments show that the vehicle is able to autonomously drive in a safe manner between two parking lots and can successfully come to a safe stop upon GPS sensor failure.
|
|
| 4. |
- Li, Yuchao, et al.
(författare)
-
A cascade control approach to active suspension using pneumatic actuators
- 2019
-
Ingår i: Asian Journal of Control. - : Wiley. - 1561-8625 .- 1934-6093. ; , s. 1-19
-
Tidskriftsartikel (refereegranskat)abstract
- Operators of forest machinery suffer from intensive whole body vibrations, which are big threats to their health. Therefore, it is important to investigate effective seat undercarriages and control methods for vibration reduction. This paper addresses the control problem of a novel seat undercarriage with pneu-matic actuators customized for forest machinery. A two-layer cascade controlstructure is developed, where the top layer consists of a group of proportional controllers to regulate the position of pneumatic actuators and the bottom layeris a sliding mode controller for force and stiffness tracking. The advantage ofthe sliding mode control is to achieve robust control performance with coarse system models. The paper demonstrates that the proposed control structure is better than a traditional PID controller. The robust stability of the sliding mode controller is proved by the Lyapunov's method. Experiments show its capability of reducing at least 20% amplitude of seat vibrations from 0.5 to 1 Hz.
|
|
| 5. |
- Li, Yuchao, et al.
(författare)
-
A Geometric Programming Approach to the Optimization of Mechatronic Systems in Early Design Stages
- 2016
-
Ingår i: 2016 IEEE INTERNATIONAL CONFERENCE ON ADVANCED INTELLIGENT MECHATRONICS (AIM). - : IEEE conference proceedings. - 9781509020652 ; , s. 1351-1356
-
Konferensbidrag (refereegranskat)abstract
- This paper evaluates geometric programming as a solver to optimize mechatronic system design in a holistic manner to aid early design decisions. Mechatronic systems design optimization requires complex and often non-convex functions as design objectives and constraints. Currently the solutions are primarily based on randomized search methods, e.g., genetic algorithms, and they are time-consuming. This paper converts complex constraints and objectives into approximate posynomial forms, which can then be used with disciplined convex optimization to significantly reduce the computation time for optimization. The approach is compared to the previous research using a mechatronic servo system design case study consisting of a motor, a shaft, two planetary gears and a rotational load. The result confirms that the geometric programming approach improves both computation speed and accuracy.
|
|
| 6. |
- Li, Yuchao, et al.
(författare)
-
A hierarchical control system for smart parking lots with automated vehicles : Improve efficiency by leveraging prediction of human drivers
- 2019
-
Ingår i: Proceedings 2019 18TH EUROPEAN CONTROL CONFERENCE (ECC). - : IEEE. ; , s. 2675-2681
-
Konferensbidrag (refereegranskat)abstract
- In this work, we introduce a hierarchical architecture for management of multiple automated vehicles in a parking lot provided the existence of human-driven vehicles. The proposed architecture consists of three layers: behavior prediction, vehicle coordination and maneuver control, with the first two sitting in the infrastructure and the third one equipped on individual vehicles. We assume all three layers share a consistent view of the environment by considering it as a grid world. The grid occupancy is modeled by the prediction layer via collecting information from automated vehicles and predicting human-driven vehicles. The coordination layer assigns parking spots and grants permissions for vehicles to move. The vehicle control embraces the distributed model predictive control (MPC) technique to resolve local conflicts occurred due to the simplified vehicle models used in the design of the prediction and coordination layers. Numerical evaluation shows the effectiveness of the proposed control system.
|
|
| 7. |
- Li, Yuchao
(författare)
-
Approximate Methods of Optimal Control via Dynamic Programming Models
- 2023
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Optimal control theory has a long history and broad applications. Motivated by the goal of obtaining insights through unification and taking advantage of the abundant capability to generate data and perform online simulation, this thesis studies the discrete-time infinite horizon optimal control problems and introduces some approximate solution methods via abstract dynamic programming (DP) models. The proposed methods involve approximation in value space through the use of data and simulator, apply to a broad class of problems, and strike a good balance between satisfactory performance and computational expenditure.First, we consider deterministic problems with nonnegative stage costs. We derive sufficient conditions under which a local controllability condition holds for the constrained nonlinear systems, and apply the results to establish the convergence of the classical algorithms, including value iteration, policy iteration (PI), and optimistic PI. These results provide a starting point for the design of suboptimal schemes. Then we propose algorithms that take advantage of system trajectory or the presence of parallel computing units to approximate the optimal costs. These algorithms can be viewed as variants of model predictive control (MPC) or rollout, and can be applied to deterministic problems with arbitrary state and control spaces, and arbitrary dynamics. It admits extensions to problems with trajectory constraints, and a multiagent structure. Via the viewpoint provided by the abstract DP models, we also derive the performance bounds of MPC applied to unconstrained and constrained linear quadratic problems, as well as their nonlinear counterparts. These insights suggest new designs of MPC, which likely lead to larger feasible regions of the scheme while costing hardly any loss of performance measured by the costs accumulated over infinite stages. Moreover, we derive algorithms to address problems with a fixed discount factor on future costs. We apply abstract DP models to analyze $\lambda$-PI with randomization algorithms for problems with infinite policies. We show that a contraction property induced by the discount factor is sufficient for the well-posedness of the algorithm. Moreover, we identify the conditions under which the algorithm is convergent with probability one. Guided by the analysis, we exemplify a data-driven approximate implementation of the algorithm for the approximation of the optimal costs of constrained linear and nonlinear control problems. The obtained optimal cost approximations are applied in a related suboptimal scheme. Then we consider discounted problems with discrete state and control spaces and a multiagent structure. When applying rollout to address the problem, the main challenge is to perform minimization over a large control space. To this end, we propose a rollout variant that involves reshuffling the order of the agents. The approximation of the costs of base policies is through the use of on-line simulation. The proposed approach is applied to address multiagent path planning problems within a warehouse context, where through on-line replanning, the robots can adapt to a changing environment while avoiding collision with each other.
|
|
| 8. |
- Li, Yuchao
(författare)
-
Approximate Solution Methods to Optimal Control Problems via Dynamic Programming Models
- 2021
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Optimal control theory has a long history and broad applications. Motivated by the goal of obtaining insights through unification and taking advantage of the abundant capability to generate data, this thesis introduces some suboptimal schemes via abstract dynamic programming models.As our first contribution, we consider deterministic infinite horizon optimal control problems with nonnegative stage costs. We draw inspiration from the learning model predictive control scheme designed for continuous dynamics and iterative tasks, and propose a rollout algorithm that relies on sampled data generated by some base policy. The proposed algorithm is based on value and policy iteration ideas. It applies to deterministic problems with arbitrary state and control spaces, and arbitrary dynamics. It admits extensions to problems with trajectory constraints, and a multiagent structure.In addition, abstract dynamic programming models are used to analyze $\lambda$-policy iteration with randomization algorithms. In particular, we consider contractive models with infinite policies. We show that well-posedness of the $\lambda$-operator plays a central role in the algorithm. The operator is known to be well-posed for problems with finite states, but our analysis shows that it is also well-defined for the contractive models with infinite states. Similarly, the algorithm we analyze is known to converge for problems with finite policies, but we identify the conditions required to guarantee convergence with probability one when the policy space is infinite regardless of the number of states. Guided by the analysis, we exemplify a data-driven approximated implementation of the algorithm for estimation of optimal costs of constrained linear and nonlinear control problems. Numerical results indicate the potentials of this method in practice.
|
|
| 9. |
- Li, Yuchao, et al.
(författare)
-
Data-driven Rollout for Deterministic Optimal Control
- 2021
-
Ingår i: 2021 60th IEEE conference on decision and control (CDC). - : Institute of Electrical and Electronics Engineers (IEEE). ; , s. 2169-2176
-
Konferensbidrag (refereegranskat)abstract
- We consider deterministic infinite horizon optimal control problems with nonnegative stage costs. We draw inspiration from learning model predictive control scheme designed for continuous dynamics and iterative tasks, and propose a rollout algorithm that relies on sampled data generated by some base policy. The proposed algorithm is based on value and policy iteration ideas, and applies to deterministic problems with arbitrary state and control spaces, and arbitrary dynamics. It admits extensions to problems with trajectory constraints, and a multiagent structure.
|
|
| 10. |
- Li, Yuchao, et al.
(författare)
-
Lambda-Policy Iteration with Randomization for Contractive Models with Infinite Policies : Well-Posedness and Convergence
- 2020
-
Ingår i: Proceedings of the 2nd Conference on Learning for Dynamics and Control, L4DC 2020. - : ML Research Press. ; , s. 540-549
-
Konferensbidrag (refereegranskat)abstract
- dynamic programming models are used to analyze λ-policy iteration with randomization algorithms. Particularly, contractive models with infinite policies are considered and it is shown that well-posedness of the λ-operator plays a central role in the algorithm. The operator is known to be well-posed for problems with finite states, but our analysis shows that it is also well-defined for the contractive models with infinite states studied. Similarly, the algorithm we analyze is known to converge for problems with finite policies, but we identify the conditions required to guarantee convergence with probability one when the policy space is infinite regardless of the number of states. Guided by the analysis, we exemplify a data-driven approximated implementation of the algorithm for estimation of optimal costs of constrained linear and nonlinear control problems. Numerical results indicate potentials of this method in practice.
|
|
| 11. |
- Li, Yuchao, et al.
(författare)
-
Linear Time-Varying Model Predictive Control for Automated Vehicles : Feasibility and Stability under Emergency Lane Change
- 2020
-
Ingår i: Ifac papersonline. - : Elsevier BV. - 2405-8963. ; , s. 15719-15724
-
Konferensbidrag (refereegranskat)abstract
- In this work, we present a novel approach based on linear matrix inequalities to design a linear-time varying model predictive controller for a nonlinear system with guaranteed stability. The proposed method utilizes a multi-model description to model the nonlinear system where the dynamics is represented by a group of linear-time invariant plants, which makes the resulting optimization problem easy to solve in real-time. In addition, we apply the control invariant set designed as the final stage constraint to bound the additive disturbance introduced to the plant by other subsystems interfacing with the controller. We show that the persistent feasibility is ensured with the presence of such constraint on the disturbance of the specified kind. The proposed method is then put into the context of emergency lane change for steering control of automated vehicles and its performance is verified via numerical evaluation.
|
|
| 12. |
- Li, Yuchao, et al.
(författare)
-
Performance Bounds of Model Predictive Control for Unconstrained and Constrained Linear Quadratic Problems and Beyond
- 2023
-
Konferensbidrag (refereegranskat)abstract
- We study unconstrained and constrained linear quadratic problems and investigate the suboptimality of the model predictive control (MPC) method applied to such problems. Considering MPC as an approximate scheme for solving the related fixed point equations, we derive performance bounds for the closed-loop system under MPC. Our analysis, as well as numerical examples, suggests new ways of choosing the terminal cost and terminal constraints, which are not related to the solution of the Riccati equation of the original problem. The resulting method can have a larger feasible region, and cause hardly any loss of performance in terms of the closed-loop cost over an infinite horizon.
|
|
| 13. |
- Liu, Hanxiao, et al.
(författare)
-
Reinforcement Learning Based Approach for Flip Attack Detection
- 2020
-
Ingår i: Proceedings of the IEEE Conference on Decision and Control. - : Institute of Electrical and Electronics Engineers Inc.. ; , s. 3212-3217
-
Konferensbidrag (refereegranskat)abstract
- This paper addresses the detection problem of flip attacks to sensor network systems where the attacker flips the distribution of manipulated sensor measurements of a binary state. The detector decides to continue taking observations or to stop based on the sensor measurements, and the goal is to have the flip attack recognized as fast as possible while trying to avoid terminating the measurements when no attack is present. The detection problem can be modeled as a partially observable Markov decision process (POMDP) by assuming an attack probability, with the dynamics of the hidden states of the POMDP characterized by a stochastic shortest path (SSP) problem. The optimal policy of the SSP solely depends on the transition costs and is independent of the assumed attack possibility. By using a fixed-length window and suitable feature function of the measurements, a Markov decision process (MDP) is used to approximate the behavior of the POMDP. The optimal solution of the approximated MDP can then be solved by any standard reinforcement learning methods. Numerical evaluations demonstrates the effectiveness of the method.
|
|
| 14. |
- Liu, Hanxiao, et al.
(författare)
-
Rollout approach to sensor scheduling for remote state estimation under integrity attack
- 2022
-
Ingår i: Automatica. - : Elsevier BV. - 0005-1098 .- 1873-2836. ; 144
-
Tidskriftsartikel (refereegranskat)abstract
- We consider the sensor scheduling problem for remote state estimation under integrity attacks. We seek to optimize a trade-off between the energy consumption of communications and the state estimation error covariance when the acknowledgment (ACK) information, sent by the remote estimator to the local sensor, is compromised. The sensor scheduling problem is formulated as an infinite horizon discounted optimal control problem with infinite states. We first analyze the underlying Markov decision process (MDP) and show that the optimal scheduling without ACK attack is of the threshold type. Thus, we can simplify the problem by replacing the original state space with a finite state space. For the simplified MDP, when the ACK is under attack, the problem is modeled as a partially observable Markov decision process (POMDP). We analyze the induced MDP that uses a belief vector as its state for the POMDP. We investigate the properties of the exact optimal solution via contractive models and show that the threshold type of solution for the POMDP cannot be readily obtained. A suboptimal solution is then obtained via a rollout approach, which is a prominent class of reinforcement learning (RL) methods based on approximation in value space. We present two variants of rollout and provide performance bounds of those variants. Finally, numerical examples are used to demonstrate the effectiveness of the proposed rollout methods by comparing them with a finite history window approach that is widely used in RL for POMDP.
|
|
