| 1. |
- Bai, Ting, et al.
(författare)
-
A Pricing Rule for Third-Party Platoon Coordination Service Provider
- 2022
-
Ingår i: ASCC 2022 - 2022 13th Asian Control Conference, Proceedings. - : Institute of Electrical and Electronics Engineers (IEEE). ; , s. 2344-2349
-
Konferensbidrag (refereegranskat)abstract
- We model a platooning system including trucks and a third-party service provider that performs platoon coordination, distributes the platooning profit within platoons, and charges the trucks in exchange for its services. This paper studies one class of pricing rules, where the third-party service provider keeps part of the platooning profit each time a platoon is formed. Furthermore, we propose a platoon coordination solution based on distributed model predictive control in which the pricing rule is integrated. To evaluate the effect of the pricing on the platooning system, we perform a simulation over the Swedish road network. The simulation shows that the platooning rate and profit highly depend on the pricing. This suggests that pricing needs to be set carefully to obtain a satisfactory platooning system in the future.
|
|
| 2. |
- Bai, Ting, et al.
(författare)
-
Approximate Dynamic Programming for Platoon Coordination under Hours-of-Service Regulations
- 2022
-
Ingår i: 2022 IEEE 61ST CONFERENCE ON DECISION AND CONTROL (CDC). - : Institute of Electrical and Electronics Engineers (IEEE). ; , s. 7663-7669
-
Konferensbidrag (refereegranskat)abstract
- Truck drivers are required to stop and rest with a certain regularity according to the driving and rest time regulations, also called Hours-of-Service (HoS) regulations. This paper studies the problem of optimally forming platoons when considering realistic HoS regulations. In our problem, trucks have fixed routes in a transportation network and can wait at hubs along their routes to form platoons with others while fulfilling the driving and rest time constraints. We propose a distributed decision-making scheme where each truck controls its waiting times at hubs based on the predicted schedules of others. The decoupling of trucks' decision-makings contributes to an approximate dynamic programming approach for platoon coordination under HoS regulations. Finally, we perform a simulation over the Swedish road network with one thousand trucks to evaluate the achieved platooning benefits under the HoS regulations in the European Union (EU). The simulation results show that, on average, trucks drive in platoons for 37 % of their routes if each truck is allowed to be delayed for 5 % of its total travel time. If trucks are not allowed to be delayed, they drive in platoons for 12 % of their routes.
|
|
| 3. |
- Bai, Ting, et al.
(författare)
-
Event-Triggered Distributed Model Predictive Control for Platoon Coordination at Hubs in a Transport System
- 2021
-
Ingår i: 2021 60TH IEEE CONFERENCE ON DECISION AND CONTROL (CDC). - : Institute of Electrical and Electronics Engineers (IEEE). ; , s. 1198-1204
-
Konferensbidrag (refereegranskat)abstract
- This paper considers the problem of hub-based platoon coordination for a large-scale transport system, where trucks have individual utility functions to optimize. An event-triggered distributed model predictive control method is proposed to solve the optimal scheduling of waiting times at hubs for individual trucks. In this distributed framework, trucks are allowed to decide their waiting times independently and only limited information is shared between trucks. Both the predicted reward gained from platooning and the predicted cost for waiting at hubs are included in each truck's utility function. The performance of the coordination method is demonstrated in a simulation with one hundred trucks over the Swedish road network.
|
|
| 4. |
- Bai, Ting, et al.
(författare)
-
Large-Scale Multi-Fleet Platoon Coordination : A Dynamic Programming Approach
- 2023
-
Ingår i: IEEE transactions on intelligent transportation systems (Print). - : Institute of Electrical and Electronics Engineers (IEEE). - 1524-9050 .- 1558-0016. ; 24:12, s. 14427-14442
-
Tidskriftsartikel (refereegranskat)abstract
- Truck platooning is a promising technology that enables trucks to travel in formations with small inter-vehicle distances for improved aerodynamics and fuel economy. The real-world transportation system includes a vast number of trucks owned by different fleet owners, for example, carriers. To fully exploit the benefits of platooning, efficient dispatching strategies that facilitate the platoon formations across fleets are required. This paper presents a distributed framework for addressing multi-fleet platoon coordination in large transportation networks, where each truck has a fixed route and aims to maximize its own fleet's platooning profit by scheduling its waiting times at hubs. The waiting time scheduling problem of individual trucks is formulated as a distributed optimal control problem with continuous decision space and a reward function that takes non-zero values only at discrete points. By suitably discretizing the decision and state spaces, we show that the problem can be solved exactly by dynamic programming, without loss of optimality. Finally, a realistic simulation study is conducted over the Swedish road network with 5,000 trucks to evaluate the profit and efficiency of the approach. The simulation study shows that, compared to single-fleet platooning, multi-fleet platooning provided by our method achieves around 15 times higher monetary profit and increases the CO2 emission reductions from 0.4% to 5.5%. In addition, it shows that the developed approach can be carried out in real-time and thus is suitable for platoon coordination in large transportation systems.
|
|
| 5. |
- Bai, Ting, et al.
(författare)
-
Rollout-Based Charging Strategy for Electric Trucks With Hours-of-Service Regulations
- 2023
-
Ingår i: IEEE Control Systems Letters. - : Institute of Electrical and Electronics Engineers (IEEE). - 2475-1456. ; 7, s. 2167-2172
-
Tidskriftsartikel (refereegranskat)abstract
- Freight drivers of electric trucks need to design charging strategies for where and how long to recharge the truck in order to complete delivery missions on time. Moreover, the charging strategies should be aligned with drivers' driving and rest time regulations, known as hours-of-service (HoS) regulations. This letter studies the optimal charging problems of electric trucks with delivery deadlines under HoS constraints. We assume that a collection of charging and rest stations is given along a pre-planned route with known detours and that the problem data are deterministic. The goal is to minimize the total cost associated with the charging and rest decisions during the entire trip. This problem is formulated as a mixed integer program with bilinear constraints, resulting in a high computational load when applying exact solution approaches. To obtain real-time solutions, we develop a rollout-based approximate scheme, which scales linearly with the number of stations while offering solid performance guarantees. We perform simulation studies over the Swedish road network based on realistic truck data. The results show that our rollout-based approach provides near-optimal solutions to the problem in various conditions while cutting the computational time drastically.
|
|
| 6. |
- Bharmoria, Pankaj, 1985, et al.
(författare)
-
Recyclable optical bioplastics platform for solid state red light harvesting via triplet-triplet annihilation photon upconversion
- 2022
-
Ingår i: Journal of Materials Chemistry A. - : Royal Society of Chemistry (RSC). - 2050-7496 .- 2050-7488. ; 10:40, s. 21279-21290
-
Tidskriftsartikel (refereegranskat)abstract
- Sustainable photonics applications of solid-state triplet-triplet annihilation photon upconversion (TTA-UC) are limited by a small UC spectral window, low UC efficiency in air, and non-recyclability of polymeric materials used. In a step to overcome these issues, we have developed new recyclable TTA-UC bioplastics by encapsulating TTA-UC chromophores liquid inside the semicrystalline gelatin films showing broad-spectrum upconversion (red/far-red to blue) with high UC efficiency in air. For this, we synthesized a new anionic annihilator, sodium-TIPS-anthracene-2-sulfonate (TIPS-AnS), that combined with red/far-red sensitizers (PdTPBP/Os(m-peptpy)(2)(TFSI)(2)), a liquid surfactant Triton X-100 reduced (TXr) and protein gelatin (G) formed red/far-red to blue TTA-UC bioplastic films just by air drying of their aqueous solutions. The G-TXr-TIPS-AnS-PdTPBP film showed record red to blue (633 to 478 nm) TTA-UC quantum yield of 8.5% in air. The high UC quantum yield has been obtained due to the fluidity of dispersed TXr containing chromophores and oxygen blockage by gelatin fibers that allowed efficient diffusion of triplet excited chromophores. Further, the G-TXr-TIPS-AnS-Os(m-peptpy)(2)(TFSI)(2) bioplastic film displayed far-red to blue (700-730 nm to 478 nm) TTA-UC, demonstrating broad-spectrum photon harvesting. Finally, we demonstrated the recycling of G-TXr-TIPS-AnS-PdTPBP bioplastics by developing a downstream approach that gives new directions for designing future recyclable photonics bioplastic materials.
|
|
| 7. |
- Bobjer, Johannes, et al.
(författare)
-
Location of retroperitoneal lymph node metastases in upper tract urothelial carcinoma : results from a prospective lymph node mapping study
- 2023
-
Ingår i: European Urology Open Science. - : Elsevier. - 2666-1691 .- 2666-1683. ; 57, s. 37-44
-
Tidskriftsartikel (refereegranskat)abstract
- Background: There is limited information on the distribution of retroperitoneal lymph node metastases (LNMs) in upper tract urothelial carcinoma (UTUC).Objective: To investigate the location of LNMs in UTUC of the renal pelvis or proximal ureter and short-term complications after radical nephroureterectomy (RNU) with lymph node dissection (LND).Design, setting, and participants: This was a prospective Nordic multicenter study (four university hospitals, two county hospitals). Patients with clinically suspected locally advanced UTUC (stage >T1) and/or clinical lymph node–positive (cN+) disease were invited to participate. Participants underwent RNU and fractionated retroperitoneal LND using predefined side-specific templates.Outcome measurements and statistical analysis: The location of LNMs in the LND specimen and retroperitoneal lymph node recurrences during follow-up was recorded. Postoperative complications within 90 d of surgery were ascertained from patient charts. Descriptive statistics were used.Results and limitations: LNMs were present in the LND specimen in 23/100 patients, and nine of 100 patients experienced a retroperitoneal recurrence. Distribution per side revealed LNMs in the LND specimen in 11/38 (29%) patients with right-sided tumors, for whom the anatomically larger, right-sided template was used, in comparison to 12/62 (19%) patients with left-sided tumors, for whom a more limited template was used. High-grade complications (Clavien grade ≥3) within 90 d of surgery were registered for 13/100 patients. The study is limited in size and not powered to assess survival estimates.Conclusions: The suggested templates that we prospectively applied for right-sided and left-sided LND in patients with advanced UTUC included the majority of LNMs. High-grade complications directly related to the LND part of the surgery were limited.Patient summary: This study describes the location of lymph node metastases in patients with cancer in the upper urinary tract who underwent surgery to remove the affected kidney and ureter. The results show that most metastases occur within the template maps for lymph node surgery that we investigated, and that this surgery can be performed with few severe complications.
|
|
| 8. |
- Chen, Xiao, et al.
(författare)
-
Safe Intersection Coordination with Mixed Traffic : From Estimation to Control
- 2023
-
Konferensbidrag (refereegranskat)abstract
- In this paper, we propose an integrated framework for safe intersection coordination of connected and automated vehicles (CAVs) in mixed traffic. An intelligent intersection is introduced as a central node to orchestrate state data sharing among connected agents and enable CAV to acknowledge the presence of human-driven vehicles (HDVs) beyond the line of sight of onboard sensors. Since state data shared between agents might be uncertain or delayed, we design the intelligent intersection to safely compensate for these uncertainties and delays using robust set estimation and forward reachability analysis. When the intersection receives state data from an agent, it first generates a zonotope to capture the possible measurement noise in the state estimate. Then, to compensate for communication and processing delays, it uses forward reachability analysis to enlarge the set to capture all the possible states the agent could have occupied throughout the delays. Finally, using the resulting set as the initial condition, a distributed model predictive control onboard the CAV will plan an invariant safe motion by considering the worst-case behavior of human drivers. As a result, the vehicle is guaranteed to be safe while driving through the intersection. A prototype of our proposed framework is implemented using.
|
|
| 9. |
- Emanuelsson, William, et al.
(författare)
-
Multiagent Rollout with Reshuffling for Warehouse Robots Path Planning
- 2023
-
Ingår i: IFAC-PapersOnLine. - : Elsevier B.V.. ; , s. 3027-3032
-
Konferensbidrag (refereegranskat)abstract
- Efficiently solving path planning problems for a large number of robots is critical to the successful operation of modern warehouses. The existing approaches adopt classical shortest path algorithms to plan in environments whose cells are associated with both space and time in order to avoid collision between robots. In this work, we achieve the same goal by means of simulation in a smaller static environment. Built upon the new framework introduced in (Bertsekas, 2021a), we propose multiagent rollout with reshuffling algorithm, and apply it to address the warehouse robots path planning problem. The proposed scheme has a solid theoretical guarantee and exhibits consistent performance in our numerical studies. Moreover, it inherits from the generic rollout methods the ability to adapt to a changing environment by online replanning, which we demonstrate through examples where some robots malfunction.
|
|
| 10. |
- Jiang, Frank, et al.
(författare)
-
Safe Teleoperation of Connected and Automated Vehicles
- 2023
-
Ingår i: Cyber–Physical–Human Systems. - : Wiley. ; , s. 251-272
-
Bokkapitel (övrigt vetenskapligt/konstnärligt)abstract
- In this chapter, we discuss teleoperation systems for connected and automated vehicles and overview the research around making these systems safe and human-centric. In recent years, teleoperation by human operators has become an important modality of supervision for connected and automated vehicles. While automated driving systems continue to mature and handle a rich variety of scenarios, we are continually reminded that unplanned exceptions and deviations from the driving systems' operational design domain – be it unpredictable human behavior or unmodeled kangaroo mechanics – can create potentially unsafe situations for automated vehicles. From decades of research in human factors in air traffic control, we know that we can robustify against automation failure by closely integrating human operators into our automation systems. To this end, teleoperation systems have become of great interest to the connected and automated vehicle research community. By establishing teleoperation systems, we can ensure that when a vehicle experiences automation failure, we have a fallback system where a human operator is able to handle the failure. Moreover, due to the recent introduction of ultra-reliable and low latency communication in 5G networks, we know that wireless communication channels are reliable enough to safely deploy such fallback systems. Since the human operator is remote, we can potentially set up “control towers” for vehicles where a few operators can manage many vehicles. However, to reach this potential, there are several safety and cyber–physical–human system design challenges that need to be addressed. In this chapter, we will introduce these challenges and discuss the recent research that has been conducted to solve these challenges. Furthermore, we will discuss the future research directions that will further enhance the safety of teleoperation systems.
|
|
| 11. |
- Jiang, Frank, et al.
(författare)
-
SVEA : an experimental testbed for evaluating V2X use-cases
- 2022
-
Ingår i: 2022 IEEE 25TH INTERNATIONAL CONFERENCE ON INTELLIGENT TRANSPORTATION SYSTEMS (ITSC). - : Institute of Electrical and Electronics Engineers (IEEE). ; , s. 3484-3489
-
Konferensbidrag (refereegranskat)abstract
- In this paper, we present a hardware and software testbed designed for evaluating vehicle-to-everything (V2X) usecases. From platooning to remote driving, there are many proposals to use V2X communication to solve sustainability or safety issues in transport networks. However, researchers mostly evaluate their proposals in only simulation studies, since setting up real, full-scale field tests can often be prohibitively expensive or time-consuming. The open-sourced Small Vehicles for Autonomy (SVEA) testbed is built around a communication software stack and a 1/10th-scale automated vehicle platform suitable for both cost-effective and time-efficient experimentation with V2X use-cases. The testbed is designed to support evaluation in a wide range of conditions, such as heterogeneous networks or vehicle fleets. To illustrate the suitability of the SVEA testbed for studying V2X use-cases, we detail and implement three use-cases: platooning, adaptive speed regulation from a road-side infrastructure camera, and remote-driving by a human operator sitting in a control tower. Finally, we conclude the paper with a discussion on the use of the platform so far and future development plans.
|
|
| 12. |
- Johansson, Alexander
(författare)
-
Coordination of cross-carrier truck platooning
- 2022
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- The need for sustainable transportation solutions is urgent as the demand for mobility of goods and people is expected to multiply in the upcoming decades. One promising solution is truck platooning, which shows great potential in reducing the energy consumption and operational costs of trucks. To utilize the benefits of truck platooning to the fullest, trucks with different schedules and routes in a road network need coordination to form platoons. This thesis addresses platoon coordination when trucks can wait at hubs to form platoons. We assume there is a reward for driving in a platoon and a cost for waiting at a hub, and the objective is to maximize the overall profit. We focus on coordinating trucks from different carriers, which is important considering that many platoon opportunities are lost if only trucks from the same carrier form platoons.In the first contributions of the thesis, we propose coordination solutions where carriers aim to maximize their own profits through cross-carrier platoon cooperation. We propose an architecture of a platoon-hailing service that stores reported platooning plans of carriers and, based on these, informs carriers about the platoons their trucks can join when they make platooning decisions. A realistic simulation study shows that the cross-carrier platooning system can achieve energy savings of 3.0% and 5.4% when 20% and 100% of the trucks are coordinated, respectively. A non-cooperative game is then formulated to model the strategic interaction among trucks with individual objectives when they coordinate for platooning and make decisions at the beginning of their journeys. The existence of at least one Nash equilibrium is shown. In the case of stochastic travel times, feedback-based solutions are developed wherein trucks repeatedly update their equilibrium decisions. A simulation study with stochastic travel times shows that the feedback-based solutions achieve platooning rates only $5\%$ lower than a solution where the travel times are known. We also explore Pareto-improving coordination guaranteeing each carrier is better off coopering with others, and models for distributing the profit within platoons.In the last contributions of the thesis, we study the problem of optimally releasing trucks at hubs when arriving according to a stochastic process, and a priori information about truck arrivals is inaccessible; this may be sensitive information to share with others. First, we study the release problem at hubs in a hub-corridor where the objective is to maximize the profit over time. The optimality of threshold-based release policies is shown under the assumption that arrivals are independent or that arrivals are dependent due to the releasing behavior at the preceding hub in the corridor. Then, we study the release problem at a single hub where the aim is to maximize the profit of trucks currently at the hub. This is realistic if trucks are only willing to wait at the hub if they can increase their own profits. Stopping time theory is used to show the optimality of a threshold-based release policy when arrivals are independent and identically distributed. These contributions show that simple coordination approaches can achieve high profits from platooning, even under limited information.
|
|
| 13. |
- Johansson, Alexander, et al.
(författare)
-
Hub-Based Platoon Formation : Optimal Release Policies and Approximate Solutions
- 2024
-
Ingår i: IEEE transactions on intelligent transportation systems (Print). - : Institute of Electrical and Electronics Engineers (IEEE). - 1524-9050 .- 1558-0016. ; 25:6, s. 5755-5766
-
Tidskriftsartikel (refereegranskat)abstract
- This paper studies the optimal hub-based platoon formation at hubs along a highway under decentralized, distributed, and centralized policies. Hubs are locations along highways where trucks can wait for other trucks to form platoons. A coordinator at each hub decides the departure time of trucks, and the released trucks from the hub will form platoons. The problem is cast as an optimization problem where the objective is to maximize the platooning reward. We first show that the optimal release policy in the decentralized case, where the hubs do not exchange information, is to release all trucks at the hub when the number of trucks exceeds a threshold computed by dynamic programming. We develop efficient approximate release policies for the dependent arrival case using this result. To study the value of information exchange among hubs on platoon formation, we next study the distributed and centralized platoon formation policies which require information exchange among hubs. To this end, we develop receding horizon solutions for the distributed and centralized platoon formation at hubs using the dynamic programming technique. Finally, we perform a simulation study over three hubs in northern Sweden. The profits of the decentralized policies are shown to be approximately $3.5\%$ lower than the distributed policy and $8\%$ lower than the centralized release policy. This observation suggests that decentralized policies are prominent solutions for hub-based platooning as they do not require information exchange among hubs and can achieve a similar performance compared with distributed and centralized policies.
|
|
| 14. |
- Johansson, Alexander, et al.
(författare)
-
Platoon Cooperation Across Carriers: From System Architecture to Coordination
- 2023
-
Ingår i: IEEE Intelligent Transportation Systems Magazine. - : Institute of Electrical and Electronics Engineers (IEEE). - 1939-1390 .- 1941-1197. ; 15:3, s. 132-144
-
Tidskriftsartikel (refereegranskat)abstract
- Truck platooning is a well-studied technology that has the potential to reduce both the environmental impact and operational costs of trucks. The technology has matured over the last 20 years, and the commercial rollout of platooning is approaching. Cooperation across carriers is essential for the viability of platooning; otherwise, many platooning opportunities are lost. We first present a cross-carrier platooning system architecture in which many carriers cooperate in forming platoons through a platoon-hailing service. Then, we present a cross-carrier platoon coordination approach in which each carrier optimizes its platooning plans according to the predicted plans of other carriers. A profit-sharing mechanism to even out the platooning profit in each platoon is embedded in the platoon coordination approach. Finally, a simulation study over the Swedish road network is performed to evaluate the potential of platooning under realistic conditions. The simulation study shows that the energy consumption of trucks in Sweden can be reduced by 5.4% due to platooning and that cooperation across carriers is essential to achieve significant platooning benefits.
|
|
| 15. |
- Johansson, Alexander, et al.
(författare)
-
Platoon Coordination in Large-Scale Networks : A Game Theoretic Approach
- 2023
-
Ingår i: Systems and Control. - : Springer Nature. ; , s. 79-100
-
Bokkapitel (övrigt vetenskapligt/konstnärligt)abstract
- The emerging commercial rollout of heavy-duty vehicle platooning necessitates the development of efficient platoon coordination solutions. The commercial vehicle fleet consists of vehicles owned by different transportation companies with different objectives. To capture their strategic behavior, we study platoon coordination that aims to maximize profits for individual vehicles. The interaction among vehicles is modeled as a non-cooperative game. In our cyber-physical system, we consider a large number of vehicles with fixed routes in a transportation network that can wait at hubs along their routes to form platoons. Each vehicle aims to maximize its utility function, which includes a reward for platooning and a cost for waiting. We propose open-loop coordination solutions when the vehicles decide on their waiting times at the beginning of their trips and do not update their decisions during their trips. It is shown that the corresponding game admits at least one Nash equilibrium. We also propose feedback solutions in which the vehicles are allowed to update their decisions along their routes. In a simulation study over the Swedish road network, we compare the proposed platoon coordination solutions and evaluate the benefits of non-cooperative platooning at a societal scale.
|
|
| 16. |
- Johansson, A., et al.
(författare)
-
Strategic Hub-Based Platoon Coordination Under Uncertain Travel Times
- 2021
-
Ingår i: IEEE transactions on intelligent transportation systems (Print). - : Institute of Electrical and Electronics Engineers Inc.. - 1524-9050 .- 1558-0016. ; , s. 1-11
-
Tidskriftsartikel (refereegranskat)abstract
- We study the strategic interaction among vehicles in a non-cooperative platoon coordination game. Vehicles have predefined routes in a transportation network with a set of hubs where vehicles can wait for other vehicles to form platoons. Vehicles decide on their waiting times at hubs and the utility function of each vehicle includes both the benefit from platooning and the cost of waiting. We show that the platoon coordination game is a potential game when the travel times are either deterministic or stochastic, and the vehicles decide on their waiting times at the beginning of their journeys. We also propose two feedback solutions for the coordination problem when the travel times are stochastic and vehicles are allowed to update their strategies along their routes. The solutions are evaluated in a simulation study over the Swedish road network. It is shown that uncertainty in travel times affects the total benefit of platooning drastically and the benefit from platooning in the system increases significantly when utilizing feedback solutions.
|
|
| 17. |
- Johansson, Alexander, et al.
(författare)
-
Truck Platoon Formation at Hubs : An Optimal Release Time Rule
- 2020
-
Ingår i: Ifac papersonline. - : Elsevier BV. - 2405-8963. ; , s. 15312-15318
-
Konferensbidrag (refereegranskat)abstract
- We consider a hub-based platoon coordination problem in which vehicles arrive at a hub according to an independent and identically distributed stochastic arrival process. The vehicles wait at the hub, and a platoon coordinator, at each time-step, decides whether to release the vehicles from the hub in the form of a platoon or wait for more vehicles to arrive. The platoon release time problem is modeled as a stopping rule problem wherein the objective is to maximize the average platooning benefit of the vehicles located at the hub and there is a cost of having vehicles waiting at the hub. We show that the stopping rule problem is monotone and the optimal platoon release time policy will therefore be in the form of a one time-step look-ahead rule. The performance of the optimal release rule is numerically compared with (i) a periodic release time rule and (ii) a non-causal release time rule where the coordinator knows all the future realizations of the arrival process. Our numerical results show that the optimal release time rule achieves a close performance to that of the non-causal rule and outperforms the periodic rule, especially when the arrival rate is low.
|
|
| 18. |
- Li, Yuchao
(författare)
-
Approximate Methods of Optimal Control via Dynamic Programming Models
- 2023
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Optimal control theory has a long history and broad applications. Motivated by the goal of obtaining insights through unification and taking advantage of the abundant capability to generate data and perform online simulation, this thesis studies the discrete-time infinite horizon optimal control problems and introduces some approximate solution methods via abstract dynamic programming (DP) models. The proposed methods involve approximation in value space through the use of data and simulator, apply to a broad class of problems, and strike a good balance between satisfactory performance and computational expenditure.First, we consider deterministic problems with nonnegative stage costs. We derive sufficient conditions under which a local controllability condition holds for the constrained nonlinear systems, and apply the results to establish the convergence of the classical algorithms, including value iteration, policy iteration (PI), and optimistic PI. These results provide a starting point for the design of suboptimal schemes. Then we propose algorithms that take advantage of system trajectory or the presence of parallel computing units to approximate the optimal costs. These algorithms can be viewed as variants of model predictive control (MPC) or rollout, and can be applied to deterministic problems with arbitrary state and control spaces, and arbitrary dynamics. It admits extensions to problems with trajectory constraints, and a multiagent structure. Via the viewpoint provided by the abstract DP models, we also derive the performance bounds of MPC applied to unconstrained and constrained linear quadratic problems, as well as their nonlinear counterparts. These insights suggest new designs of MPC, which likely lead to larger feasible regions of the scheme while costing hardly any loss of performance measured by the costs accumulated over infinite stages. Moreover, we derive algorithms to address problems with a fixed discount factor on future costs. We apply abstract DP models to analyze $\lambda$-PI with randomization algorithms for problems with infinite policies. We show that a contraction property induced by the discount factor is sufficient for the well-posedness of the algorithm. Moreover, we identify the conditions under which the algorithm is convergent with probability one. Guided by the analysis, we exemplify a data-driven approximate implementation of the algorithm for the approximation of the optimal costs of constrained linear and nonlinear control problems. The obtained optimal cost approximations are applied in a related suboptimal scheme. Then we consider discounted problems with discrete state and control spaces and a multiagent structure. When applying rollout to address the problem, the main challenge is to perform minimization over a large control space. To this end, we propose a rollout variant that involves reshuffling the order of the agents. The approximation of the costs of base policies is through the use of on-line simulation. The proposed approach is applied to address multiagent path planning problems within a warehouse context, where through on-line replanning, the robots can adapt to a changing environment while avoiding collision with each other.
|
|
| 19. |
- Li, Yuchao
(författare)
-
Approximate Solution Methods to Optimal Control Problems via Dynamic Programming Models
- 2021
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Optimal control theory has a long history and broad applications. Motivated by the goal of obtaining insights through unification and taking advantage of the abundant capability to generate data, this thesis introduces some suboptimal schemes via abstract dynamic programming models.As our first contribution, we consider deterministic infinite horizon optimal control problems with nonnegative stage costs. We draw inspiration from the learning model predictive control scheme designed for continuous dynamics and iterative tasks, and propose a rollout algorithm that relies on sampled data generated by some base policy. The proposed algorithm is based on value and policy iteration ideas. It applies to deterministic problems with arbitrary state and control spaces, and arbitrary dynamics. It admits extensions to problems with trajectory constraints, and a multiagent structure.In addition, abstract dynamic programming models are used to analyze $\lambda$-policy iteration with randomization algorithms. In particular, we consider contractive models with infinite policies. We show that well-posedness of the $\lambda$-operator plays a central role in the algorithm. The operator is known to be well-posed for problems with finite states, but our analysis shows that it is also well-defined for the contractive models with infinite states. Similarly, the algorithm we analyze is known to converge for problems with finite policies, but we identify the conditions required to guarantee convergence with probability one when the policy space is infinite regardless of the number of states. Guided by the analysis, we exemplify a data-driven approximated implementation of the algorithm for estimation of optimal costs of constrained linear and nonlinear control problems. Numerical results indicate the potentials of this method in practice.
|
|
| 20. |
- Li, Yuchao, et al.
(författare)
-
Data-driven Rollout for Deterministic Optimal Control
- 2021
-
Ingår i: 2021 60th IEEE conference on decision and control (CDC). - : Institute of Electrical and Electronics Engineers (IEEE). ; , s. 2169-2176
-
Konferensbidrag (refereegranskat)abstract
- We consider deterministic infinite horizon optimal control problems with nonnegative stage costs. We draw inspiration from learning model predictive control scheme designed for continuous dynamics and iterative tasks, and propose a rollout algorithm that relies on sampled data generated by some base policy. The proposed algorithm is based on value and policy iteration ideas, and applies to deterministic problems with arbitrary state and control spaces, and arbitrary dynamics. It admits extensions to problems with trajectory constraints, and a multiagent structure.
|
|
| 21. |
- Li, Yuchao, et al.
(författare)
-
Lambda-Policy Iteration with Randomization for Contractive Models with Infinite Policies : Well-Posedness and Convergence
- 2020
-
Ingår i: Proceedings of the 2nd Conference on Learning for Dynamics and Control, L4DC 2020. - : ML Research Press. ; , s. 540-549
-
Konferensbidrag (refereegranskat)abstract
- dynamic programming models are used to analyze λ-policy iteration with randomization algorithms. Particularly, contractive models with infinite policies are considered and it is shown that well-posedness of the λ-operator plays a central role in the algorithm. The operator is known to be well-posed for problems with finite states, but our analysis shows that it is also well-defined for the contractive models with infinite states studied. Similarly, the algorithm we analyze is known to converge for problems with finite policies, but we identify the conditions required to guarantee convergence with probability one when the policy space is infinite regardless of the number of states. Guided by the analysis, we exemplify a data-driven approximated implementation of the algorithm for estimation of optimal costs of constrained linear and nonlinear control problems. Numerical results indicate potentials of this method in practice.
|
|
| 22. |
- Li, Yuchao, et al.
(författare)
-
Performance Bounds of Model Predictive Control for Unconstrained and Constrained Linear Quadratic Problems and Beyond
- 2023
-
Konferensbidrag (refereegranskat)abstract
- We study unconstrained and constrained linear quadratic problems and investigate the suboptimality of the model predictive control (MPC) method applied to such problems. Considering MPC as an approximate scheme for solving the related fixed point equations, we derive performance bounds for the closed-loop system under MPC. Our analysis, as well as numerical examples, suggests new ways of choosing the terminal cost and terminal constraints, which are not related to the solution of the Riccati equation of the original problem. The resulting method can have a larger feasible region, and cause hardly any loss of performance in terms of the closed-loop cost over an infinite horizon.
|
|
| 23. |
- Liu, Hanxiao, et al.
(författare)
-
Reinforcement Learning Based Approach for Flip Attack Detection
- 2020
-
Ingår i: Proceedings of the IEEE Conference on Decision and Control. - : Institute of Electrical and Electronics Engineers Inc.. ; , s. 3212-3217
-
Konferensbidrag (refereegranskat)abstract
- This paper addresses the detection problem of flip attacks to sensor network systems where the attacker flips the distribution of manipulated sensor measurements of a binary state. The detector decides to continue taking observations or to stop based on the sensor measurements, and the goal is to have the flip attack recognized as fast as possible while trying to avoid terminating the measurements when no attack is present. The detection problem can be modeled as a partially observable Markov decision process (POMDP) by assuming an attack probability, with the dynamics of the hidden states of the POMDP characterized by a stochastic shortest path (SSP) problem. The optimal policy of the SSP solely depends on the transition costs and is independent of the assumed attack possibility. By using a fixed-length window and suitable feature function of the measurements, a Markov decision process (MDP) is used to approximate the behavior of the POMDP. The optimal solution of the approximated MDP can then be solved by any standard reinforcement learning methods. Numerical evaluations demonstrates the effectiveness of the method.
|
|
| 24. |
- Liu, Hanxiao, et al.
(författare)
-
Rollout approach to sensor scheduling for remote state estimation under integrity attack
- 2022
-
Ingår i: Automatica. - : Elsevier BV. - 0005-1098 .- 1873-2836. ; 144
-
Tidskriftsartikel (refereegranskat)abstract
- We consider the sensor scheduling problem for remote state estimation under integrity attacks. We seek to optimize a trade-off between the energy consumption of communications and the state estimation error covariance when the acknowledgment (ACK) information, sent by the remote estimator to the local sensor, is compromised. The sensor scheduling problem is formulated as an infinite horizon discounted optimal control problem with infinite states. We first analyze the underlying Markov decision process (MDP) and show that the optimal scheduling without ACK attack is of the threshold type. Thus, we can simplify the problem by replacing the original state space with a finite state space. For the simplified MDP, when the ACK is under attack, the problem is modeled as a partially observable Markov decision process (POMDP). We analyze the induced MDP that uses a belief vector as its state for the POMDP. We investigate the properties of the exact optimal solution via contractive models and show that the threshold type of solution for the POMDP cannot be readily obtained. A suboptimal solution is then obtained via a rollout approach, which is a prominent class of reinforcement learning (RL) methods based on approximation in value space. We present two variants of rollout and provide performance bounds of those variants. Finally, numerical examples are used to demonstrate the effectiveness of the proposed rollout methods by comparing them with a finite history window approach that is widely used in RL for POMDP.
|
|
| 25. |
- Narri, Vandana, et al.
(författare)
-
Set-Membership Estimation in Shared Situational Awareness for Automated Vehicles in Occluded Scenarios
- 2021
-
Ingår i: 2021 32nd IEEE Intelligent Vehicles Symposium (IV). - : Institute of Electrical and Electronics Engineers (IEEE). ; , s. 385-392
-
Konferensbidrag (refereegranskat)abstract
- One of the main challenges in developing autonomous transport systems based on connected and automated vehicles is the comprehension and understanding of the environment around each vehicle. In many situations, the understanding is limited to the information gathered by the sensors mounted on the ego-vehicle, and it might be severely affected by occlusion caused by other vehicles or fixed obstacles along the road. Situational awareness is the ability to perceive and comprehend a traffic situation and to predict the intent of vehicles and road users in the surrounding of the ego-vehicle. The main objective of this paper is to propose a framework for how to automatically increase the situational awareness for an automatic bus in a realistic scenario when a pedestrian behind a parked truck might decide to walk across the road. Depending on the ego-vehicle's ability to fuse information from sensors in other vehicles or in the infrastructure, shared situational awareness is developed using a set-based estimation technique that provides robust guarantees for the location of the pedestrian. A two-level information fusion architecture is adopted, where sensor measurements are fused locally, and then the corresponding estimates are shared between vehicles and units in the infrastructure. Thanks to the provided safety guarantees, it is possible to adjust the ego-vehicle speed appropriately to maintain a proper safety margin. Three scenarios of growing information complexity are considered throughout the study. Simulations show how the increased situational awareness allows the ego-vehicle to maintain a reasonable speed without sacrificing safety.
|
|
| 26. |
- Narri, Vandana, et al.
(författare)
-
Shared Situational Awareness with V2X Communication and Set-membership Estimation
- 2023
-
Ingår i: 2023 IEEE 26th International Conference on Intelligent Transportation Systems, ITSC 2023. - : Institute of Electrical and Electronics Engineers (IEEE). ; , s. 3335-3342
-
Konferensbidrag (refereegranskat)abstract
- The ability to perceive and comprehend a traffic situation and to estimate the state of the vehicles and road-users in the surrounding of the ego-vehicle is known as situational awareness. Situational awareness for a heavy-duty autonomous vehicle is a critical part of the automation platform and depends on the ego-vehicle's field-of-view. But when it comes to the urban scenarios, the field-of-view of the ego-vehicle is likely to be affected by occlusions and blind spots caused by infrastructure, moving vehicles, and parked vehicles. This paper proposes a framework to improve situational awareness using set-membership estimation and Vehicle-to-Everything (V2X) communication. This framework provides safety guarantees and can adapt to dynamically changing scenarios, and is integrated into an existing complex autonomous platform. A detailed description of the framework implementation and realtime results are illustrated in this paper.
|
|
