Search: id:"swepub:oai:DiVA.org:kth-280121" >
Robust learning and...
-
Ferizbegovic, MinaKTH,Reglerteknik,ACCESS Linnaeus Centre
(author)
Robust learning and control of linear dynamical systems
Publisher, publication year, extent ...
-
Stockholm, Sweden :Kungliga Tekniska högskolan,2020
-
electronicrdacarrier
Numbers
-
LIBRIS-ID:oai:DiVA.org:kth-280121
-
ISBN:9789178736287
-
https://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-280121URI
Supplementary language notes
-
Language:English
-
Summary in:English
Part of subdatabase
Classification
-
Subject category:vet swepub-contenttype
-
Subject category:lic swepub-publicationtype
Notes
-
QC 20200904
-
We consider the linear quadratic regulation problem when the plant is an unknown linear dynamical system. We present robust model-based methods based on convex optimization, which minimize the worst-case cost with respect to uncertainty around model estimates. To quantify uncertainty, we derive a methodbased on Bayesian inference, which is directly applicable to robust control synthesis.We focus on control policies that can be iteratively updated after sequentially collecting data. More specifically, we seek to design control policies that balance exploration (reducing model uncertainty) and exploitation (control of the system) when exploration must be safe (robust).First, we derive a robust controller to minimize the worst-case cost, with high probability, given the empirical observation of the system. This robust controller synthesis is then used to derive a robust dual controller, which updates its control policy after collecting data. An episode in which data is collected is called exploration, and the episode using an updated control policy is exploitation. The objective is to minimize the worst-case cost of the updated control policy, requiring that a given exploration budget constrains the worst-case cost during exploration.We look into robust dual control in both finite and infinite horizon settings. The main difference between the finite and infinite horizon settings is that the latter does not consider the length of the exploration and exploitation phase, but it rather approximates the cost using the infinite horizon cost. In the finite horizon setting, we discuss how different exploration lengths affect the trade-off between exploration and exploitation.Additionally, we derive methods that balance exploration and exploitation to minimize the cumulative worst-case cost for a fixed number of episodes. In this thesis, we refer to such a problem as robust reinforcement learning. Essentially, it is a robust dual controller aiming to minimize the cumulative worst-case cost, and that updates its control policy in each episode.Numerical experiments show that the proposed methods have better performance compared to existing state-of-the-art algorithms. Moreover, experiments also indicate that the exploration prioritizes the uncertainty reduction in the parameters that matter most for control.
Subject headings and genre
Added entries (persons, corporate bodies, meetings, titles ...)
-
Hjalmarsson, Håkan,1962-KTH,Signaler, sensorer och system,Reglerteknik(Swepub:kth)u10a8l40
(thesis advisor)
-
Schön, Thomas B.
(thesis advisor)
-
Rojas, Cristian R.,1980-KTH,Reglerteknik(Swepub:kth)u10v97p2
(thesis advisor)
-
Dörfler, Florian
(opponent)
-
KTHReglerteknik
(creator_code:org_t)
Internet link
Find in a library
To the university's database