Search: onr:"swepub:oai:DiVA.org:kth-327154" >
The Role Of Pretrai...
The Role Of Pretrained Representations For The Ood Generalization Of Rl Agents
-
Träuble, F. (author)
-
Dittadi, A. (author)
-
Wüthrich, M. (author)
-
show more...
-
Widmaier, F. (author)
-
Gehler, P. (author)
-
Winther, O. (author)
-
Locatello, F. (author)
-
Bachem, O. (author)
-
Schölkopf, B. (author)
-
- Bauer, Stefan (author)
- KTH,Intelligenta system,Max Planck Institute for Intelligent Systems, Tübingen, Germany
-
show less...
-
(creator_code:org_t)
- International Conference on Learning Representations, ICLR, 2022
- 2022
- English.
-
In: ICLR 2022 - 10th International Conference on Learning Representations. - : International Conference on Learning Representations, ICLR.
- Related links:
-
https://urn.kb.se/re...
Abstract
Subject headings
Close
- Building sample-efficient agents that generalize out-of-distribution (OOD) in real-world settings remains a fundamental unsolved problem on the path towards achieving higher-level cognition. One particularly promising approach is to begin with low-dimensional, pretrained representations of our world, which should facilitate efficient downstream learning and generalization. By training 240 representations and over 10,000 reinforcement learning (RL) policies on a simulated robotic setup, we evaluate to what extent different properties of pretrained VAE-based representations affect the OOD generalization of downstream agents. We observe that many agents are surprisingly robust to realistic distribution shifts, including the challenging sim-to-real case. In addition, we find that the generalization performance of a simple downstream proxy task reliably predicts the generalization performance of our RL agents under a wide range of OOD settings. Such proxy tasks can thus be used to select pretrained representations that will lead to agents that generalize.
Subject headings
- NATURVETENSKAP -- Data- och informationsvetenskap -- Datavetenskap (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Computer Sciences (hsv//eng)
Keyword
- Down-stream
- Generalisation
- Generalization performance
- High-level cognition
- Learning policy
- Low dimensional
- Real world setting
- Reinforcement learning agent
- Reinforcement learnings
- Unsolved problems
Publication and Content Type
- ref (subject category)
- kon (subject category)
To the university's database
- By the author/editor
-
Träuble, F.
-
Dittadi, A.
-
Wüthrich, M.
-
Widmaier, F.
-
Gehler, P.
-
Winther, O.
-
show more...
-
Locatello, F.
-
Bachem, O.
-
Schölkopf, B.
-
Bauer, Stefan
-
show less...
- About the subject
-
- NATURAL SCIENCES
-
NATURAL SCIENCES
-
and Computer and Inf ...
-
and Computer Science ...
- Articles in the publication
- ICLR 2022 - 10th ...
- By the university
-
Royal Institute of Technology