Sökning: WFRF:(Durisi Giuseppe 1977) >
Conditional Mutual ...
Conditional Mutual Information-Based Generalization Bound for Meta Learning
-
- Rezazadeh, Arezou, 1987 (författare)
- Chalmers tekniska högskola,Chalmers University of Technology
-
- Jose, S.T. (författare)
- King's College London
-
- Durisi, Giuseppe, 1977 (författare)
- Chalmers tekniska högskola,Chalmers University of Technology
-
visa fler...
-
Simeone, Osvaldo, 1977 (författare)
-
visa färre...
-
(creator_code:org_t)
- 2021
- 2021
- Engelska.
-
Ingår i: IEEE International Symposium on Information Theory - Proceedings. - 2157-8095. ; 2021-July, s. 1176-1181
- Relaterad länk:
-
https://research.cha... (primary) (free)
-
visa fler...
-
https://research.cha...
-
https://doi.org/10.1...
-
visa färre...
Abstract
Ämnesord
Stäng
- Meta-learning optimizes an inductive bias—typically in the form of the hyperparameters of a base-learning algorithm—by observing data from a finite number of related tasks. This paper presents an information-theoretic bound on the generalization performance of any given meta-learner, which builds on the conditional mutual information (CMI) framework of Steinke and Zakynthinou (2020). In the proposed extension to meta-learning, the CMI bound involves a training meta-supersample obtained by first sampling 2N independent tasks from the task environment, and then drawing 2M independent training samples for each sampled task. The meta-training data fed to the meta-learner is modelled as being obtained by randomly selecting N tasks from the available 2N tasks and M training samples per task from the available 2M training samples per task. The resulting bound is explicit in two CMI terms, which measure the information that the meta-learner output and the base-learner output provide about which training data are selected, given the entire meta-supersample. Finally, we present a numerical example that illustrates the merits of the proposed bound in comparison to prior information-theoretic bounds for meta-learning
Ämnesord
- NATURVETENSKAP -- Data- och informationsvetenskap -- Annan data- och informationsvetenskap (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Other Computer and Information Science (hsv//eng)
- SAMHÄLLSVETENSKAP -- Utbildningsvetenskap -- Lärande (hsv//swe)
- SOCIAL SCIENCES -- Educational Sciences -- Learning (hsv//eng)
- NATURVETENSKAP -- Matematik -- Sannolikhetsteori och statistik (hsv//swe)
- NATURAL SCIENCES -- Mathematics -- Probability Theory and Statistics (hsv//eng)
Publikations- och innehållstyp
- kon (ämneskategori)
- ref (ämneskategori)
Hitta via bibliotek
Till lärosätets databas