Sökning: onr:"swepub:oai:research.chalmers.se:989c7766-3020-453a-b627-2ee32236afe3" >
Effortless Training...
Effortless Training of Joint Energy-Based Models with Sliced Score Matching
-
- Liu, Xixi, 1995 (författare)
- Chalmers tekniska högskola,Chalmers University of Technology
-
- Staudt, Dorian, 1993 (författare)
- Chalmers tekniska högskola,Chalmers University of Technology
-
- Lin, Che-Tsung, 1979 (författare)
- Chalmers tekniska högskola,Chalmers University of Technology
-
visa fler...
-
- Zach, Christopher, 1974 (författare)
- Chalmers tekniska högskola,Chalmers University of Technology
-
visa färre...
-
(creator_code:org_t)
- ISBN 9781665490627
- 2022
- 2022
- Engelska.
-
Ingår i: Proceedings - International Conference on Pattern Recognition. - 1051-4651. - 9781665490627 ; 2022-August, s. 2643-2649
- Relaterad länk:
-
https://doi.ieeecomp...
-
visa fler...
-
https://doi.org/10.1...
-
https://research.cha...
-
visa färre...
Abstract
Ämnesord
Stäng
- Standard discriminative classifiers can be upgraded to joint energy-based models (JEMs) by combining the classification loss with a log-evidence loss. Hence, such models intrinsically allow detection of out-of-distribution (OOD) samples, and empirically also provide better-calibrated posteriors, i.e., prediction uncertainties. However, the training procedure suggested for JEMs (using stochastic gradient Langevin dynamics---or SGLD---to maximize the evidence) is reported to be brittle. In this work, we propose to utilize score matching---in particular sliced score matching---to obtain a stable training method for JEMs. We observe empirically that the combination of score matching with the standard classification loss leads to improved OOD detection and better-calibrated classifiers for otherwise identical DNN architectures. Additionally, we also analyze the impact of replacing the regular soft-max layer for classification with a gated soft-max one in order to improve the intrinsic transformation invariance and generalization ability.
Ämnesord
- NATURVETENSKAP -- Matematik -- Sannolikhetsteori och statistik (hsv//swe)
- NATURAL SCIENCES -- Mathematics -- Probability Theory and Statistics (hsv//eng)
- NATURVETENSKAP -- Data- och informationsvetenskap -- Datorseende och robotik (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Computer Vision and Robotics (hsv//eng)
Publikations- och innehållstyp
- kon (ämneskategori)
- ref (ämneskategori)
Hitta via bibliotek
Till lärosätets databas