Sökning: WFRF:(Svensson Jakob 1976 ) >
On the connection b...
On the connection between Noise-Contrastive Estimation and Contrastive Divergence
-
- Olmin, Amanda, 1994- (författare)
- Linköpings universitet,Statistik och maskininlärning,Filosofiska fakulteten,Linköping University
-
- Lindqvist, Jakob, 1992 (författare)
- Chalmers University of Technology, Department of Electrical Engineering, Gothenburg, Sweden,Chalmers tekniska högskola
-
- Svensson, Lennart, 1976 (författare)
- Chalmers University of Technology, Department of Electrical Engineering, Gothenburg, Sweden,Chalmers tekniska högskola
-
visa fler...
-
- Lindsten, Fredrik, 1984- (författare)
- Linköpings universitet,Statistik och maskininlärning,Tekniska fakulteten,Linköping University
-
visa färre...
-
(creator_code:org_t)
- 2024
- 2024
- Engelska.
-
Ingår i: Proceedings of the 38th Conference on Uncertainty in Artificial Intelligence, UAI 2022. - 2640-3498. ; 238, s. 3016-3024
- Relaterad länk:
-
https://urn.kb.se/re...
-
visa fler...
-
https://research.cha...
-
visa färre...
Abstract
Ämnesord
Stäng
- Noise-contrastive estimation (NCE) is a popular method for estimating unnormalised probabilistic models, such as energy-based models, which are effective for modelling complex data distributions. Unlike classical maximum likelihood (ML) estimation that relies on importance sampling (resulting in ML-IS) or MCMC (resulting in contrastive divergence, CD), NCE uses a proxy criterion to avoid the need for evaluating an often intractable normalisation constant. Despite apparent conceptual differences, we show that two NCE criteria, ranking NCE (RNCE) and conditional NCE (CNCE), can be viewed as ML estimation methods. Specifically, RNCE is equivalent to ML estimation combined with conditional importance sampling, and both RNCE and CNCE are special cases of CD. These findings bridge the gap between the two method classes and allow us to apply techniques from the ML-IS and CD literature to NCE, offering several advantageous extensions.
Ämnesord
- NATURVETENSKAP -- Data- och informationsvetenskap (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences (hsv//eng)
- TEKNIK OCH TEKNOLOGIER -- Elektroteknik och elektronik (hsv//swe)
- ENGINEERING AND TECHNOLOGY -- Electrical Engineering, Electronic Engineering, Information Engineering (hsv//eng)
Nyckelord
- Unnormalised models
- noise-contrastive estimation
- contrastive divergence
Publikations- och innehållstyp
- ref (ämneskategori)
- kon (ämneskategori)
Hitta via bibliotek
Till lärosätets databas