Sökning: onr:"swepub:oai:research.chalmers.se:cc38874c-a8bb-4513-82cb-7523c2496b3c" >
Automatic Different...
Automatic Differentiation of Programs with Discrete Randomness
-
- Arya, Gaurav (författare)
- Massachusetts Institute of Technology (MIT)
-
- Schauer, Moritz, 1980 (författare)
- Chalmers tekniska högskola,Chalmers University of Technology,Göteborgs universitet,University of Gothenburg
-
- Schäfer, Frank (författare)
- Massachusetts Institute of Technology (MIT)
-
visa fler...
-
- Rackauckas, Christopher Vincent (författare)
- Massachusetts Institute of Technology (MIT)
-
visa färre...
-
(creator_code:org_t)
- 2022
- 2022
- Engelska.
-
Ingår i: Advances in Neural Information Processing Systems. - 1049-5258. ; 35
- Relaterad länk:
-
https://proceedings....
-
visa fler...
-
https://research.cha...
-
https://doi.org/10.4...
-
visa färre...
Abstract
Ämnesord
Stäng
- Automatic differentiation (AD), a technique for constructing new programs which compute the derivative of an original program, has become ubiquitous throughout scientific computing and deep learning due to the improved performance afforded by gradient-based optimization. However, AD systems have been restricted to the subset of programs that have a continuous dependence on parameters. Programs that have discrete stochastic behaviors governed by distribution parameters, such as flipping a coin with probability p of being heads, pose a challenge to these systems because the connection between the result (heads vs tails) and the parameters (p) is fundamentally discrete. In this paper we develop a new reparameterization-based methodology that allows for generating programs whose expectation is the derivative of the expectation of the original program. We showcase how this method gives an unbiased and low-variance estimator which is as automated as traditional AD mechanisms. We demonstrate unbiased forward-mode AD of discrete-time Markov chains, agent-based models such as Conway's Game of Life, and unbiased reverse-mode AD of a particle filter. Our code package is available at https://github.com/gaurav-arya/StochasticAD.jl.
Ämnesord
- NATURVETENSKAP -- Matematik -- Beräkningsmatematik (hsv//swe)
- NATURAL SCIENCES -- Mathematics -- Computational Mathematics (hsv//eng)
- NATURVETENSKAP -- Matematik -- Sannolikhetsteori och statistik (hsv//swe)
- NATURAL SCIENCES -- Mathematics -- Probability Theory and Statistics (hsv//eng)
- TEKNIK OCH TEKNOLOGIER -- Elektroteknik och elektronik -- Reglerteknik (hsv//swe)
- ENGINEERING AND TECHNOLOGY -- Electrical Engineering, Electronic Engineering, Information Engineering -- Control Engineering (hsv//eng)
- NATURVETENSKAP -- Matematik -- Matematisk analys (hsv//swe)
- NATURAL SCIENCES -- Mathematics -- Mathematical Analysis (hsv//eng)
Nyckelord
- differentiable stochastic programming
- stochastic methods
- gradient based inference
- compositionality
- discrete randomness
- reparameterization trick
- automatic differentiation
- chain rule
Publikations- och innehållstyp
- kon (ämneskategori)
- ref (ämneskategori)
Hitta via bibliotek
Till lärosätets databas