Bayesian Posterior Approximation With Stochastic Ensembles

• We introduce ensembles of stochastic neural networks to approximate the Bayesian posterior, combining stochastic methods such as dropout with deep ensembles. The stochas-tic ensembles are formulated as families of distributions and trained to approximate the Bayesian posterior with variational inference. We implement stochastic ensembles based on Monte Carlo dropout, DropConnect and a novel non-parametric version of dropout and evaluate them on a toy problem and CIFAR image classification. For both tasks, we test the quality of the posteriors directly against Hamil-tonian Monte Carlo simulations. Our results show that stochastic ensembles provide more accurate posterior esti-mates than other popular baselines for Bayesian inference.

Ämnesord

NATURVETENSKAP  -- Matematik -- Beräkningsmatematik (hsv//swe)

NATURAL SCIENCES  -- Mathematics -- Computational Mathematics (hsv//eng)

NATURVETENSKAP  -- Matematik -- Sannolikhetsteori och statistik (hsv//swe)

NATURAL SCIENCES  -- Mathematics -- Probability Theory and Statistics (hsv//eng)

Nyckelord

Deep learning architectures and techniques

Deep learning architectures and techniques

Publikations- och innehållstyp

kon (ämneskategori)

ref (ämneskategori)