| 1. |
- Kammonen, Aku, 1984-, et al.
(författare)
-
Adaptive random fourier features with metropolis sampling
- 2019
-
Ingår i: Foundations of Data Science. - : American Institute of Mathematical Sciences. - 2639-8001. ; 0:0, s. 0-0
-
Tidskriftsartikel (refereegranskat)abstract
- The supervised learning problem todetermine a neural network approximation $\mathbb{R}^d\ni x\mapsto\sum_{k=1}^K\hat\beta_k e^{{\mathrm{i}}\omega_k\cdot x}$with one hidden layer is studied asa random Fourier features algorithm. The Fourier features, i.e., the frequencies $\omega_k\in\mathbb{R}^d$,are sampled using an adaptive Metropolis sampler.The Metropolis test accepts proposal frequencies $\omega_k'$, having corresponding amplitudes $\hat\beta_k'$, with the probability$\min\big\{1, (|\hat\beta_k'|/|\hat\beta_k|)^\gamma\big\}$,for a certain positive parameter $\gamma$, determined by minimizing the approximation error for given computational work.This adaptive, non-parametric stochastic method leads asymptotically, as $K\to\infty$, to equidistributed amplitudes $|\hat\beta_k|$, analogous to deterministic adaptive algorithms for differential equations. The equidistributed amplitudes are shown to asymptotically correspond to the optimal density for independent samples in random Fourier features methods.Numerical evidence is provided in order to demonstrate the approximation properties and efficiency of the proposed algorithm. The algorithm is testedboth on synthetic data and a real-world high-dimensional benchmark.
|
|
| 2. |
- Kammonen, Aku, 1984-, et al.
(författare)
-
Canonical quantum observables for molecular systems approximated by ab initio molecular dynamics
- 2018
-
Ingår i: Annales Henri Poincaré. - : Springer Nature. - 1424-0637 .- 1424-0661. ; 19, s. 2727-2781
-
Tidskriftsartikel (refereegranskat)abstract
- It is known that ab initio molecular dynamics based on the electron ground state eigenvaluecan be used to approximate quantum observables in the canonical ensemble when the temperature is low compared tothe first electron eigenvalue gap. This work proves that a certain weighted average of the different ab initio dynamics, corresponding to each electron eigenvalue, approximates quantum observables for any temperature.The proof uses the semi-classical Weyl law to show thatcanonical quantum observables of nuclei-electron systems, based on matrix valued Hamiltonian symbols, can be approximated by ab initio molecular dynamics with the error proportional to the electron-nuclei mass ratio. The resultincludes observables that depend on correlations in time. A combination of the Hilbert-Schmidt inner product for quantum operators and Weyl's lawshows that the error estimate holds %for observables and Hamiltonian symbols that have three and five bounded derivatives, respectively, provided the electron eigenvalues are distinct for any nuclei positionand the observables are in diagonal form with respect to the electron eigenstates.
|
|
| 3. |
|
|
| 4. |
- Kammonen, Aku, 1984-, et al.
(författare)
-
Canonical quantum observables for molecular systems approximated by ab inition molecular dynamics
-
Annan publikation (övrigt vetenskapligt/konstnärligt)abstract
- It is known that ab initio molecular dynamics based on the electron ground state eigenvaluecan be used to approximate quantum observables in the canonical ensemble when the temperature is low compared tothe first electron eigenvalue gap. This work proves that a certain weighted average of the different ab initio dynamics, corresponding to each electron eigenvalue, approximates quantum observables for any temperature.The proof uses the semi-classical Weyl law to show thatcanonical quantum observables of nuclei-electron systems, based on matrix valued Hamiltonian symbols, can be approximated by ab initio molecular dynamics with the error proportional to the electron-nuclei mass ratio. The resultincludes observables that depend on correlations in time. A combination of the Hilbert-Schmidt inner product for quantum operators and Weyl's lawshows that the error estimate holds %for observables and Hamiltonian symbols that have three and five bounded derivatives, respectively, provided the electron eigenvalues are distinct for any nuclei positionand the observables are in diagonal form with respect to the electron eigenstates.
|
|
| 5. |
- Kammonen, Aku, 1984-, et al.
(författare)
-
COMPUTATIONAL ALGORITHMS FOR CANONICAL ENSEMBLE OBSERVABLES
-
Annan publikation (övrigt vetenskapligt/konstnärligt)abstract
- We consider canonical ensembles of molecular systems. We propose four numerical algorithms for efficient computation of the canonical ensemble molecular dynamics observables. The four algorithms can each be efficient in different situations. For example in low temperatures we can make use of the fact that the lowest electron energy levels contributes most to the observable.
|
|
| 6. |
- Kammonen, Aku, 1984-
(författare)
-
Numerical algorithms for high dimensional integration with application to machine learning and molecular dynamics
- 2021
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- This thesis contains results on high dimensional integration with two papers, paper I and paper II, presenting applications in machine learning and two papers, paper III and paper IV, presenting applications to molecular dynamics.In paper I we present algorithms based on a Metropolis test for training shallow neural networks with trigonometric activation functions. Numerical experiments are performed on both synthetic and real data. The trigonometric activation function gives access to the Fourier transform and its inverse transform. The algorithms gives equidistributed amplitudes.In paper II we derive smaller generalization error for deep residual neural networks compared to shallow ones. An algorithm that builds the residual neural network layer by layer based on an algorithm from paper I is presented both as a stand alone algorithm as well as a pre-step for a global optimizer like Stochastic gradient descent or Adam. Numerical test are performed with promising results.In paper III we make use of the semiclassical Weyl law to show that canonical quantum observables can be approximated by molecular dynamics with an error rate proportional to the electron-nuclei mass ratio. Numerical experiments are presented that confirms the expected theoretical result.In paper IV we consider canonical ensembles of molecular systems. We propose four numerical algorithms for efficient computation of the canonical ensemble molecular dynamics observables. The four algorithms can each be efficient in different situations. For example in low temperatures we can make use of the fact that the lowest electron energy levels contributes most to the observable. The work is an extension of the results in paper III.
|
|
| 7. |
- Kammonen, Aku, 1984-, et al.
(författare)
-
SMALLER GENERALIZATION ERROR DERIVED FOR DEEP COMPARED TO SHALLOW RESIDUAL NEURAL NETWORKS
-
Annan publikation (övrigt vetenskapligt/konstnärligt)abstract
- Estimates of the generalization error are proved for a residual neural network with $L$ random Fourier features layers $\bar z_{\ell+1}=\bar z_\ell + \mathrm{Re}\sum_{k=1}^K\bar b_{\ell k}e^{\mathrm{i}\omega_{\ell k}\bar z_\ell}+\mathrm{Re}\sum_{k=1}^K\bar c_{\ell k}e^{\mathrm{i}\omega'_{\ell k}\cdot x}$. An optimal distribution for the frequencies $(\omega_{\ell k},\omega'_{\ell k})$ of the random Fourier features $e^{\mathrm{i}\omega_{\ell k}\bar z_\ell}$ and $e^{\mathrm{i}\omega'_{\ell k}\cdot x}$ is derived. This derivation is based on the corresponding generalization error for the approximation of the function values $f(x)$. The generalization error turns out to be smaller than the estimate ${\|\hat f\|^2_{L^1(\mathbb{R}^d)}}/{(LK)}$ of the generalization error for random Fourier features with one hidden layer and the same total number of nodes $LK$, in the case the $L^\infty$-norm of $f$ is much less than the $L^1$-norm of its Fourier transform $\hat f$. This understanding of an optimal distribution for random features is used to construct a new training method for a deep residual network that shows promising results.
|
|
