| 1. |
- Chang, Yu-Wei, et al.
(författare)
-
Neural network training with highly incomplete medical datasets
- 2022
-
Ingår i: Machine Learning-Science and Technology. - : IOP Publishing. - 2632-2153. ; 3:3
-
Tidskriftsartikel (refereegranskat)abstract
- Neural network training and validation rely on the availability of large high-quality datasets. However, in many cases only incomplete datasets are available, particularly in health care applications, where each patient typically undergoes different clinical procedures or can drop out of a study. Since the data to train the neural networks need to be complete, most studies discard the incomplete datapoints, which reduces the size of the training data, or impute the missing features, which can lead to artifacts. Alas, both approaches are inadequate when a large portion of the data is missing. Here, we introduce GapNet, an alternative deep-learning training approach that can use highly incomplete datasets without overfitting or introducing artefacts. First, the dataset is split into subsets of samples containing all values for a certain cluster of features. Then, these subsets are used to train individual neural networks. Finally, this ensemble of neural networks is combined into a single neural network whose training is fine-tuned using all complete datapoints. Using two highly incomplete real-world medical datasets, we show that GapNet improves the identification of patients with underlying Alzheimer's disease pathology and of patients at risk of hospitalization due to Covid-19. Compared to commonly used imputation methods, this improvement suggests that GapNet can become a general tool to handle incomplete medical datasets.
|
|
| 2. |
- Erbin, Harold, et al.
(författare)
-
Deep multi-task mining Calabi-Yau four-folds
- 2022
-
Ingår i: Machine Learning. - : IOP Publishing Ltd. - 2632-2153. ; 3:1
-
Tidskriftsartikel (refereegranskat)abstract
- We continue earlier efforts in computing the dimensions of tangent space cohomologies of Calabi-Yau manifolds using deep learning. In this paper, we consider the dataset of all Calabi-Yau four-folds constructed as complete intersections in products of projective spaces. Employing neural networks inspired by state-of-the-art computer vision architectures, we improve earlier benchmarks and demonstrate that all four non-trivial Hodge numbers can be learned at the same time using a multi-task architecture. With 30% (80%) training ratio, we reach an accuracy of 100% for h((1,1)) h((2,1)) (100% for both), 81% (96%) for h((3,1)), and 49% (83%) for h((2,2)). Assuming that the Euler number is known, as it is easy to compute, and taking into account the linear constraint arising from index computations, we get 100% total accuracy.
|
|
| 3. |
- Ghielmetti, N., et al.
(författare)
-
Real-time semantic segmentation on FPGAs for autonomous vehicles with hls4ml
- 2022
-
Ingår i: Machine Learning - Science and Technology. - : IOP Publishing. - 2632-2153. ; 3:4
-
Tidskriftsartikel (refereegranskat)abstract
- In this paper, we investigate how field programmable gate arrays can serve as hardware accelerators for real-time semantic segmentation tasks relevant for autonomous driving. Considering compressed versions of the ENet convolutional neural network architecture, we demonstrate a fully-on-chip deployment with a latency of 4.9 ms per image, using less than 30% of the available resources on a Xilinx ZCU102 evaluation board. The latency is reduced to 3 ms per image when increasing the batch size to ten, corresponding to the use case where the autonomous vehicle receives inputs from multiple cameras simultaneously. We show, through aggressive filter reduction and heterogeneous quantization-aware training, and an optimized implementation of convolutional layers, that the power consumption and resource utilization can be significantly reduced while maintaining accuracy on the Cityscapes dataset.
|
|
| 4. |
- Irwin, Ross, et al.
(författare)
-
Chemformer : a pre-trained transformer for computational chemistry
- 2022
-
Ingår i: Machine Learning. - : IOP Publishing Ltd. - 2632-2153. ; 3:1
-
Tidskriftsartikel (refereegranskat)abstract
- Transformer models coupled with a simplified molecular line entry system (SMILES) have recently proven to be a powerful combination for solving challenges in cheminformatics. These models, however, are often developed specifically for a single application and can be very resource-intensive to train. In this work we present the Chemformer model-a Transformer-based model which can be quickly applied to both sequence-to-sequence and discriminative cheminformatics tasks. Additionally, we show that self-supervised pre-training can improve performance and significantly speed up convergence on downstream tasks. On direct synthesis and retrosynthesis prediction benchmark datasets we publish state-of-the-art results for top-1 accuracy. We also improve on existing approaches for a molecular optimisation task and show that Chemformer can optimise on multiple discriminative tasks simultaneously. Models, datasets and code will be made available after publication.
|
|
| 5. |
- Larfors, Magdalena, 1978-, et al.
(författare)
-
Numerical metrics for complete intersection and Kreuzer-Skarke Calabi-Yau manifolds
- 2022
-
Ingår i: Machine Learning. - : Institute of Physics (IOP). - 2632-2153. ; 3:3
-
Tidskriftsartikel (refereegranskat)abstract
- We introduce neural networks (NNs) to compute numerical Ricci-flat Calabi-Yau (CY) metrics for complete intersection and Kreuzer-Skarke (KS) CY manifolds at any point in Kahler and complex structure moduli space, and introduce the package cymetric which provides computation realizations of these techniques. In particular, we develop and computationally realize methods for point-sampling on these manifolds. The training for the NNs is carried out subject to a custom loss function. The Kahler class is fixed by adding to the loss a component which enforces the slopes of certain line bundles to match with topological computations. Our methods are applied to various manifolds, including the quintic manifold, the bi-cubic manifold and a KS manifold with Picard number two. We show that volumes and line bundle slopes can be reliably computed from the resulting Ricci-flat metrics. We also apply our results to compute an approximate Hermitian-Yang-Mills connection on a specific line bundle on the bi-cubic.
|
|
| 6. |
- Storm, Ludvig, 1995, et al.
(författare)
-
Constraints on parameter choices for successful time-series prediction with echo-state networks
- 2022
-
Ingår i: Machine Learning: Science and Technology. - : IOP Publishing. - 2632-2153. ; 3:4
-
Tidskriftsartikel (refereegranskat)abstract
- Echo-state networks are simple models of discrete dynamical systems driven by a time series. By selecting network parameters such that the dynamics of the network is contractive, characterized by a negative maximal Lyapunov exponent, the network may synchronize with the driving signal. Exploiting this synchronization, the echo-state network may be trained to autonomously reproduce the input dynamics, enabling time-series prediction. However, while synchronization is a necessary condition for prediction, it is not sufficient. Here, we study what other conditions are necessary for successful time-series prediction. We identify two key parameters for prediction performance, and conduct a parameter sweep to find regions where prediction is successful. These regions differ significantly depending on whether full or partial phase space information about the input is provided to the network during training. We explain how these regions emerge.
|
|
| 7. |
- Yu, Yi, et al.
(författare)
-
Chemical transformer compression for accelerating both training and inference of molecular modeling
- 2022
-
Ingår i: Machine Learning: Science and Technology. - : IOP Publishing. - 2632-2153. ; 3
-
Tidskriftsartikel (refereegranskat)abstract
- Transformer models have been developed in molecular science with excellent performance in applications including quantitative structure-activity relationship (QSAR) and virtual screening (VS). Compared with other types of models, however, they are large and need voluminous data for training, which results in a high hardware requirement to abridge time for both training and inference processes. In this work, cross-layer parameter sharing (CLPS), and knowledge distillation (KD) are used to reduce the sizes of transformers in molecular science. Both methods not only have competitive QSAR predictive performance as compared to the original BERT model, but also are more parameter efficient. Furthermore, by integrating CLPS and KD into a two-state chemical network, we introduce a new deep lite chemical transformer model, DeLiCaTe. DeLiCaTe accomplishes 4× faster rate for training and inference, due to a 10- and 3-times reduction of the number of parameters and layers, respectively. Meanwhile, the integrated model achieves comparable performance in QSAR and VS, because of capturing general-domain (basic structure) and task-specific knowledge (specific property prediction). Moreover, we anticipate that the model compression strategy provides a pathway to the creation of effective generative transformer models for organic drugs and material design.
|
|
