1. |
- Gomariz, Alvaro, et al.
(author)
-
Modality attention and sampling enables deep learning with heterogeneous marker combinations in fluorescence microscopy
- 2021
-
In: Nature Machine Intelligence. - : Springer Nature. - 2522-5839. ; 3:9, s. 799-811
-
Journal article (peer-reviewed)abstract
- Fluorescence microscopy allows for a detailed inspection of cells, cellular networks and anatomical landmarks by staining with a variety of carefully selected markers visualized as colour channels. Quantitative characterization of structures in acquired images often relies on automatic image analysis methods. Despite the success of deep learning methods in other vision applications, their potential for fluorescence image analysis remains underexploited. One reason lies in the considerable workload required to train accurate models, which are normally specific for a given combination of markers and therefore applicable to a very restricted number of experimental settings. We herein propose ‘marker sampling and excite’—a neural network approach with a modality sampling strategy and a novel attention module that together enable (1) flexible training with heterogeneous datasets with combinations of markers and (2) successful utility of learned models on arbitrary subsets of markers prospectively. We show that our single neural network solution performs comparably to an upper bound scenario in which an ensemble of many networks is naively trained for each possible marker combination separately. We also demonstrate the feasibility of this framework in high-throughput biological analysis by revising a recent quantitative characterization of bone-marrow vasculature in three-dimensional confocal microscopy datasets and further confirm the validity of our approach on another substantially different dataset of microvessels in foetal liver tissues. Not only can our work substantially ameliorate the use of deep learning in fluorescence microscopy analysis, but it can also be utilized in other fields with incomplete data acquisitions and missing modalities.
|
|
2. |
- Gomariz, Alvaro, et al.
(author)
-
Probabilistic spatial analysis in quantitative microscopy with uncertainty-aware cell detection using deep Bayesian regression
- 2022
-
In: Science Advances. - : American Association for the Advancement of Science (AAAS). - 2375-2548. ; 8:5
-
Journal article (peer-reviewed)abstract
- The investigation of biological systems with three-dimensional microscopy demands automatic cell identification methods that not only are accurate but also can imply the uncertainty in their predictions. The use of deep learning to regress density maps is a popular successful approach for extracting cell coordinates from local peaks in a postprocessing step, which then, however, hinders any meaningful probabilistic output. We propose a framework that can operate on large microscopy images and output probabilistic predictions (i) by integrating deep Bayesian learning for the regression of uncertainty-aware density maps, where peak detection algorithms generate cell proposals, and (ii) by learning a mapping from prediction proposals to a probabilistic space that accurately represents the chances of a successful prediction. Using these calibrated predictions, we propose a probabilistic spatial analysis with Monte Carlo sampling. We demonstrate this in a bone marrow dataset, where our proposed methods reveal spatial patterns that are otherwise undetectable.
|
|
3. |
|
|
4. |
- Gomariz, Alvaro, et al.
(author)
-
Unsupervised Domain Adaptation with Contrastive Learning for OCT Segmentation
- 2022
-
In: Medical Image Computing and Computer Assisted Intervention, MICCAI 2022, pt viii. - Cham : Springer Nature. - 9783031164521 - 9783031164514 ; , s. 351-361
-
Conference paper (peer-reviewed)abstract
- Accurate segmentation of retinal fluids in 3D Optical Coherence Tomography images is key for diagnosis and personalized treatment of eye diseases. While deep learning has been successful at this task, trained supervised models often fail for images that do not resemble labeled examples, e.g. for images acquired using different devices. We hereby propose a novel semi-supervised learning framework for segmentation of volumetric images from new unlabeled domains. We jointly use supervised and contrastive learning, also introducing a contrastive pairing scheme that leverages similarity between nearby slices in 3D. In addition, we propose channel-wise aggregation as an alternative to conventional spatial-pooling aggregation for contrastive feature map projection. We evaluate our methods for domain adaptation from a (labeled) source domain to an (unlabeled) target domain, each containing images acquired with different acquisition devices. In the target domain, our method achieves a Dice coefficient 13.8% higher than SimCLR (a state-of-the-art contrastive framework), and leads to results comparable to an upper bound with supervised training in that domain. In the source domain, our model also improves the results by 5.4% Dice, by successfully leveraging information from many unlabeled images.
|
|
5. |
- Gomariz, Alvaro, et al.
(author)
-
Utilizing Uncertainty Estimation in Deep Learning Segmentation of Fluorescence Microscopy Images with Missing Markers
- 2021
-
Conference paper (peer-reviewed)abstract
- Fluorescence microscopy images contain several channels,each indicating a marker staining the sample. Since manydifferent marker combinations are utilized in practice, it hasbeen challenging to apply deep learning based segmentationmodels, which expect a predefined channel combination forall training samples as well as at inference for future applica-tion. Recent work circumvents this problem using a modalityattention approach to be effective across any possible markercombination. However, for combinations that do not existin a labeled training dataset, one cannot have any estimationof potential segmentation quality if that combination is en-countered during inference. Without this, not only one lacksquality assurance but one also does not know where to put anyadditional imaging and labeling effort. We herein propose amethod to estimate segmentation quality on unlabeled imagesby (i) estimating both aleatoric and epistemic uncertainties ofconvolutional neural networks for image segmentation, and(ii) training a Random Forest model for the interpretationof uncertainty features via regression to their correspond-ing segmentation metrics. Additionally, we demonstrate thatincluding these uncertainty measures during training canprovide an improvement on segmentation performance.
|
|