| 1. |
- Gupta, Ankit
(författare)
-
Adapting Deep Learning for Microscopy: Interaction, Application, and Validation
- 2023
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Microscopy is an integral technique in biology to study the fundamental components of life visually. Digital microscopy and automation have enabled biologists to conduct faster and larger-scale experiments with a sharp increase in the data generated. Microscopy images contain rich but sparse information, as typically, only small regions in the images are relevant for further study. Image analysis is a crucial tool for biologists in the objective interpretation and extraction of quantitative measurements from microscopy data. Recently, deep learning techniques have shown superior performance in various image analysis tasks. The models learn feature representations from the data by optimizing for a task. However, the techniques require a significant amount of annotated data to perform well. Domain experts are required to annotate microscopy data, making it expensive and time-consuming. The models offer no insight into their prediction, and the learned features are not directly interpretable. This poses challenges to the reliable utilization of the technique in high-trust applications such as drug discovery or disease detection. High data variability in microscopy and poor generalization performance of deep learning models further increase the difficulty in general usage of the technique. The work in this thesis presents frameworks and methods to solve the practical challenges of applying deep learning in microscopy. The application-specific evaluation approaches were presented to validate the approaches, aiming to increase trust in the system. The major contributions of this work are as follows. Papers I and III present human-in-the-loop frameworks for quick adaption of deep learning to new data and for improving models' performance based on human input in visual explanations provided by the model, respectively. Paper II proposes a template-matching approach to improve user interactions in the framework proposed in Paper I. Papers III and IV present architectural modifications in the deep learning models proposed for better visual explanation and image-to-image translation, respectively. Papers IV and V present biologically relevant evaluations of approaches, i.e., analysis of the deep learning models in relation to the biological task.This thesis is aimed towards better utilization and adaptation of the DL methods and techniques to the microscopy data. We show that the annotation burden for the user can be significantly reduced by intuitive annotation frameworks and using contemporary deep-learning paradigms. We further propose architectural modifications in the models to adapt to the requirements and demonstrate the utility of application-specific analysis in microscopy.
|
|
| 2. |
- Harrison, Philip John, 1977-
(författare)
-
Deep learning approaches for image cytometry: assessing cellular morphological responses to drug perturbations
- 2023
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Image cytometry is the analysis of cell properties from microscopy image data and is used ubiquitously in basic cell biology, medical diagnosis and drug development. In recent years deep learning has shown impressive results for many image cytometry tasks, including image processing, segmentation, classification and detection. Deep learning enables a more data-driven and end-to-end approach than was previously possible with conventional methods. This thesis investigates deep learning-based approaches for assessing cellular morphological responses to drug perturbations. In paper I we demonstrated the benefit of combining convolutional neural networks and transfer learning for predicting mechanism of action and nucleus translocation. In paper II we showed, using convolutional and recurrent neural networks applied to time-lapse microscopy data, that it is possible to predict if mRNA delivery via nanoparticles has been effective based on cell morphology changes at time points prior to the protein production evidence of successful delivery. In paper III we used convolutional neural networks, adversarial training and privileged information to faithfully generate fluorescence imaging channels of adipocyte cells from their corresponding z-stack of brightfield images. Our models were both faithful at the fluorescence image level and at the level of the features extracted from these images, features that are commonly used for downstream analysis, including the design of effective drug therapies. In paper IV we showed that convolutional neural networks trained on brightfield image data provide similar, and in some cases superior, performance to models trained on fluorescence image data for predicting mechanism of action, due to the brightfield images possessing additional information not available in the fluorescence images. In paper V we applied deep learning models to brightfield time-lapse image data to explore the evolution of cellular morphological changes after drug administration for a diverse set of compounds, compounds that are often used as positive controls in image-based assays.
|
|
| 3. |
- Partel, Gabriele, 1988-
(författare)
-
Image and Data Analysis for Spatially Resolved Transcriptomics : Decrypting fine-scale spatial heterogeneity of tissue's molecular architecture
- 2020
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Our understanding of the biological complexity in multicellular organisms has progressed at tremendous pace in the last century and even more in the last decades with the advent of sequencing technologies that make it possible to interrogate the genome and transcriptome of individual cells. It is now possible to even spatially profile the transcriptomic landscape of tissue architectures to study the molecular organization of tissue heterogeneity at subcellular resolution. Newly developed spatially resolved transcriptomic techniques are producing large amounts of high-dimensional image data with increasing throughput, that need to be processed and analysed for extracting biological relevant information that has the potential to lead to new knowledge and discoveries. The work included in this thesis aims to provide image and data analysis tools for serving this new developing field of spatially resolved transcriptomics to fulfill its purpose. First, an image analysis workflow is presented for processing and analysing images acquired with in situ sequencing protocols, aiming to extract and decode molecular features that map the spatial transcriptomic landscape in tissue sections. This thesis also presents computational methods to explore and analyse the decoded spatial gene expression for studying the spatial molecular heterogeneity of tissue architectures at different scales. In one case, it is demonstrated how dimensionality reduction and clustering of the decoded gene expression spatial profiles can be exploited and used to identify reproducible spatial compartments corresponding to know anatomical regions across mouse brain sections from different individuals. And lastly, this thesis presents an unsupervised computational method that leverages advanced deep learning techniques on graphs to model the spatial gene expression at cellular and subcellular resolution. It provides a low dimensional representation of spatial organization and interaction, finding functional units that in many cases correspond to different cell types in the local tissue environment, without the need for cell segmentation.
|
|
| 4. |
- Wieslander, Håkan
(författare)
-
Application, Optimisation and Evaluation of Deep Learning for Biomedical Imaging
- 2022
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Microscopy imaging is a powerful technique when studying biology at a cellular and sub-cellular level. When combined with digital image analysis it creates an invaluable tool for investigating complex biological processes and phenomena. However, imaging at the cell and sub-cellular level tends to generate large amounts of data which can be difficult to analyse, navigate and store. Despite these difficulties, large data volumes mean more information content which is beneficial for computational methods like machine learning, especially deep learning. The union of microscopy imaging and deep learning thus provides numerous opportunities for advancing our scientific understanding and uncovering interesting and useful biological insights.The work in this thesis explores various means for optimising information extraction from microscopy data utilising image analysis with deep learning. The focus is on three different imaging modalities: bright-field; fluorescence; and transmission electron microscopy. Within these modalities different learning-based image analysis and processing techniques are explored, ranging from image classification and detection to image restoration and translation. The main contributions are: (i) a computational method for diagnosing oral and cervical cancer based on smear samples and bright-field microscopy; (ii) a hierarchical analysis of whole-slide tissue images from fluorescence microscopy and introducing a confidence based measure for pixel classifications; (iii) an image restoration model for motion-degraded images from transmission electron microscopy with an evaluation of model overfitting on underlying textures; and (iv) an image-to-image translation (virtual staining) of cell images from bright-field to fluorescence microscopy, optimised for biological feature relevance. A common theme underlying all the investigations in this thesis is that the evaluation of the methods used is in relation to the biological question at hand.
|
|
| 5. |
- Andersson, Axel
(författare)
-
Computational Methods for Image-Based Spatial Transcriptomics
- 2024
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Why does cancer develop, spread, grow, and lead to mortality? To answer these questions, one must study the fundamental building blocks of all living organisms — cells. Like a well-calibrated manufacturing unit, cells follow precise instructions by gene expression to initiate the synthesis of proteins, the workforces that drive all living biochemical processes.Recently, researchers have developed techniques for imaging the expression of hundreds of unique genes within tissue samples. This information is extremely valuable for understanding the cellular activities behind cancer-related diseases. These methods, collectively known as image-based spatial transcriptomics (IST) techniques, use fluorescence microscopy to combinatorically label mRNA species (corresponding to expressed genes) in tissue samples. Here, automatic image analysis is required to locate fluorescence signals and decode the combinatorial code. This process results in large quantities of points, marking the location of expressed genes. These new data formats pose several challenges regarding visualization and automated analysis.This thesis presents several computational methods and applications related to data generated from IST methods. Key contributions include: (i) A decoding method that jointly optimizes the detection and decoding of signals, particularly beneficial in scenarios with low signal-to-noise ratios or densely packed signals; (ii) a computational method for automatically delineating regions with similar gene compositions — efficient, interactive, and scalable for exploring patterns across different scales; (iii) a software enabling interactive visualization of millions of gene markers atop Terapixel-sized images (TissUUmaps); (iv) a tool utilizing signed-graph partitioning for the automatic identification of cells, independent of the complementary nuclear stain; (v) A fast and analytical expression for a score that quantifies co-localization between spatial points (such as located genes); (vi) a demonstration that gene expression markers can train deep-learning models to classify tissue morphology.In the final contribution (vii), an IST technique features in a clinical study to spatially map the molecular diversity within tumors from patients with colorectal liver metastases, specifically those exhibiting a desmoplastic growth pattern. The study unveils novel molecular patterns characterizing cellular diversity in the transitional region between healthy liver tissue and the tumor. While a direct answer to the initial questions remains elusive, this study sheds illuminating insights into the growth dynamics of colorectal cancer liver metastases, bringing us closer to understanding the journey from development to mortality in cancer.
|
|
| 6. |
- Blamey, Ben, et al.
(författare)
-
Rapid development of cloud-native intelligent data pipelines for scientific data streams using the HASTE Toolkit
- 2021
-
Ingår i: GigaScience. - : Oxford University Press. - 2047-217X. ; 10:3, s. 1-14
-
Tidskriftsartikel (refereegranskat)abstract
- BACKGROUND: Large streamed datasets, characteristic of life science applications, are often resource-intensive to process, transport and store. We propose a pipeline model, a design pattern for scientific pipelines, where an incoming stream of scientific data is organized into a tiered or ordered "data hierarchy". We introduce the HASTE Toolkit, a proof-of-concept cloud-native software toolkit based on this pipeline model, to partition and prioritize data streams to optimize use of limited computing resources.FINDINGS: In our pipeline model, an "interestingness function" assigns an interestingness score to data objects in the stream, inducing a data hierarchy. From this score, a "policy" guides decisions on how to prioritize computational resource use for a given object. The HASTE Toolkit is a collection of tools to adopt this approach. We evaluate with 2 microscopy imaging case studies. The first is a high content screening experiment, where images are analyzed in an on-premise container cloud to prioritize storage and subsequent computation. The second considers edge processing of images for upload into the public cloud for real-time control of a transmission electron microscope.CONCLUSIONS: Through our evaluation, we created smart data pipelines capable of effective use of storage, compute, and network resources, enabling more efficient data-intensive experiments. We note a beneficial separation between scientific concerns of data priority, and the implementation of this behaviour for different resources in different deployment contexts. The toolkit allows intelligent prioritization to be `bolted on' to new and existing systems - and is intended for use with a range of technologies in different deployment scenarios.
|
|
| 7. |
- Gupta, Ankit, et al.
(författare)
-
Is brightfield all you need for MoA prediction?
- 2022
-
Konferensbidrag (refereegranskat)abstract
- Fluorescence staining techniques, such as Cell Painting, together with fluorescence microscopy have proven invaluable for visualizing and quantifying the effects that drugs and other perturbations have on cultured cells. However, fluorescence microscopy is expensive, time-consuming, and labor-intensive, and the stains applied can be cytotoxic, interfering with the activity under study. The simplest form of microscopy, brightfield microscopy, lacks these downsides, but the images produced have low contrast and the cellular compartments are difficult to discern. Nevertheless, by harnessing deep learning, these brightfield images may still be sufficient for various predictive purposes. In this study, we compared the predictive performance of models trained on fluorescence images to those trained on brightfield images for predicting the mechanism of action (MoA) of different drugs. We also extracted CellProfiler features from the fluorescence images and used them to benchmark the performance. Overall, we found comparable and correlated predictive performance for the two imaging modalities. This is promising for future studies of MoAs in time-lapse experiments.
|
|
| 8. |
- Harrison, Philip J., et al.
(författare)
-
Deep-learning models for lipid nanoparticle-based drug delivery
- 2021
-
Ingår i: Nanomedicine. - : Future Medicine. - 1743-5889 .- 1748-6963. ; 16:13, s. 1097-1110
-
Tidskriftsartikel (refereegranskat)abstract
- Background: Early prediction of time-lapse microscopy experiments enables intelligent data management and decision-making. Aim: Using time-lapse data of HepG2 cells exposed to lipid nanoparticles loaded with mRNA for expression of GFP, the authors hypothesized that it is possible to predict in advance whether a cell will express GFP. Methods: The first modeling approach used a convolutional neural network extracting per-cell features at early time points. These features were then combined and explored using either a long short-term memory network (approach 2) or time series feature extraction and gradient boosting machines (approach 3). Results: Accounting for the temporal dynamics significantly improved performance. Conclusion: The results highlight the benefit of accounting for temporal dynamics when studying drug delivery using high-content imaging.
|
|
| 9. |
- Harrison, Philip John, et al.
(författare)
-
Evaluating the utility of brightfield image data for mechanism of action prediction
- 2023
-
Ingår i: PloS Computational Biology. - : Public Library of Science (PLoS). - 1553-734X .- 1553-7358. ; 19:7
-
Tidskriftsartikel (refereegranskat)abstract
- Fluorescence staining techniques, such as Cell Painting, together with fluorescence microscopy have proven invaluable for visualizing and quantifying the effects that drugs and other perturbations have on cultured cells. However, fluorescence microscopy is expensive, time-consuming, labor-intensive, and the stains applied can be cytotoxic, interfering with the activity under study. The simplest form of microscopy, brightfield microscopy, lacks these downsides, but the images produced have low contrast and the cellular compartments are difficult to discern. Nevertheless, by harnessing deep learning, these brightfield images may still be sufficient for various predictive purposes. In this study, we compared the predictive performance of models trained on fluorescence images to those trained on brightfield images for predicting the mechanism of action (MoA) of different drugs. We also extracted CellProfiler features from the fluorescence images and used them to benchmark the performance. Overall, we found comparable and largely correlated predictive performance for the two imaging modalities. This is promising for future studies of MoAs in time-lapse experiments for which using fluorescence images is problematic. Explorations based on explainable AI techniques also provided valuable insights regarding compounds that were better predicted by one modality over the other.
|
|
| 10. |
- Matuszewski, Damian J., 1988-
(författare)
-
Image and Data Analysis for Biomedical Quantitative Microscopy
- 2019
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- This thesis presents automatic image and data analysis methods to facilitate and improve microscopy-based research and diagnosis. New technologies and computational tools are necessary for handling the ever-growing amounts of data produced in life science. The thesis presents methods developed in three projects with different biomedical applications.In the first project, we analyzed a large high-content screen aimed at enabling personalized medicine for glioblastoma patients. We focused on capturing drug-induced cell-cycle disruption in fluorescence microscopy images of cancer cell cultures. Our main objectives were to identify drugs affecting the cell-cycle and to increase the understanding of different drugs’ mechanisms of action. Here we present tools for automatic cell-cycle analysis and identification of drugs of interest and their effective doses.In the second project, we developed a feature descriptor for image matching. Image matching is a central pre-processing step in many applications. For example, when two or more images must be matched and registered to create a larger field of view or to analyze differences and changes over time. Our descriptor is rotation-, scale-, and illumination-invariant and it has a short feature vector which makes it computationally attractive. The flexibility to combine it with any feature detector and the customization possibility make it a very versatile tool.In the third project, we addressed two general problems for bridging the gap between deep learning method development and their use in practical scenarios. We developed a method for convolutional neural network training using minimally annotated images. In many biomedical applications, the objects of interest cannot be accurately delineated due to their fuzzy shape, ambiguous morphology, image quality, or the expert knowledge and time it requires. The minimal annotations, in this case, consist of center-points or centerlines of target objects of approximately known size. We demonstrated our training method in a challenging application of a multi-class semantic segmentation of viruses in transmission electron microscopy images. We also systematically explored the influence of network architecture hyper-parameters on its size and performance and show the possibility to substantially reduce the size of a network without compromising its performance.All methods in this thesis were designed to work with little or no input from biomedical experts but of course, require fine-tuning for new applications. The usefulness of the tools has been demonstrated by collaborators and other researchers and has inspired further development of related algorithms.
|
|
