1. |
- Vu, Minh Hoang, 1988-
(author)
-
Resource efficient automatic segmentation of medical images
- 2023
-
Doctoral thesis (other academic/artistic)abstract
- Cancer is one of the leading causes of death worldwide. In 2020, there were around 10 million cancer deaths and nearly 20 million new cancer cases in the world. Radiation therapy is essential in cancer treatments because half of the cancer patients receive radiation therapy at some point. During a radiotherapy treatment planning (RTP), an oncologist must manually outline two types of areas of the patient’s body: target, which will be treated, and organs-at-risks (OARs), which are essential to avoid. This step is called delineation. The purpose of the delineation is to generate a sufficient dose plan that can provide adequate radiation dose to a tumor and limit the radiation exposure to healthy tissue. Therefore, accurate delineations are essential to achieve this goal.Delineation is tedious and demanding for oncologists because it requires hours of concentrating work doing a repeated job. This is a RTP bottleneck which is often time- and resource-intensive. Current software, such as atlasbased techniques, can assist with this procedure by registering the patient’s anatomy to a predetermined anatomical map. However, the atlas-based methods are often slowed down and erroneous for patients with abnormal anatomies.In recent years, deep learning (DL) methods, particularly convolutional neural networks (CNNs), have led to breakthroughs in numerous medical imaging applications. The core benefits of CNNs are weight sharing and that they can automatically detect important visual features. A typical application of CNNs for medical images is to automatically segment tumors, organs, and structures, which is assumed to save radiation oncologists much time when delineating. This thesis contributes to resource efficient automatic segmentation and covers different aspects of resource efficiency.In Paper I, we proposed a novel end-to-end cascaded network for semantic segmentation in brain tumors in the multi-modal magnetic resonance imaging challenge in 2019. The proposed method used the hierarchical structure of the tumor sub-regions and was one of the top-ranking teams in the task of quantification of uncertainty in segmentation. A follow-up work to this paper was ranked second in the same task in the same challenge a year later.We systematically assessed the segmentation performance and computational costs of the technique called pseudo-3D as a function of the number of input slices in Paper II. We compared the results to typical two-dimensional (2D) and three-dimensional (3D) CNNs and a method called triplanar orthogonal 2D. The typical pseudo-3D approach considers adjacent slices to be several image input channels. We discovered that a substantial benefit from employing multiple input slices was apparent for a specific input size.We introduced a novel loss function in Paper III to address diverse issues, including imbalanced datasets, partially labeled data, and incremental learning. The proposed loss function adjusts to the given data to use all accessible data, even if some lack annotations. We show that the suggested loss function also performs well in an incremental learning context, where an existing model can be modified to incorporate the delineations of newly appearing organs semi-automatically.In Paper IV, we proposed a novel method for compressing high-dimensional activation maps, which are the primary source of memory use in modern systems. We examined three distinct compression methods for the activation maps to accomplishing this. We demonstrated that the proposed method induces a regularization effect that acts on the layer weight gradients. By employing the proposed technique, we reduced activation map memory usage by up to 95%.We investigated the use of generative adversarial networks (GANs) to enlarge a small dataset by generating synthetic images in Paper V. We use the real and generated data during training CNNs for the downstream segmentation tasks. Inspired by an existing GAN, we proposed a conditional version to generate high-dimensional and high-quality medical images of different modalities and their corresponding label maps. We evaluated the quality of the generated medical images and the effect of this augmentation on the performance of the segmentation task on six datasets.
|
|
2. |
- Astaraki, Mehdi, PhD Student, 1984-
(author)
-
Advanced Machine Learning Methods for Oncological Image Analysis
- 2022
-
Doctoral thesis (other academic/artistic)abstract
- Cancer is a major public health problem, accounting for an estimated 10 million deaths worldwide in 2020 alone. Rapid advances in the field of image acquisition and hardware development over the past three decades have resulted in the development of modern medical imaging modalities that can capture high-resolution anatomical, physiological, functional, and metabolic quantitative information from cancerous organs. Therefore, the applications of medical imaging have become increasingly crucial in the clinical routines of oncology, providing screening, diagnosis, treatment monitoring, and non/minimally-invasive evaluation of disease prognosis. The essential need for medical images, however, has resulted in the acquisition of a tremendous number of imaging scans. Considering the growing role of medical imaging data on one side and the challenges of manually examining such an abundance of data on the other side, the development of computerized tools to automatically or semi-automatically examine the image data has attracted considerable interest. Hence, a variety of machine learning tools have been developed for oncological image analysis, aiming to assist clinicians with repetitive tasks in their workflow.This thesis aims to contribute to the field of oncological image analysis by proposing new ways of quantifying tumor characteristics from medical image data. Specifically, this thesis consists of six studies, the first two of which focus on introducing novel methods for tumor segmentation. The last four studies aim to develop quantitative imaging biomarkers for cancer diagnosis and prognosis.The main objective of Study I is to develop a deep learning pipeline capable of capturing the appearance of lung pathologies, including lung tumors, and integrating this pipeline into the segmentation networks to leverage the segmentation accuracy. The proposed pipeline was tested on several comprehensive datasets, and the numerical quantifications show the superiority of the proposed prior-aware DL framework compared to the state of the art. Study II aims to address a crucial challenge faced by supervised segmentation models: dependency on the large-scale labeled dataset. In this study, an unsupervised segmentation approach is proposed based on the concept of image inpainting to segment lung and head-neck tumors in images from single and multiple modalities. The proposed autoinpainting pipeline shows great potential in synthesizing high-quality tumor-free images and outperforms a family of well-established unsupervised models in terms of segmentation accuracy.Studies III and IV aim to automatically discriminate the benign from the malignant pulmonary nodules by analyzing the low-dose computed tomography (LDCT) scans. In Study III, a dual-pathway deep classification framework is proposed to simultaneously take into account the local intra-nodule heterogeneities and the global contextual information. Study IV seeks to compare the discriminative power of a series of carefully selected conventional radiomics methods, end-to-end Deep Learning (DL) models, and deep features-based radiomics analysis on the same dataset. The numerical analyses show the potential of fusing the learned deep features into radiomic features for boosting the classification power.Study V focuses on the early assessment of lung tumor response to the applied treatments by proposing a novel feature set that can be interpreted physiologically. This feature set was employed to quantify the changes in the tumor characteristics from longitudinal PET-CT scans in order to predict the overall survival status of the patients two years after the last session of treatments. The discriminative power of the introduced imaging biomarkers was compared against the conventional radiomics, and the quantitative evaluations verified the superiority of the proposed feature set. Whereas Study V focuses on a binary survival prediction task, Study VI addresses the prediction of survival rate in patients diagnosed with lung and head-neck cancer by investigating the potential of spherical convolutional neural networks and comparing their performance against other types of features, including radiomics. While comparable results were achieved in intra-dataset analyses, the proposed spherical-based features show more predictive power in inter-dataset analyses.In summary, the six studies incorporate different imaging modalities and a wide range of image processing and machine-learning techniques in the methods developed for the quantitative assessment of tumor characteristics and contribute to the essential procedures of cancer diagnosis and prognosis.
|
|
3. |
- Brusini, Irene
(author)
-
Methods for the analysis and characterization of brain morphology from MRI images
- 2022
-
Doctoral thesis (other academic/artistic)abstract
- Brain magnetic resonance imaging (MRI) is an imaging modality that produces detailed images of the brain without using any ionizing radiation. From a structural MRI scan, it is possible to extract morphological properties of different brain regions, such as their volume and shape. These measures can both allow a better understanding of how the brain changes due to multiple factors (e.g., environmental and pathological) and contribute to the identification of new imaging biomarkers of neurological and psychiatric diseases. The overall goal of the present thesis is to advance the knowledge on how brain MRI image processing can be effectively used to analyze and characterize brain structure.The first two works presented in this thesis are animal studies that primarily aim to use MRI data for analyzing differences between groups of interest. In Paper I, MRI scans from wild and domestic rabbits were processed to identify structural brain differences between these two groups. Domestication was found to significantly reshape brain structure in terms of both regional gray matter volume and white matter integrity. In Paper II, rat brain MRI scans were used to train a brain age prediction model. This model was then tested on both controls and a group of rats that underwent long-term environmental enrichment and dietary restriction. This healthy lifestyle intervention was shown to significantly affect the predicted brain age trajectories by slowing the rats' aging process compared to controls. Furthermore, brain age predicted on young adult rats was found to have a significant effect on survival.Papers III to V are human studies that propose deep learning-based methods for segmenting brain structures that can be severely affected by neurodegeneration. In particular, Papers III and IV focus on U-Net-based 2D segmentation of the corpus callosum (CC) in multiple sclerosis (MS) patients. In both studies, good segmentation accuracy was obtained and a significant correlation was found between CC area and the patient's level of cognitive and physical disability. Additionally, in Paper IV, shape analysis of the segmented CC revealed a significant association between disability and both CC thickness and bending angle. Conversely, in Paper V, a novel method for automatic segmentation of the hippocampus is proposed, which consists of embedding a statistical shape prior as context information into a U-Net-based framework. The inclusion of shape information was shown to significantly improve segmentation accuracy when testing the method on a new unseen cohort (i.e., different from the one used for training). Furthermore, good performance was observed across three different diagnostic groups (healthy controls, subjects with mild cognitive impairment and Alzheimer's patients) that were characterized by different levels of hippocampal atrophy.In summary, the studies presented in this thesis support the great value of MRI image analysis for the advancement of neuroscientific knowledge, and their contribution is mostly two-fold. First, by applying well-established processing methods on datasets that had not yet been explored in the literature, it was possible to characterize specific brain changes and disentangle relevant problems of a clinical or biological nature. Second, a technical contribution is provided by modifying and extending already-existing brain image processing methods to achieve good performance on new datasets.
|
|
4. |
- Fuchs, Alexander, 1985-
(author)
-
Assessment of predicting blood flow and atherosclerosis in the aorta and renal arteries
- 2020
-
Doctoral thesis (other academic/artistic)abstract
- Cardiovascular diseases (CVD) are the most common cause of death in large parts of the world. Atherosclerosis (AS) has a major part in most CVDs. AS is a slowly developingdisease which is dependent on multiple factors such as genetics and life style (food, smoking, and physical activities). AS is primarily a disease of the arterial wall and develops preferentially at certain locations (such as arterial branches and in certain vessels like thecoronary arteries). The close relation between AS sites and blood flow has been well established over the years. However, due to multi-factorial causes, there exist no early prognostic tools for identifying individuals that should be treated prophylactically or followed up. The underlying hypothesis of this thesis was to determine if it is possible to use bloodflow simulations of patient-specific cases in order to identify individuals with risk for developing AS. CT scans from patients with renal artery stenosis (RAS) were used to get the affected vessels geometry. Blood flow in original and “reconstructed” arteries were simulated. Commonly used wall shear stress (WSS) related indicators of AS were studied to assess their use as risk indicators for developing AS. Divergent results indicated urgent need to assess the impact ofsimulation related factors on results. Altogether, blood flow in the following vessels was studied: The whole aorta with branches from the aortic arch and the abdominal aorta, abdominal aorta as well as the renal arteries, and separately the thoracic aorta with the three main branching arteries from the aortic arch. The impact of geometrical reconstruction, employed boundary conditions (BCs), effects of flow-rate, heart-rate and models of blood viscosity as function of local hematocrit (red blood cell, RBC, concentration) and shear-rate were studied in some detail. In addition to common WSS-related indicators, we suggested the use of endothelial activation models as a further risk indicator. The simulations data was used to extract not only the WSS-related data but also the impact of flow-rate on the extent of retrograde flow in the aorta and close to its walls. The formation of helical motion and flow instabilities (which at high flow- and heart-rate lead to turbulence) was also considered.Results:A large number of simulations (more than 100) were carried out. These simulations assessed the use of flow-rate specified BCs, pressure based BCs or so called windkessel (WK) outlet BCs that simulate effects of peripheral arterial compliance. The results showed high sensitivity of the flow to BCs. For example, the deceleration phase of the flow-rate is more prone to flow instabilities (as also expressed in terms of multiple inflection points in the streamwise velocity profile) as well as leading to retrograde flow. In contrast, the acceleration phase leads to uni-directional and more stable flow. As WSS unsteadiness was found to be pro-AS, it was important to assess the effect flow-rate deceleration, under physiological and pathological conditions. Peaks of retrograde flow occur at local temporal minima in flow-rate. WK BCs require ad-hoc adjusted parameters and are therefore useful only when fully patient specific (i.e. all information is valid for a particular patient at a particular point of time) data is available. Helical flows which are considered as atheroprotective, are formed naturally, depending primarily on the geometry (due to the bends in the thoracic aorta). Helical flow was also observed in the major aortic branches. The helical motion is weaker during flow deceleration and diastole when it may locally also change direction. Most common existing blood viscosity models are based on hematocrit and shear-rate. These models show strong variation of blood (mixture) viscosity. With strong shear-rate blood viscosity is lowest and is almost constant. The impact of blood viscosity in terms of dissipation is counter balanced by the shear-rate; At low shear-rate the blood has larger viscosity and at high shear-rate it is the opposite. This effect and due to the temporal variations in the local flow conditions the effect of blood rheology on the WSS indicators is weak. Tracking of blood components and clot-models shows that the retrograde motion and the flow near branches may have so strong curvature that centrifugal force can become important. This effect may lead to the transport of a thrombus from the descending aorta back to the branches of the aortic arch and could cause embolic stroke. The latter results confirm clinical observation of the risk of stroke due to transport of emboli from the proximal part of the descending aorta upstream to the vessels branching from the aortic arch and which lead blood to the brain.Conclusions:The main reasons for not being able to propose an early predictive tool for future developmentof AS are four-folded:i. At present, the mechanisms behind AS are not adequately understood to enable to define aset of parameters that are sensitive and specific enough to be predictive of its development.ii. The lack of accurate patient-specific data (BC:s) over the whole physiological “envelop”allows only limited number of flow simulations which may not be adequate for patientspecificpredictive purposes.iii. The shortcomings of current models with respect to material properties of blood andarterial walls (for patient-specific space- and time-variations) are lacking.iv. There is a need for better simulation data processing, i.e. tools that enable deducinggeneral predictive atherosclerotic parameters from a limited number of simulations, throughe.g. extending reduced modeling and/or deep learning.The results do show, however, that blood flow simulations may produce very useful data thatenhances understanding of clinically observed processes such as explaining helical- andretrograde flows and the transport of blood components and emboli in larger arteries.
|
|
5. |
- Jörgens, Daniel, 1988-
(author)
-
Development and application of rule- and learning-based approaches within the scope of neuroimaging : Tensor voting, tractography and machine learning
- 2020
-
Doctoral thesis (other academic/artistic)abstract
- The opportunity to non-invasively probe the structure and function of different parts of the human body makes medical imaging an indispensable tool in clinical diagnostics and related fields of research. Especially neuroscientists rely on modalities like structural or functional Magnetic Resonance Imaging, Computed Tomography or Positron Emission Tomography to study the human brain in vivo. But also in clinical routine, diagnosis, screening or follow-up of different pathological conditions build upon the use of neuroimaging.Computational solutions are essential for the analysis of medical images. While in the case of conventional photography the recorded signal comprises the actual image, most medical imaging devices require the reconstruction of an image from the acquired data. However, not only the image formation, but also further processing tasks to assist doctors or researchers in the interpretation of the data and eventually in subsequent decision making, rely more and more on automation. Typical tasks range from locating and measuring objects in a single patient, e.g. a particular organ, a tumour or a specific region in the brain, to comparing such measurements over time between groups consisting of large numbers of subjects. Automated solutions for these scenarios are required to model complex relations of data in the presence of acquisition noise and subject variability while assuring a tractable computational demand.Traditionally, the development of computational algorithms for medical imaging problems focused on rule-based strategies. Explicitly defined rules that encode the knowledge of the developer are characteristic for such approaches. Within the last decade, this paradigm began to change and learning-based models dramatically gained in popularity. These rely on fitting a complex model to large amounts of data samples, often annotated, which are representative for a particular problem. Instead of manually designing the sought-after solution, it is ‘learned from the data’. While these models have shown enormous potential, they also pose important questions for method developers. How can I get hold of enough data? How much data is enough? How can I obtain proper annotations?This thesis comprises six studies covering the development and the application of methods along the whole pipeline of medical image analysis. Studies I and II propose different extensions to the method of tensor voting to make it applicable in specific medical imaging problems. Studies III–V address the use of modern machine learning techniques, in particular neural networks, in the field of tractography. Notably, the challenge of obtaining adequately annotated data samples is a topic in Study V. In Study VI, a prospective neuroimaging study of unilateral ear canal atresia in adults is presented, covering the application of methods from data acquisition to group comparison. Overall, the compiled works contributed, in one way or the other, to the non-invasive extraction of knowledge from the human body through automated processing of medical images.
|
|
6. |
- Astaraki, Mehdi, PhD Student, 1984-, et al.
(author)
-
A Comparative Study of Radiomics and Deep-Learning Based Methods for Pulmonary Nodule Malignancy Prediction in Low Dose CT Images
- 2021
-
In: Frontiers in Oncology. - : Frontiers Media SA. - 2234-943X. ; 11
-
Journal article (peer-reviewed)abstract
- Objectives: Both radiomics and deep learning methods have shown great promise in predicting lesion malignancy in various image-based oncology studies. However, it is still unclear which method to choose for a specific clinical problem given the access to the same amount of training data. In this study, we try to compare the performance of a series of carefully selected conventional radiomics methods, end-to-end deep learning models, and deep-feature based radiomics pipelines for pulmonary nodule malignancy prediction on an open database that consists of 1297 manually delineated lung nodules.Methods: Conventional radiomics analysis was conducted by extracting standard handcrafted features from target nodule images. Several end-to-end deep classifier networks, including VGG, ResNet, DenseNet, and EfficientNet were employed to identify lung nodule malignancy as well. In addition to the baseline implementations, we also investigated the importance of feature selection and class balancing, as well as separating the features learned in the nodule target region and the background/context region. By pooling the radiomics and deep features together in a hybrid feature set, we investigated the compatibility of these two sets with respect to malignancy prediction.Results: The best baseline conventional radiomics model, deep learning model, and deep-feature based radiomics model achieved AUROC values (mean ± standard deviations) of 0.792 ± 0.025, 0.801 ± 0.018, and 0.817 ± 0.032, respectively through 5-fold cross-validation analyses. However, after trying out several optimization techniques, such as feature selection and data balancing, as well as adding context features, the corresponding best radiomics, end-to-end deep learning, and deep-feature based models achieved AUROC values of 0.921 ± 0.010, 0.824 ± 0.021, and 0.936 ± 0.011, respectively. We achieved the best prediction accuracy from the hybrid feature set (AUROC: 0.938 ± 0.010).Conclusion: The end-to-end deep-learning model outperforms conventional radiomics out of the box without much fine-tuning. On the other hand, fine-tuning the models lead to significant improvements in the prediction performance where the conventional and deep-feature based radiomics models achieved comparable results. The hybrid radiomics method seems to be the most promising model for lung nodule malignancy prediction in this comparative study.
|
|
7. |
- Astaraki, Mehdi, PhD Student, 1984-, et al.
(author)
-
Benign-malignant pulmonary nodule classification in low-dose CT with convolutional features
- 2021
-
In: Physica medica (Testo stampato). - : Elsevier BV. - 1120-1797 .- 1724-191X. ; 83, s. 146-153
-
Journal article (peer-reviewed)abstract
- Purpose: Low-Dose Computed Tomography (LDCT) is the most common imaging modality for lung cancer diagnosis. The presence of nodules in the scans does not necessarily portend lung cancer, as there is an intricate relationship between nodule characteristics and lung cancer. Therefore, benign-malignant pulmonary nodule classification at early detection is a crucial step to improve diagnosis and prolong patient survival. The aim of this study is to propose a method for predicting nodule malignancy based on deep abstract features.Methods: To efficiently capture both intra-nodule heterogeneities and contextual information of the pulmonary nodules, a dual pathway model was developed to integrate the intra-nodule characteristics with contextual attributes. The proposed approach was implemented with both supervised and unsupervised learning schemes. A random forest model was added as a second component on top of the networks to generate the classification results. The discrimination power of the model was evaluated by calculating the Area Under the Receiver Operating Characteristic Curve (AUROC) metric. Results: Experiments on 1297 manually segmented nodules show that the integration of context and target supervised deep features have a great potential for accurate prediction, resulting in a discrimination power of 0.936 in terms of AUROC, which outperformed the classification performance of the Kaggle 2017 challenge winner.Conclusion: Empirical results demonstrate that integrating nodule target and context images into a unified network improves the discrimination power, outperforming the conventional single pathway convolutional neural networks.
|
|
8. |
- Astaraki, Mehdi, PhD Student, 1984-, et al.
(author)
-
Prior-aware autoencoders for lung pathology segmentation
- 2022
-
In: Medical Image Analysis. - : Elsevier BV. - 1361-8415 .- 1361-8423. ; 80, s. 102491-
-
Journal article (peer-reviewed)abstract
- Segmentation of lung pathology in Computed Tomography (CT) images is of great importance for lung disease screening. However, the presence of different types of lung pathologies with a wide range of heterogeneities in size, shape, location, and texture, on one side, and their visual similarity with respect to surrounding tissues, on the other side, make it challenging to perform reliable automatic lesion seg-mentation. To leverage segmentation performance, we propose a deep learning framework comprising a Normal Appearance Autoencoder (NAA) model to learn the distribution of healthy lung regions and re-construct pathology-free images from the corresponding pathological inputs by replacing the pathological regions with the characteristics of healthy tissues. Detected regions that represent prior information re-garding the shape and location of pathologies are then integrated into a segmentation network to guide the attention of the model into more meaningful delineations. The proposed pipeline was tested on three types of lung pathologies, including pulmonary nodules, Non-Small Cell Lung Cancer (NSCLC), and Covid-19 lesion on five comprehensive datasets. The results show the superiority of the proposed prior model, which outperformed the baseline segmentation models in all the cases with significant margins. On av-erage, adding the prior model improved the Dice coefficient for the segmentation of lung nodules by 0.038, NSCLCs by 0.101, and Covid-19 lesions by 0.041. We conclude that the proposed NAA model pro-duces reliable prior knowledge regarding the lung pathologies, and integrating such knowledge into a prior segmentation network leads to more accurate delineations.
|
|
9. |
|
|
10. |
- Blystad, Ida, 1972-, et al.
(author)
-
Quantitative MRI using relaxometry in malignant gliomas detects contrast enhancement in peritumoral oedema
- 2020
-
In: Scientific Reports. - : Springer Nature. - 2045-2322. ; 10:1
-
Journal article (peer-reviewed)abstract
- Malignant gliomas are primary brain tumours with an infiltrative growth pattern, often with contrast enhancement on magnetic resonance imaging (MRI). However, it is well known that tumour infiltration extends beyond the visible contrast enhancement. The aim of this study was to investigate if there is contrast enhancement not detected visually in the peritumoral oedema of malignant gliomas by using relaxometry with synthetic MRI. 25 patients who had brain tumours with a radiological appearance of malignant glioma were prospectively included. A quantitative MR-sequence measuring longitudinal relaxation (R-1), transverse relaxation (R-2) and proton density (PD), was added to the standard MRI protocol before surgery. Five patients were excluded, and in 20 patients, synthetic MR images were created from the quantitative scans. Manual regions of interest (ROIs) outlined the visibly contrast-enhancing border of the tumours and the peritumoral area. Contrast enhancement was quantified by subtraction of native images from post GD-images, creating an R-1-difference-map. The quantitative R-1-difference-maps showed significant contrast enhancement in the peritumoral area (0.047) compared to normal appearing white matter (0.032), p = 0.048. Relaxometry detects contrast enhancement in the peritumoral area of malignant gliomas. This could represent infiltrative tumour growth.
|
|