| 1. |
- Tsirikoglou, Apostolia, 1985-, et al.
(författare)
-
Differential appearance editing for measured BRDFs
- 2016
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
- Data driven reflectance models using BRDF data measured from real materials, e.g. [Matusik et al. 2003], are becoming increasingly popular in product visualization, digital design and other applications driven by the need for predictable rendering and highly realistic results. Although recent analytic, parametric BRDFs provide good approximations for many materials, some effects are still not captured well [Löw et al. 2012]. Thus, it is hard to accurately model real materials using analytic models, even if the parameters are fitted to data. In practice, it is often desirable to apply small edits to the measured data for artistic purposes, or to model similar materials that are not available in measured form. A drawback of data driven models is that they are often difficult to edit and do not easily lend themselves well to artistic adjustments. Existing editing techniques for measured data [Schmidt et al. 2014], often use complex decompositions making them difficult to use in practice.
|
|
| 2. |
- Baravdish, Gabriel, 1992-, et al.
(författare)
-
GPU Accelerated Sparse Representation of Light Fields
- 2019
-
Ingår i: VISIGRAPP - 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, Prague, Czech Republic, February 25-27, 2019.. - : SCITEPRESS. - 9789897583544 ; , s. 177-182
-
Konferensbidrag (refereegranskat)abstract
- We present a method for GPU accelerated compression of light fields. The approach is by using a dictionary learning framework for compression of light field images. The large amount of data storage by capturing light fields is a challenge to compress and we seek to accelerate the encoding routine by GPGPU computations. We compress the data by projecting each data point onto a set of trained multi-dimensional dictionaries and seek the most sparse representation with the least error. This is done by a parallelization of the tensor-matrix product computed on the GPU. An optimized greedy algorithm to suit computations on the GPU is also presented. The encoding of the data is done segmentally in parallel for a faster computation speed while maintaining the quality. The results shows an order of magnitude faster encoding time compared to the results in the same research field. We conclude that there are further improvements to increase the speed, and thus it is not too far from an interacti ve compression speed.
|
|
| 3. |
- Baravdish, George, 1964-, et al.
(författare)
-
Learning via nonlinear conjugate gradients and depth-varying neural ODEs
-
Annan publikation (övrigt vetenskapligt/konstnärligt)abstract
- The inverse problem of supervised reconstruction of depth-variable (time-dependent) parameters in a neural ordinary differential equation (NODE) is considered, that means finding the weights of a residual network with time continuous layers. The NODE is treated as an isolated entity describing the full network as opposed to earlier research, which embedded it between pre- and post-appended layers trained by conventional methods. The proposed parameter reconstruction is done for a general first order differential equation by minimizing a cost functional covering a variety of loss functions and penalty terms. A nonlinear conjugate gradient method (NCG) is derived for the minimization. Mathematical properties are stated for the differential equation and the cost functional. The adjoint problem needed is derived together with a sensitivity problem. The sensitivity problem can estimate changes in the network output under perturbation of the trained parameters. To preserve smoothness during the iterations the Sobolev gradient is calculated and incorporated. As a proof-of-concept, numerical results are included for a NODE and two synthetic datasets, and compared with standard gradient approaches (not based on NODEs). The results show that the proposed method works well for deep learning with infinite numbers of layers, and has built-in stability and smoothness.
|
|
| 4. |
- Eilertsen, Gabriel, 1984-, et al.
(författare)
-
BriefMatch: Dense binary feature matching for real-time optical flow estimation
- 2017
-
Ingår i: Proceedings of the Scandinavian Conference on Image Analysis (SCIA17). - Cham : Springer. - 9783319591254 ; , s. 221-233
-
Konferensbidrag (refereegranskat)abstract
- Research in optical flow estimation has to a large extent focused on achieving the best possible quality with no regards to running time. Nevertheless, in a number of important applications the speed is crucial. To address this problem we present BriefMatch, a real-time optical flow method that is suitable for live applications. The method combines binary features with the search strategy from PatchMatch in order to efficiently find a dense correspondence field between images. We show that the BRIEF descriptor provides better candidates (less outlier-prone) in shorter time, when compared to direct pixel comparisons and the Census transform. This allows us to achieve high quality results from a simple filtering of the initially matched candidates. Currently, BriefMatch has the fastest running time on the Middlebury benchmark, while placing highest of all the methods that run in shorter than 0.5 seconds.
|
|
| 5. |
- Eilertsen, Gabriel, 1984-, et al.
(författare)
-
Classifying the classifier : dissecting the weight space of neural networks
- 2020
-
Ingår i: Proceedings of the 24th European Conference on Artificial Intelligence (ECAI 2020). - : IOS PRESS. - 9781643681016 ; , s. 1119-1126
-
Konferensbidrag (refereegranskat)abstract
- This paper presents an empirical study on the weights of neural networks, where we interpret each model as a point in a high-dimensional space – the neural weight space. To explore the complex structure of this space, we sample from a diverse selection of training variations (dataset, optimization procedure, architecture,etc.) of neural network classifiers, and train a large number of models to represent the weight space. Then, we use a machine learning approach for analyzing and extracting information from this space. Most centrally, we train a number of novel deep meta-classifiers withthe objective of classifying different properties of the training setup by identifying their footprints in the weight space. Thus, the meta-classifiers probe for patterns induced by hyper-parameters, so that we can quantify how much, where, and when these are encoded through the optimization process. This provides a novel and complementary view for explainable AI, and we show how meta-classifiers can reveal a great deal of information about the training setup and optimization, by only considering a small subset of randomly selected consecutive weights. To promote further research on the weight space, we release the neural weight space (NWS) dataset – a collection of 320K weightsnapshots from 16K individually trained deep neural networks.
|
|
| 6. |
- Eilertsen, Gabriel, 1984-, et al.
(författare)
-
Ensembles of GANs for synthetic training data generation
- 2021
-
Konferensbidrag (refereegranskat)abstract
- Insufficient training data is a major bottleneck for most deep learning practices, not least in medical imaging where data is difficult to collect and publicly available datasets are scarce due to ethics and privacy. This work investigates the use of synthetic images, created by generative adversarial networks (GANs), as the only source of training data. We demonstrate that for this application, it is of great importance to make use of multiple GANs to improve the diversity of the generated data, i.e. to sufficiently cover the data distribution. While a single GAN can generate seemingly diverse image content, training on this data in most cases lead to severe over-fitting. We test the impact of ensembled GANs on synthetic 2D data as well as common image datasets (SVHN and CIFAR-10), and using both DCGANs and progressively growing GANs. As a specific use case, we focus on synthesizing digital pathology patches to provide anonymized training data.
|
|
| 7. |
- Eilertsen, Gabriel, et al.
(författare)
-
Evaluation of tone mapping operators for HDR video
- 2016. - 1st
-
Ingår i: High dynamic range video. - London, United Kingdom : Academic Press. - 9780081004128 ; , s. 185-206
-
Bokkapitel (övrigt vetenskapligt/konstnärligt)abstract
- Tone mapping of HDR-video is a challenging filtering problem. It is highly important to develop a framework for evaluation and comparison of tone mapping operators. This chapter gives an overview of different approaches for how evalation of tone mapping operators can be conducted, including experimental setups, choice of input data, choice of tone mapping operators, and the importance of parameter tweaking for fair comparisons. This chapter also gives examples of previous evaluations with a focus on the results from the most recent evaluation conducted by Eilertsen et. al [reference]. This results in a classification of the currently most commonly used tone mapping operators and overview of their performance and possible artifacts.
|
|
| 8. |
- Eilertsen, Gabriel, 1984-, et al.
(författare)
-
Model-invariant Weight Distribution Descriptors for Visual Exploration of Neural Networks en Masse
- 2024
-
Ingår i: EuroVis 2024 - Short Papers. - 9783038682516
-
Konferensbidrag (refereegranskat)abstract
- We present a neural network representation which can be used for visually analyzing the similarities and differences in a large corpus of trained neural networks. The focus is on architecture-invariant comparisons based on network weights, estimating similarities of the statistical footprints encoded by the training setups and stochastic optimization procedures. To make this possible, we propose a novel visual descriptor of neural network weights. The visual descriptor considers local weight statistics in a model-agnostic manner by encoding the distribution of weights over different model depths. We show how such a representation can extract descriptive information, is robust to different parameterizations of a model, and is applicable to different architecture specifications. The descriptor is used to create a model atlas by projecting a model library to a 2D representation, where clusters can be found based on similar weight properties. A cluster analysis strategy makes it possible to understand the weight properties of clusters and how these connect to the different datasets and hyper-parameters used to train the models.
|
|
| 9. |
- Eilertsen, Gabriel, et al.
(författare)
-
Perceptually based parameter adjustments for video processing operations
- 2014
-
Ingår i: ACM SIGGRAPH Talks 2014. - : ACM Press.
-
Konferensbidrag (refereegranskat)abstract
- Extensive post processing plays a central role in modern video production pipelines. A problem in this context is that many filters and processing operators are very sensitive to parameter settings and that the filter responses in most cases are highly non-linear. Since there is no general solution for performing perceptual calibration of image and video operators automatically, it is often necessary to manually perform tweaking of multiple parameters. This is an iterative process which requires instant visual feedback of the result in both the spatial and temporal domains. Due to large filter kernels, computational complexity, high frame rate, and image resolution it is, however, often very time consuming to iteratively re-process and tweak long video sequences.We present a new method for rapidly finding the perceptual minima in high-dimensional parameter spaces of general video operators. The key idea of our algorithm is that the characteristics of an operator can be accurately described by interpolating between a small set of pre-computed parameter settings. By computing a perceptual linearization of the parameter space of a video operator, the user can explore this interpolated space to find the best set of parameters in a robust way. Since many operators are dependent on two or more parameters, we formulate this as a general optimization problem where we let the objective function be determined by the user’s image assessments. To demonstrate the usefulness of our approach we show a set of use cases (see the supplementary material) where our algorithm is applied to computationally expensive video operations.
|
|
| 10. |
- Eilertsen, Gabriel, et al.
(författare)
-
Real-time noise-aware tone mapping
- 2015
-
Ingår i: ACM Transactions on Graphics. - New York, NY, USA : Association for Computing Machinery (ACM). - 0730-0301 .- 1557-7368. ; 34:6, s. 198:1-198:15
-
Tidskriftsartikel (refereegranskat)abstract
- Real-time high quality video tone mapping is needed for manyapplications, such as digital viewfinders in cameras, displayalgorithms which adapt to ambient light, in-camera processing,rendering engines for video games and video post-processing. We propose a viable solution for these applications by designing a videotone-mapping operator that controls the visibility of the noise,adapts to display and viewing environment, minimizes contrastdistortions, preserves or enhances image details, and can be run inreal-time on an incoming sequence without any preprocessing. To ourknowledge, no existing solution offers all these features. Our novelcontributions are: a fast procedure for computing local display-adaptivetone-curves which minimize contrast distortions, a fast method for detailenhancement free from ringing artifacts, and an integrated videotone-mapping solution combining all the above features.
|
|
| 11. |
- Eilertsen, Gabriel, 1984-, et al.
(författare)
-
Single-frame Regularization for Temporally Stable CNNs
- 2019
-
Ingår i: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. - 9781728132938 - 9781728132945 ; , s. 11176-11185
-
Konferensbidrag (refereegranskat)abstract
- Convolutional neural networks (CNNs) can model complicated non-linear relations between images. However, they are notoriously sensitive to small changes in the input. Most CNNs trained to describe image-to-image mappings generate temporally unstable results when applied to video sequences, leading to flickering artifacts and other inconsistencies over time. In order to use CNNs for video material, previous methods have relied on estimating dense frame-to-frame motion information (optical flow) in the training and/or the inference phase, or by exploring recurrent learning structures. We take a different approach to the problem, posing temporal stability as a regularization of the cost function. The regularization is formulated to account for different types of motion that can occur between frames, so that temporally stable CNNs can be trained without the need for video material or expensive motion estimation. The training can be performed as a fine-tuning operation, without architectural modifications of the CNN. Our evaluation shows that the training strategy leads to large improvements in temporal smoothness. Moreover, for small datasets the regularization can help in boosting the generalization performance to a much larger extent than what is possible with naive augmentation strategies.
|
|
| 12. |
- Eilertsen, Gabriel, et al.
(författare)
-
Survey and Evaluation of Tone Mapping Operators for HDR-video
- 2013
-
Ingår i: Siggraph 2013 Talks. - New York, NY, USA : ACM Press. - 9781450323444
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
- This work presents a survey and a user evaluation of tone mapping operators (TMOs) for high dynamic range (HDR) video, i.e. TMOs that explicitly include a temporal model for processing of variations in the input HDR images in the time domain. The main motivations behind this work is that: robust tone mapping is one of the key aspects of HDR imaging [Reinhard et al. 2006]; recent developments in sensor and computing technologies have now made it possible to capture HDR-video, e.g. [Unger and Gustavson 2007; Tocci et al. 2011]; and, as shown by our survey, tone mapping for HDR video poses a set of completely new challenges compared to tone mapping for still HDR images. Furthermore, video tone mapping, though less studied, is highly important for a multitude of applications including gaming, cameras in mobile devices, adaptive display devices and movie post-processing. Our survey is meant to summarize the state-of-the-art in video tonemapping and, as exemplified in Figure 1 (right), analyze differences in their response to temporal variations. In contrast to other studies, we evaluate TMOs performance according to their actual intent, such as producing the image that best resembles the real world scene, that subjectively looks best to the viewer, or fulfills a certain artistic requirement. The unique strength of this work is that we use real high quality HDR video sequences, see Figure 1 (left), as opposed to synthetic images or footage generated from still HDR images.
|
|
| 13. |
- Eilertsen, Gabriel, 1984-
(författare)
-
The high dynamic range imaging pipeline : Tone-mapping, distribution, and single-exposure reconstruction
- 2018
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Techniques for high dynamic range (HDR) imaging make it possible to capture and store an increased range of luminances and colors as compared to what can be achieved with a conventional camera. This high amount of image information can be used in a wide range of applications, such as HDR displays, image-based lighting, tone-mapping, computer vision, and post-processing operations. HDR imaging has been an important concept in research and development for many years. Within the last couple of years it has also reached the consumer market, e.g. with TV displays that are capable of reproducing an increased dynamic range and peak luminance.This thesis presents a set of technical contributions within the field of HDR imaging. First, the area of HDR video tone-mapping is thoroughly reviewed, evaluated and developed upon. A subjective comparison experiment of existing methods is performed, followed by the development of novel techniques that overcome many of the problems evidenced by the evaluation. Second, a largescale objective comparison is presented, which evaluates existing techniques that are involved in HDR video distribution. From the results, a first open-source HDR video codec solution, Luma HDRv, is built using the best performing techniques. Third, a machine learning method is proposed for the purpose of reconstructing an HDR image from one single-exposure low dynamic range (LDR) image. The method is trained on a large set of HDR images, using recent advances in deep learning, and the results increase the quality and performance significantly as compared to existing algorithms.The areas for which contributions are presented can be closely inter-linked in the HDR imaging pipeline. Here, the thesis work helps in promoting efficient and high-quality HDR video distribution and display, as well as robust HDR image reconstruction from a single conventional LDR image.
|
|
| 14. |
- Emadi, Mohammad, et al.
(författare)
-
A Performance Guarantee for Orthogonal Matching Pursuit Using Mutual Coherence
- 2018
-
Ingår i: Circuits, systems, and signal processing. - : Springer. - 0278-081X .- 1531-5878. ; 37:4, s. 1562-1574
-
Tidskriftsartikel (refereegranskat)abstract
- In this paper, we present a new performance guarantee for the orthogonal matching pursuit (OMP) algorithm. We use mutual coherence as a metric for determining the suitability of an arbitrary overcomplete dictionary for exact recovery. Specifically, a lower bound for the probability of correctly identifying the support of a sparse signal with additive white Gaussian noise and an upper bound for the mean square error is derived. Compared to the previous work, the new bound takes into account the signal parameters such as dynamic range, noise variance, and sparsity. Numerical simulations show significant improvements over previous work and a much closer correlation to empirical results of OMP.
|
|
| 15. |
- Gardner, Andrew, et al.
(författare)
-
Depends : Workflow Management Software for Visual Effects Production
- 2014
-
Konferensbidrag (refereegranskat)abstract
- In this paper, we present an open source, multi-platform, workflow management application named Depends, designed to clarify and enhance the workflow of artists in a visual effects environment. Depends organizes processes into a directed acyclic graph, enabling artists to quickly identify appropriate changes, make modifications, and improve the look of their work. Recovering information about past revisions of an element is made simple, as the provenance of data is a core focus of a Depends workflow. Sharing work is also facilitated by the clear and consistent structure of Depends. We demonstrate the flexibility of Depends by presenting a number of scenarios where its style of workflow management has been essential to the creation of high-quality results.
|
|
| 16. |
- Hajisharif, Saghi, 1987-, et al.
(författare)
-
Compression and Real-Time Rendering of Inward Looking Spherical Light Fields
- 2020
-
Ingår i: Eurographics 2020 - Short Papers. - 9783038681014
-
Konferensbidrag (refereegranskat)abstract
- Photorealistic rendering is an essential tool for immersive virtual reality. In this regard, the data structure of choice is typically light fields since they contain multidimensional information about the captured environment that can provide motion parallax and view-dependent information such as highlights. There are various ways to acquire light fields depending on the nature of the scene, limitations on the capturing setup, and the application at hand. Our focus in this paper is on full-parallax imaging of large-scale static objects for photorealistic real-time rendering. To this end, we introduce and simulate a new design for capturing inward-looking spherical light fields, and propose a system for efficient compression and real-time rendering of such data using consumer-level hardware suitable for virtual reality applications.
|
|
| 17. |
- Hajisharif, Saghi, 1987-
(författare)
-
Computational Photography : High Dynamic Range and Light Fields
- 2020
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- The introduction and recent advancements of computational photography have revolutionized the imaging industry. Computational photography is a combination of imaging techniques at the intersection of various fields such as optics, computer vision, and computer graphics. These methods enhance the capabilities of traditional digital photography by applying computational techniques both during and after the capturing process. This thesis targets two major subjects in this field: High Dynamic Range (HDR) image reconstruction and Light Field (LF) compressive capturing, compression, and real-time rendering.The first part of the thesis focuses on the HDR images that concurrently contain detailed information from the very dark shadows to the brightest areas in the scenes. One of the main contributions presented in this thesis is the development of a unified reconstruction algorithm for spatially variant exposures in a single image. This method is based on a camera noise model, and it simultaneously resamples, reconstructs, denoises, and demosaics the image while extending its dynamic range. Furthermore, the HDR reconstruction algorithm is extended to adapt to the local features of the image, as well as the noise statistics, to preserve the high-frequency edges during reconstruction.In the second part of this thesis, the research focus shifts to the acquisition, encoding, reconstruction, and rendering of light field images and videos in a real-time setting. Unlike traditional integral photography, a light field captures the information of the dynamic environment from all angles, all points in space, and all spectral wavelength and time. This thesis employs sparse representation to provide an end-to-end solution to the problem of encoding, real-time reconstruction, and rendering of high dimensional light field video data sets. These solutions are applied on various types of data sets, such as light fields captured with multi-camera systems or hand-held cameras equipped with micro-lens arrays, and spherical light fields. Finally, sparse representation of light fields was utilized for developing a single sensor light field video camera equipped with a color-coded mask. A new compressive sensing model is presented that is suitable for dynamic scenes with temporal coherency and is capable of reconstructing high-resolution light field videos.
|
|
| 18. |
- Hajisharif, Saghi, 1987-, et al.
(författare)
-
Light Field Video Compression and Real Time Rendering
- 2019
-
Ingår i: Computer graphics forum (Print). - : John Wiley & Sons. - 0167-7055 .- 1467-8659. ; 38, s. 265-276
-
Tidskriftsartikel (refereegranskat)abstract
- Light field imaging is rapidly becoming an established method for generating flexible image based description of scene appearances. Compared to classical 2D imaging techniques, the angular information included in light fields enables effects such as post‐capture refocusing and the exploration of the scene from different vantage points. In this paper, we describe a novel GPU pipeline for compression and real‐time rendering of light field videos with full parallax. To achieve this, we employ a dictionary learning approach and train an ensemble of dictionaries capable of efficiently representing light field video data using highly sparse coefficient sets. A novel, key element in our representation is that we simultaneously compress both image data (pixel colors) and the auxiliary information (depth, disparity, or optical flow) required for view interpolation. During playback, the coefficients are streamed to the GPU where the light field and the auxiliary information are reconstructed using the dictionary ensemble and view interpolation is performed. In order to realize the pipeline we present several technical contributions including a denoising scheme enhancing the sparsity in the dataset which enables higher compression ratios, and a novel pruning strategy which reduces the size of the dictionary ensemble and leads to significant reductions in computational complexity during the encoding of a light field. Our approach is independent of the light field parameterization and can be used with data from any light field video capture system. To demonstrate the usefulness of our pipeline, we utilize various publicly available light field video datasets and discuss the medical application of documenting heart surgery.
|
|
| 19. |
- Hajisharif, Saghi, et al.
(författare)
-
Real-time image based lighting with streaming HDR-lightprobe sequences
- 2012
-
Ingår i: Proceedings of SIGRAD 2012. - Linköping, Sweden. - 9789175197234
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
- We present a framework for shading of virtual objects using high dynamic range (HDR) light probe sequencesin real-time. Such images (light probes) are captured using a high resolution HDR camera. In each frame ofthe HDR video, an optimized CUDA kernel is used to project incident lighting into spherical harmonics in realtime. Transfer coefficients are calculated in an offline process. Using precomputed radiance transfer the radiancecalculation reduces to a low order dot product between lighting and transfer coefficients. We exploit temporalcoherence between frames to further smooth lighting variation over time. Our results show that the frameworkcan achieve the effects of consistent illumination in real-time with flexibility to respond to dynamic changes in thereal environment.
|
|
| 20. |
- Hajisharif, Saghi, et al.
(författare)
-
Single Sensor Compressive Light Field Video Camera
- 2020
-
Ingår i: Computer graphics forum (Print). - : Wiley-Blackwell. - 0167-7055 .- 1467-8659. ; 39:2, s. 463-474
-
Tidskriftsartikel (refereegranskat)abstract
- This paper presents a novel compressed sensing (CS) algorithm and camera design for light field video capture using a single sensor consumer camera module. Unlike microlens light field cameras which sacrifice spatial resolution to obtain angular information, our CS approach is designed for capturing light field videos with high angular, spatial, and temporal resolution. The compressive measurements required by CS are obtained using a random color-coded mask placed between the sensor and aperture planes. The convolution of the incoming light rays from different angles with the mask results in a single image on the sensor; hence, achieving a significant reduction on the required bandwidth for capturing light field videos. We propose to change the random pattern on the spectral mask between each consecutive frame in a video sequence and extracting spatioangular- spectral-temporal 6D patches. Our CS reconstruction algorithm for light field videos recovers each frame while taking into account the neighboring frames to achieve significantly higher reconstruction quality with reduced temporal incoherencies, as compared with previous methods. Moreover, a thorough analysis of various sensing models for compressive light field video acquisition is conducted to highlight the advantages of our method. The results show a clear advantage of our method for monochrome sensors, as well as sensors with color filter arrays.
|
|
| 21. |
- Hanji, Param, et al.
(författare)
-
Comparison of single image HDR reconstruction methods — the caveats of quality assessment
- 2022
-
Ingår i: ACM SIGGRAPH ’22 Conference Proceedings. - New York, NY, USA : ACM. - 9781450393379
-
Konferensbidrag (refereegranskat)abstract
- As the problem of reconstructing high dynamic range (HDR) imagesfrom a single exposure has attracted much research effort, it isessential to provide a robust protocol and clear guidelines on howto evaluate and compare new methods. In this work, we comparedsix recent single image HDR reconstruction (SI-HDR) methodsin a subjective image quality experiment on an HDR display. Wefound that only two methods produced results that are, on average,more preferred than the unprocessed single exposure images. Whenthe same methods are evaluated using image quality metrics, astypically done in papers, the metric predictions correlate poorlywith subjective quality scores. The main reason is a significant toneand color difference between the reference and reconstructed HDRimages. To improve the predictions of image quality metrics, we propose correcting for the inaccuracies of the estimated cameraresponse curve before computing quality values. We further analyzethe sources of prediction noise when evaluating SI-HDR methodsand demonstrate that existing metrics can reliably predict onlylarge quality differences.
|
|
| 22. |
-
Image Analysis
- 2019
-
Proceedings (redaktörskap) (refereegranskat)abstract
- This volume constitutes the refereed proceedings of the 21st Scandinavian Conference on Image Analysis, SCIA 2019, held in Norrköping, Sweden, in June 2019.The 40 revised papers presented were carefully reviewed and selected from 63 submissions. The contributions are structured in topical sections on Deep convolutional neural networks; Feature extraction and image analysis; Matching, tracking and geometry; and Medical and biomedical image analysis.
|
|
| 23. |
- Jaroudi, Rym, 1989-
(författare)
-
Inverse Problems for Tumour Growth Models and Neural ODEs
- 2023
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- This thesis concerns the application of methods and techniques from the theory of inverse problems and differential equations to study models arising in the areas of mathematical oncology and deep learning. The first problem studied is to develop methods to perform numerical simulations with full 3-dimensional brain imaging data of reaction-diffusion models for tumour growth forwards as well as backwards in time with the goal of enabling the numerical reconstruction of the source of the tumour given an image (or similar data) at a later stage in time of the tumour. This inverse ill-posed problem is solved as a sequence of well-posed forward problems using the nonlinear Landweber regularization method. Such models and method allow to generate realistic synthetic medical images that can be used for data augmentation. Mathematical analysis of the problems solved as well as establishing uniqueness of the source are presented. The second problem includes a novel method allowing training self-contained neural ordinary differential equation networks (termed standalone NODEs) via a nonlinear conjugate gradient method, where the Sobolev gradient can be incorporated to improve smoothness of model weights. Relevant functions spaces are introduced, the adjoint problems with the needed gradients are calculated and the robustness is studied. The developed framework has many advantages in that it can incorporate relevant dynamics from physical models as well as help to understand more on how neural networks actually work and how sensitive they are to natural and adversarial perturbations. Combination of the two main problems will allow for example the training of neural networks to identify tumours in real imaging data.
|
|
| 24. |
- Jaroudi, Rym, 1989-, et al.
(författare)
-
Standalone Neural ODEs with Sensitivity Analysis
-
Annan publikation (övrigt vetenskapligt/konstnärligt)abstract
- This paper presents the Standalone Neural ODE (sNODE), a continuous-depth neural ODE model capable of describing a full deep neural network. This uses a novel nonlinear conjugate gradient (NCG) descent optimization scheme for training, where the Sobolev gradient can be incorporated to improve smoothness of model weights. We also present a general formulation of the neural sensitivity problem and show how it is used in the NCG training. The sensitivity analysis provides a reliable measure of uncertainty propagation throughout a network, and can be used to study model robustness and to generate adversarial attacks. Our evaluations demonstrate that our novel formulations lead to increased robustness and performance as compared to ResNet models, and that it opens up for new opportunities for designing and developing machine learning with improved explainability.
|
|
| 25. |
- Jönsson, Daniel, 1984-, et al.
(författare)
-
Direct Transmittance Estimation in Heterogeneous Participating Media Using Approximated Taylor Expansions
- 2022
-
Ingår i: IEEE Transactions on Visualization and Computer Graphics. - : Institute of Electrical and Electronics Engineers (IEEE). - 1077-2626 .- 1941-0506. ; 28:7, s. 2602-2614
-
Tidskriftsartikel (refereegranskat)abstract
- Evaluating the transmittance between two points along a ray is a key component in solving the light transport through heterogeneous participating media and entails computing an intractable exponential of the integrated medium's extinction coefficient. While algorithms for estimating this transmittance exist, there is a lack of theoretical knowledge about their behaviour, which also prevent new theoretically sound algorithms from being developed. For this purpose, we introduce a new class of unbiased transmittance estimators based on random sampling or truncation of a Taylor expansion of the exponential function. In contrast to classical tracking algorithms, these estimators are non-analogous to the physical light transport process and directly sample the underlying extinction function without performing incremental advancement. We present several versions of the new class of estimators, based on either importance sampling or Russian roulette to provide finite unbiased estimators of the infinite Taylor series expansion. We also show that the well known ratio tracking algorithm can be seen as a special case of the new class of estimators. Lastly, we conduct performance evaluations on both the central processing unit (CPU) and the graphics processing unit (GPU), and the results demonstrate that the new algorithms outperform traditional algorithms for heterogeneous mediums.
|
|
| 26. |
- Jönsson, Daniel, 1984-, et al.
(författare)
-
Visual Analysis of the Impact of Neural Network Hyper-Parameters
- 2020
-
Ingår i: Machine Learning Methods in Visualisation for Big Data 2020. - : Eurographics - European Association for Computer Graphics. - 9783038681137
-
Konferensbidrag (refereegranskat)abstract
- We present an analysis of the impact of hyper-parameters for an ensemble of neural networks using tailored visualization techniques to understand the complicated relationship between hyper-parameters and model performance. The high-dimensional error surface spanned by the wide range of hyper-parameters used to specify and optimize neural networks is difficult to characterize - it is non-convex and discontinuous, and there could be complex local dependencies between hyper-parameters. To explore these dependencies, we make use of a large number of sampled relations between hyper-parameters and end performance, retrieved from thousands of individually trained convolutional neural network classifiers. We use a structured selection of visualization techniques to analyze the impact of different combinations of hyper-parameters. The results reveal how complicated dependencies between hyper-parameters influence the end performance, demonstrating how the complete picture painted by considering a large number of trainings simultaneously can aid in understanding the impact of hyper-parameter combinations.
|
|
| 27. |
- Kavoosighafi, Behnaz, 1995-, et al.
(författare)
-
SparseBTF: Sparse Representation Learning for Bidirectional Texture Functions
- 2023
-
Konferensbidrag (refereegranskat)abstract
- We propose a novel dictionary-based representation learning model for Bidirectional Texture Functions (BTFs) aiming atcompact storage, real-time rendering performance, and high image quality. Our model is trained once, using a small trainingset, and then used to obtain a sparse tensor containing the model parameters. Our technique exploits redundancies in the dataacross all dimensions simultaneously, as opposed to existing methods that use only angular information and ignore correlationsin the spatial domain. We show that our model admits efficient angular interpolation directly in the model space, rather thanthe BTF space, leading to a notably higher rendering speed than in previous work. Additionally, the high quality-storage costtradeoff enabled by our method facilitates controlling the image quality, storage cost, and rendering speed using a singleparameter, the number of coefficients. Previous methods rely on a fixed number of latent variables for training and testing,hence limiting the potential for achieving a favorable quality-storage cost tradeoff and scalability. Our experimental resultsdemonstrate that our method outperforms existing methods both quantitatively and qualitatively, as well as achieving a highercompression ratio and rendering speed.
|
|
| 28. |
- Kronander, Joel, et al.
(författare)
-
Real-time HDR video reconstruction for multi-sensor systems
- 2012
-
Ingår i: ACM SIGGRAPH 2012 Posters. - New York, NY, USA : ACM Press. ; , s. 65-
-
Konferensbidrag (refereegranskat)abstract
- HDR video is an emerging field of technology, with a few camera systems currently in existence [Myszkowski et al. 2008], Multi-sensor systems [Tocci et al. 2011] have recently proved to be particularly promising due to superior robustness against temporal artifacts, correct motion blur, and high light efficiency. Previous HDR reconstruction methods for multi-sensor systems have assumed pixel perfect alignment of the physical sensors. This is, however, very difficult to achieve in practice. It may even be the case that reflections in beam splitters make it impossible to match the arrangement of the Bayer filters between sensors. We therefor present a novel reconstruction method specifically designed to handle the case of non-negligible misalignments between the sensors. Furthermore, while previous reconstruction techniques have considered HDR assembly, debayering and denoising as separate problems, our method is capable of simultaneous HDR assembly, debayering and smoothing of the data (denoising). The method is also general in that it allows reconstruction to an arbitrary output resolution and mapping. The algorithm is implemented in CUDA, and shows video speed performance for an experimental HDR video platform consisting of four 2336x1756 pixels high quality CCD sensors imaging the scene trough a common optical system. ND-filters of different densities are placed in front of the sensors to capture a dynamic range of 24 f-stops.
|
|
| 29. |
- Kronander, Joel, et al.
(författare)
-
Real-time video based lighting using GPU raytracing
- 2014
-
Ingår i: Proceedings of the 22nd European Signal Processing Conference (EUSIPCO), 2014. - : IEEE Signal Processing Society.
-
Konferensbidrag (refereegranskat)abstract
- The recent introduction of HDR video cameras has enabled the development of image based lighting techniques for rendering virtual objects illuminated with temporally varying real world illumination. A key challenge in this context is that rendering realistic objects illuminated with video environment maps is computationally demanding. In this work, we present a GPU based rendering system based on the NVIDIA OptiX framework, enabling real time raytracing of scenes illuminated with video environment maps. For this purpose, we explore and compare several Monte Carlo sampling approaches, including bidirectional importance sampling, multiple importance sampling and sequential Monte Carlo samplers. While previous work have focused on synthetic data and overly simple environment maps sequences, we have collected a set of real world dynamic environment map sequences using a state-of-art HDR video camera for evaluation and comparisons.
|
|
| 30. |
- Kronander, Joel, et al.
(författare)
-
Unified HDR reconstruction from raw CFA data
- 2013
-
Ingår i: Proceedings of IEEE International Conference on Computational Photography 2013. - : IEEE. - 9781467364638 ; , s. 1-9
-
Konferensbidrag (refereegranskat)abstract
- HDR reconstruction from multiple exposures poses several challenges. Previous HDR reconstruction techniques have considered debayering, denoising, resampling (alignment) and exposure fusion in several steps. We instead present a unifying approach, performing HDR assembly directly from raw sensor data in a single processing operation. Our algorithm includes a spatially adaptive HDR reconstruction based on fitting local polynomial approximations to observed sensor data, using a localized likelihood approach incorporating spatially varying sensor noise. We also present a realistic camera noise model adapted to HDR video. The method allows reconstruction to an arbitrary resolution and output mapping. We present an implementation in CUDA and show real-time performance for an experimental 4 Mpixel multi-sensor HDR video system. We further show that our algorithm has clear advantages over state-of-the-art methods, both in terms of flexibility and reconstruction quality.
|
|
| 31. |
- Löw, Joakim, et al.
(författare)
-
ABC - BRDF Models for Accurate and Efficient Rendering of Glossy Surfaces
- 2013
-
Ingår i: Eurographics 24th Symposium on Rendering.
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
- Glossy surface reflectance is hard to model accuratley using traditional parametric BRDF models. An alternative is provided by data driven reflectance models, however these models offers less user control and generally results in lower efficency. In our work we propose two new lightweight parameteric BRDF models for accurate modeling of glossy surface refllectance, one inspired by Rayleigh-Rice theory for optically smooth surfaces and one inspired by microfacet-theory. We base our models on a thourough study of the scattering behaviour of measured reflectance data from the MERL database. The study focuses on two key aspects of BRDF models, parametrization and scatter distribution. We propose a new scattering distributuion for glossy BRDFs inspired by the ABC model for surface statistics of optically smooth surfaces. Based on the survey we consider two parameterizations, one based on micro-facet theory using the halfway vector and one inspired by the parametrization for the Rayleigh-Rice BRDF model considering the projected devaition vector. To enable efficent rendering we also show how the new models can be approximatley sampled for importance sampling the scattering integral.
|
|
| 32. |
- Miandji, Ehsan, 1985-, et al.
(författare)
-
A Unified Framework for Compression and Compressed Sensing of Light Fields and Light Field Videos
- 2019
-
Ingår i: ACM Transactions on Graphics. - : ACM Digital Library. - 0730-0301 .- 1557-7368. ; 38:3, s. 1-18
-
Tidskriftsartikel (refereegranskat)abstract
- In this article we present a novel dictionary learning framework designed for compression and sampling of light fields and light field videos. Unlike previous methods, where a single dictionary with one-dimensional atoms is learned, we propose to train a Multidimensional Dictionary Ensemble (MDE). It is shown that learning an ensemble in the native dimensionality of the data promotes sparsity, hence increasing the compression ratio and sampling efficiency. To make maximum use of correlations within the light field data sets, we also introduce a novel nonlocal pre-clustering approach that constructs an Aggregate MDE (AMDE). The pre-clustering not only improves the image quality but also reduces the training time by an order of magnitude in most cases. The decoding algorithm supports efficient local reconstruction of the compressed data, which enables efficient real-time playback of high-resolution light field videos. Moreover, we discuss the application of AMDE for compressed sensing. A theoretical analysis is presented that indicates the required conditions for exact recovery of point-sampled light fields that are sparse under AMDE. The analysis provides guidelines for designing efficient compressive light field cameras. We use various synthetic and natural light field and light field video data sets to demonstrate the utility of our approach in comparison with the state-of-the-art learning-based dictionaries, as well as established analytical dictionaries.
|
|
| 33. |
- Miandji, Ehsan, et al.
(författare)
-
Compressive Image Reconstruction in Reduced Union of Subspaces
- 2015
-
Ingår i: Computer Graphics Forum. - : John Wiley & Sons Ltd. - 1467-8659 .- 0167-7055. ; 34:2, s. 33-44
-
Tidskriftsartikel (refereegranskat)abstract
- We present a new compressed sensing framework for reconstruction of incomplete and possibly noisy images and their higher dimensional variants, e.g. animations and light-fields. The algorithm relies on a learning-based basis representation. We train an ensemble of intrinsically two-dimensional (2D) dictionaries that operate locally on a set of 2D patches extracted from the input data. We show that one can convert the problem of 2D sparse signal recovery to an equivalent 1D form, enabling us to utilize a large family of sparse solvers. The proposed framework represents the input signals in a reduced union of subspaces model, while allowing sparsity in each subspace. Such a model leads to a much more sparse representation than widely used methods such as K-SVD. To evaluate our method, we apply it to three different scenarios where the signal dimensionality varies from 2D (images) to 3D (animations) and 4D (light-fields). We show that our method outperforms state-of-the-art algorithms in computer graphics and image processing literature.
|
|
| 34. |
- Miandji, Ehsan, et al.
(författare)
-
Learning based compression for real-time rendering of surface light fields
- 2013
-
Ingår i: Siggraph 2013 Posters. - New York, NY, USA : ACM Press. - 9781450323420
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
- Photo-realistic image synthesis in real-time is a key challenge in computer graphics. A number of techniques where the light transport in a scene is pre-computed, compressed and used for real-time image synthesis have been proposed. In this work, we extend this idea and present a technique where the radiance distribution in a scene, including arbitrarily complex materials and light sources, is pre-computed using photo-realistic rendering techniques and stored as surface light fields (SLF) at each surface. An SLF describes the full appearance of each surface in a scene as a 4D function over the spatial and angular domains. An SLF is a complex data set with a large memory footprint often in the order of several GB per object in the scene. The key contribution in this work is a novel approach for compression of surface light fields that enables real-time rendering of complex scenes. Our learning-based compression technique is based on exemplar orthogonal bases (EOB), and trains a compact dictionary of full-rank orthogonal basis pairs with sparse coefficients. Our results outperform the widely used CPCA method in terms of storage cost, visual quality and rendering speed. Compared to PRT techniques for real-time global illumination, our approach is limited to static scenes but can represent high frequency materials and any type of light source in a unified framework.
|
|
| 35. |
- Miandji, Ehsan, et al.
(författare)
-
Learning Based Compression of Surface Light Fields for Real-time Rendering of Global Illumination Scenes
- 2013
-
Ingår i: Proceedings of ACM SIGGRAPH ASIA 2013. - New York, NY, USA : ACM Press. - 9781450326292
-
Konferensbidrag (refereegranskat)abstract
- We present an algorithm for compression and real-time rendering of surface light fields (SLF) encoding the visual appearance of objects in static scenes with high frequency variations. We apply a non-local clustering in order to exploit spatial coherence in the SLFdata. To efficiently encode the data in each cluster, we introducea learning based approach, Clustered Exemplar Orthogonal Bases(CEOB), which trains a compact dictionary of orthogonal basispairs, enabling efficient sparse projection of the SLF data. In ad-dition, we discuss the application of the traditional Clustered Principal Component Analysis (CPCA) on SLF data, and show that inmost cases, CEOB outperforms CPCA, K-SVD and spherical harmonics in terms of memory footprint, rendering performance andreconstruction quality. Our method enables efficient reconstructionand real-time rendering of scenes with complex materials and lightsources, not possible to render in real-time using previous methods.
|
|
| 36. |
- Miandji, Ehsan, 1985-, et al.
(författare)
-
ON NONLOCAL IMAGE COMPLETION USING AN ENSEMBLE OF DICTIONARIES
- 2016
-
Ingår i: 2016 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP). - : IEEE. - 9781467399616 ; , s. 2519-2523
-
Konferensbidrag (refereegranskat)abstract
- In this paper we consider the problem of nonlocal image completion from random measurements and using an ensemble of dictionaries. Utilizing recent advances in the field of compressed sensing, we derive conditions under which one can uniquely recover an incomplete image with overwhelming probability. The theoretical results are complemented by numerical simulations using various ensembles of analytical and training-based dictionaries.
|
|
| 37. |
- Miandji, Ehsan, 1985-, et al.
(författare)
-
On Probability of Support Recovery for Orthogonal Matching Pursuit Using Mutual Coherence
- 2017
-
Ingår i: IEEE Signal Processing Letters. - : IEEE Signal Processing Society. - 1070-9908 .- 1558-2361. ; 24:11, s. 1646-1650
-
Tidskriftsartikel (refereegranskat)abstract
- In this paper we present a new coherence-based performance guarantee for the Orthogonal Matching Pursuit (OMP) algorithm. A lower bound for the probability of correctly identifying the support of a sparse signal with additive white Gaussian noise is derived. Compared to previous work, the new bound takes into account the signal parameters such as dynamic range, noise variance, and sparsity. Numerical simulations show significant improvements over previous work and a closer match to empirically obtained results of the OMP algorithm.
|
|
| 38. |
- Miandji, Ehsan, 1985-
(författare)
-
Sparse representation of visual data for compression and compressed sensing
- 2018
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- The ongoing advances in computational photography have introduced a range of new imaging techniques for capturing multidimensional visual data such as light fields, BRDFs, BTFs, and more. A key challenge inherent to such imaging techniques is the large amount of high dimensional visual data that is produced, often requiring GBs, or even TBs, of storage. Moreover, the utilization of these datasets in real time applications poses many difficulties due to the large memory footprint. Furthermore, the acquisition of large-scale visual data is very challenging and expensive in most cases. This thesis makes several contributions with regards to acquisition, compression, and real time rendering of high dimensional visual data in computer graphics and imaging applications.Contributions of this thesis reside on the strong foundation of sparse representations. Numerous applications are presented that utilize sparse representations for compression and compressed sensing of visual data. Specifically, we present a single sensor light field camera design, a compressive rendering method, a real time precomputed photorealistic rendering technique, light field (video) compression and real time rendering, compressive BRDF capture, and more. Another key contribution of this thesis is a general framework for compression and compressed sensing of visual data, regardless of the dimensionality. As a result, any type of discrete visual data with arbitrary dimensionality can be captured, compressed, and rendered in real time.This thesis makes two theoretical contributions. In particular, uniqueness conditions for recovering a sparse signal under an ensemble of multidimensional dictionaries is presented. The theoretical results discussed here are useful for designing efficient capturing devices for multidimensional visual data. Moreover, we derive the probability of successful recovery of a noisy sparse signal using OMP, one of the most widely used algorithms for solving compressed sensing problems.
|
|
| 39. |
- Neset, Tina-Simone, Professor, et al.
(författare)
-
AI för klimatanpassning : Hur kan nya digitala teknologier stödja klimatanpassning?
- 2024
-
Rapport (övrigt vetenskapligt/konstnärligt)abstract
- Tillgång till vädervarningar med information om förväntade konsekvenser av vädret är nödvändigt för god krisberedskap hos myndigheter, kommuner, näringsliv och privatpersoner. Vidareutveckling av varningssystem som fokuserar på förväntade störningar (konsekvensbaserade varningssystem) är därför en viktig komponent i samhällets hantering av klimatförändringar. Forskningsprojektet AI för klimatanpassning (AI4CA) har analyserat möjligheter och hinder med att inkludera AI-baserad text- och bildanalys som stöd till SMHI:s konsekvensbaserade vädervarningssystem och på sikt även stödja långsiktig klimatanpassning.
|
|
| 40. |
- Nilsson, Jens, 1970-, et al.
(författare)
-
Swedish civil air traffic control dataset
- 2023
-
Ingår i: Data in Brief. - : ELSEVIER. - 2352-3409. ; 48
-
Tidskriftsartikel (refereegranskat)abstract
- The Swedish Civil Air Traffic Control (SCAT) dataset consists of 13 weeks of data collected from the area control in Sweden flight information region. The dataset consists of detailed data from almost 170,000 flights as well as airspace data and weather forecasts. The flight data includes system updated flight plans, clearances from air traffic control, surveillance data and trajectory prediction data. Each week of data is continuous but the 13 weeks are spread over one year to provide variations in weather and seasonal traffic patterns. The dataset does only include scheduled flights not involved in any incident reports. Sensitive data such as military and private flight has been removed.The SCAT dataset can be useful for any research related to air traffic control, e.g. analysis of transportation patterns, environmental impact, optimization and automation/AI.
|
|
| 41. |
- Stacke, Karin, et al.
(författare)
-
A Closer Look at Domain Shift for Deep Learning in Histopathology
- 2019
-
Konferensbidrag (refereegranskat)abstract
- Domain shift is a significant problem in histopathology. There can be large differences in data characteristics of whole-slide images between medical centers and scanners, making generalization of deep learning to unseen data difficult. To gain a better understanding of the problem, we present a study on convolutional neural networks trained for tumor classification of H&E stained whole-slide images. We analyze how augmentation and normalization strategies affect performance and learned representations, and what features a trained model respond to. Most centrally, we present a novel measure for evaluating the distance between domains in the context of the learned representation of a particular model. This measure can reveal how sensitive a model is to domain variations, and can be used to detect new data that a model will have problems generalizing to. The results show how learning is heavily influenced by the preparation of training data, and that the latent representation used to do classification is sensitive to changes in data distribution, especially when training without augmentation or normalization.
|
|
| 42. |
- Stacke, Karin, 1990-
(författare)
-
Deep Learning for Digital Pathology in Limited Data Scenarios
- 2022
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- The impressive technical advances seen for machine learning algorithms in combination with the digitalization of medical images in the radiology and pathology departments show great promise in introducing powerful image analysis tools for image diagnostics. In particular, deep learning, a subfield within machine learning, has shown great success, advancing fields such as image classification and detection. However, these types of algorithms are only used to a very small extent in clinical practice. One reason is that the unique nature of radiology and pathology images and the clinical setting in which they are acquired poses challenges not seen in other image domains. Differences relate to capturing methods, as well as the image contents. In addition, these datasets are not only unique on a per-image basis but as a collective dataset. Characteristics such as size, class balance, and availability of annotated labels make creating robust and generalizable deep learning methods a challenge. This thesis investigates how deep learning models can be trained for applications in this domain, with particular focus on histopathology data. We investigate how domain shift between different scanners causes performance drop, and present ways of mitigating this. We also present a method to detect when domain shift occurs between different datasets. Another hurdle is the shortage of labeled data for medical applications, and this thesis looks at two different approaches to solving this problem. The first approach investigates how labeled data from one organ and cancer type can boost cancer classification in another organ where labeled data is scarce. The second approach looks at a specific type of unsupervised learning method, self-supervised learning, where the model is trained on unlabeled data. For both of these approaches, we present strategies to handle low-data regimes that may greatly increase the availability to build deep learning models for a wider range of applications. Furthermore, deep learning technology enables us to go beyond traditional medical domains, and combine the data from both radiology and pathology. This thesis presents a method for improved cancer characterization on contrast-enhanced CT by incorporating corresponding pathology data during training. The method shows the potential of im-proving future healthcare by intergraded diagnostics made possible by machine-learning technology.
|
|
| 43. |
|
|
| 44. |
- Stacke, Karin, 1990-, et al.
(författare)
-
Learning Representations with Contrastive Self-Supervised Learning for Histopathology Applications
- 2022
-
Ingår i: The Journal of Machine Learning for Biomedical Imaging. - : Melba (The Journal of Machine Learning for Biomedical Imaging). - 2766-905X. ; 1
-
Tidskriftsartikel (övrigt vetenskapligt/konstnärligt)abstract
- Unsupervised learning has made substantial progress over the last few years, especially by means of contrastive self-supervised learning. The dominating dataset for benchmarking self-supervised learning has been ImageNet, for which recent methods are approaching the performance achieved by fully supervised training. The ImageNet dataset is however largely object-centric, and it is not clear yet what potential those methods have on widely different datasets and tasks that are not object-centric, such as in digital pathology.While self-supervised learning has started to be explored within this area with encouraging results, there is reason to look closer at how this setting differs from natural images and ImageNet. In this paper we make an in-depth analysis of contrastive learning for histopathology, pin-pointing how the contrastive objective will behave differently due to the characteristics of histopathology data. Using SimCLR and H&E stained images as a representative setting for contrastive self-supervised learning in histopathology, we bring forward a number of considerations, such as view generation for the contrastive objectiveand hyper-parameter tuning. In a large battery of experiments, we analyze how the downstream performance in tissue classification will be affected by these considerations. The results point to how contrastive learning can reduce the annotation effort within digital pathology, but that the specific dataset characteristics need to be considered. To take full advantage of the contrastive learning objective, different calibrations of view generation and hyper-parameters are required. Our results pave the way for realizing the full potential of self-supervised learning for histopathology applications. Code and trained models are available at https://github.com/k-stacke/ssl-pathology.
|
|
| 45. |
- Stacke, Karin, 1990-, et al.
(författare)
-
Measuring Domain Shift for Deep Learning in Histopathology
- 2021
-
Ingår i: IEEE journal of biomedical and health informatics. - : IEEE. - 2168-2194 .- 2168-2208. ; 25:2, s. 325-336
-
Tidskriftsartikel (refereegranskat)abstract
- The high capacity of neural networks allows fitting models to data with high precision, but makes generalization to unseen data a challenge. If a domain shift exists, i.e. differences in image statistics between training and test data, care needs to be taken to ensure reliable deployment in real-world scenarios. In digital pathology, domain shift can be manifested in differences between whole-slide images, introduced by for example differences in acquisition pipeline - between medical centers or over time. In order to harness the great potential presented by deep learning in histopathology, and ensure consistent model behavior, we need a deeper understanding of domain shift and its consequences, such that a model's predictions on new data can be trusted. This work focuses on the internal representation learned by trained convolutional neural networks, and shows how this can be used to formulate a novel measure - the representation shift - for quantifying the magnitude of model specific domain shift. We perform a study on domain shift in tumor classification of hematoxylin and eosin stained images, by considering different datasets, models, and techniques for preparing data in order to reduce the domain shift. The results show how the proposed measure has a high correlation with drop in performance when testing a model across a large number of different types of domain shifts, and how it improves on existing techniques for measuring data shift and uncertainty. The proposed measure can reveal how sensitive a model is to domain variations, and can be used to detect new data that a model will have problems generalizing to. We see techniques for measuring, understanding and overcoming the domain shift as a crucial step towards reliable use of deep learning in the future clinical pathology applications.
|
|
| 46. |
- Tongbuasirilai, Tanaboon, 1983-, et al.
(författare)
-
Compact and intuitive data-driven BRDF models
- 2020
-
Ingår i: The Visual Computer. - : Springer Berlin/Heidelberg. - 0178-2789 .- 1432-2315. ; 36:4, s. 855-872
-
Tidskriftsartikel (refereegranskat)abstract
- Measured materials are rapidly becoming a core component in the photo-realistic image synthesis pipeline. The reason is that data-driven models can easily capture the underlying, fine details that represent the visual appearance of materials, which can be difficult or even impossible to model by hand. There are, however, a number of key challenges that need to be solved in order to enable efficient capture, representation and interaction with real materials. This paper presents two new data-driven BRDF models specifically designed for 1D separability. The proposed 3D and 2D BRDF representations can be factored into three or two 1D factors, respectively, while accurately representing the underlying BRDF data with only small approximation error. We evaluate the models using different parameterizations with different characteristics and show that both the BRDF data itself and the resulting renderings yield more accurate results in terms of both numerical errors and visual results compared to previous approaches. To demonstrate the benefit of the proposed factored models, we present a new Monte Carlo importance sampling scheme and give examples of how they can be used for efficient BRDF capture and intuitive editing of measured materials.
|
|
| 47. |
- Tongbuasirilai, Tanaboon, 1983-
(författare)
-
Data-Driven Approaches for Sparse Reflectance Modeling and Acquisition
- 2023
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Photo-realistic rendering and predictive image synthesis are becoming increasingly important and utilized in many application areas ranging from production of visual effects and product visualization to digital design and the generation of synthetic data for visual machine learning applications. Many essential components of the realistic image synthesis pipelines have been developed tremendously over the last decades. One key component is accurate measurement, modeling, and simulation of how a surface material scatters light. The scattering of light at a point on a surface (reflectance and color) is described by the Bidirectional Reflectance Distribution Function (BRDF); which is the main research topic of this thesis. The BRDF describes how radiance, light, incident at a point on a surface is scattered towards any view-point from which the surface is observed. Accurate acquisition and representation of material properties play a fundamental role in photo-realistic image synthesis, and form a highly interesting research topic with many applications. The thesis has explored and studied appearance modeling, sparse representation and sparse acquisition of BRDFs. The topics of this thesis cover two main areas. Within the first area, BRDF modeling, we propose several new BRDF models for accurate representation of material scattering behaviour using simple but efficient methods. The research challenges in BRDF modeling include tensor decomposition methods and sparse approximations based on measured BRDF data. The second part of the contributions focuses on sparse BRDF sampling and novel highly efficient BRDF acquisition. The sparse BRDF sampling is to tackle tedious and time-consuming processes for acquiring BRDFs. This challenging problem is addressed using sparse modeling and compressed sensing techniques and enables a BRDF to be measured and accurately reconstructed using only a small number of samples. Additionally, the thesis provides example applications based on the research, as well as a techniques for BRDF editing and interpolation. Publicly available BRDF databases are a vital part of the data-driven methods proposed in this thesis. The measured BRDF data used has revealed insights to facilitate further development of the proposed methods. The results, algorithms, and techniques presented in this thesis demonstrate that there is a close connection between BRDF modeling and BRDF acquisition; efficient and accurate BRDF modeling is a by-product of sparse BRDF sampling.
|
|
| 48. |
- Tongbuasirilai, Tanaboon, 1983-, et al.
(författare)
-
Efficient BRDF Sampling Using Projected Deviation Vector Parameterization
- 2017
-
Ingår i: 2017 IEEE International Conference on Computer Vision Workshops (ICCVW). - : Institute of Electrical and Electronics Engineers (IEEE). - 9781538610343 - 9781538610350 ; , s. 153-158
-
Konferensbidrag (refereegranskat)abstract
- This paper presents a novel approach for efficient sampling of isotropic Bidirectional Reflectance Distribution Functions (BRDFs). Our approach builds upon a new parameterization, the Projected Deviation Vector parameterization, in which isotropic BRDFs can be described by two 1D functions. We show that BRDFs can be efficiently and accurately measured in this space using simple mechanical measurement setups. To demonstrate the utility of our approach, we perform a thorough numerical evaluation and show that the BRDFs reconstructed from measurements along the two 1D bases produce rendering results that are visually comparable to the reference BRDF measurements which are densely sampled over the 4D domain described by the standard hemispherical parameterization.
|
|
| 49. |
|
|
| 50. |
- Tsirikoglou, Apostolia, 1985-, et al.
(författare)
-
A Survey of Image Synthesis Methods for Visual Machine Learning
- 2020
-
Ingår i: Computer graphics forum (Print). - : John Wiley & Sons. - 0167-7055 .- 1467-8659. ; 39:6, s. 426-451
-
Tidskriftsartikel (refereegranskat)abstract
- Image synthesis designed for machine learning applications provides the means to efficiently generate large quantities of training data while controlling the generation process to provide the best distribution and content variety. With the demands of deep learning applications, synthetic data have the potential of becoming a vital component in the training pipeline. Over the last decade, a wide variety of training data generation methods has been demonstrated. The potential of future development calls to bring these together for comparison and categorization. This survey provides a comprehensive list of the existing image synthesis methods for visual machine learning. These are categorized in the context of image generation, using a taxonomy based on modelling and rendering, while a classification is also made concerning the computer vision applications they are used. We focus on the computer graphics aspects of the methods, to promote future image generation for machine learning. Finally, each method is assessed in terms of quality and reported performance, providing a hint on its expected learning potential. The report serves as a comprehensive reference, targeting both groups of the applications and data development sides. A list of all methods and papers reviewed herein can be found at https://computergraphics.on.liu.se/image_synthesis_methods_for_visual_machine_learning/.
|
|
