| 1. |
- Eilertsen, Gabriel, et al.
(författare)
-
HDR image reconstruction from a single exposure using deep CNNs
- 2017
-
Ingår i: ACM Transactions on Graphics. - : ASSOC COMPUTING MACHINERY. - 0730-0301 .- 1557-7368. ; 36:6
-
Tidskriftsartikel (refereegranskat)abstract
- Camera sensors can only capture a limited range of luminance simultaneously, and in order to create high dynamic range (HDR) images a set of different exposures are typically combined. In this paper we address the problem of predicting information that have been lost in saturated image areas, in order to enable HDR reconstruction from a single exposure. We show that this problem is well-suited for deep learning algorithms, and propose a deep convolutional neural network (CNN) that is specifically designed taking into account the challenges in predicting HDR values. To train the CNN we gather a large dataset of HDR images, which we augment by simulating sensor saturation for a range of cameras. To further boost robustness, we pre-train the CNN on a simulated HDR dataset created from a subset of the MIT Places database. We demonstrate that our approach can reconstruct high-resolution visually convincing HDR results in a wide range of situations, and that it generalizes well to reconstruction of images captured with arbitrary and low-end cameras that use unknown camera response functions and post-processing. Furthermore, we compare to existing methods for HDR expansion, and show high quality results also for image based lighting. Finally, we evaluate the results in a subjective experiment performed on an HDR display. This shows that the reconstructed HDR images are visually convincing, with large improvements as compared to existing methods.
|
|
| 2. |
- Hajisharif, Saghi, et al.
(författare)
-
Adaptive dualISO HDR-reconstruction
- 2015
-
Ingår i: EURASIP Journal on Image and Video Processing. - : Springer Publishing Company. - 1687-5176 .- 1687-5281.
-
Tidskriftsartikel (refereegranskat)abstract
- With the development of modern image sensors enabling flexible image acquisition, single shot HDR imaging is becoming increasingly popular. In this work we capture single shot HDR images using an imaging sensor with spatially varying gain/ISO. In comparison to previous single shot HDR capture based on a single sensor, this allows all incoming photons to be used in the imaging, instead of wasting incoming light using spatially varying ND-filters, commonly used in previous works. The main technical contribution in this work is an extension of previous HDR reconstruction approaches for single shot HDR imaging based on local polynomial approximations [15,10]. Using a sensor noise model, these works deploy a statistically informed filtering operation to reconstruct HDR pixel values. However, instead of using a fixed filter size, we introduce two novel algorithms for adaptive filter kernel selection. Unlike previous works, using adaptive filter kernels [16], our algorithms are based on analysing the model fit and the expected statistical deviation of the estimate based on the sensor noise model. Using an iterative procedure we can then adapt the filter kernel according to the image structure and the statistical image noise. Experimental results show that the proposed filter de-noises the noisy image carefully while well preserving the important image features such as edges and corners, outperforming previous methods. To demonstrate the robustness of our approach, we have exploited input images from raw sensor data using a commercial off-the shelf camera. To further analyze our algorithm, we have also implemented a camera simulator to evaluate different gain pattern and noise properties of the sensor.
|
|
| 3. |
- Hajisharif, Saghi, et al.
(författare)
-
Real-time image based lighting with streaming HDR-lightprobe sequences
- 2012
-
Ingår i: Proceedings of SIGRAD 2012. - Linköping, Sweden. - 9789175197234
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
- We present a framework for shading of virtual objects using high dynamic range (HDR) light probe sequencesin real-time. Such images (light probes) are captured using a high resolution HDR camera. In each frame ofthe HDR video, an optimized CUDA kernel is used to project incident lighting into spherical harmonics in realtime. Transfer coefficients are calculated in an offline process. Using precomputed radiance transfer the radiancecalculation reduces to a low order dot product between lighting and transfer coefficients. We exploit temporalcoherence between frames to further smooth lighting variation over time. Our results show that the frameworkcan achieve the effects of consistent illumination in real-time with flexibility to respond to dynamic changes in thereal environment.
|
|
| 4. |
- Hajsharif, Saghi, 1987-, et al.
(författare)
-
HDR reconstruction for alternating gain (ISO) sensor readout
- 2014
-
Ingår i: Eurographics 2014 short papers.
-
Konferensbidrag (refereegranskat)abstract
- Modern image sensors are becoming more and more flexible in the way an image is captured. In this paper, we focus on sensors that allow the per pixel gain to be varied over the sensor and develop a new technique for efficient and accurate reconstruction of high dynamic range (HDR) images based on such input data. Our method estimates the radiant power at each output pixel using a sampling operation which performs color interpolation, re-sampling, noise reduction and HDR-reconstruction in a single step. The reconstruction filter uses a sensor noise model to weight the input pixel samples according to their variances. Our algorithm works in only a small spatial neighbourhood around each pixel and lends itself to efficient implementation in hardware. To demonstrate the utility of our approach we show example HDR-images reconstructed from raw sensor data captured using off-the shelf consumer hardware which allows for two different gain settings for different rows in the same image. To analyse the accuracy of the algorithm, we also use synthetic images from a camera simulation software.
|
|
| 5. |
- Jönsson, Daniel, 1984-, et al.
(författare)
-
Direct Transmittance Estimation in Heterogeneous Participating Media Using Approximated Taylor Expansions
- 2022
-
Ingår i: IEEE Transactions on Visualization and Computer Graphics. - : Institute of Electrical and Electronics Engineers (IEEE). - 1077-2626 .- 1941-0506. ; 28:7, s. 2602-2614
-
Tidskriftsartikel (refereegranskat)abstract
- Evaluating the transmittance between two points along a ray is a key component in solving the light transport through heterogeneous participating media and entails computing an intractable exponential of the integrated medium's extinction coefficient. While algorithms for estimating this transmittance exist, there is a lack of theoretical knowledge about their behaviour, which also prevent new theoretically sound algorithms from being developed. For this purpose, we introduce a new class of unbiased transmittance estimators based on random sampling or truncation of a Taylor expansion of the exponential function. In contrast to classical tracking algorithms, these estimators are non-analogous to the physical light transport process and directly sample the underlying extinction function without performing incremental advancement. We present several versions of the new class of estimators, based on either importance sampling or Russian roulette to provide finite unbiased estimators of the infinite Taylor series expansion. We also show that the well known ratio tracking algorithm can be seen as a special case of the new class of estimators. Lastly, we conduct performance evaluations on both the central processing unit (CPU) and the graphics processing unit (GPU), and the results demonstrate that the new algorithms outperform traditional algorithms for heterogeneous mediums.
|
|
| 6. |
- Kronander, Joel, 1983-, et al.
(författare)
-
A unified framework for multi-sensor HDR video reconstruction
- 2014
-
Ingår i: Signal Processing : Image Communications. - : Elsevier. - 0923-5965. ; 29:2, s. 203-215
-
Tidskriftsartikel (refereegranskat)abstract
- One of the most successful approaches to modern high quality HDR-video capture is to use camera setups with multiple sensors imaging the scene through a common optical system. However, such systems pose several challenges for HDR reconstruction algorithms. Previous reconstruction techniques have considered debayering, denoising, resampling (alignment) and exposure fusion as separate problems. In contrast, in this paper we present a unifying approach, performing HDR assembly directly from raw sensor data. Our framework includes a camera noise model adapted to HDR video and an algorithm for spatially adaptive HDR reconstruction based on fitting of local polynomial approximations to observed sensor data. The method is easy to implement and allows reconstruction to an arbitrary resolution and output mapping. We present an implementation in CUDA and show real-time performance for an experimental 4 Mpixel multi-sensor HDR video system. We further show that our algorithm has clear advantages over existing methods, both in terms of flexibility and reconstruction quality.
|
|
| 7. |
- Kronander, Joel, 1983-, et al.
(författare)
-
Efficient Visibility Encoding for Dynamic Illumination in Direct Volume Rendering : -
- 2012
-
Ingår i: IEEE Transactions on Visualization and Computer Graphics. - : IEEE. - 1077-2626 .- 1941-0506. ; 18:3, s. 447-462
-
Tidskriftsartikel (refereegranskat)abstract
- We present an algorithm that enables real-time dynamic shading in direct volume rendering using general lighting, includingdirectional lights, point lights and environment maps. real-time performance is achieved by encoding local and global volumetricvisibility using spherical harmonic (SH) basis functions stored in an efficient multi-resolution grid over the extent of the volume. Ourmethod enables high frequency shadows in the spatial domain, but is limited to a low frequency approximation of visibility and illuminationin the angular domain. In a first pass, Level Of Detail (LOD) selection in the grid is based on the current transfer function setting.This enables rapid on-line computation and SH projection of the local spherical distribution of visibility information. Using a piecewiseintegration of the SH coefficients over the local regions, the global visibility within the volume is then computed. By representing thelight sources using their SH projections, the integral over lighting, visibility and isotropic phase functions can be efficiently computedduring rendering. The utility of our method is demonstrated in several examples showing the generality and interactive performanceof the approach.
|
|
| 8. |
- Kronander, Joel, et al.
(författare)
-
Estimation and Modeling of Actual Numerical Errors in Volume Rendering
- 2010
-
Ingår i: COMPUTER GRAPHICS FORUM. - : Blackwell Publishing Ltd. - 0167-7055. ; 29:3, s. 893-902
-
Tidskriftsartikel (refereegranskat)abstract
- In this paper we study the comprehensive effects on volume rendered images due to numerical errors caused by the use of finite precision for data representation and processing. To estimate actual error behavior we conduct a thorough study using a volume renderer implemented with arbitrary floating-point precision. Based on the experimental data we then model the impact of floating-point pipeline precision, sampling frequency and fixed-point input data quantization on the fidelity of rendered images. We introduce three models, an average model, which does not adapt to different data nor varying transfer functions, as well as two adaptive models that take the intricacies of a new data set and transfer function into account by adapting themselves given a few different images rendered. We also test and validate our models based on new data that was not used during our model building.
|
|
| 9. |
- Kronander, Joel, 1983-, et al.
(författare)
-
Photorealistic rendering of mixed reality scenes
- 2015
-
Ingår i: Computer graphics forum (Print). - : Wiley-Blackwell. - 0167-7055 .- 1467-8659. ; 34:2, s. 643-665
-
Tidskriftsartikel (refereegranskat)abstract
- Photo-realistic rendering of virtual objects into real scenes is one of the most important research problems in computer graphics. Methods for capture and rendering of mixed reality scenes are driven by a large number of applications, ranging from augmented reality to visual effects and product visualization. Recent developments in computer graphics, computer vision, and imaging technology have enabled a wide range of new mixed reality techniques including methods of advanced image based lighting, capturing spatially varying lighting conditions, and algorithms for seamlessly rendering virtual objects directly into photographs without explicit measurements of the scene lighting. This report gives an overview of the state-of-the-art in this field, and presents a categorization and comparison of current methods. Our in-depth survey provides a tool for understanding the advantages and disadvantages of each method, and gives an overview of which technique is best suited to a specific problem.
|
|
| 10. |
- Kronander, Joel, 1983-
(författare)
-
Physically Based Rendering of Synthetic Objects in Real Environments
- 2015
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- This thesis presents methods for photorealistic rendering of virtual objects so that they can be seamlessly composited into images of the real world. To generate predictable and consistent results, we study physically based methods, which simulate how light propagates in a mathematical model of the augmented scene. This computationally challenging problem demands both efficient and accurate simulation of the light transport in the scene, as well as detailed modeling of the geometries, illumination conditions, and material properties. In this thesis, we discuss and formulate the challenges inherent in these steps and present several methods to make the process more efficient.In particular, the material contained in this thesis addresses four closely related areas: HDR imaging, IBL, reflectance modeling, and efficient rendering. The thesis presents a new, statistically motivated algorithm for HDR reconstruction from raw camera data combining demosaicing, denoising, and HDR fusion in a single processing operation. The thesis also presents practical and robust methods for rendering with spatially and temporally varying illumination conditions captured using omnidirectional HDR video. Furthermore, two new parametric BRDF models are proposed for surfaces exhibiting wide angle gloss. Finally, the thesis also presents a physically based light transport algorithm based on Markov Chain Monte Carlo methods that allows approximations to be used in place of exact quantities, while still converging to the exact result. As illustrated in the thesis, the proposed algorithm enables efficient rendering of scenes with glossy transfer and heterogenous participating media.
|
|
