CMSC 740 Fall 2017 is taught by an awesome teacher Prof. Matthias Zwicker.
Projects
The course project is based on Nori, an educational ray tracer.
Notes
Acceleration structures,
Uniform grid
Object subdivision
Bounding volume hierarchies (BVHs)
Partition space into nonoverlapping cells
Each cell stores reference to all objects
that overlap it
Spatial subdivision
Binary space partitioning (BSP) trees
Recursively divide space into two parts using
dividing planes (with arbitrary position,
orientation)
kdtrees, nearest neighbor search
Light Transport Effects
Fluorescence
Reflection
Thin film interference: https://www.shadertoy.com/view/XddXRj
Diffraction
Refraction
Dispersion
Rayleigh Scattering: https://www.shadertoy.com/view/MdXSzX (Subsurface Scattering)
Polarization
Radiometry
Quantify spatial energy distribution of light
(in a way that is compatible with the geometrical optics model)
Physical measurement of electromagnetic energy
Unit: Watts
Assume light consists of photons with
Position x
Direction of motion w
Wavelength lambda
Each photo has an energy of hv
H Planck’s constant
V = 1 / lambda frequency
Measuring energy means “counting photons”
Spectral Radiance
Energy per time per wavelength per solid angle per area
means differetial area perpendicular to
Power carried along a ray is measured in radiance
Radiance along ray is constant (in vacuum)
In practice: assume steady state, measure at discrete wavelengths R, G, B
Radiance L: power per solid angle per area, vector of 3 values for R, G, B
Irradiance: Power per unit area
Integration of radiance over hemisphere
Irradiance E: power per area perpendicular to the surface normal
Radiance L: power per area perpendicular to the incident ray \omega
When we do ray tracing, we use radiance for rays
Integration in spherical coordinates
Radiant intensity I in all direction, has a total power
Photometry
Perceptual measurement of perceived brightness
Lumen
Deep Learning
Concepts
 Layers
 Activation function
 logistic sigmoid function: \frac{1}{1+e^{x}}
 unit step
 sign
 linear
 piecewise linear
 hyperbolic tangent
 Feature map – the output of one filter applied to the previous layer
 Fully connected layer
 Convolutional layer (convolutional neural networks, CNN)
 Loss functions
 The loss function computes the error for a single training example.
 The cost function is the average of the loss functions of the entire training set.
 Stochastic gradient descent
 use ONLY ONE or SUBSET of training sample from your training set to do the update for a parameter in a particular iteration
 Pros: converges much faster compared to GD
 Cons: the error function is not as well minimized as in the case of GD
 Batch size (64, 128, or 256)

Size of subset as an approximation of the whole databaseIt, which impacts the time efficiency of training and the noisiness of the gradient estimate. Updating the parameters using all training data is not efficient. Updating by one single sample (online updating) is noisy if the sample is not a good representation of the whole data.

 Batch normalization
 It is always great if the layer get data that is always normalized(mean 0 variance 1).
 We just normalize each batch with its running mean and variance.
 Batch normalization potentially helps in two ways: faster learning and higher overall accuracy. The improved method also allows you to use a higher learning rate, potentially providing another boost in speed.
 Automatic differentiation – numerically evaluate the derivative of a function
 Error backpropagation
 compute gradients of a loss function using the chain rule
 proceed ‘backwards’ with respect to the computations performed to compute the loss itself
 Error backpropagation
 Feedforward and recurrent neural network
 Feedforward
 The connections between the units do not form a cycle
 Recurrent Neural Network
 Sequence of words and images, unsegmented, connected handwriting recognition, or speech recognition
 Feedforward
 Autoencoder / autoassociator / Diabolo network
 Used for unsupervised learning of efficient codings, earn a representation (encoding) for a set of data, typically for the purpose of dimensionality reduction, widely used for learning generative models of data.
 Encoder / Decoder
 The encoder encodes the input sequence to an internal representation called ‘context vector’ which is used by the decoder to generate the output sequence. The lengths of input and output sequences can be different, as there is no explicit one on one relation between the input and output sequences.
 An Encoder/Decoder is not an autoencoder
 Reinforcement learning
 Reinforcement learning differs from standard supervised learning in that correct input/output pairs are never presented, nor suboptimal actions explicitly corrected. Instead the focus is on online performance, which involves finding a balance between exploration (of uncharted territory) and exploitation (of current knowledge).
 Skip connections
 Residual network
 unet
Denoising Monte Carlo Renderings
KernelPredicting Convolutional Networks for Denoising Monte Carlo Renderings
Specular has large dynamic range
 Take advantage of additional information (besides RGB) produced by renderer
 Network
 8 hidden layers, 100 kernels of 5×5 in each layer
 L1 loss
 Training data
 600 frames from movie “Finding Dori”
 Noise: 32 spp
 Reference: 1024 spp
 600 frames from movie “Finding Dori”
 Test data
 25 frames from other movies
In terms of timing, for an HD image of 1920Ă—1080, our network takes about 12 seconds to evaluate and output a full denoised image
Denoising animations
Framebyframe denoising leads to temporal flickering in animations
Need to filter in 3D spacetime
How to implement this with CNNs?
Chaitanya et al. Interactive Reconstruction of Monte Carlo Image Sequences using a Recurrent Denoising Autoencoder, SIGGRAPH 2017
Input to network
 RGB colros
 Gbuffer with viewspace shading normals, depth, material roughness
 Divided by diffuse albedo (remove texture complexity from input)
Loss
 Spatial: L1, L1 on spatial gradients
 Temporal: L1 on temporal gradients
 Back propagation through time
 Replicate feed forward parts to unroll recurrent connections
 “Adam” SGD solver
 Gradients are noisy in SGD
 Compute moving average over SGD steps
 feedforward network
Input with 1spp
 54.9ms for 720p on Titan X
 Path tracer from 70ms to 260ms for 1spp
What else could we learn to improve rendering efficiency?
 Superresolution
 Foveated rendering
 Antiliasing
 Multiple Importance Sampling
 Antialiasing for light fields
 Learning Light Transport the Reinforced Way, Dahm and Keller
 Use Q learning to learn importance sampling incrementally during path tracing
Deep Learning: The Future of RealTime Rendering
Leave a Reply