Fourier Analysis of Stochastic Sampling
For Assessing Bias and Variance in Integration
Kartic
Subr
,
Jan
Kautz
University College London
g
reat sampling papers
Spectral analysis of
sampling must be
IMPORTANT!
BUT WHY?
numerical integration, you must try
assessing quality:
eg
. rendering
Shiny ball,
out of focus
Shiny ball in motion
…
pixel
multi

dim integral
v
ariance and bias
High variance
High bias
bias
and variance
High variance
High bias
predict as a function of
sampling strategy and
integrand
variance

bias trade

off
High variance
High bias
a
nalysis is non

trivial
Abstracting away the application…
0
numerical integration implies sampling
0
sampled integrand
(
N
samples)
numerical integration implies sampling
0
sampled integrand
the sampling function
integrand
sampling function
sampled integrand
multiply
sampling
func
. decides integration quality
integrand
sampled function
multiply
sampling function
strategies to improve estimators
1. modify weights
eg
.
quadrature
rules
strategies to improve estimators
1. modify weights
eg
. importance sampling
2. modify locations
eg
.
quadrature
rules
abstract away strategy: use Fourier domain
1. modify weights
2. modify locations
eg
.
quadrature
rules
analyse
sampling function in
Fourier domain
abstract away strategy: use Fourier domain
1. modify weights
a. Distribution
eg
. importance sampling)
2. modify locations
eg
.
quadrature
rules
sampling
function in the Fourier domain
frequency
amplitude (sampling spectrum)
phase (sampling spectrum)
stochastic sampling & instances of spectra
Sampler
(Strategy 1)
Fourier
transform
draw
Instances of sampling functions
Instances of sampling spectra
assessing estimators using sampling spectra
Sampler
(Strategy 1)
Sampler
(Strategy 2)
Instances of sampling functions
Instances of sampling spectra
Which strategy is better? Metric?
accuracy (bias) and precision (variance)
estimated value (bins)
frequency
reference
Estimator 2
Estimator 1
Estimator 2 has lower bias but higher variance
overview
related work
signal processing
assessing sampling patterns
spectral analysis
of integration
Monte Carlo
sampling
Monte Carlo
rendering
stochastic jitter: undesirable but unavoidable
signal processing
Jitter [Balakrishnan1962]
Point processes [Bartlett 1964]
Impulse processes [
Leneman
1966]
Shot noise [
Bremaud
et al. 2003]
we assess based on estimator bias and variance
assessing sampling patterns
Point statistics [Ripley 1977]
Frequency analysis [
Dippe&Wold
85, Cook 86, Mitchell 91]
Discrepancy [Shirley 91]
Statistical hypotheses [
Subr&Arvo
2007]
Others [
Wei&Wang
11,Oztireli&Gross 12]
recent and most relevant
spectral analysis
of integration
numerical integration schemes
[Luchini 1994; Durand 2011]
errors in visibility integration [
Ramamoorthi
et al. 12]
recent and most relevant
spectral analysis
of integration
numerical integration schemes
[Luchini 1994; Durand 2011]
errors in visibility integration [
Ramamoorthi
et al. 12]
1.
we derive estimator bias and variance in closed form
2.
we consider sampling spectrum’s phase
Intuition
(now)
Formalism
(paper)
sampling function = sum of Dirac deltas
+
+
+
Review: in the Fourier domain …
primal
Fourier
Dirac delta
Fourier transform
Frequency
Real
Imaginary
Complex plane
amplitude
phase
Review: in the Fourier domain …
primal
Fourier
Dirac delta
Fourier transform
Frequency
Real
Imaginary
Complex plane
Real
Imaginary
Complex plane
amplitude spectrum is not flat
=
+
+
+
primal
Fourier
=
+
+
+
Fourier transform
sample contributions at a given frequency
Real
Imaginary
Complex plane
5
1
2
3
4
5
At a given frequency
3
2
4
1
sampling function
the sampling spectrum at a given frequency
sampling spectrum
Complex plane
5
3
2
4
1
centroid
given frequency
the sampling spectrum at a given frequency
sampling spectrum instances
expected
centroid
centroid
variance
given frequency
expected sampling spectrum and variance
expected amplitude of sampling spectrum
variance of sampling spectrum
frequency
DC
intuition: sampling spectrum’s phase is key
•
without it, expected amplitude = 1!
–
for
unweighted
samples, regardless of distribution
•
cannot expect to know integrand’s phase
–
amplitude + phase implies we know integrand!
Theoretical results
Result 1: estimator bias
bias
reference
inner product
frequency variable
S
f
sampling spectrum
integrand’s spectrum
Implications
1.
S non zero only at 0 freq. (pure DC) =>
unbiased estimator
2.
<S> complementary to f keeps bias low
3.
What about phase?
expanded expression for bias
bias
expanded expression for bias
reference
bias
phase
amplitude
S
f
f
S
omitting phase for conservative bias prediction
reference
bias
phase
amplitude
S
f
f
S
new measure:
ampl
of expected sampling spectrum
ours
periodogram
Result 2: estimator variance
variance
frequency variable
inner product
S
 f 
2
sampling spectrum
integrand’s power spectrum
the equations say …
•
Keep energy low at frequencies in sampling spectrum
–
Where integrand has high energy
case study: Gaussian jittered sampling
1D Gaussian jitter
samples
jitter using
iid
Gaussian distributed
1D random variables
1D Gaussian jitter in the Fourier domain
real
Imaginary
Complex plane
Fourier transformed
samples at an arbitrary
frequency
Jitter in position
manifests as
phase jitter
centroid
derived Gaussian jitter properties
•
any starting configuration
•
does not introduce bias
•
variance

bias tradeoff
Testing integration using Gaussian jitter
random points
binary function
p/w constant function
p/w linear function
bias

variance trade

off using Gaussian jitter
bias
variance
Gaussian jitter
random
grid
Poisson disk
low

discrepancy
Box jitter
Gaussian jitter converges rapidly
Log

number of primary estimates
log

variance
Gaussian jitter
Random:
Slope =

1
O(1/N)
Poisson disk
low

discrepancy
Box jitter
Conclusion: Studied sampling spectrum
sampling
spectrum
integrand
spectrum
integrand
sampling function
Conclusion: bias
sampling
spectrum
integrand
spectrum
integrand
sampling function
bias depends on E( ) .
Conclusion
: variance
sampling
spectrum
integrand
spectrum
integrand
sampling function
bias depends on E( ) .
variance is V( ) .
2
Acknowledgements
Take

home messages
5
3
2
4
1
relative
phase is key
Ideal sampling spectrum
No energy in sampling spectrum
at frequencies where
integrand has high
energy
Questions?
http://www.wordle.net/show/wrdl/6890169/FMCSIG13
Sorry, what? Handling finite domain?
•
Integrand =
integrand
*
box
conclusion
Comments 0
Log in to post a comment