Fourier Analysis of Stochastic Sampling For Assessing Bias and Variance in Integration

yakzephyrAI and Robotics

Nov 24, 2013 (3 years and 6 months ago)

57 views

Fourier Analysis of Stochastic Sampling

For Assessing Bias and Variance in Integration

Kartic

Subr
,
Jan
Kautz

University College London

g
reat sampling papers

Spectral analysis of
sampling must be
IMPORTANT!

BUT WHY?

numerical integration, you must try

assessing quality:
eg
. rendering

Shiny ball,
out of focus

Shiny ball in motion



pixel

multi
-
dim integral

v
ariance and bias

High variance

High bias

bias
and variance

High variance

High bias

predict as a function of
sampling strategy and
integrand

variance
-
bias trade
-
off

High variance

High bias

a
nalysis is non
-
trivial

Abstracting away the application…

0

numerical integration implies sampling

0

sampled integrand

(
N

samples)

numerical integration implies sampling

0

sampled integrand

the sampling function

integrand

sampling function

sampled integrand

multiply

sampling
func
. decides integration quality

integrand

sampled function

multiply

sampling function

strategies to improve estimators

1. modify weights

eg
.
quadrature

rules

strategies to improve estimators

1. modify weights

eg
. importance sampling

2. modify locations

eg
.
quadrature

rules

abstract away strategy: use Fourier domain

1. modify weights

2. modify locations

eg
.
quadrature

rules

analyse

sampling function in
Fourier domain

abstract away strategy: use Fourier domain

1. modify weights

a. Distribution
eg
. importance sampling)

2. modify locations

eg
.
quadrature

rules

sampling
function in the Fourier domain

frequency

amplitude (sampling spectrum)

phase (sampling spectrum)

stochastic sampling & instances of spectra

Sampler

(Strategy 1)

Fourier

transform

draw

Instances of sampling functions

Instances of sampling spectra

assessing estimators using sampling spectra

Sampler

(Strategy 1)

Sampler

(Strategy 2)

Instances of sampling functions

Instances of sampling spectra

Which strategy is better? Metric?

accuracy (bias) and precision (variance)

estimated value (bins)

frequency

reference

Estimator 2

Estimator 1

Estimator 2 has lower bias but higher variance

overview

related work


signal processing


assessing sampling patterns


spectral analysis

of integration

Monte Carlo

sampling

Monte Carlo
rendering

stochastic jitter: undesirable but unavoidable


signal processing


Jitter [Balakrishnan1962]




Point processes [Bartlett 1964]




Impulse processes [
Leneman

1966]



Shot noise [
Bremaud

et al. 2003]

we assess based on estimator bias and variance


assessing sampling patterns

Point statistics [Ripley 1977]



Frequency analysis [
Dippe&Wold

85, Cook 86, Mitchell 91]



Discrepancy [Shirley 91]


Statistical hypotheses [
Subr&Arvo

2007]


Others [
Wei&Wang

11,Oztireli&Gross 12]



recent and most relevant


spectral analysis

of integration


numerical integration schemes
[Luchini 1994; Durand 2011]




errors in visibility integration [
Ramamoorthi

et al. 12]





recent and most relevant


spectral analysis

of integration


numerical integration schemes
[Luchini 1994; Durand 2011]




errors in visibility integration [
Ramamoorthi

et al. 12]





1.

we derive estimator bias and variance in closed form

2.

we consider sampling spectrum’s phase

Intuition

(now)

Formalism

(paper)

sampling function = sum of Dirac deltas

+

+

+

Review: in the Fourier domain …

primal

Fourier

Dirac delta

Fourier transform

Frequency

Real

Imaginary

Complex plane

amplitude

phase

Review: in the Fourier domain …

primal

Fourier

Dirac delta

Fourier transform

Frequency

Real

Imaginary

Complex plane

Real

Imaginary

Complex plane

amplitude spectrum is not flat

=

+

+

+

primal

Fourier

=

+

+

+

Fourier transform

sample contributions at a given frequency

Real

Imaginary

Complex plane

5

1

2

3

4

5

At a given frequency

3

2

4

1

sampling function

the sampling spectrum at a given frequency

sampling spectrum

Complex plane

5

3

2

4

1

centroid

given frequency

the sampling spectrum at a given frequency

sampling spectrum instances

expected
centroid

centroid

variance

given frequency

expected sampling spectrum and variance

expected amplitude of sampling spectrum

variance of sampling spectrum

frequency

DC

intuition: sampling spectrum’s phase is key


without it, expected amplitude = 1!


for
unweighted

samples, regardless of distribution



cannot expect to know integrand’s phase


amplitude + phase implies we know integrand!

Theoretical results


Result 1: estimator bias

bias

reference

inner product

frequency variable

S

f

sampling spectrum

integrand’s spectrum

Implications


1.
S non zero only at 0 freq. (pure DC) =>
unbiased estimator


2.
<S> complementary to f keeps bias low


3.
What about phase?



expanded expression for bias

bias


expanded expression for bias

reference

bias

phase

amplitude

S

f

f

S

omitting phase for conservative bias prediction

reference

bias

phase

amplitude

S

f

f

S

new measure:
ampl

of expected sampling spectrum

ours

periodogram

Result 2: estimator variance

variance

frequency variable

inner product

S

|| f ||
2

sampling spectrum

integrand’s power spectrum

the equations say …


Keep energy low at frequencies in sampling spectrum


Where integrand has high energy




case study: Gaussian jittered sampling

1D Gaussian jitter

samples

jitter using
iid

Gaussian distributed

1D random variables

1D Gaussian jitter in the Fourier domain

real

Imaginary

Complex plane

Fourier transformed

samples at an arbitrary

frequency

Jitter in position

manifests as

phase jitter

centroid

derived Gaussian jitter properties


any starting configuration



does not introduce bias



variance
-
bias tradeoff

Testing integration using Gaussian jitter

random points

binary function

p/w constant function

p/w linear function

bias
-
variance trade
-
off using Gaussian jitter

bias

variance

Gaussian jitter

random

grid

Poisson disk

low
-
discrepancy

Box jitter

Gaussian jitter converges rapidly

Log
-
number of primary estimates

log
-
variance

Gaussian jitter

Random:

Slope =
-
1

O(1/N)

Poisson disk

low
-
discrepancy

Box jitter

Conclusion: Studied sampling spectrum

sampling

spectrum

integrand

spectrum

integrand

sampling function

Conclusion: bias

sampling

spectrum

integrand

spectrum

integrand

sampling function

bias depends on E( ) .

Conclusion
: variance

sampling

spectrum

integrand

spectrum

integrand

sampling function

bias depends on E( ) .

variance is V( ) .

2

Acknowledgements

Take
-
home messages

5

3

2

4

1

relative
phase is key

Ideal sampling spectrum

No energy in sampling spectrum

at frequencies where

integrand has high
energy

Questions?

http://www.wordle.net/show/wrdl/6890169/FMCSIG13

Sorry, what? Handling finite domain?


Integrand =
integrand
*

box

conclusion