Micro Phase Shifting

minorbigarmΑσφάλεια

30 Νοε 2013 (πριν από 3 χρόνια και 4 μήνες)

67 εμφανίσεις

Micro Phase Shifting

Mohit Gupta and Shree K. Nayar

Computer Science

Columbia University


Supported by
:
NSF

and ONR


Structured Light 3D Scanning

Defect Inspection

Wafer
defect

Gaming

Archiving Heritage

Biometrics

Shape from Structured Light

camera

p
rojector

pattern

image

scene

correspondence

image plane

image plane

Structured Light Coding Schemes

time

radiance

Peak Location

time

radiance

Binary Code

time

radiance

Phase

Light Striping

Binary Codes

Phase Shifting

[
Shirai

and
Suwa
, 1971]

[
Agin

and
Binford
, 1976
]

[
Minou

et al.
, 1981
]

[
Posdamer

et al.
, 1982]

[
Sri
nivasan

et al.
, 1985]

[
Wust

and
Capson
, 1991]

Correspondence

Correspondence

Correspondence

Ambiguity

frequency

(
w
)

amplitude

Broad Frequency Band

w
max

w
mean

w
min

Phase Shifting

Unambiguous

but
Noisy

Accurate

but
Ambiguous

Phase Shifting: Issues

camera

p
rojector

interreflections

P

Q

scene

Interreflections

Defocus

scene

p
rojector

projected image

received image

Phase Shifting: Issues

camera

p
rojector

interreflections

P

Q

scene

Interreflections

Defocus

scene

p
rojector

projected image

received image

blurred

d
efocus blur

Phase Shifting and
Interreflections

camera

p
rojector

interreflections

P

Q

R

time

Interreflections

Direct Radiance

radiance

scene

camera

p
rojector

scene

P

Q

R

time

radiance

Total Radiance

Direct Radiance

Phase Error

Phase Shifting and
Interreflections

Concave Bowl

Phase Shifting and
Interreflections

point

p
rojector

interreflections

Concave Bowl

Reconstructed Shape

Errors due to
interreflections

Phase Shifting and
Interreflections

camera

p
rojector

scene

P

Q

R

𝑖
𝑝

=

interreflection

i
llumination
pattern

light transport
coefficients

×

p

Phase Shifting and
Interreflections

camera

p
rojector

scene

P

Q

R

𝑖
𝑝

=

interreflection

×

p

i
llumination
pattern

light transport
coefficients

Phase Shifting and
Interreflections

camera

p
rojector

scene

P

Q

R

𝑖
𝑝

=

interreflection

*

illumination
pattern

light transport
coefficients

pixels

pixels

p

Phase Shifting and
Interreflections

𝑖
𝑝

=

interreflection

*

illumination
pattern

light transport
coefficients

pixels

pixels

Phase Shifting and
Interreflections

frequency

frequency

×

bandlimit

Interreflections corrupt phase for low frequency sinusoids

projected patterns

Phase Shifting and
Interreflections

𝐼
𝑝

=

interreflection

illumination
pattern

light transport
coefficients

frequency

frequency

×

bandlimit

high

frequencies

projected patterns

Achieving Invariance to
Interreflections

𝐼
𝑝

=

High Frequency Illumination Invariance to Interreflections

interreflection

illumination
pattern

light transport
coefficients

Phase Shifting: Issues

camera

p
rojector

interreflections

P

Q

scene

Interreflections

Defocus

scene

p
rojector

projected image

received image

blurred

d
efocus blur

p
rojected patterns

*

=

i
deal

i
rradiance
p
rofile

p
rojector

d
efocus
k
ernel

a
ctual

i
rradiance
p
rofile

time

time

time

Phase Shifting and
Defocus

=

frequency

frequency

frequency

ideal

irradiance profile

projector

defocus kernel

actual

irradiance profile

p
rojected patterns

×

Phase Shifting and
Defocus

=

frequency

frequency

frequency

ideal

irradiance profile

projector

defocus kernel

actual

irradiance profile

projected patterns

=

frequency

frequency

frequency

ideal

irradiance profile

projector

defocus kernel

actual

irradiance profile

Large Number of Unknowns

projected patterns

×

×

Phase Shifting and
Defocus

=

frequency

frequency

frequency

ideal

irradiance profile

projector

defocus kernel

actual

irradiance profile

projected patterns

=

frequency

frequency

frequency

ideal

irradiance profile

projector

defocus kernel

actual

irradiance profile

projected patterns

Narrow Frequency Band Invariance to Defocus

×

×

Similar
amplitudes

Similar
amplitudes

Narrow

Band

Narrow

Band

Achieving Invariance to
Defocus

Micro Phase Shifting

frequency

(
w
)

amplitude

Narrow,

High
-
Frequency Band

w
max

w
mean

w
min

Invariance to
Interreflections

w
max

w
mean

w
min

High Mean Frequency

(
w
mean
)

frequency

(
w
)

amplitude

light
-
transport

bandlimit

Invariance to
Defocus

w
max

w
mean

w
min

Narrow

Bandwidth (
d
)

frequency

(
w
)

amplitude

Similar
amplitudes

How to Disambiguate Phase?

w
max

w
mean

w
min

How Can We Disambiguate Phase

Without

Low Frequency

Patterns?

How to Disambiguate Phase?

w
1

w
2
=

w
1

+

2d


+

Beat Frequency =
d

49Hz.

51Hz.

1Hz.

Phase Disambiguation:
Number Theory

period
:



pixels

correspondence
:

𝐶

pixels

𝐶

=
𝑛


+


phase
:



pixels

n
umber of periods
(unknown)

Phase Unwrapping:
Micro Phase Shifting


1

𝐶


1


𝑓

𝐶


𝑓


𝐹

𝐶


𝐹

Solve System of Simultaneous
Congruences

𝐶
=

𝑛
1

1
+

1

unknown

known

𝐶
=

𝑛
𝐹

𝐹
+

𝐹

unknown

known

𝐶
=

𝑛
𝑓

𝑓
+

𝑓

unknown

known

Chinese Remainder Theorem

There

exists

an

integer

C

solving

the

above

system

of

simultaneous

congruences
,

if

p
1

,

,

p
f

,

,

p
F

are

positive

integers

which

are

pairwise

coprime
.


[
The Mathematical
Classic by
Sun
Zi
, 3
rd

century AD]

Theorem
:

Efficient Algorithms Available for Solving

𝐶
=

𝑛
1

1
+

1

𝐶
=

𝑛
𝐹

𝐹
+

𝐹

𝐶
=

𝑛
𝑓

𝑓
+

𝑓

How Many Frequencies Are Required?



Number

of

columns
:

𝑁

Two Frequencies are Necessary


1

𝐶


𝐹

𝐶

Condition

for

unambiguous

phase

recovery:



𝑁


1
×

×

𝐹
=


p
eriods of projected
frequencies

period of emulated
low frequency

How Many Frequencies Are Required?



Number

of

columns
:

𝑁


1

𝐶


2

𝐶

Choose

two

frequencies

so

that


1
×


2

𝑁

Two Frequencies are
Sufficient

How Many Images Are Required?

𝐼
𝑘
𝑖

=



𝑔



+


𝑎
cos

𝑖
+

2
𝜋𝑘
𝐾

r
adiance for


k
th

shift of
w
i

offset

(interreflections)


amplitude

(defocus)

phase

number of
shifts

Number of Unknowns = F+2

F = number of
frequencies

How Many Images Are Required?

𝐼
𝑘
𝑖

=



𝑔



+


𝑎
cos

𝑖
+

2
𝜋𝑘
𝐾

r
adiance for


k
th

shift of
w
i

amplitude

(defocus)

phase

number of
shifts

Four Images are
Sufficient

offset

(interreflections)


Conventional vs. Micro Phase Shifting

Micro Phase Shifting:
Four Images

Conventional Phase Shifting:
Three Images

Current State
-
of
-
the
-
Art


Binary patterns

42 images

[Gupta
et al
., 2011]

[Couture
et al
., 2011]

200 images

[
Xu

and
Aliaga
, 2009]

400
-
1600 images


Modulated Phase Shifting

[
Gu

et al
.,
2011]

[Chen
et al
.,
2008]

Low SNR. 7+ images.

x

=

Ceramic Bowl: Interreflections

Projected and Input Images

Conventional Phase Shifting

[7 images, 2 Frequencies]

Micro Phase Shifting [Our]

[7 images, 5 Frequencies]

Projected

Input

Modulated Phase Shifting

[7 images, 1 Frequency]

Conventional Phase
Shifting

Micro Phase Shifting

[Our]

Modulated Phase Shifting

[
Gu

et al.
]

Shape Comparison (seven input images)

Lemon: Subsurface Scattering

point

p
rojector

subsurface
scacttering

Shape Comparison (seven input images)

Conventional Phase
Shifting

Micro Phase Shifting

[Our]

Modulated Phase Shifting

[
Gu

et al.
]

Russian Dolls: Defocus

Holes in low albedo regions

Conventional Phase Shifting

Micro Phase Shifting [Our]

Shape Comparison (seven input images)

Wax Bowl: Interreflections + Scattering

Conventional Phase
Shifting

Micro Phase Shifting

[Our]

Modulated Phase Shifting

[
Gu

et al.
]

Shape Comparison (seven input images)

Recovered Shape: Micro Phase Shifting

Failure Case: Shiny Metal Bowl

Specular
interreflections

Shape Comparison

Conventional Phase
Shifting

Micro Phase Shifting

[Our]

Modulated Phase Shifting

[
Gu

et al.
]

frequency

defocus

kernel

Discussion: Frequency Selection

frequency

light transport
bandwidth

frequency

defocus

kernel

Invariance to interreflections

Amplitude attenuation

p
rojected

frequency

Invariance to defocus

Not resolvable by projector

similar
amplitudes

frequency

projector

resolution

Shape Recovery with
I
nterreflections and Defocus


Patterns in Narrow High
-
Frequency Band

f
requency (
w
)

a
mplitude

Narrow,

High
-
Frequency Band

Summary: Micro Phase Shifting