Internet Vision - Lecture 3

unknownlippsΤεχνίτη Νοημοσύνη και Ρομποτική

16 Οκτ 2013 (πριν από 3 χρόνια και 9 μήνες)

71 εμφανίσεις

Internet Vision
-

Lecture 3

Tamara Berg

Sept 10

New Lecture Time

Mondays 10:00am
-
12:30pm in 2311


Monday (9/15) we will have a general Computer
Vision & Machine Learning review


Please look at papers and decide which one you
want to present by Monday


read topic/titles/abstracts to get an idea of which
you are interested in

Thanks to Lalonde et al for providing slides!

Algorithm Outline

Inserting objects into images

Have an image and want to add realistic looking objects to that image

Inserting objects into images

User picks a location where they want to insert an object

Inserting objects into images

Based some properties calculated about the image, possible
objects are presented.

Inserting objects into images

User selects which object to insert and the object is placed in
the scene at the correct scale for the location

Inserting objects into images


Possible approaches

Insert a clip art object

Insert a clip art object
with some idea of the
environment

Insert a rendered object
with full model of the
environment

Some objects
will be easy to
insert because
they already “fit”
into the scene

Collect a large database of objects.

Let the computer decide which examples
are easy to insert.

Allow the user to select only among those.

When will an object “fit”?

1.) When the lighting conditions of the scene and object are similar

2.) When the camera pose of the scene & object match

2D vs 3D

Use 3d information for:


1.) Annotating objects in the
clip
-
art library with camera
pose

2.) Estimating the camera pose
in the query image

3.) Computing illumination
context in both library & query
images

Phase 1
-

Database Annotation

For each object we want:


Estimate of its true size and the camera pose it
was captured under


Estimate of the lighting conditions it was captured
under


Phase 1
-

Database Annotation

Estimate object size

Objects closer to the camera appear larger than objects further from the camera

Phase 1
-

Database Annotation

Estimate object size

*If*
you know the camera pose then you can estimate the real height of an object from:


location in the image,


pixel height

Phase 1
-

Database Annotation

Estimate object size

Annotate objects with their true heights and resize examples to a common
reference size

Phase 1
-

Database Annotation

Estimate object size & camera pose



Don’t know camera pose or object heights!





Trick
-

Infer camera pose & object heights
across all object classes in the database given
only the height distribution for one class



Phase 1
-

Database Annotation

Estimate object size & camera pose

Start with known heights for people

Phase 1
-

Database Annotation

Estimate object size & camera pose

Estimate camera pose for images with multiple
people


Phase 1
-

Database Annotation

Estimate object size & camera pose

Use these images to estimate a prior over the
distribution of poses


How do people usually take pictures? Standing

on the ground at eye level.

Phase 1
-

Database Annotation

Estimate object size & camera pose

Use the learned pose
distribution to estimate
heights of other object
categories that appear
with people.


Iteratively use these
categories to learn more
categories.


Annotate all objects in
the database with their
true size and originating
camera pose.



Phase 1
-

Database Annotation

Estimate object size & camera pose

Phase 1
-

Database Annotation

For each object we want:


Estimate of its true size and the camera pose it
was captured under


Estimate of the lighting conditions it was captured
under


Phase 1
-

Database Annotation

Estimate lighting conditions

Estimate which pixels are
ground, sky, vertical


Black box for now (we’ll
cover this paper later in
the course)

Ground



Vertical



Sky


Phase 1
-

Database Annotation

Estimate lighting conditions

Distribution of pixel colors

Phase 2


Object Insertion

Query Image

Phase 2


Object Insertion

User specifies horizon line


use to calculate camera pose with respect to ground
plane (lower
-
> tilted down, higher
-
> tilted up).


Illumination context is calculated in the same way as for the database images.

Phase 2


Object Insertion

Insert an object into the scene that has matching lighting,
and camera pose to the query image

Phase 2


Object Insertion

But wait it still looks funny!

Phase 2


Object Insertion

Shadows are important!

Phase 2


Object Insertion

Phase 2


Object Insertion

Phase 2


Object Insertion

Phase 2


Object Insertion

Shadow Transfer

Categorize images for easy selection in user interface

Big Picture


It’s all about the data!



?Use lots of data to turn a hard problem into
an easier one!


Place “my car” in a scene is much harder than
place “some car” in a scene. Allow the computer
to choose from among many examples of a class
to find the easy ones.