Automated Image Registration Using Morphological Region of Interest Feature Extraction

molassesitalianAI and Robotics

Nov 6, 2013 (4 years and 2 days ago)

50 views

Automated Image Registration Using
Morphological Region of Interest
Feature Extraction

Antonio Plaza

University of Extremadura. Caceres, Spain

Jacqueline Le Moigne

NASA Goddard Space Flight Center, USA

Nathan Netanyahu

Bar
-
Ilan University, Israel & University of Maryland, USA


Automatic

Multiple Source

Integration

Prediction Models

Satellite, Aircraft

and Field Data

Improved Data Sets

Validation &

Verification

Feedback

Design of Future

Intelligent

Sensor Webs

Earth Science Data Integration

MultiTemp 2005

Jacqueline Le Moigne,
3

What is Image Registration ?


Navigation

or Model
-
Based Systematic Correction


Orbital, Attitude, Platform/Sensor Geometric Relationship, Sensor
Characteristics, Earth Model, ...



Image Registration


or Feature
-
Based Precision Correction


Navigation within a Few Pixels Accuracy


Image Registration Using Selected Features (or Control Points) to
Refine Geo
-
Location Accuracy



2 Approaches:

(1) Image Registration as a Post
-
Processing
(Taken here)

(2) Navigation and Image Registration in a Closed Loop

MultiTemp 2005

Jacqueline Le Moigne,
4

Image Registration Challenges


Multi
-
Resolution / Mono
-

or Multi
-
Instrument


Multi
-
temporal data


Various spatial resolutions


Various spectral resolutions



Sub
-
Pixel Accuracy


1 pixel misregistration=> 50% error in NDVI computation




Accuracy Assessment


Synthetic data


"Ground Truth" (manual registration?)


Use down
-
sampled high
-
resolution data


Consistency ("circular" registrations) studies


MultiTemp 2005

Jacqueline Le Moigne,
5

Image to Image Registration

Incoming Data

Image Characteristics

(Features) Extraction

• Multi
-
Temporal


Image Correlation



Landmarking



Coregistration

Feature

Matching

Compute

Transform

MultiTemp 2005

Jacqueline Le Moigne,
6

Image to Map Registration

Input Data

Map

Masking and

Feature Extraction

Feature


Matching

Compute

Transform

MultiTemp 2005

Jacqueline Le Moigne,
7

Multi
-
Sensor Image Registration

ETM/IKONOS Mosaic of Coastal VA Data

IKONOS

ETM+

MultiTemp 2005

Jacqueline Le Moigne,
8

Image Registration Components

0
Pre
-
Processing


Cloud Detection,
Region of Interest Masking, ...


1
Feature Extraction (“Control Points”)


Edges, Regions, Contours, Wavelet Coefficients, ...


2
Feature Matching


Spatial Transformation (a
-
priori knowledge)


Search Strategy (
Global vs Local
, Multi
-
Resolution, ...)


Choice of Similarity Metrics (Correlation, Optimization
Method,
Hausdorff Distance
, ...)


3
Resampling, Indexing or Fusion


MultiTemp 2005

Jacqueline Le Moigne,
9

Image Registration Subsystem

Based on a Chip Database

Landmark

Chip

Database

UTM of 4 Scene Corners Known

from Systematic Correction

Input Scene

(1)
Find Chips that


Correspond
to the


Incoming Scene

(2)
For Each Chip, Extract


Window from Scene,


Using UTM of:


-

4 Approx Scene Corners


-

4 Correct Chip Corners

(3)
Register
Each (Chip,Window)


Pair and Record Pairs of


Registered Chip Corners

(4)
Compute

Global Registration


from Multiple Local Ones

(5)
Compute Correct UTM


of 4 Scene Corners of


Input Scene

MultiTemp 2005

Jacqueline Le Moigne,
10

Image Registration Subsystem

Based on Automatic Chip Extraction

UTM of 4 Scene Corners Known from Systematic
Correction

Input Scene

(1)

Extract
Reference Chips


and Corresponding Input


Windows Using Mathematical


Morphology


(2)
Register
Each (Chip,Window)


Pair and Record Pairs of


Registered Chip Corners


(refinement step)


(3)
Compute

Global Registration


from Multiple Local Ones


(4)
Compute Correct UTM


of 4 Scene Corners of


Input Scene

Reference Scene

Advantages:



Eliminates Need for Chip Database



Cloud Detection Can Easily be Included in Process



Process Any Size Images



Initial Registration Closer to Final Registration =>


Reduces Computation Time and Increases Accuracy
.

MultiTemp 2005

Jacqueline Le Moigne,
11

Step 1: Chip
-
Window Extraction Using

Mathematical Morphology

Mathematical Morphology (MM) Concept:




Nonlinear
spatial
-
based
technique that provides a framework.



Relies on a
partial
ordering relation between image pixels.



In greyscale imagery, such relation is given by the digital value of
image pixels

Structuring
element

Dilation
3x3 structuring
element
defines
neighborhood around
pixel
P
Erosion
Max
Min
P
Original
image
Dilation
3x3 structuring
element
defines
neighborhood around
pixel
P
Erosion
Max
Min
P
Original
image
Original image

Erosion

K

K

Dilation

(4
-
pixel radius
Disk SE)

Greyscale MM Basic Operations:

MultiTemp 2005

Jacqueline Le Moigne,
12

Step 1 (Cont.)

Structuring
element

Binary Erosion

Structuring
element

Structuring
element

MultiTemp 2005

Jacqueline Le Moigne,
13

Step 1 (Cont.)

Structuring
element

Binary Dilation

Structuring
element

Structuring
element

MultiTemp 2005

Jacqueline Le Moigne,
14

K

Greyscale Morphology: Combined Operations

e.g., Erosion + Dilation = Opening

Step 1 (Cont.)

MultiTemp 2005

Jacqueline Le Moigne,
15

Step 1: Chip
-
Window Extraction Using

Mathematical Morphology


Scale
-
Orientation Morphological Profiles (SOMP)
: From Openings and
Closings with SEs=Line Segments of Different Orientations


SOMP = Feature Vector D(x,y) at each Pixel (various scales & orientations)


Entropy of D(x,y) = H(D(x,y))



Algorithm:

a.

Compute D(x,y) for each (x,y) in reference scene

b.

Extract reference chip centered around (x’,y’) with Max[H(D(x’,y’))], e.g. 256x256

c.

Compute D(X,Y) for each (X,Y) in search area input scene centered (e.g.,
1000x1000) around location (x’,y’)

d.
Compute RMSE(D(X,Y),D(x’,x’)) for all (X,Y) in search area

e.
Extract input window centered around (X’,Y’) with Min(RMSE)

f.
Return to step 2. until predefined number of chips is extracted

MultiTemp 2005

Jacqueline Le Moigne,
16

Step 1: Chip
-
Window Extraction Using

Mathematical Morphology

Results(Landsat
-
7/ETM+ Data
-

Central VA)

10 Chips Extracted from Reference Scene (Oct. 7, 1999)

10 Windows Extracted from Input Scene (Nov. 8, 1999)

MultiTemp 2005

Jacqueline Le Moigne,
17

Step 2: Chip
-
Window Refined

Registration Using Robust Feature Matching

Reference

Chip

Input

Window

Wavelet

Decomposition

Wavelet

Decomposition

Robust Feature

Matching (RFM)

Using

Hausdorff Distance


Maxima

Extraction

Maxima

Extraction

Choice of

Best

Transformation

At Each

Level of

Decomposition

{



Overcomplete Wavelet
-
type Decomposition: Simoncelli Steerable Pyramid



“Maxima” Extraction: Top 5% of Histogram

MultiTemp 2005

Jacqueline Le Moigne,
18

Step 2: Robust Feature Matching Using
Hausdorff Distance


Search Transformation Space through Hierarchical Spatial
Subdivisions



Perform Monte Carlo Sampling of Control Points



Compute Robust Similarity Measure

-
k
-
th smallest squared distance to nearest neighbors, i.e., partial Hausdorff
Distance
Partial Hausdorff Distance:




H
k
(A, B) = K
th
a
in A

min
b in B

dist (a,b)




(1≤
k

≤ |
A
|;

K
th

is the
k
th

smallest element of set; dist(a,b): Euclidean distance)



Prune Search Space by "Range" Similarity Estimates



Iterate and Refine on each Level of Wavelet Decomposition

MultiTemp 2005

Jacqueline Le Moigne,
19


From each Local Registration, Window
-
Chip:


Corrected Locations of Four corners of Each Window


i.e.: for each chip
-
window i, pair correspondences:


(UL_i_X1,UL_i_Y1) to (UL_i_X2,UL_i_Y2)


(UR_i_X1,UR_i_Y1) to (UR_i_X2,UR_i_Y2)


(LL_i_X1,LL_i_Y1) to (LL_i_X2,LL_i_Y2)


(LR_i_X1,LR_i_Y1) to (LR_i_X2,LR_i_Y2)



Use of a Least Mean Square (LMS) Procedure to Compute Global
Image Transformation (in pixels)


If n chips, 4n points used for the LMS


=> Step 4:

Use Global Transformation to Compute new UTM
Coordinates for each of the 4 Corners of the Incoming Scene

Step 3: Compute Global Registration

from All Local Registrations

MultiTemp 2005

Jacqueline Le Moigne,
20

Results of Global Registration

On Landsat
-
7 VA Test Data

MultiTemp 2005

Jacqueline Le Moigne,
21

Conclusions


Fully Automated System for Registration of Multi
-
Temporal
Landsat Scenes of Any Size, Using Mathematical Morphology and
Robust Feature Matching Techniques



MM Chip
-
Window Extractor Can be Used with Any Other
Registration Method



Eliminates Need of Database



Provides Close Initial Match => Follow
-
up Computations Faster and
More Accurate



Further Experimentation On
-
Going