Multimedia Retrieval Multimedia Retrieval Ch 5 Image Processing Ch 5 Image Processing

paradepetAI and Robotics

Nov 5, 2013 (3 years and 10 months ago)

80 views

Multimedia Retrieval
Multimedia Retrieval
Ch 5 Image Processing
Ch 5 Image Processing
AnneYlinen
AnneYlinen
Agenda
Agenda
￿
￿
Typesofimageprocessing
Typesofimageprocessing
￿
￿
Applicationareas
Applicationareas
￿
￿
Imageanalysis
Imageanalysis
￿
￿
Imagefeatures
Imagefeatures
Types of Image Processing
Types of Image Processing
￿
￿
ImageAcquisition
ImageAcquisition
￿
￿
Camera
Camera
￿
￿
Scanners
Scanners
￿
￿
X
X


rayimagers
rayimagers
￿
￿
Computertomography(CT)
Computertomography(CT)
￿
￿
Magneticresonancescanners(MR)
Magneticresonancescanners(MR)
￿
￿
Ultrasounddevices(US)
Ultrasounddevices(US)
Types of Image Processing
Types of Image Processing
￿
￿
ImageRestoration
ImageRestoration
￿
￿
Geometricdistortions
Geometricdistortions
￿
￿
Noise
Noise
￿
￿
Unsharpness
Unsharpness
Types of Image Processing
Types of Image Processing
￿
￿
ImageReconstruction
ImageReconstruction
￿
￿
usingmodels
usingmodels
￿
￿
differentviewpoint
differentviewpoint
￿
￿
anotherimagingdevice
anotherimagingdevice
Types of Image Processing
Types of Image Processing
￿
￿
ImageEnhancement
ImageEnhancement
￿
￿
Contrastenhancement
Contrastenhancement
￿
￿
amplitudescaling
amplitudescaling
￿
￿
contrastmodification
contrastmodification
￿
￿
Histogramnormalization
Histogramnormalization
￿
￿
nonadaptive
nonadaptive
histogrammodification
histogrammodification
￿
￿
adaptivehistogrammodification
adaptivehistogrammodification
￿
￿
Edgeenhancement
Edgeenhancement
￿
￿
linearedge
linearedge
crispening
crispening
￿
￿
statisticaldifferencing
statisticaldifferencing
Types of Image Processing
Types of Image Processing
￿
￿
ImageRegistration
ImageRegistration
￿
￿
Rigidregistration
Rigidregistration
￿
￿
Non
Non


rigidregistration
rigidregistration
￿
￿
Usedinmedicalapplications,cartography,face
Usedinmedicalapplications,cartography,face
recognition,etc.
recognition,etc.
Types of Image Processing
Types of Image Processing
￿
￿
ImageCompression,StorageandTransmission
ImageCompression,StorageandTransmission
￿
￿
Lossless
Lossless
￿
￿
imagecanbeexactlyreconstructed
imagecanbeexactlyreconstructed
￿
￿
Lossy
Lossy
￿
￿
approximatereconstruction
approximatereconstruction
Types
Types
of
of
Image Processing
Image Processing
￿
￿
ImageAnalysis
ImageAnalysis
￿
￿
Imageanalysisaimstogenerateadescriptionofthe
Imageanalysisaimstogenerateadescriptionofthe
imageorofobjectspresentintheimage.
imageorofobjectspresentintheimage.
Application Areas
Application Areas
￿
￿
MedicalImaging
MedicalImaging
￿
￿
MR,CT,US
MR,CT,US
￿
￿
GeoInformationSystems,Satellite,Aerialphotography
GeoInformationSystems,Satellite,Aerialphotography
andCartography
andCartography
￿
￿
Biometry
Biometry
￿
￿
Faceandfingerprintrecognition,
Faceandfingerprintrecognition,
handpalm
handpalm
recognition,
recognition,
trackingpeople
trackingpeople
￿
￿
feature
feature


basedandholisticapproaches
basedandholisticapproaches
￿
￿
OpticalCharacterRecognition
OpticalCharacterRecognition
￿
￿
IndustrialVision
IndustrialVision
￿
￿
MultimediaandImageDatabases
MultimediaandImageDatabases
Image Analysis
Image Analysis
￿
￿
extractinformationfromanimage
extractinformationfromanimage
￿
￿
detection
detection
￿
￿
classification
classification
￿
￿
parameterestimation
parameterestimation
￿
￿
structuralanalysis
structuralanalysis
Image Analysis
Image Analysis
comparison
feature
extraction
model
selection
observed
data
databaseof
models
modelfeatures
observed
features
matchcriterion
selected
model
Image Analysis
Image Analysis
￿
￿
Imageanalysistask
Imageanalysistask
￿
￿
theselectionofthefeatures
theselectionofthefeatures
￿
￿
therepresentationofthemodels
therepresentationofthemodels
￿
￿
thematchingcriterion
thematchingcriterion
￿
￿
theselectionstrategy
theselectionstrategy
Image Features
Image Features
￿
￿
Image
Image
￿
￿
2
2


dimensionalsignal
dimensionalsignal
￿
￿
representedbyamatrixFofpixelsofNrowsandM
representedbyamatrixFofpixelsofNrowsandM
columns
columns
￿
￿
Apixelvalue
Apixelvalue
f(n,m
f(n,m
)isanintensityoravectorof3
)isanintensityoravectorof3
RGBcomponents
RGBcomponents
￿
￿
mathematicaloperationsarepossiblee.g.derivative
mathematicaloperationsarepossiblee.g.derivative
andFouriertransformation
andFouriertransformation
Image Features
Image Features
￿
￿
PixelFeatures
PixelFeatures
￿
￿
NeighborhoodandImagefiltering
NeighborhoodandImagefiltering
￿
￿
eachpixelanindividualfeature
eachpixelanindividualfeature
￿
￿
neighboringpixelsgroupedtogether
neighboringpixelsgroupedtogether
￿
￿
usedtoobtainhigherlevelfeatures
usedtoobtainhigherlevelfeatures
Image Features
Image Features
￿
￿
Scalespaceandderivatives
Scalespaceandderivatives
￿
￿
scaleatwhichobjectsareseeninanimagedependsonthe
scaleatwhichobjectsareseeninanimagedependsonthe
distancebetweenobjectandcamera
distancebetweenobjectandcamera
￿
￿
scalespacetheoryforhandlingimagestructuresat
scalespacetheoryforhandlingimagestructuresat
differentscale
differentscale
￿
￿
derivativesimportantforedgedetection,pointfeature
derivativesimportantforedgedetection,pointfeature
detection,andsoon
detection,andsoon
Image Features
Image Features
￿
￿
Texture
Texture
￿
￿
smallelementarypatternrepeatedperiodicallyorquasi
smallelementarypatternrepeatedperiodicallyorquasi


periodically
periodically
￿
￿
geometricorradiometricpattern
geometricorradiometricpattern
￿
￿
importantcluesforsegmentingtheimage
importantcluesforsegmentingtheimage
￿
￿
typifiedby
typifiedby
￿
￿
thedistanceoverwhichthepatterisrepeated
thedistanceoverwhichthepatterisrepeated
￿
￿
thedirectioninwhichthepatternisrepeated
thedirectioninwhichthepatternisrepeated
￿
￿
thepropertiesoftheelementarypattern
thepropertiesoftheelementarypattern
￿
￿
co
co


occurrencematrices
occurrencematrices
Image Features
Image Features
￿
￿
PointFeatures
PointFeatures
￿
￿
Interestpoints
Interestpoints
￿
￿
cornerpointsandspots
cornerpointsandspots
￿
￿
videotracking,stereomatching,objectrecognition
videotracking,stereomatching,objectrecognition
￿
￿
Harriscornerdetector
Harriscornerdetector
Image Features
Image Features
￿
￿
Harriscornerdetector
Harriscornerdetector
￿
￿
image
image
I(x,y
I(x,y
)andsiftedimage
)andsiftedimage
I(x+u
I(x+u
,
,
y+v
y+v
)
)
￿
￿
Gaussianwindowfunction
Gaussianwindowfunction
w(x,y
w(x,y
)
)
￿
￿
E(u,v
E(u,v
)shouldchangefastforsmallsiftsof(
)shouldchangefastforsmallsiftsof(
u,v
u,v
)
)
[
]

++=
yx
yxIvyuxIyxwvuE
,
2
),(),(),(),(
cuvbvauvuE2),(
22
++
Image Features
Image Features
21
21
2
21
,
2
2
det
)(det
Mof seigenvalue λ,λ
),(
],[),(


+=
=
=







=







traceM
M
traceMkMR
III
III
yxwM
where
v
u
MvuvuE
yx
yyx
yxx
Image Features
Image Features

1

2
Corner
Edge
Edge
Flat
Rdepends only on
eigenvalues of M
Ris large for a corner
Ris negative with large
magnitude for an edge
|R| is small for a flat
region
R> 0
R< 0
R< 0|R|small
sourse(www.wisdo
sourse(www.wisdo
m
m
.weiz
.weiz
m
m
ann.ac.il/~deniss/vision_sp
ann.ac.il/~deniss/vision_sp
r
r
ing04/files/Inva
ing04/files/Inva
r
r
iantFeatu
iantFeatu
r
r
es.ppt
es.ppt)
Image Features
Image Features
￿
￿
Lineelements
Lineelements
￿
￿
linesegmentshaveawidthintheimageequaltothescaleof
linesegmentshaveawidthintheimageequaltothescaleof
theimage,Gaussianlikeprofileacrosstheline
theimage,Gaussianlikeprofileacrosstheline
￿
￿
calculatethesecondderivativeinthedirectionorthogonalto
calculatethesecondderivativeinthedirectionorthogonalto
thegradientvector
thegradientvector
￿
￿
morestableresultisobtainedbyapproximatingthe
morestableresultisobtainedbyapproximatingthe
neighborhoodofeachcandidatelineelementbyquadratic
neighborhoodofeachcandidatelineelementbyquadratic
surface:
surface:
￿
￿
(
(
n,m
n,m
)isthepositionofthecandidatelineelement
)isthepositionofthecandidatelineelement
cklblakmnflmknf2),(),(
22
+++
Image Features
Image Features
￿
￿
usingTaylorexpansion
usingTaylorexpansion
￿
￿
λ
λ
1
1
,
,
λ
λ
2
2
are
are
eigenvalues
eigenvalues
ofH
ofH
￿
￿
fortruelineelement,one
fortruelineelement,one
eigenvalue
eigenvalue
shouldbelargeand
shouldbelargeand
theothersmall
theothersmall
[]






=






+
yyxy
xyxx
ff
ff
H
where
l
k
Hlkmnflmknf),(),(
Image Features
Image Features
￿
￿
Edgeelements
Edgeelements
￿
￿
stepwisetransitioninintensities
stepwisetransitioninintensities
￿
￿
neighboringedgeelementslinkedto
neighboringedgeelementslinkedto
gether
gether
formanedge
formanedge
segment
segment
￿
￿
gradientislargeatthepositionofanedge
gradientislargeatthepositionofanedge
￿
￿
Gradient
Gradient


basedmethods
basedmethods
￿
￿
Laplacian
Laplacian


basedmethods
basedmethods
￿
￿
Canny
Canny


s
s
method
method
Image Features
Image Features
￿
￿
Canny
Canny


s
s
method
method
1.
1.
SmooththeimagewithGaussianfilter
SmooththeimagewithGaussianfilter
g(x,y
g(x,y
)=
)=
g
g
c
c
(x,y
(x,y
)
)
*
*
f(x,y
f(x,y
)
)
where
where
where
where
σ
σ
representsthewidthoftheGaussiandistribution
representsthewidthoftheGaussiandistribution
2.
2.
Computethesecondderivativeinthegradientdirection
Computethesecondderivativeinthegradientdirection
3.
3.
Findzerocrossingsofthesecondderivative
Findzerocrossingsofthesecondderivative








+
=
2
22
2
exp
2
1
),(


yx
yxgc
22
22
2
2
2
yx
yyyxyyxxxx
gg
ggggggg
n
g
+
++
=


Image Features
Image Features
￿
￿
Pros:
Pros:
￿
￿
Onepixelwideedges
Onepixelwideedges
￿
￿
Edgesaregroupedtogether
Edgesaregroupedtogether
(oftengoodfor
(oftengoodfor
segmentation)
segmentation)
￿
￿
Robustagainstnoise!
Robustagainstnoise!
￿
￿
Cons:
Cons:
￿
￿
Complicatedtounderstand
Complicatedtounderstand
andimplement
andimplement
￿
￿
Slow
Slow
References
References
￿
￿
Blanken
Blanken
etal,MultimediaRetrieval,2007,Springer
etal,MultimediaRetrieval,2007,Springer
￿
￿
Pratt,W:DigitalImageProcessing,2001,JohnWiley
Pratt,W:DigitalImageProcessing,2001,JohnWiley
&SonsINC
&SonsINC
￿
￿
Bovik
Bovik
,A:HandbookofImage&VideoProcessing,
,A:HandbookofImage&VideoProcessing,
2000,AcademicPress
2000,AcademicPress
￿
￿
Castelman
Castelman
,K:DigitalImageProcessing,1996,Prentice
,K:DigitalImageProcessing,1996,Prentice
Hall
Hall
￿
￿
Harris,C:ACombinedCornerandEdgeDetector,
Harris,C:ACombinedCornerandEdgeDetector,
1988,
1988,