Advisor: Dr. Sreela Sasi

plantationscarfΤεχνίτη Νοημοσύνη και Ρομποτική

25 Νοε 2013 (πριν από 3 χρόνια και 11 μήνες)

98 εμφανίσεις





Advisor: Dr. Sreela Sasi

Introduction

Image Colorization

WHAT:

Adding color
to
monochrome
images

WHEN:

Performed
since the early
20
th

century

WHY:

Improve visual
appeal of
illustrations

HOW:

A painstaking
and subjective
manual task

2

Introduction
(contd.)

Digital Image Colorization

Automation of colorization

Improve visual appeal of images

Color accuracy, finer details

Add relevant information to
images

Make images more
understandable

3

Introduction
(contd.)

Applications of Image Colorization

Applications

Homeland
Security

Satellite
Imaging

Old photos
and films

Medical
Imaging

Video
compression

4

Colorization Techniques

Scribble
-
based colorization


User add color scribbles
to image to be colorized

laborious, time
-
consuming, subjective,
and painstaking manual
task.


Example
-
based colorization


automation by extracting
colors from sample
image

results can vary
depending on example
image chosen

+

=

+

=

Previous Research

Image

Colorization

5

Current Research

Process Workflow

Texture
-
based
Segmentation

Image

Image

Sample
Image

Feature
Extraction

Color
Descriptors

Texture
Descriptors

New
Grayscale

Image

Texture
-
based
Segmentation

Feature
Extraction

Texture
Descriptors

Texture
Matching

Colorization
Process

Database

6

Image Segmentation

Image segmentation:



Is the partitioning of an image into
homogeneous regions based on a set of
characteristics.



Is a key element in image analysis and
computer vision.


7

Image Segmentation
(
contd.)

Clustering:



Is one of the methods available for image
segmentation.



Is a process which can be used for
classifying pixels based on similarity
according to the pixel’s color or gray
-
level
intensity.

8

Image Segmentation
(contd.)

Despite the substantial amount of research
performed to date, the design of a robust
and efficient clustering algorithm remains a
very challenging problem

9

Color
-
based Image Segmentation

Composite Image

10

Color
-
based Image Segmentation

Composite
Image

with salt & pepper noise added

11

Texture
-
based
Image
Segmentation


12

Workflow Process

Texture
-
Based Image Segmentation

Original
Image

Filtered
Image

Filtered
Image

Filtered
Image

Filtered
Image



Feature
Image

Feature
Image

Feature
Image

Feature
Image



Feature
Image

Blobs

Gabor Filters

Energy Computation

Segmentation

Add, mean smoothing, normalization

13

14

Image Segmentation

Multi
-
Channel Filtering
-

Gabor Transform

Previous Research
(contd.)

Texture
-
Based Segmentation

15

16


Image Segmentation

Normalized Sum of Gabor Responses


Current Research

Process Workflow

Texture
-
based
Segmentation

Image

Image

Sample
Image

Feature
Extraction

Color
Descriptors

Texture
Descriptors

New
Grayscale

Image

Texture
-
based
Segmentation

Feature
Extraction

Texture
Descriptors

Texture
Matching

Colorization
Process

Database

17

Previous Research
(contd.)

Clustering and Feature Extraction

18

Previous Research


The K
-
means algorithm has been used for
a fast and crisp “hard” segmentation.


The Fuzzy set theory has improved this
process by allowing the concept of partial
membership, in which an image pixel can
belong to multiple clusters.


This “soft” clustering allows for a more
precise computation of the cluster
membership, and has been used
successfully for image clustering and
segmentation.

19


The Fuzzy C
-
means clustering (FCM)
algorithm [1] is a widely used method for
“soft” image clustering.


However, the FCM algorithm is
computationally intensive.


It is also very sensitive to noise because it
only iteratively compares the properties of
each individual pixel to each cluster in the
feature domain.

Previous Research
(contd.)

[1]

James C.
Bezdek
,
Pattern Recognition with Fuzzy Objective Function Algorithms
. New York: Plenum, 1981.

20


Image Segmentation

Modified Fuzzy C
-
means Clustering


21

Previous Research

(contd.)

Fuzzy C
-
means clustering (FCM) Algorithm

22

Previous Research
(contd.)

FCM Pseudo
-
code

Step 1

Set the number c of clusters, the fuzzy parameter m, and the


stopping condition ε


Step 2

Initialize the fuzzy membership values µ


Step 3

Set the loop counter
b

=
0


Step 4

Calculate the cluster
centroid

values using (3)


Step 5

For each pixel, compute the membership values using (4) for each


cluster


Step 6

Compute the objective function
A
. If the value of
A
between


consecutive iterations < ε then stop, otherwise set
b
=
b
+1


and go to step 4

23

[2]

Stelios

Krinidis

and
Vassilios

Chatzis
, "A Robust Fuzzy Local Information C
-
means Clustering Algorithm,"
Image
Processing, IEEE Transactions on
, pp. 1
-
1, 2010.

Previous Research
(contd.)

Modified Fuzzy C
-
means clustering with
G
ki

factor

In order to improve the tolerance to noise of the
Fuzzy C
-
means clustering algorithm,
Krinidis

and
Chatzis

[2] have proposed a new Robust Fuzzy
Local Information C
-
means Clustering (FLICM)
algorithm by introducing the novel
G
ki

factor.


The purpose of this algorithm is to adjust the
fuzzy membership of each pixel by adding local
information from the membership of neighboring
pixels.


24

Previous Research
(contd.)

Modified Fuzzy C
-
means clustering with
G
ki

factor

Sliding window of size 1 around the
i
th

pixel

The
G
ki

factor is obtained by using a sliding
window of predefined dimensions:

25

Previous Research
(contd.)

Modified Fuzzy C
-
means clustering with
G
ki

factor



The
G
ki

factor is calculated by using the
following equation:


26

Current Algorithm

Modified Fuzzy C
-
means clustering with novel
H
ik

factor



This algorithm is further improved by
including both the local spatial information
from neighboring pixels and the spatial
Euclidian distance of each pixel to the
cluster’s center of gravity.


In this research, the algorithm is also
extended for clustering of color images in
the Red
-
Green
-
Blue (RGB) color space.

27

Current Algorithm
(contd.)

X
X
X
p
i
d
i1
d
i2
d
i3
c
1
c
2
c
3
Illustration of the new
H
ik

factor displaying the
spatial Euclidian distance
to the center of gravity
of each cluster

28

Current Algorithm

(contd.)

Process Workflow

Customize
Parameters

Calculate cluster
membership values

Compute
G
ki

Readjust membership
values

Compute
H
ki

Compute objective
function

Defuzzification

and clustering

-

Image

Calculate cluster
centroid

29

Current Algorithm
(contd.)

Modified Fuzzy C
-
means Clustering


30

Simulation and Results

Synthetic Grayscale Test Image

31

Natural test image


FCM segmentation

with 5 clusters


FCM segmentation

using the modified FCM algorithm

with 5 clusters,
G
ki

window=1 and
H
ik

Simulation and Results

Natural Test Image

32

Simulation and Results

Synthetic Grayscale Test Image

Synthetic 4
-
color test image

with added salt and pepper noise


FCM clustering


FCM clustering

with
G
ki

window=1 and with
H
ik



FCM clustering

with
G
ki

window=5 and with
H
ik

33

Synthetic 4
-
color test image

with added salt and pepper noise


FCM clustering


FCM clustering

with
G
ki

window=1 and with
H
ik



FCM clustering

with
G
ki

window=5 and with
H
ik

Simulation and Results

Synthetic Color Test Image

34


Image Segmentation

Clustering Demo


35

Modified Fuzzy C
-
means Clustering

Summary




In this research, the FCM with the
G
ki

factor is
modified using the
H
ik

factor, and the algorithm is
extended for the clustering of color images.



The use of the sliding window in the
G
ki

factor
improves the segmentation results by incorporating
local information about neighboring pixels in the
membership function of the clusters. However, this
resulted in a significant increase in the number of
calculations required for each iteration for each
pixel, and can be given by

36

Modified Fuzzy C
-
means Clustering

Summary
(contd.)





By combining the
G
ki

and the
H
ik

factors, this
modified FCM algorithm considerably reduced
the number of iterations needed to achieve
convergence. The tolerance to noise of the
Fuzzy C
-
means algorithm is also greatly
increased, allowing for an improved capability
to obtain coherent and contiguous segments
from the original image.


37

Modified Fuzzy C
-
means Clustering

Summary
(contd.)





However, because of the radial nature of the
spatial Euclidean distance to the cluster’s
center of gravity, this new method may not be
as effective for images containing circular
shapes, or for images where the cluster’s
center of gravity are close to each
-
other.



In this research, the FCM is extended for the
clustering of color images in the RGB color
space. The effectiveness of this algorithm may
be tested for images in other color spaces
also.

38

Current Research

Process Workflow

Texture
-
based
Segmentation

Image

Image

Sample
Image

Feature
Extraction

Color
Descriptors

Texture
Descriptors

New
Grayscale

Image

Texture
-
based
Segmentation

Feature
Extraction

Texture
Descriptors

Texture
Matching

Colorization
Process

Database

39

40


Sample Color Images

41


Image Segmentation

Normalized Sum of Gabor Responses



Image Segmentation

Feature Extraction


42


Image Segmentation

Feature Extraction

(contd.)


43

Blob Filtering for color and
texture extraction.

44

Texture and Color database

Image Segmentation

Feature
Extraction

(contd.)




45

Current Research

Process Workflow

Texture
-
based
Segmentation

Image

Image

Sample
Image

Feature
Extraction

Color
Descriptors

Texture
Descriptors

New
Grayscale

Image

Texture
-
based
Segmentation

Feature
Extraction

Texture
Descriptors

Texture
Matching

Colorization
Process

Database

46

Grayscale Image Processing




47

Current Research

Process Workflow

Texture
-
based
Segmentation

Image

Image

Sample
Image

Feature
Extraction

Color
Descriptors

Texture
Descriptors

New
Grayscale

Image

Texture
-
based
Segmentation

Feature
Extraction

Texture
Descriptors

Texture
Matching

Colorization
Process

Database

48

Previous Research

Visual descriptors


Visual
descriptors
are
descriptions of the visual features of the
contents
of images.



They
describe elementary characteristics such as the shape,
color
,
and texture.



MPEG
-
7
is a multimedia content description standard. It was
standardized in ISO/IEC 15938 (Multimedia content description
interface
).



This
description
is associated
with the content itself, to allow fast
and efficient searching for material that is of interest to the user.



MPEG
-
7
is formally called Multimedia Content Description
Interface. Thus, it is not a standard which deals with the actual
encoding of moving pictures and audio, like MPEG
-
1, MPEG
-
2 and
MPEG
-
4. It uses XML to store
metadata
.

49

Previous Research

Visual descriptors

http://chatzichristofis.info/?page_id=213

The
Img
(Rummager
)
application was
developed in the
Automatic Control Systems & Robotics Laboratory at the
Democritus University of Thrace
-
Greece
.


The
application can execute an image search based on a
query image, either from XML
-
based index files, or
directly from a folder containing image files, extracting
the comparison features in real time.

Previous Research
(contd.)

Content
-
Based Image Retrieval

50

MPEG
-
7 EHD

Fuzzy Spatial BTDH

ADS

51

Previous Research

(contd.)

Content
-
Based Image Retrieval




Image Descriptors used:


MPEG
-
7 Homogeneous Texture
Descriptor:

Edge Histogram

Descriptor
(
EHD).



CCD for Medical Radiology Images:

Brightness and Texture Directionality
Histogram

(BTDH)

Fuzzy rule based scalable composite descriptor (BTDH)

is a compact
composite descriptor that can be used for the indexing and retrieval of
radiology medical images. This descriptor uses brightness and texture
characteristics as well as the spatial distribution of these characteristics in
one compact 1D vector. The most important characteristic of the proposed
descriptor is that its size adapts according to the storage capabilities of the
application that is using it. This characteristic renders the descriptor
appropriate for use in large medical (or gray scale) image databases.





Simulation Results
(contd.)

Content
-
Based Image Retrieval (CBIR)




52

Query image

MPEG
-
7 EHD


Fuzzy Spatial BTDH

ADS

Result

Matching
color

Result

Matching
color

Result

Matching
color



















































Simulation Results
(contd.)

Content
-
Based Image Retrieval (CBIR)




53

54

Current Research

Process Workflow

Texture
-
based
Segmentation

Image

Image

Sample
Image

Feature
Extraction

Color
Descriptors

Texture
Descriptors

New
Grayscale

Image

Texture
-
based
Segmentation

Feature
Extraction

Texture
Descriptors

Texture
Matching

Colorization
Process

Database

The
RGB

color space is defined by the three

chromaticities

of
the red, green, and blue

additive primaries, and can produce
any chromaticity that is the triangle defined by those primary
colors.



The
YCbCr

color space is used
in video and digital
photography systems.


Y
is the
luma

(luminance ) component and


Cb

and Cr are the blue
-
difference and red
-
difference
chroma

components.



Simulation Results
(contd.)

Image Colorization




55

56

Image from Wikipedia

Simulation Results
(contd.)

Image Colorization




Simulation Results
(contd.)

Colorization




57

Conclusion and Future Work





New and innovative method


Automating example
-
based colorization


Combines several state
-
of
-
the
-
art techniques



Reasonably accurate results were obtained


Several of the steps require custom parameters


computationally very intensive



Texture retrieval needs improvement


Complex textures containing multiple colors


Anisotropic diffusion for preserving strong edge information



Combining these techniques in order to automatically
colorize grayscale images is a viable option


58

Conclusion and Future Work

(contd.)





I
mages segmentation and clustering methods
computationally
very intensive, P
rocessing
time for each
600x450 sample color image
took 20
minutes on a quad
-
core
Intel 2.6 GHz processor
.


Texture
retrieval methods still need to be improved for scale
and rotation
invariance


S
tore
more complete color
descriptors
to accommodate
more complex textures containing multiple colors.


Anisotropic
diffusion could also be used to smooth the
Gabor response images while preserving strong edge
information.



Testing
conducted as part of this research proved that the
ability to combine these techniques in order to automatically
colorize
grayscale

images is a viable option.


59

References




[1]

Anat

Levin,
Dani

Lischinski
, and
Yair

Weiss, "Colorization using optimization,"
ACM Transactions on Graphics
, vol. 23, no. 3, p. 689

694, 2004.

[2]

R. Irony, D. Cohen
-
Or, and D.
Lischinski
, "Colorization by example," in
Eurographics

Symposium on Rendering
, 2005, p. 277

280.

[3]

Ashikhmin

M., Mueller K. Welsh T., "Transferring Color to
Greyscale

Images,".

[4]

X., Wan L., Qu Y., Wong T., Lin S., Leung C., Heng P. Liu, "Intrinsic
colorization,"
ACM Trans. Graph.
, vol. 27, no. 5, p. 152, 2008.

[5]

Malik J. Perona P., "Preattentive texture discrimination with early vision
mechanisms,"
J. Opt. Soc. Am. A
, vol. 7, no. 5, May 1990.

[6]

A. K. Jain and F.
Farrokhnia
, "Unsupervised texture segmentation using
Gabor filters
,"
Pattern Recognition
, vol. 24, no. 12, pp. 1167
-
1186, 1991.

[7]

Seo

Naotoshi
, "Texture Segmentation using Gabor Filters," University of
Maryland, College Park, MD, Project ENEE731 , 2006.

[8]

Xiaoming

Hu
,
Xinghui

Dong,
Jiahua

Wu, Ping
Zou

Junyu

Dong, "Texture
Segmentation Based on Probabilistic Index Maps," in
International
Conference on Education Technology and Computer
, 2009, pp. 35
-
39
.


60

References

(contd.
)




[9]

X Muñoz, J
Freixeneta
, X
Cufı́
a
, and J
Martı́
a
, "Strategies for image segmentation
combining region and boundary information,"
Pattern Recognition Letters
, vol.
24
, no.
1
-
3
, pp.
375
-
392
, January
2003
.

[10]

James C.
Bezdek
,
Pattern Recognition with Fuzzy Objective Function Algorithms
.
New York: Plenum, 1981.

[11]

Chuang Keh
-
Shih, Tzenga Hong
-
Long, Chen Sharon, Wu Jay, and Chen Tzong
-
Jer,
"Fuzzy c
-
means clustering with spatial information for image segmentation,"
Computerized Medical Imaging and Graphics
, vol. 30, no. 1, pp. 9
-
15, January
2006.

[12]

Zhou
Huiyu
, Schaefer Gerald,
Sadka

Abdul H., and
Celebi

M.
Emre
, "Anisotropic
Mean Shift Based Fuzzy C
-
Means Segmentation of
Dermoscopy

Images,"
IEEE
Journal of Selected Topics in Signal Processing
, vol. 3, no. 1, pp. 26
-
34, February
2009.

[13]

Stelios

Krinidis

and
Vassilios

Chatzis
, "A Robust Fuzzy Local Information C
-
means
Clustering Algorithm,"
Image Processing, IEEE Transactions on
, pp. 1
-
1, 2010.

[14]

Gauge

Christophe

and

Sasi

Sreela
,
"Automated Colorization of
Grayscale

Images
Using Texture Descriptors and a Modified Fuzzy C
-
Means Clustering,“

Journal of
Intelligent Learning Systems and Applications (JILSA), Vol. 4 No. 2, 2012, pp. 135
-
143, DOI: 10.4236/
jilsa
.

61

62

Questions?