Image Search
Using Color and Texture
Project: NETRA
•
http://www

iplab.ece.ucsb.edu/netra/Netra.html
•
Examples: edge flow:
http://www

iplab.ece.ucsb.edu/netra/examples/Netra.ht
ml
Efficient Color Representation
•
For efficiency, Yining Den
g,et.al.
1
proposed using a
dominant color descriptor, which is the peak histogram
value for each of three colors RGB.
–
Previous implementations used colors from 10s to 100s of bins,
time and storage were excessive
–
Singular Value Decomposition used to reduce computational
complexity, however the reduction is not related to features and
will produce errors
–
Hilbert curve fitting

features close together in the original may be
far apart on the curve.
–
Color Moment Descriptors (mean, variance, and third order
moment) perform
slightly
worse than a high level histogram. The
mean colors may be quite different from any of the original colors
•
Since they perform satisfactorily even with this disadvantage, I
decided to implement this
–
The Dominant color descriptor takes the dominant colors from a
histogram and searches for colors within a distance of these colors
(each of which have a percentage of each of RGB)
Efficient Color Representation
–
Image is first segmented using the edgeflow algorithm
–
Color clustering is performed on each segment to obtain
representative colors
–
The dominant color descriptors is then:
•
F = {{c
i
, p
i
}, i = 1, …, N}
•
N is the number of color clusters
•
c
i
is the color vector
•
p
i
is its percentage where
p
i
= 1
–
Three to four colors are sufficient to describe an image region
–
Complexity of a search is O(m n) where n is the number of colors
in a query, m is the number of nodes accessed per query color
Texture Features
•
B.S Manjunath, W.Y. Ma
2,3
suggest using a Gabor function and its frequency
components, the Fourier transform
–
Higher frequency components may be sufficient to retrieve images
–
Calculating the entire Gabor function for a 512 x 512 image takes 9.3 seconds on a
SUN Spark 20, search and retrieval takes 1.02 seconds.
–
Using the adaptive filter that they suggest using four feature components takes 2.3
seconds for feature extraction and 0.7 seconds for adaptive filter selection and 0.1
seconds for retrieval
–
This was too long
–
Used edge flow to segment images
\
•
Mean and variance, presentation by Dr Lu
–
Using the Mean, Standard Deviation, Third and Forth Moment
–
Compatible with the color calculations, efficient, I have adopted this method
Implementation Parameters
•
Web based so that anyone can upload pictures and
try it out
•
Use of a robust language
–
Java:
•
Easy interface with web pages
•
Capable of performing appropriate calcs on the
server
•
Used several Java Servlets
–
ImageInfo
http://www.geocities.com/SiliconValley/Lakes/6686/
,
gets width, height, etc of an image
–
MultipartRequest
http://www.geocities.com/jasonpell/programs.html
, parses a
multipart form and writes out any files found
–
ServletMultipartRequest
http://www.millstone.org/documentation/apidocs/org/millstone/
webadapter/ServletMultipartRequest.html
, wrapper for
MultipartRequest for use in Servlets
Implementation Parameters
–
JSP
•
Java compiled on the fly
•
Java, SQL, and HTML mixed on the same page
–
SQL
•
Used Hypersonic SQL, an all Java implementation
•
Find a server that has all of these
–
Although I already have a web server (and FTP) on 2
home machines, determining how to set up a JSP and
HSQL server would be too time consuming (plus
Oracle SQL uses busy

waiting and takes up all your
clock cycles)
–
http://www.mycgiserver.com/
•
Free, although they email requests for donations for an upgrade
•
Sometimes slow, they are planning an upgrade to speed things
up
–
http://www.webappcabaret.com/
only 1 month free
–
http://www.brinkster.com/
• Moved the start page here because it is faster (and also free)
Implementation
•
I didn’t get as far as I had expected, but
have images being uploaded and added to a
DB
•
http://www25.brinkster.com/kenwschmidt/s
earch/index.html
Future Implementation
•
Texture
–
By Mean and variance
•
Mean: x = 1/n
x
i
•
Standard Deviation (second moment)
–
=
1/n
(x
i
–
x )
2
•
Third Moment = 1/n
(x
i
–
x )
3
•
Fourth Moment =
1/n
(x
i
–
x )
4
–
Search the DB for images within Y% of these
four texture parameters
•
Color
–
Record the mean of each of the three colors
–
Search for images within Y% of these
How to Implement?
•
Found some implementations which would
be easy but they were not standard SUN
Java, therefore the server that I had chosen
probably doesn’t have them available
•
Searching SUN’s documentation, I found
several ways to access pixels:
–
DataBuffer
–
SampleModel
–
BandedSampleModel
–
ColorModel
How to Implement
footnote
Java has some built in functions that are interesting
float[] elements =
{ 0.0f,

1.0f, 0.0f,

1.0f, 4.f,

1.0f, 0.0f,

1.0f,
0.0f};
BufferedImage bimg = new
BufferedImage(bw,bh,BufferedImage.TYPE_INT_RGB);
Kernel kernel = new Kernel(3, 3, elements);
ConvolveOp cop = new ConvolveOp(kernel,
ConvolveOp.EDGE_NO_OP,null);
cop.filter(bi,bimg);
How to Implement
footnote
byte reverse[] = new byte[256];
for (int j=0; j<200; j++){
reverse[j]=(byte)(256

j);
}
ByteLookupTable blut=new ByteLookupTable(0, reverse);
LookupOp lop = new LookupOp(blut, null);
lop.filter(bi,bimg);
How to Implement
footnote
RescaleOp rop = new RescaleOp(1.5f, 1.0f, null);
rop.filter(bi,bimg);
How to Implement
footnote
Sharpening with Convolution:
float[] elements =
{ 0.0f,

1.0f, 0.0f,

1.0f, 5.f,

1.0f, 0.0f,

1.0f, 0.0f};
Kernel kernel = new Kernel(3,3,elements);
ConvolveOp cop = new ConvolveOp(kernel, ConvolveOp.EDGE_NO_OP,
null);
cop.filter(bi,bimg);
How to Implement
•
Each of these did too much, I just wanted
access to the raw pixels
–
java.awt.image.PixelGrabber
Pseudo Code
for (int j = 0; j < h; j++)
{
for (int i = 0; i < w; i++)
{
pixel = pixels [ j * w + i ];
red = (pixel >> 16) & 0xff;
green = (pixel >> 8) & 0xff;
blue = (pixel ) & 0xff;
redSum += red;
greenSum += green;
blueSum += blue;
}
}
redMean = redSum / noOfPixels;
greenMean = greenSum / noOfPixels;
blueMean = blueSum / noOfPixels;
overallSum = redSum + greenSum + blueSum;
overallMean = overallSum / noOfPixels;
for (int j = 0; j < h; j++)
{
for (int i = 0; i < w; i++)
{
pixel = pixels [ j * w + i ];
red = (pixel >> 16) & 0xff;
green = (pixel >> 8) & 0xff;
blue = (pixel ) & 0xff;
redSum2 += pow ( ( red

redMean ), 2 );
greenSum2 += pow ( ( green

greenMean ), 2 );
blueSum2 += pow ( ( blue

blueMean ), 2 );
redSum3 += pow ( ( red

redMean ), 3 );
greenSum3 += pow ( ( green

greenMean ), 3 );
blueSum3 += pow ( ( blue

blueMean ), 3 );
redSum4 += pow ( ( red

redMean ), 4 );
greenSum4 += pow ( ( green

greenMean ), 4 );
blueSum4 += pow ( ( blue

blueMean ), 4 );
}
}
redMoment2 = redSum2 / noOfPixels;
greenMoment2 = greenSum2 / noOfPixels;
blueMoment2 = blueSum2 / noOfPixels;
redMoment3 = redSum3 / noOfPixels;
greenMoment3 = greenSum3 / noOfPixels;
blueMoment3 = blueSum3 / noOfPixels;
redMoment4 = redSum4 / noOfPixels;
greenMoment4 = greenSum4 / noOfPixels;
blueMoment4 = blueSum4 / noOfPixels;
overallMoment2 = ( redMoment2 + greenMoment2 + blueMoment2 ) / 3;
overallMoment3 = ( redMoment3 + greenMoment3 + blueMoment3 ) / 3;
overallMoment4 = ( redMoment4 + greenMoment4 + blueMoment4 ) / 3;
References
1.
An Efficient Color Representation for Image Retrieval, Yining Deng, B. S.
Manjunath, Charles Kenney, Michael S. Moore, Hyundoo Shin, IEEE
TRANSACTIONS ON IMAGE PROCESSING, VOL. 10, NO. 1,
JANUARY 2001
2.
Texture Features for Browsing and Retrieval of Image Data, B.S
Manjunath, W.Y. Ma, IEEE TRANSACTIONS ON PATTERN
ANALYSIS AND MACHINE INTELLIGENCE, VOL. 18, NO. 8,
JANUARY 1996
3.
A Texture Thesaurus for Browsing Large Aerial Photographs, Wei

Ying Ma and B. S. Manjunath,
JOURNAL OF THE AMERICAN
SOCIETY FOR INFORMATION SCIENCE. 49(7):633
–
648, 1998
4.
Texture Features and Learning Similarity,
Wei

Ying Ma and B. S.
Manjunath, 1063

6919/96IEEE
Comments 0
Log in to post a comment