4-1-03 - Brinkster

indexadjustmentInternet και Εφαρμογές Web

13 Νοε 2013 (πριν από 3 χρόνια και 10 μήνες)

75 εμφανίσεις

Image Search

Using Color and Texture

Project: NETRA


http://www
-
iplab.ece.ucsb.edu/netra/Netra.html



Examples: edge flow:
http://www
-
iplab.ece.ucsb.edu/netra/examples/Netra.ht
ml


Efficient Color Representation


For efficiency, Yining Den
g,et.al.
1

proposed using a
dominant color descriptor, which is the peak histogram
value for each of three colors RGB.


Previous implementations used colors from 10s to 100s of bins,
time and storage were excessive


Singular Value Decomposition used to reduce computational
complexity, however the reduction is not related to features and
will produce errors


Hilbert curve fitting
-

features close together in the original may be
far apart on the curve.


Color Moment Descriptors (mean, variance, and third order
moment) perform
slightly

worse than a high level histogram. The
mean colors may be quite different from any of the original colors


Since they perform satisfactorily even with this disadvantage, I
decided to implement this


The Dominant color descriptor takes the dominant colors from a
histogram and searches for colors within a distance of these colors
(each of which have a percentage of each of RGB)

Efficient Color Representation


Image is first segmented using the edgeflow algorithm


Color clustering is performed on each segment to obtain
representative colors


The dominant color descriptors is then:


F = {{c
i
, p
i
}, i = 1, …, N}


N is the number of color clusters


c
i

is the color vector


p
i

is its percentage where

p
i

= 1


Three to four colors are sufficient to describe an image region


Complexity of a search is O(m n) where n is the number of colors
in a query, m is the number of nodes accessed per query color


Texture Features


B.S Manjunath, W.Y. Ma
2,3

suggest using a Gabor function and its frequency
components, the Fourier transform


Higher frequency components may be sufficient to retrieve images


Calculating the entire Gabor function for a 512 x 512 image takes 9.3 seconds on a
SUN Spark 20, search and retrieval takes 1.02 seconds.


Using the adaptive filter that they suggest using four feature components takes 2.3
seconds for feature extraction and 0.7 seconds for adaptive filter selection and 0.1
seconds for retrieval


This was too long


Used edge flow to segment images
\


Mean and variance, presentation by Dr Lu


Using the Mean, Standard Deviation, Third and Forth Moment


Compatible with the color calculations, efficient, I have adopted this method


Implementation Parameters


Web based so that anyone can upload pictures and
try it out


Use of a robust language


Java:


Easy interface with web pages


Capable of performing appropriate calcs on the
server


Used several Java Servlets


ImageInfo
http://www.geocities.com/SiliconValley/Lakes/6686/
,
gets width, height, etc of an image


MultipartRequest
http://www.geocities.com/jasonpell/programs.html
, parses a
multipart form and writes out any files found


ServletMultipartRequest
http://www.millstone.org/documentation/apidocs/org/millstone/
webadapter/ServletMultipartRequest.html
, wrapper for
MultipartRequest for use in Servlets

Implementation Parameters


JSP


Java compiled on the fly


Java, SQL, and HTML mixed on the same page


SQL


Used Hypersonic SQL, an all Java implementation


Find a server that has all of these


Although I already have a web server (and FTP) on 2
home machines, determining how to set up a JSP and
HSQL server would be too time consuming (plus
Oracle SQL uses busy
-
waiting and takes up all your
clock cycles)


http://www.mycgiserver.com/


Free, although they email requests for donations for an upgrade


Sometimes slow, they are planning an upgrade to speed things
up


http://www.webappcabaret.com/

only 1 month free


http://www.brinkster.com/

• Moved the start page here because it is faster (and also free)
Implementation


I didn’t get as far as I had expected, but
have images being uploaded and added to a
DB


http://www25.brinkster.com/kenwschmidt/s
earch/index.html



Future Implementation


Texture


By Mean and variance


Mean: x = 1/n


x
i


Standard Deviation (second moment)


=
1/n


(x
i



x )
2


Third Moment = 1/n


(x
i



x )
3


Fourth Moment =
1/n


(x
i



x )
4


Search the DB for images within Y% of these
four texture parameters


Color


Record the mean of each of the three colors


Search for images within Y% of these



How to Implement?


Found some implementations which would
be easy but they were not standard SUN
Java, therefore the server that I had chosen
probably doesn’t have them available


Searching SUN’s documentation, I found
several ways to access pixels:


DataBuffer


SampleModel


BandedSampleModel


ColorModel


How to Implement
footnote

Java has some built in functions that are interesting




float[] elements =



{ 0.0f,
-
1.0f, 0.0f,
-
1.0f, 4.f,
-
1.0f, 0.0f,
-
1.0f,
0.0f};


BufferedImage bimg = new



BufferedImage(bw,bh,BufferedImage.TYPE_INT_RGB);


Kernel kernel = new Kernel(3, 3, elements);


ConvolveOp cop = new ConvolveOp(kernel,



ConvolveOp.EDGE_NO_OP,null);


cop.filter(bi,bimg);


How to Implement
footnote



byte reverse[] = new byte[256];


for (int j=0; j<200; j++){


reverse[j]=(byte)(256
-
j);


}


ByteLookupTable blut=new ByteLookupTable(0, reverse);


LookupOp lop = new LookupOp(blut, null);


lop.filter(bi,bimg);


How to Implement
footnote


RescaleOp rop = new RescaleOp(1.5f, 1.0f, null);


rop.filter(bi,bimg);



How to Implement
footnote


Sharpening with Convolution:



float[] elements =


{ 0.0f,
-
1.0f, 0.0f,
-
1.0f, 5.f,
-
1.0f, 0.0f,
-
1.0f, 0.0f};




Kernel kernel = new Kernel(3,3,elements);


ConvolveOp cop = new ConvolveOp(kernel, ConvolveOp.EDGE_NO_OP,
null);


cop.filter(bi,bimg);


How to Implement


Each of these did too much, I just wanted
access to the raw pixels


java.awt.image.PixelGrabber

Pseudo Code


for (int j = 0; j < h; j++)


{


for (int i = 0; i < w; i++)


{



pixel = pixels [ j * w + i ];



red = (pixel >> 16) & 0xff;



green = (pixel >> 8) & 0xff;



blue = (pixel ) & 0xff;



redSum += red;



greenSum += green;



blueSum += blue;



}

}

redMean = redSum / noOfPixels;

greenMean = greenSum / noOfPixels;

blueMean = blueSum / noOfPixels;

overallSum = redSum + greenSum + blueSum;

overallMean = overallSum / noOfPixels;

for (int j = 0; j < h; j++)

{


for (int i = 0; i < w; i++)


{


pixel = pixels [ j * w + i ];



red = (pixel >> 16) & 0xff;



green = (pixel >> 8) & 0xff;



blue = (pixel ) & 0xff;



redSum2 += pow ( ( red
-

redMean ), 2 );



greenSum2 += pow ( ( green
-

greenMean ), 2 );



blueSum2 += pow ( ( blue
-

blueMean ), 2 );



redSum3 += pow ( ( red
-

redMean ), 3 );



greenSum3 += pow ( ( green
-

greenMean ), 3 );



blueSum3 += pow ( ( blue
-

blueMean ), 3 );



redSum4 += pow ( ( red
-

redMean ), 4 );



greenSum4 += pow ( ( green
-

greenMean ), 4 );



blueSum4 += pow ( ( blue
-

blueMean ), 4 );


}

}

redMoment2 = redSum2 / noOfPixels;

greenMoment2 = greenSum2 / noOfPixels;

blueMoment2 = blueSum2 / noOfPixels;

redMoment3 = redSum3 / noOfPixels;

greenMoment3 = greenSum3 / noOfPixels;

blueMoment3 = blueSum3 / noOfPixels;

redMoment4 = redSum4 / noOfPixels;

greenMoment4 = greenSum4 / noOfPixels;

blueMoment4 = blueSum4 / noOfPixels;

overallMoment2 = ( redMoment2 + greenMoment2 + blueMoment2 ) / 3;

overallMoment3 = ( redMoment3 + greenMoment3 + blueMoment3 ) / 3;

overallMoment4 = ( redMoment4 + greenMoment4 + blueMoment4 ) / 3;

References

1.
An Efficient Color Representation for Image Retrieval, Yining Deng, B. S.
Manjunath, Charles Kenney, Michael S. Moore, Hyundoo Shin, IEEE
TRANSACTIONS ON IMAGE PROCESSING, VOL. 10, NO. 1,
JANUARY 2001

2.
Texture Features for Browsing and Retrieval of Image Data, B.S
Manjunath, W.Y. Ma, IEEE TRANSACTIONS ON PATTERN
ANALYSIS AND MACHINE INTELLIGENCE, VOL. 18, NO. 8,
JANUARY 1996

3.
A Texture Thesaurus for Browsing Large Aerial Photographs, Wei
-
Ying Ma and B. S. Manjunath,
JOURNAL OF THE AMERICAN
SOCIETY FOR INFORMATION SCIENCE. 49(7):633

648, 1998

4.
Texture Features and Learning Similarity,
Wei
-
Ying Ma and B. S.
Manjunath, 1063
-
6919/96IEEE