Objectives

tamerunSoftware and s/w Development

Aug 15, 2012 (5 years and 3 months ago)

209 views

Updated 2/3/06.

Objectives



Gain an understanding of segmentation



Perform a segmentation task and make use of segmentation result

Segmentation is:



To distinguish objects from each other and their background



To divide the image into regions or segments, each

of which is in some sense
homogeneous, but the union of adjacent segments is not homogeneous in the
same sense.



Homogeneity here is characterized by some properties like

o

Smoothly varying intensity, similar statistics, or colour.


Overview of Deliverables

The key functionality that should be demonstrated by your deliverable application is:


1.

The ability to load an image containing
n

distinct objects of interest (in Figure
1 we have three interesting objects),

2.

To also read an appropriate number of x and y coo
rdinates (from the
NetBeans arguments) that will specify the location of segmentation “seeds”

3.

Create a new image that segments the objects of interest from the background
of the original image (Figure 2).

4.

Calculate the distance between the centre of each o
bject and the other objects.
You can simply print the centroid coordinates and distance results in
Netbeans output window or visualise them in the interface (Figure 3).


Figure 1. An image with three objects of interest









Figure 2. Creat
ing a new image (on the right) by segmenting three objects from
the background of the image on the left

Seed

Seed

Seed




Figure 3. Calculate the distances between the centre of each object



The Segmentation Algorithm


There are a number of segmentation programs in Ni
ck Efford’s package in chapter 10.
Here let’s look at one of the programs that we have discussed in the lecture, “region
growing” method.


The algorithm of region growing is as below (from Nick’s book, pp263):


Let
f

be an image for which regions are to be

grown

Define a set of regions, R
1
, R
2
,…, R
n,
each consisting of a single seed pixel

repeat


for
i
=1 to
n
,
do


for
each pixel,
p
, at the border of
R
i
,

do


for all
neighbours of p
do


Let
x, y
be the neighbour’s coordina
tes


Let
u
i

be the mean grey level of pixels in
R
i



if

the neighbour is unassigned and |
f(x,y)


u
i
| <= delta (threshold)
then


Add neighbour to
R
i


Update
u
i



end if


end for


end for


end for

until

no more pixels are being assigned to regions



Lab Instructions


The first section of the lab is concerned with understanding what the code in
“RegionGrower.java” class does.
Y
ou do not have to run the RegionGrower.java
file

(actually you can’t as there is no main method). The steps below highlight how
each sentence in this algorithm above is implemented in Java.



93

102

82

Step1
In NetBeans, open “Open RegionGrower.java ” which is loca
ted in the
directory “javaim
-
Nick
\
Classes
\
Source
\
com
\
pearsoneduc
\
ip
\
op”.


The easiest way to do this is open an existing project you have been working on in the
labs (i.e. a project that has the image libraries ipapps.jar and iplib.jar and any of the
Chap

folders loaded). Next right click on the project name and select Properties.
Make sure the sources option is selected in the Project Properties window and click on
the “Add Folder” button in the “Source Package Folders” area.




Another dialogue box wi
ll appear. Be sure you specify the location of the folder
correctly (javaim
-
Nick
\
Classes
\
Source
\
com
\
pearsoneduc
\
ip
\
op
Please note the
Source folder should be included in the path
)



Although there appears to be no files in the window click on the “Open”

button.


IMPORTANT NOTE

Please disregard the previous lab sheet instructions regarding the removal of the
“ipapps.jar” library. You should now make sure your project has both the “iplib.jar”
and “ipapps.jar” included in the libraries section of project pr
operties. Sorry for the
confusion caused!






Now click on “OK” in the previous Project Properties window. Double click on the
“RegionGrower.java” class file in the Project view in NetBeans, you will see the
original code on the right in the code edito
r window.


Both libraries
included

Step 2
C
lick on the “View” menu and choose “show line numbers”, so that you can
see the line numbers of the code if not present, which are used as the reading
references for the rest of this document. If the “show line numbers” option is greyed
-
out please click once in the editor window (anywhere in the code) and then try to
select the option again.






Please try to read through the whole class “RegionGrower” as much as you can. The
following section provides some help for you by explainin
g some important parts in
the class.


In class “RegionGrower.java”,

Line 50
-
68
are to define the types of neighbourhood surrounding a pixel. A 4
-
neighbourhood contains only the pixels above, below, to the left and to the right of the
central pixel, An 8
-
ne
ighbourhood contains all the pixels of a 4
-
neighbourhood, plus
four diagonal neighbours. These types will be used by member variable “
private
Point[] delta”, defined in line 94, used for locating neighbourhood coordinates when
growing region.


Line 74,

sou
rceImage to represent the image to be segmented. (see the comment lines
in the code)

Line 77
-
105
, read through those member variables, in particular “List seedpixels”
and “connectivity” and “squareThreshold”.

Line 87
private int numBands is to represent t
he colour band, namely R, G, B, Grey.
If the processed image is a grey image, then numBands=0; if colour image, the value
in numbBands, 0, 1, 2, represent R, G, B channels.

Line 88,

to define a Boolean variable as a checker for each pixel to mark whether a

pixel has been assigned to a region or not. Once a pixel has been assigned to a region,
then its checker assigned[pixel.y][pixel.x] = true;

Line 89

border[y][x] is to record current pixel at [x,y] is a border of a region.
According to the algorithm, the g
rowing processing only happens at the border pixels
(for pixels that have been assigned to a region and do not belong to a region’s border,
there is no need to process them any more).

Line 97, 98,
regionSum and regionMean is to keep the sum and mean of the

current
growing region as a measure of whether a new pixel should belong to this region
based on the distance of the intensity of the new pixel and the mean of the region. (see
line306 and line 390.)

Line 119, 134
definition of RegionGrower method.

Line 1
37
-
141
initialisation

Line 142
-
175
, please read respective comments in the code.

Line 176
-

278
, define a series of public methods. Please read respective comments on
these functions.


Line 284
is the key method grow(). In order to understand this method, y
ou may need
to understand the method initialise() defined in Line 352
-
380, which has been called
in RegionGrower() constructor, in line 172. So now we are going to jump to
initialise() method to know what this method does then come back to grow() method.


In initialise(),

Line 354
, Iterator iterator = seedPixels.iterator(); seed pixels are stored in a
list, each element (each seed pixel) in the list can be accessed by “iterator”.

Line 367

out.setSample(pixel.x, pixel.y, 0, n); is to set a sequence

number to
seed location to be grown into larger region

Line 370

border[pixel.y][pixel.x] = n; to initialise border value used in
grow().


Back to the
grow()
method…

Line 288
, Raster in = sourceImage.getRaster(); to define a raster object for the
pro
cessed image, in order to access its pixel value.

Line 294
-
295
, iterate over the entire image.

Line 301, 302
, nx, ny, are coordinates of neighbourhood candidates that possibly
belong to the region

Line 303,

is to check whether this neighbour has been assig
ned to a region yet, if not,
then

Line 305, 306
, check the distance between its pixel value and the mean of the region
intensity is smaller than a threshold.

If yes, then

Line 308
-
line312
are to add this pixel to the region, and update the border of the
r
egion (because every time the program only checks the neighbourhood of the border
pixels), and update the statistics (such as the mean of the region intensity). The update
function is in line 406
-
413.


Step 3
In this step we are going to examine another cl
ass


the “RegionGrow.java”
class
.
This class can be found in the “javaim
-
Nick
\
Apps
\
Chap10” directory and you
should include this directory into your project using the same method explained in
Step 1.
At the same time you include this new directory you sho
uld also remove
the ‘
javaim
-
Nick
\
Classes
\
Source
\
com
\
pearsoneduc
\
ip
\
op’ directory that we
included in Step 1. We have finished examining the “RegionGrower.java” code,
and if we leave this directory in our project we will get several compiler errors
due to t
he way NetBeans and Java deals with the project library files.


The following section highlights the key functionality in the “RegionGrow.java” class


Line 49
-
55


public static void makeRegionWhite(BufferedImage image) {


WritableRaster raster = image.
getRaster();


for (int y = 0; y < raster.getHeight(); ++y)


for (int x = 0; x < raster.getWidth(); ++x)


if (raster.getSample(x, y, 0) > 0)


raster.setSample(x, y, 0, 255);


}


This is the place that you can manipulate /record the
segmented region in order to
complete the coursework task. You can either write a separate method similar to this
or directly change the code here. Although other ways, such as directly adding your
code to the grow() method will work, I think that is less
elegant.


Line 64
-
69


ImageDecoder input = ImageFile.createImageDecoder(argv[0]); //open input image to be


segmented


ImageEncoder o
utput = ImageFile.createImageEncoder(argv[1]); //create image endcoder for


output


int x = Integer.parseInt(argv[2]); //Integer.parse
Int(s) is a very useful method to convert user


input string (such as 4,) into actual number

Line 73
-
75


BufferedImage image = input.decodeAsBufferedImage(); // // Load image



List seeds = new ArrayList(); ////create seed pixel


seeds.add(new Point(x, y));

Line 80
-
89


// Grow region around seed pixel


RegionGrower grower =


new RegionGrower(image, seeds, connectivity, threshold);


grower
.growToCompletion();


BufferedImage outputImage = grower.getRegionImage(); //create output image as BufferedImage


makeRegionWhite(outputImage); // output image is black and white (0 and 255), with region



in white

Line 93:

output.encode(outputImage); // Write output image to file


Step 4
In this step we will u
se the “RegionGrow.java” class to call RegionGrower
class that we examined in Step 1. First open
the “RegionGrow.java” class in Chap10
by double clicking on the file in the Projects window. Read the comment section in
the beginning of the program on the instruction on how to run the code. The
arguments are:


input filename, output filename, seed pix
el x and y coordinates, connectivity,
threshold.

e.g.

java RegionGrow l1.jpg region.jpg 100 50 4 35


or in NetBeans arguments


l1.jpg region.jpg 100 50 4 35


This means you need to input 6 arguments to run this program. This program reads
an image from

a file (say l1.jpg, or mattgrey.jpg) and grows a region around a
specified seed pixel (here, say 100 50 as the seed x,y coordinate), using either 4
-

or 8
-
connectivity and a given threshold (eg, 35) on the difference between a pixel's grey
level or colour

and the mean grey level or colour of the region (the program actually
uses square distance, see details in line 140, 306 and 390 in RegionGrower()). The
output image (e.g. region.jpg) has pixels set to 255 if they belong to the region, or 0
otherwise.


N
ote
:
the class RegionGrower can actually grow from a list of seeds, whilst this
program only pass one pixel coordinate as one specified seed pixel. If you develop
your own segmentation program for the coursework, you may wish to pass more than
one seed (a
list of seed) to RegionGrower to segment the image where are there more
than one objects/regions.


Step 5

Check the result in the output file in the name that you provided in the
arguments. The output file should be under the working directory that you hav
e set up
in the properties. The result should be another image (e.g, region.jpg), which is a
binary image.


Step 6, an important note,
you can also run ImageViewer in chapter 05 to check the
object pixel locations in the image that you are running for seg
mentation (input the
same image for ImageViewer in order to do so) if you are not sure where to set the
seed coordinates.


Step 7

You also can run RegionGrowTool in chapter 10 to see an additional example
of the algorithm. For your coursework it may be bes
t to open the “RegionGrow.java”
file located in Apps
\
Chap10 and perform any coding necessary here.


When working on your coursework related to segmentation, it is perhaps very useful
to find some image examples with simple objects in it. “Simple” means wit
hin object
or components of object, the colour intensity is relatively even, so that the
segmentation algorithm is less affected by noise and can achieve better results.


Deliverable:


Choose an image with at least three simple objects (“simple” means rath
er uniform
background and uniform colour shades within one object, such as the 3friends.jpg
image provided on the course website.) This simplifies the task of segmenting our
image correctly (using more complicated images with overlapping areas of interest
will be harder to segment correctly). Once you have segmented the objects of interest
from the background you should then identify the centre of each object and calculate
the distance between them.


The following steps are suggested:

1.

Perform image segmen
tation

and record the segmented regions (constituent
pixels of the region or boundary). You can do this by using the
“RegrionGrow.java” class (available in Apps
\
Chap10) with appropriate
arguments

2.

Calculate the centre of the object.

The coordinate of the ce
ntre (centroid) is
the average of all x coordinates and y coordinates along the boundary of the
object, or the average of all x coordinates and y coordinates in the region. To
calculate the centre of segmented objects, you should create a new class called
something like “NewRegionGrow.java” in the Chap10 directory. You can then
copy the original “RegionGrow” code into this file (don’t forget to change the
class and constructor names!) and either directly add the required function the
method “MakeRegionWhite
()” or add an additional function alongside
MakeRegionWhite().For more help on how to complete this part of the task
you should also look at the additional coursework hints on the module
website.

3.

Calculate the distance between each object of interest.

Afte
r obtaining the
centroids of various objects/regions, the distance between them can be
calculated based on
Euclidean distance



||d|| =
2
2
1
2
2
1
)
(
)
(
y
y
x
x





4.

When there are more than two objects, you can compare the distances bet
ween
different objects, and check whether the calculation produced by your system
is consistent with your eye observations


Several hints have also been provided on how to approach this coursework task on the
module home page, entitled “More on Segmentatio
n
-

Coursework Hints”.