# Image Arithmetic

AI and Robotics

Nov 6, 2013 (4 years and 6 months ago)

139 views

1

Image Arithmetic

Contents

1.

-

(or constant)

2.
Subtraction

-

pointwise subtraction: image
-

image
(or constant)

3.
Multiplication

-

pointwise multiplication: im
ages *
image (or constant)

4.
Division

-

pointwise division: images / image
(or constant)

5.
Blending

-

pointwise linear combination of two
images

6.
Logical AND/NAND

-

pointwise logical ANDing/NANDing
of two binary images

7.
Logical

OR/NOR

-

pointwise logical ORing/NORing of
two binary images

8.
Logical XOR/XNOR

-

pointwise logical XORing/XNORing
of two binary images

9.
Invert/Logical NOT

-

pointwise inversion of a binary image

2

10.
Bitshift Operators

-

pointwise scaling of an i
mage

Image arithmetic applies one of the standard arithmetic
operations or a logical operator to two or more images. The
operators are applied in a pixel
-
by
-
pixel fashion which means
that the value of a pixel in the output imag
e depends only on
the values of the corresponding pixels in the input images.
Hence, the images normally have to be of the same size. One
of the input images may be a constant value, for example
when adding a constant offset to an image.

Although image ar
ithmetic is the most simple form of image
processing, there is a wide range of applications. A main
advantage of arithmetic operators is that the process is very
simple and therefore fast.

In many applications the processed images are taken from
the same
scene at different points of time, as, for example, in
reduction of random noise by adding successive images of
the same scene or motion detection by subtracting two
successive images.

Logical operators are often used to combine two (mostly

3

binary) images
. In the case of integer images, the logical
operator is normally applied in a bitwise fashion. Then we can,
for example, use a binary mask to select a particular region of
an image.

Hypermedia Image Processing Reference

©1996 R. Fisher, S. Perkins, A. W
alker and E. Wolfart.

Published by J. Wiley & Sons, Ltd. This version of HIPR may
differ from the original published version due to user
customization.

1.

Common Names: Pixel Add, Sum, Offset

Brief Description

In its most straightforward im
plementation, this operator takes
as input two identically sized images and produces as output
a third image of the same size as the first two, in which each
pixel value
[1]

is the sum of the values of the corresponding
pixel from each of the two input imag
es. More sophisticated
versions allow more than two images to be combined with a
single operation.

A common variant of the operator simply allows a specified

4

constant to be added to every pixel.

How It Works

The addition of two images is performed straig
htforwardly in a
single pass. The output pixel values are given by:

Or if it is simply desired to add a constant value C to a single
image then:

If the pixel values in the input images are actually vectors
rather than scalar values (e.g. for
color i
mages
[2]
)

then the
individual components (e.g.
red,

blue and green
components
[3]
) are simply added separately to produce the
output value.

If the image format being used only supports, say
8
-
bit integer
pixel values
[4]
, then it is very easy for the result

of the
addition to be greater than the maximum allowed pixel value.
The effect of this depends upon the particular implementation.
The overflowing pixel values might just be set to the
maximum allowed value, an effect known as
saturation
[5]
.

5

Alternatively

the pixel values might wrap around from zero
again. If the image format supports pixel values with a much
larger range, e.g. 32
-
bit integers or floating point numbers,
then this problem does not occur so much.

Guidelines for Use

ost commonly as a sub
-
step in
some more complicated process rather than as a useful
operator in its own right. As an example we show how
addition can be used to overlay the output from an
edge
detector
[6]

on top of the original image after suitable
[7]

has been carried out.

The image

shows a simple flat dark object against a light background.
Applying the Canny edge detector to this image, we obtain

6

Suppose that our task is to overlay this edge data on top of
the original image. The image

i
s the result of straightforwardly adding the two images. Since
the sum of the edge pixels and the underlying values in the
original is greater than the maximum possible pixel value,
these pixels are (in this implementation) wrapped around.
Therefore these
pixels have a rather low pixel value and it is
hard to distinguish them from the surrounding pixels. In order
to avoid the pixel overflow we need to replace pixels in the
original image with the corresponding edge data pixels, at
every place where the edge

data pixels are non
-
zero. The
way to do this is to mask off a region of the original image

The mask is made by thresholding the edge data at a pixel

7

value of 128 in order to produce

This mask is then inverted and subsequentl
y ANDed with the
original image to produce

edge data to produce

This image now clearly shows that the Canny edge detector
has done an extremely good job of localizing the edges of the
original o
bject accurately. It also shows how the response of
the edge detector drops off at the fuzzier left hand edge of the

8

object.

pixels in an image so as to brighten that image. For example,

a constant value of 50 to

yields

It is important to realize that if the input images are already
quite bright, then straight addition may produce a pixel value
overflow. Image

9

shows the results of adding 100 to the above image. Most of
the backgr
ound pixels are greater than the possible maximum
(255) and therefore are (with this implementation of addition)
wrapped around from zero. If we implement the operator in
such a way that pixel values exceeding the maximum value
are set to 255 (i.e. using a

hard limit) we obtain

This image looks more natural than the wrapped around one.
However, due to the saturation, we lose a certain amount of
information, since all the values exceeding the maximum
value are set to the same graylevel.

10

In this case, the

pixel values should be scaled down before

is the result of scaling the original with 0.8 and adding a
constant value of 100. Although the image is brighter than the
original, it has lost contrast due to the scaling. In most cases,
s
caling the image with a factor larger than 1 without using
addition at all provides a better way to brighten an image, as it
increases the image contrast. For comparison,

is the original image multiplied with 1.3.

11

Blending provides a slightly more soph
isticated way of
merging two images which ensures that saturation cannot
happen.

When adding color images it is important to consider how the
color information has been encoded. The section on 8
-
bit
color images describes the issues to be aware of when
ding such images.

Exercises

1.
Add the above Canny edge image to its original, using
different implementation's of pixel addition which handle the
pixel overflow in different ways. Which one yields the best
results for this implementation?

2.
Use skeleto
nization to produce a skeleton of

Add the skeleton to the original. Which problems do you face
and how might they be solved?

3.
Add a constant value of 255 to

12

Use two different implementations, one wrapping around from
zero all pixel values exceedin
g the maximum value and one
using a hard limit of 255. Comment on the results.

References

A. Marion An Introduction to Image Processing, Chapman and
Hall, 1991, pp 242
-

244.

D. Vernon Machine Vision, Prentice
-
Hall, 1991, pp 51
-

52.

13

2.
Pixel

Subtraction

Common Names: Pixel difference, Pixel subtract

The pixel subtraction operator takes two images as input and
produces as output a third image whose
pixel values
[1]

are
simply those of the first image minus the corresponding pixel
values from t
he second image. It is also often possible to just
use a single image as input and subtract a constant value
from all the pixels. Some versions of the operator will just
output the absolute difference between pixel values, rather
than the straightforward s
igned output.

How It Works

The subtraction of two images is performed straightforwardly
in a single pass. The output pixel values are given by:

14

Or if the operator computes absolute differences between the
two input images then:

Or if it is simply de
sired to subtract a constant value C from a
single image then:

If the pixel values in the input images are actually vectors
rather than scalar values (e.g. for
color images
[2]
)

then the
individual components (e.g.
red, blue and green
components
[3]
) are
simply subtracted separately to produce
the output value.

Implementations of the operator vary as to what they do if the
output pixel values are negative. Some work with image
formats that support negatively
-
valued pixels, in which case
the negative value
s are fine (and the way in which they are
displayed will be determined by the display
colormap
[8]
).

If
the image format does not support negative numbers then
often such pixels are just set to zero (i.e. black typically).
Alternatively, the operator may `
w
rap
[5]
' negative values, so

15

that for instance
-
30 appears in the output as 226 (assuming
8
-
bit pixel values
[4]
).

If the operator calculates absolute differences and the two
input images use the same pixel value type, then it is
impossible for the output p
ixel values to be outside the range
that may be represented by the input pixel type and so this
problem does not arise. This is one good reason for using
absolute differences.

Guidelines for Use

Image subtraction is used both as a sub
-
step in complicated
image processing sequences, and also as an important
operator in its own right.

A common use is to subtract background variations in
illumination from a scene so that the foreground objects in it
may be more easily analyzed. For instance,

16

shows some t
ext which has been badly illuminated during
capture so that there is a strong illumination gradient across
the image. If we wish to separate out the foreground text from
the background page, then the obvious method for black on
white text is simply to
thre
shold

the image on the basis of
intensity. However, simple thresholding fails here due to the
illumination gradient. A typical failed attempt looks like

Now it may be that we cannot adjust the illumination, but we
can put different things in the scene.
This is often the case
with microscope imaging, for instance. So we replace the text
with a sheet of white paper and without changing anything
else we capture a new image, as shown in

17

This image is the lightfield. Now we can subtract the lightfield
imag
e from the original image to attempt to eliminate variation
in the background intensity. Before doing that an offset of 100
is added to the first image to in order avoid getting negative
numbers and we also use
32
-
bit integer pixel values
[4]

to
avoid overf
low problems. The result of the subtraction is
shown in

Note that the background intensity of the image is much more
uniform than before, although the contrast in the lower part of
the image is still poor. Straightforward thresholding can now

18

achieve be
tter results than before, as shown in

which is the result of thresholding at a pixel value of 80. Note
that the results are still not ideal, since in the poorly lit areas
of the image the contrast (i.e. difference between foreground
and background inten
sity) is much lower than in the brightly lit
areas, making a suitable threshold difficult or impossible to
find. Compare these results with the example described under
pixel division
.

Absolute image differencing is also used for change detection.
If the a
bsolute difference between two frames of a sequence
of images is formed, and there is nothing moving in the scene,
then the output will mostly consist of zero value pixels. If
however, there is movement going on, then pixels in regions
of the image where t
he intensity changes spatially, will exhibit
significant absolute differences between the two frames.

19

As an example of such change detection, consider

which shows an image of a collection of screws and bolts.
The image

shows a similar scene with one

or two differences. If we
calculate the absolute difference between the frames as
shown in

then the regions that have changed become clear. The last
image here has been contrast
-
stretched in order to improve
clarity.

20

Subtraction can also be used to est
imate the temporal
derivative of intensity at each point in a sequence of images.
Such information can be used, for instance, in optical flow
calculations.

Simple subtraction of a constant from an image can be used
to darken an image, although scaling is
normally a better way
of doing this.

It is important to think about whether negative output pixel
values can occur as a result of the subtraction, and how the
software will treat pixels that do have negative values. An
example of what may happen can be see
n in

which is the above lightfield directly subtracted from the text
images. In the implementation of pixel subtraction which was
used, negative values are
wrapped around
[5]

starting from
the maximum value. Since we don't have exactly the same

21

reflectan
ce of the paper when taking the images of the
lightfield and the text, the difference of pixels belonging to
background is either slightly above or slightly below zero.
Therefore the wrapping results in background pixels with
either very small or very high

values, thus making the image
unsuitable for further processing (for example, thresholding).
If we alternatively set all negative values to zero, the image
would become completely black, because subtracting the
pixels in the lightfield from the pixels rep
resenting characters
in the text image yields negative results, as well.

In this application, a suitable way to deal with negative values
is to use absolute differences, as can be seen in

or as a gamma corrected ver
sion in

Thresholding this image yields similar good results as the
earlier example.

22

If negative values are to be avoided then it may be possible to
first add an offset to the first input image. It is also often useful
if possible to convert the pixel val
ue type to something with a
sufficiently large range to avoid overflow, e.g. 32
-
bit integers
or floating point numbers.

Exercises

1.
Take images of your watch at two different times, without
moving it in between, and use subtraction to highlight the
diffe
rence in the display.

2.
Use

to investigate the following method for edge detection. First
apply erosion to the image and then subtract the result from
the original. What is the difference in the edge image if you
use dilation instead of erosion? What
effects have size and
form of the structuring element on the result. How does the
technique perform on grayscale images?

23

References

A. Jain Fundamentals of Digital Image Processing, Prentice
Hall, 1989, pp 240
-

241.

R. Gonzales and R. Woods Digital Imag
e Processing,
-

51, 185
-

187.

R. Boyle and R. Thomas Computer Vision: A First Course,
Blackwell Scientific Publications, 1988, p 35.

A. Marion An Introduction to Image Processing, Chapman and
Hall, 1991, pp 238
-

241.

D. Ver
non Machine Vision, Prentice
-
Hall, 1991, pp 52
-

53.

[1]
Pixel Values

Each of the pixels that represents an image stored inside a
computer has a pixel value which describes how bright that
pixel is, and/or what color it should be. In the simplest case o
f
binary images, the pixel value is a 1
-
bit number indicating
either foreground or background. For a grayscale images, the
pixel value is a single number that represents the brightness

24

of the pixel. The most common pixel format is the byte image,
where thi
s number is stored as an 8
-
bit integer giving a range
of possible values from 0 to 255. Typically zero is taken to be
black, and 255 is taken to be white. Values in between make
up the different shades of gray.

To represent color images, separate red, gre
en and blue
components must be specified for each pixel (assuming an
RGB colorspace), and so the pixel `value' is actually a vector
of three numbers. Often the three different components are
stored as three separate `grayscale' images known as color
planes

(one for each of red, green and blue), which have to
be recombined when displaying or processing.

Multi
-
spectral images can contain even more than three
components for each pixel, and by extension these are stored
in the same kind of way, as a vector pix
el value, or as
separate color planes.

The actual grayscale or color component intensities for each
pixel may not actually be stored explicitly. Often, all that is
stored for each pixel is an index into a colormap in which the
actual intensity or colors c
an be looked up.

Although simple 8
-
bit integers or vectors of 8
-
bit integers are

25

the most common sorts of pixel values used, some image
formats support different types of value, for instance 32
-
bit
signed integers or floating point values. Such values are

extremely useful in image processing as they allow
processing to be carried out on the image where the resulting
pixel values are not necessarily 8
-
bit integers. If this approach
is used then it is usually necessary to set up a colormap
which relates part
icular ranges of pixel values to particular
displayed colors.

[2]
Color Images

It is possible to construct (almost) all visible colors by
combining the three primary colors red, green and blue,
because the human eye has only three different color
receptors
, each of them sensible to one of the three colors.
Different combinations in the stimulation of the receptors
enable the human eye to distinguish approximately 350000
colors. A RGB color image is a multi
-
spectral image with one
band for each color red, gr
een and blue, thus producing a
weighted combination of the three primary colors for each
pixel.

A full 24
-
bit color image contains one 8
-
bit value for each

26

color, thus being able to display
different
colors.

However, it is computationally expensive and

often not
necessary to use the full 24
-
bit image to store the color for
each pixel. Therefore, the color for each pixel is often
encoded in a single byte, resulting in an 8
-
bit color image. The
process of reducing the color representation from 24
-
bits to
8
-
bits, known as color quantization, restricts the number of
possible colors to 256. However, there is normally no visible
difference between a 24
-
color image and the same image
displayed with 8 bits. An 8
-
bit color images are based on
colormaps, which are

look
-
up tables taking the 8
-
bit pixel
value as index and providing an output value for each color.

[3]
RGB and Colorspaces

A color perceived by the human eye can be defined by a
linear combination of the three primary colors red, green and
blue. These thr
ee colors form the basis for the RGB
-
colorspace. Hence, each perceivable color can be defined by
a vector in the three
-
dimensional colorspace. The intensity is
given by the length of the vector, and the actual color by the
two angles describing the orienta
tion of the vector in the

27

colorspace.

The RGB
-
space can also be transformed into other
coordinate systems, which might be more useful for some
applications. One common basis for the color space is IHS. In
this coordinate system, a color is described by it
s intensity,
hue (average wavelength) and saturation (the amount of
white in the color). This color space makes it easier to directly
derive the intensity and color of perceived light and is
therefore more likely to be used by human beings.

[4]
Grayscale
Images

A grayscale (or graylevel) image is simply one in which the
only colors are shades of gray. The reason for differentiating
such images from any other sort of color image is that less
information needs to be provided for each pixel. In fact a
`gray'
color is one in which the red, green and blue
components all have equal intensity in RGB space, and so it
is only necessary to specify a single intensity value for each
pixel, as opposed to the three intensities needed to specify
each pixel in a full color

image.

Often, the grayscale intensity is stored as an 8
-
bit integer
giving 256 possible different shades of gray from black to

28

white. If the levels are evenly spaced then the difference
between successive graylevels is significantly better than the
grayl
evel resolving power of the human eye.

Grayscale images are very common, in part because much of
today's display and image capture hardware can only support
8
-
bit images. In addition, grayscale images are entirely
sufficient for many tasks and so there is

no need to use more
complicated and harder
-
to
-
process color images.

[5]
Wrapping and Saturation

If an image is represented in a byte or integer pixel format,
the maximum pixel value is limited by the number of bits used
for the representation, e.g. the pi
xel values of a 8
-
bit image
are limited to 255.

However, many image processing operations produce output
values which are likely to exceed the given maximum value.
In such cases, we have to decide how to handle this pixel
overflow.

One possibility is to
wrap around the overflowing pixel values.
This means that if a value is greater than the possible
maximum, we subtract the pixel value range so that the value
starts again from the possible minimum value. Figure 1 shows

29

the mapping function for wrapping th
e output values of some
operation into an 8
-
bit format.

Figure 1 Mapping function for wrapping the pixel values of an
8
-
bit image.

Another possibility is to set all overflowing pixels to the
maximum possible values
---

an effect known as saturation.
The
corresponding mapping function for an 8
-
bit image can
be seen in Figure 2.

30

Figure 2 Mapping function for saturating an 8
-
bit image.

If only a few pixels in the image exceed the maximum value it
is often better to apply the latter technique, especially i
f we
use the image for display purposes. However, by setting all
overflowing pixels to the same value we lose an essential
amount of information. In the worst case, when all pixels
exceed the maximum value, this would lead to an image of
constant pixel val
ues. Wrapping around overflowing pixel
retains the differences between values. On the other hand, it
might cause the problem that pixel values passing the
maximum `jump' from the maximum to the minimum value.
Examples for both techniques can be seen in the

worksheets
of various point operators.

If possible, it is easiest to change the image format, for

31

example to float format, so that all pixel values can be
represented. However, we should keep in mind that this
implies an increase in processing time and m
emory.

[6]
Edge Detectors

Edges are places in the image with strong intensity contrast.
Since edges often occur at image locations representing
object boundaries, edge detection is extensively used in
image segmentation when we want to divide the image int
o
areas corresponding to different objects. Representing an
image by its edges has the further advantage that the amount
of data is reduced significantly while retaining most of the
image information.

Since edges consist of mainly high frequencies, we can
, in
theory, detect edges by applying a highpass frequency filter in
the Fourier domain or by convolving the image with an
appropriate kernel in the spatial domain. In practice, edge
detection is performed in the spatial domain, because it is
computational
ly less expensive and often yields better results.

Since edges correspond to strong illumination gradients, we
can highlight them by calculating the derivatives of the image.
This is illustrated for the one
-
dimensional case in Figure 1.

32

Figure 1 1st an
d 2nd derivative of an edge illustrated in one
dimension.

We can see that the position of the edge can be estimated
with the maximum of the 1st derivative or with the zero
-
crossing of the 2nd derivative. Therefore we want to find a
technique to calculate t
he derivative of a two
-
dimensional
image. For a discrete one
-
dimensional function f(i), the first
derivative can be approximated by

Calculating this formula is equivalent to convolving the

33

function with [
-
1 1]. Similarly the 2nd derivative can be
estima
ted by convolving f(i) with [1
-
2 1].

Different edge detection kernels which are based on the
above formula enable us to calculate either the 1st or the 2nd
derivative of a two
-
dimensional image. There are two
common approaches to estimate the 1st derivat
ive in a two
-
dimensional image, Prewitt compass edge detection and

Prewitt compass edge detection involves convolving the
image with a set of (usually 8) kernels, each of which is
sensitive to a different edge orientation. The ker
nel producing
the maximum response at a pixel location determines the
edge magnitude and orientation. Different sets of kernels
might be used: examples include Prewitt, Sobel, Kirsch and
Robinson kernels.

Gradient edge detection is the second and more wid
ely used
technique. Here, the image is convolved with only two kernels,
one estimating the gradient in the x
-
direction, Gx, the other
-
magnitude is then given by

34

and is often approximated with

I
n many implementations, the gradient magnitude is the only
output of a gradient edge detector, however the edge
orientation might be calculated with

The most common kernels used for the gradient edge
detector are the Sobel, Roberts Cross and Prewitt ope
rators.

After having calculated the magnitude of the 1st derivative,
we now have to identify those pixels corresponding to an
edge. The easiest way is to threshold the gradient image,
assuming that all pixels having a local gradient above the
threshold mu
st represent an edge. An alternative technique is
to look for local maxima in the gradient image, thus producing
one pixel wide edges. A more sophisticated technique is used
by the Canny edge detector. It first applies a gradient edge
detector to the image

and then finds the edge pixels using
non
-
maximal suppression and hysteresis tracking.

An operator based on the 2nd derivative of an image is the

35

Marr edge detector, also known as zero crossing detector.
Here, the 2nd derivative is calculated using a Lapl
acian of
Gaussian (LoG) filter. The Laplacian has the advantage that it
is an isotropic measure of the 2nd derivative of an image, i.e.
the edge magnitude is obtained independently from the edge
orientation by convolving the image with only one kernel. The

edge positions are then given by the zero
-
crossings in the
LoG image. The scale of the edges which are to be detected
can be controlled by changing the variance of the Gaussian.

A general problem for edge detection is its sensitivity to noise,
the reason

being that calculating the derivative in the spatial
domain corresponds to accentuating high frequencies and
hence magnifying noise. This problem is addressed in the
Canny and Marr operators by convolving the image with a
smoothing operator (Gaussian) bef
ore calculating the
derivative.

[7]

A mask is a binary image consisting of zero
-

and non
-
zero
values. If a mask is applied to another binary or to a grayscale
image of the same size, all pixels which are zero in the mask
are set to zero in the outp
ut image. All others remain

36

unchanged.

Masking can be implemented either using pixel multiplication
or logical AND, the latter in general being faster.

Masking is often used to restrict a point or arithmetic operator
to an area defined by the mask. We ca
n, for example,
accomplish this by first masking the desired area in the input
image and processing it with the operator, then masking the
original input image with the inverted mask to obtain the
unprocessed area of the image and finally recombining the
t
wo partial images using image addition. An example can be
seen in the worksheet on the logical AND operator. In some
image processing packages, a mask can directly be defined
as an optional input to a point operator, so that automatically
the operator is o
nly applied to the pixels defined by the mask

[8]
Look
-
up Tables and Colormaps

Look
-
Up Tables or LUTs are fundamental to many aspects of
image processing. An LUT is simply a table of cross
-
references linking index numbers to output values. The most
common
use is to determine the colors and intensity values
with which a particular image will be displayed, and in this
context the LUT is often called simply a colormap.

37

The idea behind the colormap is that instead of storing a
definite color for each pixel in
an image, for instance in 24
-
bit
RGB format, each pixel's value is instead treated as an index
number into the colormap. When the image is to be displayed
or otherwise processed, the colormap is used to look up the
actual colors corresponding to each index

number. Typically,
the output values stored in the LUT would be RGB color
values.

There are two main advantages to doing things this way.
Firstly, the index number can be made to use fewer bits than
the output value in order to save storage space. For in
stance
an 8
-
bit index number can be used to look up a 24
-
bit RGB
color value in the LUT. Since only the 8
-
bit index number
needs to be stored for each pixel, such 8
-
bit color images
take up less space than a full 24
-
bit image of the same size.
Of course th
e image can only contain 256 different colors (the
number of entries in an 8
-
bit LUT), but this is sufficient for
many applications and usually the observable image

Secondly the use of a color table allows the user to
experiment easi
ly with different color labeling schemes for an
image.

38

One disadvantage of using a colormap is that it introduces
additional complexity into an image format. It is usually
necessary for each image to carry around its own colormap,
and this LUT must be con
tinually consulted whenever the
image is displayed or processed.

Another problem is that in order to convert from a full color
image to (say) an 8
-
bit color image using a color image, it is
usually necessary to throw away many of the original colors, a
pr
ocess known as color quantization. This process is lossy,
and hence the image quality is degraded during the
quantization process. Additionally, when performing further
image processing on such images, it is frequently necessary
to generate a new colormap
for the new images, which
involves further color quantization, and hence further image