Introduction to Algorithms
Rabie A. Ramadan
rabie@rabieramadan.org
http://www. rabieramadan.org
5
Some of the sides are exported from different sources to clarify the
topic
Chapter 4
Divide

and

Conquer
Copyright © 2007 Pearson Addison

Wesley. All rights reserved.
Acknowledgment :
Some of the slides are based
on a tutorial made by
Kruse and Ryba
Agenda
Binary Search
Tree Traversal
VLSI
Layout
Powering a number
Multiplying two large Integers
Matrix Multiplication
Closest

Pair Problem
The Problem
Find an element in a sorted array?
Binary
Search
Very efficient algorithm for searching in
sorted array
:
K
vs
A[0] . . . A[
m
] . . . A[
n

1]
If
K =
A[
m
], stop (successful search); otherwise, continue
searching by the same method in A[0..
m

1] if
K <
A[
m
]
and in A[
m
+1..
n

1] if
K >
A[
m
]
l
0;
r
n

1
while
l
r
do
m
(
l
+
r
)/2
if
K =
A[
m
] return
m
else if
K <
A[
m
]
r
m

1
else
l
m
+1
return

1
Analysis
of Binary Search
Time efficiency
Divide

and

conquer algorithm:
•
Divide: Check middle element.
•
Conquer: Recursively search
1 sub

array.
•
Combine: Trivial.
T(n) =T(n/2) +
θ
(1) =
Optimal for searching a sorted array
Limitations: must be a sorted array (not linked list)
1 sub

problem of
size n/2
Division +
Combination
Trees
A
tree
is a collection of nodes
•
The collection can be empty
•
(recursive definition) If not empty, a tree consists of a
distinguished node r (the
root
), and zero or more nonempty
subtrees
T
1
, T
2
, ....,
T
k
, each of whose roots are connected by a
directed
edge
from r
Importance of Trees
UNIX Directory
Some Terminologies
Child
and
parent
•
Every node except the root has one parent
•
A node can have an arbitrary number of children
Leaves
•
Nodes with no children
Sibling
•
nodes with same parent
Binary
Trees
A tree in which no node can have more
than two children
The depth of an “average” binary tree is
considerably smaller than N, even
though in the worst case, the depth can
be as large as N
–
1.
Example: Expression Trees
Tree traversal
Used to print out the data in a tree in a
certain order
Pre

order traversal
•
Print the data at the root
•
Recursively print out all data in the left subtree
•
Recursively print out all data in the right subtree
Preorder, Postorder and Inorder
Preorder traversal
•
node, left, right
•
prefix expression
•
++a*
bc
*+*
defg
Preorder, Postorder and Inorder
Postorder
traversal
•
left, right, node
•
postfix expression
•
abc
*+de*
f+g
*+
Inorder
traversal
•
left, node, right.
•
infix expression
•
a+b
*
c+d
*
e+f
*g
Efficiency:
Θ
(
n
). Why?
Each node is visited/printed once.
The Problem
A
set
•
An associative container data structure that is available as part of the C++
Standard Library (STL), and contains a sorted set of unique objects.
•
Sets are guaranteed to perform operations of insertion, deletion, and
testing whether an element is in it, in logarithmic time

O(log
n
).
What is the best method to represent a
Set
to satisfy
the
logarithmic
time

O(log
n
)?
Binary Search Trees
Stores keys in the nodes in a way so that searching, insertion and
deletion can be done efficiently.
Binary
search
tree property
•
For every node X, all the keys in its left
subtree
are smaller than the key
value in X, and all the keys in its right
subtree
are larger than the key
value in X
Which one is a binary search tree?
A binary search tree
Not a binary search tree
Binary search trees
Average depth of a node is O(log N); maximum depth of a node is O(N)
Representation Makes a Difference
Searching BST
If we are searching for 15, then we are done.
If we are searching for a key < 15, then we should search in the left
subtree.
If we are searching for a key > 15, then we should search in the right
subtree.
BST Analysis
Time Complexity:
Average Case = ?
Worst Case = ?
a balanced one guarantees O(
logn
)
)
(log
n
)
(
n
O
We have to traverse all of the nodes
The Problem
22
H(n) = and W(n) =
H

tree embedding
The Problem
Consider the problem of multiplying two
(large)
n

digit integers represented by arrays of
their digits such as:
A = 12345678901357986429
B = 87654321284820912836
Multiplication of Large Integers
Consider the problem of multiplying two (large)
n

digit integers represented
by arrays of their digits such as:
A = 12345678901357986429 B = 87654321284820912836
The grade

school algorithm:
a
1
a
2
…
a
n
b
1
b
2
…
b
n
(
d
10
)
d
11
d
12
…
d
1
n
(
d
20
)
d
21
d
22
…
d
2
n
… … … … … … …
(
d
n
0
)
d
n
1
d
n
2
…
d
nn
Efficiency:
Θ
(
n
2
) single

digit multiplications
First Divide

and

Conquer Algorithm
A small example: A
B where A = 2135 and B = 4014
A = (21∙10
2
+ 35), B = (40 ∙10
2
+ 14)
So, A
B = (21 ∙10
2
+ 35)
(40 ∙10
2
+ 14)
= 21
40 ∙10
4
+ (21
14 + 35
40) ∙10
2
+ 35
14
In general, if A = A
1
A
2
and B = B
1
B
2
(where A and B are
n

digit,
A
1
, A
2
, B
1
,
B
2
are
n/
2

digit numbers),
A
B = A
1
B
1
∙10
n
+ (A
1
B
2
+ A
2
B
1
) ∙10
n/
2
+ A
2
B
2
Recurrence for the number of one

digit multiplications M(
n
):
M(
n
) = 4M(
n
/2), M(1) = 1
Solution: M(
n
) =
n
2
Second Divide

and

Conquer Algorithm
A
B = A
1
B
1
∙10
n
+ (A
1
B
2
+ A
2
B
1
) ∙10
n/
2
+ A
2
B
2
The idea is to decrease the number of multiplications from 4 to 3:
(A
1
+ A
2
)
(B
1
+ B
2
) = A
1
B
1
+ (A
1
B
2
+ A
2
B
1
) + A
2
B
2,
I.e., (A
1
B
2
+ A
2
B
1
) = (A
1
+ A
2
)
(B
1
+ B
2
)

A
1
B
1

A
2
B
2,
which requires only 3 multiplications at the expense of (4

1) extra add/sub.
Recurrence for the number of multiplications M(
n
):
M(
n
) = 3
M
(
n
/2), M(1) = 1
Solution by backward substitution : M(
n
) ≈
n
1.585
Try yourself
Example of Large

Integer Multiplication
2135
4014
= (21*10^2 + 35) * (40*10^2 + 14)
= (21
*
40)*10^4 + c1*10^2 + 35
*
14
where c1 = (21+35)
*
(40+14)

21
*
40

35
*
14, and
21*40 = (2*10 + 1) * (4*10 + 0)
= (2
*
4)*10^2 + c2*10 + 1
*
0
where c2 = (2+1)
*
(4+0)

2
*
4

1
*
0, etc.
This process requires 9 digit multiplications as opposed to 16.
The Problem
Matrix Multiplication
c
00
c
01
a
00
a
01
b
00
b
01
= *
c
10
c
11
a
10
a
11
b
10
b
11
Brute

force algorithm
Conventional Matrix Multiplication
Brute

force algorithm
c
00
c
01
a
00
a
01
b
00
b
01
= *
c
10
c
11
a
10
a
11
b
10
b
11
a
00
* b
00
+ a
01
* b
10
a
00
* b
01
+ a
01
* b
11
=
a
10
* b
00
+ a
11
* b
10
a
10
* b
01
+ a
11
* b
11
8 multiplications
4 additions
Efficiency class in general:
⡮
3
)
Strassen’s Matrix Multiplication
Strassen’s algorithm for two 2x2 matrices (1969):
c
00
c
01
a
00
a
01
b
00
b
01
= *
c
10
c
11
a
10
a
11
b
10
b
11
m
1
+ m
4

m
5
+ m
7
m
3
+ m
5
=
m
2
+ m
4
m
1
+ m
3

m
2
+ m
6
m
1
= (a
00
+ a
11
)
*
(b
00
+ b
11
)
m
2
= (a
10
+ a
11
)
*
b
00
m
3
= a
00
*
(b
01

b
11
)
m
4
= a
11
*
(b
10

b
00
)
m
5
= (a
00
+ a
01
)
*
b
11
m
6
= (a
10

a
00
)
*
(b
00
+ b
01
)
m
7
= (a
01

a
11
)
*
(b
10
+ b
11
)
7 multiplications
18 additions
Strassen’s Matrix Multiplication
Strassen
observed [1969] that the product of two matrices can be computed in
general as follows: (n/2) by ( n/2)
C
00
C
01
A
00
A
01
B
00
B
01
= *
C
10
C
11
A
10
A
11
B
10
B
11
M
1
+ M
4

M
5
+ M
7
M
3
+ M
5
=
M
2
+ M
4
M
1
+ M
3

M
2
+ M
6
Formulas for Strassen’s Algorithm
M
1
= (A
00
+ A
11
)
(B
00
+ B
11
)
M
2
= (A
10
+ A
11
)
B
00
M
3
= A
00
(B
01

B
11
)
M
4
= A
11
(B
10

B
00
)
M
5
= (A
00
+ A
01
)
B
11
M
6
= (A
10

A
00
)
(B
00
+ B
01
)
M
7
= (A
01

A
11
)
(B
10
+ B
11
)
Analysis of Strassen’s Algorithm
If
n
is not a power of 2, matrices can be padded with zeros.

Therefore B11 could be a matrix
Number of multiplications:
M(
n
) = 7M(
n
/2), M(1) = 1
Solution: M(
n
)
=
≈
n
2.807
vs.
n
3
of brute

force alg.
Algorithms with better asymptotic efficiency are known but they
are even more complex and not used in practice.
The Problem
Powering a number
Powering a number
36
Closest

Pair Problem by Divide

and

Conquer
Step 0 :
Sort the points by x O(
nlogn
) (list 1) then sort based on y (list 2)
Step 1:
Divide the points given into two subsets S
1
and S
2
by a
vertical line
x
=
c
so that half the points lie to the left or on
the line and half the points lie to the right or on the line.
Closest Pair by Divide

and

Conquer
(cont.)
Step 2 Find recursively the closest pairs for the left and right
subsets.
Step 3 Set
d
= min{
d
1
,
d
2
}
We can limit our attention to the points in the symmetric
vertical strip of width 2
d
as possible closest pair. Let C
1
and C
2
be the subsets of points in the left subset S
1
and of
the right subset S
2
, respectively, that lie in this vertical
strip. The points in C
1
and C
2
are stored in increasing
order of their
y
coordinates, taken from the second list.
Step 4 For every point
P
(
x
,
y
) in C
1
, we inspect points in
C
2
that may be closer to
P
than
d.
Closest Pair by Divide

and

Conquer:
Worst Case
The worst case scenario is depicted below: (all points within the strip)
Efficiency of the Closest

Pair Algorithm
Running time of the algorithm (without sorting) is:
T(
n
) = 2T(
n
/2) + M(
n
), where M(
n
)
Θ
(
n
)
(merging the lists)
By the Master Theorem (with
a
= 2,
b
= 2,
d
= 1)
T(
n
)
Θ
(
n
log
n
)
So the total time is
Θ
(
n
log
n
).
41
42
Our Lab Today
Implement closest pair algorithms :
The brute force algorithm that computes the
distance between each pair of points and then
returns a closest pair of points.
The Divide

and

Conquer Algorithm
Comments 0
Log in to post a comment