Electronics - Devices

Nov 27, 2013 (4 years and 7 months ago)

112 views

Introduction to Algorithms

5

Some of the sides are exported from different sources to clarify the
topic

Chapter 4

Divide
-
and
-
Conquer

-

Acknowledgment :

Some of the slides are based

Kruse and Ryba

Agenda

Binary Search

Tree Traversal

VLSI
Layout

Powering a number

Multiplying two large Integers

Matrix Multiplication

Closest
-
Pair Problem

The Problem

Find an element in a sorted array?

Binary

Search

Very efficient algorithm for searching in
sorted array
:

K

vs

A[0] . . . A[
m
] . . . A[
n
-
1]

If
K =
A[
m
], stop (successful search); otherwise, continue

searching by the same method in A[0..
m
-
1] if
K <
A[
m
]

and in A[
m
+1..
n
-
1] if
K >
A[
m
]

l

0;
r

n
-
1

while
l

r

do

m


(
l
+
r
)/2

if
K =
A[
m
] return
m

else if
K <
A[
m
]
r

m
-
1

else
l

m
+1

return
-
1

Analysis

of Binary Search

Time efficiency

Divide
-
and
-
conquer algorithm:

Divide: Check middle element.

Conquer: Recursively search
1 sub
-
array.

Combine: Trivial.

T(n) =T(n/2) +
θ
(1) =

Optimal for searching a sorted array

Limitations: must be a sorted array (not linked list)

1 sub
-
problem of
size n/2

Division +
Combination

Trees

A
tree

is a collection of nodes

The collection can be empty

(recursive definition) If not empty, a tree consists of a
distinguished node r (the
root
), and zero or more nonempty
subtrees

T
1
, T
2
, ....,
T
k
, each of whose roots are connected by a
directed
edge

from r

Importance of Trees

UNIX Directory

Some Terminologies

Child

and
parent

Every node except the root has one parent

A node can have an arbitrary number of children

Leaves

Nodes with no children

Sibling

nodes with same parent

Binary

Trees

A tree in which no node can have more
than two children

The depth of an “average” binary tree is
considerably smaller than N, even
though in the worst case, the depth can
be as large as N

1.

Example: Expression Trees

Tree traversal

Used to print out the data in a tree in a
certain order

Pre
-
order traversal

Print the data at the root

Recursively print out all data in the left subtree

Recursively print out all data in the right subtree

Preorder, Postorder and Inorder

Preorder traversal

node, left, right

prefix expression

++a*
bc
*+*
defg

Preorder, Postorder and Inorder

Postorder

traversal

left, right, node

postfix expression

abc
*+de*
f+g
*+

Inorder

traversal

left, node, right.

infix expression

a+b
*
c+d
*
e+f
*g

Efficiency:
Θ
(
n
). Why?

Each node is visited/printed once.

The Problem

A
set

An associative container data structure that is available as part of the C++
Standard Library (STL), and contains a sorted set of unique objects.

Sets are guaranteed to perform operations of insertion, deletion, and
testing whether an element is in it, in logarithmic time
-

O(log
n
).

What is the best method to represent a

Set
to satisfy
the
logarithmic
time
-

O(log
n
)?

Binary Search Trees

Stores keys in the nodes in a way so that searching, insertion and
deletion can be done efficiently.

Binary

search

tree property

For every node X, all the keys in its left
subtree

are smaller than the key
value in X, and all the keys in its right
subtree

are larger than the key
value in X

Which one is a binary search tree?

A binary search tree

Not a binary search tree

Binary search trees

Average depth of a node is O(log N); maximum depth of a node is O(N)

Representation Makes a Difference

Searching BST

If we are searching for 15, then we are done.

If we are searching for a key < 15, then we should search in the left
subtree.

If we are searching for a key > 15, then we should search in the right
subtree.

BST Analysis

Time Complexity:

Average Case = ?

Worst Case = ?

a balanced one guarantees O(
logn
)

)
(log
n

)
(
n
O
We have to traverse all of the nodes

The Problem

22

H(n) = and W(n) =

H
-
tree embedding

The Problem

Consider the problem of multiplying two
(large)
n
-
digit integers represented by arrays of
their digits such as:

A = 12345678901357986429

B = 87654321284820912836

Multiplication of Large Integers

Consider the problem of multiplying two (large)
n
-
digit integers represented
by arrays of their digits such as:

A = 12345678901357986429 B = 87654321284820912836

-
school algorithm:

a
1
a
2

a
n

b
1
b
2

b
n

(
d
10
)

d
11
d
12

d
1
n

(
d
20
)

d
21
d
22

d
2
n

… … … … … … …

(
d
n
0
)

d
n
1
d
n
2

d
nn

Efficiency:
Θ
(
n
2
) single
-
digit multiplications

First Divide
-
and
-
Conquer Algorithm

A small example: A

B where A = 2135 and B = 4014

A = (21∙10
2

+ 35), B = (40 ∙10
2

+ 14)

So, A

B = (21 ∙10
2

+ 35)

(40 ∙10
2

+ 14)

= 21

40 ∙10
4
+ (21

14 + 35

40) ∙10
2

+ 35

14

In general, if A = A
1
A
2
and B = B
1
B
2
(where A and B are
n
-
digit,

A
1
, A
2
, B
1
,

B
2
are
n/
2
-
digit numbers),

A

B = A
1

B
1
∙10
n

+ (A
1

B
2
+ A
2

B
1
) ∙10
n/
2
+ A
2

B
2

Recurrence for the number of one
-
digit multiplications M(
n
):

M(
n
) = 4M(
n
/2), M(1) = 1

Solution: M(
n
) =
n
2

Second Divide
-
and
-
Conquer Algorithm

A

B = A
1

B
1
∙10
n

+ (A
1

B
2
+ A
2

B
1
) ∙10
n/
2
+ A
2

B
2

The idea is to decrease the number of multiplications from 4 to 3:

(A
1

+ A
2

)

(B
1

+ B
2

) = A
1

B
1

+ (A
1

B
2
+ A
2

B
1
) + A
2

B
2,

I.e., (A
1

B
2
+ A
2

B
1
) = (A
1

+ A
2

)

(B
1

+ B
2

)
-

A
1

B
1

-

A
2

B
2,

which requires only 3 multiplications at the expense of (4
-

Recurrence for the number of multiplications M(
n
):

M(
n
) = 3
M
(
n
/2), M(1) = 1

Solution by backward substitution : M(
n
) ≈
n
1.585

Try yourself

Example of Large
-
Integer Multiplication

2135

4014

= (21*10^2 + 35) * (40*10^2 + 14)

= (21
*
40)*10^4 + c1*10^2 + 35
*
14

where c1 = (21+35)
*
(40+14)
-

21
*
40
-

35
*
14, and

21*40 = (2*10 + 1) * (4*10 + 0)

= (2
*
4)*10^2 + c2*10 + 1
*
0

where c2 = (2+1)
*
(4+0)
-

2
*
4
-

1
*
0, etc.

This process requires 9 digit multiplications as opposed to 16.

The Problem

Matrix Multiplication

c
00
c
01

a
00

a
01

b
00

b
01

= *

c
10
c
11

a
10

a
11

b
10

b
11

Brute
-
force algorithm

Conventional Matrix Multiplication

Brute
-
force algorithm

c
00
c
01

a
00

a
01

b
00

b
01

= *

c
10
c
11

a
10

a
11

b
10

b
11

a
00
* b
00

+ a
01
* b
10

a
00
* b
01

+ a
01
* b
11

=

a
10
* b
00

+ a
11
* b
10

a
10
* b
01

+ a
11
* b
11

8 multiplications

Efficiency class in general:

3
)

Strassen’s Matrix Multiplication

Strassen’s algorithm for two 2x2 matrices (1969):

c
00
c
01

a
00

a
01

b
00

b
01

= *

c
10
c
11

a
10

a
11

b
10

b
11

m
1

+ m
4

-

m
5
+ m
7

m
3
+ m
5

=

m
2

+ m
4
m
1

+ m
3

-

m
2
+ m
6

m
1

= (a
00

+ a
11
)
*

(b
00

+ b
11
)

m
2

= (a
10

+ a
11
)
*

b
00

m
3

= a
00

*

(b
01

-

b
11
)

m
4

= a
11

*

(b
10

-

b
00
)

m
5

= (a
00

+ a
01
)
*

b
11

m
6

= (a
10

-

a
00
)
*

(b
00

+ b
01
)

m
7

= (a
01

-

a
11
)
*

(b
10

+ b
11
)

7 multiplications

Strassen’s Matrix Multiplication

Strassen

observed [1969] that the product of two matrices can be computed in
general as follows: (n/2) by ( n/2)

C
00
C
01

A
00

A
01

B
00

B
01

= *

C
10
C
11

A
10

A
11

B
10

B
11

M
1

+ M
4

-

M
5
+ M
7

M
3
+ M
5

=

M
2

+ M
4
M
1

+ M
3

-

M
2
+ M
6

Formulas for Strassen’s Algorithm

M
1

= (A
00

+ A
11
)

(B
00

+ B
11
)

M
2

= (A
10

+ A
11
)

B
00

M
3

= A
00

(B
01

-

B
11
)

M
4

= A
11

(B
10

-

B
00
)

M
5

= (A
00

+ A
01
)

B
11

M
6

= (A
10

-

A
00
)

(B
00

+ B
01
)

M
7

= (A
01

-

A
11
)

(B
10

+ B
11
)

Analysis of Strassen’s Algorithm

If
n

is not a power of 2, matrices can be padded with zeros.

-

Therefore B11 could be a matrix

Number of multiplications:

M(
n
) = 7M(
n
/2), M(1) = 1

Solution: M(
n
)
=

n
2.807
vs.
n
3
of brute
-
force alg.

Algorithms with better asymptotic efficiency are known but they

are even more complex and not used in practice.

The Problem

Powering a number

Powering a number

36

Closest
-
Pair Problem by Divide
-
and
-
Conquer

Step 0 :
Sort the points by x O(
nlogn
) (list 1) then sort based on y (list 2)

Step 1:
Divide the points given into two subsets S
1

and S
2

by a

vertical line
x

=
c

so that half the points lie to the left or on

the line and half the points lie to the right or on the line.

Closest Pair by Divide
-
and
-
Conquer
(cont.)

Step 2 Find recursively the closest pairs for the left and right

subsets.

Step 3 Set
d

= min{
d
1
,
d
2
}

We can limit our attention to the points in the symmetric

vertical strip of width 2
d

as possible closest pair. Let C
1

and C
2

be the subsets of points in the left subset S
1

and of

the right subset S
2
, respectively, that lie in this vertical

strip. The points in C
1
and C
2
are stored in increasing

order of their
y

coordinates, taken from the second list.

Step 4 For every point
P
(
x
,
y
) in C
1
, we inspect points in

C
2

that may be closer to
P

than
d.

Closest Pair by Divide
-
and
-
Conquer:
Worst Case

The worst case scenario is depicted below: (all points within the strip)

Efficiency of the Closest
-
Pair Algorithm

Running time of the algorithm (without sorting) is:

T(
n
) = 2T(
n
/2) + M(
n
), where M(
n
)

Θ
(
n
)
(merging the lists)

By the Master Theorem (with
a

= 2,
b

= 2,
d

= 1)

T(
n
)

Θ
(
n

log
n
)

So the total time is
Θ
(
n

log
n
).

41

42

Our Lab Today

Implement closest pair algorithms :

The brute force algorithm that computes the
distance between each pair of points and then
returns a closest pair of points.

The Divide
-
and
-
Conquer Algorithm