Rabie A. Ramadan

stingymilitaryΗλεκτρονική - Συσκευές

27 Νοε 2013 (πριν από 4 χρόνια και 1 μήνα)

101 εμφανίσεις

Introduction to Algorithms


Rabie A. Ramadan

rabie@rabieramadan.org

http://www. rabieramadan.org

5


Some of the sides are exported from different sources to clarify the
topic

Chapter 4


Divide
-
and
-
Conquer

Copyright © 2007 Pearson Addison
-
Wesley. All rights reserved.

Acknowledgment :

Some of the slides are based

on a tutorial made by


Kruse and Ryba

Agenda


Binary Search


Tree Traversal


VLSI
Layout


Powering a number


Multiplying two large Integers


Matrix Multiplication


Closest
-
Pair Problem

The Problem


Find an element in a sorted array?


Binary

Search

Very efficient algorithm for searching in
sorted array
:


K






vs




A[0] . . . A[
m
] . . . A[
n
-
1]

If
K =
A[
m
], stop (successful search); otherwise, continue

searching by the same method in A[0..
m
-
1] if
K <
A[
m
]

and in A[
m
+1..
n
-
1] if
K >
A[
m
]


l


0;
r



n
-
1

while
l



r

do


m




(
l
+
r
)/2



if
K =
A[
m
] return
m


else if
K <
A[
m
]
r


m
-
1


else
l


m
+1

return
-
1


Analysis

of Binary Search


Time efficiency


Divide
-
and
-
conquer algorithm:


Divide: Check middle element.


Conquer: Recursively search
1 sub
-
array.


Combine: Trivial.


T(n) =T(n/2) +
θ
(1) =




Optimal for searching a sorted array



Limitations: must be a sorted array (not linked list)



1 sub
-
problem of
size n/2

Division +
Combination

Trees


A
tree

is a collection of nodes


The collection can be empty


(recursive definition) If not empty, a tree consists of a
distinguished node r (the
root
), and zero or more nonempty
subtrees

T
1
, T
2
, ....,
T
k
, each of whose roots are connected by a
directed
edge

from r


Importance of Trees

UNIX Directory

Some Terminologies


Child

and
parent


Every node except the root has one parent



A node can have an arbitrary number of children


Leaves


Nodes with no children


Sibling


nodes with same parent


Binary

Trees


A tree in which no node can have more
than two children





The depth of an “average” binary tree is
considerably smaller than N, even
though in the worst case, the depth can
be as large as N


1.


Example: Expression Trees

Tree traversal


Used to print out the data in a tree in a
certain order



Pre
-
order traversal


Print the data at the root


Recursively print out all data in the left subtree


Recursively print out all data in the right subtree


Preorder, Postorder and Inorder


Preorder traversal


node, left, right


prefix expression


++a*
bc
*+*
defg

Preorder, Postorder and Inorder


Postorder

traversal


left, right, node


postfix expression


abc
*+de*
f+g
*+



Inorder

traversal


left, node, right.


infix expression


a+b
*
c+d
*
e+f
*g


Efficiency:
Θ
(
n
). Why?

Each node is visited/printed once.

The Problem


A
set



An associative container data structure that is available as part of the C++
Standard Library (STL), and contains a sorted set of unique objects.



Sets are guaranteed to perform operations of insertion, deletion, and
testing whether an element is in it, in logarithmic time
-

O(log
n
).




What is the best method to represent a

Set
to satisfy
the
logarithmic
time
-

O(log
n
)?


Binary Search Trees


Stores keys in the nodes in a way so that searching, insertion and
deletion can be done efficiently.


Binary

search

tree property


For every node X, all the keys in its left
subtree

are smaller than the key
value in X, and all the keys in its right
subtree

are larger than the key
value in X



Which one is a binary search tree?

A binary search tree

Not a binary search tree

Binary search trees

Average depth of a node is O(log N); maximum depth of a node is O(N)

Representation Makes a Difference

Searching BST


If we are searching for 15, then we are done.


If we are searching for a key < 15, then we should search in the left
subtree.


If we are searching for a key > 15, then we should search in the right
subtree.


BST Analysis

Time Complexity:



Average Case = ?




Worst Case = ?





a balanced one guarantees O(
logn
)

)
(log
n

)
(
n
O
We have to traverse all of the nodes

The Problem

22

H(n) = and W(n) =


H
-
tree embedding



The Problem

Consider the problem of multiplying two
(large)
n
-
digit integers represented by arrays of
their digits such as:


A = 12345678901357986429


B = 87654321284820912836



Multiplication of Large Integers

Consider the problem of multiplying two (large)
n
-
digit integers represented
by arrays of their digits such as:


A = 12345678901357986429 B = 87654321284820912836


The grade
-
school algorithm:



a
1
a
2

a
n




b
1
b
2

b
n




(
d
10
)

d
11
d
12

d
1
n


(
d
20
)

d
21
d
22

d
2
n


… … … … … … …

(
d
n
0
)

d
n
1
d
n
2

d
nn




Efficiency:
Θ
(
n
2
) single
-
digit multiplications

First Divide
-
and
-
Conquer Algorithm

A small example: A



B where A = 2135 and B = 4014

A = (21∙10
2

+ 35), B = (40 ∙10
2

+ 14)

So, A



B = (21 ∙10
2

+ 35)


(40 ∙10
2

+ 14)


= 21



40 ∙10
4
+ (21



14 + 35


40) ∙10
2

+ 35


14


In general, if A = A
1
A
2
and B = B
1
B
2
(where A and B are
n
-
digit,

A
1
, A
2
, B
1
,

B
2
are
n/
2
-
digit numbers),

A


B = A
1


B
1
∙10
n

+ (A
1


B
2
+ A
2


B
1
) ∙10
n/
2
+ A
2



B
2


Recurrence for the number of one
-
digit multiplications M(
n
):


M(
n
) = 4M(
n
/2), M(1) = 1

Solution: M(
n
) =
n
2

Second Divide
-
and
-
Conquer Algorithm

A


B = A
1


B
1
∙10
n

+ (A
1


B
2
+ A
2


B
1
) ∙10
n/
2
+ A
2


B
2


The idea is to decrease the number of multiplications from 4 to 3:



(A
1

+ A
2

)


(B
1

+ B
2

) = A
1


B
1

+ (A
1


B
2
+ A
2


B
1
) + A
2


B
2,


I.e., (A
1


B
2
+ A
2


B
1
) = (A
1

+ A
2

)


(B
1

+ B
2

)
-

A
1


B
1

-

A
2


B
2,


which requires only 3 multiplications at the expense of (4
-
1) extra add/sub.


Recurrence for the number of multiplications M(
n
):


M(
n
) = 3
M
(
n
/2), M(1) = 1

Solution by backward substitution : M(
n
) ≈
n
1.585


Try yourself

Example of Large
-
Integer Multiplication

2135


4014

= (21*10^2 + 35) * (40*10^2 + 14)

= (21
*
40)*10^4 + c1*10^2 + 35
*
14

where c1 = (21+35)
*
(40+14)
-

21
*
40
-

35
*
14, and

21*40 = (2*10 + 1) * (4*10 + 0)


= (2
*
4)*10^2 + c2*10 + 1
*
0

where c2 = (2+1)
*
(4+0)
-

2
*
4
-

1
*
0, etc.


This process requires 9 digit multiplications as opposed to 16.


The Problem

Matrix Multiplication


c
00
c
01

a
00

a
01

b
00

b
01


= *

c
10
c
11

a
10

a
11

b
10

b
11






Brute
-
force algorithm

Conventional Matrix Multiplication


Brute
-
force algorithm

c
00
c
01

a
00

a
01

b
00

b
01


= *

c
10
c
11

a
10

a
11

b
10

b
11





a
00
* b
00

+ a
01
* b
10



a
00
* b
01

+ a
01
* b
11



=


a
10
* b
00

+ a
11
* b
10



a
10
* b
01

+ a
11
* b
11


8 multiplications

4 additions

Efficiency class in general:



3
)

Strassen’s Matrix Multiplication


Strassen’s algorithm for two 2x2 matrices (1969):

c
00
c
01

a
00

a
01

b
00

b
01


= *

c
10
c
11

a
10

a
11

b
10

b
11






m
1

+ m
4

-

m
5
+ m
7

m
3
+ m
5



=


m
2

+ m
4
m
1

+ m
3

-

m
2
+ m
6



m
1

= (a
00

+ a
11
)
*

(b
00

+ b
11
)


m
2

= (a
10

+ a
11
)
*

b
00


m
3

= a
00

*

(b
01

-

b
11
)


m
4

= a
11

*

(b
10

-

b
00
)


m
5

= (a
00

+ a
01
)
*

b
11


m
6

= (a
10

-

a
00
)
*

(b
00

+ b
01
)


m
7

= (a
01

-

a
11
)
*

(b
10

+ b
11
)




7 multiplications

18 additions

Strassen’s Matrix Multiplication


Strassen

observed [1969] that the product of two matrices can be computed in
general as follows: (n/2) by ( n/2)


C
00
C
01

A
00

A
01

B
00

B
01


= *

C
10
C
11

A
10

A
11

B
10

B
11




M
1

+ M
4

-

M
5
+ M
7

M
3
+ M
5



=


M
2

+ M
4
M
1

+ M
3

-

M
2
+ M
6



Formulas for Strassen’s Algorithm

M
1

= (A
00

+ A
11
)


(B
00

+ B
11
)


M
2

= (A
10

+ A
11
)


B
00


M
3

= A
00



(B
01

-

B
11
)


M
4

= A
11



(B
10

-

B
00
)


M
5

= (A
00

+ A
01
)


B
11


M
6

= (A
10

-

A
00
)


(B
00

+ B
01
)


M
7

= (A
01

-

A
11
)


(B
10

+ B
11
)

Analysis of Strassen’s Algorithm

If
n

is not a power of 2, matrices can be padded with zeros.

-

Therefore B11 could be a matrix


Number of multiplications:


M(
n
) = 7M(
n
/2), M(1) = 1


Solution: M(
n
)
=

n
2.807
vs.
n
3
of brute
-
force alg.


Algorithms with better asymptotic efficiency are known but they

are even more complex and not used in practice.


The Problem


Powering a number


Powering a number

36

Closest
-
Pair Problem by Divide
-
and
-
Conquer

Step 0 :
Sort the points by x O(
nlogn
) (list 1) then sort based on y (list 2)


Step 1:
Divide the points given into two subsets S
1

and S
2

by a

vertical line
x

=
c

so that half the points lie to the left or on

the line and half the points lie to the right or on the line.


Closest Pair by Divide
-
and
-
Conquer
(cont.)

Step 2 Find recursively the closest pairs for the left and right


subsets.


Step 3 Set
d

= min{
d
1
,
d
2
}


We can limit our attention to the points in the symmetric


vertical strip of width 2
d

as possible closest pair. Let C
1



and C
2

be the subsets of points in the left subset S
1

and of


the right subset S
2
, respectively, that lie in this vertical


strip. The points in C
1
and C
2
are stored in increasing


order of their
y

coordinates, taken from the second list.


Step 4 For every point
P
(
x
,
y
) in C
1
, we inspect points in


C
2

that may be closer to
P

than
d.

Closest Pair by Divide
-
and
-
Conquer:
Worst Case

The worst case scenario is depicted below: (all points within the strip)

Efficiency of the Closest
-
Pair Algorithm


Running time of the algorithm (without sorting) is:



T(
n
) = 2T(
n
/2) + M(
n
), where M(
n
)


Θ
(
n
)
(merging the lists)


By the Master Theorem (with
a

= 2,
b

= 2,
d

= 1)


T(
n
)


Θ
(
n

log
n
)


So the total time is
Θ
(
n

log
n
).

41

42

Our Lab Today

Implement closest pair algorithms :


The brute force algorithm that computes the
distance between each pair of points and then
returns a closest pair of points.



The Divide
-
and
-
Conquer Algorithm