00. prelimx

cathamΤεχνίτη Νοημοσύνη και Ρομποτική

23 Οκτ 2013 (πριν από 4 χρόνια και 16 μέρες)

87 εμφανίσεις

242
-
535 ADA: 0. Preliminaries

1


Objective

o
to give some background on the course

Algorithm Design and Analysis (ADA)

242
-
535
, Semester

1 2013
-
2014

0. Preliminaries

Who I am:



Andrew Davison


WiG

Lab


ad@fivedots.coe.psu.ac.th

Please ask

questions

242
-
535 ADA: 0. Preliminaries

2

1
.

What is an Algorithm?

2
.

Meeting Times / Locations

3
.

Workload

4
.

Exercises

5
.

Course Materials

6
.

Books

7.
Videos

8.
Web Sites

Overview

242
-
535 ADA: 0. Preliminaries

3


An algorithm is a finite set of unambiguous
instructions for solving a problem.

o
An algorithm is
correct

if on all legitimate inputs, it outputs
the right answer in a finite amount of time



Can be expressed as

o
pseudocode

o
flow charts

o
text in a natural language (e.g. English)

o
computer code

1. What is a Algorithm?

242
-
535 ADA: 0. Preliminaries

4

The theoretical study of how to solve computational
problems


sorting a list of numbers


finding a shortest route on a map


scheduling when to work on homework


answering web search queries


and so on...


Algorithm
Design

242
-
535 ADA: 0. Preliminaries

5


Their impact is broad and far
-
reaching.

o
Internet
. Web search, packet routing, distributed file
sharing, ...

o
Biology
. Human genome project, protein folding, ...

o
Computers
. Circuit layout, file system, compilers, ...

o
Computer graphics.

Movies, video games, virtual reality, ...

o
Security
. Cell phones, e
-
commerce, voting machines, ...

o
Multimedia
. MP3, JPG, DivX, HDTV, face recognition, ...

o
Social networks
. Recommendations, news feeds,
advertisements, ...

o
Physics
. N
-
body simulation, particle collision simulation, ...

The Importance of Algorithms

242
-
535 ADA: 0. Preliminaries

6


Ten algorithms having "the greatest influence on the
development and practice of science and
engineering in the 20th century".

o
Dongarra and Sullivan

Top Ten Algorithms of the Century

Computing in Science and Engineering

January/February 2000


o
Barry Cipra

The Best of the 20th Century: Editors Name Top 10
Algorithms

SIAM News

Volume 33, Number 4, May 2000


http://www.siam.org/pdf/news/637.pdf

The Top 10 Algorithms of the 20th Century

242
-
535 ADA: 0. Preliminaries

7


1946: The Metropolis (Monte Carlo) Algorithm.

Uses random processes to find answers to problems
that are too complicated to solve exactly.



1947: Simplex Method for Linear Programming.


A fast technique for maximizing or minimizing a linear
function of several variables, applicable to planning
and decision
-
making.



1950: Krylov Subspace Iteration Method.


A technique for rapidly solving the linear equations
that are common in scientific computation.

What are the Top 10?

242
-
535 ADA: 0. Preliminaries

8


1951: The Decompositional Approach to Matrix
Computations.

A collection of techniques for
numerical linear algebra.



1957: The
Fortran

Optimizing Compiler.




1959: QR Algorithm for Computing Eigenvalues.

A crucial matrix operation made swift and
practical. Application areas include computer
vision, vibration analysis, data analysis.



1962: Quicksort Algorithm.


We will look at this.

242
-
535 ADA: 0. Preliminaries

9


1965: Fast Fourier Transform (FFT).

It breaks down
waveforms (like sound) into periodic components.
Used in many different areas (e.g. digital signal
processing , solving partial differential equations,
fast multiplication of large integers.)



1977: Integer Relation Detection.

A fast method for
finding simple equations that explain collections of
data.



1987: Fast Multipole Method.

Deals with the
complexity of n
-
body calculations. It is applied in
problems ranging from celestial mechanics to
protein folding.

242
-
535 ADA: 0. Preliminaries

10


Simple recursive algorithms


Divide and conquer


Backtracking


Dynamic programming


Greedy algorithms


Brute force


Randomized algorithms


Some Algorithm Types

242
-
535 ADA: 0. Preliminaries

11


A simple recursive

algorithm:

o
Non
-
recursive base case

o
Recurs with a simpler subproblem



Examples:

o
Count the number of occurrence of an element in a tree

o
Test if a value occurs in a list

o
Fibonnaci number calculation



Several of the other algorithm types use recursion in
more complex ways


Simple
Recursive

11

242
-
535 ADA: 0. Preliminaries

12


The Fibonacci sequence:

o
1, 1, 2, 3, 5, 8, 13, 21, 33, ...



Find
the n
th

Fibonacci
number
:

fib
(int
n)


if
(n <=
1)


return 1;



else
return
fib
(n
-
1) +
fib
(n
-
2);



Fibonnaci numbers

242
-
535 ADA: 0. Preliminaries

13


Execution of fib(5):

lots of repeated work,

that only gets worse

for bigger n

242
-
535 ADA: 0. Preliminaries

14


A
divide and conquer algorithm

:

o
Divide

the problem into smaller subproblems

o
Combine

the solutions to the subproblems into a solution
to the original
problem



Examples:

o
quicksort

o
merge sort

o
binary
search




Divide and
Conquer

242
-
535 ADA: 0. Preliminaries

15


Backtracking algorithms

use a depth
-
first recursive
search

o
involves choice (non
-
determinism)

o
backtracking means "go back to where you came from"



Examples:

o
search a maze

o
color a map with no

more than four colors

Backtracking

242
-
535 ADA: 0. Preliminaries

16


A
dynamic programming algorithm

remembers past
results and uses them to find new results.

o
multiple solutions exist; find the “best” one (the
optimal

one)


o
the problen contains overlapping (repeated) subproblems


solutions are stored and reused

Dynamic
Programming

242
-
535 ADA: 0. Preliminaries

17


Finding the n
th

Fibonacci number involves lots of
repeated work (overlapping subproblems) that can
be avoided using
memorization
:

Fibonacci
numbers Again

17

242
-
535 ADA: 0. Preliminaries

18


An
optimization problem
: find the
best

solution



A “greedy algorithm”
sometimes

works well for
optimization problems, and is easy to code



At each step:

o
use the best solution you can get right now, without regard for
future steps

o
You hope that by choosing a
local

optimum at each step,
you will end up with a
globally

optimum final solution

Greedy algorithms

18

242
-
535 ADA: 0. Preliminaries

19


Count out some money using the fewest possible
notes and coins



A greedy algorithm will
take the largest possible
note or coin at each step



Example: count out $6.39 using:


a $5 bill


a $1 bill


// to make $6


a 25¢ coin


// to make $6.25


a 10¢ coin


// to make $6.35


four 1¢ coins


// to make $6.39

Example: Counting
Money

242
-
535 ADA: 0. Preliminaries

20


“Krons” money come in
1
,
7
, and
10

coins



Count out 15 krons:

o
A 10 kron piece

o
Five 1 kron pieces, for a total of 15 krons



This requires 6 coins, but a better solution is two 7
kron pieces and one 1 kron piece (3 coins)



The greedy algorithm 'fails' because its final
solution is not the best (not globally optimal)

Greedy Algorithms Can 'Fail'

242
-
535 ADA: 0. Preliminaries

21


A
brute force algorithm

tries
all

possibilities until a
satisfactory solution is found.



Often, brute force algorithms require exponential
running time (very large time, so very slow)



Various
heuristics

and
optimizations

can be used

o
heuristic

means “a rule of thumb”

o
look for
sub
-
optimal solutions

(not the best), which can be
calculated more quickly than the best one

Brute
Force

242
-
535 ADA: 0. Preliminaries

22


A
randomized algorithm

uses a random number to
make a choice during the computation

o
faster than calculating a choice

o
the choice may be just as good



Examples:

o
Quicksort randomly chooses a pivot


o
Factor a large number by

choosing random numbers

as possible divisors

Randomized
Algorithms

242
-
535 ADA: 0. Preliminaries

23

The theoretical study of algorithm
performance

and
resource usage
.


Performance isn't the only important things for code:



modularity




user
-
friendliness



correctness





programmer time



maintainability





simplicity



functionality





extensibility



robustness






reliability

Analysis of Algorithms

This subject isn't about

these things.

242
-
535 ADA: 0. Preliminaries

24



It help us to understand algorithm
scalability
.




Performance often draws the line between what


is
feasible

and what is impossible.




Algorithmic mathematics provides a precise way to

talk about
program behavior
.




The lessons of program performance
generalize


to other computing resources.

Why study Algorithm Analysis?

242
-
535 ADA: 0. Preliminaries

25


Mathematical induction, running time of
programs
; growth of functions


Divide
-
and
-
conquer; comparison and linear sorts


Dynamic programming


Greedy algorithms


Elementary graph algorithms, minimum spanning
trees, shortest path problems, maximum flow


String matching


Computational geometry


NP completeness; approximation algorithms

Course Structure

242
-
535 ADA: 0. Preliminaries

26


Tuesday


9:00


10:30


R101

Wednesday

10:30


12:00

R101



I want to change these times to be

3

classes/week, each of
1 hour
.



Tell me your preferences.

2
. Meeting Times / Locations

242
-
535 ADA: 0. Preliminaries

27


Mid
-
term exam:


35%


(
2

hours)

o
week
8



Final exam:


45%


(
3

hours)

o
weeks
17
-
18



Two exercises:


20%

(2*10)


weeks 6
-
7 (July 8
-
19) and weeks 15
-
16 (September 9
-
20)

3
. Workload

242
-
535 ADA: 0. Preliminaries

28


I
may

take registration at the start of a class.



If someone is not there, they lose
1%


(unless they have a good excuse).



A maximum of
10%

can be lost

o
deducted from your final mark

Non
-
Attendence Penalty

242
-
535 ADA: 0. Preliminaries

29


The two exercises are worth a total of
20%

(each worth
10%
).



They will be maths problems

and/or

algorithms to design/write.

4
. Exercises

continued

242
-
535 ADA: 0. Preliminaries

30


Planned exercise times (which may change):

o
ex.

1

in weeks
6
-
7 (July 8
-
19)

o
ex
. 2

in weeks
15
-
16 (September 9
-
20)



Cheating will result in
0

marks.

o
YOU HAVE BEEN WARNED!!

242
-
535 ADA: 0. Preliminaries

31


All the handouts (and other materials)

will be
placed on
-
line at


http://fivedots.coe.psu.ac.th/



Software.coe/
242
-
535_
ADA
/



Print 6 slides
-
per
-
page, grayscale, and bring to
class.

5
. Course Materials

242
-
535 ADA: 0. Preliminaries

32


Introduction to Algorithms

Thomas Cormen, Charles Leiserson, Ronald
Rivest, Clifford Stein

M
cGraw Hill
,
2003
,
2nd

edition

o
mathematical, advanced,
the standard text

o
now up to version 3 (MIT)


o
lots of resources online;

see video section


6
. Books

continued

242
-
535 ADA: 0. Preliminaries

33


Algorithms

Robert Sedgewick, Kevin Wayne

Addison
-
Wesley, 2011, 4
th ed.

o
implementation (Java) and theory

o
intermediate level




Data Structures and Algorithms in Java

Robert Lafore

Sams Publishing, 2002, 2nd

ed.

o
Java examples; old

o
basic level; not much analysis


242
-
535 ADA: 0. Preliminaries

34


Algorithms
Unlocked

Thomas
H. Cormen

MIT Press, March 2013




Nine Algorithms
that Changed
the
Future

John MacCormick

Princeton University
Press, 2011

o
http://users.dickinson.edu/~jmac/9algorithms
/


search
engine indexing, pagerank, public key
cryptography, error
-
correcting codes, pattern
recognition, data compression, databases,
digital signatures, computablity

Fun Overviews

242
-
535 ADA: 0. Preliminaries

35


Algorithmic Puzzles

Anany Levitin, Maria Levitin

Oxford University Press, , 2011




Algorithmics: The Spirit of
Computing

David Harel, Yishai Feldman

Addison
-
Wesley; 3 ed., 2004

(and Springer, 2012)

o
http://www.wisdom.weizmann.ac.il/
~harel/algorithmics.html

242
-
535 ADA: 0. Preliminaries

36


The New Turing Omnibus: Sixty
-
Six

Excursions in Computer Science

A. K. Dewdney

Holt, 1993

o
66 short article; e.g. detecting primes, noncomputable
functions, self
-
replicating computers, fractals, genetic
algorithms, Newton
-
Raphson Method, viruses

242
-
535 ADA: 0. Preliminaries

37


MIT 6.046J / 18.410J Intro. to Algorithms, Fall 2005

o
http://ocw.mit.edu/6
-
046JF05


original course website for Cormen book


o
http://videolectures.net/mit6046jf05_introduction_algorithms/


video and slides side
-
by
-
side


o
http://www.catonmat.net/

category/introduction
-
to
-
algorithms


notes taken while

watching the videos


7. Videos

242
-
535 ADA: 0. Preliminaries

38


Hi
-
tech Trek

Royal Institution Christmas Lectures 2008

o
A hi
-
tech trek through the world of computer science by
Professor Chris Bishop; aimed at school kids.


Lecture 1: Breaking the speed limit


Lecture 2: Chips with everything


Lecture 3: Ghost in the machine


Lecture 4: Untangling the web


Lecture 5: Digital intelligence

o
http://richannel.org/christmas
-
lectures/

2008/

o
I have copies on a DVD

242
-
535 ADA: 0. Preliminaries

39


Algorithm Tutorials

o
http://community.topcoder.com/tc?module=Static&

d1=tutorials&d2=alg_index


Algorithmy

o
http://en.algoritmy.net/


brief explanations and code


Algorithmist

o
http://algorithmist.com/index.php/Main_Page


explanations and code



Wikipaedia page for an algorithm

o
e.g. http://en.wikipedia.org/wiki/Quicksort

8. Web Sites