Computing, Fall 2009

footballsyrupSoftware and s/w Development

Dec 1, 2013 (3 years and 8 months ago)

86 views

Introduction to Parallel
Computing, Fall 2009

Sinan Kockara

Department of Computer Science

UCA

Parallel Computing

The

use

of

multiple

computers,

or

computers

with

multiple

internal

processors,

to

solve

a

problem

at

a

greater

computational

speed

than

using

a

single

computer
.


Offers

opportunity

to

tackle

problems

that

could

not

be

solved

in

a

reasonable

time

otherwise
.


Also

problems

with
:




Higher

precision



More

memory

requirements
.


Cluster Computing

A

form

of

parallel

computing

in

which

the

computing

platform

is

a

group

of

interconnected

computers

(a

cluster)
.


We

will

concentrate

upon

cluster

computing

(
Beowulf

Cluster

)
.

References for Beowulf

Beowulf

Cluster

Computing

with

Windows
,

T
.

Sterling

(ed
.
),

The

MIT

Press,

Cambridge

Massachusetts,

2002
,

ISBN

0
-
262
-
69275
-
9
,



Beowulf

Cluster

Computing

with

Linux
,

T
.

Sterling

(ed
.
),

The

MIT

Press,

Cambridge

Massachusetts,

2002
.

Prerequisites


C/C++

if

not

already

known
.


MPI

Programming

can

be

done

in

C

or

C++

(not

Java

with

tools

we

will

use)
.



Course Text

Introduction to Parallel Computing



Publisher:

Addison Wesley; 2 edition (January
26, 2003)


Language:

English


ISBN
-
10:

0201648652


ISBN
-
13:

978
-
0201648652


Other Reference Texts


Parallel Programming: Techniques
and Application Using Networked
Workstations and Parallel
Computers
, 2
nd

edition, by B.
Wilkinson and M. Allen,


Publisher:

Prentice Hall Inc., 2005


ISBN 0
-
13
-
140563
-
2
.



Parallel Programming in C with
MPI and OpenMP,


Publisher:

McGraw
-
Hill Science/Engineering/Math; 1
edition (June 5, 2003)


ISBN
-
10:

0072822562


ISBN
-
13:

978
-
0072822564


Other References


NVIDIA CUDA development zone:
http://www.nvidia.com/object/cuda_home.html#



NVIDIA CG:


http://developer.nvidia.com/object/cg_toolkit.html



Course Outline

In

this

course,

we

will

learn

parallel

computing

techniques

and

algorithms,

and

have

practical

experiences

writing

parallel

programs

on

a

cluster

of

computers
.


We

will

concentrate

upon

the

message
-
passing

method

of

parallel

computing

and

use

the

standard

parallel

computing

environment

called

MPI

(Message

Passing

Interface)
.


Shared

memory

programming

will

be

outlined
.


New

trend

-
general

purpose

computations

on

Graphical

Processing

Unit

(GPGPU)

-

will

be

introduced

Course Outline

Course Contents



Parallel

computers
:

architectural

types,

shared

memory,

message

passing,

interconnection

networks,

potential

for

increased

speed
.



Basic

programming

techniques


GPU

for

parallel

computations


Assessment



Class tests (2)


30%


Assignments


20%


Final Project



25%


Final exam



25%


Parallel Computing


Parallel computing has been hyped for years as the next big thing in
technology


Parallel computing is dividing up tasks over multiple
microprocessors in order to make computers to run faster instead of
using a single processor to perform one task at a time.


The computer industry has taken its first steps toward parallel
computing in recent years by using "multi
-
core" chips.



To maximize computing horsepower, software makers will need to
change how software programmers work. Only a handful of
programmers in the world know how to write software code to divide
computing tasks into chunks that can be processed at the same time
instead of a traditional, linear, one
-
job
-
at
-
a
-
time approach.


Parallel hardware is useless without parallel programs.

Ref: http://www.cioinsight.com/c/a/Past
-
News/Is
-
Parallel
-
Computing
-
the
-
Next
-
Big
-
Thing/

Future of Parallel Computing


The shift to parallel computing was born out of necessity after
processor speeds ran into heat and power limitations, forcing the
semiconductor industry to assemble multiple cores, or electronic
brains, on a single chip.


Intel and Advanced Micro Devices have already assembled chips
with as many as four processors on a single chip. Tilera, a Silicon
Valley chip start
-
up, foresees a 1,000
-
core chip by 2014.


Current processors execute multiple instructions in the same cycle.


parallel computing will be certainly among top priorities of most of
the corporations


The question of how best to utilize these resources mainly depend
upon the parallel software developer and programmer:


specifying and coordinating concurrent tasks



Ref: http://www.cioinsight.com/c/a/Past
-
News/Is
-
Parallel
-
Computing
-
the
-
Next
-
Big
-
Thing/

1
st

Desktop Parallel Computer:
GPU Evolution

M. Houston, “General
-
Purpose Computation Graphics
Hardware”, Presentation Slides, SIGGRAPH 2007, 2007.

Good to Know items for Parallel
program development


Implicit Parallelism: Trends in Microprocessor
Architectures


Limitations of Memory System Performance


Dichotomy of Parallel Computing Platforms


Communication Model of Parallel Platforms


Physical Organization of Parallel Platforms


Communication Costs in Parallel Machines


Messaging Cost Models and Routing Mechanisms


Mapping Techniques


Parallel Software is dependent to hardware architecture


Applications of Parallel Computing


Applications in Engineering and Design


Finite Element Modeling (FEM)


Optimization: Simplex etc.


Scientific applications


Human genome sequencing


Commercial applications


Some of the Largest supercomputing networks reside on Wall
Street (Article, D. E. Shaw
http://www.wired.com/medtech/stemcells/news/2008/07/portfolio
_0730)


Applications in Computer systems


Intrusion detection


Cryptography

Challenges in parallel computing


Moore’s law: circuit complexity doubles
every 18 months


Problem is making this implicit increasing
rates of computation explicit by using
parallelism


Processor
-
Memory/Disk speed bottleneck
(
Read,
http://www.acm.org/crossroads/xrds5
-
3/pmgap.html
)


Data communication


Data reside at different locations