Introduction to Parallel Programming (with OpenMP)

shapecartSoftware and s/w Development

Dec 1, 2013 (3 years and 9 months ago)

67 views



Introduction to Parallel Programming
(with OpenMP)
By-
Lalith Suresh
Gyan! A FOSS Workshop
24
th
of October, Neuron '09


Once Upon A Time
Introduction to Parallel Programming; 24th of October, 2009; Neuron, 2009
Section I: Once Upon A Time


Section I: Once Upon A Time

Processor speeds used to grow at a rapid rate.

A programmer could write a piece of code today,
which he knows will run faster in a year or two.

But a lot of poor programmers were to be
disappointed later on...
Advancement of Unicore Processors
Introduction to Parallel Programming; 24th of October, 2009; Neuron, 2009


Around 2002...
Section I: Once Upon A Time
Introduction to Parallel Programming; 24th of October, 2009; Neuron, 2009


Section I: Once Upon A Time
Introduction to Parallel Programming; 24th of October, 2009; Neuron, 2009


Section I: Once Upon A Time

Unicore processor speeds are no longer
increasing.

You might have more computational power, but
your processors aren't exactly faster.

Applications must be re-written to make use of
parallel architectures.

http://www.gotw.ca/publications/concurrency-
ddj.htm
The Free Lunch is Over!
Introduction to Parallel Programming; 24th of October, 2009; Neuron, 2009


The New Paradigm
Section II: The New Paradigm
Introduction to Parallel Programming; 24th of October, 2009; Neuron, 2009


Section II: The New Paradigm

It is a subset of a process.

One process can have multiple threads.

Threads within a process, share state information
and resources.

Threads share their address space.

Context switching between threads is fast.
What is a Thread?
Introduction to Parallel Programming; 24th of October, 2009; Neuron, 2009


Section II: The New Paradigm

You can have multiple threads within the same
program.

These threads can operate in parallel depending
on how many processors you have.

Only through multi threading can a Multi-core
processor run a program faster than on a Uni-core
processor.
What is Multi-threading?
Introduction to Parallel Programming; 24th of October, 2009; Neuron, 2009


Section II: The New Paradigm
What is Multi-Threading?
Introduction to Parallel Programming; 24th of October, 2009; Neuron, 2009


Sharing Memory? Uh-oh...
Section III: Sharing Memory? Uh-oh...
Introduction to Parallel Programming; 24th of October, 2009; Neuron, 2009


Section III: Sharing Memory? Uh-oh...

Happens when two or more threads try to act on
the same piece of shared memory, in parallel.

This affects the result.

They're very hard to detect.

You need synchronisation mechanisms to avoid
this.

Famous race condition bugs include the Therac-25
bug and the power outage of North America.
Race Conditions
Introduction to Parallel Programming; 24th of October, 2009; Neuron, 2009


Section III: Sharing Memory? Uh-oh...
Race Conditions
Introduction to Parallel Programming; 24th of October, 2009; Neuron, 2009


Diving Into Parallel Programming
Section IV: Diving into Parallel Programming
Introduction to Parallel Programming; 24th of October, 2009; Neuron, 2009


Section II: The New Paradigm

OpenMP

MPI

GPGPU computing (Nvidia CUDA, ATI Brooke+)

Pthreads

Java Threads

And so on...
Parallel Programming Tools
Introduction to Parallel Programming; 24th of October, 2009; Neuron, 2009


Section II: The New Paradigm

Stands for Open Multi-programming.

Specific to shared memory parallelism.

Use compiler directives to parallelise code.

Is very easy to do, and you don't need to re-write
code.
OpenMP
Introduction to Parallel Programming; 24th of October, 2009; Neuron, 2009


Section II: The New Paradigm
Fork-Join Model used by OpenMP
Introduction to Parallel Programming; 24th of October, 2009; Neuron, 2009


Demo!
Section IV: Diving into Parallel Programming
Introduction to Parallel Programming; 24th of October, 2009; Neuron, 2009


Questions?
Introduction to Parallel Programming; 24th of October, 2009; Neuron, 2009