Directed Reading 1 - CNC

footballsyrupSoftware and s/w Development

Dec 1, 2013 (3 years and 11 months ago)

163 views

Combining the strengths of UMIST and

The Victoria University of Manchester

Matthew Livesey, Hemanth John
Jose and Yongping Men

COMP60611


Patterns of parallel programming

Task parallelism


Turning a serial program into a parallel one by understanding dependencies between
the tasks in the program


Traditionally tasks were mapped statically, but dynamic mapping algorithms may
improve performance.


It may be possible to re
-
organise a serial program to reduce dependencies, but need to
be careful that the re
-
organisation does not impact the performance of the algorithm


Scheduling algorithms which focus on the critical path perform best.

01

a = b + c;

02

d = e + f ;

03

z = a + d ;

Combining the strengths of UMIST and

The Victoria University of Manchester

Matthew Livesey, Hemanth John
Jose and Yongping Men

COMP60611


Patterns of parallel programming

Pipeline parallelism


A stream of data items is processed one after another by a sequence of threads.



Figure 4 from
Pautasso and
Alonso 2006 [9]


Combining the strengths of UMIST and

The Victoria University of Manchester

Matthew Livesey, Hemanth John
Jose and Yongping Men

COMP60611


Patterns of parallel programming

Data parallelism

General purpose programming on Graphics
processors (GPGPU).

Exploiting the parallel power of graphics chips
to execute data parallel algorithms.

E.g.
OpenCL


Allows parallel programs to run on whatever
(supported) hardware is available


E.g. Multi
-
core CPU and GPU


Provides bindings in many popular
languages


Cross platform, open source

Map Reduce


Based on a paper from Google.


Designed for processing large data sets in
parallel


Data is processed in parallel by “mappers”,
and the results recombined with “reducers”


Distributed parallelism across
heterogeneous, commodity hardware


Requires the problem at hand to fit the map
reduce pattern

Combining patterns


It is possible to combine patterns, for example a task graph approach may guide the order in which
pipeline steps are applied.


However data and task parallelism are two different ways of looking at a parallel problem.


They may be applied
heirarchically

or sequentially but represent two distinct approaches

Combining the strengths of UMIST and

The Victoria University of Manchester

Matthew Livesey, Hemanth John
Jose and Yongping Men

COMP60611


Patterns of parallel programming

Presentation and report available at:

http://www2.cs.man.ac.uk/~liveseym/parallel/


References:

1. Michael D McCool (Intel).
Structured Parallel Programming with Deterministic Patterns.
To appear in HotPar ’10, 2
nd

USENIX Workshop on Hot Topics
in Parallelism, 14
-
15 June 2010, Berkeley CA. Accessed from
http://software.intel.com/en
-
us/articles/structured
-
parallel
-
programming
-
with
-
deterministic
-
patterns/


2. Jeffrey Dean and Sanjay Ghemawat.
MapReduce: Simplified Data Processing on Large Clusters.
OSDI'04: Sixth Symposium on Operating System
Design and Implementation, San Francisco, CA, December, 2004.

3. The Apache Software Foundation.
Hadoop Map Reduce Tutorial
[Accessed 15
th

October 2010] Available from
http://hadoop.apache.org/common/docs/r0.19.2/mapred_tutorial.pdf

4. David Tarditi et al.
Accelerator: Using Data Parallelism to Program GPUs for General
-
Purpose Uses

Microsoft Research 2006

5. Nickolls et al.
Scalable Parallel Programming. ACM Queue
Volume 6 ,


Issue 2

(March/April 2008) Pages: 40
-
53


6. Aaftab Munshi,
OpenCL, Parallel Computing on the GPU and CPU
, SIGGRAPH 2008: Beyond Programmable

Shading (presentation)

7. A. Grama et al.
Introduction to Parallel Computing.
2
nd

Ed. Addison
-
Wesley, 2003

8. Thomas Rauber and Gudula R
¨
unger l.
Parallel Programming For Multicore and Cluster systems
, 1st edition, Spriner, Pages 280, 2007.

9. Pautasso, C. and


Alonso, G.

Parallel Computing Patterns for Grid Workflows, Workflows in Support of Large
-
Scale Science
, 2006. WORKS '06
Workshop on , 19
-
23 June 2006 .

10.

C. Charles Law ,William J. Schroeder, Kenneth M. Martin and Joshua Temkin.
A multi
-
threaded streaming pipeline architecture for large structured
data sets
, VIS '99: Proceedings of the conference on Visualization '99: celebrating ten years,page225
-
232,1999

11. R. Cytron, M. Hindy and W. Hsiehz.
Automatic Generation of DAG Parallelism
. In Proceedings of the ACM SIGPLAN Conference on Programming
Language Design and Implementation, June 1989, SIGPLAN Notices, Volume 24, Number 6.

12. Y. Kwok and I. Ahmad.
Benchmarking and Comparison of the Task Graph Scheduling Algorithms
. Journal of Parallel and Distributed Computing, 59,
pages 381 422 (1999)