Parallel and Cloud

footballsyrupΛογισμικό & κατασκευή λογ/κού

1 Δεκ 2013 (πριν από 3 χρόνια και 7 μήνες)

69 εμφανίσεις

Parallel and Cloud
Computing

Dr. T. Mala

Assistant Professor (Sr. Grade),

Anna University,

Chennai

Introduction to Parallel Computing


Serial Computing


Standard computing is also known as serial
computing


You have been doing serial computing for year


Traditionally software has been written for serial
computation.


Introduction to Parallel Computing


Parallel Computing


Parallel computing is the simultaneous use of
multiple compute resources (Cores, Computer) to
solve a computational problem.


The programmer has to figure out how to break
the problem into pieces, and has to figure out
how the pieces relate to each other.


For example: A parallel

chess
-
playing systems
work, including the famous IBM Deep Blue
machine that beat
kasparav
.



Serial Vs. Parallel Services

Q

Please

COUNTER

COUNTER 1

COUNTER 2

P

P

P

P

P

P

.

.

Microkernel

Multi
-
Processor Computing System

Threads Interface

Hardware

Operating System

Process

Processor

Thread

P

Applications

Computing Elements

Programming paradigms

Two Eras of Computing

Concepts and Terminology:

Why Use Parallel Computing?


Saves time


wall clock time


Cost savings


Overcoming memory constraints


It’s the future of computing


Concepts and Terminology:

Flynn’s Classical Taxonomy


Distinguishes multi
-
processor architecture by
instruction and data

1.
SISD


Single Instruction, Single Data

2.
SIMD


Single Instruction, Multiple Data

3.
MISD


Multiple Instruction, Single Data

4.
MIMD


Multiple Instruction, Multiple Data


Flynn’s Classical Taxonomy:

SISD


Serial


Only one instruction
and data stream is
acted on during any one
clock cycle


Flynn’s Classical Taxonomy:

SIMD


All

processing

units

execute

the

same

instruction

at

any

given

clock

cycle
.


Each

processing

unit

operates

on

a

different

data

element
.


Flynn’s Classical Taxonomy:

MISD


Different instructions operated on a single
data element.


Very few practical uses for this type of
classification.


Example: Multiple cryptography algorithms
attempting to crack a single coded message.


Flynn’s Classical Taxonomy:

MIMD


Can execute
different
instructions on
different data
elements.


Most common
type of parallel
computer.


Parallel Computer Memory:

Shared Memory Architecture


All processors access
all memory as a single
global address space.


Data sharing is fast.


Lack of scalability
between memory and
CPUs



Parallel Computer Memory:

Distributed Memory Architecture


Each processor has its
own memory.


Is scalable, no overhead
for cache coherency.


Programmer is
responsible for many
details of
communication
between processors.


Parallel Programming Models


Exist as an abstraction above hardware and
memory architectures


Examples:


Shared Memory


Threads


Messaging Passing


Data Parallel


Parallel Programming Models:

Shared Memory Model


Appears to the user as a single shared
memory, despite hardware implementations.


Locks and semaphores may be used to control
shared memory access.


Program development can be simplified since
there is no need to explicitly specify
communication between tasks.



Parallel Programming Models:

Threads Model


A single process may have multiple,
concurrent execution paths.


Typically used with a shared memory
architecture.


Programmer is responsible for determining all
parallelism.


Parallel Programming Models:

Data Parallel Model


Tasks performing the same operations on a set
of data. Each task working on a separate piece
of the set.


Works well with either shared memory or
distributed memory architectures.


Designing Parallel Programs:

Automatic Parallelization


Automatic


Compiler analyzes code and identifies
opportunities for parallelism


Analysis includes attempting to compute whether
or not the parallelism actually improves
performance.


Loops are the most frequent target for automatic
parallelism.


Designing Parallel Programs:

Manual Parallelization


Understand the problem


A Parallelizable Problem:


Calculate the potential energy for each of several
thousand independent conformations of a molecule.
When done find the minimum energy conformation.


A Non
-
Parallelizable Problem:


The Fibonacci Series


All calculations are dependent


Computational Power Improvement

Parallel Computing Scope


Parallel computing is fast.


There are many different approaches and
models of parallel computing.


Parallel computing is the future of computing.





Parallel Vs Distributed Systems

Parallel Systems

Distributed Systems

Memory

Tightly coupled shared

memory

Distributed

memory

Message passing, RPC, and/or used
of distributed shared memory

Control

Global clock

control

SIMD, MIMD

No global clock

control

Synchronization algorithms needed

Processor
interconnection

Order of
Tbps

Bus, mesh, tree, mesh of tree,
and hypercube (
-
related)
network

Order of
Gbps

Ethernet(bus), token ring and SCI
(ring),
myrinet
(switching network)

Main focus

Performance

Scientific computing

Performance(
cost and
scalability
)

Reliability/availability

Information/resource sharing

Introduction to Cloud Computing


The

definition

by

U
.
S
.

Governments

National

Institute

of

Standards

and

Technology
.




“Cloud

computing

is

a

model

for

enabling

ubiquitous,

convenient,

on
-
demand

network

access

to

a

shared

pool

of

configurable

computing

resources

(e
.
g
.
,

networks,

servers,

storage,

applications,

and

services)

that

can

be

rapidly

provisioned

and

released

with

minimal

management

effort

or

service

provider

interaction”

Properties of Cloud Computing


There are six key properties from the Google’s
perspective

1.
Cloud computing is user
-
centric

2.
Cloud computing is task
-
centric

3.
Cloud computing is powerful

4.
Cloud computing is accessible

5.
Cloud computing is intelligent

6.
Cloud computing is programmable


History of Cloud Computing from
Collaboration

1.
Client/Server Computing: Centralized
Applications and Storage

2.
Peer
-
to
-
Peer Computing: Sharing Resources

3.
Distributed Computing: Providing More
Computing Power

4.
Collaborative Computing: Working as a Group

5.
Cloud Computing: The Next Step in


Collaboration

Evolution of Cloud Computing


There are multiple factors that led to the
evolution of Cloud Computing as follows,


Evolution of Internet Service Provider (ISP)


Programmable Web


Virtualization


Evolution of ISP


The ISP provides more contribution to
evolution of cloud computing, by the way ISP
matured over period of time.

Evolution of ISP (Cont)

Programmable Web


Web service made the web programmable.


Protocols like


Simple Object Access Protocol (SOAP)


Representational State Transfer (REST)


JavaScript Object Notation (JSON)


Plain Old XML (POX)

fueled the growth of APIs on the web.


Today every popular search engine, social
networking site and syndication portal has APIs
offered to developers.

Virtualization


Is

the

most

discussed

tem

among

CIOs

and

IT

decision

makers
.


Through

virtualization

,

data

center

infrastructure

can

be

consolidated

from

hundred

of

server

to

just

tens

of

servers
.


This

result

in

lower

Total

Cost

of

Ownership

(TCO)

and

brings

substantial

savings

on

the

power

bills

and

reduced

cost

of

cooling

equipment
.


Delivery Models of Cloud

1.
Infrastructure as a Service (IaaS)

2.
Platform as a Service (PaaS)

3.
Software as a Service (SaaS)

1. IaaS


The consumer uses "fundamental computing
resources" such as processing power, storage,
networking components or middleware.


The consumer can control the operating
system, storage, deployed applications and
possibly networking components such as
firewalls and load balancers, but not the cloud
infrastructure beneath them.

2. PaaS


The

consumer

uses

a

hosting

environment

for

their

applications
.



The

consumer

controls

the

applications

that

run

in

the

environment

(and

possibly

has

some

control

over

the

hosting

environment),

but

does

not

control

the

operating

system,

hardware

or

network

infrastructure

on

which

they

are

running
.



The

platform

is

typically

an

application

framework
.

3. SaaS


The

consumer

uses

an

application,

but

does

not

control

the

operating

system,

hardware

or

network

infrastructure

on

which

it's

running
.

Deployment Models of Cloud

1.
Private cloud

2.
Community cloud

3.
Public cloud

4.
Hybrid cloud

1. Private cloud


The cloud infrastructure is provisioned for
exclusive use by a single organization
comprising multiple consumers (e.g., business
units).


It may be owned, managed, and operated by
the organization, a third party, or some
combination of them, and it may exist on or
off premises.

2. Community cloud


The cloud infrastructure is provisioned for
exclusive use by a specific community of
consumers from organizations that have shared
concerns (e.g., mission, security requirements,
policy, and compliance considerations).


It may be owned, managed, and operated by one
or more of the organizations in the community, a
third party, or some combination of them, and it
may exist on or off premises.

3.
Public cloud


The cloud infrastructure is provisioned for
open use by the general public.


It may be owned, managed, and operated by a
business, academic, or government
organization, or some combination of them.


It exists on the premises of the cloud provider.

4. Hybrid cloud


The cloud infrastructure is a composition of
two or more distinct cloud infrastructures
(private, community, or public) that remain
unique entities, but are bound together by
standardized or proprietary technology that
enables data and application portability (e.g.,
cloud bursting for load balancing between
clouds).

Essential Characteristics of Cloud


NIST defines five essential characteristics of
cloud computing.

1.
Rapid Elasticity

2.
Measured Service

3.
On
-
Demand Self
-
Service

4.
Ubiquitous Network Access

5.
Resource Pooling

1. Rapid Elasticity


Elasticity is defined as the ability to scale
resources both up and down as needed.


To the consumer, the cloud appears to be
infinite, and the consumer can purchase as
much or as little computing power as they
need.

2.
Measured Service


In a measured service, aspects of the cloud
service are controlled and monitored by the
cloud provider.


This is crucial for billing, access control,
resource optimization, capacity planning and
other tasks.

3.

On
-
Demand Self
-
Service



The on
-
demand and self
-
service aspects of
cloud computing mean that a consumer can
use cloud services as needed without any
human interaction with the cloud provider.

4.
Ubiquitous Network Access


Ubiquitous

network

access

means

that

the

cloud

provider’s

capabilities

are

available

over

the

network

and

can

be

accessed

through

standard

mechanisms

by

both

thick

and

thin

clients
.

5. Resource Pooling


Resource

pooling

allows

a

cloud

provider

to

serve

its

consumers

via

a

multi
-
tenant

model
.



Physical

and

virtual

resources

are

assigned

and

reassigned

according

to

consumer

demand
.



There

is

a

sense

of

location

independence

in

that

the

customer

generally

has

no

control

or

knowledge

over

the

exact

location

of

the

provided

resources

but

may

be

able

to

specify

location

at

a

higher

level

of

abstraction

(e
.
g
.
,

country,

state,

or

datacenter
)
.

Recent Trends

Gartner Hype Cycle August 2010

Gartner Hype Cycle July 2011

Blind men and an Elephant

Cloud Architecture


Cloud Architecture (Cont)

1.
Fabric layer


Contains the raw hardware level resources such as
compute, storage and network resources.

2.
Unified resource layer


Contains resources that have been
abstracted/encapsulated (usually by virtualization).

3.
Platform layer


Adds on collection of specialized tools, middleware and
services on top of the unified resource to provide
development and deployment platform.

4.
Application layer


Contains the applications that would run in the clouds.

Cloud Computing: Advantages


Lower
-
Cost Computers for Users


Improved Performance


Lower IT Infrastructure Costs


Fewer Maintenance Issues


Lower Software Costs


Instant Software Updates


Increased Computing Power



Cloud Computing: Disadvantages


Requires a Constant Internet Connection


Doesn’t Work Well with Low
-
Speed
Connections


Can Be Slow


Features Might Be Limited


Stored Data Might Not Be Secure


If the Cloud Loses Your Data, You’re Screwed


Who Benefits from Cloud Computing?


Collaborators


Road Warriors


Cost
-
Conscious Users


Cost
-
Conscious IT Departments


Users with Increasing Needs

Who Shouldn’t Be Using Cloud
Computing?


The Internet
-
Impaired


Offline Workers


The Security Conscious


Anyone Married to Existing Applications


Cloud Computing for Everyone


Cloud Computing for the Family


Cloud Computing for the Community


Cloud Computing for the Corporation

Using Cloud Services


Collaborating on Calendars, Schedules, and Task
Management


Collaborating on Event Management


Collaborating on Contact Management


Collaborating on Project Management


Collaborating on Word Processing


Collaborating on Spreadsheets


Collaborating on Databases


Collaborating on Presentations


Storing and Sharing Files and Other Online Content


Sharing Digital Photographs


THANK YOU


Any Questions?