If one ox could not do the job they did not try to grow a bigger ox, but used two oxen.

currygeckoΛογισμικό & κατασκευή λογ/κού

2 Δεκ 2013 (πριν από 3 χρόνια και 10 μήνες)

77 εμφανίσεις

1

Tuesday, December 23, 2008

If one ox could not do the job
they did not try to grow a
bigger ox, but used two oxen.


-
Grace Murray Hopper

(1906
-
1992)


Distributed Computing

Class:
BSIT
-
8

Instructor:
Raihan Ur Rasool

Chapter 04:

Inter
-
process
Communication

3


Introduction


The API for the Internet protocols


External data representation and marshalling


Client
-
Server communication


Group communication


Case study: interprocess communication in UNIX


Summary

Where are we ?

4


Why

does the communication data need external data representation and
marshalling?


[
data structures must be flattened]


Different data format on different computers, e.g., big
-
endian/little
-
endian
integer order, ASCII (Unix) / Unicode character coding


How

to enable any two computers to exchange data values?

1.
The values be converted to an agreed external format before
transmission and converted to the local form on receipt

2.
The values are transmitted in the sender’s format, together with an
indication of the format used, and the receipt converts the value if
necessary

External data representation and marshalling
introduction
[
data structures and a sequence of bytes]

5


Why does the communication data need external data representation and marshalling?


Different data format on different computers, e.g., big
-
endian/little
-
endian integer
order, ASCII (Unix) / Unicode character coding


How to enable any two computers to exchange data values?

1.
The values be converted to an agreed external format before transmission and
converted to the local form on receipt

2.
The values are transmitted in the sender’s format, together with an indication of
the format used, and the receipt converts the value if necessary


External data representation


An agreed standard for the representation of data structures and
primitive values


Marshalling


The process of taking a collection of data items and assembling them into
a form suitable for transmission in a message


Usage: for data transmission or storing in files


Three alternative approaches to EDR


CORBA’s common data representation / Java’s object serialization & XML

External data representation and marshalling
introduction

Reading Assignment ???



CORBA’s Common Data Representation (CDR)


Java Object Serialization


Extensible Markup language (XML)




Next Time:


Recap


Client
-
server communication

6



7


Represent all of the data types that can be used as arguments and
return values in remote invocations in CORBA


15 primitive types


Short (16bit), long(32bit), unsigned short, unsigned long, float, char,




Constructed types


Types that composed by several primitive types


A message example


The type of a data item is not given with the data representation
in message


It is assumed that the sender and recipient have common knowledge
of the order and types of the data items in a message.


RMI and RPC are on the contrary

CORBA’s Common Data Representation
(CDR)

8




CORBA CDR for constructed types

T

y

p

e

Re

pr

e

s

e

n

ta

t

i

o

n

s

e

q

ue

n

ce

l

e

n

g

th



(

u

n

si

g

n

ed



l

o

n

g

)

fo

ll

ow

ed

b

y



el

e

m

e

nt

s



i

n

o

r

d

e

r

s

t

ri

n

g

l

e

n

g

th



(

u

n

si

g

n

ed



l

o

n

g

)

fo

ll

ow

ed

b

y



ch

a

ra

c

te

rs

i

n o

r

d

e

r



(

ca

n


al

so

ca

n



h

av

e

w

i

de



ch

a

ra

c

te

rs)

a

r

ra

y

a

rr

ay

e

le

m

e

n

t

s i

n



o

r

de

r (

n

o l

en

g

t

h s

p

e

ci

f

ie

d b

eca

us

e

i

t

is

f

i

x

e

d

)

s

t

ru

ct

i

n t

he

or

de

r o

f



de

c

la

r

at

i

o

n o

f

t

he



co

mp

o

n

e

n

t

s

e

n

u

m

e

r

a

t

e

d

u

n

s

i

g

n

e

d



l

o

n

g

(

t

h

e

v

a

l

ue

s a

re


s

pe

c

i

f

ie

d

b

y t

he



o

r

de

r d

ec

l

ar

e

d

)

u

ni

o

n

t

y

p

e

ta

g f

o

l

l

o

we

d b

y

t

h

e s

el

e

cte

d m

e

mb

er

9




CORBA CDR message

The flattened form represents a

Person


struct with value: {‘Smith’, ‘London’, 1934}

0

3

4

7

8

11

12

15

16

19

20
-
23

24

27

5

"
Smit"

"
h___"


6

"
Lond"

"
on__"

1934

index in

sequence of bytes

4
bytes

notes

on representation

length of string


Smith


length of string


London


unsigned long

Struct Person {


string name;


string place;


long year;

};

External data
representation
-
The SUN XDR
standard

10


The entire XDR message consists of a sequence of 4
-
byte
objects:


example:

‘Hi’, ‘
There
’,
2001


2

“Hi
--


5



Ther



e
---


2001

sequence length

‘Hi’



There



Integer
2001

Must also define which end

of each object is the most significant

The marshalled message of the example:
2 Hi 5
There

2001

11


Serialization (
deserialization
)


The activity of flattening an object or a connected set of objects into a
serial form that is suitable for storing on the disk or transmitting in a
message


Include information about the class of each object


Handles: references to other objects are serialized as handles


Each object is written once only


Example


Make use of Java serialization


ObjectOutputStream.writeObject
,
ObjectInputStream.readObject


The use of reflection


Reflection

: The ability to enquire about the properties of a class, such
as the names and types of its instance variables and methods


Reflection makes it possible to do serialization (
deserialization
) in a
completely generic manner

Java object serialization

12




Indication of Java serialized form

The true serialized form contains additional type markers; h0 and h1 are handles

Serialized values

Person

3

1934


8
-
byte version number

int year

5
Smith

java.lang.String

name:

6
London

h0

java.lang.String

place:

h1

Explanation

class name, version number

number, type and name of

instance variables

values of instance variables

Public class Person implements Serializable {


private String name;


private String place;


private int year;


public Person (String aName, String aPlace, int aYear){



name = aName;



place = aPlace;



year = aYear;


}


// followed by methods for accessing the instance variables

}


Person p = new Person(

Smith

,

London

, 1934);

13




Representation of a remote object reference

Internet address

port number

time

object number

interface of

remote object

32
bits

32
bits

32
bits

32
bits


An identifier for a remote object that is valid throughout a
distributed system


Must ensure uniqueness over space and time


Existence of remote object references


Life of the remote object references


Remote object references may not be used for relocated objects

Objectives of this lecture

14


To

study

the

general

characteristics

of

interprocess

communication

and

the

particular

characteristics

of

both

datagram

and

stream

communication

in

the

Internet
.


To

be

able

to

write

Java

applications

that

use

the

Internet

protocols

and

Java

serialization
.


The

design

issues

for

Request
-
Reply

protocols

and

how

collections

of

data

objects

may

be

represented

in

messages

(RMI

and

language

integration

are

left

until

Chapter

5
)
.


15


Introduction


The API for the Internet protocols


External data representation and marshalling


Client
-
Server communication


Group communication


Case study: interprocess communication in UNIX


Summary

Where are we ?

16


The request
-
reply protocol


Normally synchronous
[
client blocks until the reply arrives]


May be reliable


Reply is ACK


Asynchronous may be alternative


Client may retrieve the reply later.


Mostly implemented over UDP but not TCP


Acknowledgements are redundant since requests are followed by
replies


Establishing a connection involves two extra pairs of messages in
addition to the pair required for a request and a reply


Flow control is redundant for the majority of invocations, which pass
only small arguments and results

The request
-
reply protocol

17



The

request
-
reply protocol

Request

Server

Client

doOperation

(
wait)

(
continuation)

Reply

message

getRequest

execute

method

message

select object

sendReply

public byte[] doOperation (RemoteObjectRef o, int methodId, byte[] arguments)

sends a request message to the remote object and receives the reply.

The arguments specify the remote object, the method to be invoked and the
arguments of that method.

public byte[] getRequest ();

acquires a client request via the server port.

public void sendReply (byte[] reply, InetAddress clientHost, int clientPort);


sends the reply message reply to the client at its Internet address and port.

18



Request
-
reply message structure

messageType

requestId

objectReference

methodId

arguments

int (0=Request, 1= Reply)

int

RemoteObjectRef

int or Method

array of bytes

Internet address

port number

time

object number

interface of

remote object

32
bits

32
bits

32
bits

32
bits

information to be transmitted in a request or reply message

Contains message identifier

Representation of Remote Object Reference

19



Request
-
reply message structure


Fields:


1. Message Type


Request or reply


2. A doOperation generates
requestId for each message



A server copies in the reply
message


3 Remote object reference


marshalled


4. methodId



Message Identifier:


requestId

increasing sequence of
U_integers

(reset to zero)


Identfier

for the sender process, for example its port and internet
address

4,294,967,295

20


Failure model


Problems:


Omission failures
. (and failure of processes

crash failure)


No guarantee of delivery in order
.


Timeout


doOperation

return exception when repeatedly issued requests are all
timeout


Duplicate request messages:



filter out duplicates by
requestID


if the server has not yet sent the reply, transmit the reply after finishing
operation execution


Lost Reply Messages


If the server has already sent the reply, execute the operation again to
obtain the result. Note
idempotent operation,

e.g.,
add an element to a
set
,
and a contrary example, append an item to a sequence


History


History: server contains a record of reply messages that have been
transmitted to avoid re
-
execution of
operations

memory cost

The request
-
reply protocol

21


HTTP used by web browsers to make requests to web
servers and to receive replies from them.


Client requests specify a URL that includes the DNS
hostname of a web server, an optional port number and
an identification of a resource.


HTTP specifies:


Messages.


Methods.


Arguments.


Results.


Rules for representing data in messages.


Fixed set of methods, unlike other RMI


Content negotiation and password style authentication

HTTP:
an example of a request


reply protocol

22


If over TCP


Each client
-
server interaction consists of the following steps


The client requests and the server accepts a connection at the default server
port or at a port specified in the URL


The client sends a request message to the server


The server sends a reply message to the client


The connection is closed


Persistent connection (http1.1, RFC 2616)


Connections that remain open over a series of request
-
reply exchanges
between client and server


Closing


Marshalling


Request and replies are marshalled into messages as ASCII text string


Resources are represented as byte sequences and may be compressed


MIME (Multipurpose Internet Mail Extensions) to send text, images etc,
in emails


HTTP Methods


GET, HEAD, POST, PUT, DELETE, OPTIONS, TRACE

HTTP: an example of a request


reply protocol

HTTP
-

methods

23


GET: Requests the resource whose URL is given as argument.


HEAD: Similar to GET, but no data returned.


POST: Can deal with data supplied with the request.


PUT: data supplied in the request is stored with the given URL as
its identifier either as modification of the an existing resource or
as a new resource.


DELETE: Server deletes resource identified by the URL.


OPTIONS: Server supplies client with list of methods that it
allows.


TRACE: Server sends the request back.


Each client request specifies the name of a method to be applied
to a resource at the server and the URL of that resource.


24



HTTP request / reply messages

GET

//
www.dcs.qmw.ac.uk/index.html

HTTP/ 1.1

URL or pathname

method

HTTP version

headers

message body

HTTP/1.1

200

OK


resource data

HTTP version

status code

reason

headers

message body


Request/Reply:


Header: for example acceptable content type


The message body contains data associated with the
URL specified in the request






Contents of HTTP request message

Contents of HTTP reply message

/303

Assignment




Install LAM
-
MPI on Linux


Home Assignment 2
[30/12/08]


25

Objectives of this lecture

26


To study the general characteristics of
interprocess

communication and the particular characteristics of both
datagram and stream communication in the Internet.


To be able to write Java applications that use the Internet
protocols and Java serialization.


To be aware of the design issues for Request
-
Reply
protocols and how collections of data objects may be
represented in messages (RMI and language integration
are left until Chapter 5).


To be able to use the Java API to IP multicast and to
consider the main options for reliability and ordering in
group communication
.


27


Introduction


The API for the Internet protocols


External data representation and marshalling


Client
-
Server communication


Group communication [multicast operation]


Case study: interprocess communication in UNIX


Summary

Where are we ?

28


Fault tolerance based on replicated services


Client request are multicast to all the members of the group, each of
which performs an identical operation


Finding the discovery servers in spontaneous networking


Multicast message can be used by servers and clients to locate available
discovery services to register their interfaces or to look up the
interfaces

of other services

The characteristics of
Multicast

Service Interface provides an entry point that consumers use to
access the functionality exposed by the application
[
network addressable]

29


Fault tolerance based on replicated services


Client request are multicast to all the members of the group, each of
which performs an identical operation


Finding the discovery servers in spontaneous networking


Multicast message can be used by servers and clients to locate available
discovery services to register their interfaces or to look up the
interfaces of other services


Better performance through replicated data


Data are replicated to increase the performance of a service, e.g., Web
Cache. Each time the data changes, the new value is multicast to the
processes managing the replicas



[bandwidth]


Propagation of event notification


Multicast to a group may be used to notify processes when something
happens, e.g., the
Jini

system uses multicast to inform interested clients
when new
lookup services
advertise their existence.
[newsgroup]

The characteristics of
Multicast

30


A multicast group is specified by a class D Internet address


Built on top of IP


Sender is unaware of the identities of the individual recipients


Available only via UDP


The membership of a group is dynamic


It is possible to send datagram to a multicast group without being a member and join
or leave the group at any time.


IPv4


Multicast routers
[Internet]


use the broadcast capability of the local network
[Ethernet]


MTTL (time to live)
-

specify the number of routers a multicast message is allowed
to pass.


[to limit the distance of propagation]


Multicast address allocation


Permanent group


exist even without members 224.0.0.1 to 224.0.0.255


Temporary group


the other addresses, set
TTL to a small value


Temporary groups, free multicast address, avoid accidents,
X IP multicast


Failure model: due to UDP,
[no guarantees, omission failure

unreliable multicast]


Java API to IP multicast


IP Multicast Protocol


group communication

31



Multicast peer joins a group and sends and
receives datagrams

import java.net.*;

import java.io.*;

public class MulticastPeer{


public static void main(String args[]){




// args give message contents & destination multicast group (e.g. "228.5.6.7")


MulticastSocket s =null;




try {




InetAddress group = InetAddress.getByName(args[1]);




s = new MulticastSocket(6789);




s.joinGroup(group);





byte [] m = args[0].getBytes();




DatagramPacket messageOut =




new DatagramPacket(m, m.length, group, 6789);




s.send(messageOut);









// this figure continued on the next slide

32



Multicast peers example…
continued



//
get messages from others in group




byte[] buffer = new byte[1000];





for(int i=0; i< 3; i++) {





DatagramPacket messageIn =




new DatagramPacket(buffer, buffer.length);





s.receive(messageIn);





System.out.println("Received:" + new String(messageIn.getData()));





}




s.leaveGroup(group);






}catch (SocketException e){System.out.println("Socket: " + e.getMessage());



}catch (IOException e){System.out.println("IO: " + e.getMessage());}


}finally {if(s != null) s.close();}


}




}

33


Failures


Router failure prevent all recipients beyond it receiving the message


Members of a group receive the same array of messages in different
orders


Some examples of the effects of reliability and ordering


Fault tolerance based on replicated services


if one of them misses a request, it will become inconsistent with the
others


Finding the discovery servers in spontaneous networking


an occasional lost request is not an issue when locating a discovery
server


Replicated Data [lost messages, ordering]

replication technique


Reliable multicast or unreliable multicast?


According to application’s requirement

Reliability and ordering of multicast

34


Introduction


The API for the Internet protocols


External data representation and marshalling


Client
-
Server communication


Group communication


Case study: interprocess communication in UNIX


FOR SELF STUDY


Summary

Where are we ?

35


Two alternative building blocks


Datagram Socket: based on UDP, efficient but suffer from
failures


Stream Socket: based on TCP, reliable but expensive


Marshalling


CORBA’s CDR and Java serialization


Request
-
Reply protocol


Base on UDP or TCP


Multicast


IP multicast is a simple multicast protocol

Summary

Home Assignment 2

36

Q.1 From the book, answer the following
questions

a)
4.1

b)
4.2

c)
4.5

d)
4.6

e)
4.7

f)

4.25

g)
4.27



Quiz 02

Reading Material

Home Assignment 2:
Learn MPI Programming


37

Discussion date: 1
st

January, 2009

http://www.netlib.org/utk/papers/mpi
-
book/mpi
-
book.html


Extra Slides for Reference


38

39


Datagram communication


Datagram

Socket


Bind


Sendto


recvfrom


Stream communication


stream
socket

,

bind


Accept


Connect


Write
and

read


UNIX socket

40



Sockets used for datagrams

ServerAddress
and
ClientAddress

are socket addresses

Sending a message

Receiving a message

bind(s, ClientAddress)

sendto(s, "message", ServerAddress)

bind(s, ServerAddress)

amount = recvfrom(s, buffer, from)

s = socket(AF_INET, SOCK_DGRAM, 0)

s = socket(AF_INET, SOCK_DGRAM, 0)

41



Sockets used for streams

Requesting a connection

Listening and accepting a connection

bind(s, ServerAddress);

listen(s,5);

sNew = accept(s, ClientAddress);

n = read(sNew, buffer, amount)

s = socket(AF_INET, SOCK_STREAM,0)

connect(s, ServerAddress)

write(s, "message", length)

s = socket(AF_INET, SOCK_STREAM,0)

ServerAddress
and
ClientAddress

are socket addresses