NOD Architecture - kollektivanbud

batterycopperInternet and Web Development

Nov 12, 2013 (3 years and 9 months ago)

234 views


IO A/S




Side
1

av
7


NOD
Architecture


Table of Contents

NOD Architecture

................................
................................
................................
................................
................
1

About this document

................................
................................
................................
................................
.......
1

PL4/NOD physical and logical architecture

................................
................................
................................
.....
2

Designchoices based on the supplemental requirements

................................
................................
..............
3

Overall Design Goals

................................
................................
................................
................................
...
3

Technology

................................
................................
................................
................................
..................
3

Robustness

................................
................................
................................
................................
..................
4

Performance

................................
................................
................................
................................
................
5

Maintainability

................................
................................
................................
................................
..............
6

Monitoring

................................
................................
................................
................................
....................
6

Availability

................................
................................
................................
................................
....................
6

Security

................................
................................
................................
................................
........................
6

Designchoices based on select Usecases

................................
................................
................................
......
7




Change history

Date

Version

Responsible

Description

16.03.2011

0.1

Karl Ivar Da
hl

Initial version

29.03.2011

0.2

Karl Ivar Dahl


13.04.2011

0.9

Karl Ivar Dahl

Proposed ar
ch
itecture for POC

17.08.2011

1.0

Karl Ivar Dahl

Updated after POC


References

ID

Description

1

NOD

Supplementary_Specification
.doc

2

NOD

Client

Interfaces

(C
oconet WIKI)

3

NOD

PLUGIN Interface

(Coconet WIKI)

4

NOD

PTO Interfaces

(Coconet WIKI)

5

NOD

Administrative Interface

(Coconet WIKI)




About this document

This document aims to describe the logical and physical view of the
NOD

Server
Architecture.


T
he architecture is primarily a result of the
supplementary specification
[1]
, and we recommend reading the
s
e
requirements first.






Side
2

av
7

The goal is to establish a modern architecture with excellent scalability and future flexibility, all while keeping
the solutio
n as simple as possible.


PL4/
NOD

physical and logical
architecture





















Side
3

av
7

MongoDB
MongoDB Replica Set
PL
4
Tomcat
PL
4
Services
PL
4
Admin
Order
Service
PL
4
WS
WS
-
Order
Order
Publisher
Order
Update
Consumer
PL
4
Import
PL
4
Export
Transaction
Consumer
NAL Tomcat
NAL Services
Security
Service
Order
Service
Admin
Service
Order
Consumer
Order
Updater
Plugin
Service
Apache CXF
Load Balanced Nodes
REST
-
Action
Service
Custom
Transport
(
application
/
exi etc
)
Transaction
Publisher
Oracle
AQ
-
Transac
tions
PL
4
DB
AQ
-
Orders
SAM
Load Balancer
NOD Client
IOS Filshare
AQ
-
Process
ed
Orders
PTO
Order
AQ
-
Process
ed
Orders
Transac
tion
Order
Mapping
DIFF
Service
PLUGIN Tomcat
Plugin WEB Container
HB
206
+++
Comma
nd Set



Designchoices based on the supplemental requirements

Overall Design Goals

The
NOD

interface will be based on the principles of RESTful commu
nication; Request processing is done
stateless over HTTP
S

with simple and lean protocols.


The
NOD Client

interface
services should be
idempotent

or provide a receipts that the action has already
been performed.

This means that resubmitting an identical re
quest should not cause an error.


Technology

The
NOD

is based on the same basic architecture as PL4: Pure Java objects exposing services through the
Spring Framework, deployed on Tomcat. This has proven to be a very simple and solid architecture.


To maint
ain a common technical fundament for both PL4 and NOD, PL4 should be upgraded to use the
same version of the frameworks that NOD will use.


Communication protocol





Side
4

av
7

Webservices in PL4 are exposed through the high
-
performing XFire api. This API has since been

integrated
into Apache CXF (
http://cxf.apache.org/
):

Apache CXF is an open source services framework. CXF helps you build and develop services using
frontend programming APIs, like JAX
-
WS and JAX
-
RS. These services
can speak a variety of protocols
such as SOAP, XML/HTTP, RESTful HTTP, or CORBA and work over a variety of transports such as
HTTP, JMS or JBI.


Upgrading XFire to CXF will give us the ability to expose services using other encodings than SOAP in a
modular

fashion, while still maintaining a tight integration with the Spring Framework. This will enable the
NOD

to be "future proof" in terms of communication.


We do however recommend staying with the HTTP
/HTTPS

protocol for all communication as this is a
prot
ocol that is well
understood and scales easily with load balancers. To avoid the overhead of repeated
connections we should instead look into the
keep
-
alive

HTTP attribute instead of considering persistent TCP
connect
ions.
The
NOD

will have a known, stable

number of clients (the
NOD Client
s), so we should be able
to forecast the numbe
r

of simult
aneous connections this will res
ult in.


Request payloads

Traditional WebServices are quite heavy both on the client and server

side
. In addition they consume a lot
of
network bandwith.
Changes to the schema will often result in required changes on the client as well. In the
case of
NOD CLIENT
's we can not expect all
NOD Clients to

support the same version of the interface at the
same time.


This is solved by using a
modular XML "core" payload where extensions can be deployed using XML Name
Spaces.


By using the XML Infoset the interfaces will have a robust specification language (XML Schema). Combined
with the HTTP Content Negotiation support, the NOD sever may in the

future add support for alternative
serializations of the XML infoset, such as
Efficient XML

or
Fast Infoset
.

Robustness

Application Server

Failure of one Java application server node will result in a lost connection from the PUD.
Given the nature of
the i
dempotent services, the PUD is however free to resubmit the request many times, and should be
redirected to a functional node by the load balancer on the next request.


Plugins

The plugins should be a separate module that may be implemented using various t
echnologies such as
C/C++.
Calling existing C/C++ code from Java is

however

quite
dangerous, as any failure will be able to take
down the entire JVM.
Different plugins may also require different resources such as database connections or
file system access.


Each plugin is therefore deployed as a separate Web Application that is accessed by the NOD server by a
common REST Plugin interface. This allows easy management of plugins without any consequences for the
main NOD Server. Plugins may also be developed i
n any suitable language that may implement the REST
Plugin API such as COM+ exposed by .NET on Internet Information Server, with a custom database.


Plugin WEB application should only be deployed on the same Tomcat Nodes as the NOD Server after
thorough te
sting.


Database





Side
5

av
7

While PL4 uses Oracle today, an additional database is recommended to drive the NOD Server. MongoDB is
a extremely high
-
performing Open Source database NO
-
SQL database that is used to
cache

the information
needed when an Order Group is dis
tributed through the NOD.

This design choice enables near linear scalability on most database
-
related access in the NOD Server while
still having the benefits of transactional control of all Orders stored in PL4.


MongoDB

has high robustness;
A properly
-
co
nfigured MongoDB shard cluster will have no single point of
failure.


Potential single points of failure

If the PL4 Oracle database

or the queue itself

fails,
the queue will

be unavailable
to
the
NOD
. This will result
in
NOD

res
p
onse failures.


IO should c
onsider

improving the robustness of the PL4 oracle Database by clustering it.


Performance

The critical performance
path
in the
NOD

is the request/response between the
NOD Client
and the
NOD
.


All communication not strictly necessary to facilitate this
sy
nchronous
communication should be
asynchronous.


This is solved by establishing JMS queues for the communication between
the
NOD

and PL4 based on
Oracle

AQ
.


In addition, all synchronous database access in the
NOD

will be against MongoDB, a NoS
Q
L database

with
extremely good performance characteristics.

MongoDB itself scales well with both partitioning(sharding) and
clustering(replica sets).


The application stack avoids network distribution

of the architecture layers as much as possible
. This means
the en
tire Java stack executes locally in one Tomcat node. Communication with
plugins

may also be

local
access if the plugin is deployed on the same node as the NOD Server.


The entire stack is stateless, if session state must be stored, we use MongoDB. This ena
bles near linear
scalability in the layer above MongoDB

by simple load balancing of the incoming HTTP requests. The exact
load balancing solution should be decided upon in cooperation with the hosting provider.




Performance Bottlenecks

We expect synchron
ous PUD performance to be very good, given enough load balancing nodes the
bottleneck will be the incoming network capacity and the MongoDB performance.


The JMS queue must
however
scale with the
NOD
. If publishing messages to the queue is swamped,
NOD

res
ponse times will suffer. Note that the queue is processed asynchronously, the publishing of a message
should be a
relatively
undemanding job as long as the messages are small
. A long

queue will result in
delayed updates of the Orders in PL4, but should not

affect
NOD

performance.


NOTE: It is currently unclear how the 3rd pary SAM server scales and uses state, this may be a limiting factor
to the scalability of the arhitecture.






Side
6

av
7

Resources

CXF performance:
http://www.ibm.com/developerworks/java/library/j
-
jws14/index.html

MongoDB performance:
http://www.mongodb.org/display/DOCS/Benchmarks

Protobuf

performance
:

https://github.com/eishay/jvm
-
serializers/wiki/


Maintainability

Monitoring

Performance

NOD

will use JAMON, the same performance monitoring tool as PL4 for monitoring HTTP response times.


Testin
g

MongoDB is an extremely flexible datastore, and given the corect configuration it is possible to reserve a
separate MongoDB shard that records all communication on a series of cards without affecting the nodes that
contain production data. This capabilit
y is currently not
planned
. This can also enable
future
integration
between
IO Test Center and the
NOD

for testing and fraud detection.


Security monitoring/auditing

Currently unspecified.


Availability

Planned Downtime

To reduce the planned downtime to a
minimun, the application is bunded as simply as possible. The goal is
one WAR deployed in the application server with a minimum of manual configuration.

In addition there will be database scripts
plugin

deployments to upgrade the entire stack.


Unplanned D
owntime

To avoid loss of critical data during the event of unplanned downtime or connection failures, all non
-
reproducible/non
-
derived data should be handled in a transactional context.


All
orders will be added through PL4 and stored in the Oracle Databas
e and added to the outgoing
NOD

queue in the same transaction. The guaranteed delivery nature of the JMS queue ensures that the PTO can
be sure that the action will be delivered to the
NOD

if the WS
-
order call is successful.


No data will be removed from t
he MongoDB dataset without first adding the corresponding synchronization
message on the queue back to PL4. This means that we should be able to recreate the correct actions with
relevant status based on the data found in PL4 even in the event of a total f
ailure of MongoDB.
Note that
while the data and architecture may allow this, development of administrative functionality is required to
support this.

Security

To be discussed in workshops.


Confidentiality

Integrity

Availability

Authenticity





Side
7

av
7

Non
-
repudiatio
n





Designchoices based on select Usecases

To be discussed in workshops.