OOICI_R2_LCO_CyberPoP_CI_2011-08-30_ver_1x - Confluence

enginestagΔίκτυα και Επικοινωνίες

26 Οκτ 2013 (πριν από 3 χρόνια και 10 μήνες)

81 εμφανίσεις

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

Ocean Observatories Initiative

OOI CI Release 2 Life Cycle
Objectives Review

CyberPoPs & Network
(Build2)

OOI CI
Operations Team


R1 PRR and R2 LCO Review

La Jolla, CA Aug
-
30 to Sep
-
1 2011

1

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

Agenda


CyberPoP

Points


R1 Development & Test Infrastructure


Integrated Observatory Network


CyberPoP

Operational Deployment


Use Case Overview


Accomplishments

2

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

OOI Deployment Topology


Observatory Requirements:

Provide observatory assets to
support measurements of
scientific
processes. Provide
computational and network assets
to enable real
-
time, secure access
and processing for the
observatory users


Engineering Drivers:

P
roximity to observatory assets,
geographic redundancy. High
-
bandwidth, low latency access to
national and international
network peering points. Access to
commercial and academic
compute clouds

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

CyberPoP

‘Points’


Types of Points


Acquisition Point: Collects data (e.g., from marine
environments)


Distribution Point: Distributes data around the
network, provides access to data


Management Point: Enables operator management of
OOI system


CyberPoP can be configured for any of these
purposes, including several at once

4

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

Observatory Acquisition Point


Functions


Data acquisition from marine observatories


Real
-
time data processing and product generation


Data forwarding to Data Distribution Point


Data preservation


Requirements


Protected data facility with built
-
in redundancy


Sufficient
s
torage capacity, network resources, and
computational resources


Connection to marine networks

5

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

Observatory Distribution Point


Hardware environment for the Capability Containers for
data and content distribution


Web services for supporting end users


End users communicate through the Distribution Points,
never directly to the Acquisition Points


Provides layer 2 peering with external network providers


Links to cloud execution and storage providers such as
Amazon EC2 and TeraGrid

6

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

Observatory Management Point


Supports network and resource operations from a central
location


Interfaces to marine management systems


Provides state of the health monitoring activities


shows
availability of resources


Monitors all network traffic, protocols and availability


Generates automated alerts when system capacities are
approaching warning levels or when it detects that
resources are no longer available


7

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

Buildout Sequence

8

Buildout 1

Buildout 2

Commercial
Clouds

Academic

Clouds & HPC

National
Repositories

Buildout 3

Buildout 4

Buildout 5

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

Peering Environment with

National Research & Academic Networks

9

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

Peering Environment with

International Research & Academic Networks

10

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

CyberPoP Functions & Locations

Function

Location

Speed

Initial
Capacity

Aggregate
Cores

Acquisition

Point

Portland

10 Gbps

34

TB

144

Acquisition

Point

Woods Hole

1.54

Mgbs

15 TB

48

Engineering
Environment

San Diego

10 Gbps

34 TB

176

Distribution Point

Seattle

10 Gbps

34 TB

96

Distribution

Point

Chicago

10 Gbps

34 TB

80

Management
Point

San Diego

10 Gbps

18 TB

80

11

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

Server Environment

12


4 Dell R810 Servers


Used in Development
and Test
environments for R1


Duplicate hardware
that will be used in
Staging and
Production for R2

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

Engineering Center

Service Deployment Infrastructure

13

ION

Development

Environment

ION

Testing

Environment

ION

Staging

Environment

ION

Production

Environment

Virtualization

Capacities

Physical Computing

Infrastructure

OOI Program

Application

Environment

Management

Environment

San Diego CyberPoP

Virtualization

Capacities

Physical Computing

Infrastructure

First Article Build of Operation Grade

CyberPoP Design

First Article Build of Production Grade

CyberPoP Design

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

Software Development/Service Maturity Cycle

14

Development

Environment

Testing

Environment

Staging

Environment

Production

Environment

San Diego Engineering Center

Portland CyberPoP

Development Team

Integration & Test Team

Operations Team

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

R1 Application and Middleware Support


Virtual Machine Support


Hardened Centos LINUX: Base VM Image


Cassandra: Multi
-
site, Multi Process Data Store


RabbitMQ: Message Broker


Grails: Web Application Platform


THREDDS: Provides Access to Scientific Data


Nimbus: Elastic Computing Management Framework


ION Capability Containers: OOI Service Container


Virtualization Infrastructure


Citrix (KVM)


VMWare and VCenter

15

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

R1 Software Configurations

16

EPU

EPU

EPU

EPU

EPU

EPU

EPU

ION Services

EPU

EPU

EPU

EPU

EPU

EPU

EPU

ION Services

Test and Development Environments

Cassandra

(multiple instances)

RabbitMQ
(multiple instances)

Grails
(multiple instances)

THREDDS
(multiple instances)

Nimbus Service Nodes

Nimbus 2.7

Nimbus 2.8

KVM/VMM

KVM/VMM

Test Environments

ion
-
beta

buildbot

ion
-
test

Images

cc
-
worker (4 GB)

cc
-
cfchecker (10 GB)

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

Integrated Observatory Network

17

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

OOI Network Deployment


P

or

tland, OR

A

ccess P

oint

Management N

e

tw

or

k

P

aci
fi

c City

, OR

Shore Station

Axial Seamount

N

ode 3

Mid
-
Plat

e

N

ode 5

Hydr

at

e Ridge

N

ode 1

Endurance

Ar

r

a

y

Pioneer

Ar

r

a

y

Argentine Basin


Ar

r

a

y

S

t

ation P

apa

Ar

r

a

y

Southern


Ocean

Ar

r

a

y

Sat

ellit

e

Communications

Mar

ine Oper

ations

Management Point

Sat

ellit

e A

ccess Point

W

oods Hole, MA

A

ccess P

oint

Ir

minger Sea

Ar

r

a

y

1

0 GbE

Int

er

ne

t

T1

Sat

ellit

e

User

Mar

ine

Cyber

R

egional Scale N

odes

Coast

al Scale N

odes

Global Scale N

odes

R

esear

c

h & Educations

F

acilities

End User

Applications

Chicago, IL

Distr

ibution P

oint

Seattle, W

A

Distr

ibution P

oint

Obser

v

at

or

y N

e

tw

or

k

Public Internet

Obser

v

at

or

y

Oper

ations

Management Point

P

or

tland, OR

A

cq

uisition Point

W

oods Hole, MA

A

cq

uisition Point

A

cademic HPC &

Cloud F

acilities

Ex

ecution Point

Commer


cial HPC &

Cloud F

acilities

Ex

ecution P

oint

N

ational Dat

a

R

eposit

or

ies

Ex

ecution Point

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

R2 Network Connectivity Requirements


Provide high
speed
backbone (10 Gbps
)
wide area
network to connect the CyberPoPs located at San Diego,
Portland, Seattle, and Chicago



Provide
T
-
1 connectivity to Woods Hole off of a spur
from Chicago


19

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

Initial Components Required for Build2


Wide Area Network (National Network Infrastructure)


Network Messaging Infrastructure


Based on Layer 2 National Network Infrastructure


Provides direct application
-
level message connectivity between
Capability Containers in ION and at users’ sites


Acquisition Points


Portland


Woods Hole


Distribution Points


Seattle


Chicago

20

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

Network Functional Capabilities


10
-
40 Gbps connectivity to
storage and computing devices


High availability/fault tolerance


Local and Wide Area strategies


WAN 10 Gbps eBGP peering


10 Gbps message processing


1 Gbps network firewall and
intrusion detection


1 Gbps load balancing


Dynamic VLAN support

21

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

CyberPoP

Operational Deployment

22

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

CyberPoP

Constructions


Build first instances of all CyberPoPs in San Diego


Engineering Center as first instance


Follow with SD Management CyberPoP


Full functional capabilities co
-
located


Configure Portland
CyberPoP

for supporting PRR activities


Later differentiate and distribute capabilities geographically


All CyberPoPs
will be
run as a lights
-
out
operations


23

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

Remote
CyberPoP

Equipment Locations



Use secure co
-
location facilities to house racks
and equipment


Pittock Building (Portland Oregon), contracted through
the University of Washington


University of Washington (RSN facility)


Woods Hole Oceanographic Institution


StarLight Facility (Northwestern University)

24

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

Server, Storage and Network Deployment

25

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

San Diego Management Point Assembly

26

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

San Diego CyberPoP Detailed Rack Design

27

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

San Diego CyberPoP Detailed Network Design

28

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

Use Case Overview

29

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

R2 Operational Support

30


UC.R2.55 Manage Help

Ticket

Process problems and request for
services



啃⹒㈮.㘠䵯M楴潲i䥏丠乥瑷潲o

剥獰s湤 瑯t湥瑷潲欠楳獵敳e慦晥捴楮朠
瑨t⁳ 獴敭

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

Accomplishments

31

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

Accomplishments to Date


Deploy and operate OOI Program Operations


Build the Engineering Center Environment


Development and Test Environments


Representative of CyberPoP hardware/software configuration


Management Point


acquire network, servers and software


Continual support for the ION Development team


Deploy, (re)configure needed systems and applications


Provide monitoring and analysis tools


Assist with network performance issues


Gained experience in supporting the operational
requirements for ION

32

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

Operations & Maintenance Support


Create High Availability Environments


Help Desk Support (approx. 1000 tickets)


Immediate responses to tickets


Problems/Issues, request for new services


Creation of Metrics


Availability of resources (
99.969
%)


Servers


Storage


Network components


Software applications


Databases


Coordinated Hardware and Software Upgrades


Minimize outages

33

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

R1 Development Services


Create OS configurations and CentOS imaging


Installation of the hardware


Network connectivity (DNS/DHCP/VPN access)


Initial installation of monitoring tools


Installation of service components with final
configurations in conjunction with developers


Development of deployable scripts/templates


Load Balancer for Cassandra cluster, also NAT services


Authentication support

34

OOI CI R2 Life Cycle Objectives Review

Aug 30
-

Sep 1 2011

Thanks!

35