Atmospheric Science Data Center

lilactruckInternet και Εφαρμογές Web

4 Δεκ 2013 (πριν από 3 χρόνια και 4 μήνες)

64 εμφανίσεις

Atmospheric Science Data Center

HPC User Forum


Michael Little

September 10, 2009

Broomfield, CO

9/9/09

NASA ASDC HPCU Forum

Page:
2

Agenda


September 09, 2009


Recent NASA Science Mission Directorate Activities


SMD Modeling Needs Study 2008/2009


Tsengdar Lee

http://www.hec.nasa.gov/news/reports.html


Decadal Survey Data Systems (DSDS)

Martha Maiden


June, 2009

http://dsds.nasa.gov


Atmospheric Science at NASA Langley Research Center


Science Focus


Atmospheric Science Data Center


NextGen Net
-
centric Operations


Atmospheric Science @LaRC


Remote Sensing Instruments &in situ measurements


Earth Radiation Budget (ERBE, CERES)


40 years of internally consistent measurements (<1% err)


TRMM, AQUA, TERRA, NPP*, NPOESS*


Atmospheric Chemistry (SAGE)


Clouds and aerosols (CALIPSO)


Analysis of measurements


Radiation and Clouds in Climate


Clouds properties


Data Fusion is key to all higher order data products

9/9/09

Page:
3

NASA ASDC HPCU Forum

Earth Radiation Budget

CERES Measures the Earth’s Radiant Energy Balance

02S00220
-
T0051b
-
154
02S00220
-
T0051b
-
154
Figure 1
-
1. Earth Radiation Balance
9/9/09

Page:
4

NASA ASDC HPCU Forum

CALIPSO LIDAR Data

9/9/09

Page:
5

NASA ASDC HPCU Forum

Climate Science Needs for Computation


High accuracy (reduce error to <1% in measurement)


Calibrate, calibrate, calibrate


Current weather measurements are ~15%


Re
-
processing long time series


Internal consistency


100x (100 days of observations re
-
processed in 1 day of compute
time)


Cannot guarantee re
-
processing will give same data


Easy to Use Development Environment


Physical Scientists frequently have weak computer science skills (no
formal training)


Resistance to change


Scientists are more valuable than compute cycles


MATLAB and IDL/ENVI

9/9/09

Page:
6

NASA ASDC HPCU Forum

Atmospheric Science Data Center


ESDIS Digital Active Archive Center


Data Sets


ERBE, CERES, CALIPSO, SAGE, TES, MISR


6 Functions: Ingest, Archive, Produce, Distribute, Report, Support


Produce/re
-
process instrument data products



CERES, CALIPSO, MISR, SAGE


Distribute Released Data Products


Science Computing Facility


Local research and missions(300 users)


Develop/test code to produce data products


Instrument calibration (including pre
-
launch ground cal)


Data Product calibration


Analysis of data


Commercial Applications: MatLab and IDL/ENVI

9/9/09

Page:
7

NASA ASDC HPCU Forum

Recent Changes at ASDC


Replace legacy systems with computational environment with high I/O to
disk systems


Big endian and little endian due to code migration issues


Increase processor availability with minimum system
-
wide outages


Fiberchannel interconnect


Improve reliability, maintainability


Reduce SysAdmin workload


Put all interesting data online (2PB)


IBM x3950, x3650, P6 processors


IBM DS4800, DS5300 Disk Arrays with lots of redundancy


Brocade M48k Director


Sun Grid Engine


Improve stewardship and data protection


All sites have local environmental risks


Key products stored offsite on tape


Tape archive and backup


SunSTK 8500 with Quantum StoreNext


Code conversion took > 5 years


Scientifically identical data products

9/9/09

Page:
8

NASA ASDC HPCU Forum

64 x86 SMP

256 GB

ASDC/SCF Fiberchannel Connectivity

Fiberchannel Switch

64 x86 SMP

256 GB

56 P6

256GB

56 P6

256GB

56 P6

256GB

56 P6

256GB

220TB

DS4800

220TB

DS4800

220TB

DS4800

220TB

DS4800

220TB

DS4800

64 x86 SMP

128 GB

16 x86 HS12

64 GB

Common

SANhead

(NFS)

(AFP)

(Samba)

ASDC JBOSS

Server

& TS
-
1

& Ingest

ASDC SDP Production

64 x86 SMP

256 GB

56 P6

256GB

SCF Cluster

(Access through

Sun Grid Engine)

220TB

DS4800

220TB

DS4800

SCF Disk Array

Instrument

Instrument

ERBE
-
like

SRB/Power

SRB/Power

SARB

14 4P6@8GB

SCF Interactive Processors

ASDC Disk Archive

(R/W Production)

(R/O SCF)

Consodine

TISA

SOFA

Clouds

Clouds

TISA

220TB

DS4800

WWW

WWW

64 x86 SMP

128 GB

Cloud Modeling

Cluster (Xu)

64 x86 SMP

128 GB

ECS DUE

Processing

Db

Server

Db

Server

Tesla

Tesla

3755

3755

GPU Experiment (YongHu)

Sim

Server

FC

Monitor

LB1

LB1

9/9/09

Page:
9

NASA ASDC HPCU Forum

Projected Work at ASDC


Improve performance of processing and storage


Tune SGE


Fiberchannel switch configurations


Improve quality of SysAdmin Support


Train Users to leverage investment


Model computational environment


SIMSCRIPT III


Improve Distribution Function


Google Earth


Permit direct access of data by non
-
local research community


Service Oriented Architecture (SOA)


9/9/09

Page:
10

NASA ASDC HPCU Forum

NextGen National Airspace System


Trajectory Based Operations requires Common Operating
Picture among
everyone

using NAS


Avionics Roadmap created in November, 2008


Scope is curb
-
to
-
curb instead of gate
-
to
-
gate


NextGen Joint Planning and Development Office


Created by Public Law 108
-
176, VISION 100 Act


Coordination function, not PMO


3x air traffic, safer, cheaper with less environmental impact by 2025


http://www.jpdo.gov


Industry participates via NextGen Institute to avoid Conflict of Interest


Governance by Collaboration among Partner Agencies and
Companies


FAA, Defense, Homeland Security, Commerce, NASA, EPA


Airlines, Airports


9/9/09

Page:
11

NASA ASDC HPCU Forum

NextGen Net
-
centric Operations


Leverage DoD experience with Net
-
centric Operations


JPDO Net
-
centric Operations Division leads collaboration


USAF CIO, Electronic Systems Center, AF Weather Agency


FAA SWIM PM, FAA Telecommunications Infrastructure


NOAA Office of the Federal Coordinator for Meterological Services and
Supporting Research


NWS Aviation Weather Office


NASA LaRC, GSFC and eventually ARC and DFRC


DHS Chief Technology Officer and CIO


NASA, DARPA and NSF supporting R&D


Industry partnership via Net
-
centric Working Group


Membership cleared through NextGen Institute


https://jpdo
-
ncowg.larc.nasa.gov/wiki/index.php/Main_Page


Lead business process re
-
engineering, info exchange design


Info owners to implement information exchanges


Ensure access to information for all parties in NAS


Semantic Service Oriented Architecture (SOA)


Machine
-
to
-
machine readability is essential

9/9/09

Page:
12

NASA ASDC HPCU Forum

NextGen Semantic SOA


4D Weather Cube is first implementation


2013 Initial Operational Capability for Info Exchange Services


Development at MIT Lincoln Labs and NCAR


FAA NNEW and Weather in the Cockpit programs


NWS Aviation Weather Office


Other Business Areas with Information for Exchange


Safety


Aircraft


Airports


Flight/Flow


Unmanned Aerial Vehicles


Key Issues


Cultural Change


NCO Governance


Cybersecurity


Infrastructure: computation and data communications



9/9/09

NASA ASDC HPCU Forum

Page:
13

-
Integrated Surveillance

-
Environmental Impact

-
Airports

-
Special Use Airspace