Integrated Data-Focus Information Environment

fishglugSoftware and s/w Development

Dec 13, 2013 (3 years and 7 months ago)

86 views

Integrated Data
-
Focus Information Environment

Fleet C2 Capabilities

13 November 2013

Bobby Junker

bobby.junker@navy.mil

Head, ONR C4ISR Department

2


2


Operational Imperative
-

Shortening the Kill Chain

‘ Watch’ Chain

MDA

Intel

Surveil

Trk/Char/ID

Recon

Assess

MDT

Assess

Engage

Find

Fix

Track

Target

Engage

Assess

(1) Reduce Uncertainty

Maintain High OPTEMP (2)

Seamless, Transparent, Integrated, Data Centric, Agile

‘Kill’ Chain

(3)
Reduce

Manpower

Legacy Information
-
Based Warfare

3

4

A2AD Operational Imperatives


Dynamic/Optimal Force Integration through seamless, automated, mission


prioritized Combat, C2 & ISR Machine
-
to
-
Machine data distribution in D
-
DIL Comms

Future Information
-
Based Warfare

5

Combat
Systems (CS)
Network

Includes UxV
Common
Control
Services



Configurable


Mediation



Adaptable Rules
Engine

Includes:



Force Discovery


Service



Maestro



IM Services

C2/ISR LAN/ACS
Systems

UxV

Control

C2RPC

FFDS

Universal

Gateway



Adaptable Rules
Engine



Configurable Mediation



RTI Router

Integrated C2, CS, and ISR Construct

In A2/AD D
-
DIL Environment


Goal: Transparency of data and information services across
disparate enclaves in a D
-
DIL environment to support Force
-
Level
A2AD integrated
UxV

planning and execution

Data Exchange Goals

1.
Increased automation of and reduced
timelines for plan
-
act
-
assess
-
replan

cycle


Maximize
information transparency

across
Force while maintaining appropriate level of
information assurance


Minimize application
design complexity

by
defining interoperable middleware services in
each domain

2.
Federated Force Discovery Service


Automated
Force Composition
&
Synchronization


On demand
Federation
Across the Force


Information

Support to Applications

3.
A2AD Enhancements


Maximize effectiveness of
Disconnected
Intermittent Low Bandwidth (DIL) networks


Mission based
Information Prioritization

Combat
System
CS Gateway
C
2
Gateway
C
2
Core Services
C
2
Systems and
Applications
C
2
Information
Management
Two
-
Way
CS/C2 Data
Exchange



Messaging (M)
Data:



tracks



readiness



Bulk (B) Data



ATO



data base
updates



readiness



Etc.



Video Data (V)

ONI

IC Data
Center

DoD/Svcs

MOC

CVN

LCC

LHA

RSC/COCOM

CORE
NODES

EDGE

NODES

Distributed Tactical Cloud Construct

EDGE:
Ultra
-
thin clients, band
-
width
informed, greater mobility, better data
management

TOC

DaaS

Tactical Cloud Construct and Issues

9

COCs

Ships

Naval Tactical Cloud

Mgt

.

Policies

Cloud Mgt

.

Tools

PaaS

Naval

Tactical Cloud

Platform

Naval

Cloud

Platform

U

t

i

l

i

t

y

D

a

t

a

S

t

o

r

a

g

e

Naval

Tactical Cloud

Data Science

Commodity Purchases

Naval

Tactical Cloud

Experimentation

Naval

Tactical Cloud

Widgets

&

Apps

PEOs

/

PoRs

SYSCOMs

/

PEOs

/

ONR

Naval Tactical Cloud

System Engineering

Overall Naval Cloud

Naval

Tactical Cloud

Analytics



ISR

-

LITE Cloud Racks



DCGS

-

A Tactical Edge Nodes

LEVERAGE



IC Gov Cloud



ACS



FFDS

LEVERAGE



C

2

RPC



Magic Mirror



USMC Advanced Analytics

LEVERAGE



ISR

-

LITE UOM



Army UCD



IC GEM

LEVERAGE

FFC

/

MARFORCOM

FLTCYBERCOM

MARFORCYBER

DaaS

IaaS

Naval

Tactical Cloud

Infrastructure

Virtualization


Maintaining information consistency in dynamic, D
-
DIL
comms

environment


Optimization of available band
-
width to the highest
priority information


Distributed, dynamic Identity and Authentication
Management


Software and data security in cloud environment


Appropriate scaling across physical platforms


Data normalization across large heterogeneous data
types


Automated information prioritization


Real
-
time / Near real
-
time operations







Tactical Cloud Technical Issues

10

10/30/2013
-

v0.11

NAVAL DATA SPACE

The Naval C2 Big Data Challenge

11

Historical
Data
Current
Data
Predictive
(Future)
Data
Unit
Group
Force
Green
Blue
White
Red
C2 Support Systems

C2 Decision Makers

Bring in all possible Data to support
non
-
expert, C2 Decision Makers


Leverage Historical/Forensic data to bring
expertise to the “layman”


Leverage Future/Predictive data to bring
enhanced understanding to the “layman”

OBJECTIVE:

Optimize the Naval
Force

vs. a Near
-
Peer Adversary

Force

Commander

Platform

Commanders

Group

Commanders


Reduce Uncertainty


Maintain Op Tempo


Reduce Man Power

ISR

METOC

Cyber

C2

Log

. . .

Optimize multi
-
mission
Naval Platforms/Sensors
across the Force

10/30/2013
-

v0.11

Leveraging the Complete

Naval “Data Space”

12

Historical

Data

Current

Data

Predictive (Future)

Data

Unit

Group

Force

Scope of Naval
“Data Space”

as it is today

Expand Naval “Data
Space” with NTC

Naval

“Data Space”



Extend to Force Level


More powerful computation
enables greater span of C2
optimization


Extend from today’s Unit level
optimization to
Group and
Force
level optimization



Extend to Predictive Data


More powerful analytics enable
generation of Predictive
(Future) data


Extend from today’s Current
data to store
Predictive

data
sets afloat



Extend to Historical Data


Enhanced storage enables
much greater data storage
afloat


Extend from today’s Current
data set to store
Historical

data sets afloat

Operations


How often will batches be processed?


How will new data types be accommodated?


Will adding a new tagging scheme and new
taskings

require
recertification?


How will ad hoc queries be handled?


Can the inefficiency of HIVE et. al. be overcome?


What are the personnel implications?


Will new skills be needed


Will more SCI clearances be needed?



10/09/2013
-

v0.14

Role of Forensic Data in C2

14

10/30/2013
-

v0.11

Naval Problem Characterization

15

Entity Types/Entity Complexity

Volume of Data

Few

Many

Small

Huge

Naval

Patterns of Life

/Forensics

Naval

Planning

Naval

Effects

Increasing Level of Data Science Design Difficulty

Increasing Level
of

Computational Resources

RTRG

Naval

Readiness

Naval

Situational

Awareness

Naval

Predictive

Analysis/

Forecasting

Naval

Target ID/

Classification

Architecture


What are the Big Data?


What are the appropriate processes?


Should some data be processed before ingest?


Should structured data be kept in a RDBMS?


What processes can be parallelizable?


Does cloud technology scale down?


How many real nodes can be / need to be furnish?


Do we need shared memory?


Computational power appropriate for
each platform or
node?

10/30/2013
-

v0.11

Naval Data Science Challenges

17


Scoping Challenges:


What is the data that is important to Naval Operations?


What metadata do we use to describe Naval Data?


What are the entities that are important to Naval Operations?


What are the relationships that need to be established?


Between Entities and the Data?


Between Entities?


Alignment Challenges:


How do we get the Naval Community to adopt common Domain Models (e.g.,
metadata types, entity types, and relationship types)?


How do we develop broadly useful indexing strategies across all domains?


How do we ensure that the Naval Community’s Data Science Approach is
interoperable with other Communities (e.g., IC, Army, Air Force, . . .)

10/30/2013
-

v0.11

Data Science Framework

18


A Data Science Framework defines how metadata, entities, and
relationships are structured within the cloud platform


A Data Science Framework consists of the patterns and
constraints that are placed on how metadata, entities, and
relationships are stored and used


The major elements of a Data Science Framework are:


The nouns that are used to define metadata and entities


The verbs that are used to define relationships


The indexing strategies


The design of the Data Science Framework is critical because it
has a significant effect on:


How hard/easy it is to ingest data into the cloud


How hard/easy it is to create the desired metadata, entities, and relationships


How hard/easy it is to write analytics


How hard/easy it is for applications to interact with the metadata, entities, and
relationships

10/30/2013
-

v0.11

Data Science Methodology

19

Operational

Use Cases

Analytic

Capabilities

Computer Science

Implementation

What are the operational use cases
that you want to address?

What are the analytic capabilities
that need to be developed to
support the use case?

What are the entities that you need to
define to support your analytic
questions?

What analytics algorithms are required
to extract the information needed to
provide the analytic capabilities?

How is everything to be implemented
on top of the cloud software platform?

What data sources need to be ingested
and how do they need to be indexed?

Operational

SME

Data

Scientist

Entity

Models

Data Sources

and Ingest

Analytic

Development

Computer

Scientist

12/14/2012
-

v0.01


Scoping Use Cases for Experimentation

20

Analytical
Questions
Operational
Use Cases
Object
Models
Analytic
Development
D
-
DIL
/
A
2
AD
NTC Platform
Fleet
&
Marine Corps
Data
Scientists
Computer
Scientists
What are the operational use
cases that you want to
investigate
?
What are the analytic questions
that need to be answered to
support the use case
?
What are the object models that
will best allow us to answer the
analytical questions
?
What analytics are required to
extract the information needed
by the use case
?
What software services are
required to make everything
work and how do integrate the
Cloud with the CS
?


Data Ingest services


Data Query and Pub
/
Sub services


Analytic Services


Cloud Security


CS
-
C
2
data exchange
(
HAG
)


In
-
line NRT analytic jobs


Batch analytic jobs


Object Modeling Frameworks


Object Model definition


Object Model services
Data
Ingestion
What data sources need to be
injested to address the analytic
questions
?


Data ingest scripts


Data visibility tagging


Data labeling
SLICE

Need to define Use Case
Slices

through the Naval Big Data
Problem Space to develop
standards, patterns and practices

12/14/2012
-

v0.01


Example of an Operational Scenario

21

Data Analytics performed across Warfare Areas


Data
-
driven decision guides shaped by
commander’s intent, historical decisions/results,
and COA/ECOA input across all available data


Collaborative CS/C2/ISR/Environmental
information through data exposure and
advanced analytics (including cross
-
domain)


Adaptive fleet
-
wide data sharing in a non
-
DIL and
DIL environment


Autonomous predictive SA across warfare
domains


Automated consolidation of
UxV

capabilities/
status, to align/de
-
conflict tasking


Automated data security tagging at ingest


ASW
IAMD
EXW
Data Cloud
Enhanced
EXW Analytics
Data Cloud
Enhanced
IAMD Analytics
Data Cloud
Enhanced
ASW Analytics
Acoustic Data
MAD Data
Ocean Data
Submarine Intel
Radar Data
Tracks
METOC
SIGINT
INT SUMs
Message Traffic
SIGINT
METOC Data
Acoustic
Data
Tags
Non
Acoustic
Tags
Blue
ASW
Platforms
Tags
Blue
ASW
Sensors
Tags
Ocean
Data
Tags
Blue
ASW
Weapons
Tags
Blue ASW
Combat
Systems
Tags
Enemy
Sub OOB
Data
Tags
Enemy
Sub Perf
.
Data
Tags
SIGINT
Data
Tags
Comm
.
Shipping
Data
Tags
Cloud Data Services
Data Ingest
….

22


22


Operational Imperative
-

Shortening the Kill Chain


C
onsistent Cloud enabled SA
/C2/Execution
tools & CS/C2/ISR
integrated data set across the Force


Tactical Cloud enabled rapid
/smooth integration with legacy feeds,
processes, and
uses
both kinetic and non
-
kinetic in D
-
DIL environment


Cloud enabled r
apid
force composition changes to meet dynamic planning
and execution
fo
r
success simultaneously across all A2AD missions


Seamless, fully integrated Fires across the Joint Force

‘ Watch’ Chain

MDA

Intel

Surveil

Trk/Char/ID

Recon

Assess

MDT

Assess

Engage

Find

Fix

Track

Target

Engage

Assess

(1) Reduce Uncertainty

Maintain High OPTEMP (2)

Seamless, Transparent, Integrated, Data Centric, Agile

‘Kill’ Chain

(3)
Reduce

Manpower

Summary


There are major S&T, Acquisition, and Policy challenges associated
with bringing Cloud Computing capabilities to the Fleet, particularly
for A2AD conditions


ONR has developed a plan for systematically addressing the
challenges via its Limited Technology Experiments (LTE) process
that will result in:


Naval Tactical Cloud Reference Implementation


Naval Tactical Cloud Architecture and Design Guidance


Naval Tactical Cloud Data Models and Ingested Data Sources


Preliminary Naval Tactical Cloud Analytics, Widgets, and Apps


The framework supports Combat System, C2, logistics, personnel, ISR
(DCGS
-
N/MC),
etc

data structures in context of providing integrated
warfighting

decision (C2) support


The proposed effort could significantly accelerate the transition
of
cloud
technology to the Fleet


Reduce acquisition timelines and risk


Inform manpower planning and training development


Facilitate co
-
evolution of cloud technology related CONOPs/governance