Integrated Risk and Knowledge

Integrated Risk and Knowledge
Management for Exploration

David M. Lengyel

Risk and Knowledge Management Officer

Exploration Systems Mission Directorate

NASA Headquarters

dlengyel@hq.nasa.gov

2

Introduction

Objective:

Introduce and discuss
real work process improvements

that utilize organizational management innovations and
leverage existing ESMD information technology

resources

Customer:

The ESMD civil servants and contractor work force

Goal:

No nonsense, straight
-
up, “Real Deal” approaches to
make your job more fun and make you more effective



-

Work more effectively and efficiently



-

Make better
–

more risk informed decisions



-

Manage risks in a proactive fashion


Not another burdensome management / administrative demand

on your time …….. This stuff will save you time !

dlengyel@hq.nasa.gov

3

Why Integrate Risk and Knowledge Management?

Practice 1: Establish
Pause and Learn

Processes


Practice 2: Generate and Infuse Knowledge
-
Based Risks (KBRs)


Practice 3: Web
-
Enabling High Performance Teams


Practice 4: Provide Knowledge Sharing Forums


Practice 5: Promote Experienced
-
Based Training

Designing a

complex architecture of hardware, software, ground and
space
-
based assets to return to the Moon and then go on to Mars will
require:

1)
an effective strategy to learn from past lessons, and

2)
a set of inter
-
related practices to generate and share knowledge for reuse
as we progress forward.


ESMD risk and knowledge management
communities have embarked on an effort to integrate risk and knowledge
management (KM) over the lifecycle of the Constellation and Advanced
Capabilities Programs using a set of inter
-
related strategies, which
include:

dlengyel@hq.nasa.gov

4

ESMD and Stealth KM

“Knowledge
-
enabling processes (i.e. process improvement) will

lay a solid KM foundation for future organizational evolution and

help align KM with business
-
based goals and objectives


Improving processes also provides an opportunity to deploy

supporting KM tools and techniques such as collaboration or

CRM software and processes
–

this can give important momentum

to knowledge workers, and can help them to work in a more

holistic and community
-
based way


Bottom
-
line
: Process evolution equals culture evolution”










Niall Sinclair








Author of
Stealth KM


dlengyel@hq.nasa.gov

5

Practice 1: Pause and Learn

“The Need to Pause, Reflect, and Learn”

PaL is modeled after the Army After

Action Review (AAR) process by

Dr. Ed Rogers KM Architect at the
GSFC.


The idea is to create a learning event

at the end of selected critical events in

the life of a project. End of project

reflections are good but are too
infrequent for the organization to learn
in a timely manner.


PaL meetings are intended to be
integrated into the project life cycle at
key points as a natural part of the
process. PaL meetings

are structured and facilitated by
specialists who are not project
members




https://ice.exploration.nasa.gov/ice/site/km/pal/

dlengyel@hq.nasa.gov

6

Attributes of a PaL

Informal, facilitated roundtable discussion (1/2 hour to full day)

–
Includes moderator and rapporteur

–
Focuses on tasks and goals that were to be accomplished


Not for attribution

–
Does not judge success or failure (not a critique)

–
Encourage employees to surface lessons


Focused on particular area of project life (phase and function)

–
Management PaL, Technical PaL, Conceptual PaL, et. al.

–
Team participation may vary, depending on PaL focus and objective


Maximizes participation

–
Primary benefactors are the
participants themselves

–
More project activity can be recalled and more lessons shared


Must be conducted
inside a project’s schedule
, not outside or later

–
Recall of key details more likely and insights can be immediately
applied

–
Affirms learning as integral part of project life cycle

dlengyel@hq.nasa.gov

7

PaL as a Process

Step 1

–
Identify when PaLs will occur

–
Determine who will attend PaLs

–
Select Moderators, Rapporteurs

–
Select potential PAL sites

–
Review the PAL plan


Step 2

–
Review what was supposed to happen

–
Establish what happened (esp. dissenting points of view)

–
Determine what was right or wrong with what happened

–
Determine how the task should be done differently next time


Step 3

–
Review objectives, tasks, and common procedures

–
Identify key events

–
Rapporteurs collect
ALL

observations

–
Organize observations (identify key discussion or teaching points)


1

Adapted from United States Army Manual:
A Leader’s Guide To After Action Reviews

dlengyel@hq.nasa.gov

8

Practice 2: Knowledge
-
Based Risks

Knowledge
-
Based Risk

n.

1.

A risk based
on lessons learned from previous experience.
2.

A closed risk with documented lessons
learned appended.
3.

A means of transferring
knowledge in a risk context.

Definition

dlengyel@hq.nasa.gov

9

•
Start Early

•
Need to Capture, Learn From and Repeat Successes
--
Need to Learn from and
Prevent Failures, Mishaps, Near Misses

•
There was a limited number of useful lessons learned in the NASA Lessons
Learned Information System database. The good ones are masked by the
hundreds of poor ones, so that extensive effort is required to sort them out.

•
Lesson Learned
–

Well
-
understood mechanisms for “transfer of knowledge”
during Program development are crucial to a successful long
-
term Program.

•
Flow
all

applicable Lessons Learned into Requirements, Processes, and Plans.
Institutionalize the Use of Lessons Learned.

•
Provide Sufficient Resources, Planning, and Management Support to Analyze
and Incorporate Lessons Learned. NASA and Contractor Must Work Together

•
The best lessons learned for running a major program should be captured in a
living handbook of best practices
. New lessons learned should be screened for
applicability, and included in the handbook.

Lessons Learned on Lessons Learned


ESMD Is Taking a New Approach to Lessons Learned…..

dlengyel@hq.nasa.gov

10

Knowledge
-
Based Risks Strategy

The ESMD KBR strategy is intended to convey risk
-
related lessons learned and best practices to ESMD
personnel. This strategy integrates the existing
Continuous Risk Management (CRM) paradigm used
at NASA with knowledge management
--
with the
primary focus on integrating transfer of knowledge
through existing work processes and not adding an
additional burden to the workforce to incorporate
new KM tools and concepts.



5


4


3


2


1


1 2 3 4 5

CONSEQUENCE

L

I

K

E

L

I

H

O

O

D

dlengyel@hq.nasa.gov

11

KBR Process Flow Chart

IDENTIFY
Identify risk issues and
concerns
Control
Make risk decisions
TRACK
Monitor risk metrics and
verify
/
validate mitigation
actions
PLAN
Decide what
,
if anything
,
should be done about
risks
ANALYZE
Evaluate impact
/
severity
,
probability
,
timeframe
,
classify and prioritize risks
Candidate
KBR
?
Replan Mitigation
Program
/
project data
(
Metrics Information
)
Close or Accept Risks
Invoke Contingency Plans
Continue to Track
Risk Status Reports on
:
Risks
Risk Mitigation Plans
Statement of Risk
Risk Classification
:
Likelihood
Consequence
Timeframe
Risk Prioritization
Risk Source Checklists
,
Lessons Learned
,
Review of
WBS and Requirements
,
Hazard Analysis
,
FMEA
,
FTA
,
KBRs
,
etc
.
Risk Data
,
Test Data
,
Expert
Opinion
,
Lessons Learned
,
Review of WBS and
Requirements
,
Hazard
Analysis
,
FMEA
,
FTA
,
Technical Analysis
,
KBRs
,
etc
.
Resources
No
No Action
Research
Watch
Acceptance Rationale
Mitigation Plans
Yes
Notify Cognizant
RMO
Brief to ESMD
RMWG
Accepted
?
No
No Action
Yes
Upon Closure
:
ESMD RMO coordinate
documenting lessons learned
,
video nugget
(
s
)
,
and associated
documentation
KBR documenbted in Risk
Database and KBR Portal
Pushed to Subscribers
•
KBRs are documented as a
requirement in ESMD Risk
Management Plan
–

this flows
down to Levels 2 and 3 (Program
and Project) Risk Management
Plans

•
Leverages Standard Continuous
Risk Management (CRM) paradigm

•
Adds filtering process for
identifying significant risks as KBR
candidates

•
Captures “What worked
–

OR
–

Didn’t work in terms of mitigation
strategies

•
Provides Infusion Process for
KBRs Back Into Risk Management
and other processes Which current
NASA Lessons Learned System
lacks

dlengyel@hq.nasa.gov

12

KBR Criteria

Risks that are "Candidate KBRs" should meet several of the
following criteria (listed in order of importance):


(1) Were mitigated (not accepted or watched)


(2) Will likely appear again in other programs / projects


(3) Included a particularly effective mitigation approach /
implementation, or an error in mitigation planning or
implementation could have been avoided


(4) Was on the performing organization's Top Risk List at some
point during the life cycle


(5) Was owned (and/or worked on) by a particularly knowledgeable
person who could serve as a "expert" on the risk topic

dlengyel@hq.nasa.gov

13

Application of Risk Management Assurance Mapping

Multiple KBR Capture Points
–

Multiple Delivery Points

Internal and External to ESMD

HRP
PCB
ETDP
PCB
HRP
PCB
MO Project
EVA Project
GO Project
ELP Project
1
st Stage
ECB
/
ERB
Up Stage
ECB
/
ERB
UpStgEng
ECB
/
ERB
ARES
-
V
ECB
/
ERB
Flight T
&
I
ECB
/
ERB
Orion Project
SM RB
LAS RB
PP
&
C RB
T
&
V RB
S
&
MA RB
VIO RB
CM RB
Constellation Program
ESMD Level
1
Field Centers
Mission Support Offices
Mission Support Offices
SSC CD
Gilbrech
SSC Risk
Management
MSFC CD
King
MSFC Risk
Management
LaRC CD
Roe
LaRC Risk
Management
KSC CD
Kennedy
KSC Risk
Management
JPL CD
Elachi
JPL Risk
Management
JSC CD
Coats
JSC Risk
Management
GSFC CD
Weiler
GSFC Risk
Management
GRC CD
Whitlow
GRC Risk
Management
DFRC CD
Petersen
DFRC Risk
Management
Tools
Tech
.
Dev
.
R
.
Boyle
Suit
Cupples
Vehicle
B
.
Moses
S
&
MA
Carrington
PP
&
C
B
.
Johnson
SE
&
I
J
.
Davis
TV
&
V
J
.
Patrick
EVA RMO
Mulligan
Project Controls
SE
&
L Level III
CEV Mgmt
Ops
&
Int
.
Kunz
Business Mgmt
Staff Office
LV Mgmt
Ground Systems
CC
&
C
Risk Manager
Risk Manager
Anderson
Risk Manager
Risk Manager
Risk Manager
Risk Manager
Risk Manager
C
3
PO Program
Flight Test
&
Int
Taylor
ARES
-
V
Cook
Vehicle Integration
Reuter
First Stage
Burt
Upper Stage
Davis
Upper Stage Engine
Kynard
NASA Administrator
M
.
Griffin
Deputy Administrator
S
.
Dale
Associate Administrator
R
.
Geveden
Office of Safety
and Mission
Assurance
B
.
O’Connor
Office of Chief
Engineer
C
.
Scolese
Space Operations
Mission Directorate
W
.
Gerstenmaier
Science Mission
Directorate
C
.
Hartman
Aeronautics Research
Mission Directorate
L
.
Porter
PA
&
E
S
.
Pace
Cx PM
J
.
Hanley
NASA
HQ
Adv
.
Cap
.
C
.
Walz
JSC
PP
&
C
B
.
Waddell
T
&
V
W
.
Arceneaux
Ops Integration
R
.
Castle
SE
&
I
C
.
Hardcastle
SR
&
QA
L
.
Hansen
MSFC
NASA Centers
Board
/
Panel
Inline Report
NASA Exploration Systems Mission Directorate
Risk Integration Critical Process Map
LEGEND
Orion Project Manager
R
.
Hanson
ARES
-
I
Project Manager
S
.
Cook
Ground Operations
T
.
Talone
Mission Operations
D
.
Webb
DIO
N
.
Woodward
Orion Contractor
Lockheed Martin
Orion VIO
Gahring
Orion CM
Piatel
Orion SM
Free
Orion PP
&
C
Floyd
Orion LAS
Stover
Orion T
&
V
Hurlber
Orion S
&
MA
Thiessel
Orion
Chief Engineer
Prime
Contractor
GSFC
SI
&
M
M
.
Allen
OCE
H
.
Lyles
First Stage Contractor
ATK
Upper Stage Contractor
PWR
EVA
Lutz
JPL
GRC
KSC
LaRC
ARC
Transition
J
.
Olsen
SOMD
Transition
J
.
Kearn
RMO
Perera
Information Flow
Project
Risk Manager
Curiel
Risk Manager
Schwarz
Risk Manager
Holsomback
Risk Manager
Nise
Risk Manager
Pacera
Risk Manager
See
Risk Manager
Wood
Risk Manager
Manuel
Risk Manager
Peterson
Risk Manager
Schubert
Risk Manager
Wood
/
Cole
Risk Agent
Chief Engineer
SE
&
I
LV RMO
Risk Manager
Mabry
Risk Manager
Ogle
Risk Manager
Coleman
Risk Manage
r
Cole
/
Wood
Risk Manager
Kulpa
/
Coleman
Risk Manager
Kulpa
/
Wood
Program Manager
Brasfield
Risk Manager
Sanofsky
Program Manager
Risk Manager
Burkevics
Logistics Mgmt
Project
Mgmt
Req
&
Prj Int
–

Lvl IV
Proj
.
Support
Risk Manager
Risk Manager
Risk Manager
Lvl
5
FITO RM
Kline
MO
RM
Smith
OSPP
CMO
R
.
Williams
OSPP
Risk Management
CMO
Risk Management
Liskowsky
ARC CD
Wordon
ARC Risk
Management
SCaN Program
ISS Program
Space Shuttle Program
JICB
ESMD
QPMR
ESMD
“HQ Only”
RMWG
CxCB
Cx RMWG
PRCB
PRCB
Mission
Operations Risk
Review Board
EVA Risk Review
Board
ELP Project
Control Board
ELP Risk
Management
Panel
Vehicle
Integration
CB
/
RB
Ground
Operations Risk
Review Panel
Ground Operation
Project Control
Board
Orion
/
CEV
Project Control
Board
CTN PCB
Program Manager
Ondrus
Risk Manager
Naves
Program Manager
Hale
Risk Manager
Turner
Program Manager
Suffedin
Risk Manager
Lurtomski
C
3
PO
PCB
I
&
A RRB
CFO
G Sykes
OCIO
J
.
Pettus
General Counsel
M
.
Wholley
Integrate Ent
.
Mgmt
Program
B
.
German
Innovative
Partnership Prog
.
D
.
Comstock
Strategic
Communications
R
.
Hopkins
TCB
Inspector General
R
.
Cobb
OI
&
M
T
.
Luedtke
Communication
Planning
R
.
Hopkins
Education
J
.
Winterton
Legislative and
Intergovmnt Affairs
J
.
Winterton
Public Affairs
D
.
Mould
Diversity and Equal
Opportunity
B
.
Manuel
Human Capital
Management
T
.
Dawsey
Internal Controls
and Mgmt Systems
J
.
Henn
Procurement
S
.
Goddard
Small Business
Programs
G
.
Delgado
Security and
Program Protection
D
.
Saleeba
NSSC
R
.
Arbuthnot
Infrastructure and
Administration
O
.
Dominguez
OI
&
A RM
Hoyt
ETD Program
ETD Program
Manager
Risk Manager
HR Program
HR Program
Manager
Risk Manager
LPR Program
LPR Program
Manager
Risk Manager
LRO
Project Manager
LCROSS
Project Manager
CxSD
Risk Manager
D
.
Lengyel
Risk Manager
Risk Manager
Risk Manager
Risk Manager
Risk Manager
N
.
Saiyeed
SMC
PMC
OMC
Risk Manager
Turner
Config
.
Mgmt
.
Risk
Resrc
Sch
IT
LM RB
Lvl
5
FITO RM
Cagle
Risk
Mgr
Risk
Mgr
Risk
Mgr
SR
&
QA Chair
Chair
Advanced Proj
.
Risk Manager
DIO RM
Chair
DIO RM
Chair
Chair
Program Manager
Lindenmoyer
Risk Manager
Eminger
Chair
Chair
Chair
Program Manager
C
.
Lacefield
Risk Manager
L
.
Piccola
Advisor
Advisor
Chair
R
R
R
R
Chair
Chair
R
R
R
ESMD
DRWG
AA Chair
Program and
Institutional
Integration
R
.
Keegan
Chief of Staff
P
.
Morell
Exploration Systems
Mission Directorate
S
.
Horowitz
Deputy AA
D
.
Cooke
Tri
-
Chair
Tri
-
Chair
Tri
-
Chair
Administrator
,
Chair
Co
-
Chair
Co
-
Chair
Activity Vector
R
Risk Review Board
dlengyel@hq.nasa.gov

14

Knowledge
-
Based Risks (Continued)

2006

2007

2008

2009

2010

2011

2013

2012

2006

2007

2008

2009

2010

2011

2013

2012

2006

2007

2008

2009

2010

2011

2013

2012

Ground Ops

Orion

Ares
-
1

Number of Documented Risks

Number of Documented Risks

Number of Documented Risks

KBRs

Standalone Program Risks

in ARM

Integrated Program Risks

in ARM

Integrated and Knowledge
-

Based Risks in ARM

More access to risk information is
required to close “knowledge gaps”

KBRs will become a living reference
over time as risks are identified,
mitigated and closed

Ground Ops

Orion

Ares
-
1

Ground Ops

Ares
-
1

Orion

dlengyel@hq.nasa.gov

15

Knowledge
-
Based Risks (Continued)

NASA Standard WBS

ARM allows automated delivery of new KBRs

dlengyel@hq.nasa.gov

16

Knowledge
-
Based Risks (Continued)

•
Embedded 3
-
8 min
Video Nugget with
Transcript

•
Related Knowledge
Bundles

•
Related Content
–

reports, documents, etc.

•
Threaded discussion
(blog) feature to be
added to comment on
each KBR

•
Hosted on ESMD R&KM
portal

https://ice.exploration.nasa.gov/ice/site/km/menuitem.d785354e3fd1af6d86dba1102a55d40c/

dlengyel@hq.nasa.gov

17

•
The design of the LRO propulsion tanks was influenced by a number of
factors including launch vehicle characteristics. The Delta II Expendable
Launch Vehicle’s (ELV) spin stabilized upper stage made the Nutation Time
Constant (NTC) a key parameter in assessing the stability of the spacecraft.
The uncertainty in predicting the effects of liquid propellant motions and the
relatively large propellant load and mass fraction for the LRO tank resulted in
the identification of a potential risk. Close coordination and communication
with all levels of management early in the design trade study process allowed
for the effective mitigation of the risk and provided additional lunar
exploration opportunity.

LRO Spacecraft

Atlas V Booster

Delta II Booster

LCROSS

Spacecraft

First Closed Risk KBR
–

Lunar Recon Orbiter

dlengyel@hq.nasa.gov

18

Practice 3: Web
-
Enabling High Performance Teams

Knowledge resides with people and is often lost via actions like:



-

Downsizing


-

Retirements


-

Shuttle Transition


-

People Movement


The notion of using communities of practice (CoPs) as a fundamental
building block of a solid knowledge management system was reviewed
by ESMD.


The implementation of CoPs
—
especially discipline
-
specific, top
-
down
approaches
—
demands change, as evidenced by the lack of support for
participating in these types of CoPs and the explosion of virtual teams in
ESMD’s wiki environment
.


ESMD has developed a strategy to enhance team communication and
performance in a
virtual environment

through the promotion of both
workgroup and wiki functionality tools.


dlengyel@hq.nasa.gov

19

High
-
Performance Team Development Lifecycle

Web
-
Enabled Team Support (Continued)



People

Process

Products

Policing

Community Building

Seed Posting

Team Start

Along the entire lifecycle, Team
Development is rooted in
defining the people, process,
and product

People
–

Define the team with

roles and responsibilities

Process
–

Consider how the

Web
-
enabling software fits

into the context of daily

work process

Product
–

Define your

team’s purpose and

give it a direction

Evaluating top
-
down and bottoms
-
up implementation options (possibly employ both
approaches).
Both approaches will utilize:


•

External and internal NASA virtual team benchmarking (CIA, Pfizer, Fidelity
Investments, Knowledge Leadership Forum, etc.)

•

Collaboration Workshops

•

ESMD business process analysis / business unique knowledge architecture
templates

•

Systems Engineering, Functional Engineering Disciplines, Sub
-
System
Management, CxP SMA, Acquisition/Procurement, Human Resources

dlengyel@hq.nasa.gov

20

Web
-
Enabling ESMD Teams in a Secure Environment

The PBMA toolkit
provides ESMD Teams
with a secure
environment to share
documents, conduct
threaded discussions,
polls, manage
calendars, locate
expertise, collaborate
and learn. Over 30
ESMD Teams are
serviced by PBMA.

The ESMD Wiki
provides secure
collaborative
functionality within
the ESMD Integrated
Collaborative
Environment (ICE).
ESMD Wiki spaces
now number over 130

https://ice.exploration.nasa.gov/confluence/dashboard/

https://secureworkgroups.grc.nasa.gov/

dlengyel@hq.nasa.gov

21

Practice 4: Knowledge Sharing Forums

ESMD Alumni Sharing Events:



•
These events bring in alumni from Apollo, Space Shuttle, and other
programs to discuss their experiences and lessons learned

•
This is an extensive, under
-
utilized knowledge base

•
ESMD has invited selected alumni to brown bag lunches and other
lessons learned forums


Knowledge Sharing Workshops and Seminars:


•
At Knowledge Sharing Workshops, senior project leaders share their
insights, what they learned and what they might have done differently
based on a recent project experience.

•
These workshops are attended by emerging project leaders who want
to understand the wisdom of successful project managers


APPEL Master’s Forums:


•
Conducted twice annually

•
ESMD has and will continue to participate in these events

•
See: http://appel.nasa.gov/node/19


dlengyel@hq.nasa.gov

22

Practice 5: Experienced
-
Based Training

Project Management and Engineering Training

•
Already conducted by APPEL and NESC Academy

•
ESMD will focus its efforts in training on leveraging the existing
infrastructure of training courses throughout NASA

•
ESMD will help shape existing courses by providing ESMD
-
related
experiences, gleaned from case studies, KBRs, and other sources of
lessons

•
NESC Academy: http://www.nescacademy.org/home/index.aspx

•
APPEL: http://appel.nasa.gov/



Case Studies

•

ESMD will facilitate the development of case studies that will help
transfer the context of program/project decisions to the workforce and
emerging leaders

•
Senior ESMD managers would help shape the content based on their
experiences and leadership

•
Case studies will make existing training programs more relevant and
useful to upcoming ESMD leaders who participate

dlengyel@hq.nasa.gov

23

KM Practices and Tool Integration

Portals

Engineering /
Management
Training

Knowledge
-
Sharing
Forums

Wikis/CoPs

dlengyel@hq.nasa.gov

24

ESMD Risk & KM Teaming

ESMD is teamed with:

•
Space Operations Mission Directorate

•
Office of Safety & Mission Assurance

•
Office of the Chief Engineer

•
NASA HQ Institutions & Administration

•
Academy of Program / Project & Engineering Leadership

•
NASA Engineering & Safety Center (NESC) Academy

•
JSC Chief Knowledge Officer

•
GSFC Chief Knowledge Officer

•
MSFC / Ares Chief Knowledge Officer

•
Constellation Program

•
ISS Program

•
SSP Program

•
Pratt
-
Whitney
-
Rocketdyne Chief Knowledge Officer

•
Lockheed
-
Martin

•
ATK
-
Thiokol

•
United Space Alliance, Office of the Chief Engineer

•
The Aerospace Corporation

•
NASA Alumni Association

•
Defense Acquisition University
–

Best Practices Clearinghouse



dlengyel@hq.nasa.gov

25

Summary

“ESMD faces exciting opportunities and formidable challenges. To
reduce risk and apply knowledge more effectively, ESMD should
integrate its KM, RM and OL initiatives into a comprehensive plan
that will accomplish more with less bureaucracy. The goal is not
compliance with detailed processes and procedures but
compliance with intent: the intent to learn, to share and probe
every possible angle so ESMD’s missions have the highest
possible chance of success. ESMD must take risks with ‘eyes
wide open’ and ‘minds fully engaged’ at every decision, every
trade and with every residual risk.”

From: Strategy for Exploration Systems Mission Directorate

Integrated Risk Management, Knowledge Management

and Organizational Learning Whitepaper

Dave Lengyel & Dr. Ed Rogers

dlengyel@hq.nasa.gov

26



Questions?

Contact Information:


dlengyel@hq.nasa.gov

Office: (202) 358
-
0391

Cell: (202) 253
-
1762