Management of Mission Failure Risk

downtownbeeΜηχανική

18 Νοε 2013 (πριν από 3 χρόνια και 7 μήνες)

65 εμφανίσεις

© The Aerospace Corporation
2012

Logic
-
Quantitative Framework for
Decisionmaker's

Management of Mission Failure Risk


USC


CSSE Annual Research Review Workshop

7 March 2012

Dr. Sergio
Guarro

Distinguished Engineer, The Aerospace Corporation

Background and Context of The Aerospace Corporation
Mission Assurance and Risk Framework


Space missions are unforgiving


The rule of the game is essentially “one strike and you are out”


i.e., minimal
possibility exists for remedying problems during mission execution


Volume and mass constraints also limit the amount of redundancy that can be used
as insurance against failures


The possibility of failure must be understood and managed at the lowest levels
of system design detail


Because of the above Mission Assurance and Mission Risk Assessment
processes are given great attention and priority in the range of activities our
company executes on behalf of our U.S. Government space program
Customers


Comprehensive Program Offices’ mission assurance task plans and assessment
processes


Special issues addressed with specialized Engineering & Technology Group support


Aerospace specialists’ analyses in support of Customers’ independent review team
assessments

2

APR / ASMR Framework & Process


The Aerospace Corporation (“Aerospace”) President Review / Senior Management Review (APR / ASMR)
process is the concluding synthesis of a full cycle of assurance and risk assessment applied
to supported
National Security Space (NSS) programs,
to provide the decision
-
maker with the analytical means to judge
and manage the risk of mission failure


Structured integration of Risk Assessment (RA) and Management (RM) information produced by program
contractor(s) and Government / Aerospace Program Office is key to success of APR/ASMR process


The logic


quantitative risk framework presented here is the result of the most recent development to
provide reference guidance for the APR / ASMR risk assessment processes


The guidance is documented in a corporate Technical Instruction published by The Aerospace Corporation Corporate
Chief Engineer Office and supported by more detailed documentation produced by The Aerospace Corporation Systems
Engineering Division

3

PO RM
Process

Indpdt
.

Review Team

RA Process

APR / ASMR

RA
Process

Contractor RM Process

Indpdt
. Program

Reviews

Focus on

Mission Risk
Only

Address both
Programmatic &

Mission Risk

Objectives and Flow of Logic
-
Quantitative Risk
Framework


Key objectives

A.
Clear identification of key factors and events that can determine a mission impact

B.
Assessment of risk in objective probability and mission
-
impact dimensions


Avoid qualitative definition of likelihood and consequences that are intrinsically
subject to different interpretations by different audiences

C.
Separation of risk definition and assessment from decision process


Value judgment of risk is the decision
-
maker’s, not the assessor’s, responsibility



Typical execution flow

4

Program

Office

Eng. &Tech.

Group

MA Plan

& Scope

Preliminary

Identification

& Evaluation

Risk

Screening

Risk

Scenario

Definition

Risk

Assessment

& Rating

Risk

Communication

& Decision

Support

Risk Identification Using Mission Assurance Baseline


Risk identification proceeds from the basic concept of
risk as deviation from “mission assurance
baseline” (MAB)
:


Each
space system mission item (SSMI)
within the assessment scope is evaluated from this perspective


A potential
SSMI risk item
is identified as a significant deviation from the desired level of quality in a set of
reference
mission assurance attributes


Guidance document s define the
set of attributes to be evaluated
and the
severity criteria
to determine whether any
existing deviations are significant enough to call for the formal definition of an associated risk



This risk identification concept is the application of a general concept that relates risk directly to the
Aerospace MA (Mission Assurance) processes

i
-
th
SSCI
System
Acquisition Gates
SRR
SDR
PDR
. . .
APR/FRR
Baseline
Evaluation
Evidences
System
Acquisition Development
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
System
Risk ID & Assessment
5

Risk Item Screening


Apply filter to preliminarily
-
identified potential risks


Apply full assessment and quantification technique to mission
impacting major technical risks


6

Preliminary

Risk
Identification

100s of non
-
quantitative
potential risks
preliminarily
identified
(e.g., in
MA Baseline task
executions)

no

yes

Cost &
Schedule
Risks

Mission
Impact?

no

yes

Lower
Level
Issues

yes

Lower
Severity
Risks

Technical
risks

10
-
20 Mission
Impacting Major
Technical Risks

no

Define &
Assess

Risk

Scenario

Significant
Deviation from
Baseline?

Technical

Impact?


A
risk scenario
is defined to initiate the analysis / assessment portion of the process for post
-
screen
mission risks


Definition:
A
RISK SCENARIO

is a system or mission condition that can be formally described as a cause
-
effect sequence of events the occurrence of which may cause a mission risk impact and associated
consequences to be realized.


The reference risk scenario identifies in logic event sequence diagram (ESD) format the
key chance
events / conditions
that may affect the outcome of a given risk in terms of probability and consequence
severity


This may include risk control measures expressly introduced by a program to counter an identified
risk:


preventive control measures (PCMs)
,

when executed successfully, eliminate altogether the potential mission
impact of a given risk


mitigative

control measures (MCMs)
, when executed successfully, reduce the potential mission impact of a
given risk by some predicted amount that can be quantified as a consequence reduction factor

Risk Scenario Definition

7

SCENARIO EVENT SEQUENCE DIAGRAM (ESD)
-

including PCM & MCM events


Initiating
Event

occurs


Intermediate

Event Y

occurs

PCMs

are present

and

successful

Intermediate

Event X

occurs

MCMs

are present

and

successful

Unmitigated

Mission

Impact

is realized

Mitigated

Mission

Impact

is realized

No Mission

Impact

is realized

no

no

yes

yes

yes

no

no

no

yes

yes

Risk Rating


Once a reference risk scenario has been defined and expressed in
standard ESD form risk can be assessed by estimating:


Likelihood / probability of initiating event


Conditional likelihood / probability of intermediate events


Including probability of success of PCMs and MCMs


Severity / magnitude of mission performance shortfall resulting from any
mission impact scenario outcomes


Performance shortfall reduction factors associated with MCM
-
event
successful outcomes


The guidance documentation provides the simple formulations by
which risk scenarios can be quantified and rated in summary
“probability of consequence severity” form, using the above ESD
quantification parameters


8

Legend

Performance
Parameter

Shortfall

(% of required
value)

Performance
Requirement

Iso
-
consequence

calibration line

1

No
Missn
.

Value

0.25

0.50

0.75

Mission Shortfall
Metric (MSM)

Image

Resolution

Shortfall

40%

10 %

20%

30%

0 %

0

Full
Missn
.

Value



Data Rate

Shortfall

80%

20 %

40%

60%

0 %

Rating of Multiple Performance Consequence Effects


When

a risk involves consequences in multiple mission performance

dimensions a
combined Mission Shortfall Metric (MSM) needs to be developed


This can be done by mapping hypothetical shortfall magnitudes relative to individual key
performance parameters into a single MSM scale, i.e., essentially defining a simple “mission
utility function” (in the potential shortfall direction)

9

Risk Communication and Decision Support


The recommended format of risk communication is a “probability vs. consequence
severity” risk map on which appropriate areas of risk have been pre
-
identified for
reference according to decision makers’ input and directives


Uncertainty in both probability and consequence magnitude is also displayed

10

Estimates

of individual

Mission Risks

with low uncertainty

Estimates

of individual

Mission Risks

with significant
uncertainty

Use in Decision
-
Making: Power Distribution Shorts
Scenario Example


Risks flow from initiator through intermediate events to impacts


The initiator is not the risk


Include and show “delta effect” of any preventive or
mitigative

control
measures (PCMs, MCMs)


Benefits:


Easier to understand and more thorough “risk statement”


Clearly identifies key events and factors, which strongly influence risk
outcome


Shows effectiveness of prevention and mitigation


Makes it easier to resolve disputes


11

Short

Occurs

Short Is in
Unprotected
Section

Short Load


> 20 Amps

Mission Loss

PCM1

Add Extra

Insulation

PCM2

Add Diode

Protection

MCM1

Sectorize

Solar Panel

Solar Panel

Wiring Insulation
Cracked / Frayed

Short Load

6 to 20 Amps

Short Load


< 6 Amps

Mission
Degradation

No / Minor
Impact

Potential

Controls

Power Shorts Scenario Example Details

12


Scenario
Outcome

Probability
of Scenario
Outcome

Mission Shortfall

Probability

Deviation from
SPF Control
Requirements



































No Impact 1

x1%





No Impact 1

97.22%



Short
Develops


NO (P1)


























YES





























No Impact 2

x2
%



11% to 64% Mission
Shortfall

0.11%



Short Is in
Protected
Section


NO (P2)




















YES





















A





Mission Loss

2.68%

Short to
Structure vs.
Wire


STRUCTURE
(P3)






















WIRE















B























A















No Impact 3

x3%





Open Circuit
Follows


YES (P4)


















NO





















Large Bus Hot
Load
a
to
b
A

z1%



Addtl. Short
from Melting
Insulation in
Yoke


YES (P5)
















NO















Bus Hot Load
c
to
d
A

y1%





























B















No Impact 4

x4%





Open Circuit
Follows


YES (P6)


















NO





















Large Hot
Load
e
to
f
A

z2%



Additl. Short
from Melting
Insulation in
Nearby Wire


YES (P7)
















NO















Hot Load ~
g
A

z3%

























Total No
Impact

97.22%





















Mission Shortfall

Probability





















No Impact 1

XX%

























































































11% to 64% Mission
Shortfall

YY%





































Mission Loss

ZZ%





















Power Shorts Scenario Results



RISK SCENARIO OUTCOMES

MISSION OUTCOMES

Large Amp
Short to Bus

Moderate Amp
Short to Bus

Moderate Amp
Short to
Wire(s)

Large Amp
Short to
Wire(s)

No Significant
Mission Impact

Mission
Performance
Shortfall

Total Mission
Loss

[a

to b
A]

[c
to
d
A]

[e
to
f
A]

[~
g
A]



[m
to
n%
shortfall]



Probability
Distribution
Parameters

















Mean

-

-

-

-

-

-

-

5
th

Percentile

-

-

-

-

-

-

-

Median (
50th
Percentile)

-

-

-

-

-

-

-

95th Percentile

-

-

-

-

-

-

-


Assessment results suggested that some risk control measures would be
warranted, if their introduction were technically feasible


14


Defined and formulated to support Decision
-
makers’ assessment
and management of risk of mission failure


Clear, unequivocal definition / description of all “selected risks”


“Reference Scenario” Format


Distinction between assessment, display/communication, and decision
-
support aspects of risk process


Assessment via objective, quantifiable metrics


Quantification recommended for
objectivity, not to project impression of
precision


Strong recommendation to explicitly display assessment uncertainty

In Summary: Key Points of Logic
-
Quantitative Risk
Framework

15

Backup Charts

Example of MA Baseline Attributes for Risk Identification

16



APR MISSION

-

ASSET BASELINE ATTRIBUTES



BASELINE DEFINITION



DEVIATION FROM BASELINE



"PROCESS" / "PRODUCT"



NATURE of D

EVIATION



Modest



Significant



Large



Process



Product



1



Design Assurance Factors



1.1



Residual issues with SSMI design / engineering and



interface specifications



No residual issues



X



2



Manufacturing Assurance Factors



2.1



Indication of SSMI manufacturing technology issues



No indication of issues



X



2.2



Deviations from of SSMI manufacturing quality control



processes



No deviations



X



3



IT&E Factors



3.1



SSMI TLYF exceptions



No unassessed / unjustified



exceptions



X



3.2



IT&E process deviations from SSMI requirement



verification objectives



No deviations





















































X



3.3



SSMI requirements, including interface and reliability



requirements, not verified by results of IT&E process



(test, analysis, or demonstration)



No IT&E results deviations from



requirements



X



4



Operations Readiness Factors



4.1



Product evidence of SSMI integration and mission



readiness issues



No evidence of issues



X



4.2



Deviations from SSMI anomaly resolution plans



No deviations



X



4.3



Residual liens against SSMI on Orbit Testing and



Operations Certification requirements



No residual liens





















X



5



MA Disciplines Factors



5.1



Deviations from Specifications and Standards



requirements applicable to SSMI



No deviations



X



6



Other Factors



6.1



SSMI

-

specific issue #1 (describe)



No issue



TBD















6.2



SSMI

-

specific issue #N (describe)



No issue



TBD



Initiating Event Identification in Risk Scenario ESD


The initiating event in a risk scenario is identified according to the
nature of the baseline deviation(s) initially identifying the risk

17

SSMI BASELINE FACTORS TO BE EXAMINED TO DEFINE RISK SCENARIO INITIAL CONDITION


Requirements

Deviations ?



Design
Deviations ?



Manufacturing

& Assembly

Deviations ?



IT & E

Deviations ?



Operational

Readiness

Deviations ?



MA Discipline

Specs &
Stds

Deviations?


If evidence of deviations exist, is it in
process

or
product attributes
?

If any deviations exist, is their
magnitude
moderate
,
significant
, or
large

(
M
,
S
, or
L
) ?

RISK
-
SCENARIO INITIAL CONDITION

DEFINED IN TERMS OF ANSWERS

TO ABOVE QUESTIONS

Examples of ESD

Templates Provided in Risk Guidance

Document


SSMI product

exhibits

[ M / S / L ]

deviation from

[req./des./… ]

baseline


Is SSMI deviation

fully controlled

by built
-
in system design
features (e.g., redundancy,
operational options, etc.) ?


Is SSMI deviation

fully controlled

by PCMs added after risk
identification ?

Is SSMI deviation

mitigated by MCMs added
after risk identification ?

Unmitigated

Mission

Shortfalls

are realized

Less severe

Mission

Shortfalls

are realized

No Mission

Shortfalls

are realized

no

no

yes

no

yes

yes

ESD Template for Risk Scenario Driven by SSMI
Product Attribute
Deviation

18


SSMI process

exhibits

deviation from

baseline


Is a
SSMI product

deviation
from

baseline produced

as a result ?


Is SSMI product deviation

“moderate” (M)?

Enter “product

deviation” ESD

w/ “S” deviation

condition

Enter product

deviation ESD

w/ “M” deviation

condition

No Mission

Shortfalls

are realized

yes

no

yes

no

yes

Is SSMI product deviation

“significant” (S)?

Enter “product

deviation” ESD

w/ “l” deviation

condition

no

ESD Template for Risk Scenario Driven by SSMI
Process Attribute
Deviation