Challenges to Adoption

goodyearmiaowΜηχανική

18 Νοε 2013 (πριν από 3 χρόνια και 11 μήνες)

73 εμφανίσεις

Formal Model
-
Based Development
in Aerospace Systems:

Challenges to Adoption

Mats P. E. Heimdahl

University of Minnesota Software Engineering Center

Critical Systems Research Group

Department of Computer Science and Engineering

University of Minnesota

and a Plea for Help

Domain of Concern

How we Develop Software

Concept
Formation

Requirements
Specification

Design

Implementation

Integration

System

Unit Test

Integration
Test

System Test

Object Code

Test

Analysis

Model
-
Based Development

Specification

Model

Visualization

Prototyping

Testing

Code

Analysis

Properties

Model
-
Based Development Tools


Commercial Products


Esterel Studio and
SCADE Studio from
Esterel Technologies


Rhapsody from I
-
Logix


Simulink and Stateflow
from Mathworks Inc.


Rose Real
-
Time from
Rational


Etc. Etc.



System

Specification/Model

How we
Will

Develop Software

Concept
Formation

Requirements

Implementation

Integration

Properties

Analysi
s

Integration
Test

Syste
m
Test

Specification
Test

What Does Industry Want?

Better / Safer

Cheaper

Faster

Model
-
Based Development

Examples

Company

Product

Tools

Specified & Autocoded

Benefits Claimed

Airbus

A340

SCADE

With Code
Generator



70% Fly
-
by
-
wire Controls



70% Automatic Flight Controls



50% Display Computer



40% Warning & Maint Computer



20X Reduction in Errors



Reduced Time to Market

Eurocopter

EC
-
155/135
Autopilot

SCADE

With Code
Generator



90 % of Autopilot





50% Reduction in Cycle Time

GE &
Lockheed
Martin

FADEDC Engine
Controls

ADI Beacon



Not Stated





Reduction in Errors



50% Reduction in Cycle Time



Decreased Cost

Schneider
El
ectric

Nuclear Power
Plant Safety
Control

SCADE

With Code
Generator



200,000 SLOC Auto Generated
from 1,200 Design Views





8X Reduction in Errors while
Complexity Increased 4x


US
Spaceware

DCX Rocket

MATRIXx



Not Stated





50
-
75% Reduction in Cost



Reduced
Schedule & Risk

PSA

Electrical
Management
System

SCADE

With Code
Generator



50% SLOC Auto Generated



60% Reduction in Cycle Time



5X Reduction in Errors

CSEE
Transport

Subway
Signaling System

SCADE

With Code
Generator



80,000 C SLOC Auto Generated



Improved
Productivity from
20

to 300 SLOC/day

Honeywell
Commercial
Aviation
Systems

Primus Epic
Flight Control
System

MATLAB
Simulink



60% Automatic Flight Controls



5X Increase in Productivity



No Coding Errors



Received FAA Certification



Problem 1

Believing Testing Can be Eliminated

Testing will always be a crucial
(and costly) component

How we Develop Software

Concept
Formation

Requirements
Specification

Design

Implementation

Integration

System

Unit Test

Integration
Test

System Test

Analysis

Object Code

Test

System

Specification/Model

Testing Does not go Away

Concept
Formation

Requirements

Implementation

Integration

Properties

Extensive Testing
(MC/DC)

System

Specification/Model

It Simply Moves

Concept
Formation

Requirements

Implementation

Integration

Properties

Extensive Testing
(MC/DC)

System

Specification/Model

Do it the Right Way

Concept
Formation

Requirements

Implementation

Integration

Properties

Analysi
s

Integration
Test

Syste
m
Test

Specification
Test

Unit Test

Example: ADGS
-
2100 Adaptive
Display & Guidance System

Requirement

Drive the Maximum Number of Display Units
Given the Available Graphics Processors

Counterexample Found in 5 Seconds!

Checking 573 Properties

Found 98 Errors

883 Subsystems


9,772 Simulink Blocks


2.9 x 10
52

Reachable States

Remedy


Be honest about the capabilities of model
-
based development and formal methods


Done right, provides outstanding requirements,
models, analysis, etc., etc.


May greatly reduce the effort spent in testing



Problem 2

Believing the Model is Everything

The model is never enough

Modeling is
so much
fun

Properties

Specification/Model

Modeling Frenzy

Concept
Formation

Requirements

Implementation

Integration

How do we
know the model
is “right”?

System

System

Specification/Model

Do it the Right Way

Concept
Formation

Requirements

Implementation

Integration

Properties

Analysi
s

Integration
Test

Syste
m
Test

Specification
Test

Unit Test

Remedies


Recognize the Role of Software Requirements


The model is not everything


Development Methods for Model
-
Based
Development Badly Needed


Model
-
Based Software Development Process


Develop Tools and Techniques for Model,
Properties, and Requirements Management


Develop Inspection Checklists and Style Guidelines
for Models


Problem 3

Trusting Verification

To really mess things up,

you need formal verification

Model Checking Process

Does the system

have property X?

Model

Engineer

SMV

Automatic Translation

SMV Properties

Properties

Yes!

SMV

Spec.

Automatic Translation

Model Checking Process

Does the system

have property X?

Model

Engineer

SMV

Automatic Translation

SMV Properties

Properties

SMV

Spec.

Automatic Translation

Counter Example

No!

Property or Model: Who is Right?

AG
(Onside_FD_On
-
> Mode_Annunciations_On)

The Mode Annunciations shall be turned on

when the Flight Director is turned on

AG
( (Is_This_Side_Active & Onside_FD_On)


-
> Mode_Annunciations_On)

If this side is active, the Mode Annunciations shall

be turned on when the Flight Director is turned on

If this side is active and the Mode Annunciations are off, the Mode
Annunciations shall be turned on when the Flight Director is turned on

AG
( ! Mode_Annunciations_On
-
>


AX
((Is_This_Side_Active & Onside_FD_On)


-
> Mode_Annunciations_On)))

Translated All the “Shalls” into
SMV Properties

Analysis Process Steps


All properties verified (!), or…


Counterexamples found for
some properties


Simulate counterexample in
MBD environment and make
corrections to:


model


properties


requirements


assumptions

(invariants)

Formal
Analysis Model
MBD Model
Shall
Statements
CTL
Properties
Corrections
Corrections
Corrections
Corrections
Create Model
(
Manual
)
Formalize
Properties
(
Manual
)
Merge
(
Automated
)
Translate
(
Automated
)
Simulation
/
Corrections
Formal
Verification
Remedies


Develop techniques to determine adequacy of model and
property set


How do we know they are any “good”


Techniques for management of invariants


How do we validate the assumptions we make


Methodology and guidance badly needed


Tools with training wheels


“Verification for Dummies”


All we need is one high
-
profile verified system

to fail spectacularly to set us back

a decade or more

Model Checking Process

Why?

Guru

Does the system

have property X?

Model

Engineer

SMV

Automatic Translation

SMV Properties

Properties

SMV

Spec.

Automatic Translation

?

Problem 4

Believing One Tool Will Be Enough

To be effective, we need a suite of

notations and analysis tools

(and the ability to continually integrate new ones)

Original Tool Chain

RSML
-
e

NuSMV Model Checker

PVS Theorem Prover

Rockwell Collins/U of Minnesota

SRI International

RSML
-
e

to NuSMV

Translator

RSML
-
e

to PVS

Translator

Conversion to SCADE

Design

Verifier

SCADE

Lustre

NuSMV

PVS

Safe State

Machines

Simulink

Simulink

Gateway

StateFlow

SPY

Esterel Technologies

MathWorks

University of Minnesota/Rockwell Collins (NASA LaRC Funded)

University of Minnesota (NASA IV&V Funded)

Reactive Systems

Esterel Technologies

MathWorks

SRI International

University of Minnesota/Rockwell Collins (NASA LaRC)

University of Minnesota (NASA IV&V)

Current(?) Tool Status

Design

Verifier

SCADE

Lustre

NuSMV

PVS

Safe State

Machines

SAL

ICS

Symbolic

Model Checker

Bounded

Model Checker

Infinite

Model Checker

Simulink

Simulink

Gateway

StateFlow

Reactis

SPY

Three Conjectures


No one modeling language will be
universally accepted, nor universally
applicable


No one verification/validation tool will
satisfy the analysis needs of a user


Languages and tools must be tested on
real world problems by practicing engineers


Preferably in commercial tools

Translation


with no IL

Effort

=
m

*
n

High quality translations



Lustre ++

poly

tables



SCADE



RSML
-
e



PVS

poly’



SMV



C

m modeling languages n target languages

poly

Translation


with IL

Effort

=
m

+
n

Low quality translations



Lustre IL



Lustre ++

poly

tables



SCADE



RSML
-
e



PVS

poly’



SMV



C

m modeling languages n target languages

poly

A Proposed Framework (Van Wyk)


Based on techniques from
extensible programming
languages
, specifically attribute grammars
extended with
forwarding
.


Hypothesis:



An
extensible language

may serve as a
host language

for domain specific extensions (to construct new
modeling languages),


while
forwarding

enables the
feasible construction

of high
quality translations from source specification languages
to target analysis languages.


Provided to spur discussion only! There may be
better solutions.

Translation


with lang. exts.

Effort

=
m

+
n

+
Σ

t I
High quality translations



Lustre Host



Lustre ++

poly

tables



SCADE



RSML
-
e



PVS

poly’



SMV



C

m modeling languages n target languages

forwarding

poly

pvs_trans (t2)

pvs_trans (t1)

c_trans (t3)

forwarding

forwarding

c_trans

smv_trans

pvs_trans

Remedies


Next generation tools must allow easy
extension and modification of notations to
meet domain specific needs


They must allow easy construction of high
-
quality translations from modeling notations
to analysis tools


They also must enable controlled reuse of
tool infrastructure to make tool extensions
cost effective

Problem Summary


Believing Testing Can be
Eliminated


Believing the Model is
Everything


Trusting Verification


Believing One Tool Will
Be Enough

Thank You


Rockwell Collins


Steven Miller


Michael Whalen


Alan Tribble


Michael Peterson


NASA Langley


Ricky Butler


Kelly Hayhurst


Celeste Bellcastro


NASA Ames


Michael Lowry


NASA IV&V Facility


Kurt Woodham (L3
-
Titan)


My Students at Minnesota


Anjali Joshi


Ajitha Rajan


Yunja Choi,


Sanjai Rayadurgam


Devaraj George


Dan O'Brien


Opinions in talk are mine.

Do not blame the innocent.

Discussion

For More Information


Michael W. Whalen et. al., Formal Validation of Avionics Software in a Model
-
Based Development Process, Formal Methods in Industrial Critical Systems
(FMICS’2007), July 2007.


Steven P. Miller, Alan C. Tribble, Michael W. Whalen, Mats P. E. Heimdahl,
Providing the Shalls, International Journal on Software Tools for Technology
Transfer (STTT), Feb 2006.


Michael W. Whalen, John D. Innis, Steven P. Miller, and Lucas G. Wagner,
ADGS
-
2100 Adaptive Display & Guidance System, NASA Contractor Report
NASA
-
2006
-
CR213952, Feb. 2006. Available at
http://hdl.handle.net/2002/16162
.


A lot of good reading at
http://shemesh.larc.nasa.gov/fm/fm
-
collins
-
intro.html



Eric Van Wyk and Mats Heimdahl. Flexibility in modeling languages and tools: A
Call to Arms. To appear in Software Tools for Technology Transfer.