Aircraft Weapons Systems Testing

AGARD
o
graph 300
-

YY


1
-

1








Aircraft Weapons Systems Testing


AGARDOgraph 300
–

YY

Draft V 0.1


RTO SCI FT3




Wing Commander Malcolm G. Tutty, MEng, FIE(Aust), FRAeS

Air Force Headquarters

RAAF Base Edinburgh, SA 5111

Royal Australian Air Force and University of South Austra
lia




Preliminary Draft for discussion and other National Input


AIRCRAFT WEAPONS SYSTEMS TESTING



AGARDOgraph 300
-
YY


1
-

2



section

content

written by

Title page

AGARDograph number

Title

Author names

Sponsorship (SCI, FT3)

executive

Introduction to RTO

Organization description

executive

Introduction to 160 &
300 series

Purpose of series

FT3 team

Executive Summary (English)

Relation of subject matter to the
mission of NATO. Some
abstract
-
like qualities

FT3 champion

Executive Summary (French
copy of English version)

Relation of subject matter to the
mission o
f NATO. Some
abstract
-
like qualities

executive

Acknowledgements

(
optional
)

institutions visited and individual
contributions.

author

Foreword

(
optional
)

Content determined by author or
champion.

author/champion

Table of Contents

Compiled by publisher.

(author
must provide one for editing
purposes.)

publisher

List of figures, tables

Compiled by publisher.

(author must provide one for
editing purposes.)

publisher

Symbols list

symbols and nomenclature used
in the volume

author

Introduction

First chapte
r of text

author

Body of text

Including figures and tables (all
provided electronically)

author

Conclusions or concluding
remarks

if any

author

References & Bibliography

including article titles and source,
authors, date

author

Annexes

(
optional
)

Conte
nt determined by author.

author

Annex listing 160 & 300 series
previous publications

Titles, AGARDograph number,
author(s), publication date

executive

Report documentation page

compiled by executive

executive


including abstract

author


Author name(s),

titles, telephone
number, FAX, mail, & email
addresses

author





AGARD
o
graph 300
-

YY


1
-

3


Aircraft Weapons Systems Testing

AGARDOgraph 300
–

YY


RTO SCI FT3

Wing Commander Malcolm G. Tutty, MEng, FIE(Aust), FRAeS

Director Simulation, Trials & Ranges
-

Air Force Headquarters

RAA
F Base Edinburgh, SA 5111

Royal Australian Air Force and University of South Australia


malcolm.tutty@defence.gov.au



All aeroplanes share problems in common. When the airspeed is too slow they lose
lift, reach a minimum level flying speed,
or they stall, sometimes quite violently.

An aircraft designer, like a test pilot, must be streetwise in all matters which affect conduct of an effective flight with t
he highest
level of safety for it’s intended pu
rpose. He or she must be practical above all.

The test pilot and the designer must know their history
–

of what has been attempted before, and what has failed and why?
Nature has its own laws and never breaks them.





S
tanton

(2001), pg xix.

For they learned that true safety was to be found in long previous training, and not in eloquent exhortations uttered when th
ey
were going into action.

Thucydides, History of the Peloponnesian Wars, v, c. 404 BC


Figure 1 RAAF F/A
-
18 hea
ding downtown Baghdad, March 2003



AIRCRAFT WEAPONS SYSTEMS TESTING



AGARDOgraph 300
-
YY


1
-

4


ABSTRACT

This AGARDOgraph provides an overview of the contemporary approaches used for the ground
and flight testing of aircraft weapon systems.

1.0

INTRODUCTION

1.1

The contemporary conventional wisdom is that on the battlefiel
ds and battlespaces of the
near future our soldiers, sailors and airmen will be presented with ever increasingly voluminous
and often conflicting data from multiple sources and will face ‘cognition overload’. The very
nature of asymmetric warfare
1

requir
es that such data and information must be rapidly processed,
coordinated and systematic responses to these threats coming from all dimensions be exploited
effectively with synchronised, appropriate and balanced responses, Corbin et al, (2007).
As noted
in

AGARDOgraph 300
-
xx
, there has been a revolutionary shift

in the focus of the profession of
arms. The shift has occurred away from the stand
-
alone platform
-
centric view to that of a
capability management construct that is to be network
-
centric, interopera
ble and effects based. A

key question being asked by all interested parties is what confidence do we have in the most
important parts of our joint ‘network enabled’ forces being able to operate as planned whilst
retaining the level of interoperability betw
een all these families of systems at acceptable levels of
cost, schedule and performance risks.

1.2


The conduct of key validation and verification (V&V) activities and the ground and flight
testing conducted during the systems engineering activities is centra
l to answering these questions,
rationally and independently
. This article will discuss the importance of rigorous, all
encompassing research into what has gone before, undertaking the test planning, conduct and
reporting expected of today’s weapon system
s test organisations and what the future will entail.

2.0

BACKGROUND

Now, there are two ways of learning to ride a flying machine;

-

if you are looking for perfect safety, you will do well to sit on a fence and watch the birds; but if you wish to learn you m
ust
mount a machine and become acquainted with its tricks by actual trial.
Wilbur Wright, Miracle at Kitty Hawk

-


2.1

During all successful integration programs, the ground and flight testing of weapon systems
is fully integrated with the capability developmen
t and systems engineering conducted. In general,
the competencies needed by test and laboratories to undertake responsible testing are as described
by ISO/IEC 17025 (1999). For aircraft stores clearance and certification activities as discussed at
AGARDO
graph 300
-

xx,
this also involves responsible test organisations addressing the minimum
acceptable certification requirements and the test and evaluation (T&E) methods agreed to by all
the US and Allied Services initially in
MIL
-
HDBK
-
244
, Guide to Aircraf
t Stores Compatibility

and MIL
-
STD
-
1763 (1984),
Aircraft/Stores Certification Procedures

and more recently with MIL
-
HDBK
-
1763 (1998),

Aircraft Stores Compatibility: Systems Engineering Data Requirements and
Test Procedures
. Both of these documents provid
e the detailed engineering and test background



1


A military term
originally referring to war between two or more actors, or groups of actors, whose relative power differed by a
significant amount. Contemporary military thinkers tend to broaden this original meaning to include asymmetry of s
trategy or
tactics; today "asymmetric warfare" can describe a military situation in which two
belligerents

of unequal power interact and attempt
to take advantage of their opponents'

weaknesses.

Wikipedia (2007)


AIRCRAFT WEAPO
NS SYSTEMS TESTING




AIRCRAFT WEAPONS SYSTEMS TESTING

1
-

5

needed for most integration activities. As aircraft systems head to greater integration levels and
service oriented architectures there is a need for a broader, more holistic context that will be
explored in

this article.

2.2

Delimitation.

This article purposely does not address the following: certain aspects of the
safety testing of munitions, such as insensitive munitions, rough handling, storage, accelerated
aging, or transportation tests: tests of basic ma
terials or piece parts, such as transistors, and
integrated circuits; and munition effectiveness.

3.0

SYSTEMS ENGINEERING
AND AIRCRAFT STORES
COMPATIBILITY


[In 1909] the chief engineer was almost always the chief test pilot as well.

That had the fortunate re
sult of eliminating poor engineering early in aviation.


Igor Sikorsky


3.1

Systems Engineering.

As outlined at AGARDOgraph 300
-
XX
, what is commonly known
as the
systems engineering process

is basically an iterative process of deriving / defining
requi
rements at each level of the system, beginning at the top (the system level) and propagating
those requirements through a series of steps which eventually leads to a preferred system concept.
Depending on the
maturity
of the stores and/or aircraft, there
are four separate compatibility
situations involved when authorisation of a store on an aircraft is required that will fundamentally
drive the scope of the Validation & Verification (V&V) and T&E needed. The four situations, in
order of increasing risk, a
re:

1.

Adding ‘old’ stores to the authorised stores list of ‘old’ aircraft
–

and the least ‘fun’!.

2.

Adding ‘old’ stores to the authorised stores list of a ‘new’ aircraft.

3.

Adding ‘new’
2

stores to the authorised stores list of an ‘old’ aircraft.

4.

Adding ‘new’ or

modified stores to the authorised stores list of ‘new’ or modified aircraft
–

the most
‘fun’!.

3.2

T
wo of the vital tools that a systems engineer doing aircraft stores compatibility needs
therefore to address these four situations, are appropriate:



Risk man
agement
of all the constituent elements of the system which was discussed at
AGARDOgraph 300
-
XX

, and



Experimentation & systems modelling

at the necessary level of fidelity across the broad range of
engineering, scientific and programmatic disciplines.

3.3

Not

only are these tools vital to the ultimate systems performance and safety in it’s use but
they are two of the most commonly misused terms
3

and sources of ‘activity traps’ if used
inappropriately or in the place of positive management and active decision m
aking for the system,
its subsystems and for the super
-
system that it belongs to.

4.0

EXPERIMENTATION & T&E

All models are wrong, some are useful.
Engineering Axiom

One Flight Test is worth a 1000 analyses…
SQNLDR M.G. Tutty, USAF Test Pilot Scho
ol, Sept 1990




2

Or adding new aircraft stores configurations and/or expanding the flight operating envelope.

3

Probably even more so than systems engineering itself!


AIRCRAFT WEAPONS SYSTEMS TESTING



AGARDOgraph 300
-
YY


1
-

6


4.1

Unfortunately, there are no internationally agreed "T&E Standards". Prior to cancellation, MIL
-
STD
-
831
(1963) provided the primary guidance on test reporting, MIL
-
STD
-
810F (200) and some other
standards identified at MIL
-
HDBK 1763 (1998) id
entifies others, however, there are no military of civilian
standards agreed within NATO or the active ISO community that addresses such a fundamental issue at the
core of delivering systems that work! That said, ISO 17025 (1999) and ISO 9001 (2004) estab
lishes the
general requirements any ‘responsible test organisation’ needs to be able to satisfy to be shown to be
competent. Furthermore excellent texts such as Bock (2001), Heuer (2003) and Kass (2006) provide
excellent approaches to designing rational
and comprehensive experiments and tests. This concern has been
recognised by several professional societies such as the International Test and Evaluation Association
(ITEA) and they have agreed in principle to address the concerns in the broadest of conte
xts as to how the
international community also assesses how systems engineering should also be done in a more cooperative
manner between nations and between commercial and military systems.

4.2

Previous research by the author into future Australian and allied
air armament and military aircraft
electronic (avionic) systems engineering best practices addressed this risk to future ADF operational needs
and levels of interoperability to ensure the appropriate and timely allocation of funds for aircraft stores
capab
ilities. Since that research was published, the candidate has also taken up a position with industry as
the Chief Engineer for P
-
3 and Force Applications with Tenix Defence Aerospace prior to returning into the
RAAF. In that job it become apparent that

some of the principles ingrained by the conduct of systems
engineering and experimentation to determine the extent of aircraft stores compatibility
4

over 25 years may
be of extreme benefit to the broader and much more difficult problem of assessing the co
mplete avionic
mission system’s compatibility and/or interchangeability
5

by using such inductive logic. The author has
also been interested for some time in researching the potential for similarly integrating the extensive
Modelling and Simulation (M&S)
done for aircraft stores compatibility into the ADF’s AAP 7001.067
(2004) and MIL
-
HDBK
-
1763 (1998) frameworks. MIL
-
HDBK
-
1763 (1998), for which the candidate was
responsible for Chairing the US Tri
-
service Committee involved in 1990 and was one of the maj
or
contributors for the revision in 1998, identifies for some 40 types of ground and flights tests that the
Responsible Test Organisation is to be able to meet minimum criteria for: Purpose, Data Requirements, Test
Preparation, Acceptance Criteria, Test Pr
ocedure, and Test Reporting.

4.3

Integrating a consistent framework for experimentation and the M&S conducted in support of
demonstrating the aircraft avionics ground and flight testing for the capability required may provide a basis
for determining the ana
logous nature of such systems for certification and for effective network enabled
operations at higher levels of interoperability without insisting on the use of common systems.


4.4

Gartska (2000) and (2005) note that ‘
A network
-
centric force has the capabi
lity to share and
exchange information among the geographically distributed elements of the force: sensors, regardless of
platform; shooters, regardless of service; and decision makers and supporting organizations, regardless of
location. In short, a
net
work
-
centric force is an interoperable force, a force that has global access to

assured

information whenever and wherever needed
’
6
. Portions of Gartska (2000) are worth noting here
(as it was worth noting in Tutty (2005)) due to it’s applicability to aer
ospace mission systems and a
refocusing since that time is particularly important to the research:

“Continued exploration of the relationships between information and combat power requires
both new analytic tools and new mental models. Ongoing activities

to develop metrics for
the information domain are hacking through dense conceptual “underbrush” in an attempt to



4


T
ypically this is treated as a “two system compatibility problem” wi
th the "aircraft" and the "stores" it “carries and employs”
-

which uses a consistent aircraft stores compatibility T&E framework via MIL
-
HDBK
-
1763 (1998) and the ADF’s AAP 7001.067
(2004).

5


Which needs to be treated as a
n “open ended number of system of

systems compatibility problem” and hence is also an
interchangeability problem.

6


Garstka (2000)
notes that ‘a force with these capabilities is not known to currently exist in any of the US Military services or in the
armed forces any our Allied or Coali
tion partners.’ Which was true at Tutty (2005) and is still true today.


AIRCRAFT WEAPO
NS SYSTEMS TESTING




AIRCRAFT WEAPONS SYSTEMS TESTING

1
-

7

identify a path that can be navigated. A key element of the model is a focus on three
domains: the physical domain, the cognitive domain, an
d the information domain. This
conceptual model builds upon a construct proposed initially by Fuller [(1926)],

and refined by
Cebrowski (1998)

[and later in Hayes & Alberts (2002) p 10]:



Physical Domain:

The physical domain is the traditional domain of w
arfare. It is domain where
strike, protect and maneuver take place across the environments of ground, sea, air and space.
Comparatively, the elements of this domain are the easiest to measure, and consequently, combat
power has traditionally been measu
red primarily in this domain. Two important metrics for measuring
combat power in this domain, lethality and survivability, have been and continue to be cornerstones of
military operations research.



Information Domain:
The information domain is the dom
ain where information is created,
manipulated, and shared. The force has the capability to collect, share, access and protect
information. The force has the capability to collaborate. This becomes the most sensitive of the
domains to protect and defe
nd.



Cognitive Domain:

The cognitive domain is the domain of the mind of the warfighter and the
supporting populous. This is the domain where battles and wars are won and lost. This is the
domain of intangibles: leadership, morale, unit cohesion, level

of training and experience, situational
awareness, and public opinion. This is the domain where tactics, techniques and procedures [TTP]
reside. Much has been written about this domain, and key attributes of this domain have remained
relatively constan
t since Sun Tzu in

500 BCE
. The attributes of this domain are extremely difficult to
measure, and each sub
-
domain (each individual mind), is unique. Consequently, explicit treatment
of this domain in analytic models of warfare is rare. However, a meth
odology that begins to address
key attributes and relationships of this domain has been proposed by Harmon (1997) in the context of
“entropy based warfare.’ ‘…
With network
-
centric operations a fourth input is added, digital
information that is exchanged f
rom external sources, such as other fighter aircraft, or airborne
surveillance and C
2

aircraft, over a network [see Tutty (2005)] Figures 2
-
6 and 2
-
7].

The issue then really becomes one of “data fusion and confidence in the provenance of the data shared
an
d presented to the required User.”

4.4

As noted in numerous texts and papers since Gartska (2000), such as
Signori et al (2001)
,
Alberts & Hayes (2005), Gartska (2005) and
Singer (2007)

a fourth domain now seems warranted:
namely a social
-
cultural one.

Alberts & Hayes (2005) defines this domain as follows:



“Social Domain
: The social domain is the domain covering those set of interactions between
and among force entities”.

4.5

This is a welcomed development
as the
Cognitive Domain
can now rightly focus
on the
implementing ability of the persons involved rather than on the C2, management and coordination functions
of the
Social Domain
. How this relates to the progression from peace through war and (hopefully) to
peace again and these domains

of war is k
ey to the national effects
-
based approach (which not just about
weapons effects) of Smith (2003) and has been elegantly summarised by Singer (2007) as shown at Figure
8.7.10
eae544
-
2. This directly impacts on the time that experimentation can be seen to
influence. Also of
interest here is the distinction being made between the “Art of War” and the “Science of War” during the all
too critical “Stability Operations and Transition” Phase to Peace
–

which the media have highlighted as
having not been particu
larly well done in recent major conflicts
–

the so
-
called “winning the peace”!


AIRCRAFT WEAPONS SYSTEMS TESTING



AGARDOgraph 300
-
YY


1
-

8


Figure 2. Application of the Art & Science of War and the “Domains of Warfare”
–

courtesy Singer (2007)

4.6

The proposed University of SA and RT
O research will explore
inter alia

how interoperability
initiatives for air armament systems may also be applied to future military avionic mission systems and the
systems engineering and experimentation being applied during acquisition of new Australian a
ircraft
capabilities. Network enabling will see the need for the system of systems needing to be certified for use
of weapons which will require the rigour of safety
-
critical, high reliability systems engineering and vastly
improved experimentation / M&S

/ T&E techniques needing to be applied. The research into whether a
code of best practice for an integrated systems engineering and experimentation / test and evaluation
framework is achievable for optimising and demonstrating future network enabled cap
abilities needing
aircraft avionic systems in effects based operations will be investigated. Whether contemporary systems
engineering and T&E practices are suitable for application with complex adaptive networks will also need
to be thoroughly explored.

Alberts & Hayes (2007) suggests that a “cookbook” is needed (the good news)
but is “clearly premature at this time” and questions whether can be written and what form it would take
(the bad news). It is proposed that the MIL
-
HDBK
-
1763 (1998), Hayes and

Alberts (2002) and the more
recently published TTCP
GUIDEx (2006)

approaches where a tailorable framework is available as shown at
Figure
3

and Appendix A Table 1 as a starting point and indeed these documents should help provide the
compilation of techni
ques that have been found to be successful by others doing aircraft stores compatibility
and experimentation respectively. Such an experimentation “cookbook” will provide a means for planning,
comparing the results, assessing technology readiness consist
ently and thereby enhancing their reliable use
in future research/testing that will be of immediate use to practitioners.

4.7

To answer this research problem, the North Atlantic Treat Organisations (NATO) Research &
Technology Organisations (RTO) SCI Fligh
t Test Technology Team (FT3) are cooperating with the RAAF
and University of South Australia’s research for development of a ‘Code of Best Practice’ as the first big

AIRCRAFT WEAPO
NS SYSTEMS TESTING




AIRCRAFT WEAPONS SYSTEMS TESTING

1
-

9

step to a consensus based standard "
Joint

Aerospace Integrated Mission Environment for

Exp
erimentation CoBP".

4.8

Before that discussion, what are the fundamentals of contemporary weapons systems testing?

Figure 3. The military experimentation process
(adapted from Alberts and Hayes (2002))


AIRCRAFT WEAPONS SYSTEMS TESTING



AGARDOgraph 300
-
YY


1
-

10


5.0

WEAPONS SYSTEMS GROUND & FLIGHT TEST
–

THE FU
NDAMENTALS

5.1

Introduction.

Broad description of weapons systems Ground & Flight Test as a risk reduction
exercise, maximising the potential capability, a through life activity etc The SFTE Flight Test Handbook is
extremely valuable. Of significance i
s that RTO AGARDographs Series 160 and 300 identify Flight Test
Instrumentation and Flight Test Techniques respectively. The AG
-
160 or AG 300 Volume number will be
referenced where relevant will be cited wherever they exist or are being actively being wo
rked on …

5.2

The Specialisations.

Flight test activities can be sub
-
divided into the following sub
-
specialisations:



Research
-

exploratory work on a novel, fundamental idea or concept



Development
-

developmental work on an existing product, concept or t
echnique



Certification of Civil Aircraft & Systems
-

largely demonstrative against formal
requirements



Certification of Military Aircraft & Systems
-
demonstrative against formal requirements
plus the proof of fitness for purpose



Operational Test & Evaluat
ion
-

exploitation of an existing equipment standard, technique
or capability



Production and Post Flight Maintenance Flight Test
-

engineering and continued
airworthiness



Flying Laboratory Operations



Flight Demonstration and Customer Training



Flight Te
st Training
-

of test pilots and flight test engineers


5.3

Flight Test Planning.

Including: literature search, safety assessment and failure analysis, first and
subsequent flight envelope expansion, avionics, loads and weapons testing,
reference to bandw
idth
problem. S
ee

RTO AGARDOgraph AG
-
160 Volume 1
-

Basic Principles of Flight Test Instrumentation
Engineering
, Volume 13
–

Practical Aspects of Instrumentation System Installation
, AG
-
160 Volume 12
–

Aircraft Flight Test Data processing
, Volume 14
–

The

Analysis of Random Data
, AG 300 Volume 14
–

Introduction to Flight Test Engineering
and AG
-
300 Volume 19

–

Simulation in Support of Flight Testing.

Testers must make sure that their analysis tools to be used during the program actually are suitable and
ready to ensure that all key data is collected and that test missions do not have to be redone to address such
a fundamental test planning deficiency.


AIRCRAFT WEAPO
NS SYSTEMS TESTING




AIRCRAFT WEAPONS SYSTEMS TESTING

1
-

11

Max Lorenzo Need to add in here a JTEM discussion wrt experimentation at joint level for exercises etc

…


Figure 3 JTEM Approach
–

Courtesy of
JTEM (2009)
…


George Rumsford / Chip Ferguson…. JMETC paragraph


5.4

Ground and Flying Handling Qualities, Engine Handling in Flight
Including: taxiing and
handling on the ground, longitudinal handling, lateral
/directional handling, rapid rolling, stalling and
spinning, human factors and crew workload, engine handling at various AOA/betas, confirmation of relight
envelopes, fuel flows etc. See
MIL
-
STD
-
1797A (1995),

RTO AGARDOgraph AG 160 Volume 16
–

Trajectory M
easurements for Take
-
Off & Landing Tests
, AG 300 Volume 21
–

FQ Flight Testing of Digital
Flight Control Systems

5.5

Structural Performance Flight Testing.

Including: flutter, structural temperature assessment,
vibration and noise, radar cross
-
section and

infra
-
red signature measurement. See Chapter
7.5.05

and RTO
AGARDOgraph AG 160 Volume 7
–

Strain Guage Measurments on Aircraft
.

5.6

Avionic Systems Flight Testing.
Including: navigation and autopilot systems, including auto land,
sensor systems (includin
g: radar systems, radar altimeter, defensive aids, electro
-
optic sensors)
communications (including data link), IFF and TACAN and helmet
-
mounted sights and displays etc.,
human factors and crew workload. See

RTO AGARDOgraph AG 300 Vol 15
–

Introduction t
o Avionics
Flight Test,
AG 300 Volume 4
–

Determination of Antennae Patterns and Radar Reflection Characteristic

AIRCRAFT WEAPONS SYSTEMS TESTING



AGARDOgraph 300
-
YY


1
-

12


of Aircraft,

AG 300
–

Volume 18
–

Flight Testing of Radio Navigation Systems
and AG 300 Volume 20
–

Logistics T&E in Flight Test
.

5.7

High Ord
er Flight Controls Flight Test

Including: flight laws, speed and alpha protection systems.
See

RTO AGARDOgraph AG 300
–

The Principles of Flight Test Assessment of Flight
-
Safety
-
Critical
Systems in helicopters.

5.8

Environmental Ground & Flight Testing a
nd Noise

Including: hot weather, cold weather and
icing testing, contaminated surface “shapes”, noise, water trough testing, noise verification and
measurement. See also

MIL
-
STD
-
810F (200)
and RTO AGARDOgraph AG 160 Volume 2
–

Inflight
Temperature Measu
rements
, AG 300 Volume 9
–

Aircraft Exterior Noise Measurement and Analysis
techniques,

AG 300 Volume 8
–

Flight Testing under Extreme Environmental Conditions
.

5.9

Aircraft Performance Testing

Including: pressure error calibration, takeoff, climb and c
ruise
performance, range & endurance, OEI performance, landing performance and braking, VMC, VMCG &
VMU etc. See

SFTE Handbook
and RTO AGARDOgraph AG 160
–

Volume 3 and 4
–

The Measurement
of Fuel Flow and Engine Rotation Speed, AG 237
–

Guide to In
-
flig
ht Measurment of Turbo
-
jets and Fan
Engines.

5.11

Fixed Wing Vertical and Short Take off & Landing Trials [V/STOL]

Essentially the
difference and additions necessary for V/STOL research, development & clearance activities, i.e. hover
performance and hand
ling in the V/STOL range

5.12

Rotary Wing Aircraft Testing

Including: rotary wing testing techniques, hover performance,
OEI flight, auto
-
rotation/dead man’s curve, sloping ground limitations, winching and under
-
slung loads.
See also

RTO AGARDOgraph AG
160 Volume 10
–

Helicopter F
light Test Instrumentation
, AG 300
Volume 9
–

Aircraft Exterior Noise Measurement and Analysis techniques
, and AG 300 Volume 22
–

Helicopter/Ship Qualification

Testing
.

5.13

Operational Test & Evaluation
Including: preparati
on and planning, safety assessment and
hazards analysis, flight test profiles, reporting. See AG 300
–

Volume 13
–

Reliability & maintainability.

5.14

Post Maintenance Flight Testing

Including: preparation and planning, safety assessment and
hazards an
alysis, flight test profiles, airworthiness reporting

5.15

Unmanned Aerial Systems [UAS] Flight Testing

Including: the principal differences between
manned & unmanned vehicle flight test, control mechanisms, emergency reversion and self destruct
mechanism
s, data links, certification challenges

5.16

Role Related Flight Testing
: Activities to include:



Air to Air Refuelling Trials
–

see AG
-
300 Volume 11
–

The Testing of Fixed Wing Tanker &
receiver Aircraft…



Air Delivery Trials & Paradropping
–

see AG 300 Vol
ume 6
–

Developmental Airdrop Testing
Techniques and Devices.



Air
–

air radar and AEW&C systems
–

see AG 300 Volume 7
–

Air to Air Radar Flight Testing
and Volume 16

–

Introduction to Airborne Early Warning Radar Flight Testing


AIRCRAFT WEAPO
NS SYSTEMS TESTING




AIRCRAFT WEAPONS SYSTEMS TESTING

1
-

13



ASC Weapons Carriage & Rele
ase Trials
–

MIL
-
HDBK
–
244A (1990) and MIL
-
HDBK
-
1763
(1998), AG 160 Volume 9
–

Aeroelastic Flight Test Techniques & Instrumentation
, and AG
Volume 5
–

Store Separation Flight Testing



Weapons Accuracy/Aiming Trials
–

see AG 300 Volume 10
–

Weapon Delivery A
nalysis &
Ballistic Testing



Ship Helicopter Operational Limits Trials [SHOL]



Electronic Warfare (EW)
–

see AG 300 Volume 17
–

EW T&E



Night Vision Goggles
–

AG
-
SCI
-
089,
Flight Test of Night Vision Systems in Rotary Wing
Aircraft.




Survivability
–

MIL
-
STD
-
20
72 and
AIAA Survivability text



Experimentations
–

see Guidex (2003) and Alberts & Hayes (2002).



Reliability & Maintainability
–

MIL
-
STD
-
217F and AG 300 Volume 13
–

R&M



NASA R&M


5.17

Flight Test Approvals.
Including: design and engineering approvals, ai
rworthiness flight
limitations and approval mechanisms, flight test organisation and flight crew approvals
–

reference to 2
annexes explaining the specific roles and responsibilities of test pilots and flight test engineers as debated
within EASA [proposed

Annexes attached to this Framework]

6.0

AIRCRAFT STORES COMPATIBILITY TESTING.

6.1

The ASC tests and factors from MIL
-
HDBK
-
1763 (1998) Section 4 requiring consideration for all
weapons systems testing to ensure an effects based outcome can be summarised
as:

“a. Ground test procedures which may include:

(1) Tests 101 and 102, Fit and Function. Store loading configurations, clearances
electrical/mechanical interfaces, electrical compatibility, armament weapons
support equipment or checkout and loading pro
cedures.

(2) Test 110, Static Ejection. Determine reactive force loads, store velocities, separation
characteristics, correct lanyard function and arming control system reliability.

(3) Test 120, Aeroelastic Ground Vibration Test (GVT). Determine frequen
cy, damping and
mode shape of the aircraft
-
store/suspension equipment combination for flutter analyses.


(4) Test 130, Structural Integrity. Verify aircraft
-
store and suspension equipment
structural integrity and compatibility to the most critical flight c
onditions including
those necessary for carrier suitability.

(5) Test 140, Wind Tunnel. Determine effect of aircraft on captive stores, captive stores on
aircraft, aeroelastic effects, separation and ballistics for the aircraft
-
store configuration.

(6) T
est 150, Environmental. Determine the ability of the store configuration(s) to withstand or

AIRCRAFT WEAPONS SYSTEMS TESTING



AGARDOgraph 300
-
YY


1
-

14


operate in the vibration, aeroacoustic electromagnetic, temperature and thermal
environments.

(7) Test 160, Gun/Rocket/Missile Firing. Quantifies the safety, compa
tibility and performance
requirements of the aircraft gun, rocket or missile systems.

b. Flight test procedures may include:

(1) Test 200, Inflight Loads and Structural Integrity. Inflight instrumented tests used to verify the
structural integrity and ob
tain loads survey data for the store and aircraft stores
combinations.


(2) Test 210, Flutter. Inflight instrumented tests used to substantiate that the aircraft with stores
is free of any aeroelastic instability and has satisfactory damping characteristic
s in the
defined carriage envelope.

(3) Test 220, Environmental. Determines if the aircraft
-
store and suspension equipment can
withstand the actual flight environment, validate design specification levels and substantiate
predicted/test levels for vibrati
on, aeroacoustic and thermal environments.

(4) Test 230, Flying Qualities. Inflight instrumented tests used to demonstrate that the
aircraft/stores configuration meets the requirements for the flying qualities of military piloted
aircraft.

(5) Test 240,
Performance and Drag. Determines any degradation in mission performance
caused by the carriage of external stores.

(6) Test 250, Captive Flight Profile (CFP). Verifies the effect of a store configuration on aircraft
flying qualities and store structural i
ntegrity throughout the required flight envelope, usually
performed on uninstrumented aircraft.

(7) Test 260, Carrier Suitability. Verifies store compatibility under actual catapult launches and
arrested landings.

(8) Test 270, Employment. Verifies store
s separation characteristics under various modes
(including failure mode), tactics and conditions, store release and propulsion effects on the
aircraft and adjacent stores, electromagnetic interference effects and compatibility with
shipboard electromagnet
ic environment.

(9) Test 280, Jettison. Verifies store jettison modes, configurations and conditions,
effects on aircraft and adjacent stores, propulsion effects, functional differences
in store between employment and jettison.

(10)Test 290, Ballistics.
Determines free
-
stream ballistic coefficients, evaluates and verifies the
aircraft/stores ballistic accuracy.

c.

Armament system software changes. One extensive aspect of stores certification
is the software changes required to the Stores Management Set
and/or Fire
Control Computer/Mission Computer for control and release of the store being
authorized. These software changes are a necessity for proper interface and
function of the store with the aircraft. It is probable that for new aircraft, the
software

changes will be accomplished by the prime contractor; however, as the
aircraft enters service, the Operating Service will make input to any necessary
changes. Verification of the acceptability of the changes to OFPs affecting
operation of stores will norm
ally be conducted as part of Test 102.


AIRCRAFT WEAPO
NS SYSTEMS TESTING




AIRCRAFT WEAPONS SYSTEMS TESTING

1
-

15

d.

Human factors and mission effectiveness. When developing the certification plan for any
aircraft
-
store combination, dedicated mission effectiveness or human factors testing should
be considered. This includes mun
ition preparation, loading, aircraft checkout, preflight,
cockpit weapon switchology, takeoff, ingress, target tracking/acquisition, target
engagement, safe escape maneuvers, egress, landing, integrated combat turn capability
and downloading. The results o
f mission effectiveness and human factors considerations
should be published as part of any certification report to provide a data base for further
development of existing weapons systems as well as new technologies.”


So what does that summary mean in pra
ctical terms?

6.2

Ground Testing

Mavens Corollary to Von Moltkes Adage on Battle Plans and the Enemy: No Test Plan survives contact with the Aircraft.

SQNLDR M.G. Tutty, USAF Test Pilot School, Sept 1990


6.2.1

For new aircraft with little or no previous captiv
e compatibility testing to draw upon, ground testing is a
vital link to reducing some of the uncertainties before flight.

6.2.2

Fit and function ground mounts and/or modeling and simulation analysis using computerised physical
fit programs are essential to dete
rmining if a new store is compatible. Mounting of the store on its suspension
equipment and aircraft validates physical fit while also certifying loading procedures. A power
-
on ground
mount is the first step to verify that the store and aircraft systems ca
n talk to each other over the weapons
buses and that the store can function as designed. Such a power
-
on ground mount can also meet many of the
requirements for electromagnetic compatibility and electromagnetic interference (EMC/EMI) certification to
deter
mine if unintentional interactions will adversely affect flight safety or mission success.

6.2.3

Weapon developers have also tried to bridge the reliability gap of weapons flying in the real world
with environmental ground testing. Such “shake and bake” testing

is designed to subject the store to the
predicted worst
-
case critical conditions and for establishing the expected life span of the structures and
electronics. Whilst this is still an imperfect science, the failure to do any such activities constitutes
e
xtremely poor engineering and programmatic practices. The initial environmental analyses should
compare the natural and induced environment of the aircraft stores combination(s) in which the store and
store suspension and release equipment must operate to

that which has been or will be qualified. Of primary
importance for these stores and equipment during this initial analysis is the operational or mission life
-
span.
For some stores, this life
-
span would in the past have been typically a one
-
time flight w
hile today with
extended flight times and multiple missions flying combat air patrols stores, such as fuel tanks, ECM pods,
and air
-
to
-
air missiles, the life
-
span could be greater than the using aircraft. Vibration and acoustic ground
testing are often co
mbined to determine if a store can survive and still function effectively under the
combined effects of the lower frequency airframe
-
induced motions and the higher frequency acoustic
energy. Thermal testing ensures a new store can withstand the extreme te
mperatures in both the ground and
flight environments.

6.2.4

Aeroelastic ground vibration testing is conducted to determine frequency, damping, and mode shape in
order to validate wind tunnel test results and theoretical flutter calculations. Structural integri
ty ground testing
subjects the aircraft stores combination to the loads that are predicted to occur at the most critical flight
conditions. For carrier
-
based aircraft, these critical flight conditions are expanded to include the dynamic
conditions of catap
ult launches and arrested landings. The goal of this ground testing is to ensure that aircraft
and store have sufficient strength to meet the ultimate design conditions.

6.3

Flight Testing

6.3.1

Even after extensive research, advanced analysis, and thoro
ugh ground testing, uncertainty about
captive store compatibility will usually remain. The amount of this remaining uncertainty will determine the

AIRCRAFT WEAPONS SYSTEMS TESTING



AGARDOgraph 300
-
YY


1
-

16


level of effort required in flight testing. For new stores being carried on new aircraft, flight testing will

need to
explore areas of flutter, aircraft and store loads, and flying qualities. On the other end of the spectrum, a store
previously tested on one aircraft may need certification on another aircraft that has tested similar stores. In this
case, the effe
cts of vibration and airflow on a store’s suspension equipment, umbilical connections, lanyards,
control actuators, integrated guidance and control systems, and other associated hardware may remain
unknown but will be expected to have less risk. Additiona
l questions may linger about the effect of the store
configuration on aircraft flying qualities. This remaining uncertainty can be eliminated by flying a limited
scope flight test mission known as a captive compatibility flight profile (CFP).

6.3.2

Captive Comp
atibility Flight Profiles.
CFP test missions are flown to qualitatively evaluate aircraft
flying qualities, store structural integrity, and physical interfaces throughout the required carriage envelope.
Although these missions can be flown to gather quant
itative data, they are usually flown on uninstrumented
aircraft and data requirements are minimal. Three different types of tests are flown during CFP missions:
flying qualities, structural integrity, and endurance.



A CFP flying qualities evaluation is re
quired for a store whose shape, mass properties, or configuration
are not analogous to another store that has been previously certified on that aircraft. Large, heavy stores
which approach aircraft mass, inertia, or aerodynamic boundaries usually require
a flying qualities CFP.
This evaluation is also required to clear a store to higher carriage limits than previously certified. Store
configurations that could potentially exceed aircraft limits are not good candidates for a qualitative CFP
and should be mo
nitored real
-
time with an instrumented aircraft.



In order to assess the change in flying qualities from a given store, a baseline CFP mission should be
flown with the same aircrew and the same aircraft, but without the test store. The test profile inclu
des all
classic flying qualities flight test techniques flown throughout the carriage envelope. CFP structural
integrity testing is flown to certify a configuration when no previous analogous testing has been
conducted. Testing of a store on another aircra
ft or in a different configuration may allow for certification
of a new configuration by analogy based on ASC engineering judgment. CFPs are generally not required
for one
-
of
-
a
-
kind stores that do not require certification for fielding. If a store already

in inventory is
significantly modified, it should undergo a CFP structural integrity test. For a store that requires
certification on multiple aircraft, the structural integrity test requirements for all aircraft may be met by
flying the CFP on the aircra
ft and in the configuration judged to be most critical. However, testers must be
careful if this approach is taken, since it is very hard to predict which aircraft might have the most severe
carriage environment or if the store components will be affected
in the same way by all aircraft. The CFP
structural integrity tests are conducted to subject the aircraft
-
store combination to worst
-
case inertial and
aerodynamic. The wind up turns and loaded rolls, both positive and negative g, are flown in a build up
f
ashion by starting in the heart of the envelope at 80% of the maximum load factor before proceeding to
100%. In general, maneuvers should next progress to maximum Mach number at altitude, then to the
maximum Mach and airspeed point, and finally to maximum
airspeed at low altitude for the highest
dynamic pressure. Some configurations may also require engine spillage testing to determine the effects
of transient engine inlet backwash on stores mounted near the inlet or in other vulnerable positions. To
meet

this objective, “throttle chops” are conducted at the critical test conditions, generally in the transonic
regime.



CFP endurance, or “speed soak”, testing is conducted at the airspeed and altitude within the carriage
envelope that causes the worst
-
case

straight and level aircraft vibration. Previous flight testing on aircraft
loaded with external stores has shown that worst
-
case vibration occurs around 0.9 Mach at the lowest
practical altitude. For this reason, the speed soak should be flown at or below

1,000 ft MSL and at 0.9
Mach. This testing should have a cumulative total of 50% of total mission radius or approximately 30
minutes. If the speed soak must be terminated or suspended for any reason, for instance to perform air
refueling, it can be resume
d and still certify store endurance as long as the cumulative time at the critical
flight condition exceeds 30 minutes.


AIRCRAFT WEAPO
NS SYSTEMS TESTING




AIRCRAFT WEAPONS SYSTEMS TESTING

1
-

17



Another part of CFP endurance testing is to ensure that the configuration can withstand the operational
flight environment for periods

longer than a single mission. To meet this objective, the cumulative time
for flying qualities, structural integrity, and speed soak endurance testing must exceed 150% of the
aircraft’s mission radius. If more than one mission is needed to meet this requi
rement, then the store
should not be downloaded and any discrepancies noted on the aircraft stores combination resulting from
the first sortie should not be corrected between sorties. In order to verify store structural integrity and
endurance, the test s
tore and suspension equipment should be photographed both before and after the test
mission. During preflight, any existing aircraft stores anomalies should be carefully noted and marked on
the store or aircraft to aid in post
-
flight review. Additionally,
the store’s function should be checked
before, during, and after the mission to ensure the flight environment does not cause degradation.

6.3.3

Environmental Testing.
Stores that are still under development generally require more rigorous
environmental
testing than can be accomplished on a CFP. MIL
-
STD
-
810F (2000) is the Bible along with AG
300 Volume 8 as noted earlier. This testing will not only ensure the new store can hold up and function
throughout the carriage envelope, but it may also focus on cr
itical flight conditions or a specific operational
environment. For example, weapon developers may need to ensure their new store can survive and function
when subjected to the vibration caused by aircraft gunfire or, if carried internally, in the harsh ae
ro
-
acoustic
environment of an open weapon bay. Likewise, it may be necessary to expose the store to rain or other
environmental factors if these represent critical design criteria. Whenever possible, aircrew should evaluate
store function during both envir
onmental and CFP testing. In particular, aircraft vibrations or limit cycle
oscillations might affect a weapon seeker’s ability to lock onto and track a target. In this case, operationally
representative maneuvers should be flown to assess the mission impa
ct.

6.3.3

Test Aircraft Configurations.
An often overlooked aspect of captive compatibility testing is
software integration between aircraft and store Operational Flight Programs (OFPs).



Most of today’s advanced weapons rely heavily on information from the ai
rcraft to tell them either where
they are, where the target is, or both. Unfortunately, many aircraft stores certification efforts are
completed with patch tapes and other workarounds that provide the weapon with the information needed
to carry out its mi
ssion. The aircraft OFP is usually an after
-
thought and in some cases has driven
additional weapon OFP development once full software integration was attempted. Whenever possible,
development of store and aircraft OFPs should be conducted concurrently in
order to discover and correct
deficiencies as early as possible.



For safety reasons, captive compatibility testing should only be accomplished with inert stores. These test
articles must have production representative mass and aerodynamic properties. Any
hardware that may
affect drag or flow field must be mounted on the store. Mass properties, including weight, center of
gravity, and inertias, must be accurately measured. If necessary, ballast must be added to the store to make
its mass properties producti
on representative.



Although testing of flutter, loads, and flying qualities can be done with a “dummy” store, flight with an all
-
up
round (AUR) will enable the aircrew to determine if the store can survive structurally and function effectively
even at th
e worst
-
case environmental conditions. An AUR includes all mission
-
necessary subassemblies,
electrical connections, and associated hardware. When used for dedicated environmental testing, an AUR is
heavily instrumented to measure and record environmental c
onditions. Even when “dummy” stores are used,
all physical and electrical interfaces must be adequately simulated.

In recent years, most weapon OFPs have included diagnostic features that can pass health information to the
cockpit. For warfighters, such d
iagnostics warn aircrew of weapon failures or degradation before any attempted
release. For testers, this information is crucial in troubleshooting problems during environmental testing. If an
aircraft OFP is not capable of monitoring and displaying this d
ata, then it must be recorded inside the store to be
downloaded and analyzed after flight. This is another good reason for conducting stores certifications with

AIRCRAFT WEAPONS SYSTEMS TESTING



AGARDOgraph 300
-
YY


1
-

18


production representative aircraft OFPs.

6.3.4

Telemetry.
During environmental testing, store
environmental and health data are often transmitted via
telemetry to a control room for real
-
time monitoring. Such telemetry usually comes directly from the store.
However, if the aircraft OFP supports it, aircraft weapons bus data can also be transmitted
for control room
monitoring of store health. During edge
-
of
-

the
-
envelope captive stores certification, the control room can also
play an important safety role by monitoring aircraft parameters and terminating a test point if limits are
approached or excee
ded. Control room monitoring of safety of flight parameters is critical for flutter, loads, and
flying qualities envelope expansion testing. Beyond monitoring safety and store health, the control room can also
significantly improve test efficiency by evalu
ating test point quality and providing feedback to the pilot. For low
altitude captive compatibility test points, telemetry reception can be a definite problem. If real
-
time monitoring of
store health or aircraft flight conditions is required, test points
should be planned to remain close to the telemetry
receiver. Other possible solutions are to use another aircraft to relay telemetry and to perform the test points with
the store facing telemetry antennas so as to minimize the aircraft’s masking effects.

6.3.5

Risk Mitigation.
Since much of captive compatibility flight testing calls for taking unproven aircraft
stores configurations to new limits, its risks include many of the same ones that are inherent in any envelope
expansion test effort. On the far s
ide of the risk spectrum, captive compatibility testing could result in severe
aircraft damage or the loss of aircraft control. These risks are generally associated with flutter, aircraft and store
loads, and flying qualities testing of new aircraft stores

combinations and are addressed in detail in
AGARDOgraph 300
-

XX
. Specific captive compatibility risks come from maneuvering an aircraft right up to the
edge of its limits, often at the far right side of its flight envelope. Additional risk is introduced
by the requirement
to prove compatibility while flying with heavyweight or asymmetrically loaded configurations.

6.3.6

Test Limits.
In order to mitigate the risk involved with testing at the very edge of aircraft limits, testers
must first know the speci
fic reason for each limit. Both load factor and airspeed limits are usually imposed to
protect the structural integrity of the aircraft stores combination. However, in some cases, these limits are needed
to prevent the pilot from going into a part of the f
light envelope with degraded handling qualities or increased
flutter susceptibility. At times, elevated risk may result from approaching a test limit that is simultaneously
indicated by independent risk assessments of the different engineering disciplines.

This convergence of flutter,
loads, or flying qualities risks can be particularly challenging and will require thorough planning of mitigation
procedures. By knowing the specific rationale for each limit, test aircrew can look for ways to mitigate risk if

the
limits are inadvertently exceeded. Knowing the basis for limits of a specific aircraft configuration might also allow
the flight clearance authority to plan for testing right up to the operational limit without the risk of having to
terminate testing
early for a momentary excursion beyond a more conservative limit. Although flight test limits
may cut into the margin of safety that is built into the operational aircraft limits, the proficiency and training of test
aircrew allow for a small increase in r
isk which is usually worth the benefit of being able to carefully proceed
through the test profile. MIL
-
HDBK
-
1763 also prescribes over testing such operating limitations during
developmental testing to ensure that the operating limitations for the aircra
ft stores configurations going into
service are more robust for operational usage (when test pilot are not flying the mission on a two way range
against the enemy!). For example, when limits are based on degraded handling qualities or flutter susceptibil
ity,
test objectives may call for exceeding a planned operational aircraft limit in order to confirm that a cliff does not
exist just past that limit. Such testing might actually allow the flight clearance authority to expand the operational
limit if they
determine the margin of safety is higher than predicted. On the other hand, if handling qualities or
limit cycle oscillations are worse, then that aircraft stores combination might need to be further limited.

6.3.7

High Speed Risks.
Flight testing at the
far right side of the aircraft envelope introduces additional risks
which must be taken into account. The high airspeed flight regime subjects aircraft and stores to increased
dynamic pressure and temperature, yielding a higher potential for structural fai
lure. Any failure that degrades
aircraft stability and control could be catastrophic, since a departure from controlled flight at such high speeds will
most likely cause additional aircraft breakup well above the safe ejection envelope. To mitigate the ri
sk of high

AIRCRAFT WEAPO
NS SYSTEMS TESTING




AIRCRAFT WEAPONS SYSTEMS TESTING

1
-

19

speed structural failure, the test aircraft should be fully inspected before flight using all non
-
destructive means
available. All major stability and control surfaces should be checked for delamination, water intrusion, and other
structural def
ects. Additionally, all door latches should be pull tested to ensure they do not release at high speed
due to the effects of high dynamic pressure and fuselage flexure. Additional risk can be mitigated by limiting
exposure to the high speed flight regime.

Although CFP testing calls for subjecting the store to positive and
negative g loaded rolls at maximum allowable airspeed, such flight conditions may be so difficult to achieve that
operational pilots will never be expected to go there. Likewise, certain
maneuvers at high speed may not achieve
the desired effect. For example, flight control roll rate limits at high airspeeds might not subject the store to the
greatest structural forces, where a reduced
-
speed loaded roll may. Test objectives might be better

served by
maneuvering to maximum load factors at a lower, more operationally representative airspeed. Ultimately, the
benefits of conducting testing at high speeds must be based on sound engineering judgment to add value that
outweighs the risk of going t
here in the first place.

6.3.9

Safety Chase.
The effective use of a chase aircraft is crucial to mitigating risk during captive
compatibility testing. In addition to clearing for the formation, the safety chase can monitor flight conditions and
check the
aircraft and store for discrepancies either during or between test points. During high speed flight testing,
the chase aircraft may need to lag behind the test aircraft based on its own more restrictive aircraft limits or in
order to conserve fuel. The are
a chase approach will also limit the exposure of the chase aircraft to the increased
risk of the high speed flight regime. An aircraft flying area chase must maintain awareness of the test aircraft’s
position and be ready for a quick rejoin, if needed.

6.
3.9

Configuration Limits.
Some captive compatibility tests call for aircraft
-
store configurations at the very
edge of aircraft asymmetric load and gross weight limits. Such configurations can increase risk throughout the
flight profile, but particularly d
uring takeoff, landing, and maneuvering flight.

6.3.10

Asymmetric Risks.
The first approach to reduce this risk should be to minimize exposure to it. For some
tests, takeoff asymmetry can be reduced by loading a counterbalancing store on the opposite win
g or specifying
unique fuel configurations for takeoff. To get the aircraft in the proper configuration before the first test point, the
counterbalance can be jettisoned or fuel balance adjusted through refueling or consumption. Before landing, stores
shou
ld be released or jettisoned to reduce asymmetry and gross weight, if possible. Sometimes, however, this
approach may be cost prohibitive, especially for high cost test items. The risks induced by asymmetric stores come
from increased pilot workload to off
set the asymmetry and the subsequent reduction in remaining flight control
power. These configurations are also more susceptible to departures from controlled flight. With asymmetric
stores loaded onto an aircraft, the center of gravity (cg) will be shifte
d into the heavy wing. This shifted cg
requires unbalanced flight control inputs to keep the aircraft balanced. A lateral stick input away from the heavy
wing is required to hold it up. This in turn causes adverse yaw into the heavy wing, requiring a rudde
r input away
from the heavy wing to maintain coordinated flight. The increased drag from these inputs will require additional
thrust. Increased thrust will cause additional yaw around the cg into the heavy wing, requiring additional rudder
away from the he
avy wing. This cross coupling between aircraft axes is further exacerbated when the pilot
commands changes from the steady state conditions. An increase in power will cause a yaw into the heavy wing.
For traditional swept wing aircraft, sideslip from this
yaw will cause a roll into the heavy wing. When power is
reduced for landing, the yaw and roll will be away from the heavy wing. During maneuvering flight, an increase
in load factor will also result in a roll into the heavy wing. In order to counter the
se effects and reduce the risks of
departure from controlled flight, stick and throttle inputs should be smooth. Additionally, turns should be made
away from the heavy wing whenever possible. Some flight control systems might mask the symptoms of
asymmetr
y by maintaining zero roll rate when commanded and keeping the aircraft in coordinated flight.
Although this will reduce pilot workload, control surfaces will still be displaced to balance the aircraft. This will
not only result in increased drag, but will

also limit lateral
-
directional flight control power away from the heavy
wing. Before a takeoff with asymmetric stores, lateral trim may need to be set away from the heavy wing in order
to prevent a roll at liftoff. A crosswind into the light wing will fu
rther complicate roll control during takeoff and
landing, and may exceed lateral control power. Aircrew must plan for unique crosswind limits prescribed as flight
test risk mitigation procedures. During landing, symmetric braking will cause a turn away fro
m the heavy wing.
Since differential braking will be required to keep the aircraft rolling straight down the runway, maximum braking

AIRCRAFT WEAPONS SYSTEMS TESTING



AGARDOgraph 300
-
YY


1
-

20


will not be available.

6.3.11

Heavyweight Risks.
Testing of heavyweight configurations brings additional unique risks.
No
twithstanding density altitude effects, aircraft control will be much more sluggish due to the increased inertia.
Computation of takeoff and landing data is critical to ensure these can be accomplished with an adequate margin
of safety. During heavyweight
landings, brake performance is vitally important. Aircrew must be ready to take
quick action for degraded brakes, including taking the aircraft back into the air if stopping distance is questionable.
If an aircraft is equipped with a digital brake system,
consider downloading the brake data between sorties to
assess brake performance. This may allow the test team to determine that one brake is not as effective as the other.
Correcting this problem can help ensure that maximum symmetric braking is available
during subsequent
heavyweight landings. Aircrew must also reference brake energy tables to ensure brake limits are not exceeded. In
most cases, brakes will be extremely hot following these heavyweight landings, so hot brake procedures should be
followed.

6.3.12

Performance Limitations.
In order to fully assess aircraft handling qualities and store structural
integrity, CFP testing requires aircraft maneuvers at the far right side of the operational envelope. Unfortunately,
this part of the envelope is not

only at the edge of the aircraft’s structural limits, but it is also often at the very edge
of an aircraft’s performance capabilities. When test conditions are above the maximum attainable speed at a given
altitude, the pilot must execute a diving entry
into the test point. These tests points must be started by climbing
5,000
-

10,000 ft above the desired test condition. In extreme cases, the pilot may need to climb to an even higher
altitude. Next, with MAX power selected, the pilot accelerates to the te
st Mach number and then holds this Mach
number in a dive. As altitude decreases during the constant Mach dive, airspeed will build. The test point is
executed when approaching the desired airspeed or altitude data band. If the dive angle required to hold
constant
Mach number is too excessive, the pilot will not have enough time to execute the test point before blasting through
the data band. If the far side of the data band is an airspeed limit, the pilot must be particularly careful not to over
-
speed the
aircraft during the dive. This is particularly easy to do during negative g test points. As a rule of thumb,
dive angle should be reduced to 10° or less before entering the data band. Even though Mach number may bleed
off slightly, this dive angle should p
revent flying through the data band too quickly. In most cases, maneuvering
will also be required during these test points. With just enough energy to maintain Mach number in the dive, there
is no energy left to put into maneuvering. As such, Mach number
will start to bleed off quickly once the test point
is started. The challenge for the test pilot is to hit the test conditions before dropping out of the bottom of the Mach
data band. Mach bleed can be particularly troublesome in the transonic region. Duri
ng windup turns in this area of
the flight envelope, many aircraft
-
store combinations have a tendency to dig in, resulting in a g spike and
subsequent early mission termination for exceeding limits. Pilots must be prepared when executing these transonic
ma
neuvers. A timely unload is essential to being able to continue the mission.

6.3.13

Loaded Roll Techniques.
The easiest way to hold g constant during loaded rolls is by using an
adjustable g
-
limiter. If the test aircraft is not equipped with this feature
, then trim can sometimes be very helpful in
holding load factor within the desired data band throughout the step roll input. Another technique is to use two
hands, one for the constant pitch input and the other for the step roll input. However, if handlin
g qualities issues
are anticipated, these test points should be flown using normal control inputs to assess the operational impact.

6.4

Separation Testing

6.4.1

Introduction.

One of the most important preflight analyses is that of determining the separation
ch
aracteristics of a store for employment and jettison
–

MIL
-
HDBK
-
244, MIL
-
HDBK 1763 and AG 300
–

Volume 5 document current practices in detail. Store separation characteristics are important because they directly
affect aircraft safety and weapon delivery
accuracy. Also, safety hazards associated with flight testing are
increased due to the mass of the stores being released, the uncertainty in the predictions, and the opportunities for
inflicting serious aircraft damage because of the relatively large numbe
rs of stores released at or near the
boundaries of the acceptable flight envelope. Experience has shown that proper use of predictive methods for

AIRCRAFT WEAPO
NS SYSTEMS TESTING




AIRCRAFT WEAPONS SYSTEMS TESTING

1
-

21

store separation will enhance the safety of store delivery and jettison during flight testing and subsequent
operational employment. Aircraft stores certification flight test programs can be considerably reduced, both in
scope and in cost, by reducing full scale flight tests based on a comprehensive analysis and positive correlation of
predictions with the fligh
t test data.
In future there is a continued trend toward increased stand
-
off distance (for
pilot safety), and a commensurate decrease in store aerodynamic stability (to increase glide time). We also see
a tendency toward folded fins and wings, for enhanc
ed packing density. Finally, with more concern about
reducing collateral damage, there is a trend toward smaller and lighter stores. All of these trends conspire to
make future stores more sensitive to unsteady weapon bay aerodynamic forces.

6.4.2

Method sele
ction.

Predicting accurate store separation trajectories on today’s high speed aircraft under
the varying conditions of altitude, Mach number, dive angle, load factor, and other factors related to delivery
techniques (particularly where multiple carriage

of stores is involved), is an extremely difficult task, requiring a
skilled and experienced analyst. Several techniques are available for store separation analysis, and these are
documented throughout the scientific literature.
There well proven wind tunn
el and Computational Fluid
Dynamic M&S experience that has supported advanced weapon
development and integration. Most NATO nations uses a
variety of unique CFD codes to augment wind tunnel testing.
These techniques have been extensively validated for ex
ternal
store separation. During the past decade, various AIAA
Challenges have seen great progress and the US, under the
auspices of the DoD High Performance Computing (HPC)
Modernization Program Office have combined each of the
Services initiatives to esta
blish an Institute for HPC
Applications to Air Armament (IHAAA) which has included
key NATO nations.
Some are purely analytical in nature,
utilizing theoretical aerodynamics and complex mathematical manipulation and analyst interpretation. Others
utilize

wind tunnel testing of small scale models of the store and aircraft, while still others involve a combination
of theoretical and wind tunnel data, utilizing a high speed digital computer for data reduction. Wind tunnel test
data for store separation may b
e obtained from one, or a combination of, the following:

a.
Captive trajectory.
This test uses a strain gauge balance within the separating store to continually measure the
forces and moments acting on the store. An on
-
line computer simulation determines
successive positions of the
store through its trajectory.

b
. Grid data.

An instrumented store or pressure probe is used to measure the forces and moments acting on the
store in the flowfield through which the store must separate. Trajectories are calculat
ed off
-
line using this
information as inputs to a trajectory program.

c
. Dynamic drop.

The dynamic drop tests use dynamically scaled models that are physically separated in the wind
tunnel. Data can either be photographical or telemetry. (This method is g
enerally limited to simulated level flight
releases only.)

d.
Carriage loads.

In this test forces and moments are measured on the store, with the store or weapon attached to
the aircraft in its correct carriage position. These data are used as inputs to t
rajectory computation programs.

6.4.3

No one technique will suffice for all cases. Rather, the analyst must examine the particular case to be
analyzed and select the technique that, in his opinion, offers the most advantages for his particular situation.
Most
purely theoretical techniques available today suffer severe degradation when applied to transonic store separation,
or where multiple carriage is involved.


AIRCRAFT WEAPONS SYSTEMS TESTING



AGARDOgraph 300
-
YY


1
-

22



6.4.4

The recent advances in computational fluid dynamics (CFD) and semi
-
empirical techniqu
es provide
excellent tools for engineering estimations or for use in conjunction with experimental data. As noted at Cenko,
Tutty et al (2003), captive trajectories and dynamic drops are expensive in that they are only for a specific flight
condition. Grid

data are superior because many flight and store characteristics can be changed while the grid data
are used as input for 6 degrees
-
of
-
freedom equations or other analytical tools. Wind tunnel testing is cheaper than
flight testing when the cost of aircraft

flight time, weapons assets, telemetry packages, and photogrammetric
analyses are considered. For these reasons, most analysts today employ hybrid methods which reduce costs while
retaining wide applicability. Several AIAA stores separation workshops hav
e seen M&S predictions challenged
by (hidden) actual flight test results. CFD has matured so that given one has enough time (and funds for advanced
computer time for the number of cases required), CFD can now clearly predict trajectories
–

however, time
and
cost effective M&S are still a trade
-
off against accuracy and fidelity.

6.4.5

Analyses.

As a first step in store separation analysis, all available flight test and predicted data pertaining
to the separation characteristics of the store in question, either
from the aircraft being examined or others with
similar installations, should be accumulated and screened for completeness of flight envelope coverage and for
trends. If existing data covers the store’s separation characteristics from the proper aircraft t
hroughout the desired
flight envelope, delivery conditions (speed, dive angle, load factors, altitude), delivery configuration and mode
(single, pair, ripple, etc), little or no additional testing may be required to allow certification. If this is not the
case,
however, additional data must be obtained in accordance with the method of store separation prediction chosen.
As noted by Dr Stanek in Cenko et al (2008), t
he mainstay analysis tool of the weapons clearance community is
the CTS system. This tool
was / is usually the right level of fidelity for external clearance problems, because it
has matched a very rapid prediction capability (due to the assumption of quasi
-
steady flow), with an external
store flowfield which WAS quasi
-
steady. External flow ov
er an aircraft in steady level flight (conditions when
you typically release weapons) is designed to be non
-
separated and steady. If the aircraft is well
-
designed, then
that is precisely what the external flow will be
–

attached, non
-
separated, and relati
vely steady. For external
weapon carriage and release, the flowfield that the store is immersed in is therefore predominantly steady, and
CTS works well for the prediction of store trajectory in the majority of those cases. The exception (where CTS
may n
ot work well) for external store integration is cases where aeroelastic effects predominate, and the
carriage structure movement itself is partly responsible for driving the flow unsteadiness.


AIRCRAFT WEAPO
NS SYSTEMS TESTING




AIRCRAFT WEAPONS SYSTEMS TESTING

1
-

23

6.4.6

Weapons bay cavity flows, on the other hand, are naturally and
rather dramatically unsteady,
due to a robust self
-
reinforced acoustic resonance phenomenon, coupled to and driven by an equally
robust free shear layer instability. Within the vicinity of the open weapons bay cavity, the quasi
-
steady assumption (the assu
mption that the store sees a single value of forces and moments at each
position and orientation which is constant and independent of time) does not hold. All weapons bays
with dimensions typical in modern aircraft installations exhibit this self
-
sustaine
d acoustic resonance,
to varying degrees. This would lead one to conclude that it may not be very conservative to use the
CTS process to clear weapons released from within weapons bays. The exception to the strong
unsteady weapons bay behavior are cases
where effective flow control devices (such as properly
designed leading edge spoilers) have been employed to suppress self
-
sustained oscillations, and the
resultant high acoustic levels and unsteady loads have been suppressed. Unsteady pressure levels in
unsuppressed weapons bays can be high enough (160 to 180 dB) to damage aircraft bulkheads, or to
“instantly” tear off weapon components

6.4.7

It is useful at this point to clearly define what is meant by unsteady weapon trajectory effects. It is
easiest to des
cribe the most dramatic case
–

which Dr Stanek at Cenko et al (2008) calls “bifurcation”. The
unsteady shear layer in the weapons bay is the dominant source of flow unsteadiness. When a weapons bay is
in strong acoustic resonance, the flow tends to take
on a two
-
dimensional character, with the formation of
coherent 2D vortical “rollers” which span the cavity. In this situation, an unsteady component of normal force
is created along the bay which changes sign from instant to instant
–

from “into the bay”
to “out of the bay”,
and vice versa. It is possible in this circumstance, depending on the time of release of the weapon, for the store
to tend to fly toward or away from the bay, depending on the time of release. This “pitch bifurcation” behavior
is the

most dramatic example of unsteady weapon trajectory effects.

6.4.8

Separation operating limitations.

Store employment covers separating the store from the aircraft in its
normal operational mode. It should cover separations at all speeds up to the allowable

in level and maneuvering
flight, both in the single release mode, and in multiple release (ripple) mode down to the minimum release
interval. Particular attention should be given to releases of unpowered stores in high dive angles (60o or greater) at
the
attendant low g (cosine of the dive angle). Such separations can be, and often are, extremely dangerous,
particularly for unstable or low density stores. In determining the separation envelope, the review should also
consider that some parts of the flight
envelope will not require analyses due to a more restrictive dive recovery or
safe escape limitation. It should also be kept in mind that proper store employment denotes not only safe
separation from the aircraft, but also that the separation be relatively

unperturbed so as not to adversely affect
delivery accuracy.

6.4.9

Analyzing the launch transient phase of store separation is extremely difficult. It generally involves
guided stores, such as electro
-
optical guided bombs, which contain autopilot and guidance
systems that are active
during store separation to avoid target breaklock or radical store movements caused by release perturbations. If
every component functions properly, separation will be completely safe and unperturbed. However, control failure
or spu
rious guidance signals causing abnormal control deflections at release can cause high
-
energy collisions with
the aircraft. Because of these possibilities, a reliability analysis of the store guidance and control system will be
performed, and the results of

possible failures identified and examined for probability of occurrence and effect on
store separation. Although no specific pass
-
fail criteria can be used in all cases, probabilities of failure of a single
component causing an impact on the aircraft shou
ld be kept in the realm of 10
-
6
. If this cannot be done, store
redesign should be effected prior to flight testing.

6.4.10

Internal weapons carriage is being used to improve the aircraft aerodynamic performance and low
observable characteristics. The separation

of stores from a weapons bay may be significantly impacted by the
unsteady flow in the bay. These temporal effects may not be captured during wind tunnel testing which use a
quasi
-
steady approach to run the Captive Trajectory System (CTS) to determine th
e store trajectories. One of
the first IHAAA tasks undertaken by the store separations team was the Store Separation from Cavity (SSC)

AIRCRAFT WEAPONS SYSTEMS TESTING



AGARDOgraph 300
-
YY


1
-

24


project. The goal of this project was to determine CFD application best practices for the separation of stores
from w
eapons bays. This is a current real
-
world problem that can benefit from the optimal application of HPC
technology.

6.4.11

Jettison criteria.

Jettison of a store (or a store/suspension equipment combination) involves the
releasing of items from the aircraft duri
ng emergencies (emergency jettison) or as normal operation after
expenditure of cargo or submunitions (selective jettison). Examples of these would be fuel tanks, gun pods,
dispensers and multiple bomb racks complete with some or all of its weapons. The p
rimary concern of any
jettison is to separate the item, or items, from the aircraft safely, without collision, because there is no requirement
for accurate delivery. This phase of store separation is by far the most dangerous to the releasing aircraft sinc
e
many items jettisoned are aerodynamically unstable, usually of low density, and their separation behavior is
generally erratic and unrepeatable. If at all possible, the jettison envelope of a store should be close to the full
authorized carriage flight e
nvelope. Jettisons are, however, commonly limited to level flight (plus and minus a
reasonable g tolerance, and sideslip of helicopters). Jettison envelopes that are limited to a single speed, or those
that specify a very narrow speed, altitude or dynamic
pressure band, should be avoided, if at all possible.


7.0

NETWORK ENABLED OPERATIONS

Australias NEO test and training

Axioms:


1. Experiment, test and train as we Australians need to fight operationally.

2. Safety and netwo
rk enabled effects based operations is to be the sine qua non.

3. Design our systems so it’s hard to use them wrong and it’s easy to gain insight into their performance!

4. Joint Mission Rehearsal is the aspiration for undertaking network enabled experime
ntation, test & training on ranges.


AIRCRAFT WEAPO
NS SYSTEMS TESTING




AIRCRAFT WEAPONS SYSTEMS TESTING

1
-

25

7.1

Test and Training Range network enabling.
The US has been quite aggressively pursuing
improvements to the test and training ranges for a number of years. Based on funding from the
Central Test & Evaluation Invest
ment Program (CTEIP) run by the Office of the Secretary of Defense
(that is separate from that provided by the Services or Acquisition
–

an important point as the
Australian tribalism on funding for infrastructure is actually much worse in the US!) numerou
s GPS
and range data links have been conceptualised, prototyped, demonstrated and developed. Of
particular interest is the Joint Advanced Missile Instrumentation project to implement modular
instrumentation packages that will dramatically reduce the siz
e and improve the interoperability of
flight termination, TSPI and vector
-
scoring of future weapons on US and Allied ranges
.

Such
systems are now in the prototype stages of development and, when used in conjunction with the
proposed US Weapon Data Link A
rchitecture this will be a potent improvement to test and
instrumented training operations for air armament. The Foundation Initiative 2010 was established by
the U.S. DoD to create a new infrastructure for DoD Range interoperability. The Foundation Init
iative
2010 organization, in partnership with the user community, created an advanced middleware product
called TENA (Test and Training Enabling Architecture). TENA directly targets the interoperability
and reuse needs of networked aerospace applications b
y defining:



A standard mechanism for efficiently communicating structured data (objects)
over networks



Pre
-
engineered vendor
-
neutral interfaces for common aerospace objects as well
as a powerful extension mechanism for creating new interfaces.

7.2

TEN
A defines general
-
purpose objects that represent telemetry
-

and range
-

related entities
such as radar systems, telemetry feeds, aircraft and other moving objects, and time/space coordinates.
For example, a complex operation such as retrieving the real
-
ti
me coordinates of an aircraft becomes a
call to a TENA function such as aircraft time and spatial position information. TENA middleware
communicates over any IP network, initial applications are now starting on US test ranges and will be
available for use
by Australia shortly. A more complete description of TENA (and the graphic in
great detail) is available at Cannon (2004) and the distributed object computing middleware at
Noseworthy (2002).

7.3

Of equal importance to TENA is the Next Generation Range I
nstrumentation (called NexRI)
and the ‘instrumentation NETwork’ (called iNET) initiatives of the USAF and USN. Range
Instrumentation Systems, Air Combat Support, Air Armament Center, at Eglin AFB, Fl, plans to
award a sole source contract to Rockwell Coll
ins (who has also won a contract for the Weapon Data
Link Architecture Concept Demonstration phase) for the follow
-
on phases to the current Next
Generation Range Instrumentation Tactical Targeting Network Technology
-
to
-
Range Instrumentation
Waveform runnin
g through FY07 (see Tutty (2005)) where each phase builds upon the previous phase.
A sole source contract was identified as the only way to ensure the P
5
Combat Training Systems
proposed for all US ranges and F
-
35 JSF will meet schedule requirements to be
gin integration for
JTRS compliance by 2007. JTRS compliance is the first step to permit interoperability of test and
training range systems by FY09. The iNET study seeks to incorporate developments in network
operation into the telemetry networks avai
lable at US ranges to ensure that test and training ranges are
able to collect the data needed during network enabled testing.

The third axiom is driven by the JMETC design philosophy to

reduce the likelihood that the systems can be use
wrong and the goal
that such systems will be collecting higher levels of system performance without pain to the


AIRCRAFT WEAPONS SYSTEMS TESTING



AGARDOgraph 300
-
YY


1
-

26


Users.

8.0

TEST REPORTING


8.1

While the test planning and test conduct are often seen as the most important phases of a test
program, the most important is the th
oroughness and rigour of the test reporting. A test report must
clearly document all the findings and recommendations not just for the system under test but also for
improving the test methods of other programs. A summary of the recommended practice appli
cable to all
contemporary and future weapons systems is included at AGARDOgraph

300
-

ZZ
.


Note this test reporting document may as well be inserted,

unless a stand alone document is better...


9.0

CONCLUSIONS

9.1

This article has identified the wide rang
e of weapons systems testing used to ensure aircraft
stores compatibility and effects
-
base outcomes in contemporary weapons systems.


10.0

BIBLIOGRAPHY / REFER
ENCES

AGARDographs Series 160, Flight Test Instrumentation, NATO RTO, see
www.rta.org

and
stinet.dtic.mil/agardographs.html

AGARDographs Series 300, Flight Test Techniques, NATO RTO see
www.rta.org

and
stinet.dtic.mil/agardographs.html


Alberts, Dr D.S. and Hayes Dr R.E., 2002,
E
xperimentation; Code of Best Practice
, Command and
Control Research Program, [Online, accessed 1 December 2004]. URL: http://www.dodccrp.org

Alberts, Dr D.S. and Hayes, Dr R.E., 2005,
Power to the Edge: Command… Control… in the
Information Age
, 3
rd

Prin
ting April 2005, Command and Control Research Program, [Online,
accessed 31 December 2006]. URL: http://www.dodccrp.org

Alberts, Dr D.S. and Hayes, Dr R.E., 2007,
Planning: Complex Endeavours.
Command and Control
Research Program, April 2007 [Online, acc
essed 15 July 2007]. URL: http://www.dodccrp.org

Ball, Robert. E., 2003,
The Fundamentals of Aircraft Combat Survivability: Analysis and Design
,
AIAA Education Series

Bock, P., 2001,
Getting it Right: R&D Methods for Science & Engineering
, Academic Pres
s

Cebrowski, VADM Arthur K. USN, and Garstka, J. J. 1998, Network Centric Warfare: Its Origin and
Future,
Proceedings of the Naval Institute

124, Jan 1998, 232
-
35.
[Online, accessed 10 May 2004].

Cenko,

A., Benek,

J., Deslandes,

R.,
Dillenius, M., Stane
k,

M.,2008, Unsteady Weapon Bay
Aerodynamics
-

Urban Legend or Flight Clearance Nightmare,
AIAA

2008
-
0189


AIRCRAFT WEAPO
NS SYSTEMS TESTING




AIRCRAFT WEAPONS SYSTEMS TESTING

1
-

27

Farrier, A SQNLDR, Appla, D & Chadwick, J. 2004,
As Easy as ABC
, ADF Experimentation
Symposium, Defence Science & Technology Organisation, Jun 2004

Ga
rstka, J.J. 2000,
Network Centric Warfare: An Overview of Emerging Theory
, Joint Staff
Directorate for C4 Systems
,

US DoD, Washington, USA [Online, accessed 10 May 2004]. URL:
htt
p://www.mors.org/publications/phalanx/dec00/feature.htm


Gartska, J.J., 2005,
Fighting the Networked Force
, Presentation to the NDIA Network Centric
Operations Conference, 21 March 2005, [Online, accessed 15 July 2007]. URL:
http://www.dtic.mil/ndia/2
005netcentric/2005netcentric.html

Grove, J., et al, “USAF/RAAF F
-
111 Flight Test with Active Separation Control,”
AIAA paper 2003
-
9
, 41st Aerospace Sciences Meeting and Exhibit, 6
-
9 January 2003, Reno, Nevada.

GUIDEx (2006),
TTCP Guide for Understanding an
d Implementing Defense Experimentation
(GUIDEx),

The Technical Cooperation Program, [Online, accessed 15 July 2007]. URL:
http://www.dtic.mil/ttcp/guidex.htm

Harmon, M, 1997,
Entropy Based Warfare: A Unified Theory for Modeling the Revolution in Military

Affairs
. Booz
-
Allen & Hamilton, USA

Heuer, R Jr, 2003, The Psychology of Intelligence Analysis, Center for the Study of Intelligence, CIA,
3
rd

Edition, Washington DC ISBN 1920667000

ISO 9001, 2004, Quality Management, ISO, 2004

ISO/IEC 17025, 1999,
Gene
ral Requirements for the Competence of Testing & Calibration
Laboratories,
ISO and the IEC, 1999 See URL:
http://www.fasor.com/iso25/

or
http://www.omnex.com/Standards/iso_17025/iso_17025.html


Kass, R.A., 200
6,
The Logic of Warfighting Experiments
, Command & Control Program, DoD, August
2006, see dod.ccrp.org

Leedy, P.D, & Ormrod, J.E., 2001,
Practical Research: Planning & Design
, 7
th

Ed, Prentice
-
Hall, Inc,
United States of America


Leugers, J., et al, “Flig
ht Test Demonstration of Miniature Munitions Release from Internal Weapons
Bay
–

Final Test Report”,
AFRL
-
MN
-
EG
-
TR
-
2002
-
7011
, January 2002.

MIL
-
HDBK
-
244A, 1990,
Guide to Aircraft Stores Compatibility,
US Department of Defence, USA.
dated 6 April 1990.


MI
L
-
HDBK
-
1763, Aircraft Stores Compatibility, Design and Test Requirements, US DoD, USA

MIL
-
HDBK
-
516B, 2005, Airworthiness Certification Criteria, US DoD, USA. 26 Sep 2005

MIL
-
STD
-
810F, 2000,
Environmental Engineering Considerations and Laboratory Tests,

US
DoD,
USA.

MIL
-
STD
-
882C, 2002,
Standard Practice for System Safety
, US DoD, USA.


AIRCRAFT WEAPONS SYSTEMS TESTING



AGARDOgraph 300
-
YY


1
-

28


MIL
-
STD
-
1289D, 2003
,
Requirements For Airborne Stores Ground Fit and Compatibility
, US DoD,
24
September 2003


MIL
-
STD
-
1760D
,

2004,
Interface Standard for Aircraft/Store Interc
onnection System,

US DoD,
2004

MIL
-
STD
-
1797A, 1995, Flying Qualities of Piloted Vehicles, US DoD XXXX 1995

MIL
-
STD
-
2072, 1977,
Survivability, Aircraft, Establishment and Conduct of Programs for
, US DoD,

25 Aug 1977

Noseworthy, J,R., 2002, IKE 2
–

Implement
ing the Stateful Distributed Object paradigm,
5
th

IEEE
International Symposium on Object Orientated Real
-
time Distributed Computing
, ISOROC 2002,
See URL: http://www.cs.wustl.edu/~schmidt/PDF/OORTDCSpaper.pdf

SFTE Handbook,

Smith, Dr E.R., 2002, Effects Ba
sed Operations
–

Applying network centric warfare in peace, crisis
and war, Command and Control Research Program, USA [Online, accessed 1 December 2004].
URL: http://www.dodccrp.org

St
i
nton,

D. 2001,
The Design of the Aeroplane,
2
nd

Ed, Blackwell Science

Ltd, UK.

Tutty, M.G., 2005,
Australian Aircraft Stores Capabilities in a Network Enabled World
, University of
South Australia, 31 January 2005.


11.0

KEY WORDS

Test and evaluation, experimentation, experimental, ground and flight test, aeroacoustics, aero
elastic
ground vibration test, aircraft flutter, aircraft loads, aircraft stores compatibility, armament system
software changes, ballistics, captive flight profile, carrier suitability, electromagnetic analysis,
employment, environmental testing, fit and
function, flying qualities, jettison, operational flight
program, safe escape, static ejection, store mass properties, store separation, structural integrity,
vibration and endurance, wind tunnel, weapons systems test

12.0

ABOUT THE AUTHOR

The author joine
d the RAAF

as an engineering cadet in 1980 and has served in the Air Force and Australian
Public Service in a multitude of engineering and test roles including an exchange tour with the 3246 Test Wing
/ TY until the Gulf War OT&E I, then as Director ASCENG

for over 800 aircraft stores combinations for over
20 aircraft types that have serviced so many DIMPIs / targets it is now best conservatively described by the
power law: P
>x

=

x
–


PetaDIMPI’s Serviced, and the Director of the worlds largest land
-
based W
oomera Test
Range prior to joining the dark side for an outstanding time as

Chief Engineer for Maritime Patrol and Force
Applications, Tenix Defence Aerospace for the AP
-
3C Orion $1Billion upgrade and was in the Active Reserve
as the Red Weapons Analyst, D
efence Intelligence Organisation. In 2008, he was invited to rejoin the RAAF as
Director Trials & Range Management, AFHQ. He has a

Bachelor of Electronic Engineering with Distinction
from RMIT
and a Masters in Systems Engineering from the University of S
A. He is doing a PhD in his spare
time.

He was Listed in Who’s Who in
the World for
Science & Engineering in 2003, and has been a Fellow of
the Royal Aeronautical Society and the Institution of Engineers (Australia) since 2002. His interests include:

AIRCRAFT WEAPO
NS SYSTEMS TESTING




AIRCRAFT WEAPONS SYSTEMS TESTING

1
-

29

his

family, reading, shooting, flying, the study of haute couture and aesthetics, golf, travel, drinking good red,
bourbon and beer, and investigating novel applications of air power.

13.0

ACKNOWLEDGEMENTS

Kevin Christensen, LtCol USAF, SETP

Alex Cenko, USN N
AVAIR Separations SME

William D. Hack, Colonel USAF, ASC SME

Ron Hack, WGCDR (RAAF Rtd), QANTAS Captain

Neal Siegel, NAVAIR, USN

WGCDR Mark Washusen, CO ASCENG, RAAF

UK???

CA???

Lt Col Dave Fawcett

Roger H Beazley

Dr Viv Crouch

Professional Societies invol
ved representatives from
–


International Test and Evaluation Association

Society of Experimental Test Pilots

Society of Flight Test Engineers

NATO Research & Technology Organisation Flight Test Technical Team

Flight Test Society of Australia


ANNEXES:

A.

De
finitions and Acronyms

B.

Definition of Flight Test Team


AIRCRAFT WEAPONS SYS
TEMS TESTING


AGARDOgraph 300
-
YY


1
-

30



ANNEX A:

DEFINITIONS & ACRONYMS

ARDU.
Aircraft Research & Development Unit. The Royal Australian Air Force unit responsible for
planning, conducting and reporting of Air Force and Army aerospa
ce T&E.

Critical Operational Issue (COI).

A key operational effectiveness or operational suitability
issue that must be examined during T&E to determine the system capability to perform its mission.
A COI is normally phrased as a question to be answere
d in evaluating a systems operational
effectiveness/suitability.

Critical Technical Characteristic (CTI)
. A quantitative or qualitative parameter of system
performance whose measurement is a principal indicator of technical achievement. Critical
techni
cal characteristics must be testable, measurable and verifiable.

Experimentation.
n. the act or practice of making experiments; the process of experimenting; a
product that is the result of a long experiment. Macquarie Concise Dictionary (1998)

In th
e
scientific method, an
experiment

is a set of actions and observations, performed to verify or falsify
a hypothesis
or

identify a causal relationship between phenomena. The experiment is a
cornerstone in the empirical approach to knowledge. Wikipaedia

(2005)




Type of
Experimentation

Experimentation
Application

Aims

Discovery

Analysis:

•

Problem
Space
Examination &
Reproduction

To provide some insight into the nature of any
new performance variables or relationships
that have been demonstrated by the
in
troduction of the concept or capability.
To
give an idea of future research that may be
undertaken in order to further refine the
innovation.

Hypothesis
Testing

Verification:

•

Unit Test

•

Integration
Test

•

Set To Work

•

DT&E

•

AT&E

To make comparisons between a
lternative
cases.

To show that an effect was created.

To show that the cause of that effect is
demonstrable. To show how findings can be
generalised to some extent to real
-
world
military operations.

Demonstration
Experimentation

Validation:

•

OT&E

To demon
strate a capability or concept that
has been proven.
To explore the range
of conditions over which a capability will exist
and to ensure the capability is robust in the
military context.

Table 1. Types of Experimentation
–

courtesy TTCP GUIDEx

(2006) and Smith (2007)



AIRCRAFT WEAPONS SYS
TEMS TESTING


AIRCRAFT WEAPONS SYSTEMS TESTING

1
-

31

Fidelity

–

The degree to which a model realistically represents the system or process it is
modelling

–

not necessarily the level of granularity, detail or complexity of the model.

Model
–

Any representation of a function or proce
ss, be it mathematical, physical, or descriptive.

They are typically of two categories
–

representations (employing some logical or mathematical
rule) and simulations (which mimic the

detailed structure of the system and may include
representations of sub
systems or components) that may be made up of one or several of: physical,
graphical, mathematical (deterministic) and statistical (probabilistic).


Similarity.
State of being similar, a point of resemblance.

Simulation.

A computer program that repre
sents the operation of a function or process to the
degree of accuracy necessary to meet its purpose.

Typically realistic or representative scenarios
are run in the time domain to simulate the behaviour(s) of the proposed or real system. INCOSE
SE Hand
book (2000)


Responsible Test Organisation.


RTO.
Responsible Test Organisation.


Standard.

A description of a process, material, or product meant for repeated use in one of more
applications and covers materials, processes, products and services. NATO A
AP 6

Test and Evaluation (T&E).
T&E is the process by which a system is

compared against technical or
operational criteria
through testing and the results are

evaluated to assess performance against agreed
criteria
. T&E is usually conducted to assist i
n making engineering, programmatic or process decisions
, and
to reduce the risks associated with the outcome of those decisions. In control theory and management
terms, T&E can be best thought of as the negative feedback loop on the capability life cycle
management
process.


V&V.
Validation & Verification. T
he process of checking that a product, service, or system
meets specifications and that it fulfills its intended purpose. V&V is one of the disciplines of the
overarching function of T&E. V&V ar
e key components of Quality Management Systems such as
ISO 9001 (2004). Independent V&V is the particular form of the V&V discipline that, due to the
increasing complexity of systems, is growing in importance due to the independent techniques and
methodol
ogies employed in IV&V which are well suited to the acquisition of complex systems.


Validation.
The process of evaluating a system or component during or at the end of the
development process to determine whether it satisfies specified requirements.

T
he process of
determining the degree to which a
model

is an accurate representation of the real world from the
perspective of the intended uses of the model.
American Institute for Aeronautic & Astronautics
(1998)

Confirms that the system, as built, wil
l satisfy the user’s needs

–

ensures that “
you
built the right thing
”. INCOSE SE Handbook (2000)

Verification.
The process of evaluating a system or component to determine whether the products of a
given development phase satisfy the conditions imposed

at the start of that phase.

The process of
determining that a
model implementation

accurately represents the developers conceptual description of the
model and the solution to the model.
American Institute for Aeronautic & Astronautics (1998)

Address
es
whether the system, its elements, its interfaces, and incremental work products satisfy their
requirements

-

ensures that “
you built it right
”. INCOSE SE Handbook (2000)

ANNEX B:

DEFINITION OF FLIGHT TEST TEAM

AIRCRAFT WEAPONS SYS
TEMS TESTING


AGARDOgraph 300
-
YY


1
-

32


The Definition of a Test Pilot

A test pil
ot is a pilot who undertakes the testing of an aircraft, its parts or associated systems within the
bounds of a specific approval granted for that purpose.

Activities

A test pilot may conduct flight testing of an aircraft, its parts or associated systems f
or one or more of the
following purposes:



To establish or expand the flight envelope



To establish whether the handing qualities, performance, flight characteristics, systems,
displays and human factors associated with the aircraft are safe and comply with
the
regulatory and certification requirements



To establish whether the handling qualities, performance, flight characteristics, systems,
displays and human factors associated with the aircraft are fit for purpose



To provide data and observations in support

of experimental or development programmes



To assess novel, unusual systems, displays or procedures



To establish new piloting techniques



To display, demonstrate and provide flight training on certified and non type certified
aircraft

In addition, a test pi
lot may be required [with or without other personnel] to provide:



Flight Safety and risk assessment advice including the determination of higher risk
activities



Flight Test programme and scheduling advice, instrumentation requirements, advice of
flight tes
t profiles planning and special test requirements



Advice as necessary on flight manual, operational and environmental; [noise, public
nuisance, weather] matters



Verbal and written debriefs as necessary



AIRCRAFT WEAPONS SYS
TEMS TESTING


AIRCRAFT WEAPONS SYSTEMS TESTING

1
-

33

The Definition of a Flight Test Engineer

A flight te
st engineer is an individual who is responsible for the technical management and/or coordination
of a flight test programme [or part thereof], including the active participation in the airborne task.

Activities

A flight test engineer may be responsible for

any or a combination of the following activities:



Preparation of the flight test programme, provision of advice



Co
-
ordination and specification of instrumentation and/or telemetry requirements and
recording needs



Drafting the flight test instruction/order



Participation in the test flight, provision of airborne advice



Participation in the debrief process including the drafting of reports and retrieval of
instrumentation and telemetry data

In addition, a flight test engineer may be required [with or without
other personnel] to provide:



Flight safety and risk assessment advice including the determination of higher risk
activities



Flight test programme and scheduling advice, instrumentation requirements, advice on
flight test profile planning and special test r
equirements



Advice centred on the flight test process including liaison with design personnel on
development matters, modifications and changes, and with airworthiness authorities in
relation to the certification process



Verbal and written debriefs as nece
ssary