Interoperability Testing using the

fishecologistMobile - Wireless

Dec 12, 2013 (3 years and 7 months ago)

53 views

Acceptance testing &

Interoperability Testing using the
RTD

Our Message


3G terminal behaviour needs careful
examination to make sure that they perform
efficiently and correctly on live Networks


Live network testing will still be used as the
ultimate ‘end user’ test but has limitations and
disadvantages


The RTD provides a fast way to create and run
tests that are required for UE acceptance & IOT
before deployment


There are many RTD Libraries available now to
use as a good basis for a test regime


How is UE testing done today


Live networks


Test networks / TestFests


System simulators / test
beds / approved test beds

Field Trials

final test of
product in the
true operating
environment and
with ‘real’
customers


Test networks

initial evaluation of
product in ‘real world’
controlled environments.

TestFests used for
applications

Laboratory system simulators

controlled testing for repeatability and
fault investigation

test conditions are easily re
-
configured

Future network upgrades can be tested


Time
consuming,
very expensive


Susceptible to
local / external
conditions


Unrepeatable,
and diagnostic
data limited to
location



Fast, low cost and flexible implementation


Fully repeatable and reproducible, replicate real
network problems


Potential scenarios
-

‘What if?’


Excellent diagnostics


Costly and limited
network size



Limited scope and
limited capability


Constrained inter
-
network operation


Test that different manufacturer’s equipments work
together correctly in a real network


Ensure terminals stay on their (or preferred) networks


NW Manufacturer A

Software version 1

NW Manufacturer A

Software version 2

NW Manufacturer B

Node B Mfr C

Node B Mfr D

UE Mfr E

UE Mfr F

UE Mfr G

Interoperability

(Network Operators Needs)

What is RTD
-

Rapid Test Designer


Rapid Test Designer
(RTD) is a unique
tool that significantly
speeds up the
testing of W
-
CDMA /
GSM devices by
greatly simplifying
the way in which
tests are created,
executed, and
analyzed.

Handset

under

test






Test Execution Engine







Building Blocks

for Tests






Controls

NAS

RRC

SS Proc

User Tests



Test

Criteria Editor


High Level Analyzer

Detailed

Test Log Analyzer

Reference

Tests


Test Result

Archives


Tools within the RTD

PROCEDURE LIBRARY

Anritsu 3G Track Record

Innovation then commercialisation


System Simulator MD8480

Set the industry benchmark with the
earliest, highest functionality and
most stable 3G layer 1 network
simulator, which is now the
worldwide market leader with over
1500 sold to date


3GPP Protocol Test System

Set another industry benchmark with
the earliest, highest functionality
and most stable test system for
3GPP Layer 2&3 protocols


MD8480

MD8480
A

W
-
CDMA

MD8480
B

W
-
CDMA

GSM / GPRS

MD8480
C

W
-
CDMA

GSM / GPRS

HSPA+

(3Cells)

(3+1Cells)

(4+2Cells)

MD8480 has evolved from W
-
CDMA only and now is able to simulate up to 4 HSPA cells

Evolution of MD8480

The RTD uses a true
graphical interface
allowing the purpose
of the test to be
easily understood

Tests can be
commented and user
prompts enabled

Test flow is easily
followed

Procedure Picker

Test Editor Window

Information Window

In this example the only
sensible options are to

either configure the system or
shut it down.

Procedures

Include

Shows test path
progress during

execution

Graphical interface


Parameterisation dependent on the users Network


Multiple catalogue support in a test


Creation/Deletion of user defined catalogues


Copying of catalogues


Cut & paste copying between catalogues


Editing of catalogue entries

ExE
E
Confidential

Network models and catalogues
simplify creation of variants

RUN VIDEO TO SEE

Key Features


Cuts Development Time


Provides an effective way of creating tests (and test
variants) in a fraction of the time taken by traditional
methods


Hides Complexity Of 3GPP Protocol Testing


Allows the user to concentrate on testing specific
functions and protocols within the UE without having
to be an expert on all the protocol layers.


No Need For TTCN Knowledge


Many engineers are unfamiliar with TTCN and do not
want to use it.


Improves Productivity


Facilitates creating many different network
simulations and testing the behavior of many UEs.

Benefits to Network operators


Reduce the time taken to get a UE available
on the network


Beat the competition TTM


Additional applications possible


Frequent model changes possible


Reduce the number of UE related problems
encountered in the field


Happy customers


Less churn


Increased revenue per user

Why Use the RTD ?


Simple to create simulation of network
conditions and test specific functions


Some tests cannot be done reliably on a
live network


Fast objective results


Can be performed before network features
are present


Tests can be updated in line with 3GPP
standards

What sort of Tests


Selection and re
-
selection of networks


Emergency call procedures


Handovers


Packet calls and state transitions


RAB tests


Measurements





The Future


AT&T (and other operators) gating tool


Support for eHSPA / LTE


Realisation that IOP needs Test Platforms


Libraries will assist organisations that do
not have resource to create test
campaigns

10

Things you need to know


Fast

Test creation, execution, and maintenance


Flexible


Easy to create test variants and test campaigns

Multi
-
RAT & multi
-
functional


Future proof

HSPA and beyond before the competition


Reliability and repeatability

Campaigns of tests can be run & each test can include automatic control of the terminal


Robust platform

System runs unattended for long periods


Multi
-
RAT in a small package

GSM, GPRS, E
-
GPRS, W
-
CDMA, HSPA in one compact system


Results criteria can be built into tests

Quick analysis of performance and results


Test libraries available

Hundreds of tests available immediately


Annotation on screen

Test guidance and run time notes for users


Secure test traceability

Watermarking in tests to show if there have been no changes to agreed requirements.

Additional slides


These slides may help with explaining
features within the RTD

Automation


Key Points

Detailed breakdown

1
-

Test Executive Control

1.
The Test Executive (RTD) has now been successfully
trialled against many UEs from different vendors

2.
The combined AT/MMI proxy was developed as a
flexible interface which can adapt to many different
types of UEs through the use of a configuration file
‘proxy.xml’

3.
Full control of a UE is only possible if the UE vendor
has made provision for an automated testing interface

4.
In general automated testing can be carried out via the
use of the AT command set [3GPP TS27.007]

2


Automated Test Scripts

1.
In general the AT&T and T
-
Mobile USA UMTS Protocol
Libraries have been designed with automation in mind

2.
Where possible AT and MMI commands have been
added to each script so they are ‘Individually
Automated’. These AT commands are sent directly to
the UE via the combined proxy or in the case of MMI
commands they are interpreted by the ‘proxy.xml file’
and then sent to the UE via the combined proxy.

3


Test Throughput

1.
In order to execute many tests sequentially the CLI
(Command Line Interface) was developed.

2.
This interface is relatively easy to use, it is in a text file
format and currently forms the primary regression
engine for all library testing in Anritsu

3.
Although a great deal of ‘Batch Automated’ testing was
carried out using the AT&T and T
-
Mobile USA UMTS
Protocol Libraries with a beta version we have now
included the proxy as part of RTD

4


UE Control

1.
In general automated testing can be carried out via the use of the
AT command set [3GPP TS27.007]

2.
Where the vendor has not implemented a given AT command, for
instance AT+CFUN which is used to turn off the UE between tests,
it may be possible to use the Keypad Control command
(AT+CKPD) which simulates the pressing of the UE key pad via
the AT command interface.

3.
Some vendors have not implemented an automated testing
solution. In this case the Protocol Libraries have been designed to
cope with this, however an operator must be present at all times to
carry out the operations indicated by the executing test script.
[Note : T
-
Mobile will not accept a UE in this condition, even if it
passes all the tests in the library]

RTD upgrades tests to latest 3GPP
release in seconds !


Existing user tests are automatically upgraded to
the latest version of the RTD and 3GPP release

A backup of the test has been
created automatically during
the upgrade

Campaign Management


Key
Points

Detailed breakdown

Tests are dropped into the
campaign creator page

Test runs and Repeat
conditions can be set.

Progress is shown as the tests
are run

Statistical test reports can be
produced

Results can be
exported

to .xml or .csv format