Building The Ultimate Device Matrixx - Meetup

fearlessquickMobile - Wireless

Dec 12, 2013 (3 years and 11 months ago)

80 views

Picking The Right Set of
Mobile Devices

By Brian Kitchener

Software Quality Architect

bkitchener@prototest.com

Overview


About me


Some Background


The Problem


Understanding Android


How Apps Work


Building a Device Matrix


Example Matrices


Conclusion






About Me


Software Quality Architect at
ProtoTest


We're
a mobile test lab

that combines usability testing with quality
assurance

to

cultivate

a great user
experience


Project Architect, Technical Lead, Trainer.


Started in QA in 2001


BA in Applied Computing from University of Denver


Testing background : Functional, Performance, UAT,
Security, API, Database.


Automation : Selenium,
WebDriver
,
WatiN
,
MonkeyTalk
, SOASTA,
Fitnesse
, QTP,
EggPlant
, Squish


Languages : C#, Java, Ruby,
Javascript

BACKGROUND
INFORMATION

Some Stats for 2012


Mobile Apps achieved $17 billion in sales


5.2 Mobile Subscribers


1.2 Billion PC’s


4.2 Billion people use a toothbrush


1 Billion Smartphones


722 Million Smartphones sold


1.4 Million
iOS

+ Android Apps


25 developers
= half of app revenue

iPhone
-

June 2007

About the iPhone


Steve Ballmer :
Microsoft CEO


“There’s no chance the iPhone is going to gain significant market
share. No chance
.”


Patrick Stewart:



Last Wednesday, I stupidly dropped my iPhone in the bath, and my
life has sort of spiraled almost out of control
.”


Jon Rubinstein


Palm CEO


Is there a toaster that also knows how to brew coffee? There is no such
combined device, because it would not make anything better than an
individual toaster or coffee machine. It works the same way with the
iPod, the digital camera or mobile phone: it is important to have
specialized devices
.


Mike
Lazaridis



Blackberry CEO


And
so what
[the iPhone] has actually
done is increased our sales
.

ANDROID IS THE
PROBLEM

And Then There Were Two…


Android unveiled November 2007


First device was sold in October
2008.


Over 11,000 models have been
released.


48 Billion app installs


Over 1 Billion Android devices
activated


8 OS Revisions

OS Fragmentation

Device Fragmentation


Let’s do some math!


16 device display categories


20 different common resolutions


8 OS versions


6 Hardware Manufacturers


4 Major cellular networks


16
x 20 x 8 x 6 x 4 =
76,800 permutations


Pairwise approach = over 30 permutations


Who can afford to increase testing by 30X?


Our Approach


Efficiency, not coverage


Flexible: support small or large number of devices


Understand how apps work to logically select criteria


Use Market research to pick most common
configurations


Pick minimum and maximum boundary values for each
criteria


Choose a value that matches an edge case or abnormal
configuration.


Pick values that stress or tax the system



UNDERSTANDING
ANDROID

Android


Built and Maintained by Google


Open Source


Built on Linux kernel


ARM


X86 Ports


Built to support almost any type of device


Phones, tablets, phablets, media players,
tv’s
,
watches, etc.


Device Manufacturers customize code.




Example: Kindle Fire


Forked Android 2.3


Not updateable


Customized UI


Separate App store


Not all android apps work


Custom web browser




The Operating System


Google Releases “stock”
versions


10 Major Releases since
2008


API Level, not Version


Device manufacturers like
to customize the OS


Drivers, libraries, UI


“Stock” OS available in
Nexus devices or an
Emulator



Simulators / Emulators


Simulator imitates the software layer


OS and Libraries


Apple provides a simulator in
xCode

IDE


Emulator duplicates the hardware and software


Processor and Memory


Cannot mimic GPU, GPS, accelerometer


Always run stock OS


Can be used to test some functionality


Should always test on a physical device too

The Processor


ARM RISC
-
based instruction set


Specification defined by ARM holdings


32 bit


Same as
iOS


X86 patches and ports


It’s only a spec, can be modified.





SoC



System On A Chip


Main Board


Processor


RAM Bus


GPU


May include :


Cellular


WiFi


NFC


GPS


Bluetooth



2 Samsung Galaxy S4s

Quallcomm

Snapdragon


Quallcomm

Krait 300


Quad core ARMv7
Cortex A15 Architecture


Adreno

320 GPU


Dual Channel 533Mhz
Bus


Integrated LTE

Samsung
Exynos

5
Octa


Samsung
Big.little

processor


Quad core Cortex A 15


Quad core Cortex A7


PowerVR

SGX 544 GPU


Dual Channel 800Mhz
Bus


No Integrated LTE

Common
SoC

Manufacturer

Device Name

Cores

Processor

GPU

Qualcomm

Snapdragon S4

2 or 4

ARM Cortex
-
A15

Adreno

Nvidia

Tegra 3

4+1

ARM Cortex
-
A9

Geforce

Samsung

Exynos 4

2 or 4

ARM Cortex
-
A9

Mali

Intel

Medfeld

1

Intel x86

PowerVR

Texas
Instruments

OMAP 4

2+2

ARM Cortex
A9

PowerVR

ST
-
Ericcson

NovaThor

2

ARM Cortex
-
A9

PowerVR

Resolution is not enough


Unlimited number of
screen sizes available


Screens range from 3” to 11”


Each screen has a resolution, same as a
monitor


If you increase the resolution everything shrinks!


Pixels per Inch = Density


Screen Size + Density = Display Bucket


Resolution is not enough!


The Display Buckets

Size :


Xlarge

: 8”
-

10” tablet.


Large : 5”
-

7” tablet.


Normal : 3.5”
-

5” phones.


Small : 3”
-

3.5” phones.

Density :


ldpi

= Low DPI (~120)


mdpi

= Medium DPI (~160)


hdpi

= High DPI (~240)


xhdpi

= Extra High DPI
(~
320)


Low density
(120),

ldpi

Medium
density
(160),

mdpi

High
density
(240),

hdpi

Extra high
density
(320),

xhdpi

Small
screen

QVGA
(240x320)

480x640

Normal
screen

WQVGA400
(240x400)

WQVGA432
(240x432)

HVGA
(320x480)

WVGA800
(480x800)


WVGA854
(480x854)


600x1024

640x960

Large
screen

WVGA800*
*
(480x800)


WVGA854*
* (480x854)

WVGA800*
(480x800)


WVGA854*
(480x854)


600x1024

Extra
Large
screen

1024x600

WXGA
(1280x800)


1024x768

1280x768

1536x1152

1920x1152


1920x1200

2048x1536

2560x1536


2560x1600

Display Buckets


Galaxy S3


1280 x 720


Xhdpi

density (331ppi)


Normal screen (4.7”)


Galaxy Tab 10.1


1280 x 800


ldpi

density (149ppi)


Xlarge

sceren

(10.1”)


Galaxy Note LTE


1280 x 800


hdpi

density (285ppi)


Large Screen (5.5”)




Market Analysis

Aspect Ratio


UI is manipulated from code


Density Pixels adjust for screen size


But can use regular pixels!


Need to take both X and Y into account!


Easy to overlap or hide things


Includes orientation


Some devices include an aspect ratio changer!
(LG
Optimus

Vu)

Cellular Carrier


Four Major US Networks


Verizon, Sprint, AT&T, T
-
Mobile


Some phone interoperability


2 protocols



GSM


T
-
Mobile AT&T


CDMA


Verizon and Sprint


Carriers assigned specific frequency bands


LTE will be new standard
-

But spectrum issues will
prevent cross
-
network phones


So if the phone supports the carrier’s protocol
and band it can theoretically connect.


HOW APPS WORK

How Apps work


Apps need to work on all screen sizes



May not be functional


May be wasted space


May not make sense


Apps define XML layouts similar to HTML



Node structure


Static Content


Images,
etc


Dynamic Content


Color, Text, etc.

Layouts and Fragments


XML Fragments are
reusable components


Layouts stitch together
fragments for a specific
sized device


App may need different
flow for tablet
vs

phone


BUILDING THE

DEVICE MATRIX

Our Criteria


Operating System


OS customizations, missing libraries, driver issues,


Screen Size


Rendering issues, usability, missing layouts


Pixel Density


Density Independence, missing layouts.


Aspect Ratio


X,Y calculations, overlapping panels, display issues


SoC


Hardware performance, Instruction set, battery, signal


Carrier


Network protocol, speed, responsiveness, packet loss



The Goal


Efficiency, not coverage!


Build a set of devices to be used for app and
website testing.


Know when to update them


Define a list of simple categories of devices


Pick devices that offer broad coverage


Adjust the number of devices based upon
needed coverage




Categorical Approach


Define scope



Android,
iOS
, phone, tablet, etc
.


Understand Testing requirements


Self
-
descriptive Names


Help to broaden coverage


Will adjust devices chosen to cover our criteria


Should be apparent when to update a device


Spread coverage :


Usage
-
> Edge Cases
-
> Strange
-
> Stress

Example Categories


Common


Matches most common
display configuration


Newest


Latest OS version, largest
screen, highest resolution


Oldest


Oldest, slowest, smallest
device.


Abnormal


Non
-
standard OS, aspect
ratio, orientation, size


Popular


Most popular device in
terms of sales


Budget


Low
-
priced new model.
Tend to have strange specs


Flagship


Nexus device running stock
Android OS


Catch
-
All


Cover any missing
criteria

Android Phone Matrix

March

2012

Device Name

OS

Display

Aspe
ct

SoC

Carrier

Newest

HTC Droid DNA

4.2

Normal
-
xhdpi

9:16

Snapdragon S4

Verizon

Oldest

HTC Tattoo

1.6

Small
-
ldpi

3:4

Snapdragon S1

AT&T

Common

Motorola Droid 3

2.3

Normal
-
hdpi

9:16

TI OMAP 4

Verizon

Popular

Samsung Galaxy
S3

4.1

Normal
-
xhdpi

9:16

Exynos 4

Sprint

Abnormal

LG Optimus VU

4

Large
-
hdpi

3:4

Nvidia Tegra 3

Tmobile

Flagship

LG Nexus 4

4.2

Normal
-
xhdpi

3:5

Snapdragon S4

TMobile

Budget

Dell Venue

2.2

Normal
-
mdpi

3:5

Snapdragon S3

AT&T

Catch
-
All

Sony Xperia P

2.3

Normal
-
hdpi

9:16

Sony NovaThor

AT&T

iOS

Matrix

March
2012

Device
Name

OS

Display

Aspect

SoC

Carrier

Newest

iPhone 5S

7

4” 1136 x 640 326ppi

9㨱:

䅰灬攠e4扩W⁁

T
-
䵯扩le

Oldest

iPhone 3g

6

3.5” 320 x 480 165ppi

2㨳

䅰灬攠䄳

A否T

Common

iPhone 5

6

4” 1136 x 640 326ppi

9㨱:

䅰灬攠䄵

V敲iz潮o

Popular

iPhone 4

6

3.5” 640x960 330ppi

2㨳

䅰灬攠䄴

印物SW

iPad
(Retina)

iPad 3

7

10” 1536x2048 264ppi

3㨴

䅰灬攠䄵X

V敲iz潮

iPod

iPod Touch
(4
th

gen)

5

3.5” 640x960 326ppi

2㨳

䅰灬攠䄴

坩䙩

Mini

iPad Mini

6

7” 1024 x 768 162ppi

3㨴

䅰灬攠䄵

A否T

Conclusion


Understanding how everything works allows
us to
logically select devices.


A large number of permutations can be
covered in few devices


If additional coverage is needed additional
devices can be added


White Paper :


http://www.prototest.com :


Building the Ultimate Device Matrix