Picking The Right Set of
Mobile Devices
By Brian Kitchener
Software Quality Architect
bkitchener@prototest.com
Overview
•
About me
•
Some Background
•
The Problem
•
Understanding Android
•
How Apps Work
•
Building a Device Matrix
•
Example Matrices
•
Conclusion
About Me
•
Software Quality Architect at
ProtoTest
•
We're
a mobile test lab
that combines usability testing with quality
assurance
to
cultivate
a great user
experience
•
Project Architect, Technical Lead, Trainer.
•
Started in QA in 2001
•
BA in Applied Computing from University of Denver
•
Testing background : Functional, Performance, UAT,
Security, API, Database.
•
Automation : Selenium,
WebDriver
,
WatiN
,
MonkeyTalk
, SOASTA,
Fitnesse
, QTP,
EggPlant
, Squish
•
Languages : C#, Java, Ruby,
Javascript
BACKGROUND
INFORMATION
Some Stats for 2012
•
Mobile Apps achieved $17 billion in sales
•
5.2 Mobile Subscribers
–
1.2 Billion PC’s
–
4.2 Billion people use a toothbrush
–
1 Billion Smartphones
•
722 Million Smartphones sold
•
1.4 Million
iOS
+ Android Apps
•
25 developers
= half of app revenue
iPhone
-
June 2007
About the iPhone
•
Steve Ballmer :
Microsoft CEO
–
“There’s no chance the iPhone is going to gain significant market
share. No chance
.”
•
Patrick Stewart:
–
“
Last Wednesday, I stupidly dropped my iPhone in the bath, and my
life has sort of spiraled almost out of control
.”
•
Jon Rubinstein
–
Palm CEO
–
Is there a toaster that also knows how to brew coffee? There is no such
combined device, because it would not make anything better than an
individual toaster or coffee machine. It works the same way with the
iPod, the digital camera or mobile phone: it is important to have
specialized devices
.
•
Mike
Lazaridis
–
Blackberry CEO
–
And
so what
[the iPhone] has actually
done is increased our sales
.
ANDROID IS THE
PROBLEM
And Then There Were Two…
•
Android unveiled November 2007
•
First device was sold in October
2008.
•
Over 11,000 models have been
released.
•
48 Billion app installs
•
Over 1 Billion Android devices
activated
•
8 OS Revisions
OS Fragmentation
Device Fragmentation
Let’s do some math!
•
16 device display categories
•
20 different common resolutions
•
8 OS versions
•
6 Hardware Manufacturers
•
4 Major cellular networks
•
16
x 20 x 8 x 6 x 4 =
76,800 permutations
•
Pairwise approach = over 30 permutations
•
Who can afford to increase testing by 30X?
Our Approach
•
Efficiency, not coverage
•
Flexible: support small or large number of devices
•
Understand how apps work to logically select criteria
•
Use Market research to pick most common
configurations
•
Pick minimum and maximum boundary values for each
criteria
•
Choose a value that matches an edge case or abnormal
configuration.
•
Pick values that stress or tax the system
UNDERSTANDING
ANDROID
Android
•
Built and Maintained by Google
•
Open Source
•
Built on Linux kernel
•
ARM
•
X86 Ports
•
Built to support almost any type of device
–
Phones, tablets, phablets, media players,
tv’s
,
watches, etc.
•
Device Manufacturers customize code.
Example: Kindle Fire
•
Forked Android 2.3
–
Not updateable
•
Customized UI
•
Separate App store
•
Not all android apps work
•
Custom web browser
The Operating System
•
Google Releases “stock”
versions
•
10 Major Releases since
2008
–
API Level, not Version
•
Device manufacturers like
to customize the OS
–
Drivers, libraries, UI
•
“Stock” OS available in
Nexus devices or an
Emulator
Simulators / Emulators
•
Simulator imitates the software layer
–
OS and Libraries
–
Apple provides a simulator in
xCode
IDE
•
Emulator duplicates the hardware and software
–
Processor and Memory
–
Cannot mimic GPU, GPS, accelerometer
•
Always run stock OS
•
Can be used to test some functionality
•
Should always test on a physical device too
The Processor
•
ARM RISC
-
based instruction set
•
Specification defined by ARM holdings
•
32 bit
•
Same as
iOS
•
X86 patches and ports
•
It’s only a spec, can be modified.
SoC
–
System On A Chip
•
Main Board
•
Processor
•
RAM Bus
•
GPU
•
May include :
–
Cellular
–
WiFi
–
NFC
–
GPS
–
Bluetooth
2 Samsung Galaxy S4s
Quallcomm
Snapdragon
•
Quallcomm
Krait 300
–
Quad core ARMv7
Cortex A15 Architecture
•
Adreno
320 GPU
•
Dual Channel 533Mhz
Bus
•
Integrated LTE
Samsung
Exynos
5
Octa
•
Samsung
Big.little
processor
–
Quad core Cortex A 15
–
Quad core Cortex A7
•
PowerVR
SGX 544 GPU
•
Dual Channel 800Mhz
Bus
•
No Integrated LTE
Common
SoC
Manufacturer
Device Name
Cores
Processor
GPU
Qualcomm
Snapdragon S4
2 or 4
ARM Cortex
-
A15
Adreno
Nvidia
Tegra 3
4+1
ARM Cortex
-
A9
Geforce
Samsung
Exynos 4
2 or 4
ARM Cortex
-
A9
Mali
Intel
Medfeld
1
Intel x86
PowerVR
Texas
Instruments
OMAP 4
2+2
ARM Cortex
A9
PowerVR
ST
-
Ericcson
NovaThor
2
ARM Cortex
-
A9
PowerVR
Resolution is not enough
•
Unlimited number of
screen sizes available
•
Screens range from 3” to 11”
•
Each screen has a resolution, same as a
monitor
–
If you increase the resolution everything shrinks!
•
Pixels per Inch = Density
•
Screen Size + Density = Display Bucket
•
Resolution is not enough!
The Display Buckets
Size :
•
Xlarge
: 8”
-
10” tablet.
•
Large : 5”
-
7” tablet.
•
Normal : 3.5”
-
5” phones.
•
Small : 3”
-
3.5” phones.
Density :
•
ldpi
= Low DPI (~120)
•
mdpi
= Medium DPI (~160)
•
hdpi
= High DPI (~240)
•
xhdpi
= Extra High DPI
(~
320)
Low density
(120),
ldpi
Medium
density
(160),
mdpi
High
density
(240),
hdpi
Extra high
density
(320),
xhdpi
Small
screen
QVGA
(240x320)
480x640
Normal
screen
WQVGA400
(240x400)
WQVGA432
(240x432)
HVGA
(320x480)
WVGA800
(480x800)
WVGA854
(480x854)
600x1024
640x960
Large
screen
WVGA800*
*
(480x800)
WVGA854*
* (480x854)
WVGA800*
(480x800)
WVGA854*
(480x854)
600x1024
Extra
Large
screen
1024x600
WXGA
(1280x800)
†
1024x768
1280x768
1536x1152
1920x1152
1920x1200
2048x1536
2560x1536
2560x1600
Display Buckets
•
Galaxy S3
–
1280 x 720
–
Xhdpi
density (331ppi)
–
Normal screen (4.7”)
•
Galaxy Tab 10.1
–
1280 x 800
–
ldpi
density (149ppi)
–
Xlarge
sceren
(10.1”)
•
Galaxy Note LTE
–
1280 x 800
–
hdpi
density (285ppi)
–
Large Screen (5.5”)
Market Analysis
Aspect Ratio
•
UI is manipulated from code
•
Density Pixels adjust for screen size
–
But can use regular pixels!
•
Need to take both X and Y into account!
–
Easy to overlap or hide things
•
Includes orientation
•
Some devices include an aspect ratio changer!
(LG
Optimus
Vu)
Cellular Carrier
•
Four Major US Networks
–
Verizon, Sprint, AT&T, T
-
Mobile
–
Some phone interoperability
–
2 protocols
•
GSM
–
T
-
Mobile AT&T
•
CDMA
–
Verizon and Sprint
–
Carriers assigned specific frequency bands
–
LTE will be new standard
-
But spectrum issues will
prevent cross
-
network phones
•
So if the phone supports the carrier’s protocol
and band it can theoretically connect.
HOW APPS WORK
How Apps work
•
Apps need to work on all screen sizes
–
May not be functional
–
May be wasted space
–
May not make sense
•
Apps define XML layouts similar to HTML
–
Node structure
–
Static Content
–
Images,
etc
–
Dynamic Content
–
Color, Text, etc.
Layouts and Fragments
•
XML Fragments are
reusable components
•
Layouts stitch together
fragments for a specific
sized device
•
App may need different
flow for tablet
vs
phone
BUILDING THE
DEVICE MATRIX
Our Criteria
•
Operating System
–
OS customizations, missing libraries, driver issues,
•
Screen Size
–
Rendering issues, usability, missing layouts
•
Pixel Density
–
Density Independence, missing layouts.
•
Aspect Ratio
–
X,Y calculations, overlapping panels, display issues
•
SoC
–
Hardware performance, Instruction set, battery, signal
•
Carrier
–
Network protocol, speed, responsiveness, packet loss
The Goal
•
Efficiency, not coverage!
•
Build a set of devices to be used for app and
website testing.
•
Know when to update them
•
Define a list of simple categories of devices
•
Pick devices that offer broad coverage
•
Adjust the number of devices based upon
needed coverage
Categorical Approach
•
Define scope
–
Android,
iOS
, phone, tablet, etc
.
•
Understand Testing requirements
•
Self
-
descriptive Names
•
Help to broaden coverage
•
Will adjust devices chosen to cover our criteria
•
Should be apparent when to update a device
•
Spread coverage :
–
Usage
-
> Edge Cases
-
> Strange
-
> Stress
Example Categories
•
Common
–
Matches most common
display configuration
•
Newest
–
Latest OS version, largest
screen, highest resolution
•
Oldest
–
Oldest, slowest, smallest
device.
•
Abnormal
–
Non
-
standard OS, aspect
ratio, orientation, size
•
Popular
–
Most popular device in
terms of sales
•
Budget
–
Low
-
priced new model.
Tend to have strange specs
•
Flagship
–
Nexus device running stock
Android OS
•
Catch
-
All
–
Cover any missing
criteria
Android Phone Matrix
March
2012
Device Name
OS
Display
Aspe
ct
SoC
Carrier
Newest
HTC Droid DNA
4.2
Normal
-
xhdpi
9:16
Snapdragon S4
Verizon
Oldest
HTC Tattoo
1.6
Small
-
ldpi
3:4
Snapdragon S1
AT&T
Common
Motorola Droid 3
2.3
Normal
-
hdpi
9:16
TI OMAP 4
Verizon
Popular
Samsung Galaxy
S3
4.1
Normal
-
xhdpi
9:16
Exynos 4
Sprint
Abnormal
LG Optimus VU
4
Large
-
hdpi
3:4
Nvidia Tegra 3
Tmobile
Flagship
LG Nexus 4
4.2
Normal
-
xhdpi
3:5
Snapdragon S4
TMobile
Budget
Dell Venue
2.2
Normal
-
mdpi
3:5
Snapdragon S3
AT&T
Catch
-
All
Sony Xperia P
2.3
Normal
-
hdpi
9:16
Sony NovaThor
AT&T
iOS
Matrix
March
2012
Device
Name
OS
Display
Aspect
SoC
Carrier
Newest
iPhone 5S
7
4” 1136 x 640 326ppi
9㨱:
䅰灬攠e4扩W⁁
T
-
䵯扩le
Oldest
iPhone 3g
6
3.5” 320 x 480 165ppi
2㨳
䅰灬攠䄳
A否T
Common
iPhone 5
6
4” 1136 x 640 326ppi
9㨱:
䅰灬攠䄵
V敲iz潮o
Popular
iPhone 4
6
3.5” 640x960 330ppi
2㨳
䅰灬攠䄴
印物SW
iPad
(Retina)
iPad 3
7
10” 1536x2048 264ppi
3㨴
䅰灬攠䄵X
V敲iz潮
iPod
iPod Touch
(4
th
gen)
5
3.5” 640x960 326ppi
2㨳
䅰灬攠䄴
坩䙩
Mini
iPad Mini
6
7” 1024 x 768 162ppi
3㨴
䅰灬攠䄵
A否T
Conclusion
•
Understanding how everything works allows
us to
logically select devices.
•
A large number of permutations can be
covered in few devices
•
If additional coverage is needed additional
devices can be added
•
White Paper :
–
http://www.prototest.com :
–
Building the Ultimate Device Matrix
Enter the password to open this PDF file:
File name:
-
File size:
-
Title:
-
Author:
-
Subject:
-
Keywords:
-
Creation Date:
-
Modification Date:
-
Creator:
-
PDF Producer:
-
PDF Version:
-
Page Count:
-
Preparing document for printing…
0%
Σχόλια 0
Συνδεθείτε για να κοινοποιήσετε σχόλιο