Mobile I/O virtualization

fearlessquickMobile - Wireless

Dec 12, 2013 (3 years and 7 months ago)

59 views

© 2010 VMware Inc. All rights reserved

Mobile I/O virtualization

Harvey Tuch, Staff Engineer, Mobile Virtualization Platform

January
25
th

2012

Sponsored by MIT and VMware Academic Programs

VMware: www.vmware.com

VMware Labs: labs.vmware.com


2

Agenda


Smartphone hardware


I/O virtualization techniques


Examples from MVP hypervisor


Storage virtualization


GPS virtualization


Telephony virtualization

3

Smartphone hardware

4

Computer System Organization

NIC

LA
N

CPU

MMU

Memory

Controller

Local Bus

Interface

High
-
Speed

I/O Bus

Controller

Bridge

Frame
Buffer

Low
-
Speed

I/O Bus

USB

CD
-
ROM

5

Evolution of mobile phone

to Mobile Personal Computer

6

Recent smartphone specs sample (source:
pdadb.net
)

6

Copyright ® VMware, Inc. All Rights Reserved.

7

Copyright ® VMware, Inc. All Rights Reserved.

Hardware trends


Example smartphone circa
2012


> 1GHz
dual or quad
core ARM Cortex
-
A9/A15
or Snapdragon


1GB SDRAM


512MB internal NAND Flash


32GB microSD


Integrated GPS, > 10MP camera,
3G/4G (LTE),
802.11, Bluetooth


480x800 OLED


Ice Cream Sandwich Android
OS


Tablets
, smartbooks, etc.

8

Smartphone physical hardware

Sma
rt
p
h
o
n
e

i
n
t
e
rn
a
l
s
Syst
e
m-o
n
-C
h
i
p

(So
C
)
AR
M
co
re
(e
.
g
.

C
o
rt
e
x
A-8
)
MMU
I
n
t
e
rru
p
t

co
n
t
ro
l
l
e
r
T
i
me
rs
C
a
ch
e
G
PU
D
MA
engine
D
SP
LCD
co
n
t
ro
l
l
e
r
U
SB
co
n
t
ro
l
l
e
r
SD

ca
rd

co
n
t
ro
l
l
e
r
U
AR
T
G
PI
O
C
a
me
ra

co
n
t
ro
l
l
e
r
Po
w
e
r
ma
n
a
g
e
me
n
t
Me
mo
ry
co
n
t
ro
l
l
e
r
SD
R
AM
N
AN
D

F
l
a
sh
G
SM
mo
d
e
m
G
PS
An
t
e
n
n
a
Acce
l
e
ro
me
t
e
r/
co
mp
a
ss
WiFi
L
C
D
/
t
o
u
ch
scre
e
n
Bl
u
e
t
o
o
t
h
Ba
t
t
e
ry
C
a
me
ra
Mi
cro
p
h
o
n
e
/
sp
e
a
ke
r

Processor core


ARM


Power efficiency, battery life


E.g. 8 DMIPS/mW (Cortex
-
A9)


ARMv7 ISA


Devices


highly integrated


SoC organization


Plethora of devices (IP blocks)


Limited standardization


Storage


solid state NAND

9

Agenda


Smartphone hardware


I/O virtualization techniques


Examples from MVP hypervisor


Storage virtualization


GPS virtualization


Telephony virtualization

10

I/O virtualization techniques

11

Device virtualization goals



Present to guest a “virtual” platform (set of devices)


Guest applications, middleware, OS expect standard set of mobile devices


E.g. touchscreen, display, storage, GPS, Bluetooth, WiFi, GSM, GPU,
cameras, accelerometers, audio, keyboard, etc.


Multiplex physical hardware between VMs


Securely sharing physical hardware across domains


Efficiency


Fidelity


Portability


Interposition (checkpointing, live migration)


Minimizing engineering effort (cost)


12

Device virtualization techniques



Device emulation


VMM emulates registers and memory map of physical HW devices


Paravirtualization


VMM provides optimized high
-
level API for guest devices


Pass
-
thru


VMM provides guest direct access to physical HW devices

13

Device virtualization techniques


device emulation


VMM emulates registers and memory map of physical HW devices


Leverage existing guest OS driver support


Interposable


Emulation overhead


VMM implementation cost

Hypervisor

Guest Operating System

ACME UART driver

ACME UART device model

Physical FOOBAR UART

Write FIFO ‘a’

Read status register CSR

Write FIFO ‘b’

…..

Write FIFO ‘a’

Write FIFO ‘b’

Read status register FSR

…..

14

Device virtualization techniques
-

paravirtualization



VMM provides optimized high
-
level API for guest devices


Higher performance possible


Simplified VMM


Interposable


Requires custom drivers/libraries per guest

Hypervisor

Guest Operating System

PV UART driver

PV UART device model

Physical FOOBAR UART

Write shared buffer
‘abcdefghijklmno…’

…..

Write FIFO ‘a’

Write FIFO ‘b’

Read status register FSR

…..

15

Device virtualization techniques


pass
-
thru


VMM provides guest direct access to physical HW devices


Highest performance, even simpler VMM


I/O MMU required for safety (DMA danger)


Multiplexing HW devices requires extra HW support


Interposition hard (hybrid models separate control/data paths)

Hypervisor

Guest Operating System

ACME UART driver

Physical ACME UART

Write FIFO ‘a’

Read status register

Write FIFO ‘b’

…..

16

Bare
-
metal (Type
-
1) hypervisor architecture


Execute directly on physical HW


Preferred architecture for high
-
end servers


E.g. VMware ESX, Xen, Microsoft Hyper
-
V

C
o
p
yri
g
h
t


®

VMw
a
re
,

I
n
c.

Al
l

R
i
g
h
t
s
R
e
se
rve
d
.
B
a
r
e
-m
e
ta
l

a
r
c
h
i
te
c
tu
r
e

(ty
p
e
-1
)

Exe
cu
t
e
s
d
i
re
ct
l
y
o
n

p
h
ysi
ca
l

H
W

Pre
f
e
rre
d

a
rch
i
t
e
ct
u
re

f
o
r
h
i
g
h
-e
n
d

se
rve
rs

E.
g
.

VMw
a
re

ESX,

Xe
n
,

Mi
cro
so
f
t

H
yp
e
r-V
8
Har
d
w
ar
e
Vi
rt
ual
Mac
hi
ne Moni
t
or
Li
nux
Li
nux
XP
Vi
s
t
a
Mac
O
S
Wednesday, December 15, 2010
17

Hosted (Type
-
2) hypervisor architecture


Executes alongside existing
host

OS, e.g. Mac OS X, Linux, Windows


Leverage host device drivers and resources management


E.g. VMware Workstation/Fusion, KVM, Parallels Desktop


C
o
p
yri
g
h
t


®

VMw
a
re
,

I
n
c.

Al
l

R
i
g
h
t
s
R
e
se
rve
d
.
H
o
s
te
d

a
r
c
h
i
te
c
tu
r
e

(ty
p
e
-2
)

Exe
cu
t
e
s

a
l
o
n
g
si
d
e

e
xi
st
i
n
g

host

O
S,

e
.
g
.

W
i
n
d
o
w
s,

L
i
n
u
x,

O
S
X

L
e
ve
ra
g
e

h
o
st

d
e
vi
ce

d
ri
ve
rs
a
n
d

re
so
u
rce

ma
n
a
g
e
me
n
t

E.
g
.

VMw
a
re

W
o
rkst
a
t
i
o
n
,

KVM,

Pa
ra
l
l
e
l
s
D
e
skt
o
p
9
Har
d
w
ar
e
Vi
rt
ual
Mac
hi
ne Moni
t
or
Gues
t
O
S

(Li
nux)
Ho
st
OS
(
W
i
n
d
o
w

XP
)
U
s
er A
pp
Ke
r
n
e
l

Modul
e
World
Wednesday, December 15, 2010
18

VMware MVP system architecture


Personal applications run
natively



Corporate applications run in a
virtual machine

1
8

Copyright ® VMware, Inc. All Rights Reserved.

19

MVP as a hosted (Type
-
2)
hypervisor


Hosted hypervisor
benefits:


Tackle SoC diversity


Reduced TTM


Zero virtualization overhead
for personal phone.


Almost all components can
be downloaded
OTA (app
store).

1
9

Copyright ® VMware, Inc. All Rights Reserved.

20

Device virtualization stack


Frontend (RHS)


Guest OS


VMM interface for virtual devices


Backend (LHS)


Hypervisor implementation of virtual device semantics on host

C
o
p
yri
g
h
t


®

VMw
a
re
,

I
n
c.

Al
l

R
i
g
h
t
s
R
e
se
rve
d
.
Pl
a
tfo
r
m

v
i
r
tu
a
l
i
za
ti
o
n

V
i
r
tu
a
l
i
ze
d

s
m
a
r
tp
h
o
n
e

d
e
v
i
c
e
s
/
s
u
b
s
y
s
te
m
s

N
e
t
w
o
rki
n
g
,

st
o
ra
g
e
,

t
e
l
e
p
h
o
n
y
,

G
PS,

f
ra
me

b
u
f
f
e
r
,

t
o
u
ch
scre
e
n
,

b
a
t
t
e
ry
,

a
u
d
i
o

Pa
r
a
v
i
r
tu
a
l
i
za
ti
o
n

u
s
e
d

e
x
te
n
s
i
v
e
l
y

D
i
f
f
e
re
n
t

l
e
ve
l
s
o
f

t
h
e

st
a
ck
(H
W
,

ke
rn
e
l

su
b
syt
e
m,

An
d
ro
i
d

mi
d
d
l
e
w
a
re
)
32
Har
d
w
ar
e
Vi
rt
ual
Mac
hi
ne Moni
t
or
Gues
t
O
S

(Li
nux)
Ho
st
OS
(
W
i
n
d
o
w

XP
)
U
s
er A
pp
Ke
r
n
e
l

Modul
e
De
v
i
c
e

I
/
O
Net
work,

Di
s
k
,
Di
s
p
l
a
y,

Wednesday, December 15, 2010
21

Frontend virtualization


where to slice?

Virtualization Layer

R: 137

G: 203

B: 223

Operating System

Application

Libraries, middleware

R: 192

G: 192

B: 192

R: 217

G: 84

B: 30

R: 248

G: 152

B: 29

Device emulation

Device level PV

Library level PV

Application level PV

22

Backend implementation


Host user space (application level)


Translate guest device I/Os to standard kernel/library calls (e.g. POSIX)


E.g. write ‘a’ to /dev/ttyS0 for UART transmit


Portable, simple, language independent


System call overhead, latency


Host kernel


Translate guest device I/Os to kernel level function calls


Lowest overhead, direct access to device subsystem


Kernel dependent, modularity issues, increased TCB


Hardware (pass
-
thru)

23

Agenda


Smartphone hardware


I/O virtualization techniques


Examples from MVP hypervisor


Storage virtualization


GPS virtualization


Telephony virtualization

24

Storage virtualization

25

Smartphone storage devices


Internal storage


NAND flash devices


Software Flash Translation Layer (FTL)


Limited size (256MB


several GB)


Kernel, application code, libraries, middleware


External storage


microSD cards


Hardware FTL


Up to 32GB today (2TB future)


Economics of semiconductor scaling


Optimized for cost, media workloads


Application data (and some code)

26

Storage virtualization stack

C
o
p
yri
g
h
t


®

VMw
a
re
,

I
n
c.

Al
l

R
i
g
h
t
s
R
e
se
rve
d
.
Sto
r
a
g
e

v
i
r
tu
a
l
i
za
ti
o
n
34
VF
S
libc
G
u
e
st
Ap
p
l
i
ca
t
i
o
n
e
xt
3
Bl
o
ck
L
a
ye
r
N
AN
D
L
a
ye
r
FTL
N
AN
D

C
h
i
p

D
ri
ve
r
N
AN
D
CHIP
PV
Bl
o
ck
D
ri
ve
r
VF
S
H
o
st

l
i
b
c
e
xt
3
VMX
(St
o
ra
g
e

t
h
re
a
d
,

g
u
e
st

i
ma
g
e

f
o
rma
t

p
ro
ce
ssi
n
g
,

e
n
cryp
t
i
o
n
)
Bl
o
ck
L
a
ye
r
N
AN
D

F
l
a
sh
F
i
l
e
syst
e
m
VF
A
T
MMC
/
SD
L
a
ye
r
MMC
/
SD
D
ri
ve
r
mi
cro
SD
C
a
rd
Mksck
+
Sh
a
re
d

Me
m
Ph
y
s
i
c
a
l
Hardware
V
i
r
tu
a
l
Hardware
G
u
e
s
t
L
i
n
u
x
Kernel
H
o
s
t
L
i
n
u
x
Kernel
G
u
e
s
t
"
u
n
i
x
"
user space
H
o
s
t
"
u
n
i
x
"
user space
FTL
N
AN
D
Wednesday, December 15, 2010
27

VM image storage on SD cards


Why?


Storage footprint of guest may be several GB (including checkpoint images)


Challenges


Performance


Reliability


Security

28

VM image storage on SD cards
-

performance


FTL optimized for cost, media workloads


Poor non
-
sequential I/O performance


VM workloads exhibit far more non
-
sequential I/O than MP3 streaming,
photos, videos, etc.

29

VM image storage on SD cards
-

performance

.

30

VM image storage on SD cards
-

performance

.

31

VM image storage on SD cards
-

performance

.

32

VM image storage on SD cards


other challenges


Reliability


Dropped phone, whoops


Battery dies


Host software crashes (in particular at kernel level)


Security


SD cards are FAT formatted for compatibility


No access controls


M
alicious host apps can read/write VM images


33

Log structured filesystems (LSF)


An excursion back in time to… 1991…


Slow seek latency on mechnical disks, CPUs getting faster


Non
-
sequential file writes expensive


Solution: every file update gets appended to a sequential “log”


Maintain meta
-
data to later locate the most recent block for a file in the log


Reads mostly hit in the page cache, don’t worry about non
-
sequentiality


Mendel Rosenblum and John K. Ousterhout. The design and implementation
of a log
-
structured file system. In ACM Symposium on Operating System
Principles, 1991.


Benefits beyond performance…


Meta
-
data only updated after new data is on disk


O
ld data still exists earlier in log


Crash recovery discovers latest valid block on disk

34

LSF example

Log
L
o
g
i
ca
l

b
l
o
ck
3
L
o
g

l
o
ca
t
i
o
n
0
1
2
3
4
...
-
-
-
0
-
...
L
a
t
e
st

co
n
t
e
n
t
s
f
o
r
b
l
o
ck
O
l
d

co
n
t
e
n
t
s
f
o
r
b
l
o
ck
35

LSF example

Log
L
o
g
i
ca
l

b
l
o
ck
3
L
o
g

l
o
ca
t
i
o
n
0
1
2
3
4
...
1
-
-
0
-
...
L
a
t
e
st

co
n
t
e
n
t
s
f
o
r
b
l
o
ck
O
l
d

co
n
t
e
n
t
s
f
o
r
b
l
o
ck
0
36

LSF example

Log
L
o
g
i
ca
l

b
l
o
ck
3
L
o
g

l
o
ca
t
i
o
n
0
1
2
3
4
...
1
2
-
0
-
...
L
a
t
e
st

co
n
t
e
n
t
s
f
o
r
b
l
o
ck
O
l
d

co
n
t
e
n
t
s
f
o
r
b
l
o
ck
0
1
37

LSF example

Log
L
o
g
i
ca
l

b
l
o
ck
3
L
o
g

l
o
ca
t
i
o
n
0
1
2
3
4
...
1
2
-
3
-
...
L
a
t
e
st

co
n
t
e
n
t
s
f
o
r
b
l
o
ck
O
l
d

co
n
t
e
n
t
s
f
o
r
b
l
o
ck
0
1
3
38

Storage backend
-

Logging block store (LBS)


Backend represents VM disk & checkpoint images with files on
host


Similar format to a LFS (performance, reliability)


Split data and meta
-
data between internal & external storage


Encryption & integrity checks on data file


Meta
-
data and encryption keys located on protected internal storage (security)

39

LBS performance

.

40

Agenda


Smartphone hardware


I/O virtualization techniques


Examples from MVP hypervisor


Storage virtualization


GPS virtualization


Telephony virtualization

41

GPS virtualization

42

GPS virtualization

C
o
p
yri
g
h
t


®

VMw
a
re
,

I
n
c.

Al
l

R
i
g
h
t
s
R
e
se
rve
d
.
G
PS
v
i
r
tu
a
l
i
za
ti
o
n
33
MVPL
o
ca
t
i
o
n
Se
rve
r
(Ja
va
)
An
d
ro
i
d

Ja
va

L
o
ca
t
i
o
n

l
i
b
ra
ri
e
s
l
i
b
g
p
s_
p
h
ysh
w
.
so

(JN
I
)
H
o
st

ke
rn
e
l

G
PS
d
e
vi
ce

d
ri
ve
r
G
PS
p
h
ysi
ca
l

h
a
rd
w
a
re
G
u
e
st

a
p
p
l
i
ca
t
i
o
n

(Ja
va
)
An
d
ro
i
d

Ja
va

L
o
ca
t
i
o
n

l
i
b
ra
ri
e
s
l
i
b
g
p
s_
mvp
.
so

(JN
I
)
T
C
P

l
o
o
p
b
a
ck
o
ve
r
vi
rt
u
a
l
i
ze
d

n
e
t
w
o
rk
H
o
st
G
u
e
st
Wednesday, December 15, 2010
43

Agenda


Smartphone hardware


I/O virtualization techniques


Examples from MVP hypervisor


Storage virtualization


GPS virtualization


Telephony virtualization

44

Telephony virtualization

45

Telephony virtualization


GSM


Dual SIM smartphones (geography dependent)


Multiple International Mobile Subscriber Identities (IMSIs) per SIM (geo dep)


GSM Alternate Line Service (ALS) (network dependent)


Network


Call forwarding, ala Google Voice, Line2


Voice
-
over
-
IP (VoIP)


Realistic with 4G/LTE


Integration with enterprise Unified Commmunications (UC)