Optimize Oracle on Vmware - Guy Harrison - Squarespace

mangledcobwebΛογισμικό & κατασκευή λογ/κού

14 Δεκ 2013 (πριν από 3 χρόνια και 9 μήνες)

70 εμφανίσεις

Optimize Oracle RDBMS on
VMware






Guy
Harrison

Director, R&D Melbourne

www.guyharrison.net


Guy.harrison@quest.com

@
guyharrison


Introductions

0
10
20
30
40
50
60
70
80
Blue
Yellow
Red
Pct

Star trek shirt fatality analysis

10

©2011 Quest Software, Inc. All rights reserved..

Agenda


Motivations for Virtualization


VMware ESX resource management:


Memory


CPU


IO


Paravirtualization

(OVM)
vs

Hardware Assisted
Virtualization (ESX)


RAC on VMware

11

©2011 Quest Software, Inc. All rights reserved..

Motivations for Virtualization


Better utilization of server resources


Reduced power consumption

Server
Consolidation


Fewer physical machines


Backup, cloning, rapid provisioning

Manageability


Adjust resources on demand


A complement to the physical “grid”
vision

Elastic computing

12

©2011 Quest Software, Inc. All rights reserved..

Resistance to Database virtualization


Virtual CPU & IO


Sharing of virtual resources

Performance


Large databases too big for a
single VM


RAC
-
style clustering problematic

Scale


Oracle’s stance often
misunderstood


See
MyOracleSupport

249212.1

Support

13

©2011 Quest Software, Inc. All rights reserved..

DB virtualization is happening


14

©2011 Quest Software, Inc. All rights reserved..

Oracle virtualization is lagging....

0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
SQL Server
File and Print Servers
IIS
Active Directory
Apache
Exchange
Sharepoint
Java
Oracle
None of the Above

Which of the following do you run in VMs?

Quest survey of
vFoglight

users , 2010

15

©2011 Quest Software, Inc. All rights reserved..

ESX Memory management

16

©2011 Quest Software, Inc. All rights reserved..

Managing ESX memory


ESX can “
overcommit
” memory


Sum of all VM physical memory allocations > actual ESX physical memory


Memory is critical to Oracle server performance


SGA memory to reduce
datafile

IO


PGA memory to reduce sort & hash IO


ESX uses four methods to share memory:


Memory Page Sharing


Memory compression


“Ballooning”


ESX swapping


DBA needs to carefully configure to avoid disaster

Configuring VM memory

Relative Memory
Priority for
this
VM

Minimum Memory for
this VM

VMs Compete
for memory in this
range

Maximum memory for
the VM (dynamic)

18

©2011 Quest Software, Inc. All rights reserved..

Monitoring VM memory


ESX Swap

ESX virtual
memory

ESX physical
memory

VM

VM virtual
memory

ESX swap

Effective VM
physical
memory

ESX and VM memory

ESX Swap

ESX virtual
memory

ESX physical
memory

VM

Apparent VM
physical
memory

ESX swap

Effective VM
physical
memory

Vmmemctl

“balloon”

VM Swap

VM Swap

ESX Ballooning

ESX Ballooning

As memory grows, ESX balloon driver (
vmmemctl
) forces VM to page out
memory to VM
swapfile

ESX Ballooning


Inside the VM, paging to the
swapfile

is observed.


The guest OS will determine
which pages are paged out


If LOCK_SGA=TRUE, then the
SGA should not be paged.

ESX Swap

ESX virtual
memory

ESX physical
memory

ESX swap

ESX Swapping

VM virtual
memory

VM

Effective VM
physical
memory

ESX Swap

ESX virtual
memory

ESX physical
memory

ESX swap

Apparent VM
physical
memory

VM

Effective VM
physical
memory

ESX Swapping

ESX Swapping


ESX swaps out VM memory to
ESX

swapfile

ESX Swapping


Within the VM, swapping cannot
be detected.


ESX will determine which
memory pages go to disk


Usually occurs when VMware
tools are not installed


Even if LOCK_SGA=TRUE,
SGA memory might be on disk

Avoiding Ballooning and swapping

memory reservations help avoid ballooning or ESX
swapping

28

©2011 Quest Software, Inc. All rights reserved..

Ballooning vs. Swapping

Swingbench

workload running on Oracle database


from
VMWare

whitepaper:
http://www.vmware.com/files/pdf/perf
-
vsphere
-
memory_management.pdf

29

©2011 Quest Software, Inc. All rights reserved..

VMware memory recommendations


Paging or swapping of PGA or SGA is almost always a Very Bad
Thing
TM
.


Use memory reservations to avoid swapping or ballooning


Install VMware tools to allow ballooning instead of swapping


Set Memory reservation =
PGA+SGA+process

Overhead


Be realistic about memory requirements:


In physical machines, we are used to using all available memory


In VM, use only the memory you need, freeing up memory for other VMs


Oracle advisories (or Spotlight) can show you how much memory is
needed


Reduce VM reservation and Oracle memory targets
in tandem
to
release memory

31

©2011 Quest Software, Inc. All rights reserved..

ESX CPU management

32

©2011 Quest Software, Inc. All rights reserved..

ESX CPU management


If more virtual CPUs than ESX CPUs, then
vCPUs

will
sometimes wait for physical CPU


Time “stops” inside the VM when this occurs


For multi
-
CPU VMs, it’s (nearly) all or nothing.


A
vCPU

can be in one of three states:


Associated with an ESX CPU but idle


Associated with an ESX CPU and executing instructions


Waiting for ESX CPU to become available


Shares and reservations determine which VM wins access
to the ESX CPUs

Configuring VM CPU

VMs compete for CPU in this
range

Shares determine relative CPU
allocated when competing

CPU utilization VM

“CPU Ready” is the
amount of time VM
spends waiting on
ESX for CPU

Inside the VM, CPU
stats can be
misleading

SMP

for
vCPUs


ESX usually has to
schedule all
vCPUs

for a
VM simultaneously


The more CPUs the
harder this is


Some CPU is also
needed for ESX


More is therefore not
always better

(Thanks to Carl Bradshaw for letting me reprint
this diagram
from

his Oracle on
VMWare

whitepaper
)

36

©2011 Quest Software, Inc. All rights reserved..

ESX CPU performance comparisons


Vs 2 core 1.8 GHz physical machine

VT enabled

37

©2011 Quest Software, Inc. All rights reserved..

Programmatic performance

0
10
20
30
40
50
PLSQL
PLSQL compiled
Java Stored Proc
Simple C program
Standalone Java
Elapsed Time (s)

ESX 2 VCPU 2.26 GHz VT enabled
ESX 2 CPU 3.5 GHz no-VT
Physical 2 CPU 1.8 GHz
NB: Not a benchmark! Just some informal measurements!!

38

©2011 Quest Software, Inc. All rights reserved..

Programmatic performance (2)

333

72

0
50
100
150
200
250
300
350
ESX 2 CPU 3.5 GHz no-VT
ESX 2 VCPU 2.26 GHz VT enabled
Elapsed time Pct relative to Physical CPU adjusted for GHz

39

©2011 Quest Software, Inc. All rights reserved..

ESX CPU recommendations


Use up to date chipsets and ESX software



Allocate as few VCPUs as possible to each VM



Use reservations and shares to prioritise access to ESX CPU



Monitor ESX Ready time to determine the “penalty” of competing
with other virtual machines




40

©2011 Quest Software, Inc. All rights reserved..

ESX IO management

Typical
VMWare

disk configuration

IO Resource Allocation


Disk shares can be used to prioritize IO bandwidth.


This is poorly implemented prior to
vSphere

4.1

43

©2011 Quest Software, Inc. All rights reserved..

Storage IO Control


Prior to
vSphere

4.1:



disk shares could be used only at the VM level, and only within a single
ESX host



vSphere

4.1 Storage IO Control (SIOC):


Manages disk share priorities for all VMs attaching to the same
datastore


Is triggered by high (“congested”) latency


Can be enabled globally at the
datastore

level


Enables equitable distribution even when set to defaults


Storage IO Control

45

©2011 Quest Software, Inc. All rights reserved..

vSphere

4.1 SIOC


46

©2011 Quest Software, Inc. All rights reserved..

SOIC won’t make up for

a poorly configured IO layout

Performant

VMware disk configuration

48

©2011 Quest Software, Inc. All rights reserved..

Optimal configuration


See “
Oracle Database Scalability in VMware® ESX” at
www.vmware.com/oracle


Each virtual disk directly mapped via RDM to dedicated
RAID 0 (+1) group


41 Spindles
!

49

©2011 Quest Software, Inc. All rights reserved..

ESX
IO recommendations


Follow normal best practice for physical disks


Avoid sharing disk workloads


Dedicated
datastores

using VMFS


Align virtual disks to physical disks?


Consider Raw Device Mapping (RDM)


Consider SIOC in
vSphere

4.1


If you can’t optimize IO, avoid IO:


Tune, tune, tune SQL


Prefer indexed paths


Memory configuration


Don’t forget about temp IO (sorts, hash joins)

50

©2011 Quest Software, Inc. All rights reserved..

Shameless plugs

52

©2011 Quest Software, Inc. All rights reserved..


53

©2011 Quest Software, Inc. All rights reserved..


54

©2011 Quest Software, Inc. All rights reserved..

Paravirtualization

vs

Hardware Virtualization

55

©2011 Quest Software, Inc. All rights reserved..

Paravirtualization

vs

“Hardware Virtualization”


Virtualization is not emulation....


Where
-
ever possible, Hypervisor runs native code from
OS against underlying hardware


Because a virtualized operating system is running
outside privileged x86 “ring 0”, direct calls to hardware
need special handling.


The three main approaches are:


Full Virtualization (
VMWare

on older hardware)


ParaVirtualization

(
Xen
, Oracle VM)


Hardware Assisted Virtualization (Intel VT, AMD
-
V)



Full virtualization


Hardware calls from the VM
are handled by the hypervisor
by:


Catching the calls as they occur at
run time


Re
-
writing the VM image at load
time (binary translation)


Requires no special
hardware


Supports any guest OS


Relatively Poor performance


Used by ESX on older chip
-
sets


VM

Hypervisor

Hardware

Ring 0

Hardware Assisted virtualization


Intel VT and AMD
-
V chips
add a non
-
root mode Ring 0.


VM can issue instructions
from non
-
root Ring 0.


CPU can divert these to
hypervisor


No changes to OS required


Good performance


Requires modern chipsets

Hypervisor

Hardware

VM

Ring 0

Root Mode

Non
-
Root
Mode

Paravirtualization


VM operating system is
rewritten to translate
device calls to “
hypercalls



Hypercalls

are handled

by
a special VM (
dom
0 in
Xen
/OVM)


Good

performance but
requires modified VM OS


Xen

can use either
paravirtualization

or
hardware assist

Hypervisor

Hardware

VM

(
domU
)

Ring 0

VM

(dom0)

59

©2011 Quest Software, Inc. All rights reserved..

RAC and ESX

60

©2011 Quest Software, Inc. All rights reserved..

Paravirtualization
, ESX and RAC


Prior to 11.2.0.2, Oracle relied on
paravirtualized

kernels to
maintain time synchronization for RAC clusters.



From 11.2.0.2 Oracle uses Cluster Time Synchronization
Service (CTSS) to maintain clock sync, and this works on
ESX



Therefore, Oracle supports RAC on
Vmware

ESX only
from 11.2.0.2 onwards



See Oracle
MySupport

Note
249212.1

61

©2011 Quest Software, Inc. All rights reserved..

References


Latest version of this presentation:


http://www.slideshare.net/gharriso/optimize
-
oracle
-
on
-
vmware
-
5271530


My blog (
www.guyharrison.net

):


http://guyharrison.squarespace.com/blog/2010/2/22/memory
-
management
-
for
-
oracle
-
databases
-
on
-
vmware
-
esx.html


http://guyharrison.squarespace.com/blog/2010/4/9/esx
-
cpu
-
optimization
-
for
-
oracle
-
databases.html


http://guyharrison.squarespace.com/blog/2010/7/12/stolen
-
cpu
-
on
-
xen
-
based
-
virtual
-
machines.html


http://www.vmware.com/files/pdf/perf
-
vsphere
-
memory_management.pdf


http://www.vmware.com/files/pdf/Oracle_Databases_on_vSphere_Deployment_Tips.pdf


http://www.vmware.com/files/pdf/techpaper/VMW
-
vSphere41
-
SIOC.pdf




COLLABORATE 12

April 22
-
26, 2012

Mandalay Bay Convention Center

Las Vegas, Nevada, USA

www.collaborate12.org

www.collaborate12.ioug.org