Memcached - Achieve Internet

yieldingrabbleInternet and Web Development

Dec 7, 2013 (3 years and 8 months ago)

113 views

Revolutionizing

enterprise web development

Memcache Optimization

Agenda


What is memcache and what components do you
need?


Why use it and how do you know it

s working?


Limitations and configuration options


Basic troubleshooting


Common configuration strategies


Tips


What

is memcache?



Memcached is an in
-
memory key
-
value store for
small chunks of arbitrary data



A system to cache data in RAM rather than in a
database.


Replaces Drupal

s default database cache.



Who Uses It?


Facebook


YouTube


Craigslist


Twitter


Flicr


LiveJournal


Wikipedia


Bebo


Wordpress.com


Digg


Typepad


Yellowbot


Many
others…

Sources: http://memcached.org and http://lineofthought.com/tools/memcached

How does memcache.d fit into an overall
performance architecture?


Only 1 component of the
architecture


Compliments other tools such
as Varnish and APC.

Why use
memcache
?


Helps improve the scalability of your site.


Reduces request execution time on the web
server.


Reduces database load.


Allows more distribution of resources and
processing.


In some use cases, can improve page response
time*




* This is very application specific, for most sites the improvement will be so small that most
users will not be able to tell the difference. (e.g. 0.1 second improvement on stock
Drupal

install)

Stats


Performance and Scalability


Cache read performance: 2x faster


Cache write performance: 40x faster


Reduces number of database queries by up to
50%


Actual statistics* (Drupal core vs. Memcache)


Cache Read Operations: 3.5 sec vs. 1.5 sec


Cache Write Operations: 100 sec vs 2.5 sec


DB Queries: 70 vs. 33.

* Cache Read/Write stats are for performing 10,000 operations

** Your results will vary, tests performed on a single dev VM with default D7 install.

Components of a memcache system


Memcache.d Daemon (v 1.4.x)


PHP Extension


PECL Memcache (> v 2.2.1)


Use version 3.0.5 or later


PECL Memcache.d (> 2.x)


TIP: Use the setting

memcache_extension


to
select which library to use if you have both
installed.


Memcache Module


Drupal 7.9+, Pressflow 6, or Drupal 6 with core
patch

PECL Memcache vs. Memcached

PECL

Memcache

PECL
Memcached

Stable, not as frequently
updated

Newer, more features

Easy to install

More

complicated install
with multiple
dependencies

Slightly faster in some
cases

igbinary

support


What is
igbinary
?


Alternate to built
-
in
php

serialize/
unserialize

functions.


Higher performance and lower memory utilization in
many cases.


Binary serialization rather than string.


Requires special php.ini configuration to fully take
advantage of. Requires special calls to use.


Resources:


http://codepoets.co.uk/2011/php
-
serialization
-
igbinary/


https://github.com/igbinary/igbinary/


http://drupal.org/node/839444

Installing Memcache.d


On Debian based systems:


Sudo apt
-
get install memcached


Sudo apt
-
get install php
-
pear


For PECL Memcache


Sudo pecl install memcache
-
beta


For PECL Memcached


Sudo apt
-
get install libmemcached libmemcached
-
dev
pcre3 pcre3
-
dev


(optional) Sudo pecl install igbinary


Be sure to do the config steps!


Sudo pecl install memcached

How do you know it

s working?


Memcache Statistics (Memcache Admin Module)


Enable

Show memcache statistics at bottom of
each page



You should see at least 10


20 cache operations
listed on several bins including:

cache

,

cache_menu

, and

cache_bootstrap

.


Refresh the page a couple times, you should see
a hit number greater than 0 on almost every line
and almost no cache set operations.



How do you know it

s working?


Devel Module


Enable the option

Display query log


sorted by
source.


Should see no queries hitting the main Drupal
cache tables (e.g.

cache

,

cache_bootstrap

,

cache_menu

, etc.)

Diagnostic/Troubleshooting Techniques


Use Telnet to directly query cache instances


Useful commands:


Stats


displays general stats


Stats slabs


displays per slab stats


Stats items


displays per item stats


Flush_all


Deletes everything in the cache


Get / set


Gets or sets a cache item


Use the memcache stats on each page and Memcache
Stats Report to see what

s happening


Most of the same info as via telnet.


TIP: You may get an error from the report if you don

t
have at least one memcache bin defined.


Important Memcache Stats


Cache Hit/Miss Ratio > 99% hits ideal


Connections < 1024 (less than configured limit)


Evictions == 0


Free/Used Memory > 20% free memory

Sizing Memcache.d


Look at the cache_* tables in your database.


Sum the total size of these tables (e.g. in
phpmyadmin)


Multiply by at least 2 and round up to the nearest
multiple of 8MB.


TIP: Try to keep instance size under 512MB.


Example:


Cache table size=30MB, x2 = 60MB.


Round up to 64MB.

Settings.php Options:


cache_backends


cache_default_class


cache_class_form (cache_class_<bin name>)


Drupal 7 only


session_inc (D6 only for now)


memcache_servers


memcache_bins


memcache_key_prefix


memcache_options

Configuration options

Configuration options

Recommended
Memcached

Options (in
settings.php
):

$
conf
['
memcache_options
'] = array(

Memcached
::OPT_COMPRESSION => FALSE,

Memcached
::
OPT_DISTRIBUTION =>
Memcached
::DISTRIBUTION_CONSISTENT,

Memcached
::OPT_BINARY_PROTOCOL => TRUE,

)
;


For
a single
memcache.d

instance, this option is slightly faster:

Memcached
::
OPT_DISTRIBUTION =>
Memcached
::DISTRIBUTION_MODULA
,



NOTE: These settings are only valid if using PECL
Memcached

Limitations


Max key size: 250 bytes


Can use larger keys, but pay a hashing penalty


Max data size: 1 MB


Has no redundancy or locking

* Most of these limitations can be overcome by custom compiling the
Memcache.d

libraries or adding
other extensions to
Memcache.d
.

Common configuration strategies

3 Common Strategies:


Basic configuration


Basic distributed configuration


Multiple bins distributed configuration

Basic Configuration

Good for:


Sites that run on 1
-
2 web servers.


Mostly read
-
only content or
infrequently changing content.


$conf['
cache_backends
'][] = 'sites/all/modules/
memcache
/memcache.inc';

$conf['
cache_default_class
'] = '
MemCacheDrupal
';

$conf['
cache_class_form
'] = '
DrupalDatabaseCache
';

$conf['
memcache_servers
'] = array(

mysql.localdomain:11211' => 'default',);

$conf['
memcache_key_prefix
'] =

mysitename
';

$conf['
memcache_options
'] = array(


Memcached
::OPT_COMPRESSION => FALSE,



Memcached
::OPT_DISTRIBUTION =>
Memcached
::DISTRIBUTION_MODULA,


Memcached
::OPT_BINARY_PROTOCOL => TRUE,

);

Basic Configuration:

settings.php

Simple distributed configuration

Good for:


Sites running on 2 or more web
servers.


Sites that are a mix of read
-
only
and regularly changing content,
but mostly read
-
only.


$conf['
cache_backends
'][] = 'sites/all/modules/
memcache
/memcache.inc';

$conf['
cache_default_class
'] = '
MemCacheDrupal
';

$conf['
cache_class_form
'] = '
DrupalDatabaseCache
';

$conf['
memcache_servers
'] = array(



web1.localdomain:11211' => 'default',



web2.localdomain:11211


=>

default

,

);

$conf['
memcache_key_prefix
'] =

mysitename
';

$conf['
memcache_options
'] = array(


Memcached
::OPT_COMPRESSION => FALSE,


Memcached
::OPT_DISTRIBUTION =>
Memcached
::DISTRIBUTION_CONSISTENT,


Memcached
::OPT_BINARY_PROTOCOL => TRUE,

);


Simple distributed configuration:

settings.php

Dedicated bins distributed configuration

Good for:


Site running on 3 or
more web servers.


Frequently added or
changing content.


Large amounts of
cached content.

// Same settings as previous configs

$conf['memcache_servers'] = array(



memcache1.localdomain:11211' => 'default',



memcache2.localdomain:11211


=>

default

,



memcache3.localdomain:11211' =>

static',

);

$conf[

memcache_bins

] = array(



cache


=>

default

,



cache_bootstrap


=>

static

,



cache_menu


=>

static

,



cache_field


=>

static

,



cache_filter


=>

static

,



cache_path


=>

static

,

);

// Same settings as previous configs

Dedicated bins distributed configuration:

settings.php

Other Tips


Do not set

Minimum cache lifetime

.


Do not set expiration more than 30 days.


You do not need to enable the Memcache module or
enable Drupal caching to use Memcache.


When using multiple bins, make sure you explicitly define
at least one cache table to each bin to avoid errors on the
stats report.


When splitting out multiple servers/bins, move things out in
this order:


Mostly read
-
only content (menus, filters, paths)


Page cache


High traffic cache tables (session, cache)


References


Memcache.d: http://memcached.org/


PECL Memcache:

http://pecl.php.net/package/memcache


PECL Memcached:

http://pecl.php.net/package/memcached


Drupal Module:
http://drupal.org/project/memcache


Memcached.php:
http://www.phpdeveloper.org/news/10247


References


PECL
Memcache

vs.
Memcached
:


http://code.google.com/p/memcached/wiki/P
HPClientComparison


http://brian.moonspot.net/php
-
memcached
-
issues


Alternate
Memcache.d

Init script:
http://www.bxtra.net/articles/2011
-
07
-
27/memcached
-
multiple
-
instances
-
startup
-
script
-
initd
-
on
-
centos
-
56
-
64
-
bits


Thank You

Shawn Smiley, Lead Engineer

d.o: Shawn_Smiley

e: shawn.smiley@achieveinternet.com