Drupal in the Cloud

twodotcuddlyInternet και Εφαρμογές Web

4 Δεκ 2013 (πριν από 3 χρόνια και 6 μήνες)

71 εμφανίσεις

Drupal in the Cloud
by Azhan
Founder/Director
S & A Solutions

S & A Solutions – who?

doing it with Drupal since 2007

Over 70 projects in 5 years

More than 20 clients

99% Drupal projects

We love Drupal (obviously!)
> Drupal and S & A Solutions
Drupal in the Cloud
>
Case Study 1 – http://theasiamag.com

Content publishing of stories from Asia

Launched in 2010 using Amazon EC2+RDS

Drupal 6, on
shared hosting
during development

Despite performance tuning efforts, still slow pages
"Ok S & A, so what do we do now?"
Drupal in the Cloud
>
Case Study 1 – http://theasiamag.com
S & A commissioned to do a site performance evaluation
We need to

Find out what is causing the slow pages

Find out what server size is suitable

Decide between single tier or multi-tier

Provide a solution for short term/future scalability

Prove with benchmarking test results
Drupal in the Cloud
>
Case Study 1 – http://theasiamag.com
Drupal and other optimizations done:

Turned on Boost module, CSS/Javascript aggregation

Optimized images, custom module SQL queries

Removed references to missing files/images
Results:

Reduced page loading times from 8 secs to 5 secs

average ~ 30-40% improvement

Apparent network latency stays at 3-4 secs
> Hints on bottleneck in the network, web server, database
Drupal in the Cloud
>
Case Study 1 – http://theasiamag.com
Network test:

wget download test >> 132kb/sec (good)

ping latency hovering 200-300ms
(normal for SG to USA)
Network no longer the suspect

Apache ab –k -n100 -c1 >> 2-5 secs per page load (SLOW!)

70% content errors when ramped up to n=100, c=10
Final suspects:
Web server, MySQL needs more CPU and/or memory
Drupal in the Cloud
>
Case Study 1 – http://theasiamag.com
Other opportunities in improving performance

Serve static files using a CDN (> 1GB of static files)

Load balanced web servers

Memcached / Varnish

Reverse proxy eg. Nginx

Tune & recompile Apache/MySQL for performance

Allocate more resources to Apache and MySQL (easy)
Drupal in the Cloud
>
Case Study 1 – http://theasiamag.com
Other opportunities in improving performance

Serve static files using a CDN (> 1GB of static files)

Load balanced web servers

Memcached / Varnish

Reverse proxy eg. Nginx

Tune & recompile Apache/MySQL for performance

Allocate more resources to Apache and MySQL (easy)
Drupal in the Cloud
>
Case Study 1 – http://theasiamag.com
TheAsiaMag
asked for
Amazon web services

US West Virginia availability zone

mostly worldwide audience

4 types of server configurations were tried


involves a Load balancer, EC2 with EBS, RDS
Drupal in the Cloud
>
Case Study 1 – http://theasiamag.com
1
st
run: Load balancer, 2x EC2 micro instances, small RDS
2
nd
run: Load balancer, 1x LARGE EC2 instance, small RDS
3
rd
run: Load balancer, 1x small EC2 instance, small RDS
4
th
run: Load balancer, 1x medium EC2 instance, small RDS
Benchmarking tools used:

Apache ab

Drupal Devel module, MySQL slow logs

Firebug
Drupal in the Cloud
>
Case Study 1 – http://theasiamag.com
RESULTS of 1
st
run: Apache ab -n10000 -c10
Load balancer, 2x EC2 micro instances, small RDS
EC2 CPU 100%
EC2 RAM 100% (no swap)
RDS CPU 100%
Apache server dies at a random time within a day
and could not revive itself
Drupal in the Cloud
>
Case Study 1 – http://theasiamag.com
RESULTS of 2
nd
run: Apache ab -n10000 -c10
Single LARGE EC2 instance, small RDS
10,000 requests completed in under 20mins
EC2 CPU 5-10%
EC2 RAM 30-50%
RDS CPU <5%
Impressive, but overkill. And expensive!
Drupal in the Cloud
>
Case Study 1 – http://theasiamag.com
RESULTS of 3
rd
run: Apache ab -n10000 -c10
Single small EC2 instance, small RDS
10,000 requests completed in 1hr
EC2 CPU 15-20%
EC2 RAM 85%
RDS CPU 50%
Performance is just OK, but cost is within budget!
No need for 4
th
run.
Drupal in the Cloud
>
Case Study 2 – http://holidaycube.com

Property listing portal, launched in 2010

Drupal 5, dedicated server hosting

In 2011 - slow pages, occasional server deaths

S & A, what do we do?”
Drupal in the Cloud
>
Case Study 2 – http://holidaycube.com
The problem

Just 1000+ properties

10k page views per month

Very complex search+filters using native Drupal search

Real time statistics killing the site

Watchdog?!!!
First aid

Identified slow queries, MySQL always using too much RAM

Had to “kill” watchdog

Plenty of opportunity for code optimization, but have to wait

Hardware upgrade to rescue the site first
Drupal in the Cloud
>
Case Study 2 – http://holidaycube.com

Current hosting at a leading US-based reputable company

Amazon web services suggested as an alternative
Drupal in the Cloud
>
Scalability challenges

Code, database, OS optimizations

Single, monster server vs multi-tier architecture

How to handle spikes in traffic/load

Security, reliability

Cost for all the fancy setup

Time to do all that?!
without breaking the bank!
> I know we could do all that with Amazon web services
Drupal in the Cloud
>
Case Study 2 – http://holidaycube.com
Proposed architecture on Amazon web services

Load balancer

2x small EC2 for web server

1x small EC2 for hosting static files

Shared by all web servers via NFS

1x LARGE RDS for MySQL database
Drupal in the Cloud
>
Case Study 2 – http://holidaycube.com
Final architecture on Amazon web services

Load balancer (later, perhaps)

Single LARGE EC2 for web server + MySQL
Keep it simple!
Building the business case for the move to AWS:

Benchmarking the AWS setup vs the current site

AWS setup has identical CPU, double RAM
Drupal in the Cloud
>
Case Study 2 – http://holidaycube.com
Amazon setup vs dedicated server
Upgrading “current” to 8GB, RHEL will cost USD700!
Current
Amazon web services
CPU
Dual core AMD 64bit CPU
4 ECUs (Intel Xeon dual core equivalent)
OS
64bit RHEL 4
64bit Ubuntu 11.10
RAM
4GB
7.5GB
Location
USA
USA (West Virginia)
Average Cost
USD 300 per month
USD 200 per month
Drupal in the Cloud
>
Case Study 2 – http://holidaycube.com
Benchmarking test results comparison
Apache ab requests per sec
0
2
4
6
8
10
12
4
11
Current
AWS
Benchmarking Tests
No of
requests
per
second
(
more
is bet
ter)
Drupal in the Cloud
>
Case Study 2 – http://holidaycube.com
Benchmarking test results comparison
Apache ab timing per request
0
2
4
6
8
10
12
14
12
4.4
Current
AWS
Benchmarking Tests
T
ime i
n s
ecs
Drupal in the Cloud
>
Case Study 2 – http://holidaycube.com
Benchmarking test results comparison
Devel PHP timings on Homepage
0
500
1000
1500
2000
2500
1941
102
Current
AWS
Benchmarking Tests
Ti
me
in
ms
Drupal in the Cloud
>
Case Study 2 – http://holidaycube.com
Benchmarking test results comparison
Devel PHP timings on Query page
0
20
40
60
80
100
120
100
18
Current
AWS
Benchmarking Tests
T
ime i
n s
ecs
Drupal in the Cloud
>
Case Study 2 – http://holidaycube.com
Benchmarking test results comparison
Firebug network latency
0
1
2
3
4
5
6
5.46
1.44
Current
AWS
Benchmarking Tests
T
ime i
n s
ecs
Drupal in the Cloud
>
Case Study 2 – http://holidaycube.com
Benchmarking test results comparison
Firebug full page loading time
0
10
20
30
40
50
60
70
60
20
Current
AWS
Benchmarking Tests
T
ime i
n s
ecs
Drupal in the Cloud
>
Case Study 2 – http://holidaycube.com
Client is convinced!
(Yeay!!)
Drupal in the Cloud
>
Why Amazon web services?
No to AWS

Could easily have been other hosting providers

Other cloud vendors can address scalability issues

Getting over the Amazonian barriers

AWS may be viewed as “difficult” by some
Drupal in the Cloud
>
Why Amazon web services?
Yes, AWS!

A platform for benchmarking, server sizing

Easy and quick to create/kill servers, run tests

Actually a safe, cheap and controlled means of
hosting your site or application

Easily spawn additional servers to cater to “high” periods
eg. A product launch lasting 4 weeks
terminate the additional servers after site traffic has died down
You can get really creative designing your architecture
Drupal in the Cloud
>
Drupal web load balancing
Amazon Load Balancer
Normally a complex setup with traditional hosting,
the AWS Console makes it so easy to set this up in minutes
Drupal in the Cloud
>
Drupal and memcached
http://drupal.org/project/memcache

Amazon ElastiCache
Normally a complex setup with traditional hosting
the AWS Console makes it so easy to set this up in minutes
if paired with the Drupal memcache module
Drupal in the Cloud
>
Drupal and CDN
http://drupal.org/project/cdn
http://drupal.org/project/cloudfront

Amazon Cloudfront or S3
Drupal in the Cloud
>
Drupal and high availability
Amazon multi-Availability Zones for EC2 and RDS
Drupal in the Cloud
>
Drupal and MySQL
Running MySQL can be done using one of
the following options:
MySQL on Amazon EC2
MySQL on Amazon RDS
Multi-AZ RDS
Drupal in the Cloud
>
Drupal & Amazon web services
Amazon CloudFormation

Templates” to quickly deploy a Drupal stack into the cloud
Drupal in the Cloud
>
Drupal & Amazon web services
It works!
Drupal in the Cloud
>
Questions?
Email: azhan@snasolutions.com.sg
Website: http://www.snasolutions.com.sg

Thank you!
Drupal in the Cloud