Drupal on Virtualised Infrastructure - AWS - Generic ... - Runegate.org

boundlessbazaarServers

Dec 9, 2013 (3 years and 7 months ago)

152 views

IT STARTED WITH A
KICKSTARTER CAMPAIGN


http
://
www.kickstarter.com/projects/224590870/the
-
guide
-
to
-
glorantha



Wild success, raised much more than expected!



Attracted a lot of traffic to the website



Shared hosting provider told us we were using too much CPU



After warnings, provider cut our service for 24 hours



We needed an alternative. Quickly!

RESCUED BY DRUPAL PLANET

Just before that happened, I flagged a blog posting by Randall Knutson


http://
www.leveltendesign.com/blog/randall
-
knutson/create
-
high
-
performance
-
drupal
-
server
-
just
-
30
-
month



I ended up not following his
recipe exactly


WHAT I WILL COVER

Choosing a provider


Creating a server in AWS


Choosing an operating system


Installing and configuring the software stack on Ubuntu

DEFINITIONS

Virtualisation


Partitioning a ‘large’ physical resource in such a way that a user of a virtual
partition appears to have full access to and control of a smaller physical resource.


Computer


CPU + Memory


Storage


a
nd much more, outside the scope of this discussion


VPS


sir瑵al mriv慴e
perver

Atp


Amazon teb⁓er癩c敳

PROVIDERS


Acquia



https://
www.acquia.com/products
-
services/acquia
-
cloud



Drupal

specialists, value add
layered on AWS


AWS

(Amazon
Web
Services)


http://
aws.amazon.com/



datacenter in Singapore


Softlayer



http://www.softlayer.com/cloudlayer
/



datacenter in
Singapore


Azure

(Microsoft)


http://www.windowsazure.com
/



Windows, datacenter
in
Singapore


Rackspace


http
://www.rackspace.com/cloud/overview
/


many more



Decision

c
riteria include

Pricing

Functionality

Performance:

CPU & Memory

Service

Flexibility & Scaling

Performance: Storage IOPS

Location

Reputation & Reliability

Performance: Network and Internet

WHAT WE DECIDED

We ended up choosing a cheap
provider,
local to my friend’s business and home


A2 Hosting


http://www.a2hosting.com/


AWS


AMAZON WEB SERVICES

Amazon provide a number of different
virtualisation

services via the web

The base virtual computing service is EC2 or Elastic Cloud Compute


Can create configurations from
basic
up to multiple load
-
balanced
servers


A basic EC2 instance can service a moderately loaded Drupal site


costs around US$30 / SG$40 per month


the first month should be free


Getting started documentation

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EC2_GetStarted.html

CREATING A VPS WITH AWS


Sign up to AWS at
http://aws.amazon.com/



have your credit card ready!


Go to the EC2 console at
https://console.aws.amazon.com/ec2/v2/home


Choose a region by price (USA regions can be slightly cheaper);

or by location (close to users for performance or legal reasons)


Click on the “
Launch Instance
µ?EXWWRQ


Name the virtual server (important if you will have multiple servers)


Create (or re
-
use) an SSH key pair to allow you to login to the server


Choose the base operating system image (the Launch Configuration)


Click
Continue


Optionally click
Edit Details

for advanced settings


Click
Launch

EC2


CREATE INSTANCE

EC2


CREATE INSTANCE

YOUR NEW VIRTUAL SERVER

A short while later (half a minute), your new server should be up and running.


More to do, more to do…


Set up and assign a fire
-
wall policy, through
Security Groups



If you want a publicly accessible server, create and assign an
Elastic
IPs

address



If you want additional ‘disks’, create and assign volume(s) in
Elastic Block Store



Connect to the new server, using SSH and the Key Pair generated earlier



Install the additional packages and configure services

CHOOSING AN OPERATING SYSTEM

Ubuntu


Desktop and Server variants


LT匠⡬潮o t敲e 獵灰潲琩t

獥浩
-
慮湵nl 異u牡摥r


䉡獥搠潮o
䑥Di慮


F牥r

SUSE Enterprise Linux Server


P慩a 獵灰潲t

Wi湤n睳


P慩a 䱩捥湳L湧

慮搠卵灰潲t


乯琠瑨攠畳畡氠
w敢獴慣s

f潲o䑲異Dl

Red Hat Enterprise Linux

(RHEL)


P慩a 獵灰潲t


R数e瑡瑩潮⁦潲⁨o杨

t散桮楣慬
ex捥汬敮捥

CENTOS


䉡獥搠潮oRH䕌E晲潭⁒敤⁈慴


印S瑴礠牥捯c搠f潲⁴業敬礠牥l敡獥s


F牥r

䄠灯獳pbili瑹tf潲⁁W匠i猠慮 䅍䤠⡁(慺潮o䵡捨M湥⁉浡来⤬)睨w捨⁩猠愠灲e
-
b畩l琠
潰o牡瑩湧r獹獴敭⁡湤n慰ali捡瑩潮猠獴慣sⰠ獵捨 慳a
䉩瑎B浩

䑲異Dl

AMI


PRE
-
CONFIGURED APPLICATION STACKS

SETTING UP AN UBUNTU SERVER


BASICS

Command

Explanation

apt
-
get update

Updates the list of available software packages

apt
-
get upgrade

Upgrades all installed packages to latest versions,
particularly

security releases

The above two commands

should be run in sequence as the first thing when
you first login to the server. Thereafter, run again about once a week to
ensure that the needed security updates are applied in a timely fashion
.


I haven’t yet found a way to automate this.


卯p攠灡捫慧攠異杲a摥猠睩wl r敱畩r攠愠獥sv敲 r敢潯琮

SETTING UP AN UBUNTU SERVER


BASICS 2

Command

Explanation

apt
-
get install
<package
-
name>



Install a

software package.

If a package depends on other
packages, then the system will prompt
to confirm whether to download and
install both the requested package
and also the additional dependencies

You may be asked additional
questions during the installation of
some packages. For example, the new
root password for MySQL

apt
-
get install

<p1> <p2> <p3>

Install a list of packages

UBUNTU 12.04 “LAMP” STACK
PACKAGES

Package

Description

Version

apache2

Apache webserver

2.2.22

mysql
-
client

MySQL database client software

5.5.29

mysql
-
server

MySQL database server software

5.5.29

libapache2
-
mod
-
php5

PHP
in Apache

5.3.10

php5
-
cli

PHP command line

5.3.10

php5
-
mysql

MySQL library for PHP

memcached

Memory cache server


PERFORMANCE

64Mb should be more than enough for a
small site

1.4.13

drush

Drupal

command line utilities

4.5

ADVICE: SETTING UP AN UBUNTU SERVER


Try to locate Ubuntu packages rather than installing
software from other sources


Installing from other sources may
compromise your ability to cleanly upgrade
the operating system in
future, with potential security implications


Exceptions: Drupal


and

other web
-
facing packages written in scripting
languages



Try to avoid directly updating the settings files created by Ubuntu


For major packages, Ubuntu provides simple
ways
to safely override settings


Directly updating settings files may compromise your ability to cleanly upgrade
the operating system in future, possibly leading to your important changed
settings being lost during an upgrade

UBUNTU POSSIBLE PACKAGES
-

1

Package

Description

php5
-
curl

Files
via internet
library

php5
-
gd

G
raphics
library

php5
-
geoip

IP
geography library

php5
-
imagick

G
raphics
library

php5
-
mcrypt

E
ncryption
library

php5
-
memcache

M
emory
cache
library


PERFORMANCE

php5
-
memcached

M
emory
cache
library


PERFORMANCE

php
-
apc

Keeps
'compiled'
php

in
memory


PERFORMANCE

UBUNTU POSSIBLE PACKAGES
-

2

Package

Description

awstats

Webserver
statistics
-

requires manual configuration

curl

http utilities

diff

Comparison utilities

geoip
-
database

Database linking
ip

addresses to countries

git
-
core

Version control
software

less

Tool for
looking through logs

patch

Software management utility, part

of a VCS

make

Software build tool

UBUNTU POSSIBLE PACKAGES
-

3

Package

Description

phpmyadmin

Web
front end to manage MySQL
databases

php
-
pear

T
ool
to install the
PEAR

and PECL

libraries for PHP

unzip

F
ile
compression management

vim

Programmers’ editor

webalizer

Webserver statistics


needs manual setup

wget

Tools
to get files via the internet

NOTES:


周攠l楳i敤e灡捫慧敳a慲攠桩湴猬 獯浥s潦o睨楣栠y潵o睩wl 湯琠湥敤t


Y潵o浡礠湥敤慮礠浯牥 灡捫慧敳⁤数a湤n湧n潮⁹潵o r敱畩r敭敮瑳t

PHP


Create
files with settings overrides
in


/
etc
/php5/
conf.d



Typical PHP
overrides



exact values depend on your requirements


upload_max_filesize

= 4M


memory_limit

= 256M


include_path

= ".:/
usr
/share/
php




APC settings


apc.shm_size

=
64M

; add 48
-

64M per additional Drupal site


apc.apc.stat

=
0



Memcache

settings


memcache.hash_strategy
="consistent"


APACHE


Enable modules
by adding symbolic links
in

/
etc
/apache2/mods
-
enabled


to the required file(s) from



/
etc
/apache2/mods
-
available



There may be 2 settings files for 1 module:
e
g
.
a
lias.load

and
alias.conf




Remove unwanted modules by removing the link(s) in






/
etc
/apache2/mods
-
enabled



Create your site definitions in



/
etc
/apache2/sites
-
available


and enable the site by creating a symbolic link in

/
etc
/apache2/sites
-
enabled



Particularly useful when creating several virtual hosts



Create files with settings overrides in


/
etc
/apache2/
conf.d

NEARLY THERE

Install
uploadprogress
;

type the following commands


pecl

install
uploadprogress


echo
'extension=uploadprogress.so' > /
etc
/php5/
conf.d
/uploadprogress.ini


I haven’t found an Ubuntu package for this


The 2
nd

line is an example of how to
cleanly
set up a PHP override


Reboot the server


Start setting up Drupal in the default location of /
var
/www


After Drupal is set up and confirmed working…

ENABLE
MEMCACHE

Drupal
6: add the following lines
to


sites/default/
settings.php


$
conf
['
cache_inc
']
= 'sites/all/modules/
memcache
/memcache.inc
';


$
conf
['
memcache_key_prefix
'] = '
something_unique
';


Drupal
7: add the following lines to



sites/default/
settings.php


$
conf
['
cache_backends
'][] = 'sites/all/modules/
memcache
/memcache.inc';


$
conf
['
cache_default_class
'] = '
MemCacheDrupal
';


$
conf
['
memcache_key_prefix
'] = '
something_unique
';


Only important
if you are
hosting multiple
sites

WEB ADMINISTRATION

Webmin



http
://webmin.com
/


Not packaged


Moderately complex to install


More focused on server administration



ISPConfig



http://www.ispconfig.org
/


Not packaged


Difficult to work out and install all dependencies


Very complex to install, required a lot of research and hand editing to complete


Easier to use for common tasks, such as to create
virtualised

email and websites
within the virtual server

BACK
-
UPS

I have yet to do this correctly!


I am currently only backing up database and keeping the backups on the virtual
server!


Drupal Planet and Randall Knutson to the rescue again

http://
www.leveltendesign.com/blog/randall
-
knutson/yet
-
another
-
simple
-
amazon
-
s3
-
backup
-
script
-
drupal

QUESTIONS AND DISCUSSION

apt
-
get install apache2
mysql
-
client
mysql
-
server libapache2
-
mod
-
php5 php5
-
cli
php5
-
mysql
memcached

drush

php5
-
curl php5
-
gd php5
-
geoip php5
-
imagick
php5
-
mcrypt php5
-
memcache php5
-
memcached
php
-
apc

awstats

curl diff
geoip
-
database
git
-
core
less patch make
phpmyadmin

php
-
pear unzip vim
webalizer

wget