Drupal Continuous Integration

architectgroundhogInternet και Εφαρμογές Web

4 Δεκ 2013 (πριν από 3 χρόνια και 10 μήνες)

106 εμφανίσεις

Drupal Continuous Integration


With
Jenkins


Deploy


Classic Graphics


Charlotte, NC


By: John W Smith

License
: Creative Commons CC0


1.
In

the

following

slides,

we

will

be

describing

simple

Jenkins

jobs

that

can

be

used

to

automate

the

deployment

your

git

repository

based

code

(not

specific

to

Drupal,

although

that

is

our

purpose)

to

your

Demo,

Live,

Staging

and/or

Testing

environments
.


First,

we

will

configure

a

Jenkins

job

that

has

the

simple

task

of

deploying

code

to

a

single

server

/

environment
.

Although

this

is

not

optimal,

as

it

requires

a

separate

Jenkins

job

for

each

site

/

server

/

environment,

it

is

simple

to

setup

and

requires

only

basic

scripting
.


Second,

we

will

introduce

a

couple

of

scripts

that

will

allow

deploying

multiple

repositories

to

their

corresponding

servers

based

on

parameters

passed

to

the

Jenkins

job
.

This

goes

back

to

our

original

recommendation

to

name

your

repositories,

servers,

Jenkins

jobs,

etc

using

a

common

simple

format
.

2.
We

assume

you

have

already

followed

the

previously

released

document

“Drupal

Continuous

Integration

with

Jenkins”

describing

the

Basic

install,

setup

and

configuration

of

Jenkins

for

Drupal

Continuous

Integration
.


Description / Purpose


This

document

does

NOT

go

into

the

complexities

of

securing

your

system

from

internal

or

external

attackers,

please

ensure

your

system

is

secure

whether

or

not

it

is

accessible

from

the

internet
.


Also,

please

consult

with

your

organizations

security

and

system

administrators

before

making

any

changes

to

existing

systems

configuration

or

authentication

models,

and

before

introducing

new

systems

into

the

development

and

/

or

production

server

environments
.

Disclaimer


2GHz
+ Multi
-
Core
CPU


4GB
+
RAM


250
GB

or

more

depending

on

the

number

of

concurrent

builds,

and

build

history,

artifacts,

etc

that

you

will

keeping

around

for

review
.


Software needed:


Jenkins
1.xxx (I used
1.433.x)


Java
JDK/JRE
1.x


MySQL 5.x


PHP 5.x


Drush
(latest
version)


Drupal
6.x Core SimpleTest patch
file (included with
SimpleTest Drupal Module)


System Requirements

1.
Points
of
Contact


The
Points of Contact (POC) for this documentation are:


Technical
Lead:
David Norman

(
davidn@knowclassic.com
)


Project
Lead:
John Smith

(
johns@knowclassic.com
)

or

(JSmith@i1Technologies.com)


References

Jenkins

is

here

to

help
.

Specifically,

Jenkins

is

an

"extensible

continuous

integration

(CI)

server"
.


From

a

10
,
000
ft

view,

Jenkins

is

a

tool

for

offloading

the

deployment

and

automated

testing

of

a

software

application
.


You

write

your

code,

push

it

into

version

control,

and

Jenkins

will

take

over

the

task

of

grabbing

that

code,

running

an

installation

/

deployment

process,

testing

the

application

(if

you

provide

it

with

a

test

environment),

and

reporting

back

to

you

those

test

results
.



~
(
http://thinkshout.com/blog/2010/09/sean/beginners
-
guide
-
using
-
hudson
-
continuous
-
integration
-
drupal
)


There

are

a

number

of

different

CI

tools

out

there,

but

we

(Classic

Graphics)

have

chosen

Jenkins,

as

it

seems

to

be

the

leader

in

the

Drupal

world
.


Jenkins
,

for

us,

provides

a

web
-
based

user

interface,

easy

integration

via

LDAP

to

our

Windows

AD

Domain,

a

friendly

way

to

run

scripts,

integration

with

CVS

/

Subversion

/

Git

repositories,

and

of

course

it

works

seamlessly

with

multiple

projects

/

builds

simultaneously
.

Also,

as

an

added

bonus,

Jenkins

has

a

Debian

repository

that

we

can

use

to

install

it

on

our

Ubuntu

servers

without

worrying

about

clobbering

other

libraries

and

avoid

any

dependency

nightmares
.



Welcome!

Here,

we

will

be

creating

a

simple

server

specific

deployment

job

for

our

Drupal

environment,

this

job

could

then

be

used

as

the

base

for

other

deployment

sites

/

servers
.

Since

every

shops

environment,

testing

and

verification

requirements

will

differ,

we

leave

the

advanced

deployment

scripting

to

the

user

of

this

document
.


The Jenkins job setup in this tutorial makes a few assumptions:

1.
You

have

setup

pub
-
key

based

auth

over

ssh

to

your

git

repository

and

possibly

the

destination

web

servers,

depending

on

your

configuration

this

type

of

setup

may

or

may

not

work

for

you
.

2.
You

have

Drupal

core,

all

required

modules,

and

all

custom

modules

located

within

your

git

repository
.

However,

the

script

could

be

easily

modified

to

retrieve

the

latest

released

version

of

Drupal

and

any

other

required

modules

using

Drush

if

this

is

not

the

case

in

your

organization
.

3.
Access

to

deployment

account

private

and

public

keys

(if

required)

and

/

or

the

accounts

password
.

Deployment

account

MUST

have

SSH

access

to

the

destination

web

server(s)

AND

write

permission

to

each

sites


DocumentRoot


directory
.

Obviously

this

type

of

setup

has

security

implications,

please

consult

with

your

systems

security

officer

/

administrator
.


Create
a Server Specific
Deployment
Job

1.
First,

open

up

your

favorite

browser

and

point

it

to

the

Jenkins

server

(
http
:
//jenkins
.
ourdomain
.
com
:
8080
)
.

Click

the


log

in


link

in

the

upper

right
-
hand

corner

of

the

browser

window
.

2.
Enter

your

login

credentials

and

click


log

in

.


3.
For this job, we will need an
additional plugin installed and
enabled. Click the
“Manage Jenkins”

link on the left.

4.
Click the “Manage Plugins” link.

5.
Scroll down to find the
“SSH Plugin”
,
click the check box to enable it.
Scroll to the bottom of the page and
click
“Install without restart”.


6.
Click the
“Return to Dashboard”

link
in the left menu to return to the
Jenkins Dashboard once the plugin
has been installed.

7.
Click

the


Manage

Jenkins


link

in

the

menu
.

8.
Click

the

“Configure

System”

link
.

9.
Scroll

down

to

the

SSH

section,

and

add

the

appropriate

information

for

your

destination

server
.

We

have

the

SSH

Server

advanced

view

shown

in

this

screen

shot

which

allows

entry

of

server

login

/

password

information

for

the

SSH

account

to

be

used
.

10.
Add

as

many

entries

as

needed
.

You

will

need

one

for

each

server

you

plan

to

use

as

a

destination
.

11.
Click

“Save”

when

done
.

12.
Click

the


New

Job


link

in

the

menu
.

Enter


Deployment

Test


and

select


Build

a

free
-
style

software

project

.

Click


Ok


when

done
.


NOTE
:
When creating these jobs, in order to
simplify the creation of the “git” post
-
receive bash
script to launch a build remotely, you should
consider naming your jobs the same as the name of
your repository/repo
-
directory name. So, if you
have your repos on your git server in the
/
var
/git/
reponame.git

directory,then

this job would
be called
reponame
.
As we progress through this
tutorial,
it will become apparent why this makes
things easier
.

13.
Now

let’s

configure

this

simple

job
.

14.
Scroll

down

to

the

Build

Environment

and

we’ll

enable

the


Send

files

or

execute

commands

over

SSH

after

the

build

runs


option
.

15.
Select

the

server

you

will

be

deploying

to

from

the

Drop

down

list
.

16.
Enter

the

following

text

in

the

“Exec

Command”

text

area

without

the

quotes
.

“git
-
update
.
sh

user
.
account

branch”

where
:


git
-
update
.
sh



is

a

script

defined

later

in

this

document



user
.
account



is

the

user

the

script

will

use

to

retrieve

the

git

repository

code

(should

be

the

same

account

as

the

SSH

account

in

a

perfect

world)


branch



the

repository

branch

you

want

to

deploy

with

this

script

(most

likely

master

in

all

instances

except

the

test

server

environment)

17.
Enable

the

“Exec

in

pty


option
.

18.
Click

“Save”
.

Here,

we

will

be

creating

a

generic

deployment

job

for

our

d
eployment

environment
.

Since

every

shops

environment,

testing

and

verification

requirements

will

differ,

we

leave

the

advanced

deployment

scripting

to

the

user

of

this

document
.


The Jenkins job setup in this tutorial makes a few assumptions:

1.
You

have

setup

pub
-
key

based

auth

over

ssh

to

your

git

repository

and

the

destination

web

servers,

depending

on

your

configuration

this

type

of

setup

may

or

may

not

work

for

you
.

Alternately,

you

could

use

an

expect

script

to

enter

the

login

credentials

for

you,

however

this

is

somewhat

insecure

as

the

password

would

be

available

in

the

expect

script
.

2.
You

have

code

(Drupal

core,

all

required

modules,

and

all

custom

modules)

located

within

your

git

repository
.

However,

the

script

could

be

easily

modified

to

retrieve

the

latest

released

version

of

Drupal

and

any

other

required

modules

using

Drush

if

this

is

not

the

case

in

your

organization
.

3.
Access

to

the

deployment

account

private

and

public

keys

(if

required)

and

/

or

the

accounts

password
.

Deployment

account

MUST

have

SSH

access

to

the

destination

web

server(s)

AND

write

permission

to

each

sites


DocumentRoot


directory
.

Obviously

this

type

of

setup

has

security

implications,

please

consult

with

your

systems

security

officer

/

administrator
.


Create a Generic Deployment Job

1.
First,

open

up

your

favorite

browser

and

point

it

to

the

Jenkins

server

(
http
:
//jenkins
.
ourdomain
.
com
:
8080
)
.

Click

the


log

in


link

in

the

upper

right
-
hand

corner

of

the

browser

window
.

2.
Enter

your

login

credentials

and

click


log

in

.

3.
Click

the


New

Job


link

in

the

menu
.

Enter


Deployment

Test


and

select


Build

a

free
-
style

software

project

.

Click


Ok


when

done
.


NOTE
:
When creating these jobs, in order to
simplify the creation of the “git” post
-
receive bash
script to launch a build remotely, you should
consider naming your jobs the same as the name of
your repository/repo
-
directory name. So, if you
have your repos on your git server in the
/
var
/git/
reponame.git

directory,then

this job would
be called
reponame
.
As we progress through this
tutorial,
it will become apparent why this makes
things easier
.

4.
Now

we

need

to

fill

in

some

basic

configuration

for

our

newly

created

Deploy
-
Test

job

as

shown

in

the

following

screenshots
.

a.
Enter a description for our job (not
required)

b.
If

you

would

like

to

setup

automatic

deletion

of

old

builds,

check

the


Discard

Old

Builds


checkbox

(not

needed

for

this

build,

as

nothing

will

be

downloaded

or

created

in

this

deployment

scenario)
.

c.
You

will

probably

want

to

leave

project

based

security

enabled,

add

any

additional

users

and

set

permissions

as

required
.


d.
We

will

configure

3

build

parameters

for

this

build
.

e.
The

first

parameter

is


REPO


and

will

be

used

to

select

the

git

repository

to

deploy
.

f.
The

second

parameter

is


BRANCH


and

will

determine

the

branch

of

the

given

repository

to

deploy
.

The


BRANCH


parameter

should

default

to

master

in

most

cases
.

g.
The

third

parameter

is


SRVENV


and

will

contain

the

server

enviroment

we

want

to

deploy

the

git

repository

to

(in

our

shop

we

use

[test,

demo,

staging,

and

live

environments]

)
.

h.
Check

the

“Execute

concurrent

builds

if

necessary”

option
.

i.
No

Source

Code

Management

needed

for

deployment

jobs
.

j.
Check

“Trigger

Builds

remotely”

option,

and

enter

a

token

that

can

be

used

to

deploy

the

sites
.

For

security

reasons,

only

persons

designated

to

deploy

code

to

the

production

environment

should

be

given

this

token
.

k.
Check

the

“Color

ANSI

Console

Output”

option
.

(not

required)

l.
Now

we’ll

add

a

BASH

script

to

our

job,

shown

to

the

right

here
.

This

will

get

you

started,

and

can

be

expanded

upon

depending

on

your

requirements
.


m.
That’s

it

for

Jenkins

using

this

deployment

method
.

Scroll

down

to

the

bottom

of

the

job

creation

screen

and

click


Save

.

n.
We’ll

now

move

on

to

the

server

side

script

that

will

be

used

with

both

of

the

above

J
enkins

deployment

jobs
.


#!/bin/bash
-
x

######

## This script is an example ONLY and should be
customized for
the site/modules you are working on...

######


######

# This is pretty straight forward actually, and can
be accomplished
a couple of different ways.
This

#

script
is
designed to be used inside the Jenkins environment,
with
a job that can deploy a
repository

#

to a matching
server
environment.

#

# Requirements:

# SSH Keys setup for the Jenkins user account

# SSH
authorized_keys

on destination servers must
contain
the
jenkins

account pub key in
the

#
destination
servers deployment account

# The generic version of git
-
update.sh script
included with
this job script.

# The deployment account on each destination server
needs to
have SSH keys setup to allow
read
-

# only
access to
the
git repository server. This will enable the
deployment account
to retrieve
the

#
appropriate repository
and branch
to deploy without the need for human interaction.

#

# Jenkins job parameters:

# REPO = GIT Repository to deploy

# BRANCH = GIT
Repostiry

branch to deploy

# SRVENV = Should be one of the following for
CG's environment
/ configuration
. Your
setup
may

#
differ.

# test
-

test server environment

# staging
-

staging environment for final QA

# demo
-

demonstration server environment

# live
-

production server environment

######


###
Configuration Variables ###

# Your deployment user account

DEPLOY_ACCOUNT="
deploy_user
"


# The base domain for all your servers in your test/production
environment
this could also be
moved

#

up to the Jenkins job parameters section if
you
need to deploy to several different domains

DOMAIN="exampledomain.com"


# set the deployment server based on supplied information

if [ "x"${
SRVENV%live
} = "x" ]; then


# deploying to the live / production environment


DEPLOY_SERVER="${REPO,,*}.${DOMAIN,,*}"

else


# deploying to an environment other than live / production


DEPLOY_SERVER="${REPO,,*}
-
${SRVENV,,*}.${DOMAIN,,*}"

fi


# now we'll simply execute the git
-
update.sh script (installed on the deployment

# destination server) with the appropriate parameters

ssh ${DEPLOY_ACCOUNT}@${DEPLOY_SERVER} git
-
update.sh ${DEPLOY_ACCOUNT} ${BRANCH
}

1.
The

following

script

will

need

to

be

placed

on

each

server

that

will

be

used

as

a

destination

for

one

of

our

Jenkins

deployment

jobs,

and

should

be

placed

within

the

path

and

accessible

/

executable

by

the

deployment

account
.

Generic
g
it Deployment Script

#!/
bin/bash


script
=`
basename

$0
`;

usage
=$(cat <<
EOF_USAGE

USAGE
: $script <
cgnum
> [branch
]

EOF_USAGE);


if
[[ $1 == '' ]];
then



echo
$usage >&
2;



exit
1
;

fi


gitconfig
=$(cat <<
EOF_USAGE

You
need to configure your git identity and try again.
\
n

This
information is required once for identifying who
is
\
n

responsible
for each tag this script creates during a deploy.
\
n

\
n

Example
:
\
n

$
git
config

--
global user.name
“Git User"
\
n

$
git
config

--
global
user.email

“guser@ourdomain.com
"
\
n

\
n

Templates
to copy/paste:
\
n

\
n

git
config

--
global
user.name
\
n

git
config

--
global
user.email
\
n

EOF_USAGE);


if
[ !
-
e "$HOME/.git/
config
" ];
then



echo
-
e $
gitconfig

>&2
;



exit
1
;

Fi


HOSTNAME
=`hostname
`


if
[ ${HOSTNAME:(
-
8
)}
== "
-
staging" ];
then



REPO
=${HOSTNAME%
-
staging}



ENVIRON
="
staging“

elif

[ ${HOSTNAME:(
-
5
)}
== "
-
test" ];
then



REPO
=${HOSTNAME%
-
test
}



ENVIRON
="test
"

elif

[

$
{HOSTNAME
:
(
-
5
)}

==

"
-
demo"

]
;

then



REPO
=
$
{HOSTNAME
%
-
demo
}



ENVIRON
="
demo“

e
lse



REPO
=
$
{HOSTNAME
}


ENVIRON="
live“

fi


if

[[

$
2

==

''

]]
;

then



BRANCH=“master”

e
lse



BRANCH
=
$
2

fi


if

[

-
d

"/
tmp
/clone/"

]
;

then



echo

"Removing

clone

directory
...




rm

-
rf

"/
tmp
/clone
/“

fi


echo

"Cloning

$
BRANCH
...


mkdir

"/
tmp
/clone
/“


git

clone

-
b

"
$
BRANCH"

ssh
:
//
$
1
@code
.
cgraphics
.
com/var/git/
$
REPO
.
git/

/
tmp
/clone
/


cd

"/
tmp
/clone
/“


TAG
=`date

+
%
Y
-
%
m
-
%
d
-
%
H
-
%
M
`

HASH
=`git

log

"
$
BRANCH"

-
n

1

--
pretty=format
:
%
h`


echo

"Tagging

local

clone
...


git

tag

-
a

-
m

"
$
BRANCH

$
HASH

to

$
ENVIRON

by

$
1
"

"
$
ENVIRON
-
$
TAG“


echo

"Pushing

tag

to

origin
...


git

push

origin

"
$
ENVIRON
-
$
TAG“


echo

"Syncing

the

$
BRANCH

clone

to

public_html
...


rsync

-
rl

--
exclude=private

--
exclude="
.
git"

--
exclude="
.
gitignore
"

/
tmp
/clone/

/
var
/www
/


echo

"Removing

clone

directory
...


rm

-
rf

/
tmp
/clone
/


echo

"Launch

complete
!“

echo

"Template

for

pasting

to

a

related

issue
:


echo

"Deployed

$
HASH

with

tag

$
ENVIRON
-
$
TAG
.
"

2.
That’s

it,

just

place

this

script

in

the

path,

setup

one

of

the

previous

Jenkins

jobs

modified

appropriately

for

your

environment,

and

you

will

be

on

your

way

to

automated

deployment

of

your

code

via

your

Jenkins

CI

Server
.

Drupal Continuous Integration with
Jenkins
-

Deploy

Enjoy, and feel free to contact us with any suggestions,
corrections or mistakes we may have made or that
inadvertently slipped between the cracks.


John W Smith (
johns@knowclassic.com
) or (
Jsmith@i1Technologies.com
)