IMPLEMENTATION PLAN FOR NEW POLOPOLY SEARCH

barristerbedroomSoftware and s/w Development

Oct 28, 2013 (3 years and 7 months ago)

67 views

IMPLEMENTATION PLAN FOR NEW POLOPOLY SEARCH



1.

POLOPOLY
REDIRECTS

Edit /usr/local/apache2.2
-
is/conf/uws
-
redirect
-
and
-
proxy.conf (please take a backup copy first)

Replace all the existing SEARCH section by this one:


## search
-

ammended for new Google
Search Engine (October 2012, Joseph Farthing, UWP)

## search
-
test.ed.ac.uk represents whatever server we use for the live search engine

## RedirectPermanent is used to move old search engine locations to the new search
interface


JkUnmount /search/website
-
results polopoly

ProxyPass /search/website
-
results http://corpwebsearch.is.ed.ac.uk


JkUnmount /search/website
-
results polopoly

ProxyPass /search http://corpwebsearch.is.ed.ac.uk


JkUnmount /search.html polopoly

RedirectPermanent /staff
-
results

http://www.ed.ac.uk/search


JkUnmount /search.html polopoly

RedirectPermanent /student
-
results http://www.ed.ac.uk/search



2.

POLOPOLY
CHANGES

2.1



QUICK WAY

In this case we deploy the jsp directly to the directory in the Front servers
under the Tomcat
and also in the backend serve and the new Search Box should appear straight away.


I leave file /u01/app/polopoly/apache
-
tomcat
-
5.5.31/webapps/ROOT/jsp/search.jsp so we
can rollback easier.


Also the code should be copied to Backend server

to have everything synchronized.


1.

Copy file
K:
\
ISAPPS
\
general
\
Organisation
\
DSG
\
Scripts
\
UWS
\
Support_releases
\
release_Searc
h_Nov_2012
\
WWWTopDepartmentView.jsp

to server
: Pine


path:
/u01/app/polopoly/project/edin916/src/java/uk/ac/ed/uws/view/department


a
nd to here

/u01/app/polopoly/apache
-
tomcat
-
5.5.31/webapps/ROOT/WEB
-
INF/views/department


2.

Copy file
K:
\
ISAPPS
\
general
\
Organisation
\
DSG
\
Scripts
\
UWS
\
Support_releases
\
release_Searc
h_Nov_2012
\
WWWTopDepartmentView.jsp

to server:
Olive and Mint

path:

/u01/app/p
olopoly/apache
-
tomcat
-
5.5.31/webapps/ROOT/WEB
-
INF/views/department




2.2

FULL DEPLOYMENT

Please pipe the output from the install and deploy commands to files
where specified
just in case there is a problem we need to diagnose.

For the sake of ease of reading, this document will refer to the
Admin CM server

and
the
Front Tomcat servers
, rather than by their individual names. The Admin CM
Server is PINE; the Front Tomcat servers are OLIVE and MINT.

It is assumed that code syncing between the active and failover Admin CM server
(SPRING)

will be completed also.

Part 1.

Deploy Edinburgh code

The following steps should be followed in order to update the Admin CM Serve
r.

1.

Put the two
Front Servers

(Mint & Olive)

into offline mode (disconnect)

with the following
command:

/u01/app/polopoly/scripts/controlcache.sh disconnect


2.

On the
Live Polopoly Admin CM server

(Pine)
,

Backup

the contents of the following directories in case a rollback of changes is required:


/u0
1/app/polopoly/project/edin916/src


3.

Copy the
file from


K:
\
ISAPPS
\
general
\
Organisation
\
DSG
\
Scripts
\
UWS
\
Support_releases
\
release_Search_Nov_201
2
\
WWWTopDepartmentView.jsp

to server
:
Live Polopoly Admin CM server

(Pine)

path:
/u01/app/polopoly/project/edin916/src/java/uk/ac/ed/uws/view/department

4.

Remove file: search.jsp

from
/u01/app/polopoly/project/edin916/src/java/uk/ac/ed/uws/jsp


5.

On the
Live Polopoly Admin CM server

(Pine)
, run the following commands to shutdown Tomcat:

5.1


cd /u01/app/polopoly/apache
-
tomcat/bin/
;

5.2


./shutdown.sh

6.

On the
Live polopoly Admin CM server

(Pine)
, run the following commands to deploy changes
(note that some of the output is sent to stdout and simultaneously redirected to
log

files for
checking purposes following the deployment):

6.1


cd

/u01/app/polopoly/project/edin916

6.2


ant clean

6.3


ant deploy
-
all
-
clean
-
with
-
checks | tee deploy_all_clean_
291112
.
log

6.4


ant deploy
-
front
-
webapp

| tee deploy_front_webapp_
291112
.
log

7.

Before continuing,
verify:

7.1

that the output contained in
the log files produced by the previous stage

is as expected
and that there are no errors or significant warnings (note that compiler warnings such as
deprecation warnings are expected during deployment and these can be
ignored);


In particular, ensure that neither of the following warnings are present in
deploy_all_clean_
291112
.
log

near the top of the file:

o

WARNING: ROOT directory does not exist in webapps

o

WARNING: polopoly directory does not exist in webapps

If either warning is present, verify that the specified location for webapps is correct.

It is valid for one or both of these directories to be missing (for example, during a re
-
deploy) but these warnings should be checked.

7.2

that the poloply.war
and

ROOT.
war files in the following location have been updated (i.e.
show the date and time consistent with the current deployment):

/u01/app/polopoly/apache
-
tomcat/webapps/

7.3

that the ROOT.war file in the following location has been updated:

/u01/app/polopoly/polopo
ly9.16.1/pear/work/ears/front
.

8.

On the
Live Polopoly Admin CM server
, run the following commands to start Tomcat:

8.1


cd /u01/app/polopoly/apache
-
tomcat/bin/;

8.2


.
/startup.sh

9.

Before continuing with steps to update the Front servers, verify that the Live Admin GU
I
is available and functioning as expected.

https://www.polopoly.mis.ed.ac.uk/polopoly/




Part 2.

Update the Front Servers

10.

On the
Live Polopoly Admin CM server
,

run the following commands to copy the
ROOT.war to the front server in a temporal location

10.1


cd /u01/app/polopoly/polopoly9.1
6
.3/pear/work/ears/front

10.2


scp ROOT.war
olive
:/u01/ap
p/polopoly/

(you will be prompted to enter the front server password)

10.3


scp ROOT.war
mint
:/u01/ap
p/polopoly/

(you will b
e prompted to enter the front server password)

11.

Ensure that both Front servers

(Olive & Mint)


are running in Disconnected mode:

/u01/app/polopoly/scripts/controlcache.sh status

12.

R
un the following command
on
both Front servers (Olive & Mint)
to create a copy

of the
existing ROOT.war file in case a rollback of changes is required:

12.1


cd /u01/app/polopoly/apache
-
tomcat/webapps

12.2


cp ROOT.war /u01/app/polopoly/ROOT.war.
old.291112



13.

To begin with, we update

one

Front server
only by carrying out the following steps on the
Front server named
Olive
:

13.1

run the following commands to move aside the healthcheck.jsp page for the Polopoly web
application on Olive in order to prevent the load balancer from serving content from that
serve
r after tomcat & apache have been started.

13.2


cd /u01/app/polopoly/apache
-
tomcat/webapps/ROOT/jsp

13.3


mv healthcheck.jsp healthcheck
-
disabled.jsp


13.4

run the following commands to shutdown Tomcat

and apache on
:

13.4.1

As apache unix user

sudo
/usr/sbin/svcadm disable
site/webserver/apache2
-
2
-
is

13.4.2


cd /u01/app/polopoly/apache
-
tomcat/bin/;

13.4.3


./shutdown.sh;

Note

that you may need to kill any existing Polopoly process if the following error
appears when trying to shut down Tomcat:

java.net.BindException: Address already in u
se


13.5

run the following command

to move file ROOT.war from the temporary location to
tomcat
webapps, remove folder ROOT and check the new ROOT.war has been copied properly

13.5.1


cd /u01/app/polopoly/apache
-
tomcat/webapps; mv
/u01/app/polopoly/ROOT.war .;rm
-
r /u0
1/app/polopoly/apache
-
tomcat/webapps/ROOT; ls

l

13.6

run the following commands to start Tomcat:

13.6.1


cd /u01/app/polopoly/apache
-
tomcat/bin/;

13.6.2


./startup.sh

13.6.3

tail
-
f /u01/app/polopoly/apache
-
tomcat/logs/catalina.out

Watch out for a message like this one

19
-
Nov
-
200
8 15:55:23 org.apache.catalina.startup.Catali na start

INFO: Server startup in 14523 ms

13.7

run the following commands to
move aside the healthcheck.jsp page for the Polopoly web
application on
Olive

in order to prevent the load balancer from serving content from that
server after
tomcat & apache have been started
.

13.7.1


cd /u01/app/polopoly/apache
-
tomcat/webapps
/ROOT/jsp

13.7.2


mv healthcheck.jsp healthcheck
-
disabled
.jsp

13.8

run the following command

to start
apach
e

13.8.1

sudo
/usr/sbin/svcadm
en
able site/webserver/apache2
-
2
-
is

13.9

On the
Olive
, run the following commands to put the server into online mode (connected)
and check the status:

13.9.1



/u01/app/polopoly/scripts/controlcache.sh connect

13.9.2


/u01/app/polopoly/scripts/controlcache.sh status


13.10

Populate the cache by running the following
wget
command (this spiders the University
website to 4 link levels):


NOTE

the machine on which this command is being run must be pointing directly to Olive
to a
ccess the www.ed.ac.uk site. One possibility is to use the server mylo

to which we
have root access
-
, and modify its /etc/hosts file to make
www.ed.ac.uk

point to

Olive


wget
-
nc
-
r
-
l 4
-
Dwww.ed.ac.uk
http://www.ed.ac.uk/


13.11

Access
Olive
directly through the IP address through a browser to verify that content is
being served correctly (this requires a change to your hosts file, outlined below).


In order to access
Olive
directly,
users must add following line to their local hosts file:


192.168.40.114

www.ed.ac.uk

# olive


Before testing, users should check they are on the correct server by requesting the
following page and checking that the serv
er name returned is

Olive
:

http://www.ed.ac.uk/jsp/healthcheck
-
disabled.jsp

14.

When it has been agreed that
Olive
is operating correctly, the
server

must be
brought into
service alongside
Mint
so that the external website is serve
d from both servers for a pe
riod of
time, introducting site load to
Olive

gradually
.

14.1

On

Olive
,
run the following commands to reinstate
the healthcheck.jsp page for the
Polopoly web application on the Front server in order to
allow

the load balancer
to start
serving content from tha
t server.

14.1.1


cd /u01/app/polopoly/apache
-
tomcat/webapps
/ROOT/jsp

14.1.2


mv healthcheck
-
disabled
.jsp

healthcheck.jsp

15.

Verify that the external website content is being served correctly before continuing.


Ensure
the
machine
used for testing
does not have
an
IP address for
Olive or Mint
in
the
hosts file

A period of around an hour is required at this stage to verify that Olive is functioning
correctly and ensure that around half the University website load is distributed on each
server.

To assist in judging the delay required, the grep command can be run against the
polopoly_access_log on
Olive

and
Mint

to compare the number of objects requested from each
server and check when they are roughly equal.

For example,
to find all requests made

between 13:00:00 and 13:09:59 on 7th December

2011:

grep
'07/Dec/2011:13:0
' /usr/local/apache2.2
-
is/logs/polopoly_access_log | wc
-
l

Replace the
red text

in the above command with an appropriate time from which to search the
logs.

16.

Having confirmed that
all is OK on Olive, run the following on Mint

16.1

run the following commands to
move aside the healthcheck.jsp page for the Polopoly web
application on
Mint

in order to prevent the load balancer from serving content from that
server after
tomcat & apache have
been started
.

16.1.1


cd /u01/app/polopoly/apache
-
tomcat/webapps
/ROOT/jsp

16.1.2


mv healthcheck.jsp healthcheck
-
disabled
.jsp


16.2

run the following commands to shutdown Tomcat and apache on:

16.2.1

As apache unix user

sudo /usr/sbin/svcadm disable site/webserver/apache2
-
2
-
is

16.2.2


cd
/u01/app/polopoly/apache
-
tomcat/bi n/;

16.2.3


./shutdown.sh;

Note

that you may need to kill any existing Polopoly process if the following error appears
w
hen trying to shut down Tomcat:
java.net.BindException: Address already in use


16.3

run the following command to

move file ROOT.war from the temporary location to tomcat
webapps, remove folder ROOT and check the new ROOT.war has been copied properly

16.3.1


cd /u01/app/polopoly/apache
-
tomcat/webapps; mv

/u01/app/polopoly/ROOT.war .;rm
-
r /u01/app/polopoly/apache
-
tomcat/web
apps/ROOT; ls

l

16.4

run the following commands to start Tomcat:

16.4.1


cd /u01/app/polopoly/apache
-
tomcat/bin/;

16.4.2


./startup.sh

16.4.3

tail
-
f /u01/app/polopoly/apache
-
tomcat/logs/catalina.out

Watch out for a message like this one

19
-
Nov
-
2008 15:55:23
org.apache.catalina.startup.Catali na start

INFO: Server startup in 14523 ms

16.5

run the following commands to
move aside the healthcheck.jsp page for the Polopoly web
application on
Mint

in order to prevent the load balancer from serving content from that
serv
er after
tomcat & apache have been started
.

16.5.1


cd /u01/app/polopoly/apache
-
tomcat/webapps
/ROOT/jsp

16.5.2


mv healthcheck.jsp healthcheck
-
disabled
.jsp

16.6

run the following command

to start
apache

16.6.1

sudo
/usr/sbin/svcadm
en
able site/webserver/apache2
-
2
-
is

16.7

run the
following commands to put the server into online mode (connected) and check the
status:

16.7.1


/u01/app/polopoly/scripts/controlcache.sh connect

16.7.2


/u01/app/polopoly/scripts/controlcache.sh status

16.8

Populate the cache by running the following
wget
command (this spid
ers the University
website to 4 link levels):


NOTE

the machine on which this command is being run must be pointing directly to
MINT

to access the www.ed.ac.uk site. One possibility is to use the server mylo

to which we
have root access
-
, and modify its

/etc/hosts file to make
www.ed.ac.uk

point to

Mint


wget
-
nc
-
r
-
l 4
-
Dwww.ed.ac.uk
http://www.ed.ac.uk/


17.

Access
Mint

directly through the IP address through a browser to verify that

content is being
served correctly

(this requires a change to your hosts file, outlined below)
.

In order to access
Mint

directly, users must add following line to their local hosts file:

192.168.40.
113

www.ed.ac.uk

# m
int

Before testing, users should check they are on the correct server by requesting the following
page and checking that the server name returned is

Mint
:

http://www.ed.ac.uk/jsp/healthcheck
-
disabled.jsp


18.

On

Mint
,
run the following commands to reinstate
the healthcheck.jsp page for the Polopoly web
application on the Front server in order to
allow

the load balancer
to start
serving content from that
server.

18.1


cd /u01/app/polopoly/apache
-
tomcat/webapps
/ROOT/jsp

18.2


mv healthcheck
-
disabled
.jsp

healthcheck.jsp

19.

V
erify that the external website content is being served correctly before continuing.


Ensure
the
machine
used for testing
does not have
an
IP address for
Olive or Mint
in
the
hosts file

A period of around an hour is required at this stage to verify that Ol
ive is functioning
correctly and ensure that around half the University website load is distributed on each
server.

To assist in judging the delay required, the grep command can be run against the
polopoly_access_log on
Olive

and
Mint

to compare the number of objects requested from each
server and check when they are roughly equal.

For example,
to find all requests made between 13:00:00 and 13:09:59 on 7th December

2011:

grep
'07/Dec/2011:13:0
' /usr/local/apache2.2
-
is/logs/polopoly_acc
ess_log | wc
-
l

Replace the
red text

in the above command with an appropriate time from which to search the
logs.

Part 3.

Update Cold Backup Application Server

To be done in
the cold backup Application Server
(SPRING).

20.

On the
cold backup Application Server

(Spring), backup the existing code:

/u01/app/polopoly/project/edin916/src


21.

Copy the
file from


K:
\
ISAPPS
\
general
\
Organisation
\
DSG
\
Scripts
\
UWS
\
Support_releases
\
release_Search_Nov_201
2
\
WWWTopDepartmentView.jsp

to server
:
Live Polopoly Admin CM server

(
Spring
)

path:
/u01/app/polopoly/project/edin916/src/java/uk/ac/ed/uws/view/department

22.

Remove file: search.jsp

from
/u01/app/polopoly/project/edin916/src/java/uk/ac/ed


23.

Go to
/u01/app/polopoly/apache
-
tomcat/webapps

and make a copy of the old ROOT.war

23.1



cp ROOT.war
/u01/app/polopoly/ROOT.war.ol d.
291112


24.

Copy ROOT.war from Pine to Spring

From
Live Polopoly Admin CM server

(Pine)

go to

24.1


cd /u01/app/polopoly/apache
-
tomcat/webapps

24.2


scp ROOT.war
spring
:/u01/ap
p/polopoly/

(you will be prompted to enter the fro
nt server password)


25.

run the following command

to move file ROOT.war from the temporary location to
tomcat
webapps,
remove folder ROOT and check the new ROOT.war has been copied properly

25.1


cd /u01/app/polopoly/apache
-
tomcat/webapps; mv
/u01/app/polopoly/ROOT.war .;rm
-
r
/u01/app/polopoly/apache
-
tomcat/webapps/ROOT; ls

l







ROLLBACK PLAN FOR NEW POLOPOLY SEARCH



1.

POLOPOLY
REDIRECTS

Edit /usr/local/apache2.2
-
is/conf/uws
-
redirect
-
and
-
proxy.conf (please take a backup copy
first)

Replace
all the existing SEARCH section by this one:


## search

## once polopoly servers are back up the search proxy info should go here..


JkUnmount /search/website
-
results polopoly

ProxyPass /search/website
-
results
http://webdata.ed.ac.uk/polopacheproxy/search/infoseek.php


JkUnmount /search/staff
-
results polopoly

ProxyPass /search/staff
-
results http://webdata.ed.ac.uk/polopacheproxy/search/staff.html


JkUnmount /search/student
-
results polopoly

ProxyPass /search/stud
ent
-
results
http://webdata.ed.ac.uk/polopacheproxy/search/student.html

2.

POLOPOLY CHANGES

2.1

QUICK WAY


In this case we
copy backup files

directly to the directory in the Front servers under the
Tomcat and also in the backend serve and the
OLD

Search Box shoul
d appear straight away.


I leave file /u01/app/polopoly/apache
-
tomcat
-
5.5.31/webapps/ROOT/jsp/search.jsp so we
can rollback easier.


Also the code should be copied to Backend server to have everything synchronized.


3.

Copy file
K:
\
ISAPPS
\
dsg
\
SupportTeam
\
UWS
\
backup
\
bk_live_131112
\
WWWTopDepartmentV
iew.jsp

to server
: Pine


path:
/u01/app/polopoly/project/edin916/src/java/uk/ac/ed/uws/view/department


and to here

/u01/app/polopoly/apache
-
tomcat
-
5.5.31/webapps/ROOT/WEB
-
INF/views/department


4.

Copy file
K:
\
ISAPPS
\
ds
g
\
SupportTeam
\
UWS
\
backup
\
bk_live_131112
\
WWWTopDepartmentV
iew.jsp

to server:
Olive and Mint

path:

/u01/app/polopoly/apache
-
tomcat
-
5.5.31/webapps/ROOT/WEB
-
INF/views/department


2.3

FULL DEPLOYMENT

Part 1.
Deploy Edinburgh code

The following steps should be
followed in order to update the Admin CM Serve
r.

1.

Put the two
Front Servers

(Mint & Olive)

into offline mode (disconnect)

with the following
command:

/u01/app/polopoly/scripts/controlcache.sh disconnect


2.

On the
Live Polopoly Admin CM server

(Pine)
,

Restore
the
backup copy from step 2

and rename as

/u01/app/polopoly/project/edin916/src



3.

On the
Live Polopoly Admin CM server

(Pine)
, run the following commands to shutdown Tomcat:

3.1


cd /u01/app/polopoly/apache
-
tomcat/bin/
;

3.2


./shutdown.sh

4.

On the
Live polopoly
Admin CM server

(Pine)
, run the following commands to deploy changes
(note that some of the output is sent to stdout and simultaneously redirected to
log

files for
checking purposes following the deployment):

4.1


cd

/u01/app/polopoly/project/edin916

4.2


ant clea
n

4.3


ant deploy
-
all
-
clean
-
with
-
checks | tee deploy_all_clean_
291112_2
.
log

4.4


ant deploy
-
front
-
webapp

| tee deploy_front_webapp_
291112_2
.
log

5.

Before continuing, verify:

5.1

that the output contained in
the log files produced by the previous stage

is as expected
and

that there are no errors or significant warnings (note that compiler warnings such as
deprecation warnings are expected during deployment and these can be ignored);


In particular, ensure that neither of the following warnings are present in
deploy_all_clean_
291112_2
.
log

near the top of the file:

o

WARNING: ROOT directory does not exist in webapps

o

WARNING: polopoly directory does not exist in webapps

If either warning is present, verify that the specified location for webapps is correct.

It is

valid for one or both of these directories to be missing (for example, during a re
-
deploy) but these warnings should be checked.

5.2

that the poloply.war
and

ROOT.war files in the following location have been updated (i.e.
show the date and time consistent wi
th the current deployment):

/u01/app/polopoly/apache
-
tomcat/webapps/

5.3

that the ROOT.war file in the following location has been updated:

/u01/app/polopoly/polopoly9.16.1/pear/work/ears/front
.

6.

On the
Live Polopoly Admin CM server
, run the following commands
to start Tomcat:

6.1


cd /u01/app/polopoly/apache
-
tomcat/bin/;

6.2


.
/startup.sh

7.

Before continuing with steps to update the Front servers, verify that the Live Admin GUI
is available and functioning as expected.

https://www.polopoly.mis.ed.ac.uk/polopoly/


Part 2.
Update the Front Servers

8.

On the
Live Polopoly Admin CM server
,

run the following commands to copy the
ROOT.war to the front server in a temporal location

8.1


cd /u01/app/polopoly/polopoly9.1
6
.3/pear/work/ears/front

8.2


scp ROOT.war
olive
:/u01/ap
p/polopoly/

(you will be prompted to enter the front server password)

8.3


scp ROOT.war
mint
:/u01/ap
p/polopoly/

(you will be prompted to enter the front server password)

9.

Ensure that both Front servers

(Olive &

Mint)


are running in Disconnected mode:

/u01/app/polopoly/scripts/controlcache.sh status

10.

R
un the following command
on
both Front servers (Olive & Mint)
to create a copy of the
existing ROOT.war file in case a rollback of changes is required:

10.1


cd
/u01/app/polopoly/apache
-
tomcat/webapps

10.2


cp ROOT.war /u01/app/polopoly/ROOT.war.
old.291112



11.

To begin with, we update

one

Front server
only by carrying out the following steps on the
Front server named
Olive
:

11.1

run the following commands to move aside the
healthcheck.jsp page for the Polopoly web
application on Olive in order to prevent the load balancer from serving content from that
server after tomcat & apache have been started.

11.2


cd /u01/app/polopoly/apache
-
tomcat/webapps/ROOT/jsp

11.3


mv healthcheck.jsp hea
lthcheck
-
disabled.jsp


11.4

run the following commands to shutdown Tomcat

and apache on
:

11.4.1

As apache unix user

sudo
/usr/sbin/svcadm disable site/webserver/apache2
-
2
-
is

11.4.2


cd /u01/app/polopoly/apache
-
tomcat/bin/;

11.4.3


./shutdown.sh;

Note

that you may need to kill any existing Polopoly process if the following error
appears when trying to shut down Tomcat:

java.net.BindException: Address already in use


11.5

run the following command

to move file ROOT.war from the temporary location to
tomcat
w
ebapps, remove folder ROOT and check the new ROOT.war has been copied properly

11.5.1


cd /u01/app/polopoly/apache
-
tomcat/webapps; mv
/u01/app/polopoly/ROOT.war .;rm
-
r /u01/app/polopoly/apache
-
tomcat/webapps/ROOT; ls

l

11.6

run the following commands to start Tomcat
:

11.6.1


cd /u01/app/polopoly/apache
-
tomcat/bin/;

11.6.2


./startup.sh

11.6.3

tail
-
f /u01/app/polopoly/apache
-
tomcat/logs/catalina.out

Watch out for a message like this one

19
-
Nov
-
2008 15:55:23 org.apache.catalina.startup.Catali na start

INFO: Server startup in 14523 ms

11.7

run
the following commands to
move aside the healthcheck.jsp page for the Polopoly web
application on
Olive

in order to prevent the load balancer from serving content from that
server after
tomcat & apache have been started
.

11.7.1


cd
/u01/app/polopoly/apache
-
tomcat/webapps
/ROOT/jsp

11.7.2


mv healthcheck.jsp healthcheck
-
disabled
.jsp

11.8

run the following command

to start
apache

11.8.1

sudo
/usr/sbin/svcadm
en
able site/webserver/apache2
-
2
-
is

11.9

On the
Olive
, run the following commands to put the server into

online mode (connected)
and check the status:

11.9.1



/u01/app/polopoly/scripts/controlcache.sh connect

11.9.2


/u01/app/polopoly/scripts/controlcache.sh status


11.10

Populate the cache by running the following
wget
command (this spiders the University
website to 4 link
levels):


NOTE

the machine on which this command is being run must be pointing directly to Olive
to access the www.ed.ac.uk site. One possibility is to use the server mylo

to which we
have root access
-
, and modify its /etc/hosts file to make
www.ed.ac.uk

point to

Olive


wget
-
nc
-
r
-
l 4
-
Dwww.ed.ac.uk
http://www.ed.ac.uk/


11.11

Access
Olive
directly through the IP address through a browser to verify that content is
being served correctly (this requires a change to your hosts file, outlined below).


In order to access
Olive
directly, users must add following line to their local hosts file:


192.168.40.114

www.ed.ac.uk

# olive


Before testing, users should check they are on the correct server by requesting the
following page and checking that the server name returned is

Olive
:

http://www.ed.ac.uk/jsp/healthc
heck
-
disabled.jsp

12.

When it has been agreed that
Olive
is operating correctly, the
server

must be
brought into
service alongside
Mint
so that the external website is serve
d from both servers for a period of
time, introducting site load to
Olive

gradually
.

12.1

On

Olive
,
run the following commands to reinstate
the healthcheck.jsp page for the
Polopoly web application on the Front server in order to
allow

the load balancer
to start
serving content from that server.

12.1.1


cd /u01/app/polopoly/apache
-
tomcat/webapps
/ROOT/jsp

12.1.2


mv healthcheck
-
disabled
.jsp

healthcheck.jsp

13.

Verify that the external website content is being served correctly before continuing.


Ensure
the
machine
used for testing
does not have
an
IP address for
Olive or Mint
in
the
hosts file

A period of
around an hour is required at this stage to verify that Olive is functioning
correctly and ensure that around half the University website load is distributed on each
server.

To assist in judging the delay required, the grep command can be run against the
p
olopoly_access_log on
Olive

and
Mint

to compare the number of objects requested from each
server and check when they are roughly equal.

For example,
to find all requests made between 13:00:00 and 13:09:59 on 7th December

2011:

grep
'07/Dec/2011:13:0
'
/usr/local/apache2.2
-
is/logs/polopoly_access_log | wc
-
l

Replace the
red text

in the above command with an appropriate time from which to search the
logs.

14.

Having confirmed that all is OK on Olive, run the following on Mint

14.1

run the following commands to
mov
e aside the healthcheck.jsp page for the Polopoly web
application on
Mint

in order to prevent the load balancer from serving content from that
server after
tomcat & apache have been started
.

14.1.1


cd /u01/app/polopoly/apache
-
tomcat/webapps
/ROOT/jsp

14.1.2


mv
healthcheck.jsp healthcheck
-
disabled
.jsp


14.2

run the following commands to shutdown Tomcat and apache on:

14.2.1

As apache unix user

sudo /usr/sbin/svcadm disable site/webserver/apache2
-
2
-
is

14.2.2


cd /u01/app/polopoly/apache
-
tomcat/bi n/;

14.2.3


./shutdown.sh;

Note

that you may need to kill any existing Polopoly process if the following error appears
w
hen trying to shut down Tomcat:
java.net.BindException: Address already in use


14.3

run the following command to move file ROOT.war from the temporary location to tomcat
w
ebapps, remove folder ROOT and check the new ROOT.war has been copied properly

14.3.1


cd /u01/app/polopoly/apache
-
tomcat/webapps; mv

/u01/app/polopoly/ROOT.war .;rm
-
r /u01/app/polopoly/apache
-
tomcat/webapps/ROOT; ls

l

14.4

run the following commands to start Tomcat
:

14.4.1


cd /u01/app/polopoly/apache
-
tomcat/bin/;

14.4.2


./startup.sh

14.4.3

tail
-
f /u01/app/polopoly/apache
-
tomcat/logs/catalina.out

Watch out for a message like this one

19
-
Nov
-
2008 15:55:23 org.apache.catalina.startup.Catali na start

INFO: Server startup in 14523 ms

14.5

run the following commands to
move aside the healthcheck.jsp page for the Polopoly web
application on
Mint

in order to prevent the load balancer from serving content from that
server after
tomcat & apache have been started
.

14.5.1


cd /u01/app/polopoly/apache
-
tom
cat/webapps
/ROOT/jsp

14.5.2


mv healthcheck.jsp healthcheck
-
disabled
.jsp

14.6

run the following command

to start
apache

14.6.1

sudo
/usr/sbin/svcadm
en
able site/webserver/apache2
-
2
-
is

14.7

run the following commands to put the server into online mode (connected) and check the
status:

14.7.1


/u01/app/polopoly/scripts/controlcache.sh connect

14.7.2


/u01/app/polopoly/scripts/controlcache.sh status

14.8

Populate the cache by running the following
wget
command (this spiders the University
website to 4 link levels):


NOTE

the machine on which this co
mmand is being run must be pointing directly to
MINT

to access the www.ed.ac.uk site. One possibility is to use the server mylo

to which we
have root access
-
, and modify its /etc/hosts file to make
www.ed.ac.uk

point
to

Mint


wget
-
nc
-
r
-
l 4
-
Dwww.ed.ac.uk
http://www.ed.ac.uk/


15.

Access
Mint

directly through the IP address through a browser to verify that content is being
served correctly

(this requires a change to your hosts file, outlined below)
.

In order to access
Mint

directly, users must add following line to their local hosts file:

192.168.40.
113

www.ed.ac.uk

# mint

Before testing, users should c
heck they are on the correct server by requesting the following
page and checking that the server name returned is

Mint
:

http://www.ed.ac.uk/jsp/healthcheck
-
disabled.jsp


16.

On

Mint
,
run the following commands to reinstate
the healthcheck.jsp page for the Polopoly web
application on the Front server in order to
allow

the load balancer
to start
serving content from that
server.

16.1


cd /u01/app/polopoly/apache
-
tomcat/webapps
/ROOT/jsp

16.2


mv healthcheck
-
disabled
.jsp

healthcheck.jsp

17.

V
erify that the external website content is being served correctly before continuing.


Ensure
the
machine
used for testing
does not have
an
IP address for
Olive or Mint
in
the
hosts file

A period of around an hour is required at this stage to verify that Ol
ive is functioning
correctly and ensure that around half the University website load is distributed on each
server.

To assist in judging the delay required, the grep command can be run against the
polopoly_access_log on
Olive

and
Mint

to compare the number of objects requested from each
server and check when they are roughly equal.

For example,
to find all requests made between 13:00:00 and 13:09:59 on 7th December

2011:

grep
'07/Dec/2011:13:0
' /usr/local/apache2.2
-
is/logs/polopoly_acc
ess_log | wc
-
l

Replace the
red text

in the above command with an appropriate time from which to search the
logs.

Part 4.

Update Cold Backup Application Server

To be done in
the cold backup Application Server
(SPRING).

26.

On the
Live Polopoly Admin CM server

(Spring)
,

Restore the
backup copy from step 2
0

and rename as

/u01/app/polopoly/project/edin916/src


27.

Go to
/u01/app/polopoly/apache
-
tomcat/webapps

and make a copy of the old ROOT.war

27.1



cp ROOT.war
/u01/app/polopoly/ROOT.war.ol d.
291112


28.

Copy ROOT.war from Pine to
Spring

From
Live Polopoly Admin CM server

(Pine)

go to

28.1


cd /u01/app/polopoly/apache
-
tomcat/webapps

28.2


scp ROOT.war
spring
:/u01/ap
p/polopoly/

(you will be prompted to enter the front server password)


29.

run the following command

to move file ROOT.war from the temporary location to
tomcat
webapps,
remove folder ROOT and check the new ROOT.war has been copied properly

29.1


cd /u01/app/polopoly/apache
-
tomcat/webapps; mv /u01/app/polopoly/ROOT.war .;rm
-
r
/u01/app/polopoly/apache
-
tomcat/
webapps/ROOT; ls

l