Tungsten Installation Guide for PostgreSQL - Amazon S3

frightenedfroggeryΔιαχείριση Δεδομένων

16 Δεκ 2012 (πριν από 4 χρόνια και 8 μήνες)

654 εμφανίσεις

Tungsten Installation
Guide for PostgreSQL
Tungsten Installation Guide for PostgreSQL - Document issue 1.3.2
Tungsten version 1.3.2
Tungsten Installation Guide for PostgreSQL
Tungsten version 1.3.2
This document was published on 11 August 2011.
www.continuent.com [http://www.continuent.com]
Copyright © 2008 - 2011 Continuent
The trademarks, logos, and service marks in this Document are the property of Continuent or other third parties. You are not permitted to use these Marks
without the prior written consent of Continuent or such appropriate third party. Continuent, Tungsten, uni/cluster, m/cluster, p/cluster, uc/connector, and the
Continuent logo are trademarks or registered trademarks of Continuent in the United States, France, Finland and other countries.
All Materials on this Document are (and shall continue to be) owned exclusively by Continuent or other respective third party owners and are protected under
applicable copyrights, patents, trademarks, trade dress and/or other proprietary rights. Under no circumstances will you acquire any ownership rights or other
interest in any Materials by or through your access or use of the Materials. All right, title and interest not expressly granted is reserved to Continuent.
All rights reserved.
Tungsten Installation Guide for PostgreSQL
Tungsten Installation Guide for PostgreSQL - Document issue 1.3.2
iii
Tungsten version 1.3.2
Table of Contents
1. Introduction ............................................................................................................................................ 1
2. Product Installation and Configuration ..................................................................................................... 2
2.1. Tungsten Enterprise Software Installation ..................................................................................... 2
2.1.1. Unpack Software .............................................................................................................. 2
2.1.2. Run Tungsten Configuration Procedure .............................................................................. 2
2.1.3. Post-Configuration Replication Setup ................................................................................. 3
2.1.4. Other Configuration Adjustments ....................................................................................... 3
2.2. Checking the Installation .............................................................................................................. 4
2.2.1. Starting to Manage the Cluster .......................................................................................... 4
2.2.2. Running Performance Tests .............................................................................................. 4
3. Host Prerequisites .................................................................................................................................. 5
3.1. Tungsten Login ........................................................................................................................... 5
3.1.1. Configuring Remote Host Access for PostgreSQL Login ..................................................... 5
3.2. Operating System Prerequisites ................................................................................................... 6
3.2.1. Notes on Operating System Prerequisites .......................................................................... 6
3.2.2. Linux Prerequisites ........................................................................................................... 6
3.2.3. Mac OS X Prerequisites ................................................................................................... 6
3.2.4. Solaris Prerequisites ......................................................................................................... 7
3.3. Java Virtual Machine ................................................................................................................... 7
3.4. Sudo Prerequisites ...................................................................................................................... 8
3.4.1. Sudo Configuration Hints .................................................................................................. 8
3.5. Network Prerequisites .................................................................................................................. 9
3.6. PostgreSQL Database Prerequisites ........................................................................................... 10
3.6.1. Configuring PostgreSQL Server Account Access .............................................................. 11
3.6.2. Preparing the Archive Folder ........................................................................................... 12
3.6.3. Configuring Primary and Standby Servers ........................................................................ 13
4. Connecting Applications to Tungsten ..................................................................................................... 14
4.1. Application Configuration ............................................................................................................ 14
4.2. Configuring Applications to Use Connector ................................................................................. 14
4.3. Configuring Java Applications to use Tungsten SQL Router ......................................................... 14
4.3.1. Configuring Java Applications In General ......................................................................... 14
4.3.2. Configuring Apache Tomcat ............................................................................................. 15
5. Detailed Cluster Configuration ............................................................................................................... 17
5.1. Logger Configuration ................................................................................................................. 17
5.1.1. Introducing Log4j ............................................................................................................ 17
5.1.2. Log4j Configuration Settings ............................................................................................ 17
5.2. Java Service Wrapper Configuration ........................................................................................... 18
5.2.1. Introducing Service Wrapper ........................................................................................... 18
5.2.2. Wrapper Configuration Settings ....................................................................................... 18
5.3. Network Configuration ................................................................................................................ 18
5.3.1. Network Port Access ....................................................................................................... 18
5.3.2. Overview of Group Communications Configuration ............................................................ 19
5.3.3. Group Communications with Multicast .............................................................................. 19
5.3.4. Group Communications with Ping Protocol ....................................................................... 20
5.3.5. Using a Gossip Server .................................................................................................... 20
5.4. JMX Configuration ..................................................................................................................... 21
5.4.1. Introducing JMX .............................................................................................................. 21
5.4.2. JMX Configuration Settings ............................................................................................. 21
Tungsten Installation Guide for PostgreSQL
Tungsten Installation Guide for PostgreSQL - Document issue 1.3.2
iv
Tungsten version 1.3.2
5.4.3. JMX on NAT-enabled hosts ............................................................................................. 21
5.5. Monitoring & Alerting ................................................................................................................. 22
5.5.1. Nagios Monitoring ........................................................................................................... 22
5.5.2. Other Monitoring Tools .................................................................................................... 22
Introduction
Tungsten Installation Guide for PostgreSQL - Document issue 1.3.2
1
Tungsten version 1.3.2
Chapter 1. Introduction
This document describes installation and configuration procedures for Tungsten Enterprise releases. Tungsten
Enterprise releases integrate multiple Tungsten components with add-on software and procedures intended to
make it easier for customers to deploy and operate solutions for data protection, database availability, performance
scaling, and data integration.
Note
This document covers installation and configuration from Tungsten Enterprise 1.3, which contains a num-
ber of new features not available in previous releases. Please check with the Continuent website at
www.continuent.com to get documentation for Tungsten 1.2.x and before.
Product Installation and Configuration
Tungsten Installation Guide for PostgreSQL - Document issue 1.3.2
2
Tungsten version 1.3.2
Chapter 2. Product Installation and Configuration
2.1. Tungsten Enterprise Software Installation
This section describes installation and configuration of Tungsten software on a single host. The installation proce-
dure must be repeated on each host participating in the cluster.
2.1.1. Unpack Software
Unpack the release in a convenient directory. We recommend installing builds in /opt/continuent or the home
directory of the tungsten login.
$ cd /opt/continuent
$ tar -xvzf ~/tungsten-enterprise-1.3.2.tar.gz
$ cd tungsten-enterprise-1.3.2

2.1.2. Run Tungsten Configuration Procedure
The configure script queries installation choices, sets up components of Tungsten, and starts services. You must
run this script before using Tungsten.
$ ./configure

Note
If your Ruby installation has an unsupported Ruby version or does not have required system libraries in-
stalled, configure will fail with an error messages explaining what is missing.
The configure script stores configuration choices in file tungsten.cfg. You can rerun configuration at a later time
using configure -b. This repeats the installation in batch mode.
You can store tungsten.cfg in an alternative location using the configure -c command. The -c option takes a
file specification. This allows you to rename the file or store it in a different directory. Here is an example.
$ ./configure -c ~/configs/tungsten.cfg

Tip
For quick configuration changes on the same version of Tungsten, edit tungsten.cfg and run batch
configuration. You can also save time in deployments by running interactively on one host and then editing
tungsten.cfg manually (for example changing host names) so you can run batch configuration on the
remaining hosts in the cluster.
Warning
When upgrading you must rerun configure interactively on the old tungsten.cfg file from the previous
version. See the Release Notes for more information on upgrading Tungsten.
Product Installation and Configuration
Tungsten Installation Guide for PostgreSQL - Document issue 1.3.2
3
Tungsten version 1.3.2
• Most questions default to reasonable values. If in doubt, accept the default and continue. You can rerun the script
at a later time if necessary to change the configuration values.
• If you are enabling the replicator for the first time, do not choose the configuration option to start services imme-
diately unless all databases have already been synchronized by loading dumps. Instead, you'll need to copy
data from the master to all slaves and then start up the cluster.
• Tungsten for PostgreSQL users should use the 'postgres' operating system account to run Tungsten. This is
necessary to avoid problems with PostgreSQL file permissions. For other databases you can use any account
desired.
• For slave replicators you must enter the name of the master host. This may change later due to failover but you
must start with an initial master during configuration.
• The database login used by the replicator must have full administrative privileges as described in the Tungsten
Replicator Guide. The configuration script does not create this account automatically.
• Depending on the backup method you select you may need to perform additional setup after configuration com-
pletes. For example, pg_dump backups are fully self-configuring whereas LVM backups are not. You will need
to edit the Tungsten Replicator replicator.properties file to complete configuration.
• The Connector read/write splitting places special requirements on applications that use it. You should consult
Tungsten documentation before enabling this option.
• The application client login, password, and database are used to configure the Connector and the Bristlecone
test tools. The script currently accepts only a single user name. If there are additional users who will connect
through the Connector, you must add them manually to the Connector user.map after installation completes.
Note
Applications that use SQL Router libraries set their own login and password when connecting. This is
not set as part of configuration. There are no limitations on the number of logins that may be used by
the application.
• The configure script cannot install service scripts when using a non-root account. You should see a notification
in the output if this is the case. In this case you will need to configure service scripts manually by running
cluster-home/bin/deployall using an account with root privileges.
2.1.3. Post-Configuration Replication Setup
After completing configuration, you should start all Tungsten services if you have not already done so during the
automatic configuration procedure. You can start services by running cluster-home/bin/startall.
2.1.4. Other Configuration Adjustments
You may need to make additional configuration changes to tune the cluster. Specialized configuration information
is provided in Chapter 5, Detailed Cluster Configuration.
Product Installation and Configuration
Tungsten Installation Guide for PostgreSQL - Document issue 1.3.2
4
Tungsten version 1.3.2
2.2. Checking the Installation
2.2.1. Starting to Manage the Cluster
Tungsten is immediately ready for use after installation. You can begin to administer the cluster using the cctrl
script. The ls command will output a list of the current cluster members and their status. There is a help command
to get a list of the available commands.
$ tungsten-manager/bin/cctrl
[LOGICAL] /services/default/> ls
...

For more information, refer to the Tungsten Concepts and Management Guide.
2.2.2. Running Performance Tests
You can test the cluster using Bristlecone, a performance testing framework. Bristlecone automatically generates
load on the cluster and demonstrates read/write load balancing using a tool called Evaluator. To run a Bristlecone
Evaluator test, run the following script:
cluster-home/bin/evaluator_readwrite
You can also run a TPC-B test to check master update performance and replication throughput. Follow the steps
shown below to configure and run the test.
1.Cd to the bristlecone directory in the Tungsten release.
2.Edit config/benchmark/TPCB.properties to set correct connection URLs. Note that there are connection URLs
for both master and slave databases.
3.Run the test: bin/benchmark.sh -props TPCB.properties
For more information on Bristlecone, see the Bristlecone documentation at http://www.continuent.com/communi-
ty/lab-projects/bristlecone.
Host Prerequisites
Tungsten Installation Guide for PostgreSQL - Document issue 1.3.2
5
Tungsten version 1.3.2
Chapter 3. Host Prerequisites
This chapter contains a list of prerequisites for hosts that participate in a Tungsten cluster. Review these carefully
to avoid problems during configuration and deployment.
3.1. Tungsten Login
It is recommended to run Tungsten with a separate user such as "postgres".
Table 3.1. Tungsten Checklist
Description
Check
Expected Result
Check account.
Login using the postgres ac-
count.
You should arrive at writable home direc-
tory.
Ensure install locations
are writable.
Ensure the postgres account
can write to the directory where
the installation is unpacked.
The directory should be writable.
Ensure release directo-
ry locations are writable.
Ensure locations like /opt/
tungsten exist and are writable
by the postgres account.
The locations should be writable.
3.1.1. Configuring Remote Host Access for PostgreSQL Login
Remote host access uses certificate-based ssh access. Here are the steps to generate certificates for host access
for the 'postgres' account.
Note
File and directory names may vary according to your operating system and the type of key generated.
1.Use the ssh-keygen program to generate a key. Do not put a passphrase on the key. Here is an example of use.
pasayten:~/.ssh # ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/var/lib/pgsql/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /var/lib/pgsql/.ssh/id_rsa.
Your public key has been saved in /var/lib/pgsql/.ssh/id_rsa.pub.
The key fingerprint is:
4c:cf:40:4e:1e:a3:be:91:11:f1:9a:48:4a:5d:f8:33 postgres@pasayten
2.Copy the private key id_rsa to ~postgres/.ssh for each host that will participate in the cluster.
3.Similarly add the public key to the authorized_hosts file using a command like the following:
cat id_rsa.pub >> ~/.ssh/authorized_keys
Host Prerequisites
Tungsten Installation Guide for PostgreSQL - Document issue 1.3.2
6
Tungsten version 1.3.2
4.Ensure the .ssh directory and files within it are readable only to the 'postgres' account.
chmod 700 ~/.ssh;chmod 600 ~/.ssh/*
5.Test ssh access by logging in between hosts. You may need to do this once for each host-to-host combination,
including the local hostname, to eliminate prompts like the following:
$ ssh pasayten
The authenticity of host 'pasayten (172.16.238.128)' can't be established.
RSA key fingerprint is 16:54:e3:e4:e8:88:45:fc:62:07:5e:69:59:86:4c:c4.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'pasayten,172.16.238.128' (RSA) to the list of known hosts.
3.2. Operating System Prerequisites
3.2.1. Notes on Operating System Prerequisites
Tungsten has minimal prerequisites for installation and configuration. The following sub-sections describe require-
ments for each supported operating system. Note that Ruby is required for all operating systems.
3.2.2. Linux Prerequisites
Table 3.2. Linux Checklist
Description
Check (commands in
this font)
Expected Result
Check Ruby release.
ruby --version
You should see a Ruby 1.8 or later version, as in
the following example:
ruby 1.8.5 (2006-08-25) [i386-linux]
You can install Ruby using yum install ruby for
Redhat distributions or apt-get install ruby for
Ubuntu.
3.2.3. Mac OS X Prerequisites
Table 3.3. Mac OS X Checklist
Description
Check (commands in
this font)
Expected Result
Check Ruby release.
ruby --version
You should see a Ruby 1.8 or later version, as in
the following example:
ruby 1.8.6 (2008-03-03 patchlevel 114) [univer-
sal-darwin9.0]
Host Prerequisites
Tungsten Installation Guide for PostgreSQL - Document issue 1.3.2
7
Tungsten version 1.3.2
3.2.4. Solaris Prerequisites
Table 3.4. Solaris Checklist
Description
Check (commands in
this font)
Expected Result
Check Ruby release.
ruby --version
You should see a Ruby 1.8 or later version, as in
the following example:
ruby 1.8.5 (2006-08-25) [i386-linux]
Rudy is not installed by default on all So-
laris hosts. You can obtain Ruby from http://
www.sunfreeware.com.
Check for GNU tar.
[/usr/local/bin/]tar --
version
You should see GNU tar, for example:
tar (GNU tar) 1.21 Copyright (C) 2008 Free
Software Foundation, Inc.
GNU tar is not installed by default on most So-
laris hosts. You can obtain GNU tar from http://
www.sunfreeware.com.
3.3. Java Virtual Machine
Tungsten requires the Sun JDK 5 or greater. Incorrect Java versions will result in failures. Users must install the
Sun JDK, which is available at http://java.sun.com.
Table 3.5. Java Checklist
Description
Check (commands in
this font)
Expected Result
Check JAVA_HOME.
echo $JAVA_HOME
Should point to Sun JDK install location.
Check Java release.
java -version
You should see the Sun JDK, for example:
java version "1.6.0_22"
Java(TM) SE Runtime Environment \
(build 1.6.0_22-b04)
Important
Many Linux distributions ship non-Sun Java versions such as GNU Java. These versions use different com-
mand line flags and different libraries from the Sun JDK. We recommend users remove non-Sun versions
if present as they may cause Tungsten processes to fail at start-up or otherwise misbehave.
Warning
Do not mix Java versions. All hosts running Tungsten must run Java version 1.6.
Host Prerequisites
Tungsten Installation Guide for PostgreSQL - Document issue 1.3.2
8
Tungsten version 1.3.2
3.4. Sudo Prerequisites
Non-root accounts typically require sudo to start and stop database servers through the Tungsten Manager as well
as to manage manager virtual IP addresses.
Sudo access is also required to manage PostgreSQL Warm Standby replication, which requires server restart
during slave provisioning.
Table 3.6. Network Checklist
Description
Check (commands in this
font)
Expected Result
Ensure sudo is config-
ured for non-root ac-
counts and does not re-
quire a password.
sudo -l
Must show a list of commands that this
login can execute without a password.
For example like (ALL) NOPASSWD:
ALL.
Allow batch sudo com-
mands from ssh.
ssh localhost sudo -l re-
quiretty is unset.
Ensure the command works. If you get
a message like sudo: sorry, you
must have a tty to run sudo you
will need to unset requiretty as de-
scribed below.
3.4.1. Sudo Configuration Hints
The sudo program is configured using the sudoers file. You must run visudo as root to edit this file.
Recent sudo releases by default prevent commands like ssh from issuing sudo commands remotely. Tungsten
needs this capability, which you can enable by commenting out requiretty. More recent versions of sudo may
also require that you enable visiblepw. Actual sudo privileges must be accessible without a password or Tung-
sten will hang while attempting to execute commands.
# Batch sudo commands do not work if requiretty is set.
#Defaults requiretty
Defaults visiblepw
...
## Allow postgres to run any command
postgres pasayten=(root) NOPASSWD: ALL

Ubuntu users may have to use different semantics for visudo:
Defaults:postgres !authenticate
...
## Allow postgres to run any command
postgres ALL=(ALL) ALL

Host Prerequisites
Tungsten Installation Guide for PostgreSQL - Document issue 1.3.2
9
Tungsten version 1.3.2
3.5. Network Prerequisites
Tungsten requires that machines have assigned host names and that all hosts used in a Tungsten cluster have
names that resolve reliably to non-loopback IP address. IP addresses and ports used by Tungsten may not be
blocked by firewalls.
Table 3.7. Network Checklist
Description
Check (commands in this
font)
Expected Result
Check hostname.
uname -n
Should resolve to the unique name of
host.
Check local host IP.
hostname --ip-address
Must resolve to a "real" IP address; not
to the loopback address 127.0.0.1. Pri-
vate IP addresses are accepted.
Ensure that all other
host names in cluster
resolve to their IP ad-
dresses from this host.
hostname --ip-address
All host names must resolve to a non-
loopback IP address. There should be no
overlaps or inconsistency in names be-
tween hosts.
Determine a witness
host IP address to use.
The witness host should be
on the same network as the IP
address identified above and
should not be one of the clus-
ter database servers. The IP ad-
dress should be used in case
there is an issue with the DNS
server.
ping {witness IP address}
Ensure that you are able to ping the wit-
ness host IP from each cluster database
server.
Check visibility of TCP/
IP ports.
Check visibility of ports from an-
other host. You can use telnet
hostname port to verify acces-
sibility. Here are the key ports
that must be open.
• 2112 - Port for replication
THL. Must be open on any
host that runs a replicator.
• 7800-7805 - Group communi-
cations ports. Must be open
on any host that runs a man-
ager.
• 9999 - Client port for the Con-
nector (not required for hosts
that do not run the Connector)
The ports must be visible. For test instal-
lations we recommend you remove fire-
walls between hosts. Also, do not forget
normal database ports like 5432 for Post-
greSQL.
Host Prerequisites
Tungsten Installation Guide for PostgreSQL - Document issue 1.3.2
10
Tungsten version 1.3.2
Description
Check (commands in this
font)
Expected Result
• 10000 - Listener port for the
replicator. Must be open on
any host that runs a replicator.
• 11999 - Tungsten manager
client port used by Connec-
tor to connect and receive up-
dates on status of cluster data
sources.
• 12001 - Gossip router port
(not required when using mul-
ti-cast communications)
Warning
Tungsten is highly dependent on correct and consistent resolution of host names to IP addresses. Incon-
sistencies due to different /etc/hosts files or other name resolution issues are one of the most common
causes of cluster failures. Always check host name resolution very carefully on each new host and recheck
if you encounter problems in operation.
3.6. PostgreSQL Database Prerequisites
Tungsten installations that use PostgreSQL must be able to connect using TCP/IP without a password to both the
local server as well as the master server. In addition, warm standby clusters require certificate-based ssh access
in order to rsync files from the master to slave server during provisioning.
Warning
Tungsten for PostgreSQL accounts MUST use the PostgreSQL system account to run Tungsten. For stan-
dard PostgreSQL installations, you must perform the following checks while logged in as the 'postgres' user
on the operating system.
Table 3.8. PostgreSQL Database Checklist
Description
Check (commands in this
font)
Expected Result
Confirm the presence
of the pg_standby exe-
cutable.
pg_standby --version
This command should return version in-
formation for pg_standby, as shown in
the following example:
pg_standby (PostgreSQL) 8.4.0

Host Prerequisites
Tungsten Installation Guide for PostgreSQL - Document issue 1.3.2
11
Tungsten version 1.3.2
Description
Check (commands in this
font)
Expected Result
pg_standby is available in the post-
gresql-contrib-VERSION package.
You may need to add the PG_HOME/bin
folder to your path.
Check TCP/IP access
to local Postgres server
using PostgreSQL sys-
tem account without a
password.
psql -hlocal_host_name
Must be able to login without a pass-
word. See the note on configuring ac-
count access below if this check does
not succeed.
You may also have to configure the
listen_addresses parameter in
postgresql.conf.
Check TCP/IP access
to the master Post-
gres server using Post-
greSQL system account
without a password.
psql -hmaster_host
Must be able to login to master host with-
out a password. See the note on config-
uring account access below if this check
does not succeed.
Check ability to restart
database
[sudo] /etc/init.d/postgresql
[start | stop]
PostgreSQL login must be able to restart
the database. See sudo configuration in-
structions in Section 3.4, “Sudo Prereq-
uisites” to set up sudo access if this test
does not work. The database start script
name and location may vary according
to your operating system and version of
PostgreSQL.
Check ability to ssh
to other servers in the
cluster without a pass-
word. Do this for each
server that participates
in the cluster.
ssh host_name ls
Command must show files on the remote
host without prompting for a password
or host key authentication. If you do not
have certificates configured see the note
below on host access.
3.6.1. Configuring PostgreSQL Server Account Access
Tungsten depends the PostgreSQL system account being able to log in without a password to the local PostgreSQL
server. Warm standby clustering must also login to the master server without a password in order to start backups.
PostgreSQL supports such logins in a secure manner but needs to be configured carefully to avoid introducing
security holes.
For local access only we recommend configuring ident authentication using the pg_hba.conf file, which controls
server access. Use an editor to make these changes. Here is an example of typical ident settings for local access
that also preserves password access for logins from remote hosts.
# "local" is for Unix domain socket connections only
local all all ident
Host Prerequisites
Tungsten Installation Guide for PostgreSQL - Document issue 1.3.2
12
Tungsten version 1.3.2
# IPv4 local connections:
host all all 127.0.0.1/32 ident
# TCP/IP connections
host all all 172.16.238.1/24 md5
Note
Whenever you make changes to the pg_hba.conf file you should restart the PostgreSQL server and
double check settings to ensure they work correctly.
In cases where both local and remote access are required, we recommend using a .pgpass file, which contains
passwords to be used for logins. The following instructions describe how to create a .pgpass file for the 'postgres'
account.
1.While logged in as 'postgres', edit file .pgpass in the home directory using a command like the following:
vi ~/.pgpass
2.Add a line for the 'postgres' account like the following example, where 'secret' is the password.
*:*:*:postgres:secret
3.Ensure the file is readable only for the 'postgres' login using the following chmod command.
chmod 600 ~/.pgpass
If Streaming Replication is used, pg_hba.conf must specify a connection for replication explicitly (it is not
considered a part of option all). For example:
host all all 172.16.210.0/24 md5
host replication all 172.16.210.0/24 md5

Switch and failover will not work unless you enable connections for replication on all your PostgreSQL servers.
Note
Tungsten relies on the psql console command. Tungsten does not specify a password to this command as
a parameter. Instead, it relies on you to preconfigure it in the .pgpass file. Thus, if the postgres user has
a password, define it in the .pgpass file and check whether a plain psql call can connect to the database.
3.6.2. Preparing the Archive Folder
Tungsten saves new WAL files into what is known as the archive folder. On a master, new WAL files stay in this
folder until they are sent out to slaves, and on a slave until they are recovered and the pg_standby process
decides that they are no longer needed.
You must create this folder before running ./configure, which will prompt for it. The archive folder must not be
a subfolder of the PostgreSQL data directory and the postgres user must own it.
Host Prerequisites
Tungsten Installation Guide for PostgreSQL - Document issue 1.3.2
13
Tungsten version 1.3.2
Note
In the case of a lagging slave, the master's archive folder (to be more specific, the outbox folder inside of
it) will accumulate WAL files, which are waiting to be sent out to the lagging node. Depending on the lag,
the archive folder can drastically increase in size. Slaves should not lag in a healthy cluster slaves, but be
prepared for a corner case like this by providing a couple of gigabytes of extra disk space for the archive
folder. In the case of an unseen noticeable slave lag, troubleshoot and solve the issue.
For more information, see chapter How Tungsten Replicator Implements PostgreSQL WAL Shipping in Tungsten
Replicator Guide.
3.6.3. Configuring Primary and Standby Servers
This section provides a list of recommendations on configuring the primary and standby servers in the PostgreSQL
environment:
• Create the primary and standby servers in a way that they are as similar as possible, at least from the perspective
of the database server.
• The path names associated with tablespaces will be passed across unmodified. If you use tablespaces, the
primary and standby servers must have the same mount paths for tablespaces.
• The hardware architecture on the primary and standby servers must be the same. A 32-bit and 64-bit system
are not interoperable.
• In general, log shipping between servers running different major PostgreSQL release levels is not possible.
Because PostgreSQL Global Development Group does not make changes to disk formats during minor release
upgrades, it is likely that running different minor release levels on the primary and standby servers will work.
However, no formal support for that is offered and we recommend that you the keep the primary and standby
servers at the same release level as much as possible. When updating to a new minor release, the safest policy
is to update the standby servers first - a new minor release is more likely to be able to read WAL files from a
previous minor release than vice versa.
Connecting Applications to Tungsten
Tungsten Installation Guide for PostgreSQL - Document issue 1.3.2
14
Tungsten version 1.3.2
Chapter 4. Connecting Applications to Tungsten
4.1. Application Configuration
Tungsten requires minor reconfiguration for applications that wish to connect to the cluster through either the
Connector or the Tungsten SQL Router. This chapter contains advice on setup.
4.2. Configuring Applications to Use Connector
The Connector allows applications to connect simply by changing connection strings. To use the Connector, appli-
cations should change connection strings to connect to the Connector host and port. The following example shows
PHP configuration.
<?php
$hostname = "ganesh";
$dbname = "test?qos=RW_SESSION&sessionId=12345678971";
/* Connect */
$link = mysqli_connect($hostname, “tungsten”, “secret”, $dbname, 9999);
/* Create database */
mysqli_query($link, "create database if not exists test");
/* Close the connection */
mysqli_close($link);
?>

For more information on using the Connector, please consult Tungsten Connector Guide.
Note
Any application can use the Connector to access a Tungsten cluster. This includes applications written in
Java. All you need to to is change the application connection string to point to the Connector instead of
connecting directly to the database.
4.3. Configuring Java Applications to use Tungsten SQL Router
Java applications may connect to the cluster using the Tungsten SQL Router JDBC driver. This avoids an extra
"hop" through a proxy like the Connector. The following sub-sections describe configuration steps for common
types of Java applications.
For more information on using the Connector, consult Tungsten SQL Router Guide.
4.3.1. Configuring Java Applications In General
The SQL Router introduces the following dependencies for Java applications.
• All JAR files required by SQL Router must be added to the Java class path.
Connecting Applications to Tungsten
Tungsten Installation Guide for PostgreSQL - Document issue 1.3.2
15
Tungsten version 1.3.2
• You must set the property cluster.home to the location of the cluster-home directory in the Tungsten re-
lease.
• You must enable the application to find the router.properties configuration file. This is normally located in
the cluster-home/conf directory, in which case the SQL Router will find it automatically. However, if you use
a different location you must set the router.properties property to the file location.
In addition, the application may need to be changed to connect using Tungsten SQL Router connection URLs.
Depending on the method used to connect, this may require a code change. In most cases it is only necessary to
change a property. Here is an example of Java code to connect to the SQL Router.
Class.forName("com.continuent.tungsten.router.jdbc.TSRDriver").newInstance();
Connection conn = DriverManager.getConnection(
“jdbc:t-router://myservice/mydb?qos=RO_RELAXED&maxAppliedLatency=120”,
“tungsten”, “secret”);

To configure a Java application (that is, an application invoked from the command line), follow the steps shown
below.
1.Change web-app connection strings to use the Tungsten SQL Router URL rather than a vendor JDBC URL.
2.Add all files from the SQL Router lib directory to the class path of your application. You may do this by adding
the files to the JAR files already used by your application or referencing them directly in the lib directory. Either
way, the SQL Router JAR files must be in the application class path.
3.Ensure that the application class path also includes the vendor JDBC driver JAR file.
4.Edit your application start script to add the cluster.home property, as shown in the following example:
java -Dcluster.home=/opt/continuent/tungsten/cluster-home
5.Optionally set the router.properties property if the router.properties file name is stored outside of
cluster-home/conf as shown in the following example:
java -Dcluster.home=/opt/continuent/tungsten/cluster-home -Drouter.properties=/home/apps/conf/
router.properties
Warning
Beware of the cluster home location shifting when you do a Tungsten upgrade. It is a good practice to
define a link to the current Tungsten release so that application paths are not affected by Tungsten version
changes.
4.3.2. Configuring Apache Tomcat
Apache Tomcat runs Java web applications. The normal way to handle JAR files used by applications is to place
them inside the WEBINF/lib directory. However, the SQL Router is limited in that there may be only one SQL
Router per Java process. This affects the recommended configuration for Tomcat.
The following configuration instructions are based on Tomcat 6.0.x. Note that class loading has changed between
Tomcat versions. For earlier versions of Tomcat, consult the Tomcat documentation and adapt the configuration
accordingly.
1.Change connection properties to specify the SQL Router driver class and JDBC connection URL. Settings will
vary according to the application in use.
Connecting Applications to Tungsten
Tungsten Installation Guide for PostgreSQL - Document issue 1.3.2
16
Tungsten version 1.3.2
2.Add all files from the SQL Router lib directory to the $CATALINA_HOME/lib directory.
3.Add the vendor JDBC driver JAR file to $CATALINA_HOME/lib.
4.Edit the catalina.sh script to set the cluster.home property within the script. Alternatively, you can specify
the value using the CATALINA_OPTS environmental variable, as shown below.
export CATALINA_OPTS=-Dcluster.home=/opt/continuent/tungsten/cluster-home
5.Optionally set the router.properties property if the router.properties file name is stored outside of
cluster-home/conf as shown in the following example:
export CATALINA_OPTS="-Dcluster.home=/opt/continuent/tungsten/cluster-home -Drouter.properties=/
home/apps/conf/router.properties"
Tip
You may need to alter Log4j settings to ensure log messages go into a file with automatic roll-over rather
than logging to the console.
Warning
Beware of class loading problems when placing SQL Router JARs in the shared $CATALINA_HOME/lib.
All SQL Router dependencies must be available here or you will get ClassNotFoundException on start-up.
After completing these settings, you should be able to start Tomcat and have your applications connect to the
database. Check the Tomcat logs for error messages.
Detailed Cluster Configuration
Tungsten Installation Guide for PostgreSQL - Document issue 1.3.2
17
Tungsten version 1.3.2
Chapter 5. Detailed Cluster Configuration
This chapter contains information about detailed configuration of clusters. This is intended to help you make con-
figuration settings above and beyond those supplied by the automated Tungsten configuration script.
5.1. Logger Configuration
5.1.1. Introducing Log4j
Tungsten uses Apache log4j to log messages. Each service has a log4j.properties file that controls the level
of log messages and whether they go to the console, a file, or to Unix syslog facility.
For further information on log4j.properties, check the documentation at the Apache Log4j website, which is
located at http://logging.apache.org/log4j/1.2/index.html.
5.1.2. Log4j Configuration Settings
All Tungsten services come with pre-configured log4j settings. You may alter these as you wish in order to adjust log
levels or destination by editing the log4j.properties file. After changing settings, you must restart the service
for new settings to take effect.
Generally speaking, Tungsten services define the following output destinations.
• stdout - Sends output to the console. This setting is the default for all Tungsten services. The service wrappers
automatically redirect this to a file.
• file - Send output to a log file with defined rules for roll-over.
• syslog - Send output to Unix syslog facility.
Most Tungsten services write at the INFO level, which produces a reasonable level of notifications, warnings, and
error messages. The DEBUG level may be enabled for diagnostic purposes. Examples are provided at the end of
each log4j.properties file.
The following example shows log4j settings to enable debug logging on Tungsten replicator.
# Example of how to turn on debugging. Specify the name of a package or
# a Java class. This turns on debugging for all replicator classes.
log4j.logger.com.continuent.tungsten.replicator=DEBUG, Console
log4j.additivity.com.continuent.tungsten.replicator=false

Note
Changing Log4j settings only affects the service whose log4j.properties you alter. Other services are
not affected.
Warning
Debug logging generates voluminous output. It should not be enabled on production systems unless re-
quired to diagnose problems.
Detailed Cluster Configuration
Tungsten Installation Guide for PostgreSQL - Document issue 1.3.2
18
Tungsten version 1.3.2
5.2. Java Service Wrapper Configuration
5.2.1. Introducing Service Wrapper
Tungsten uses Java Service Wrapper (http://wrapper.tanukisoftware.org/) to allow Java processes that implement
cluster functions to operate as operating system services. The Java Service Wrapper uses a configuration file that
specifies arguments for the Java process. From time to time it is necessary to make configuration changes to the
wrapper.
5.2.2. Wrapper Configuration Settings
Tungsten services define Java Service Wrapper configuration in file wrapper.conf. To make changes you can
edit the file and change settings as needed. You must restart the service for new settings to take effect. Most
settings are documented with comments. The most commonly changed settings are the following.
• Memory - You can raise Java VM memory by editing the wrapper.java.maxmemory property.
• Java VM Options - You can supply additional Java VM options using wrapper.java.additional properties.
• Debugging - You can enable Java remote debugging by uncommenting the debug properties included in most
Tungsten wrapper.conf files. These are for developer use only and should not be enabled unless needed.
5.3. Network Configuration
5.3.1. Network Port Access
Tungsten nodes require open access to the following TCP/IP ports on each host. Local environments may require
adjustment of firewalls to ensure access. In Amazon cloud environments you must use a security group that opens
these ports to other members of the cluster.
Note
Some ports are particular to MySQL or PostgreSQL and are so noted. Ports in general are default values;
actual values may differ for particular installations.
• 22 - Default port for ssh and rsync access. Tungsten requires this port to manage PostgreSQL warm standby
clusters.
• 2112 - Default port for master Tungsten Replicator to accept client connections. This must be open for any host
that runs a replicator that will act as a master.
This port is not currently used for PostgreSQL clustering.
• 5432 - Default port for PostgreSQL listener. Tungsten requires remote access to this port in order to run backups
in PostgreSQL clusters. It must also be open for applications to connect.
• 7800-7805 - Communication ports allocated by group communications for Tungsten Manager and Tungsten
Monitor. Tungsten starts with the first port in the range and allocates additional ports as needed.
• 10000 - Listener port for the replicator. Must be open on any host that runs a replicator.
• 11999 - Manager port used by SQL Router and Connector to get information about cluster state. The port must
be accessible to any host that uses these products or applications will hang.
Detailed Cluster Configuration
Tungsten Installation Guide for PostgreSQL - Document issue 1.3.2
19
Tungsten version 1.3.2
• 12001 - Listener for gossip router. This port only needs to be open on on hosts that run the gossip router.
5.3.2. Overview of Group Communications Configuration
Tungsten uses group communications to exchange messages between monitors and middleware components like
the Tungsten SQL Router. Group communications allow processes to find each other (membership management)
and send reliable messages to all of the members of the group.
Group communications setup is handled by the automated configuration script described in Chapter 2, Product
Installation and Configuration. Users do not normally need to perform additional configuration.
Group communications are controlled by two files presents in the conf directory of most Tungsten components. File
hedera.properties selects the type of group communications and provides specific configuration. Tungsten
uses JGroups, which is a popular group communications library written in Java. A typical hedera.properties
looks like the following example.
# Hedera properties file

# Using JGroups
hedera.factory=org.continuent.hedera.factory.JGroupsGroupCommunicationFactory
hedera.channel.jgroups.config=/jgroups_tcp.xml
hedera.channel.jgroups.fragmentSize=32000

The hedera.channel.jgroups.config provides the location of a second configuration file specific to JGroup.
This file specifies the communication protocol between members as well as the mechanism to find group members
in the first place. The JGroups configuration file must be identical for all members of the group.
Warning
The forward slash preceding the JGroups configuration file name is required.
Note
By default JGroups will use TCP port 7800 for communication. If unavailable it will try the next port in
succession. You can tune the port ranges used by JGroups by changing the start_port and end_port
properties in the <TCP> tag in the relevant JGroups configuration file.
5.3.3. Group Communications with Multicast
The default JGroups configuration is in jgroups_tcp.xml. It uses multicast to locate group members and TCP/IP
to communicate between members once they join the group. This configuration works for most local area networks
that support multicast routing. To use it, ensure that hedera.properties contains the following reference.
hedera.channel.jgroups.config=/jgroups_tcp.xml

On networks that do not support multicast, Tungsten components will hang without locating each other. When this
happens it is necessary to use an alternate configuration.
Detailed Cluster Configuration
Tungsten Installation Guide for PostgreSQL - Document issue 1.3.2
20
Tungsten version 1.3.2
5.3.4. Group Communications with Ping Protocol
The JGroups TCPPING protocol uses fixed IP addresses to identify cluster members. It is a good approach for
clusters that have stable membership and cannot use multicast. Ping protocol works in virtually any network con-
figuration. To use it, ensure that hedera.properties contains the following reference.
hedera.channel.jgroups.config=/jgroups_tcp_ping.xml

If you add or subtract hosts on the network, you must change the host names in the jgroups_tcp_ping.xml
file. This can be done manually by editing the TCPPING tag or by rerunning the configure script. You must restart
all managers following a membership change, so changes will typically require a short maintenance window.
Warning
Inconsistent host addresses on different nodes will lead to unpredictable cluster behavior. Ensure that you
specify cluster hosts consistently for each node and change them consistently as new hosts are added or
old hosts are removed.
5.3.5. Using a Gossip Server
The final JGroups configuration uses a gossip server to add new group members. The gossip server maintains a
list of the current members. New members contact the gossip server, which then distributes updated membership
information to the new member as well as the rest of the group.
Gossip servers are somewhat more complex to manage but work in cases where cluster membership changes
regularly and multicast is not available. The following procedure describes how to configure JGroups with a gossip
server.
1.Select a host to run the gossip server and then cd to the gossiprouter/bin directory of your Tungsten
release. Start the gossip server as follows:
gossiprouter start
Tip
The gossip server by default listens on port 12001. If this port is not suitable, locate the following
port option setting in the gossip server configuration file named gossipwrapper.conf and select a
different port:
# Application parameters. Add parameters as needed starting from 1
wrapper.app.parameter.1=org.jgroups.stack.GossipRouter
wrapper.app.parameter.2=-port
wrapper.app.parameter.3=12001

Update hedera.properties to set the configuration file as shown below.
hedera.channel.jgroups.config=/jgroups_tcp_gossip.xml

Detailed Cluster Configuration
Tungsten Installation Guide for PostgreSQL - Document issue 1.3.2
21
Tungsten version 1.3.2
2.Edit file jgroups_tcp_gossip.xml to set the gossip server host name(s). Locate the <TCPGOSSIP> tag
and fill in the host names, as shown below. Note the port 12001. If you start the gossip server with a different
port number you must update this tag accordingly.
<TCPGOSSIP initial_hosts="myhost1[12001],myhost1[12001]"
gossip_refresh_rate="10000"
num_initial_members="2" />

Tip
To ensure maximum availability of the cluster users should run at least two gossip servers.
Warning
Gossip server failures can cause problems with membership management in Tungsten clusters. If you
experience problems with managers joining the cluster, ensure that all gossip routers are running properly.
5.4. JMX Configuration
5.4.1. Introducing JMX
Tungsten uses JMX for management and monitoring. JMX enables remote management of Java processes. The
protocol and its use are a standard feature of Java Virtual Machines.
For further information on JMX, consult the documentation available at the following location: http://java.sun.com/
j2se/1.5.0/docs/guide/management. At this location you will also find definitions for property settings used to control
JMX behavior.
5.4.2. JMX Configuration Settings
Generally speaking there is little need for users to make JMX configuration changes. That said, there are a number
of properties that control JMX behavior. These can be set directly when starting the Java process, as in the following
example.
java -Dproperty=value ...
You can also set multiple properties at once using a JMX properties file. A sample JMX properties file is included
the conf directories of certain Tungsten services and is named sample.jmx.properties. To use this file, you
must add the following property when starting Java.
java -Dcom.sun.management.config.file=path/to/propfile
5.4.3. JMX on NAT-enabled hosts
JMX requires the host name used for connection to match the name of the host itself. This can be a problem in
some cases where the host is accessed through NAT, in which case the host name likely resolves to a different
name and IP from that used by remote JMX clients. This will result in refused connections.
To cure this problem you must set the following Java property either directly when invoking Java or using the JMX
properties file. You can use either the host name or the IP address to which you are binding.
java.rmi.server.hostname=public.host.name
Detailed Cluster Configuration
Tungsten Installation Guide for PostgreSQL - Document issue 1.3.2
22
Tungsten version 1.3.2
For further information consult JMX documentation. This problem is also discussed in various online forums. You
can search on the java.rmi.server.hostname property to find online descriptions of the problem and how
to resolve it.
5.5. Monitoring & Alerting
5.5.1. Nagios Monitoring
There are four checks bundled with the Tungsten Enterprise package. Each is written to be used with the NRPE
daemon as part of a complete monitoring installation. Each script should be run from the cluster-home/bin
directory. Refer to the documentation for NRPE or your monitoring solution on how to use these scripts.
• check_tungsten_services [-r] [-c] - Check if the Tungsten services for this host are running.
This check should be run on every server running Tungsten services. The -r option will confirm that both the
manager and the replicator are running. The -c option will check the connector service.
• check_tungsten_online - Check if all cluster services are online.
This check is only required on a single server. Look at each database server, manager and replicator to make
sure they are ONLINE. The check will return WARNING if just one is not ONLINE. It will return CRITICAL if there
is more than one.
• check_tungsten_latency -w # -c # [--perfdata] [--perslave-perfdata] - Compare the
maximum latency to warning and critical levels.
This check is only required on a single server. Return WARNING if the maximum latency is between -w and -c.
Return CRITICAL if the maximum is greater than -c.
You can also retrieve maximum latency data that is compatible with Nagios 3 using the --perfdata option.
Append the --perslave-perfdata option if you would like the performance data to be given.
# Maximum latency performance data
$ check_tungsten_latency -w 1 -c 2 --perfdata
WARNING: mdb2=1.734s | max_latency=1.734;1;2;;
# Per slave latency performance data
$ check_tungsten_latency -w 1 -c 2 --perfdata --perslave-perfdata
WARNING: mdb2=1.734s | mdb2=1.734;1;2;; mdb3=.962;1;2;;
• check_tungsten_progress -t # - Check if the local replicator is progressing.
Return CRITICAL if the replicator is not ONLINE. Return WARNING if the replicator does not complete one event
during the given number of seconds.
5.5.2. Other Monitoring Tools
These scripts are written to be compatible with the Nagios system. Some systems are able to use these checks
without any modification. Continuent is available to customize them for your environment or you may use the scripts
as the basis for your own modifications.