Installing PEM Watchdog - FTP Directory Listing - Parallels

herbunalaskaΔιαχείριση Δεδομένων

30 Ιαν 2013 (πριν από 4 χρόνια και 5 μήνες)

311 εμφανίσεις



SWsoft

PEM 2.6 Update 2

Watchdog User's Guide

Revision 2.0 (March 28, 2007)


(c) 1999
-
2007




ISBN: N/A

SWsoft.

13755 Sunrise Valley Drive

Suite 325

Herndon

VA 20171 USA

Phone: +1 (703) 815 5670

Fax: +1 (703) 815 5675


Copyright 1999
-
2007 SWsoft. All righ
ts reserved.

Distribution of this work or derivative of this work in any form is prohibited unless prior written
permission is obtained from the copyright holder.

PEM is a registered trademark of SWsoft, Inc.

Linux is a registered trademark of Linus Torva
lds.

RedHat is a registered trademark of Red Hat Software, Inc.

UNIX is a registered trademark of The Open Group.

Intel, Pentium, Xeon and Celeron are registered trademarks of Intel Corporation.

SSH and Secure Shell are trademarks of SSH Communications Sec
urity, Inc.

Request Tracker is a trademark of Best Practical Solutions, LLC.





3


Contents

Preface

5

About This Guide

................................
................................
................................
............................

6

Who Should Read This Guide

................................
................................
................................
........

6

Org
anization of This Guide

................................
................................
................................
............

6

Documentation Conventions

................................
................................
................................
..........

6

Typographical Conventions

................................
................................
................................
.

6

Feedback

................................
................................
................................
................................
........

8

Introduction

9

About Watchdog

................................
................................
................................
...........................
10

Why You Should Use Watchdog

................................
................................
................................
..
10

Services to Monitor

................................
................................
................................
.......................
10

References

................................
................................
................................
................................
...
11

Installing PEM Watchdog

12

PEM Watchdog Installation Requirements and Prerequisites

................................
.....................
13

PEM Watchdog Distribution and Packaging

................................
................................
................
14

Installation Instructions

................................
................................
................................
.................
15

Configuring PEM Watchdog

16

PEM Watchdog Configuration File Structure

................................
................................
...............
17

PEM Watchdog Directives

................................
................................
................................
...........
17

#

................................
................................
................................
................................
.........
17

instance [ = name ]

................................
................................
................................
.............
18

perio
d

................................
................................
................................
................................
.
18

timeout(<service>) = <timeout>

................................
................................
.........................
18

default_timeout
................................
................................
................................
...................
19

retries(<service>) = <number>

................................
................................
..........................
19

service = <name>@<host>

................................
................................
................................
20

include(<name>)

................................
................................
................................
................
20

input = <name>

................................
................................
................................
..................
21

allow/deny(<service>)

................................
................................
................................
........
22

loglevel

................................
................................
................................
...............................
23

pulse

................................
................................
................................
................................
...
23

use_dns

................................
................................
................................
..............................
23

use_postponed
................................
................................
................................
...................
24

use_doublecheck

................................
................................
................................
...............
24

use_suckers

................................
................................
................................
.......................
24

notify_always

................................
................................
................................
......................
24

notify(<event type>) = <email>

................................
................................
..........................
25

cc(<event type>)
-

<email>

................................
................................
................................
25

mailer(<event_type>)

................................
................................
................................
.........
26

template(<event_type>) = /path/to/file

................................
................................
...............
26

subject(<event_type>) = <subject_template>

................................
................................
....
27

dbstore

................................
................................
................................
...............................
28

PEM Watchdog Check Modules

................................
................................
................................
..
29

Apache Web Server

................................
................................
................................
...........
29



ApacheAdmin25
-

PEM UI

................................
................................
................................
.
29

Dotster
-
specific

BillingManager

................................
................................
.........................
30

BM

................................
................................
................................
................................
......
30

PEM ClusteredQmail Service

................................
................................
............................
31

Linux File Manager

................................
................................
................................
............
31

Horde Webmail

................................
................................
................................
..................
32

Linux LoadAv
erage

................................
................................
................................
............
32

MS Exchange Server

................................
................................
................................
.........
33

Microsoft IIS Server

................................
................................
................................
...........
33

Microsoft SQL Server

................................
................................
................................
.........
34

MySQL Server

................................
................................
................................
....................
34

Outlook Web

Access

................................
................................
................................
..........
34

PEM Accounts (Deprecated)

................................
................................
.............................
34

PEM Service Controllers

................................
................................
................................
....
35

PEM pg_autovacuum service

................................
................................
............................
35

PostgreSQL Server

................................
................................
................................
............
35

Ping

................................
................................
................................
................................
....
35

Plesk Server Administrator

................................
................................
................................
35

ProFTPD Service

................................
................................
................................
...............
36

Qmail Service (Deprecated)

................................
................................
...............................
36

WebComStore Service

................................
................................
................................
......
37

PEM Watchdog Integration Modules

................................
................................
............................
37

File

................................
................................
................................
................................
.....
37

PEM

................................
................................
................................
................................
...
38

Configuration File Sample

................................
................................
................................
............
39

Calculating Checking Time

................................
................................
................................
..........
40

General Operations on PEM Watchdog

42

Upgrading PEM Watchdog

................................
................................
................................
...........
42

Starting/Stopping Watchdog

................................
................................
................................
........
42

Index

43





5


In This Chapter

About This Guide

................................
................................
..............................

6

Who Should Read This Guide

................................
................................
...........

6

Organization of

This Guide

................................
................................
................

6

Documentation Conventions

................................
................................
.............

6

Feedback

................................
................................
................................
..........

8



C
H A P T E R
1

Preface



Preface

6


About This Guide

The purpose of the
PEM Watchdog User's Guide

is to describe in details the functional
features and management tools of the PEM Watchdog daemon.

Who Should Read This Guide

This guide is intended for people engaged in administering PEM Watchdog
-

a daemon
that periodica
lly checks PEM processes and services in order to detect the possible
problems and notify the PEM Watchdog administrator or other PEM support stuff about
them. The reader of this guide is expected to be familiar with PEM
-

the hosting
platform developed an
d distributed by SWsoft Inc., and have good knowledge in using
and administering Unix/Linux OSs.

Organization of This Guide

This guide has the following structure:

Chapter 2,
Introduction

(on page
9
),
acquaints you with the PEM Watchdog application
and its main features.

Chapter 3,
Installing PEM Watchdog

(on page
12
), tells about requirements and
preconditions you need to observe before installing Watchdog. It also contains the
in
stallation procedure itself.

Chapter 4,
Configuring PEM Watchdog

(on page
16
), depicts in details the structure of the
PEM Watchdog configuration file. All PEM Watchdog directives and check modules are
described in this chapter.

Chapt
er 5,
General Operations on PEM Watchdog

(on page
42
), contains information of how
to start, stop and upgrade Watchdog.

Documentation Conventions

Before you start using this guide, it is important to un
derstand the documentation
conventions used in it.

Typographical Conventions

The following kinds of formatting in the text identify special information.



Preface

7


Formatting convention

Type of Information

Example

Special Bold

Items y
ou must select, such
as menu options, command
buttons, or items in a list.

Go to the
QoS

tab.


Titles of chapters, sections,
and subsections.

Read the
Basic Administration

chapter.

Italics

Used to emphasize the
importance of a point, to
introduce a term
or to
designate a command line
placeholder, which is to be
replaced with a real name or
value.

The system supports the so called
wildcard character

search.

Monospace

The names of commands,
files, and directories.

The license file is located in the
httpdoc
s/common/licenses

directory.

Preformatted

On
-
screen computer output
in your command
-
line
sessions; source code in
XML, C++, or other
programming languages.

# ls

al /files

total 14470

Preformatted Bold

What you type, contrasted
with on
-
screen computer
ou
tput.

# cd /root/rpms/php

<>

The value enclosed in angle
brackets is variable, i.e.
depending on the context it
can take this or that value.

Go to unpacked OS directory:

#

cd <PEM_DIST>/rpms/<OS

CAPITALS

Names of keys on the
keyboard.

SHIFT, CTRL, ALT

K
EY+KEY

Key combinations for which
the user must press and hold
down one key and then press
another.

CTRL+P, ALT+F4







Preface

8


Feedback

If you have found a mistake in this guide, or if you have suggestions or ideas on how to
improve this guide, ple
ase send your feedback to
userdocs@swsoft.com
. Please
include in your report the guide's title, chapter and section titles, and the fragment of
text in which you have found an error.







9


In this chapter you will get the general idea of what the PEM Watchdog is and
understand its main functions.

In This Chapter

About Watchdog

................................
................................
................................

10

Why You Should Use Watchdog

................................
................................
.......

10

Services to Monitor
................................
................................
............................

10

References

................................
................................
................................
........

11



C
H A P T E R
2

Introduction



Introduction

10


About Watchdog

PEM Watchdog is a a

Linux daemon written in Perl that monitors the state of different
services running in PEM or under any other hosting platform. Its general purpose is to
send notifications on problems either occurred in the system or that are about to occur.
When a PEM Wa
tchdog administrator receives a notification, he/she is supposed to
take measures to fix or prevent the problem manually basing on notification message
and log attached to it.

You also have an option to save and keep the Watchdog check results in the exter
nal
MySQL database.

Why You Should Use Watchdog

The purpose of using the Watchdog service seems to us rather obvious, but
nevertheless some words on its usefulness and advantages won't be out of place.

If you read this gui
de, it means that you have the honor to be a staff member of PEM
support and one of your tasks is to monitor the stability of this hosting platform. It would
not be true to say that services and applications used in present
-
day hosting industry
run like cl
ock
-
work. Machines reboot, programs crash, filesystems go up and down.
And as any other complex mechanism, they require steadfast control. The bigger the
company offering hosting services, the better quality it must grant to their customers.
Stable 24
-
hour

fail
-
safe hosting is the most weighty argument the hosting company can
adduce in a competitive struggle. Ensuring that a collection of services is always
running can require a large amount of manual activity in the face of these challenges.
Setting up PEM

Watchdog once, it investigates all the services concurrently on a
round
-
the
-
clock basis and immediately informs if any problems are detected.

Another good reason to use PEM Watchdog is the prevention of malicious attacks
targeted to slow down or put the s
ystem out of action. When a Watchdog marks the
unusual overload in one of the system components or registers multiple failed login
attempts, it calls an alert.

Services to Monitor

The current PEM Watchdog version is able to monito
r the state of the following PEM
objects, processes and services:



Apache webserver



Apache webadmin



Billing Manager



bind



Clustered qmail



File manager



Introduction

11




Horde



Load average



Melin



MS Exchange



MS IIS



MSSQL



MySQL



OWA



PEM Accounts



PEM Service Controllers



pg_autovac
uum



PgSQL



ping



Plesk



proFTPD



Qmail



WebComStore


References

Administering PEM Watchdog may require you deeper understanding of PEM and its
components. You can get this information from PEM User's Guides that come with
PEM distribution packa
ge, namely:



PEM Deployment Guide



PEM Provider's Guide




12


In This Chapter

PEM Watchdog Installation Requirements and Prerequisites

............................

13

PEM Watchdog Distribution and Packaging

................................
......................

14

Installation Instructions

................................
................................
......................

15



C
H A P T E R
3

Installing PEM Watchdog



Installing PEM Watchdog

13


PEM Watchdog Installation Requirements and
Prerequisites

PEM Watchdog can be installed either on physical server running under RedHat
Enterprise Linux 3 or Virtuozzo VPS based on RedHat Enterprise Linux 3 Template.
The following hardware characteristics a
re recommended:



CPU
-

Pentium II class, if no more services are running on this machine.



RAM
-

256 Mb (ECC)



HDD
-

not less that 50 Mb available diskspace (IDE interface).


Before installing PEM Watchdog, make sure the following requirements are met:



The se
rvices to be monitored are properly configured and running.



The following ports and protocols on hosts to be checked are open:

Billing Manager server

443, 5224

UI node server

443

Web server

80, 19002, FTP, 2021

MySQL server

3306

PostgreSQL server

5432

DNS servers

53 (UDP, TCP)

Mail server

25, 110, 143

Any host internal interface

22

Melin Website Creator Server

8443

Plesk server

8443

Webmail (Horde) server

80, 443

MS Exchange server

25,110, 143

MS IIS server

FTP, 80

MS SQL server

???

OWA server

443



The hostnames that are to be checked must be resolved from the Watchdog server.



If you intend to check the PostgreSQL database, you need to know the root access
username and password to it.



For notification purposes you must have either local sendmai
l server or external
SMTP server.




Installing PEM Watchdog

14


PEM Watchdog Distribution and Packaging

PEM Watchdog has modular structure and is distributed as a set of RPMs. In the PEM
Watchdog distribution package besides this User's Gu
ide you will find two directories:
contrib

and
dist
.

The
dist

directory contains the following files:

File Name

Description

watchdog
-
5.5
-
*.rpm

Watchdog engine

watchdog
-
checker
-
*.rpm

Check modules. At least one check module should be installed, so
watchdo
g could check at least one type of services.

watchdog
-
input
-
PEM
-
*.rpm

Module that fetches all services from PEM system DB.

If you plan to use this Watchdog to monitor the services running
under PEM, it's recommended to install this package. It dynamicall
y
adds all services registered in PEM.

watchdog
-
input
-
File
-
*.rpm

Module that fetches services from file. Services should be declared
in the same format as they are described in this guide.

Install this package if you plan to store the services list that w
ill be
checked by Watchdog in a separate file.



Installing PEM Watchdog

15



The
contrib

directory contains files that might be required by dependencies while
installing files from the
dist

directory.

Installation Instructions

This instruction describe
s the steps necessary to deploy PEM Watchdog and set it
running.

1

Deliver the PEM Watchdog distribution package into the
root

directory of the
server/VPS intended to run Watchdog.

2

Install watchdog engine and RPMs required by dependency.

3

Decide which service

type will be checked and install appropriate check modules
(and RPMs required by dependency).

Note
: Watchdog and its modules will require some RPMs which are not present in
the Watchdog contributed RPMs, for example, perl RPM. Such RPMs can be found
in th
e latest available PEM distribution, in PEM
-
RPMS
-
RHES
-
3.tar or in RedHat
Enterprise Linux AS3 distribution.

4

Configure watchdog: edit watchdog config file, define check instances and service
filters; alter notification subject, 'from' address, mail server a
nd notification
recipients. Follow the instructions in the
Configuring PEM Watchdog chapter of this
guide

(on page
16
).

5

Make sure the PEM Watchdog service is always turned on:


chkconfig
--
level 345 watchdog on

6

Start PEM Watchdog:



service watchdog start



16


After you have installed the PEM Watchdog engine and the corresponding modules,
you should configure the Watchdog work. All Watch
dog checks are split into separate
check processes called
'instances
'. Each instance checks services sequentially, while
all instances are running in parallel.

List of services on hosts to be checked is defined via config file for each instance. It is
poss
ible to create service lists dynamically using integration modules. Watchdog
configuration file can be found at
/etc/watchdog.conf
.

In This Chapter

PEM Watchdog Configuration File Structure

................................
.....................

17

PEM Watchdog Directives

................................
................................
.................

17

PEM Watchdog Check Modules

................................
................................
........

29

PEM Watchdog Integration Modules

................................
................................
.

37

Configura
tion File Sample

................................
................................
.................

39

Calculating Checking Time

................................
................................
................

40



C
H A P T E R
4

Configuring PEM Watchdog



Configuring PEM Watchdog

17


PEM Watchdog Configuration File Structure

Configuration file has following structure:

[global settings]

[All settings before `instance' keyword are considered to be in the
global scope. These settings affect all the instances included in the
file and usually contain timeouts information, number of retries and
mail settings.]

instance =
Ins
tance1


[Settings for
Instance1
. The settings defined here affect only this
very instance and override the settings in the global scope.]


[All directives here are considered to be directives in instance
scope]

module =
foo

{


[input module '
foo
' definitio
n]


[All directives here are considered to be directives in module
scope]

}

instance =
Instance2


[Settings for
Instance2
. The settings defined here affect only this
very instance and override the settings in the global scope.]


[All directives here are co
nsidered to be directives in instance
scope]

module =
bar

{


[input module '
bar
' definition]


[All directives here are considered to be directives in module
scope]

}

As you see, global settings are settings that do not fall under any 'instance =' entry.
In
tegration module definition can be written in any place of config file.

To see the example of the PEM Watchdog configuration file, refer to the

corresponding
topic of this guide

(on page
39
).

PEM Watchdog

Directives

In this section you will learn all the directives and settings. Each directive can be issued
in one or more of the three scopes:
global
,
instance
,
module

and can have specific
values.

Important
: Each time you edit the configuration file, restar
t the Watchdog for the
changes to take effect.

To restart watchdog, you should issue the following command:

# watchdog restart

#



Configuring PEM Watchdog

18


Directive Name

Directive Definition

Scope

Notes

#

A comment. Any characters in line after # are
ignored.

global

instan
ce

module

If you need to specify '#'
as a part of the
parameter value (i.e. as
a part of the password),
enter the backslash (
\
)
symbol before it. For
example, 'abc
\
#def' will
actually be interpreted
as 'abc#def'.



instance [ =
name ]

Directive Name

Directive Definition

Scope

Notes

instance [ = name
]

New check instance definition. Name parameter is
not mandatory and is used only for logging
purposes.

global




period

Directive Name

Directive Definition

Scope

Note
s

period

Sets the time period in seconds:

-

in instance scope

it defines the idle time between
checking all services included in this instance;

-

in module scope

this directive sets the time
period for refreshing the list of services included in
this modu
le.

instance

module

If not defined, the period
is 0 seconds.



timeout(<service>) = <timeout>

Directive Name

Directive Definition

Scope

Notes

timeout(<service>)
= <timeout>

Sets timeout for service type <service>, in
seconds. Example:

timeout(Apache) = 10 # Set 10 sec
timeout for Apache check.

It means that Watchdog will wait for 10 second
for Apache 'answer'. If no reply was received,
the attempt is considered failed.

Be aware that some services like Billing
Manager
usually require much time to pass the
data because of long authentication procedures.

global

instance

This directive can take
any positive integer.



Configuring PEM Watchdog

19




default_timeout

Directive Name

Directive Definition

Scope

Notes

default_timeout

Se
ts default timeout for instance checks (refresh
list for modules).

See the
period

directive

(on page
18
).

global

instance

module

This directive can take
any positive integer.

If not defined, the default
timeout is 10 seconds.



retries(<service>) = <number>

Directive Name

Directive Definition

Scope

Notes

retries(<service>)
= <number>

When service check is failed, PEM Watchdog
can immediately arrange a retry in case it was a
temporary glitch. Numbe
r of such retries is set
with the retries directive on per service type
basis.

For example:

retries(Apache) = 5

global

instance

Note that
use_doublecheck
and
use_postponed

have nothing to do with
retries
. For example,
if
retries

is set to 3,
and both
use_p
ostponed

and
use_doublecheck

are
enabled, there will be 12
check attempts ( = 3
retries during first fail, 3
retries during postponed
check, 3 retries during
next iteration because
doublecheck
presupposes that
service fails 2 iterations
in a row, and final
ly 3
retries during postponed
check in second
iteration). Only after 12
failures service will be
considered failed and
notification will be sent.





Configuring PEM Watchdog

20


service = <name>@<host>

Directive Name

Directive Definition

Scope

Notes

serv
ice =
<name>@<host>


Adds service of type <name> on host <host> to
be checked. Service properties are non
-
mandatory and can be passed in curly brackets
after service definition.

For example,

service = <name>@<host> {


property1 = value1



property2 = value2


...

}

Properties are different for each service. To
see what properties can be used with this or
that service, refer to the
PEM Watchdog
Check Modules chapter

(on page
29
).

global

instance

mod
ule





include(<name>)

Directive Name

Directive Definition

Scope

Notes

include(<name>)


Includes the services from the services list
source <name> or from PEM DB into the check
instance.

instance

The path to this file
should be sp
ecified
under the module
directive.





Configuring PEM Watchdog

21


input = <name>

Directive Name

Directive Definition

Scope

Notes

input = <name>


Defines services list source with name <name>,
which uses module <module> to generate
services list. Additional modu
le
-
specific
properties can be passed in curly brackets.

For example,

input = <name> {


module = <module>


property1 = value1


...

}

To see what properties can be used within
the input directive, refer to the
PEM
Watchdog Integration Mo
dules chapter

(on
page
37
).

global

Use this directive only
for File and PEM
Modules.





Configuring PEM Watchdog

22


allow/deny(<service>)

Directive Name

Directive Definition

Scope

Notes

allow(<service>)

allow(<service>@<host>)

allow(
wild*card@wild.car
d
)

deny(<service>)

deny(<service>@<host>)

deny(
wild*card@wild.card
)

Adds filters for included services.

It is useful, for example, when you
want to c
reate two separate check
instances to check all Apache
services registered in PEM in one
check instance, and all ProFTPD
services registered in PEM in another.
In this case, you should:

1

Define service list source, which
will get all services registered in
PEM:


input = pem
-
hosts { module =
PEM }.

2

Define check instance, which will
check Apache services:


instance = Apache check
instance


allow(Apache)

3

Define another check instance,
which will check ProFTPD
services:


instance = ProFTPD check
instanc
e


allow(ProFTPD)

By default, all services are allowed. If
any filter is defined, all the other
services become 'denied'.

It is possible to use wildcards in filters:

*
-

matches 0 or more characters.

.
-

matches any single character.

For example, A* mean
s all services,
which names start with 'A'. A.ache will
match Apache, Abache, Achache, etc.

instance

It makes sense to use
filters only when the list
of services is fetched
from external file or PEM
database. Thus, you can
sort out the services
between ins
tances
easily.





Configuring PEM Watchdog

23


loglevel

Directive Name

Directive Definition

Scope

Notes

loglevel


Defines loglevel in the global scope or for the
single instance. It can take the following values:

0

-

no logging

1

-

errors only

2

-

warnings and errors

3

-

warnings, errors, raw informational messages
(default value)

4

-

debug info + all from 1
-
3 levels

5

-

lots of special information

global

instance

For production systems
it is recommended to
keep loglevel in range 1
-
3.



pulse

Directive Nam
e

Directive Definition

Scope

Notes

pulse


Defines interval between any watchdog actions,
in seconds. A
Watchdog action

is any check
attempt or check retry attempt.

It is very useful in case of Apache web server
checking. If Apache is temporarily down, it
makes
sense to wait for some time before the next
check attempt.

In
module

scope it defines the interval between
sql queries to PEM database (for PEM module)
or two attempts to read the file (for File module).

global

instance

module

The default pulse value

is 0 seconds.



use_dns

Directive Name

Directive Definition

Scope

Notes

use_dns


Possible values:
0
,
1
.

Defines whether hosts should be resolved before
check. If
use_dns

is set to
1
, then while
checking
Apache@myhost.com

Watchdog will try
to resolve
myhost.com

and after the host is
resolved, Watchdog will try to connect via IP (not
via hostname!).

global

instance






Configuring PEM Watchdog

24


use_postponed

Directive Name

Directive Definition

Scope

Notes

use_postponed


Possible values:
0
,
1
.

Defines whether the failed services should be re
-
checked again after all services in the list had
been checked.

global

instance

See also the
retries

(on page
19
) and
use_doublecheck

(on
page
24
) directives.



use_doublecheck

Directive Name

Directive Definition

Scope

Notes

use_doublecheck


Possible values:
0
,
1
.

Defines whether the failed services should be
checked in the next iteration and only after th
e
second service failure in a row, notification will be
sent.

global

instance

See also the
retries

(on page
19
) and
use_postponed

(on
page
24
) directives.



use_suckers

Directive Name

Directive

Definition

Scope

Notes

use_suckers


Possible values:
0
,
1
.

If set to
1
, Watchdog will use special
'sucker'

API
to get check logs for failed service. If set to
0
, no
check logs will be available in Watchdog
notifications.

global

instance

All check modules

that
come with the Watchdog
distribution have sucker
API, but it is not
mandatory to use them.
Therefore, you may get
no check log with third
-
party module even in
case use_suckers is set
to
1
.



notify_always

Directive Name

Directive
Definition

Scope

Notes

notify_always

Possible values:
0
,
1
.

Defines whether Watchdog should send
notification each time service is found dead or
only when the failure was detected for the first
time.

global

instance






Configuring PEM Watchdog

25


notify(<event type>) = <email>

Directive Name

Directive Definition

Scope

Notes

notify(<event
type>) = <email>

Defines the list of e
-
mail addresses to send
notifications to about various event types.

The following event types are recognized:



service_fail
ed

-

the most common event



input_failed

-

input module failed. To use
this event, notify() directive should be
used in the global scope.



facility_failed

-

instance failed to get the
services list from the integration module.

Usage examples:

notify(service_
failed) =
root@localhost,
responsible@example.com


notify(input_failed) =
notsoresponsible@example.net

global

instance

E
-
mails should be
separated with com
mas.



cc(<event type>)
-

<email>

Directive Name

Directive Definition

Scope

Notes

cc(<event type>)
-

<email>

Adds e
-
mail addresses in CC when sending
notifications.

The same event types as in the
notify(<event
type>) = <e
mail> directive

(on page
25
) can be
used.

global

instance

E
-
mails should be
separated with commas.

Mails are not actually
sent to these addresses,
they are just added to
CC field. It can be useful
when there is a need to
"Reply All" o
n Watchdog
notifications.





Configuring PEM Watchdog

26


mailer(<event_type>)

Directive Name

Directive Definition

Scope

Notes

mailer(<event_typ
e>)

Sets the mailer parameters for certain types of
events.

The same event types as in the
notify(<event
type>) =

<email> directive

(on page
25
) can be
used.

Possible values: strings, special format.

Mailer parameters should be specified in form:

property1=value1, property2=value2,
...

Examples:

mailer = remote, host=smtp.host.com,
port=25, user
=watchdog,
password=secret, timeout=15,
from=watchdog@mail.com

or

mailer = local,
from=watchdog@mail.com
, timeout=10

global

instance




template(<event_type>) = /path/to/file

Directive Name

Directive Definition

Scope

Notes

template(<event_t
ype>) =
/path/to/file

Defines the path to the mail template file to be
used for sending Watchdog notifications.

There is a number o
f placeholders that can be
issued to the template:

@@failcount@@

is substituted with failed services
count

@@body@@

is substituted with list of failed
services, hosts and failure reasons

@@start_time@@

and
@@end_time@@

are
substituted with check start ti
me and end time,
respectively.

global

instance

The path to the file
should be absolute.



Configuring PEM Watchdog

27


Here is the example of the mail template:

<html>

<head>

<title>Watchdog notification</title>

</head>

<body>

Hello,<br>

@@failcount@@ problem(s) detected:

<hr><pre>

@@b
ody@@

</pre><hr>

Check was started at <b>@@start_time@@</b>

Check was finished at <b>@@end_time@@</b>

<br>

Check log should be attached if any additional data is available.

</html>

This template would produce the following result in the notification e
-
mail
:

Hello,

1 problem(s) detected:

LoadAverage@192.168.0.0
: Maximum load exceeded.

Check was started at Wed 7
-
Jun
-
06 10:48:13pm Check was finished at Wed 7
-
Jun
-
06
10:48:53pm

Check log should be attached if any

additional data is available.


subject(<event_type>) = <subject_template>

Directive Name

Directive Definition

Scope

Notes

subject(<event_ty
pe>) =
<subject_template
>

Subject template defines the subject of
the
Watchdog notifications.

The following placeholders can be used with this
directive:

@@failcount@@

is substituted with failed services
count

@@start_time@@ and @@end_time@@

are
substituted with check start time and end time,
respectively.

@@failed_serv
ice@@

is substituted to failed
service name, if there is only one failed service,
and to 'multiple services' otherwise.

global

instance




Configuring PEM Watchdog

28


Here is the example of the subject template:

subject(service_failed) = [Local watchdog] @@failed_service@@ failed

dbstore

Directive Name

Directive Definition

Scope

Notes

dbstore

Is is possible to store the check results in the
MySQL database. You will have to create
database manually and initialize database
structure from the file
/usr/local/watchdog/watch
dog.sql

If it is
specified for one instance only, the directive will
override the global value and the instance will
use the locally specified values.

global

instance




Configuring PEM Watchdog

29


Usage examples:

dbstore(db) = test # database name

dbstore(user) = root # user to conne
ct to the database

dbstore(password) = secret # password to connect to the database

dbstore(dbhost) = 193.124.215.6 # MySQL server host

dbstopre(dbport) = 3306 # MySQL server port

The example of the
watchdog.sql

file:

--

Table structure for table `watchdog
`

--


DROP TABLE IF EXISTS watchdog;

CREATE TABLE watchdog (


id int(11) NOT NULL auto_increment,


service_name varchar(128) NOT NULL default '',


host varchar(128) default NULL,


status varchar(16) NOT NULL default '',


message varchar(255) default N
ULL,


check_start datetime NOT NULL default '0000
-
00
-
00 00:00:00',


check_time int(11) NOT NULL default 0,


retries_used int(11) NOT NULL default 0,


PRIMARY KEY (id)

) TYPE=MyISAM;

PEM Watchdog Check Modules

In this s
ection you will get to know the service definitions as they could be included into
any instance to check appropriate services. Each parameter of the check modules is
commented.

Important
: Some check modules like Dotster
-
specific BillingManager checker are
customer
-
specific and can not be used on other installations.

Apache Web Server

service =
Apache@www.web.com

{


port = 80 protocol = http # alternative: https


path = / #
HTTP path to request on server


ok_codes = 200|401 # HTTP codes which are considered
successful

}


ApacheAdmin25
-

PEM UI

Note
: For PEM version 2.4, 2.4 RC 1
-
4 and lower, use ApacheAdmin check module
with the same prop
erties.

service =
ApacheAdmin25@pem.swsoft.com

{


user = admin # User to log in PEM UI


password = setup # Password to log in PEM UI


protocol = https # Protocol: https or http

(https is the
default)

}




Configuring PEM Watchdog

30


Dotster
-
specific BillingManager

service =
bmDoster@bm.swsoft.com

{


user = admin # User to log in to PEM.Billing UI


password
= setup # Password to log in to PEM.Billing UI


protocol = https # http or https; protocol for PEM UI


user_id = 1 # ID for user passwd in "user =" clause


ssh_user = root # user to log in PEM.Billing host via ssh


ssh_p
assword = root # password to log in PEM.Billing host
via ssh

}


BM

Note
: BillingManager check routine can take up to 1 minute, so it's recommended to
set the timeout for BillingManager in the range of 40
-
60 secs.

service =
BillingManager@bm.swsoft.com

{


user = admin # User to log in to PEM.Billing UI


password = setup # Password to log in to PEM.Billing UI


protocol = https # http or https; protocol for PEM.Billing
UI


user_id = 1 # ID for user passwd in "user =" clause


ssh_user = root # user to log in PEM.Billing host via ssh


ssh_password = root # password to log in PEM.Billing host
via ssh

}




Configuring PEM Watchdog

31


PEM ClusteredQ
mail Service

service =
ClusteredQmail@mail.pem.com

{


smtp_server = smtp.mail.pem.com # SMTP server hostname


# default is the same as
checked host,



# mail.pem.com in this
example


pop3_server = pop3.mail.pem.com # POP3 server hostname


# default is the same as
checked host,



# mail.pem.com in this
example


imap_server = imap.mail.pem.com # IMAP server hostname


# default is the same as
checked host,


# mail.pem.com in this
example


# Ports to use for appropriate protocols


# default values are standard


smtp_port = 25


pop3_port = 110


imap_port = 143



# Credentials to use for POP3 and IMAP login


user = watchdog



password = watchdog



# mailbox and domain form watchdog test mailbox


# in the following example, test weatchdog mailbox


# will be
watchdog@swsoft.com


mailbox = watchd
og


domain = swsoft.com



delay = 1 # delay before each mail command, in seconds.


#

default is 1 sec


# Set these parameters if you want to suppress appropriate
checks


# norm
ally they should not be set unless you really do not
want


# to check POP3, SMTP or IMAP


no_pop3 = 1


no_smtp = 1


no_imap = 1

}


Linux File Manager

service =
filemanager@web.ve.com

{


port = 19002 # 19002 is the default port

}




Configuring PEM Watchdog

32


Horde Webmail

service =
Horde@webmail.swsoft.com

{


user =
admin@swsoft.com

# username to log in Horde UI


password = setup # password to log in Horde UI


protocol = https # protocol to use to log in Horde UI


mail_server = mail.swsoft.com # mail server to be used by
Horde.


# default is the same as service host,


# webmail.swsoft.com in this example.

}


Linux LoadAverage

service =
LoadAverage@192.16
8.0.1

{


user = root # user to log in to checked host via SSH


password = secret # password to log in to checked host via
SSH


max_load = 20 # Maximum load_average before sending
notification

}




Configuring PEM Watchdog

33


MS Excha
nge Server

service =
MSExchange@mail.pem.com
{


smtp_server = smtp.mail.pem.com # SMTP server hostname


# default is the same as
checked host,



# mail.pem.com in this
example


pop3_server = pop3.mail.pem.com # POP3 server hostname


# default is the same as
checked host,


# mail.
pem.com in this
example


imap_server = imap.mail.pem.com # IMAP server hostname


# default is the same as
checked host,


# mail.pem.com in this
example


# Ports to use for appropriate protocols


# default values are standard


smtp_port = 25


pop3_port = 110


imap_port = 143



# Credentials to use for POP3 and IMAP login


user = watchdog


passw
ord = watchdog



# mailbox and domain form watchdog test mailbox


# in the following example, test weatchdog mailbox


# will be
watchdog@swsoft.com


mailbox = watchdog



domain = swsoft.com



delay = 1 # delay before each mail command, in seconds.


#

default is 1 sec


# Set these parameters if you want to suppress appropriate
checks


# normally they s
hould not be set unless you really do not
want


# to check POP3, SMTP or IMAP


no_pop3 = 1


no_smtp = 1


no_imap = 1

}


Microsoft IIS Server

service =
msii
s@host.com

{


# IIS serves as both http and ftp server.


# by default, port values are standard:


http_port = 80


ftp_port = 21

}




Configuring PEM Watchdog

34


Microsoft SQL Server

service =
MsSQL@mysql.server.ip

{


port = 3306 # default value is 3306


user = root # user to log in MsSQL DB


password = '' # password to log in MsSQL DB


db = mysql # DB name to connect to


query = 'SHOW
TABLES' # Test query to execute.


# test query must return non
-
empty
result

}


MySQL Server

service =
MySQL@mysql.server.ip

{


port = 3306 # default v
alue is 3306


user = root # user to log in MySQL DB


password = '' # password to log in MySQL DB


db = mysql # DB name to connect to


query = 'SHOW TABLES' # Test query to execute.


# test
query must return non
-
empty
result


# 'SHOW TABLES' is default

}


Outlook Web Access

service =
OWA@owa.webinterface.host.com

{


# Windows

credentials to log in OWA


domain = windomain


user = watchdog


password = secret


# Protocol to use for web interface. Default is 'https'.


protocol = https

}


PEM Accounts (Depreca
ted)

Important
: It makes sense to use this directive for PEM 2.4 RC1 only, when PEM
crashed down if the accounts limit per one VPS was exceeded. In the latter versions,
this bug was fixed.

This checker monitors the number of accounts in PEM and reports whe
n there are not
enough web VPSs.

service =
PEMAccounts@pem.db.host.ip

{


user = plesk # user to log in PEM DB


password = '' # password to log in PEM DB


db = plesk # PEM DB



port = 5432 # PEM DB port


maximum_accounts = 100 # maximum accounts per VPS

}




Configuring PEM Watchdog

35


PEM Service Controllers

This checker sends notifications when one or more Service Controllers are down.

service =
PEMSC@pem.db.host.ip

{


user = plesk # user to log in to PEM DB


password = '' # password to log in to PEM DB


db = plesk # PEM DB


port = 5432 # PEM DB port

}


PEM pg_autovacuum service

service =
pg_autovacuum@pem.db.host.ip

{


# credentials to log in to PEM DB host via SSH


user = root


password = secret

}


PostgreSQL Server

service =
PgSQL@mysql.server.ip

{


port = 5432 # default value is 5432


user = postgres # user to log in to PgSQL DB


password = '' # password to log in to PgSQL D
B


db = template1 # DB name to connect to


# Test query to execute, it must return non
-
empty result #
this is the default query:


query = SELECT datdba FROM pg_database WHERE
datname='template1'

}


Ping

This module pi
ngs remote hosts.

service = ping@localhost {


protocol = icmp # alternative: tcp

}


Plesk Server Administrator

This module checks Plesk Admin UI availability.

service =
Plesk@h
ost.com

{


port = 8443 # default port for Plesk admin UI is 8443


path = /index.php3 # path to Plesk login form, default value
is


# /index.php3

}




Configuring PEM Watchdog

36


ProFTPD Service

service =
Proftpd@ftp.store.com

{


# by default, port is 21


port = 21 # alias for port property:


ftp_port = 21

}


Qmail Service (Deprecated)

This checker is deprecated. Yo
u should use Clustered Qmail checker.

service =
Qmail@mail.pem.com

{


smtp_server = smtp.mail.pem.com # SMTP server hostname


# default is the same as
checked ho
st,


# mail.pem.com in this
example


pop3_server = pop3.mail.pem.com # POP3 server hostname


# default is the same as
checked host,



# mail.pem.com in this
example


imap_server = imap.mail.pem.com # IMAP server hostname


# default is the same as
checked host,


# mail.pem.com in this

example


# Ports to use for appropriate protocols


# default values are standard


smtp_port = 25


pop3_port = 110


imap_port = 143



# Credentials to use for POP3 and IMAP login


user = watch
dog


password = watchdog



# mailbox and domain form watchdog test mailbox


# in the following example, test weatchdog mailbox


# will be
watchdog@swsoft.com


mailbox
= watchdog


domain = swsoft.com



delay = 1 # delay before each mail command, in seconds.


#

default is 1 sec


# Set these parameters if you want to suppress appropriate
checks



# normally they should not be set unless you really do not
want


# to check POP3, SMTP or IMAP


no_pop3 = 1


no_smtp = 1


no_imap = 1

}




Configuring PEM Watchdog

37


WebComStore Service

service =
WebComStore@store.domain.com

# No properties available for
this # checker

PEM Watchdog Integration Modules

PEM Watchdog has two integration modules. Their function is to fetch services list

either from file or from PEM database.

File

You may want to keep the services list in a separate file. Then, you have to install
watchdog
-
input
-
File
-
*.rpm

and specify the input directive with the File module
in the Watchdog configuration file.

input = any_name {


# Mandatory line for any integration module


module = File


# Mandatory parameter is File:


file = /etc/watchdog.include # path to file with service
definitions.


retries = 3 # 3 times to read specifi
ed file, after that
trigger


# input_failed
event


period = 100 # reread file each 100 seconds


timeout = 3 # fail if file was not read/parsed in 3 seconds

}


The list of services in a separ
ate file should be specified in the same format as they are
described in this guide. For example,

service = Apache@localhost

service =
ping@192.168.0.1

{protocol = icmp}




Configuring PEM Watchdog

38


PEM

Using this module, you don't n
eed to define in the configuration file all services you
wish to check in PEM, they all will be checked by default. Though you may use filters
(allow/deny) not to check some of them.

The list of services that PEM Module checks automatically:

Apache, CQMail
, File Manager, MS IIS, MS SQL, MySQL, PEM Service Controllers,
PostgreSQL, Proftpd. If you need to check some other services, you have to add them
into the configuration file manually using the service directive.

input = any_name {


# Mandatory lin
e for any integration module


module = PEM


# Credentials to log into PEM DB


host = 10.10.0.1


port = 5432


db = plesk


user = plesk


password = ''



# Mailbox to test Qmail/Clustered Qmail


# If not specified, Qmail/Clustered Qmail services


# won't be fetched from PEM DB


test_mailbox =
watchdog@swsoft.com




# From field for test watchdog messages which are used to t
est
PEM mail


# servers from =
watchdog@swsoft.com



# Set this properties for all Apache services


# fetched from PEM DB.


# Useful for Parked domain servers,


# because th
ey return 404 on HTTP GET /


apache_ok_codes = 200|401|302


apache_path = /index.html

}







Configuring PEM Watchdog

39


Configuration File Sample

In general, the typical Watchdog config file looks like this:


#

# Begin /etc/watchdog.co
nf


#

# Global settings: timeouts, retries, mail settings

timeout(Apache) = 10 # seconds

timeout(Proftpd) = 5 # seconds

retries(Apache) = 3

default_retries = 1


use_suckers = 1

use_dns = 0

use_doublecheck = 1

use_postponed = 0


mailer(service_failed) = rem
ote, host=local.smtp.relay, port=25,
timeout=10

notify(service_failed) =
john@example.com
,
bigboss@example.com


loglevel = 2 # only errors and warnings


#

# Now, define i
ntegration modules

input = file1 {


# this module will simply provide service lists from file


# /etc/watchdog.d/include1.wd, which should contain only
service


# definitions in the same format as in watchdog.conf


module = Fil
e


file = /etc/watchdog.d/include1.txt

}


# This module will add all services from PEM

input = pem_services {


module = PEM


db = plesk


user = plesk


host = 127.0.0.1 }


#

# Finally, define check instances

instance = ch
eck Apache


# Local override: It will override global retries


retries(Apache) = 5


allow(Apache@*) # Allows to check only Apache within this
instance


# and ignores all other service types



# that may be included from integration
modules


include(file1) # Include services defined in
/etc/watchdog.d/include1.txt


include(pem_services) # Include all services registered in PEM



Configuring PEM Watchdog

4
0



service = Apache@localhost { port = 8443 } #

Explicit service
definition

instance = check everything except Apache


allow(*@*)


deny(Apache@*)


include(file1)


include(pem_services)



# Explicit service declarations


service = ping@localhost


service
=
ping@192.168.0.1


# ... may be next instance, etc..


#

# End of /etc/watchdog.conf

Calculating Checking Time

Let's assume the following situation. You have 10 hosts with Apache web
servers you
wish to monitor with the help of PEM Watchdog.

These servers have IP addresses in the range 192.168.0.1
-

192.168.0.10

You create a check instances:

instance = check Apache


service =
Apache@192.
168.0.1

{ port = 8443 }


service =
Apache@192.168.0.2

{ port = 8443 }


service =
Apache@192.168.0.3

{ port = 8443 }


service =
Apache@192.168.0.4

{ port = 8443 }


service =
Apache@192.168.0.5

{ port = 8443 }


service =
Apache@192.168.0.6

{ port = 8443 }



service =
Apache@192.168.0.7

{ port = 8443 }


service =
Apache@192.168.0.8

{ port = 8443 }


service =
Apache@192.
168.0.9

{ port = 8443 }


service =
Apache@192.168.0.10

{ port = 8443 }

Define the checking time settings and the number of retries.

timeout = 10 # 10 seconds to wait for each Apache server reply

retries(A
pache) = 3 # Watchdog should query Apache on host 3 times if
it's failed.

pulse = 1 # One second of idle time between two retries or checks of
neighbouring Apaches.

use_postponed = 1 # Recheck again the failed Apache server after the
rest of the servers in

the list are checked.

use_doublecheck = 1 # Recheck the service on the next check iteration
and only then consider it failed and send notification.

period = 30 # The idle time between two checking iterations of a
single instance

Now let's assume that Apac
he on host 192.168.0.5 has failed. When do you receive
notification about it, taking into account the above settings?

1

Watchdog notices that Apache on host 192.168.0.5 is down. It will wait for 10
seconds if Apache would send a reply.
timeout = 10



Configuring PEM Watchdog

41


2

Watchdog
waits for 1 more second before starting another check attempt.
pulse =
1

3

Watchdog undertakes the second and the third attempt.
retries = 3

4

Watchdog leaves the failed server and continues checking the other Apaches in a
row. After the last Apache (192.168.0
.10) is checked, Watchdog returns to the
failed Apache and starts checking it again for 3 times.
use_postponed = 1
,
retries = 3
.

5

If nothing is changed with the Apache state, Watchdog goes on checking the other
Apaches. When the last Apache in a row is chec
ked, Watchdog starts checking all
the servers from the beginning.
use_doublecheck = 1
.

6

When it reaches the failed Apache in this iteration and checks it again for 3 times, it
sends a notification at last.

Now let's count how much time has left since Apache

had failed.

(10+1) x 3 = 33
-

3 retries

(10+1) x 5 = 55
-

checking other servers in a row

(10+1) x 3 = 33
-

postponed check

(10+1) x 5 = 99
-

checking other servers

30
-

idle time before the second iteration

(10+1) x 4 = 44 beginning new iteration

(10+1
) x 3 = 33
-

checking the failed server again in a new iteration

33+55+33+55+30+44+33 = 283 sec

Thus, if one out of ten servers is down, you will receive notification in 283 seconds.



42


These operations include starting, stopping and upgrading Watchdog.

In This Chapter

Upgrading PEM Watchdog

................................
................................
................

42

Starting/Stopping Watchd
og

................................
................................
..............

42



Upgrading PEM Watchdog

If you need to upgrade your current Watchdog version to the newer one, follow this
instruction:

1

Upload watchdog distribution to the Watchdog host.

2

Execute


# rpm
-
Fhv dis
t/*.rpm

3

Restore old config file:


# mv
-
f /etc/watchdog.conf.rpmsav /etc/watchdog.conf

4

Restart watchdog:


# service watchdog restart

Starting/Stopping Watchdog

Watchdog runs as standard Linux daemon. Thus, watchdog can
be started with
following command:

# service watchdog start

Watchdog can be stooped with following command:

# service watchdog stop

C
H A P T E R
5

General Operations on PEM Watchdog



43


#

# • 17

A

About This Guide • 5

About Watchdog • 8

allow/deny(<service>) • 21

A
pache Web Server • 28

ApacheAdmin25
-

PEM UI • 28

B

BM • 28

C

Calculating Checking Time • 39

cc(<event type>)
-

<email> • 24

Configuration File Sample • 37

Configuring PEM Watchdog • 15

D

dbstore • 27

default_timeout • 18

Documentation Conventions • 6

Dots
ter
-
specific BillingManager • 28

F

Feedback • 7

File • 35

G

General Operations on PEM Watchdog • 41

H

Horde Webmail • 30

I

include(<name>) • 20

input = <name> • 20

Installation Instructions • 14

Installing PEM Watchdog • 11

instance [ = name ] • 17

Introdu
ction • 8

L

Linux File Manager • 29

Linux LoadAverage • 30

loglevel • 22

M

mailer(<event_type>) • 25

Microsoft IIS Server • 31

Microsoft SQL Server • 32

MS Exchange Server • 31

MySQL Server • 32

N

notify(<event type>) = <email> • 24

notify_always • 23

O

Or
ganization of This Guide • 6

Outlook Web Access • 32

P

PEM • 36

PEM Accounts (Deprecated) • 32

PEM ClusteredQmail Service • 29

PEM pg_autovacuum service • 33

PEM Service Controllers • 33

PEM Watchdog Check Modules • 27

PEM Watchdog Configuration File Struc
ture •
16

PEM Watchdog Directives • 16

PEM Watchdog Distribution and Packaging •
13

PEM Watchdog Installation Requirements and
Prerequisites • 12

PEM Watchdog Integration Modules • 35

period • 17

Ping • 33

Plesk Server Administrator • 33

PostgreSQL Server
• 33

Preface • 5

ProFTPD Service • 34

pulse • 22

Q

Qmail Service (Deprecated) • 34

R

References • 10

Index



Index

44


retries(<service>) = <number> • 19

S

service = <name>@<host> • 19

Services to Monitor • 10

Starting/Stopping Watchdog • 41

subject(<event_type>) = <subject
_template> •
26

T

template(<event_type>) = /path/to/file • 25

timeout(<service>) = <timeout> • 18

Typographical Conventions • 6

U

Upgrading PEM Watchdog • 41

use_dns • 22

use_doublecheck • 23

use_postponed • 23

use_suckers • 23

W

WebComStore Service • 35

W
ho Should Read This Guide • 5

Why You Should Use Watchdog • 9