SNMP AGENT AND MANAGER DEVELOPMENT

pheasantarrogantSoftware and s/w Development

Aug 15, 2012 (4 years and 10 months ago)

348 views



1


SNMP

AGENT AND MANAGER
DEVELOPMENT





Team Members:

1)

Arnav Aggarwal

2)

Brian

Walshe

3)

Paul Gildea

4)

Paul
Soprovici

5)

Peter Hannon













2




Table of Contents



1.

Team members and responsibilities

................................
................................
........................

3

2.

Motivation and Objective

................................
................................
................................
........

3

3.

State of the art (in relevant area)

................................
................................
.............................

4

4.

Design

&
Implementation

................................
................................
................................
.....

13

5
.

Evaluation

................................
................................
................................
..............................

21

6
.

Conclusion

................................
................................
................................
.............................

26

7
.

Appendix

................................
................................
................................
...............................

27




3


1.

Team members and responsibilities


Arnav Aggarwal
-

He is
a

writer and researcher for the team. He has

read lots of papers
on SNMP thus
understand the various

technologies in which SNMP protocol is still used.

Brian


Brian is
an architect/d
eveloper for our team. His main responsibilities were to
design the
SNMP
Manager.

Paul


Paul is the manager of the
team,

he has contributed by managing and keeping
track o
f all the developments of the team. Also Paul had been

responsible for preparing
the s
lides for the presentations
.

Paul Gildea
-

He is a

writer and researcher for the team. He has
also read many

papers on
SNMP giving him

understand
ing of

various technologie
s
in which SNMP protocol is
used.

Peter Hannon
-

Peter is also an architect/
developer for our team. His main
responsibilities were to design the SNMP Agent and interfac
e it with the manager.

2.

Motivation and
Objective


Our primary objective is to build a
simple SNMP agent and M
anager

to monitor,
measure various attributes of the web server application

using the Java technology
.

We
connect the manager agent to the web server to poll for some information from time to
time. We have also programmed our Agent f
or sending traps to the manager.







4



3.

State of the art (in relevant area)




We studied several papers to understan
d the problem at hand to find
efficient and
scalable solution

to the problem. In the process we studied about various technologies
,
network management architectures

that are being used in monitoring, managing and
maintaining the remote devices in heterogeneous
and homogeneous
networks.
At present
there are two majorly used protocols in the field of network management of them one is
t
he CMIP/CMIS proposed by OSI and the other
is SNMP (Simple Network Manager
protocol) forwarded by IETF.



Our study was primarily related to the technologies u
sing the Simple Network
Manager

Protocol to
manage and control and observe the behaviour

of the web
applications servers using a SNMP agent associated/ controlled by a m
anager allowing
human interface
.



SNMP


The SNMP agent on the managed device serves to provide access to data
(managed object) stored on the managed device. The SNMP manager
can monitor and
control the managed device through the agent

by simple SET and GET commands
.



In complex

network systems

there is a need to
decentralise

management

functions
performed in the network enabling better
efficiency

and resource u
tilisation
.

It
is

also
clear that in
very large networks it is simply not possible to mo
nitor many sub
-
networks
over a w
ide area network without causing a bottleneck at a centralized manager. This
problem can only be handled properly by employing several distributed mid
-
level
managers or immobile data collectors. SNMPv2 attempted to introduce the manager
-
to


5


manager MIB’s for such purposes but failed due to the reasons such as added complexity
etc.
The SNMP is an accepted and widely supported standard and offers a low
-
cost,
small
-
footprint
solution for elements with scarce resources. There are many applications
out there in practice which are built on top of SNMP.


Distributed SNMP


The support for d
ecentralised was also built into the earlier versions of the SNMP
which is the version1 in the form of remote monitoring system (RMON). The SNMP
version1 lacked support for the manager to manager interface (to provide hierarchical
architecture) which was l
ater provided by the version2 of the SNMP protocol.
The
SNMPv3 management framework makes it possible to develop a set of distributed
entities which are composed of several interacting modules.

The

distributed
SNMP
provides enables creation of entities
which can act as both the agents and the manager






In today’s

management systems

there may be

several agents which implement new
protocols such as Distributed Protocol Interface

(DPI)

or the more advanced Agent
extensibility Protocol

(AEP)

than usi
ng the old legacy protocols such as the SNMP.
One
spe
cific system that we studied
made

use of

more complex protocols on the mobile
agents

to monitor and manage
SNMP agents in various network topologies

discussed
below
[3]
:
-


1.

A mobile agent
arrive
s at the

node
to

access attributes, variables and the current
state data

from the resident SNMP agent’s MIB.
Th
e mobile agent

has managerial
capabilities to act as a manager and
issue SNMP request packets to th
e resident
SNMP agent. The
mobile agent

can also
open
a socket connection with the
resident MIB without being present at the resident MIB

but
this will not be
a good
practise
due to

security reasons.
Such an approach has been

used

in [1] and in [2
]


6


using readily available Java SNMP classes
. The key
advantages are that there is
no need to
modify the

SNMP agent to provide access to i
ts MIB.


2.

A mobile agent

arrive
s at the node and has

the capability of extending the

resident
agent’s
MIB with certain meta
-
variables that it brings with it.
The SNMP DPI
p
rotocol
and the
AgentX protocol
are
designed to allow
a sub
-
agent process to
enhance the

SNMP age
nt by first registering with it

and communicating with it

using the specified protocol.


3.


A mobile agent, having acquired information during its migratory activity, may

arrive at the affected node( with lower performance etc.) or elsewhere in the
network. It then sends a
defined SNMP

trap to a remote SNMP manager. Both the
AgentX and the DPI p
rotocol are

designed to enable the SNMP agent to receive
such a trap from a subagent/

mobile agent and transmit it to an SNMP manager.


4.

It is possible

though less like that

some entity send
s

an

SNMP trap t
o a mobile
agent which acts as an

SNMP manager.

This assumes that the sender can actually
locate the mobile "manager" using the

mobile agent environment’s location
service.


Our aim is

to implement a simple SNMP

based

agent

which
retrieves,
monitors and
controls the

parameters of the Web application
server internally
( through the SNMP
messages)

rather than externally
as it
s
aves us crucial traffic, since monitored information
can be analyzed, filtered, summarized inside the server and it also makes the

management
fault tolerant. For example if the web

application server is being managed externally
using the Http port, and there is a service failure on the same port, the manager will
neither be able to GET or SET any commands on the web application server.





7



The enhanced SNMPv2 provides a manager
-
t
o
-
manager (M2M) MIB to support
hierarchical management architecture.

The services an SOA
-
compliant distributed system
is built from are expected to support introspection, be discoverable, loosely coupled and
platform
-
, location
-

and transport
-
independent.
There are a number of middleware
platforms that help to meet these requirements. A good example of such platform for
implementing a monitoring system are Web Services (WS) providing access to the
system’s functionality built with a Java Management Extensio
n (JMX) support.


We studied a number of syst
ems before settling in on
the

Apache Tomcat
Application server as it provides an interface to monitor using
JMX technology
. JMX
provides an interface between the Agent application and the web application server
allowing the allowing the agent to implement the managerial functions such as retrieval
and setting of information on some conditions.




The JMX architecture consists of three levels: instrumentation, agent and

distributed services. JMX provides

interface
s and services adequate to the requirements
of monitoring and management

systems
.

JMX provides access to detailed information
about the program through objects called MBeans. Java provides a set of standard
MBeans which can be used to access information on

the Java Virtual Machine itself
-

memory usage, threads, etc
-

and these MBeans can be extended to provide more specific
details about the state of the program itself.

In the project our developers make use of the
SNMP4J java Library to create the agents and the manager.
E
xtensive use
has been made
of the Agent Toolkit Java Edition

6.0
.



Web Server


Once we had settled on using the JMX
technology to interface the agent web
application server we looked at various web servers on which we could run our
developed system. The two main contenders for this were Apache HTTP Server and


8


Apache Tomcat

(Tomcat for short)

due to functionality availab
le
, open source status

and
the range of support provided to these web servers.

This is shown by the massive amount
of use that Apache HTTP Server (Apache for short)

receives, being the first web server
to handle over one
hundred
million web sites
.

Apache and Tomcat are open source
software;
they

are
dev
eloped and maintained by a

wide, decentralized,
community of
peers with the support of the Apache Software Foundation

(ASF).
These peers
collaborate on forwarding the functionality of the web server
s via a consensus based
software
development process.
Main contributors to a project are chosen as managers for
a project with t
he ASF
granting membership to these
volunteers who have contributed to
the
various
Apache projects.


The advantages and disadvantages of each of the web server technologies
were

investigated to gather information on which technology would be best suited to our
project.

The main features which get covered by one or both when looking at security and
dynamic

content are: basic access authentication, digest access authentication, https,
virtual hosting, CGI, fast CGI, servlet, SSI, administration console and IPv6 etc. Both
run on an array of operating systems/platforms
. Apache is seen to be faster than Tomca
t
when handling static pages, as wel
l as being more robust,
suppor
ting CGI scripts, PHP,
Perl etc,

however it is important to use the right tool for the job at hand. Tomcat is used
for servlets/JSP

which Apache does not support
.
Tomcat

can also handle html

pages

but

Apache implements

it better.

Summing up the web servers available the conclusion that
Tomcat should be used was reached.
As Tomcat provides an extensive library of custom
MBeans that can be used to monitor and manage it, it was

decided that the
best way to
proceed, would be to create an agent in Java that would run as its own process and collect
information about the Tomcat server it was monitoring through JMX calls.

This
along
with ease of implementation and integration with JMX

and the
apt conf
iguration settings
and administration tools available

made Tomcat a good choice
.

The n
ewest version of
Tomcat is 6.x which uses the latest servlet and JSP versions.
Tomcat is

made up of
several components
; Coyote, Catalina, and Jesper 2.



9


Coyote

Coyote is a

HTTP connecter which supports HTTP 1.1 for the web server. After
listening for a connection and receiving one on a TCP port Coyote is responsible for
sending the data to the Tomcat Engine where it is processed and a response is sent back
to the client fro
m which the request originated.


Catalina

Catalina is the servlet container used by Tomcat
.
A servlet container is a
compiled, executable program, it is used for loading, initialising and running the servlets.
It acts as an interface for the web server and the
servlets. Upon the arrival of a request,
Catalina maps the request to a servlet and sen
ds

the request to the servlet

in question
.
This

servlet proc
esses the request and returns an appropriate

response.
Next t
he container
translates the
response and sends it back to the w
eb server.

Large numbers of requests are
catered for and the container/o
bjects in the container are multithreaded. Each thread is
created and managed by the container and requests are handled concurrently.


Jasper

2

Jasper is Tomcat's
JavaServer Pages(
JSP
)

Engine.
The more recent versions of
Tomcat
use Jasper 2 which has more
functionality included such as Background JSP
compilation and JSP Tag library pooling etc. Jasper 2 follows
Sun Microsystems's

JavaServer Pages 2.0 specification. It parses JSP files which are then

compile
d
into Java
code as servlets
.







10


AdventNet WebNMS
SNMP Agent Toolkit Java Edition 6



AdventNet WebNMS SNMP Agent Toolkit Java Edition 6 is a tool for network
management by use of the industry established standard management technologies such
as SNMP, JMX, and TL1. It is java based. This product is used
for development of our
SNMP agent. This software can generate stub code for an Agent from a MIB. The agent
produced is capable of sending and receiving messages and must only be instrumented to
so that these messages contain the relevant values.

It is free

for evaluation for forty five
days and so suited our purposes for this project. It also was chosen as it caters well to our
specific needs by offering a wide selection of development tools, framework, and API’s.
It caters for multiple platforms: Windows,
Linux Solaris etc.


The complete agent development process is handled. This includes everything
from building the agent to testing of the agent. AdventNet WebNMS SNMP Agent
Toolkit Java Edition 6 is built on SNMP, TL1, HTML, RMI, CORBA, JMX, Java Beans,

XML, JFC, JDBC, etc and so the necessary functionality for the system was embedded in
the software. Bundled with the software is a command line interface as well as a
graphical user interface for agent design, development and testing.[4]



Fig. 1. Develop
ment of Advent SNMP Agent[5]








11


Features and Benefits


1.

Master Subagent Architecture.

For multiple
SNMP agents who are located at
different IP addresses, a proxy agent must manage the distributed network of
SNMP agents. Management efficiency in distributed applications is increased and
this is defined by the master
-
subagent architecture. Here the SNMP p
roxy acts as
the master agent and the managed SNMP agents are registered as sub
-
agents to it.


2.

Trap notifications.
Automatic alarms sent to manager when unwanted events
occur.



3.

Rule Engine.

Logic rules must be configured for the systems and this is catered
for without needing to edit code.



4.

Support.
Atomicity, IPv6, transport protocols and persistent storage supported.[5]



As making calls to JXM are relatively complicated, involving remote

procedure
calls, and the need to deal with potential errors in unmarshalling the objects returned, we
used the Adapter design pattern to simplify things. A class called TomcatInfo was created
to act as the adapter, and allows us to instrument the agent wi
th the minimal amount of
change to the generated code. How this class interacts with the system is shown in a class
diagram further on in the document.








12




The TomcatInfo class is responsible for managing the RMX connection to the
Tomcat instance. It
provides the information that we require from the server as simple
method calls. The methods of TomcatInfo handle unmartial
l
ing the values returned by the
RMI calls and return them as primitive types where appropriate. They do not handle
errors, throwing t
hem, and they provide values "as is" converting to appropriate units is
left to the instrumentation code. (For example, the method getUptime() returns its value
in milliseconds, and this must be converted to hundredths of a second in the
instrumentation)



The TomcatInfo class also provides information that is not directly available
from the Tomcat MBeans, but which can be calculated from polling. This includes
calculating the number of requests per second the server is receiving, and telling the
agent to s
end traps when this goes above a defined level.


SNMP4J/Swing Libraries/NetBeans

To develop the manager the following software was used:




NetBeans

o

This is a Java IDE which was used for writing the java code.

o

Written using the NetBeans Platform.

o

Open
source software platform for developing Java applications.

o

All modules needed for development included.

o

Additional language support added with plugins.

o

Chosen for ease of use, robustness and functionality.






13




Swing Libraries

o

Used for the development of the

graphical user interface of the manager.

o

Bundled with NetBeans.





SNMP4J Library

o

Used for SNMP development with Java.

o

Used for developing agents and managers.

o

Downloaded separately.

o

Open source.


4.

Design

& Implementation



At the outset of the project a design had to be created which would cover the
objectives outlined in our project. The main areas that were to be worked on were
build
ing a

SNMP agent and Manager to monitor

this. These would be used to
administrate a web ser
ver along with monitoring this web server. To do this

a variety of
attributes
should be measured.
The agent
’s

purpose is to

monit
or the status of the web
server and the manager decides what to do with the information.


To show that the web
server is being

managed correctly a few simple goals had to be obtained:



a.

Basic Communication

i.

Get and Set Attributes.

ii.

Restart Server.


b.

Polling

i.

Dealing With Expected Events
.





14


c.

Traps Directed Polling.

i.

Reacting and Analyzing Unexpected Events.

ii.

Denial of Service

etc.



These were the basic conditions that must be met in the project, with some
possible examples mentioned. An incremental development process was best; get
something simple working and then work on including more functionality. The first step
was to get some
basic communication between the agent and the manager
working.
Some simple get and set requests were designed and implemented.

Next polling was
thought about and building on from that the next step was to think about trap directed
polling.

Design and functionality of traps and how there were managed were thought
about before being designed. Following on from this development
of get
/set and trap
directed polling
functionalit
y

was successfully implemented. The particular examples
chosen to s
how our server being managed will be discussed in more detail

along with the
components of the system

in this section.



System D
esigned

Using the Following C
omponents:


1.

SNMP Manager

(MS)


The management station has a set of management applications for
data
analysis, fault recovery etc; an interface by which the network manager
monitors and controls the network element (Web Application server in our
case).


2.

Management Agent
/
SNMP Agent


The management agent acts as an interface between the manager and the

network elements
through the internet.

The management agent responds to
requests for information from management station and also performs specified


15


action as requested by the manager station. The Agent may also be
responsible for sending Traps to the Man
ager when specific predefined
condition is achieved.



When designing the agent there was two main options, the agent could be
implemented as an extension to the web server, or we could have it running as its own
process. Running as its own process would b
e more robust, but would require it to
communicate with the web server through some means.

The
Management Agent

was
designed

using the WEBNMS
package;

the
MIB

was designed

in accordance with the
given specifications and then created java code using t
he
given package. The

Agent then
had to be

the
interfaced

with the Web server application server using JMX.





3.

Java Management Extensions (JMX)


JMX

is a
Java

technology that supplies tools for managing and monitoring
applications
, system objects, devices and
service oriented networks. The

resources are represented by objects called MBeans
(Managed Bean)
.

It
provides
tools for building distributed, Web
-
based and dynamic solutions for
managing and monitoring devices, applications, a
nd service
-
driven networks.
By design, this standard is suitable for adapting to legacy systems which
implement SNMP for providing
managem
e
n
t functions. The JMX technology
provides us

the means to create
Java agents,

implement
interface between the
agent a
nd
manager

(
at
a
remote location
)
, and easily integrate these solutions
into existing management and monitoring systems.



We use JMX which is supported by Apache Tomcat6 to integrate with the
agent existing at the same location. The Agent registers the MBeans on the


16


MBean server and allows passing (GET, SET) values on the web application
server.


4.

Network Element

(Tomcat Se
rver)


Apache Tomcat is a
n

open implementation of Java Server and Java Server
Technologies. We implement the managerial functions to monitor the Apache
Tomcat Application Server, JMX provides us interface with the Tomcat as
Tomcat is implemented in Java.


Some of the functions that we implement are restarting the Tomcat server,
getting number of connections to
it, sending traps if the number
of
connections exceed
ed

the given specified limit etc.


5.

Management Information Base (MIB)


MIB is a type of
database used for managing the devices in a communications
network. It comprises of collection of objects in a database used to manage
entities (network elements such as routers and switches etc.) in a network. The
database is hierarchical (tree
-
structured
) and entries are addressed through object
Identifiers.


6.

Simple Network Manager Protocol (SNMP)


It is a protocol through which the management station and the managed
entities exchange control and management information. SNMP is the
most widely
accepted

protocol for sending messages to the network elements to be managed.
We use both SNMP version one and SNMP version to in implementing our
designs.




17



Figure

2
: SNMP based Management Figure
.


In the above figure we can see the outline of the system that
was designed and
implemented.




Management user launches the manager application.



Logic settings can be set in the manager applications graphical user interface.




An example of this is the threshold of requests per second that the server will
accept before sending a trap. The manager application will be looked at in detail
later in the report.




Messages between the agent and manager applications are sent over UDP(U
ser
Datagram Protocol
) as this is the standard and is a simple connectionless protocol,
with SNMP being designed to be as simple as possible.



Messages are received on the agent side
by the SNMP Agent Process. This
process encapsulates the SNMP Agent, the M
anagement Information Base and
any MBean information retrieved from the MBean Server.



The Agent application acts as an interface between the manager and the
rest of
its
process and the Web Server Process

and is managed by the management
information base
.



It responds to requests from the management application
.



When a trap is triggered the Agent
sends a message
to the management
application o
ver UDP.



18




If the manager application requires information the agent is responsible for
retrieving the MBean from the
MBean Server.



JMX allows the SNMP Agent Process to connect with the Web Server Process.



This Process consists of the MBean Server and the Apache Tomcat Web server
which is to be managed.



Class Diagrams


These

show the final design and implementation of the manager class and agent
class systems. The
relationship

between each component along with the information each
component contains is show in the following two class diagrams:



SNMP Manager Class (Figure 3)



SNMP Agent Class (Figure 4)




19


Class
Diagram
s

showing SNMP Manager Class
and SNMP Agent

Class.



20





21


5.

Evaluation



The project overall has been a success. The initial outline of the project and what
was expected from a system managing a server has been
completed.

The objective
was to create a

SNMP agent and Manager to monitor

and manage a web server and
this has been implemented

effectively
.

Each feature that was added to the system was
tested as it was introduced. This started off with basic communicati
on between the
MIB
, the agent application and the manager application.



As

the application became more complex this scaled up to testing management of
a Tomcat server with the mana
gement graphical user interface shown in the figure
below.

Various test cas
es were carried out to ensure each part of the system worked
as it should.

To test these currently one can enter various inputs into the management
application system, such as
entering text into a text field, selecting values from a drop
down menu or press
ing a JButton on the GUI. The following
fields can be modified:





Polling Interval

o

Various time options



Restart Tomcat

o

Restarts web server



Threshold/Set Threshold

o

Enter amount of requests server will accept per second without
yielding a trap.



HTTP Interface

o

Enable/Disable HTTP Interface



Get Server Info

o

Update server information fields



22




Get Servlet Info

o

Update servlet information fields



Clear Traps

o

Reset Traps field



Figure

5
: Web server management application graphical user interface.





23


Polling Interval

This is how frequently the systems will poll for traps. This can be configured
from the drop down menu to pick a desired time. The default value is a trap will be sent
every ten seconds
.


Restart Tomcat


This JButton tests the restarting
of the Tomcat web server when a restart
command is sent.


Threshold/Set Threshold


Within this interface the functionality to set the threshold for connections to the
server per second was implemented.

To test that poll directed traps are working the
f
ollowing simple process was used:


1.

Download
Python.

2.

Start Tomcat, the Agent and the Manager.

3.

On the Manager set the connections per second threshold to

30

.

4.

After hitting the "Set Th
reshold" button a

dialog saying "The SNMP SET was
successful".

5.


If this was not shown there was
a configuration problem with the Agent,
Manager or Tomcat.

6.

Open IDLE (the Python GUI).

7.

T
ype in and run the following Python script:


from urllib import urlopen

import time





24


for i in range(1000):


urlopen("http://localhost:8080")


time.sleep(0.01)



This code generates a loop that connects multiple time
s

to the Tomcat host and
port
. Since the connections to the Tomcat server are over
the
30 per second

threshold

a
trap is sent to the manager
.
This shows

that the implemented trap directed polling
functionality
has been correctly

put in
place.

The code example above generates roughly
50 connections per second.



By altering the sleep time in the code or the threshold value in the GUI it can be
s
een that if the number of connections to the server is below the threshold value a trap
will not be sent and if the number of connections to the server per second is above the
threshold value a trap will be sent. This testing is also used in conjunction wi
th the
polling interval drop down menu to make sure that a trap is only sent once for each
specified amount of time.



HTTP Interface


This JButton

is used for enabling and disabling the HTTP interface. Upon
pressing the JButton the text will change to ‘Enabled’ or ‘Disabled’ respectively

showing
the user the current status of the HTTP Interface
.

By visiting the interface it was easy to
see if the te
st was successful by checking if the interface was active or not.

Get Server Info

This retrieves the current information from the server. This includes:




The last time and date that the Get Server info JButton was pressed.



Host name of the
web
server.



Port number of the
web
server.



25




Up time of the
web
server in days, hours, minutes and seconds.



CPU usage of the web server; measured in percentage CPU of the system



Memory usage of the web server.


Get Servlet Info

This retrieves the current information
from the servlet. This includes:




The last time and date that the Get Servlet info JButton was pressed.



A list of servlet names.



Request count

-

amount of requests that a particular servlet name has
received
.

In the screenshot the localhost has received 20
03 requests
from the generated Python script.


Clear Traps


The Clear Trap JButton

clears all the information from the traps field of the GUI.
This includes:



Date the traps were received.



Times the traps were received at



The OID’s



The trap error message. I
n this case the threshold of connections to
the web server was breached.






26



6.

Conclusion



This section should draw conclusions and indicate any caveats about the

prototype/project execution.

This should be about 1 or 2 pages or so (can be longer if
required)

7.

Appendix

Put a URL to a file where the source code of your project is zipped

Put in any screen shots or anything else you believe relevant here



REFERENCES



[1]

Zapf M., Hermann K., and Geihs K., "Decentralized SNMP Management with Mobile Agents."
In
Proceedings of IM ’99, Boston, May 1999.

[2]

Simoes P., Silva L., Boavida F., "Integrating SNMP into a Mobile Agents Infrastructure" Proc. of
IEEE/IFIP DSOM '99, Zurich Oct.1999.

[3]

B. Pagurek, Y. Wang, and T. White Department of Systems and Computer Engineeri
ng Carleton
University, Ottawa Ontario Canada

“Integration of Mobile agents with SNMP: Why and How”.

[4]

http://www.webnms.com/javaagent/index.html
,

(
2009
).


WebNMS SNMP Agent Toolkit Java Edition 6
”.

ZOHO Corp.

[5]

http://www.webnms.com/javaagent/snmp
-
agent.html
,

(
2009
).

“SNMP Agent”.

ZOHO Corp.