Title : MQ Network Mapper

backdamagedInternet and Web Development

Jul 30, 2012 (4 years and 8 months ago)

370 views

COMP6701


eScience Project Proposal


Title : MQ Network Mapper



Student :

Tim Goodwill


-

tim.goodwill@eds.com



Academic supervisor:
Peter Strazdins

-

peter@cs.anu.edu.au



Client:
Trung Nguyen (EDS), MQ Administrator, Customs Connect Facility


trung.nguyen@eds.com



Period : this semester



Keywords:
MQ
,
MQ Queue Manager
,
MQ Channel, MQ Network, Programmable Command Formats
(PCF).



Report:



Page
2

of
18

Abstract


The IBM WebSphere Message Queueing
application (WMQ) is designed to provide a provide
a robust, rapid deployment multi
-
server application environment. However while the WM
Q
application base is relatively invisible to developers and application support staff, it’s
administration can be complex.

IBM supply with their product a heavy
-
duty enterprise explorer application that allows the
administrator to undertake any and all MQ

deployment and administration tasks. However
the
configuration and
use of this application requires a good deal of administration
experience and knowledge of the existing WMQ network
, and navigation is cumbersome.

The purpose of this project is to create
a lightweight application that will cater for the
everyday administration activities of the MQ administrator, and make these low
-
risk
administration tasks available to non
-
specialist staff via an auto
-
discovering, intuitively
designed interface.

Page
3

of
18


Contents


Introduction

................................
................................
................................
................................
..............................

4

Messaging

................................
................................
................................
................................
.........................

4

MQ Components

................................
................................
................................
................................
...............

5

Queue Manager

................................
................................
................................
................................
.............

5

Queue

................................
................................
................................
................................
............................

5

Channel

................................
................................
................................
................................
.........................

5

Process

................................
................................
................................
................................
..........................

5

Cluster

................................
................................
................................
................................
...........................

5

The Problem

................................
................................
................................
................................
.............................

5

Purpose

................................
................................
................................
................................
................................
.....

6

Project Goals:

................................
................................
................................
................................
.......................

6

Portability

................................
................................
................................
................................
.........................

6

‘Out
-
of the box’ functionality

................................
................................
................................
............................

6

Auto Discovery

................................
................................
................................
................................
.................

6

Easily accessible diagnostic information and high level admin commands

................................
.....................

6

Project Outcomes

................................
................................
................................
................................
......................

7

System design

................................
................................
................................
................................
...........................

8

Discovery algorithm

................................
................................
................................
................................
.............

8

Flow chart
-

Discoveryalgorithm:

................................
................................
................................
.....................

9

Class Process Overview


Discovery:Class Process Overview


Queue InterActions:

................................
.........

10

Cl
ass Process Overview


Queue InterActions:

................................
................................
................................
.....

11

Class Process Overview


Object Update:

................................
................................
................................
.............

11

User Guide

................................
................................
................................
................................
..............................

12

Project Timetable

................................
................................
................................
................................
....................

17

References:

................................
................................
................................
................................
.............................

18

More Information:

................................
................................
................................
................................
..............

18


Page
4

of
18

Introduction

Messaging

Websphere MQ is IBM’s middleware for commercial messaging and queuing


a method of
program
-
to
-
program communication.
Middleware messaging enables programs to
communicate with each other across a network of unlike components, such
as processors,
subsystems, operating systems and communication protocols. Websphere MQ programs
use a consistent application program interface (API) across all platforms.
Websphere MQ
is

used in every major industry in many countries around the world
, and
is the
pseudo
-
standard

for program
-
to
-
program communication
in the Financial industry
.


Middleware messaging

speeds implementation of distributed applications by simplifying
application development and test.
Programs within an

application communicate by w
riting
and retrieving application
-
specific data (messages) to/from

queues, without having a
private, dedicated, logical connection to link them.

Each remote system has its own queue
manager with its own queues. The queue manager transfers messages to other

queue
managers via channels using existing network facilities, such as TCP/IP, SNA, SPX.




Messaging means that programs communicate with each other by sending data in
messages and

not by calling each other directly.

Programs

communicating through

queues
need not be executed concurrently.





The programmer does not have to worry about the target program being busy or not
available. He or she isn’t even concerned about the server being down or havin
g no
connection to it. The programmer sends messages to a queue that is associated with an
application; and the application may or may not be available at the time of the request. the
Queue Manager takes care of the transport to the target application and
even starts it, if
necessary. If the target program is not available, the messages stay in a queue and get
processed later.

Page
5

of
18

MQ Components

Queue Manager

A Queue Manager (QMgr) manages application access to queues, channels and processes. A qmgr is created w
ith
the crtmqm command. Once created, queues, channels and processes can be defined within the qmgr. There can be
multiple qmgrs in a server.

Queue

A queue represents a storage area for pieces of data, called messages. There are several queue types, namely

Qremote, etc…..

Channel

A channel provides a way for application to access a qmgr remotely or for qmgrs to communicate with other qmgrs.

Process

Process definitions provide details of those applications that will be triggered by events on an associated qu
eue.

Cluster

A Cluster allows QMgrs to be logically grouped together regardless of OS and location. Queues defined to a
Cluster are automatically available to all QMgrs in the Cluster without defining transmission queues, remote
queues or channels between
each QMgr.


The Problem

T
he administration of distributed networks can become very complex. The problems of
administration will continue to grow as networks increase in size and complexity.






















Examples of administration specific to mess
aging and queuing include:



Resource management.

o

For example, queue creation and deletion.



Performance monitoring.

o

For example, maximum queue depth or message rate.



Control.

o

For example, tuning queue parameters such as maximum queue depth,
maximum messa
ge length, and enabling and disabling queues.



Message routing.



Definition of alternative routes through a network.

QMGR
13
QMGR
12
QMGR
7
QMGR
8
QMGR
1
QMGR
2
QMGR
3
QMGR
4
QMGR
11
QMGR
10
QMGR
5
QMGR
6
QMGR
9
Cluster

Page
6

of
18


Purpose

WebSphere MQ PCF commands can be used to simplify queue manager administration and
other network administration. PCF commands allow

you to use a single
external
application
to perform network administration from a single queue manager within the network.


The
aim of this project

is to design a Java program to make use of the power of Websphere
MQ PCF commands to
traverse the MQ networ
k, retrieve data remotely from connected
queue managers, and
create a visual representation of all queue managers connected to
the

MQ network.

Simple administration tools would be made available via the GUI to display Queue Manager,
MQ Cluster and Queue s
tatus information, and to issue basic administration commands,
such as put disabling queues.


It is intended that the software would be portable, and operate without modification on any
IBM Websphere MQ network.


P
roject
Goals:

Portability

a.

The product will

be written in Java (1.4.2), and is to run on any java
-
enabled
operating system including Windows, Linux and Unix.


Out
-
of the box
’ functionality

b.

The product is to run on any machine or platform on which an MQ Client with a
properly configured connection t
o a production WMQ server is installed.
The port
number of a running Queue Manager must be known to the program user. These
are

the only pre
-
requisite
s
, or configuration required.

Auto Discovery

c.

The product will
discover the network from one known node


r
equiring the user
to be aware of the server name (or IP address) running MQ server, the port upon
which a running Queue Manager is listening, and the name of an open MQ client
channel.

Easily accessible diagnostic information and high level admin commands

d.

A QManager will be represented graphically as a node with ‘leaves’, representing
the
managed
object classes
Queues
,
and

Channels
.

e.

Selecting one of these ‘leaves’ wil
l display a list of the object’s attributes
.

f.

The program should be able to pull down status

information for a

selected

object.
For Queues, this will include Description, Put Status (enabled/disabled)

QDepth,
and

Maximum Depth. For Channels, this will include Description, current Status
(running/stopped/retryin
g etc…), Type (Sender/Reciever)

and

connection name
(Target IP and port)
.

g.

The program should be able to issue high
-
level admin commands
for a

selected

object
to allow common administration tasks to be carried out via the console.

For a Queue, these options will include
‘Refresh’



to obtai
n updated attribute
status
,
‘Put Enable/Disable’, ‘
Put Messages
’ and ‘Browse

Messages

. For Channels,
commands will limited to ‘Stop/Start’
.


Page
7

of
18

Project Outcomes

The data object compiled by the Java application may be large and complex, and it would
require

a flexible drawing object to represent it elegantly.

I have selected the ‘HyperTree’ to model the data. HyperTree is an open
-
source Java
library that graphically represents a tree structure on an elliptical hyperbolic ‘canvas’.


Essentially, a central ar
ea of the tree provides focus and detail, while connected objects
trail off hyperbolically toward the edge of the ellipse. Primary relationships to a selected
object are in focus, and while more remote relationships are evident, the details do not
clutter
the screen unless they are selected or dragged toward the
c
ent
re
.

Th
e

following initial
mock
-
up
produced at the beginning of the project
include
d

a working
hyperTree, which is a visualisation of, and an abstraction from a nodal
-
type data object.
The associ
ated Java classes require
d

some modification to display information in a manner
appropriate to this project.

























The

mock
-
up was created using the open
-
source Net
-
Beans

IDE.
At the initial stages of the
project, it was decided that

basing the project on the ‘Netbeans Platform’ could provide
much of the enhanced formatting and windowing required of the project. The Netbeans
Platform based project is shown for comparison.

Clus
ter1

Page
8

of
18

System design

The interactions between this java application a
nd the MQ WMQ product are made possible
using IBM supplied
‘Programmable Command Format’ (PCF) java classes


‘MS0B
-

MQSeries® Java classes for PCF’. These classes give the programmer access to the
‘Message Que
u
ing Administration Interface’ (MQAI), essen
tially allowing command
references in code to be use meaningful names that are interpreted run
-
time into the non
-
text PCF commands that are transmitted to MQ admin queues.

A discovery algorithm will handle the order and manner of server connection via the
com.ibm.mq library, and once connected, the PCF commands will be fed into the
com.ibm.mq.pcf library, and data requested via the MQ command facility.


Discovery

algorithm

h.

The program will connect via the WMQ server to the Qmgr listening on a pre
-
determined

port.

i.

As the program first connects to any Queue Manager (QMgr), it will collect, via
PCF commands, a standard set of data
-

both managed object information
(queues, channels), and details of any QMgrs and Clusters to which it is
connected.

j.

If the QMgr i
s part of a cluster, the program will query the QMGR to discover
whether it is host to a FULL or PARTIAL cluster repository.

k.

If the QMgr is host to a full repository, the program will query the QMgr for
details of the cluster, including all connected QMgrs
.

l.

If the QMgr is host to a partial repository, the program will identify a QMgr that
hosts a full repository, connect to that QMgr, collect the standard set of
information, as well as details of the cluster, including all connected QMgrs

m.

Having collected d
etails of QMgrs connected to the cluster, the program will then
need to connect to each of these newly identified QMgrs in turn to retrieve the
standard data set, and so
-
on.

n.

Clusters will, as a rule, be investigated before directly connected QMgrs. Cluster
s
represent primary relationships. Once investigation of clustered connections has
been exhausted, undiscovered directly connected QMgrs will be mapped, until
and unless further clusters are discovered, and so on…

o.

Once the network has been mapped, the data

will be serialised to disk, and re
-
loaded upon the next restart. The reason for this is that the discovery process for
a large network may take several minutes. The user will start with a pre
-
loaded
network structure, sanitized of warning data, that will
be updated as the new
data is retrieved.


In this way, the program is to build up and store a data object representing a
comprehensive map of the entire MQ network


without any prior configuration or
knowledge of the network.


Page
9

of
18

Flow chart
-

Discoveryalgorit
hm:

MQ Client installed
on workstation
Collect standard data set
(
via PCF
):
IP address of WMQ server
known and reachable
Port No
.
of QMgr on
WMQ server known
managed object
information
(
queues
,
channels
)
Connected Clusters
,
Full repository
location
Directly connected
QMgrs
1
2
Cluster currenltly
being mapped
?
Member of
Cluster
?
Known
,
unexplored
cluster
(
full repository
)
?
Stored data
object
?
Unexplored
clustered QMgr
?
Known directly
connected
QMgr
?
Serialise data
,
end Process
3
Yes
No
Yes
No
No
Yes
No
Yes
No
Yes
No
Yes
PCF Commands
Page
10

of
18


Class Process Overview


Discovery
:
NodeExplorer

TopComponent

DiscoveryAction

USER


ConnectPoint

Discovery

QMgrDiscovery

ConnectPoint

ConnectPoint

MQObject

ObjectRepositor
y

HyperView

HTree

TopComponent

IBM Libraries

Page
11

of
18

Class Process Overview


Queue InterActions:

















Class Process Overview


Object Update:
















NodeExplorer

TopComponent

Browse

TopComponent

USER

Q
ueueUtils

ConnectPoint

ConnectPoint

MQObject

ObjectRepository

NodeExplore
r

TopComponent

Properties

USER

QMgrUtils

ConnectPoint

ConnectPoint

MQObject

ObjectRepository

HTree

TopComponent

IBM Libraries

IBM Libraries

Page
12

of
18

User Guide

Upon opening, the Explorer view presents a preformatted Connect
-
Point for connection to the MQ Network. The
connection details are presented in the Properties dialog, and should be amended as appropriate.




Connect to the chosen QMgr using t
he right
-
click context item “Discover…”, or the File Menu item “Discovery”.


Page
13

of
18

The application will proceed to map the network, populating the explorer and creating a ‘Hypertree’ visualisation
as show:



The number and type of objects displayed in the Hyp
erTree are controlled via the check
-
boxes at the top of the
window:


Page
14

of
18

Nodes are selected vie either the explorer or the HyperTree, and will display a property sheet for each object
selected:



Selecting a Channel or Queue object will display both editab
le and non
-
editable properties. Editable properties may
be changed by selecting the value in the properties window, and either overtyping, or selecting a checkbox as
appropriate.


Page
15

of
18

Connect
-
Points may be added by selecting “Add New Connect Point” via the
root context.

The Connect
-
Point
will need to be edited, and discovered in the usual way. A Connect
-
Point may be deleted or
renamed at any time.







Selecting a Qu
eue Managers “Queues” or “Channels” list will provide a status list for each class of obj
ect in the
properties window:


Page
16

of
18

Queue messaging options are
accessed by right clicking a queue object in the explorer, or opening the Browser
window from the “Window” Menu.

From this window, it is possible to Browse messages, Get messages (Browse + remove
) or Put messages to a queue.



Page
17

of
18

Project Timetable

Week

Dates

Notes

4

16 Mar

Modelling and Learning of Chosen Technics Phase

9

4 May

Modelling phase completed.

10

11 May

Tidy Comments and
Debugging

11

18 May

Mid
-
project result (prototype)

13

1 Jun

Im
plementation
t
o Supervisors.

14

15

Jun

Project presentations.

16

2
8

Jun

Final Reports


Page
18

of
18

Discussion, Conclusions and Future Work

//
There is a lot of work to be done.

//
I need to understand the HyperTree Java library
and its classes fairly well
in order t
o
//
make important modifications. Documentation is sparse, and classes not well annoted, so
//
this will be a challenge.

The modifications I desire to make include:

//
1. Extra, strait, unobtrusive (grey?) node to node lines underneath the tree to depict
//
additional
connection relationship.

//
2. Run
-
time add/delete/recolour node without resetting the view.

//
3.
Possibly e
xpand the las
t nodule to reveal/hide leaves (
actual
//
queues/channels/triggers).

//
4. Possibly display
different shape
s

to
represent Cl
usters, QMgrs, Object Classes and
//
Managed Objects.

//
I need to learn the

Message Queui
ng Administration Interface’

belonging to the PCF Java
//
classes. I have downloaded excellent IBM resources on the subject.

//
At this stage, in order to keep the inter
face clean and simple I intent to make
//
administration commands, such as Put Disable/Enable, Get object information, Browse
//
Queue etc… available via a context menu.


References:

More Information
:



“Introduction to Messaging and Queuing”

at:
http://www.elink.ibmlink.ibm.com/public/applications/publications/cgibin/pbi.cgi?CTY=US&FNC=SRX&
PBL=GC33
-
0805
-
01




Programming guides
: “
WebS
phere MQ Using Java
” “WebSphere MQ Application
Programming Guide”, at:
http://www
-
306.ibm.com/software/integration/wmq/library/library53.html



References
:



HyperTree J
ava Library



http://www.loria.fr/~bouthier/



IBM Redbooks



http://www.redbooks.ibm.com/



IBM Software
-

WebSphere MQ


Library



http://www
-
306.ibm.com/software/integration/wmq/library/library53.html



IBM Software
-
WebSphere MQ
-

Support



http://www
-
30
6.ibm.com/software/integration/wmq/library/library53.html



IBM Websphere MQ Iformation Center



http://publib.boulder.ibm.com/infocenter/wmqv6/v6r0/index.jsp