Performing Problem Determination in the WebSphere Application ...

pityingmushroomInternet και Εφαρμογές Web

7 Αυγ 2012 (πριν από 4 χρόνια και 11 μήνες)

285 εμφανίσεις


Presentation subtitle:

20pt Arial Regular,

green R223 | G255 | B102


Recommended

maximum length: 2 lines


Confidentiality/date line: 13pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Disclaimer information may also be appear in this area. Place
flush left, aligned at bottom, 8
-
10pt Arial Regular, white


IBM logo must not
be moved, added
to, or altered in any
way.

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements


Presentation title:

28pt Arial Regular, white


Recommended maximum
length: 2 lines


Group name:

17pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

© 2002 IBM Corporation

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

April 2003 | IBM Academy
Proactive Problem Prediction, Avoidance, and Diagnosis Conference


For client
presentations,
client’s logo may
go in this area

Performing Problem Determination in the
WebSphere Application Server Environment


The Art and Science of Problem Determination

Title slide


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

2

Agenda


The philosophy of problem determination




What needs to happen in problem determination for it to work



Agenda slide


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

3

Introduction


The Typical High Volume Environment

Z900 Sysplex
Distributor

Sun 6800’s (20 CPU’s)

2 domains, 249 JVMs

Across 20 servers

Data Center 1

WAS

Web servers

DB2/UD
B

WAS

Web Servers

CoreBuilder
9000

Backend
Mainframe
s

CoreBuilder
9000

Failover
Backend

WWW

Reaching millions of TBA Customers!

Data Center 2

14 CICS

7 CICs WOR

DB2/UD
B

19 AOR

14 CICS

7 CICs WOR

19 AOR

DB2


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

4

Philosophy 101

From Merriam
-
Webster's Dictionary


Philosophy:



2 a : pursuit of wisdom ... c : an analysis of the grounds of and
concepts expressing fundamental beliefs



3 b : a theory underlying or regarding a sphere of activity or thought
<the philosophy of war> <philosophy of science>



The Philosophy of Problem Determination

Basic text slide


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

5

Philosophy 101

What is the objective? What are we trying to achieve?



The ability to conduct post
-
mortem analysis


can the problem be recreated?


did we collect enough data to understand the issue?


did we perturb significantly?


an iterative process


Heisenbugs


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

6

Philosophy 101

What is the objective? What are we trying to achieve?



Understanding the problem



avoid speculation



look at the data



point the finger at yourself first


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

7

Team work is the key to successful resolution
of any problem. Remember that IBM Support
is part of your team too.

Philosophy 101

Achieving the Objective

Working together


Operations and Developers need to cooperate


Control changes to source and patches to the system.


Involve IBM Support immediately


IBM Support personnel have extensive experience and are
actually good at debugging “over the phone”



It could be a WebSphere Application Server problem. That is what
efixes are for.

Highlight:

18pt Arial Italic,

green R223 | G255 | B102

Text slide with

subheading and highlight

Subheading:

20pt Arial Italics

green R223 | G255 | B102


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

8

A typical environment is not trivial requiring
teams of experts

Philosophy 101

Expertise


2 : the skill of an expert

Do research / background investigation
-

nobody can possibly know
everything


Java



WebSphere Application Server Infrastructure



The Frontend
-

ip sprayers, reverse proxy, web servers, plugin



The Backend
-

databases, MQ, CTG



Hardware and networks

Highlight:

18pt Arial Italic,

green R223 | G255 | B102

Text slide with

subheading and highlight

Subheading:

20pt Arial Italics

green R223 | G255 | B102


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

9

Tools are essential for conducting problem
determination. While one can do PD without
such tools it makes the process PAINFUL

Philosophy 101

Arsenal


b: a collection of weapons


HyperProbe


w3.research.ibm.com/hyperprobe



WebSphere Application Server tracing facility (data flood)



Introscope



www.wilytech.com



Jinsight


www.alphaworks.ibm.com




PerfAnal


developer.java.sun.com



aktools, LoadRunner, Rational, JMeter, network sniffers, etc



Other tools from 3
rd

parties & IBM being released more often lately

Highlight:

18pt Arial Italic,

green R223 | G255 | B102

Text slide with

subheading and highlight

Subheading:

20pt Arial Italics

green R223 | G255 | B102


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

10

There is a difference between load and stress

Philosophy 101

Load and Stress


Load
-

b : the demand on the operating resources of a system


Number of users attaching to the system



Stress


STRAIN, PRESSURE <the environment is under stress to
the point of collapse>


Numerous users doing work at a higher than normal rate

Highlight:

18pt Arial Italic,

green R223 | G255 | B102

Text slide with

subheading and highlight

Subheading:

20pt Arial Italics

green R223 | G255 | B102


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

11

Philosophy 101


Scientific Method

What is load and stress testing for?


Measurable and
reproducible
results!


1.
Peak number of users


2.
Where are the application or
infrastructure bottlenecks


3.
Expectable response times and
responses per second


4.
Capacity Planning

Customers actively reproduce the results of
IBM tests in order to VALIDATE their own
environments


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

12

Anticipate that you
will

have problems and
give yourself time to resolve them

Philosophy 101

Anticipate


1 : to give advance thought, discussion or treatment to


Load and stress test from Day One



Determine the application's performance base lines



Client response times



Application server side response times



Load test every release



Compare against the base line

Highlight:

18pt Arial Italic,

green R223 | G255 | B102

Text slide with

subheading and highlight

Subheading:

20pt Arial Italics

green R223 | G255 | B102


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

13

What development practices work in your
environment? Can you afford to skip any one
of these steps?

Philosophy 101

Development Practices


Unit test everything

www.junit.org

httpunit.sorceforge.net



Embrace change


www.extremeprogramming.org



Write effective use cases


www.usecases.org/



Conduct code reviews



www.infotectonica.com

Highlight:

18pt Arial Italic,

green R223 | G255 | B102

Text slide with

subheading and highlight

Subheading:

20pt Arial Italics

green R223 | G255 | B102


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

14

Change Control
Development
Functional
Verification

Testing
System Test
Load Test
Stress Test
Integration
Testing
CHANGE CONTROL PROCESS
only infrastructure changes
can return to integration testing

IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

15

Philosophy 101

Production Monitoring



Collect necessary data



Minimal impact



Precision


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

16

Philosophy 101


Open PMRs immediately


Call back on a PMR


2 hour time limit for sev 1 PMRs


page the "duty manager"


call back every day


Check status of PMR online


Send every file support requests


does not matter if you think they need the file or not ... they WILL
NOT look at your problem until all requested documentation has
arrived


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

17


Test the fix


verify that the fix(es) supplied solve the problem


re
-
open PMRs if prematurely closed or the problem reappears after
the close


PMRs live for only x number of days


save text of PMRs and place into a problem determination
subdirectory everyone can access


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

18

Table of contents

The philosophy of problem determination


What needs to happen in problem determination for it to work



IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

19

Introduction


The Typical High Volume Environment

Z900 Sysplex
Distributor

Sun 6800’s (20 CPU’s)

2 domains, 249 JVMs

Across 20 servers

Data Center 1

WAS

Web servers

DB2/UD
B

WAS

Web Servers

CoreBuilder
9000

Backend
Mainframe
s

CoreBuilder
9000

Failover
Backend

WWW

Reaching millions of TBA Customers!

Data Center 2

14 CICS

7 CICs WOR

DB2/UD
B

19 AOR

14 CICS

7 CICs WOR

19 AOR

DB2

DB2


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

20

Components of a WebSphere Application Server

CPU

(AIX, Solaris, Windows)

Component interactions

Production JVM

(AIX, AS400, HP
-
UX, Linux, Solaris, Unix, Windows, OS/390, z/OS
)

Application Server

J2EE Application



EJB





Servlet


EJB

CICS

Transaction

Gateway

MQSeries

Connector

JDBC Driver

Thread

Pool

EJB

Pools

JDBC

Pools

Mainframe

Back
-
end systems

Database

Memory
Management

J2EE Services

File and
Network I/O

Customer

Transactions

J2EE components

Back
-
end connectors


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

21

Making Problem Determination Work


None of this is simple.



Several tiers adds complexity



Multiple network components and software integration points




IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

22

Making Problem Determination Work


None of this is simple.



Several tiers adds complexity



Multiple network components and software integration points



Start by building a decision tree




IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

23

The Decision Tree

Root Problem #1

Root Problem n

Root Problem #2

. . .


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

24

The Decision Tree

Root Problem #1

Root Problem n

Root Problem #2

. . .


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

25

Several disparate components means that
someone or something needs to understand
how they work and how they interact with
each other

Components of the decision tree

WebSphere Application Server


Java Virtual Machine



Data Source / Connection Pooling



Sessions and Persistence



Web Server Plug
-
in



EJB Container

Highlight:

18pt Arial Italic,

green R223 | G255 | B102

Text slide with

subheading and highlight

Subheading:

20pt Arial Italics

green R223 | G255 | B102


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

26

Components of a Decision Tree


JVM Arguments

Heap Size

Generational Garbage Collection

WebSphere Classpath

Application's Classpath


V5 vs. v4 vs. v3.5.x migration issues



Max OSE Connections

Number of Web servers

Number of worker threads to process incoming requests



Classloader


Synchronization


Threads


Servlet Context


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

27

And even more detail…


Data Source minimum and maximum


Orphan Timeout


Transaction Timeout


StaleConnectionObjectException


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

28

Sessions and Persistence


Best practices generally ignored. Typical problems include:


Large sessions sizes


Failing to setAttribute data after it changes



Problems that result


Slow(er) response times


Missing session data / fail over appears to be defective


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

29

WebSphere plug
-
in


Not well understood



Commonly people “hack” the plug
-
in to get “desired” effect which is
error prone and results in more problems



Little more than tracing is available to debug the plug
-
in


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

30

EJBs


Entity bean vs. self
-
persistence


CMP vs BMP


Stateful Session Beans vs. Stateless Session Beans


Persisting state


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

31

Monitor and understand each piece of the
network and what it is doing otherwise it is
impossible to verify conclusions

Infrastructure

Network Components


Reverse Proxies
have swallowed requests



Firewalls
rules have kicked in and negatively impacted performance



Routers
flow control and routes causing problems



IP Sprayers
misconfigured led to inefficient processing



Highlight:

18pt Arial Italic,

green R223 | G255 | B102

Text slide with

subheading and highlight

Subheading:

20pt Arial Italics

green R223 | G255 | B102


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

32

Each of these problems was VERIFIED by
monitoring the network and watching the
interaction between components

Infrastructure

Integration Points


Database
a disk failure was causing negative performance for
random INSERTs, another case the HACMP solution failed to detect
the disk failure and copied the failure. There was no backup!!



CTG
on the mainframe was misconfigured resulting in fewer
available connections than were thought



DB2 Connect
misconfigured led to poor response times



Improperly written stored procedures
resulted in deadlock conditions



Web servers
misconfigured had 1000:1 Web server to application
server ratio resulting in severely overwhelmed the network


Highlight:

18pt Arial Italic,

green R223 | G255 | B102

Text slide with

subheading and highlight

Subheading:

20pt Arial Italics

green R223 | G255 | B102


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

33

Overlook NOTHING


Everything we just discussed
can contribute to negative
performance



Ruling out a component,
hardware or software, without
data to support the decision is
a common mistake (i.e.
guessing)



Use a decision tree

Text slide with

large image


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

34

Start by building a decision tree from
experience. This is where IBM can help a
customer.

What does a decision tree look like?

A Primer


Identifies a problem



Identify the components that participate in the problem

Identify component configuration

Verify by collecting data/examining logs that configuration is correct

User load can also be considered a component



Identify the data, if any, that can participate in the problem

User data or action initiated

Triggers or other non
-
user initiated interactions



Identify the possible fixes.

There can be multiple possible solutions to any problem.

Some solutions result in a previously undiscovered problem


Highlight:

18pt Arial Italic,

green R223 | G255 | B102

Text slide with

subheading and highlight

Subheading:

20pt Arial Italics

green R223 | G255 | B102


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

35

Decision trees provide a documented solution
to known problems.

Decision Tree

Why do we need them?


Unfamiliarity with the environment



Staff turnover



Defines known problems and provides known solutions



Is easily expanded over time to accommodate new components




Highlight:

18pt Arial Italic,

green R223 | G255 | B102

Text slide with

subheading and highlight

Subheading:

20pt Arial Italics

green R223 | G255 | B102


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

36

Slow response time is a common problem.
Examine each component and its relationship
with its peers.

What does a decision tree look like?

An Example

SLOW RESPONSE TIME
(negative user experience)



Identify the components that participate in the problem

All network components
(IP sprayer, Web servers, routers, firewalls)

J2EE Services



session persistence, RMI, XML, Web Services, etc

Application Code


requires monitoring

Backend resources


databases, CICS transactions, MQ Series, etc

What was the user load during the negative event?



Identify the data, if any, that can participate in the problem

What action did the user(s) initiate?

What data was associated with that action?

Examine network activity



Identify the possible fixes.

Correct network component fault(s) or configurations

Correct application code

Correct backend resource contention(s)


Highlight:

18pt Arial Italic,

green R223 | G255 | B102

Text slide with

subheading and highlight

Subheading:

20pt Arial Italics

green R223 | G255 | B102


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

37

Each component in the decision tree has its
own low level detail that describes the data to
collect, analyze and remedy to apply

What does a decision tree look like?

From High Level to Detail

SLOW RESPONSE TIME
(negative user experience)



Component:
Application Code


Data Collection

Monitoring data (is the application itself “slow”)

If the application is slow then



identify components of application that are slow



collect data using
kill
-
3


Analyze collected data and determine what fix to apply

Apply fix, examine environment, restart data collection if no improvement
OR THINGS ARE WORSE!

Highlight:

18pt Arial Italic,

green R223 | G255 | B102

Text slide with

subheading and highlight

Subheading:

20pt Arial Italics

green R223 | G255 | B102


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

38

Systematic Problem Determination


Is an iterative approach



Generally not the same
problem twice



Requires perseverance and
management support ($$)



Crucial for every production
environment



A problem can strike at any
moment, usually at the worst
moment and with code that has
“been running for a while”

Text slide with

large image


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

39

Problems are generally self inflicted


Poorly written code due to

Lack of adherence to best practices

No code reviews conducted

Little to no testing of the application



Misunderstanding of configuration parameters and not testing
changes



No change control process



Ignorance is not bliss

Text slide with inset image

and two
-
line heading


IBM logo must not
be moved, added
to, or altered in

any way.





Background should

not be modified,
except for quotes,
which use gray
background.


Title/subtitle/confidentiality line: 10pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Slide heading:

28pt Arial Regular, light
blue R204 | G204 | B255


Maximum length: 2 lines








Slide body:

18pt Arial Regular, white


Square bullet color:

green R223 | G255 | B102


Recommended maximum
text length: 5 principal
points


Group name:

14pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

Performing Problem Determination in the WebSphere Application Server Environment

© 2003 IBM Corporation

Optional slide number:

10pt Arial Bold, white

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements

40

Bureaucracy is the best policy


Procedures, while intimidating, are the best thing you can do to your
environment and helps enforce change control policies


Restrict developer access even in their own environments

Strict change control procedures

Strict QA procedures that include code reviews and testing/SLA

Stringent load and stress testing

Text slide with inset image

and two
-
line heading


Presentation subtitle:

20pt Arial Regular,

green R223 | G255 | B102


Recommended

maximum length: 2 lines


Confidentiality/date line: 13pt Arial Regular, white

Maximum length: 1 line


Information separated by vertical strokes,

with two spaces on either side


Disclaimer information may also be appear in this area. Place
flush left, aligned at bottom, 8
-
10pt Arial Regular, white


IBM logo must not
be moved, added
to, or altered in any
way.

Indications in green = Live content


Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional elements


Presentation title:

28pt Arial Regular, white


Recommended maximum
length: 2 lines


Group name:

17pt Arial Regular, white


Maximum length: 1 line


Copyright: 10pt Arial

Regular, white

© 2002 IBM Corporation

Template release: Oct 02

For the latest, go to http://w3.ibm.com/ibm/presentations

IBM SWG Software Services for WebSphere

April 2003 | IBM Academy
Proactive Problem Prediction, Avoidance, and Diagnosis Conference


For client
presentations,
client’s logo may
go in this area

The End

We put the fear in WebSphere

Closing slide