Performance Measurement Tools for Justice Information Technology Projects

spotlessstareSecurity

Nov 29, 2013 (3 years and 9 months ago)

60 views

Performance Measurement Tools for Justice
Information Technology Projects

Illinois/ Cook County

Focus Group Experience

Using “scenario” approach to reach
agreement and define performance


Bring stakeholders together to develop a
criminal justice scenario


Reach consensus on the desired state of
integration


Define the current state of integration
(baseline)


Quantify gap between current state and
desired state


Define desired outcomes


Develop objectives and performance
measures

IIJIS Strategic Plan: Existing
performance measures


2.1.4.

Number of mechanisms identified to reduce paper
-
based processes

5.2.4.

Number of stakeholders adopting functional
standards promoting interoperability by September 2003

5.2.5.

Number of stakeholder agencies recognized through
the certification program

6.1.3.

Percent of recommended infrastructure solutions
implemented

6.3.3.

Year 1: Number of users utilizing the resource
center

6.3.4.

Year 2: Percent increase of users utilizing the
resource center


IIJIS Strategic Plan: Existing
performance measures


6.4.5.

Year 2: Percent increase of stakeholders adopting
enterprise
-
wide disaster recovery plans

6.4.6.

Year 1: Number of stakeholders performing disaster
recovery tests

6.4.7.

Year 2: Percent increase of stakeholders
performing disaster recovery tests.

7.3.1.

Number of research projects on biometric
technological solutions completed by September 2004

7.3.2.

Number of research projects on costs and benefits
of biometrics completed by September 2004


Illinois/Cook County Focus Group
on Performance Measures



Sixth in a series of facilitated workgroups by
CSLJ to refine & validate PMs


All participants had contributed to development
of state and/or Cook County strategic plans


Participants were provided information on PM
Project and the PM toolkit


Participants applied toolkit to IIJIS Strategic
Plan; then answered questions and shared their
reactions with CSLJ staff


PARTICIPANT CONCLUSIONS:
Performance measures cannot be
objectively selected; stakeholders
must reach consensus



Stakeholders, depending on their role within the
justice enterprise, are likely to have strongly
divergent views


Higher
-
level goals and outcomes will usually be
easier to agree upon than specific performance
measures


Performance measure selection, however, is not
entirely subjective, and some measures are
likely to be agreed upon by all


PARTICIPANT CONCLUSIONS:
It is nearly impossible to
definitively establish causal
linkages to public safety outcomes


Outcomes become more diffuse the further out
you get from actual processes


Improved information sharing is only one of a
number of factors affecting public safety


A single performance measure may be
inadequate to indicate that a system is working
(or not)


A carefully selected family of measures is
preferable, and should be thoroughly tested
before being deployed.


PARTICIPANT CONCLUSIONS:
Public sector outcome measures
are driven by different factors than
for the private sector


Greater integration success by private industry due to
clearer lines of accountability


While the private sector is driven by the bottom line,
government is driven by the competing needs of many
agencies with differing missions and purposes; lines of
accountability are therefore more diffuse.


Government is accountable not only to the governor,
legislature, and public officials, but to the public as well


Concern that the public believes that the justice system
works as it is portrayed on television creates another
type of accountability


PARTICIPANT CONCLUSION:
Performance measures derived
from strategic planning are
different from those derived from
tactical planning


Strategic plans are often written at a a very high
level in order to convey the conceptual
framework (particularly in state
-
level plans)


Strategic performance measures often address
pre
-
implementation “process” activities, thus
producing a preponderance of “outputs” rather
than “outcomes”


Two
-
step planning process could make logic
model/theory of change more straightforward


Performance measurement caveats


Most people (including your employees and
consultants) can learn to make the measures
come out the way they think
you

want them
to, while exerting a minimum of effort in
actually improving a process


Always question the measures you’ve
defined, keeping in mind that the people
applying them could find ways of boosting the
measures without really improving anything


Test each measure to determine if it operates
as expected. Does it always go one way
when things get better and the other when
things get worse?

Performance measurement musts



Early and often, measure and evaluate
progress toward the goals and objectives that
have been defined by the governing body


Continually verify that defined measures
actually correspond with the achievement of
goals and objectives


Resist overreaching : measures of long
-
term,
global outcomes tend to be unreliable.


Performance measurement is complex and
requires significant expertise

Strategic Goal 3:
Identify and recommend cost
-
effective

biometric identification applications


Objective 3.1
:

By September 2004, research, identify, and recommend technological
applications that support biometrics for rapid identification.

Objective 3.2:

By September 2004, research, identify, and evaluate the costs and
benefits of

biometric identification applications.

Outcomes:


Increased knowledge of biometric technologies


Improved cost
-
effective biometric identification solutions

Performance Measures:


Number of research projects on biometric technological solutions
completed by

September 2004


Number of research projects on costs and benefits of biometrics
completed

by September 2004


Number of research reports presented to the IIJIS Governing Body


Increased knowledge of biometric technologies

Increased knowledge of cost
-
effective biometric solutions for

courtroom identification of defendants

Eliminate problem of sentencing wrong individual

More accurate reporting of court dispositions

to criminal history repository

More complete criminal history records

Enhanced Public Safety

Revised Performance measures for Strategic Goal #3

Assumed Causal Chain as Inputs, Outputs, and Outcomes

Output

Output

Short
-
term Outcome

Short
-
term Outcome

Intermediate Outcome

Intermediate Outcome

Staff work time conducting research on biometric solutions

Input

More informed justice decision
-
making

Final Outcome

Why do I have to worry about all this?


Projects have been
de
-
funded

to lack of
performance measure data



Need to justify capital
expenditures



Need for public
accountability



It’s part of professional “
tool kit