Teacher Incentive Fund Program

lapclassManagement

Nov 6, 2013 (3 years and 11 months ago)

85 views

Data Quality Innovation through
Knowledge Management in the
Teacher Incentive Fund Program

Sara Kraemer, Lexy Spry, Jeffery Watson

Center for Data Quality and Systems Innovation

University of Wisconsin
-
Madison


Jenna Scott

Westat


MIS Conference 2012

Overview of the Teacher Incentive
Fund 3 (TIF 3)


U.S. Department of Education (ED)
discretionary grant program



Overarching premise: To support efforts to
develop and implement performance
-
based
teacher and principal compensation systems
in high
-
need schools


Overview of TIF 3, cont.


Goals:

1)
Improve student achievement by increasing teacher and
principal effectiveness

2)
Reform teacher and principal compensation systems so
that teachers and principals are rewarded for increases
in student achievement

3)
Increase the number of effective teachers teaching poor,
minority, and disadvantaged students in hard
-
to
-
staff
subjects

4)
Create sustainable performance
-
based compensation
systems


62 grantees (i.e., SEAs, LEAs, charters, and other not
-
for
-
profit organizations)



Overview of TIF 3, cont.


To meet ED’s and grantees’ needs, ED awarded a
contract to our team to provide:

(1)
Technical assistance (TA) to the TIF grantees and ED

(2)
Compliance monitoring assistance to ED

(3)
Logistical support (i.e., conferences)



Our team:


Westat, University
of Wisconsin, J. Koppich
&
Associates
,
American
Institutes for Research, and
Synergy Enterprises, Inc.



Knowledge Management (KM)


Creating knowledge from data via organization
and interpretation


Data

with context becomes useful information


When enough people use

information over time,
it becomes knowledge


Goals as a Knowledge Manager


Collaboration


Expertise location


Support distributed work


Share lessons and experiences


Access


Find needed information


Security


Comprehension


Understand complex tasks

Evidence of Knowledge Management


Deliverables are on time


Duplication of work is avoided


Successes are iterative

Process and Tools


Mandated use


Centralized store of
data/information/knowledge


Flexible system, responsive to new needs,
user
-
focused


Actively managed (by KM)


Process and Tools for TIF3


Mandated use


Centralized store of
data/information/knowledge


Flexible system, responsive to new needs,
user
-
focused


Actively managed (by KM)

Opportunities for Improvement


Access/Trust Permissions


Methods of notification

Successes


TA Request List



Brought clarity to complex work


Useful now and in the future


Risk Assessment


Structure with flexibility


Method to manage multiple data points over time

Technical Assistance
(TA) Team



Interdisciplinary team of
experts in Performance
-
Based Compensation Systems (PBCS) and human
capital management


Distributed
team:


University
of Wisconsin
-
Madison (lead institution
)


AIR


Westat


Independent
experts


Role of the TA Provider to
Grantees


Manage the coordination of TA activities
for grantees


Deliver TA across
7 domains of PBCS:


Teacher evaluation


Principal evaluation


Value
-
Added and Measurement


Data quality and data systems


Communications and stakeholder engagement (includes
Union engagement)


Fiscal sustainability


Program evaluation


Role of
TA to U.S.
Department of
Education


Individual and cross
-
grantee assessments of:


TA needs


Risk


Deliver targeted expertise to
program officers in
technical
areas


Example: Assess if technical aspects of PBCS meets
compliance requirements (e.g., data
quality, principal
observation)

Grantee Needs Assessment
-

1


Purpose


One
-
stop
, up
-
to
-
date summary of grantee
PBSC


Track grantee progress over life cycle of program


Created in collaboration with knowledge
manager
(workspace)


Design
criteria


Consistent categories and naming conventions


Iterative
inputs


ability to update as
grantees’
progress in
real time


Shareable
with/among partners


Grantee Needs Assessment


2

How was it used?


Reflection with grantees,
feedback


Report
to
Department of Education


Prioritize TA activities


Coordination
of group
-

or individual
-
level TA


Group: workshops
, communities of practice (online), written
products, and webinars


Individual:
one
-
on
-
one conversations and feedback


Management of TA activities within/across
grantees


Cross
-
Grantee Risk
Analysis
-

1


Purposes


Identify and prioritize areas of greatest
risk


Organized by domain


Identify grantees most in need of
TA support


Internal
TA coordination


Report to Department
of Education
on status
of
program (end of Year 1)


Cross
-
Grantee Risk Analysis
-

2


Design


Based upon expert assessment of risk


Data source: Needs assessment


1
-
5 rating of risk, supplemental explanation/summary


Group
-
based analytic process


Workspace


Generated analytics for report


Visualizations of cross
-
grantee risk


(e.g., red, yellow, green)

Benefits of Knowledge Management
Approach
-

1


Managed
a complex space
of data
,
communication streams, content domains,
multiple
actors and analytic
goals


2
stakeholders (grantees, Department of Education)


Enhanced data quality in analysis and reporting


Coordinated resources
across distributed TA
providers


Benefits of Knowledge Management
Approach
-

2


Provided focus for targeted TA activities,
program directives


Utilized flexible
team organization and
technical system (
workspace)


Adapt
to emergent deliverables, project needs


Managed TA by functional goals


not possible
without explicit
KM design principles

Knowledge Management System
(KMS): Management’s Perspective


Overarching benefits of the KMS for Westat’s
work:


1)
Provides a systematic organization


2)
Allows for refined coordination


3)
Builds enriched collaboration

KMS Organization


Contact information for all team members


Including across TIF grants



History of the project


Examples: annual performance reports,
conference documents, email correspondence,
meeting and visit notes, needs and risk
assessment


KMS Organization, Cont.


Current working documents


Examples: Conference or meeting planning,
content papers



Management calendar


Houses all important project dates


KMS Coordination



Monthly report to client (ED)


Previously, not automated



Management calendar


(noted earlier)


Generates automatic notifications


Example: Weekly partners’ meeting



Task management


Assigns tasks and ensures that the tasks are completed on
time


Examples: Technical assistance (TA) requests, bimonthly phone
calls with grantees


KMS Coordination, Cont.


TA Requests


Catalogs the requests


Notifies all involved parties (i.e., program officer and
TA lead)


Follows the progress of the request to completion



Bimonthly Phone Calls


Coordinates time of call


Notifies monitor of phone call


Ensures that the task is completed

KMS Collaboration


Sharing of knowledge



Access to and/or collaborative development
of:



(1) meeting notes; (2) conference
presentations; (3) reports or papers; (4) needs
assessment and risk assessment; (5) grantee
letters; (6) annual performance reports (APRs)


Version control




KMS Collaboration, Cont.


Reports to client


Examples: risk assessment, needs assessment,
bimonthly grantee phone calls, TA site visit


Version control



Fast turnaround requests to client


Via the Knowledge Manager


Previously, more difficult and less robust


KMS: Monitors’ Perspective


Overarching benefits of KMS for
monitors’ work:


1)
Provides a systematic organization


1)
Facilitates enriched collaboration


KMS Organization


Stores and archives important information
that monitors write or complete, and/or
frequently consult


ED documents: grantee applications, APRs, goals
and objectives worksheets


TA documents: needs assessments


Email correspondence


Meeting and visit notes and reports

KMS Organization, Cont.


Organizes monitors’ tasks



Examples: Bimonthly phone calls, site visits



Sends automated reminder of meeting


Sends automated reminder to complete
meeting notes


KMS Collaboration


Monitor


TA relationship


Strengthened collaboration


central repository for
ALL information


Site visit notes and reports


Bimonthly meeting notes



Monitors’ interactions


Allows for monitors to “fill
-
in” for one another

Summary


The KMS provides systematic organization,
refined coordination, and strengthened
collaboration for both the management team
and monitors


allowing our team to provide
more efficient and effective technical
assistance and monitoring to TIF 3.

Questions?

Sara Kraemer

sbkraeme@wisc.edu


Jeff Watson

jgwatson@wisc.edu

Lexy

Spry

spry@wisc.edu


Jenna Scott

JennaScott@westat.com

Center for Data Quality and Systems Innovation

dataquality.wceruw.org