Journal of Personnel Evaluation in Education, 11

rescueflipUrban and Civil

Nov 16, 2013 (3 years and 10 months ago)

85 views

What factor had the largest effect on student achievement?

A Research Brief


Mixed Ability Grouping?

4

Class Size?

3

Prior Achievement?

2

The Teacher?

1


This slide is based on the

1997 research study by Wright and colleagues

Wright, S. P.,
Horn, S. P., & Sanders, W. L. (1997). Teacher and classroom context effects on student
achievement: Implications for teacher evaluation.
Journal of Personnel Evaluation in
Education, 11
, 57
-
67.

which was a seminal study examining the effect
s of schools and
teachers on student achievement. In this

study
, the authors used
the
Tennessee Value
-
Added Assessment System (TVAAS) for data analysis. TVAAS is a

system for

multivariate, longitudinal analysis of student achievement data.
The

main purpose

of this
study wa
s to produce estimates of school and teacher effects on student achievement
gains
that are separated from the effects of other confounding factors, such as intra
-
classroom heterogeneity, student
prior achievement
, and class size.

Student a
cademic
gain was calculated by stu
dents’ scale score

this year minus that student’s scale score last
year. Classroom heterogeneity in achieve
ment is defined as the standard

deviation of the
achievement level scores of the students in the class. The larger the standard deviation,
the more heterogeneous in achievement
are

the students in the class.


The findings are:



T
eacher
effect is the dominant factor affecting student acad
emic gain.



Classroom context variables of heterogeneity among students and class sizes have
relatively little influence on academic gain.



“Effective teachers appear to be effective with students of all achievement levels,
regardless of the level of
heterogeneity in their classrooms” (p. 63). That means if
the teacher is ineffective, students under the teacher’s tutelage will achieve
inadequate
ly
, regardless of how similar or different they are in terms of their
prior
academic achievement.


Detailed b
reakdown z
-
values are listed below (instead of reporting effect sizes, the
author
s

used

z
-
values

to describe the effects of variables on students’ achievement on
various subjects
):






(Set: 1=30 East Tennessee school systems
.


2=24 Middle
Tennessee school systems
.
)


These findings have

also

been supported
by many more

recent studies. For the last
several years, numerous researchers have explored the “value
-
added effects” of a
particular school or teacher through the use of sophisticated sta
tistical models involving
longitudinal data on student achievement.

Consider the impact of teacher effectiveness on
student achievement drawn from a sampling of studies presented in the following table:



Study

Key Findings

Sanders & Rivers
(1996)




Teacher effect on student achievement is cumulative. With
an even start at the second grade, differences in student
achievement of 52 to 54 percentile points were observed as
a result of two extreme teacher sequences after only three
years (low
-
low
-
low se
quence versus high
-
high
-
high
sequence).



Teacher effects on student achievement have been found to
be both cumulative and residual. Subsequent assignment of
effective teachers cannot offset the effects of prior
ineffective ones.



The residual effects of both

effective and ineffective teachers
are measurable two years later, regardless of the
effectiveness of subsequent teachers.

Hanushek, Kain, &
Rivkin (1998)



Lower bound estimates suggest that variations in teacher
quality account for at least 7.5% of the
total variation in
measured achievement gains, and there are reasons to
believe that the true percentage is considerably larger.

Mendro, Jordan,
Gomez, Anderson, &
Bembry (1998a,
1998b).



The research findings in these studies on teacher
effectiveness
found not only that teachers have large effects
on student achievement, but also the measures of
effectiveness are stable over time.



Ineffective teachers have negative longitudinal effects on
student learning. If the students have a less effective teacher
in the first year and the highest level teachers for remaining
years, their achievement could never exceed that of the
students who have been assigned with effective teachers for
all the years.

Nye,
Konstantopoulos, &
Hedges (2004)




If primary grade teacher effects are normally distributed, the
difference in achievement gains between having a 25th
percentile teacher (a not so effective teacher) and a 75th
percentile teacher (an effective teacher) is over one third of
a standard deviat
ion in reading and almost half a standard
deviation in mathematics.



The difference in achievement gains between having a 50th
percentile teacher (an average teacher) and a 90th percentile
teacher (a very effective teacher) is about one third of a
standard

deviation in reading and somewhat smaller than
half a standard deviation in mathematics

Rivkin, Hanushek, &
Kain (2005).



Differences between teachers explained about 15% of the
measure variance in student test scores.



In both reading and mathematics, a o
ne standard deviation
increase in teacher quality for a grade raises student
achievement by about one
-
tenth of standard deviation.

Aaronson, Barrow, &
Sander (2007)



A standard deviation increase in teacher effectiveness over a
full year raises student
math test scores by 0.15 standard
deviations.



Controlling for sampling error, a one standard deviation,
one semester improvement in math teacher quality raises
student math scores by 0.15 standard deviations. Thus, over
two semesters, a one standard deviat
ion improvement in
math teacher quality translates into an increase in math
achievement equal to 22% of the average annual gain.



Estimates of teacher effects are relatively stable over time,
reasonably impervious to a variety of conditioning
variables, and

do not appear to be driven by classroom
sorting (i.e., student/teacher assignment) or selective use of
test scores.

Leigh, (2010)



Moving from a teacher at the 25th percentile to a teacher at
the 75th percentile would raise test scores by one
-
seventh of
a standard deviation. Since a 0.5 standard deviation increase
in test scores is equivalent to a full year’s learning, this
implies that a 75th percentile teacher can achieve in three
-
quarters of a year what a 25th percentile teacher can achieve
in a full y
ear.



Moving from a teacher at the 10th percentile to a teacher at
the 90th percentile would have even more dramatic effects,
raising test scores by one quarter of a standard deviation.
This implies that a teacher at the 90th percentile can achieve
in half
a year what a teacher at the 10th percentile can
achieve in a full year.

Stronge, Ward,
Tucker, & Grant
(2011)



After controlling for variables such as class size, prior
student achievement and a host of individual student
variables (e.g., gender,
ethnicity, socio
-
economic level,
English Second Language learners), the students residual
gain scores (difference between predicted and actual
achievement levels) were calculated. In reading, students
taught by bottom quartile teachers could expect to scor
e, on
average, at the 21
st

percentile on the state’s reading
assessment, whereas students taught by the top quartile
teachers could expect to score at approximately the 54
th

percentile. This striking difference, more than 30 percentile
points can be attrib
uted to the quality of teaching occurring
in the classrooms during one academic year. Similar results
were reached for mathematics, with the students in the
bottom quartile teachers’ classrooms scoring, on average, at
the 38
th

percentile; while students in

the top quartile
teachers’ classrooms scored at the 70
th

percentile. In both
reading and math, there were no statistically significant
differences in student achievement levels at the beginning of
the school year between the top and bottom quartile
teache
rs’ classes.



Class size affects how much time and attention teachers give individual students, as well
as the social

dynamics among students. However,
extant
research finds a weak
relationship between
reduced
class

size and student performance (
Ehrenber
g, Brewer,
Gamoran, & Willms, 2001)
. Various studies have found that reducing class size in
primary grades from about 23 or 24 to 15 can yield an effect size in the range of .15 to
.26 (
Hattie, 2009;

United States Department of Education, 1998). This trans
lates, at best,
to a 7 to 8 percentile point rise in achievement over the course of a year (Barber, &
Mourshed, 2007).

A seven to eight percentile gain is a nice increase in student
achievement, but it comes from one of the most expensive educational refor
m policies.
When juxtaposed with improving teacher quality (as illustrated in the table above), the
effect of class size reduction pales in comparison.

What will work best is reduced class
size with a highly effective teacher teaching the class.


References:

Aaronson, D., Barrow, L., & Sander, W. (2007). Teachers and student achievement in the
Chicago public high schools.
Journal of Labor Economics, 25
(1), 95
-
135.

Barber, M., & Mourshed, M. (2007). How the world’s best
-
performing school systems
com
e out on top. Retrieved from
http://www.mckinsey.com/locations/ukireland/publications/pdf/Education_report.pdf
.

Ehrenberg, R., Brewer, D. J., Gamoran, A., & W
illms, J. D. (2001). Class size and
student achievement.
Psychological Science in the Public Interest, 2
(1), 1
-
30.

Hanushek, E. A., Kain, J. F., & Rivkin, S. G. (1998).
Teachers, schools, and academic
achievement.

Cambridge,

MA: National Bureau of Economic

Research. Retrieved
from
ht
tp://www.nber.org/papers/w6691
.

Hattie, J. (2009).
Visible learning: A synthesis of over 800 meta
-
analyses relating to
achievement.
New York: Routledge.

Leigh, A. (2010). Estimating teacher effectiveness from two
-
year changes in students’
test scores.
Economics of Education Review, 29,

480
-
488.

Mendro, R. L., Jordan, H. R., Gomez, E., Anderson, M. C., & Bembry, K. L. (1998a,
April).
Longitudinal teacher ef
fects on student achievement and their relation to
school and project evaluation.

Paper presented at the 1998 Annual Meeting of the
American Educational Research Association, San Diego, CA.

Mendro, R. L., Jordan, H. R., Gomez, E., Anderson, M. C., & Bembry
, K. L. (1998b,
April).
An application of multiple linear regression in determining longitudinal
teacher effectiveness.

Paper presented at 1998 Annual Meeting of the American
Educational Research Association, San Diego, CA.
United States Department of
Educa
tion. (1998, April).
Research on the academic effects of small class size.
Retrieved April 25, 2009, from
http://www.ed.gov/pubs/ClassSize/academic.html
.

Nye, B., Konstantopoulos, S., & Hedges,

L. V. (2004). How large are teacher effects?
Educational Evaluation and Policy Analysis, 26
(3), 237
-
257.

Rivkin, S. G., Hanushek, E. A., & Kain, J. F. (2005). Teachers, schools, and academic
achievement.
Econometrica, 73
(2), 417
-
458.

Sanders, W. L, & Rive
rs, J. C. (1996, November).
Cumulative and residual effects of
teachers on future student academic achievement.

Knoxville, TN: University of
Tennessee Value
-
Added Research and Assessment Center.

Stronge, J. H., Ward, T. J., & Grant, L.W. (2011). What makes

good teachers good? A
cross
-
case analysis of the connection between teacher effectiveness and student
achievement.
Journal of Teacher Education
, 62
(4), 339
-
355.

Wright, S. P., Horn, S. P., &

Sanders, W. L. (1997). Teacher and classroom context
effects on student achievement: Implications for teacher evaluation.
Journal of
Personnel Evaluation in Education, 11
, 57
-
67.