Do cross
-
sectional student assessment
data make a reasonable proxy for
longitudinal data?
Elena Zaitseva and Mantz Yorke
Liverpool John Moores University
HEIR Conference, University of Liverpool
2nd year underperformance
2
nd
year students are characterised as a cohort where ‘
many suffer
from reduced motivation or apathy, declining grade point averages,
or a letdown from their first year
’ (Pattengale & Schreiner, 2000)
Underperformance of 2
nd
year students is well researched by
North American scholars
; the topic receives less attention in the
UK and Europe
‘The Forgotten Year: Tackling the ‘Sophomore Slump’ is a HEA
funded institutional research project
http://secondyearexperience.ljmu.ac.uk/
Project:
•
Examines patterns in student performance across the levels of
study
and longitudinal dynamics of slump (quantitative research)
•
Investigates underlying factors behind the 2
nd
year students’
underperformance and diminished engagement (qualitative
research )
•
Develops and evaluates interventions
to enhance experience of
the 2
nd
year cohort
S
lump is a dynamic phenomenon
Longitudinal
performance of
programmes
with 20+ students
2008
-
09
2009
-
10
2010
-
11
N
Consistent
non
-
slumpers
no slump
no slump
no slump
14
Consistent
slumpers
slump
slump
slump
8
‘Recovered’
slump
slump
no slump
2
‘In and out’
no slump
slump
no slump
4
‘Plunged into
recession’
no slump
slump
slump
6
New
slumpers
no slump
no slump
slump
2
Instead of waiting for a programme’s longitudinal data,
can we use cross
-
sectional student performance data
to predict slump?
Data from 08
-
09 and 09
-
10
Data from 09
-
10 and 10
-
11
Number of valid records
(56 programmes, N=20+ students)
08
-
09 Level 1 3006
08
-
09 Level 2 2519
09
-
10 Level 1 2975
Number of valid records
(49 programmes, N=20+ students)
09
-
10 Level 1 2807
09
-
10 Level 2 2385
10
-
11 Level 1 2984
Valid records:
Students who were awarded marks in respect of 120 credits at the Level stated
Exception to the above:
Social Work where 36 credits are awarded on pass/fail basis, i.e. 84 credits with marks
The data used
Starters
AY2008
-
09
Starters
AY2009
-
10
Starters
AY2010
-
11
Year
1
Year
2
Year
1
Year
1
Year
2
Longit
Longit
X
-
sect
X
-
sect
AY2008
-
09
AY2009
-
10
AY2010
-
11
B
C
A
True
cohort
True
cohort
Quasi
-
cohort
Quasi
-
cohort
The analytical frame
Cross
-
sectional
No slump
Slump
Longitudinal
No slump
22
11
Slump
4
12
Match
69.4 per cent
Calculating the match
–
Analysis A
Starters
AY2008
-
09
Starters
AY2009
-
10
Starters
AY2010
-
11
Year
1
Year
2
Year
1
Year
1
Year
2
Longit
Longit
X
-
sect
X
-
sect
AY2008
-
09
AY2009
-
10
AY2010
-
11
69.6%
match
69.4%
match
69.4%
match
The outcomes
Starters
AY2008
-
09
Starters
AY2009
-
10
Starters
AY2010
-
11
Year
1
Year
2
Year
1
Year
1
Year
2
Longit
Longit
X
-
sect
X
-
sect
AY2008
-
09
AY2009
-
10
AY2010
-
11
+ 1.53%
+ 1.49%
+ 1.40%
A complexity
Starters
AY2008
-
09
Starters
AY2009
-
10
Starters
AY2010
-
11
Year
1
Year
2
Year
1
Year
1
Year
2
Longit
Longit
X
-
sect
X
-
sect
Acad Year
2008
-
09
Acad Year
2009
-
10
Acad Year
2010
-
11
73.2%
match
81.6%
match
67.3%
match
Revised outcomes
Academic Year
2008
-
09
Academic Year
2009
-
10
Year 1 mean
Year 2 mean
Year 2 mean
Cross
-
sectional slump
Mark
Year 1 mean
A paradox
(not the only one)
What proportion of students enrolled on the programme
are ‘captured’ by the most popular 120 credits?
The question of homogeneity
Module
Max N
students
Enrolled on
module
Fraction of Max N
'Top 10' fraction
sum
Homogeneity
Index
xx01
78
78
1
xx02
78
78
1
xx03
78
78
1
xx04
78
78
1
xx16
78
50
0.64
xx16
78
50
0.64
xx13
78
42
0.54
xx13
78
42
0.54
xx12
78
37
0.47
xx14
78
35
0.45
7.28
0.73
xx14
78
35
0.45
xx07
78
29
0.37
xx15
78
29
0.37
xx15
78
29
0.37
xx06
78
26
0.33
xx05
78
23
0.29
xx08
78
17
0.22
xx10
78
12
0.15
xx09
78
10
0.13
xx11
78
2
0.03
A worked example
Programme
N with 120
credits Yr1+2
Homog
Yr1
Homog
Yr2
Applied Social Science 1
82
1.00
0.87
Applied Social Science 2
97
1.00
0.92
Applied Social Science 3
74
1.00
0.80
Applied Social Science 4
149
1.00
0.86
Arts 1
45
1.00
0.62
Arts 2
55
0.99
0.71
Education 1
21
1.00
1.00
Education 2
78
1.00
0.73
Humanities & Social Science 1
26
0.99
1.00
Humanities 1
47
1.00
0.92
Humanities 2
66
1.00
0.75
Science 1
27
0.98
0.94
Science 2
145
1.00
0.99
Science 3
26
1.00
0.80
Technology 1
63
1.00
1.00
Selected results from homogeneity analysis
Programme
N with 120
credits Yr1+2
Homog
Yr1
Homog
Yr2
Applied Social Science 1
82
1.00
0.87
Applied Social Science 2
97
1.00
0.92
Applied Social Science 3
74
1.00
0.80
Applied Social Science 4
149
1.00
0.86
Arts 1
45
1.00
0.62
Arts 2
55
0.99
0.71
Education 1
21
1.00
1.00
Education 2
78
1.00
0.73
Humanities & Social Science 1
26
0.99
1.00
Humanities 1
47
1.00
0.92
Humanities 2
66
1.00
0.75
Science 1
27
0.98
0.94
Science 2
145
1.00
0.99
Science 3
26
1.00
0.80
Technology 1
63
1.00
1.00
Selected results from homogeneity analysis
- Home
- Browse
- All Categories
- AI and Robotics
- Biotechnology
- Clean Technology
- Data Management
- Electronics - Devices
- Enterprise Applications
- Entertainment
- Internet and Web Development
- Management
- Mechanics
- Mobile - Wireless
- Networking and Communications
- Oil and Offshore
- Security
- Semiconductor
- Servers
- Software and s/w Development
- Storage
- Telecommunications
- Urban and Civil
- Community
- Upload
Comments 0
Log in to post a comment