Possible evaluation framework for the BBC Skillswise web site: http://www.bbc.co.uk/skillswise

normalpetsSoftware and s/w Development

Nov 4, 2013 (3 years and 5 months ago)

57 views

Possible evaluation framework for the BBC Skillswise web site:
http://www.bbc.co.uk/skillswise




Evaluation
stakeholders,
their concerns

and associated
questions


Stakeholder

Concerns


Evaluation Question

1. BBC
Management

Value for money

Whether students learn

Whether the whole site is used.


Is anything irrelevant or unused or
useless? Are there measurable gains
for the students, so that we can
ascertain value
-
for
-
money?

2. Tutors/

teachers

How easy it

is to use.

Whether people learn anything.

How flexible it is.

How well can you incorporate the
site into your own teaching?

3.
Students/use
rs of the site

How easy it is to use.

How they feel about using it.

To what extent has the site motivated
you to
learn?

Data capture
methods used


1. BBC
-

Weblog of server traffic


page hits/”stickiness” of
pages/analysis of user
feedback/discussion fora/tracked
scores for interactive quizzes
-

to
establish what concrete benefits
from using the site there are, an
d
what sections of the site are not
used.

2. Ethnographic study of 10
groups of students


to establish a
clear perception of student
response to the site


Data capture
methods used


3. Focus groups with teachers


perceptions of the applications the
mate
rial has/ease of integration
with the curriculum/strengths and
weaknesses/perceptions of learner
response



Time Line


all projects to run concurrently
-

2 weeks


p污lning⽡/min⽢ooking 瑲ave氠lnd wr楴楮g
數p散瑡瑩tns for ou瑣tmes

All run at the
same
time

Compiling quantitative data


from BBC


慮d qu慬楴慴av攠d慴愠


from 捯汬lg敳e


one month

Analysing it
-

six weeks

Writing report, getting feedback, correcting it, and producing executive summary


one
month.


THREE MONTHS TOTAL


Costs

Director



d慹s 瑯 wr楴攠propos慬a 瑨敮 on攠day 愠w敥k un瑩氠瑨攠fin慬afiv攠days. (18 d慹s)

tr楴敲⽒敳敡e捨 Off楣敲s


x2


for 污獴 month, 瑯 wr楴攠r数ort

o敳敡e捨敲⽤慴愠捯汬散瑩ln


x10 groups x 2 d慹s (on攠for fo捵s group, on攠for obs敲v慴楯n)

Admin suppor琠


Ri
sks

Server
failure

Not finding
enough of a user
group

No determinate
outcomes…inco
n捬csiv攠d慴愠or
瑯o mu捨 d慴愠or
no琠tnough d慴愠
g慴a敲敤

Budg整eov敲run
(慬汯w 慮 ex瑲愠
20┩

E汯p敭en琠瑯
Br慺楬f
䵡n慧楮g
䑩D散瑯r