3x

exhaustedcrumMechanics

Oct 24, 2013 (3 years and 11 months ago)

79 views


“There are known
knowns
. These are
things we know that we know. There
are known unknowns. That is to say,
there are things that we know we
don't know. But there are also
unknown unknowns. There are things
we don't know we don't know.”


Donald Rumsfeld, February 2002



The use of regular multiple choice
quizzes to enhance the development
of students self
-
assessment and self
-
feedback skills

Pecha

Kucha

Presentation

Alan Brown

CHEP Annual Conference

January 2013

4 types of student?


A thirteenth
-
century
Persian

poet,
Ibn

Yamin

said there are four types of men:

1.
One who knows and knows that
he knows... His horse of wisdom
will reach the skies.

2.
One who knows, but doesn't
know that he knows... He is fast
asleep, so you should wake him
up!


Source: Wikipedia, 2013

4 types of student?

3.
One who doesn't know, but knows that
he doesn't know... His limping mule will
eventually get him home.

4.
One who doesn't know and doesn't
know that he doesn't know... He will be
eternally lost in his hopeless oblivion!



Our first job is to get students away from 4

Source: Wikipedia, 2013

Fluid Mechanics in Year 1


Brand new subject


Not on ‘A’ level physics or maths syllabus


Several new concepts


Pressure


Viscosity


Density


Fluid Mechanics in Year 1


Several ‘Old’ concepts


Forces


Velocity and Acceleration


Momentum


But these don’t look the
same as in physics



Assessments historically
based on applied
mathematical problems

The problem


Enable students to find out what they know
and do not know


This is the role of Feedback


Several new concepts requires lots of rapid
feedback



I taught this module to 88 students

The solution


Multiple choice quiz questions each week on
Blackboard


Based on the previous week’s material



The adaptive release system is used to enforce
the quiz questions


Any attempt at the questions allows access to
the next set of resources


Answers and feedback are given instantly



The questions are designed to test conceptual
knowledge, not learned definitions



Class test uses similar types of multiple choice
questions

Result 1: Class test marks


Scores in the class test were similar to the
previous year (applied maths problems)


Quiz designed to test the understanding of the
new and old concepts


misunderstandings much easier to spot and correct
than using applied mathematics type questions


Result 1: Class test marks

Feedback to correct conceptual misunderstandings given
1 week after the test

Result 2: Predicting performance


Are students more aware of their own
knowledge?


Students were asked to predict their grade:


Before they sat the test


After they sat the test


After they had seen all the answers

Result 2: Predicting performance


With MCQs the students were much better at
predicting their marks

With MCQs

Without MCQs

Result 3: Engagement


Out of 88 students


4 attempted none


7 attempted 1 or 2


15 attempted 3 or 4


62 attempted 5 or
more

Test attempts

Mean overall
mark

0

24

1

12

2

25

3

42

4

52

5

48

6

53

7

54

Result 3: Engagement


Student feedback on the MCQ system :


87 % of respondents found the MCQs
helpful or very helpful,


5 % said they were unhelpful


Many students answered multiple times


What have I learned


New concepts are difficult to learn


Students need to start from a position of self
-
knowledge


Applied mathematics is not an effective way of
assessing conceptual knowledge


Most students will engage with feedback
opportunities

What I will do next


Check MCQ answers to try and determine
misunderstandings earlier


Don’t wait until the class test


Refine teaching based on misunderstandings
from last year


Enhance the automated feedback in the MCQs


Use MCQs to monitor engagement

Conclusions


Rapid feedback should enhance students self
-
knowledge


MCQs used to give rapid automated feedback
to 88 students on a weekly basis


Adaptive release gives an incentive without
marking and assessment

Conclusions


Students are better able to predict their marks


Students were overwhelmingly positive about
the quiz questions


Engagement easy to assess



Hopefully fewer “Unknown Unknowns”