of Key Competences

natureplaygroundAI and Robotics

Nov 14, 2013 (3 years and 8 months ago)

60 views

The Use of ICT for the Assessment
of Key Competences

Irish Presidency Conference

Better Assessment and Evaluation to Improve Teaching and Learning

19
-
20 March 2013


Christine Redecker

Yves Punie




IPTS
: Part of Joint Research Centre of
the EC: 7 Research Institutes across
Europe

Mission
: “to provide customer
-
driven
support to the EU policy
-
making
process

by developing science
-
based
responses to policy challenges that have
both a
s
ocio
-
economic

as well as a
scientific/technological

dimension”

IPTS
-

Who we are

Future Learning Spaces in 2020

2006 2007 2008 2009 2010 2011 2012

2013

Learning 2.0 in formal E&T

Learning 2.0 in informal settings


Creativity & Innovation in
schools in EU

eLearning in the EU10 MS

-

Digital Competence

-

Active Ageing & Learning

Tell
-
Net Teacher Networking

Review ICT
Impact

IPTS Studies on ICT for Learning

The Future of Learning

ICT for the Assessment of Key Competences

Digital Competence

Mainstreaming Creative Classrooms

Open Education in Europe

1to1 Learning initiatives in Europe

Mapping Technologies for Learning


DC Follow up

Where we stand

Multiple Choice

Short Answer

Generation 1

Automated administration
and/or scoring of
conventional tests

Adaptive Testing

Generation 2

Generation 4

Intelligent Tutors

Automated Feedback

Personalised
feedback and
tutoring

Learning 2.0

Generation 3

Continuous
integrated
assessment

Data Mining & Analysis

Behavioural tracking

Generation Re
-
invention

Transformative strategy: use technology to
change test formats

Peer assessment

Collaborative multimedia learning
environments

Online collaboration

Virtual laboratories

Games

Simulations

Virtual Worlds

Replication of complex
real life situations within a
test

Bunderson, Inouye & Olsen (1989). The four generations of computerized educational
measurement. In R. L. Linn (Ed.),
Educational measurement

(Third ed., pp. 367
-
407). New
York: Macmillan

Bennett, R. E. (2010). Technology for Large
-
Scale Assessment. In P. Peterson, E. Baker & B.
McGaw (Eds.),
International Encyclopedia of Education

(3rd ed., Vol. 8, pp. 48
-
55). ford:
Elsevier.

1990 1995 2000 2005 2010 2015 2020


2025

Efficient testing

Personalised learning

Learning Analytics

Computer
-
Based Assessment (CBA)

Technology
-
enhanced learning

Current use of ICT in
assessment in schools

Central

recommendations

on

the

use

of

ICT

in

pupil

assessment

in

compulsory

education

in

primary

and

general

secondary

education

(ISCED

1
,

2
,

3
),

2009
/
10

Source
:

Eurydice

(
2011
)
.

Key

Data

on

Learning

and

Innovation

through

ICT

at

School

in

Europe

2011
,

Education,

Audiovisual

and

Culture

Executive

Agency
.

There is scope for improvement

1
st

Generation CBA:
Problem
-
Solving

http://www.worldclassarena.org/v5/flash/09_y
ear_old/backtofront.swf

World Class Arena Sample Test: Problem
Solving (9 years)

Examples

http://www.worldclassarena.org/v5/flash/13
_year_old/pyramids.swf

Problem Solving 13 years

Generation Re
-
Invention:
Problem
-
Solving

Examples

1
st

Generation CBA:
Situational Judgement


EPSO sample situational judgement test questions;
http://europa.eu/epso/discover/prepa_test/sample_test/index_en.htm


Examples

Generation Re
-
Invention:

e
-
tray exercise

EPSO sample assessment centre e
-
tray exercise

http://europa.eu/epso/discover/prepa_test/sample_test/index_en.htm

Examples

Future eAssessment
Strategies

Multiple Choice

Short Answer

Generation 1

Automated administration
and/or scoring of
conventional tests

Adaptive Testing

Generation 2

Generation 4

Intelligent Tutors

Automated Feedback

Personalised
feedback and
tutoring

Learning 2.0

Generation 3

Continuous
integrated
assessment

Data Mining & Analysis

Behavioural tracking

Generation Re
-
invention

Transformative strategy: use technology to
change test formats

Computer
-
Based Assessment (CBA)

Technology
-
enhanced learning

Peer assessment

Collaborative multimedia learning
environments

Online collaboration

Virtual laboratories

Games

Simulations

Virtual Worlds

Replication of complex
real life situations within a
test

Bunderson, Inouye & Olsen (1989). The four generations of computerized educational
measurement. In R. L. Linn (Ed.),
Educational measurement

(Third ed., pp. 367
-
407). New
York: Macmillan

Bennett, R. E. (2010). Technology for Large
-
Scale Assessment. In P. Peterson, E. Baker & B.
McGaw (Eds.),
International Encyclopedia of Education

(3rd ed., Vol. 8, pp. 48
-
55). ford:
Elsevier.

Learning Analytics

1990 1995 2000 2005 2010 2015 2020


2025

Efficient testing

Personalised learning

Towards Generation 3:
Datamining

Signals

http://www.itap.purdue.edu/tlt/sig
nals/signals_final/index.htm

Similar: GPS at Northern Arizona
University

http://resourceconnect.nau.edu/gr
ade
-
performance
-
status.asp

Examples

Towards Generation 3: Behaviour Tracking

Example: SNAPP

SNAPP is a software tool that
allows users to visualize the
network of interactions
resulting from discussion
forum posts and replies.
Teachers can rapidly identify
patterns of user behaviour.
SNAPP can extract all user
interactions from various
commercial and open source
learning management
systems, such as BlackBoard
and Moodle.

Examples

Towards Gen 4:

Intelligent Tutoring Systems

GnuTutor
(http://gnututor.com/)

OpenSource version of AutoTutor

https://www.youtube.com/watch?v=k6HDE
EnYuIA&feature=player_embedded

AutoTutor

https://www.youtube.com/watch?v=aPcoZPj
L2G8


Examples

Generation 1

Generation 2

Generation 4

Personalised
feedback and
tutoring

Learning 2.0

Generation 3

Continuous
integrated
assessment

Generation Re
-
invention

Computer
-
Based Assessment (CBA)

Technology
-
enhanced learning

Collaborative multimedia learning
environments

Bunderson, Inouye & Olsen (1989). The four generations of computerized educational
measurement. In R. L. Linn (Ed.),
Educational measurement

(Third ed., pp. 367
-
407). New
York: Macmillan

Bennett, R. E. (2010). Technology for Large
-
Scale Assessment. In P. Peterson, E. Baker & B.
McGaw (Eds.),
International Encyclopedia of Education

(3rd ed., Vol. 8, pp. 48
-
55). ford:
Elsevier.

-

mainstream use for high stakes testing

-

further development of innovative test formats

-

increased ease of use at school level

-

trend towards automated scoring


→ cultural change:

increased acceptance of CBA

Automated administration
and/or scoring of
conventional tests

Adaptive Testing

Replication of complex real life
situations within a test

-

Multimedia collaborative learning
environments support key competence
development.


-

Peer and portfolio assessment evolve
naturally.


-
Data mining and analysis will make
integrated assessment and tutoring possible
in the future.


We are in a transitional period: Moving from the “Testing
Paradigm” to the (future) “Tutoring Paradigm” in which explicit
testing could become obsolete

Technological trends

1990 1995 2000 2005 2010 2015 2020


2025

Efficient testing

Personalised learning

Key Questions

1.
Which ICT

tools, formats, environments, devices etc. are
most suitable for the assessment of
which Key
Competences
?



2.
How can ICT be used to support
competence
-
based
assessment

strategies that adequately capture the
skills

and
attitudes

dimensions of all eight Key Competences and
their
transversal

nature?


Assessing

Key Competences

F = Formative Assessment

D = Diagnostic Assessment

S = Summative Assessment

1

Communication in the mother tongue

D
S

F
D

FS

2

Communication in foreign languages

D
S

FS

3a

Mathematical competence

D
S

F
D

FD
S

FDS

3b

Basic competences in science and technology

FS

FDS

4

Digital competence

FDS

5

Learning to learn

FS

FDS

6

Social and civic competences

FS

7

Sense of initiative and entrepreneurship

FS

8

Cultural awareness and expression

FS

FS

CBA

ePortfolios

Peer
Assessment

Self
Assessment

Quizzes /
simple
games

Virtual worlds
games

Simulations

Intelligent
Tutors etc.

Focus Areas


PART 1



INSPIRATION

Mother Tongue Literacy

1

Communication in the mother tongue

CBA (DS)

Quizzes / simple games

(FD)

ePortfolios
(FS)

Matching spoken and written language

http://www.familylearning.org.uk/sight_word_games.html

Foreign language learning

2

Communication in foreign languages

CBA (DS)

Quizzes / simple games

(FD)

ePortfolios
(FS)

http://www.nickjr.de/games/107
-
swipers
-
buchstabier
-
buch

Mathematical games

3a

Mathematical competence

CBA (DS)

Quizzes / simple games

(FD)

Simulations (FDS)

Inteligent tutoring systems (FDS)

Mathematical games using simulation for secondary school education

Source:
http://hotmath.com/games.html
.

5

Learning to Learn

CBA (F)

Peer Assessment (FS)

Self Assessment (FS)

http://spark.uts.edu.au/

Example:
SparkPlus

Peer Assessment for
Collaborative Work


http://www.youtube.com/watch?v=Tp8v2UBbmN4


www.moodle.org

Moodle is a free web
application that educators can
use to create effective online
learning sites.

Moodle Statistics

Registered sites: 77,213

Countries: 229

Courses: 7,116,466

Users: 66,349,750

Teachers: 1,292,731

Enrolments: 56,306,274

Forum posts: 118,413,272

Resources: 63,891,942

Quiz questions: 147,057,434

Peer Assessment

using moodle

ePortfolios

1

Communication in the mother tongue

CBA (DS)

Quizzes / simple games

(FD)

ePortfolios
(FS)

2

Communication in foreign languages

CBA (DS)

Quizzes / simple games

(FD)

ePortfolios
(FS)

8

Cultural Awareness and expression

ePortfolios
(FS)

Virtual worlds / games (FS)

http://sites.google.com/site/eportfolioapps/overview/model
-
k
-
12


ePortfolios

ePortfolios

1

Communication in the mother tongue

CBA (DS)

Quizzes / simple games

(FD)

ePortfolios
(FS)

2

Communication in foreign languages

CBA (DS)

Quizzes / simple games

(FD)

ePortfolios
(FS)

8

Cultural Awareness and expression

ePortfolios
(FS)

Virtual worlds / games (FS)

http://sites.google.com/site/eportfolioapps/overview/levels


Level 1
-

ePortfolio as Storage


Collection:

Creating the Digital Archive (regularly


weekly/monthly)

A Focus on Contents & Digital Conversion


Level 2
-

ePortfolio as Workspace


Collection + Reflection:

Immediate Reflection on Learning & Artifacts in Collection (
regularly)

A Focus on Process

& Documentation of
Learning


Level 3
-

ePortfolio as Showcase


Selection/Reflection/Direction

(each semester? End of year?)

A Focus on Product & Documentation of
Achievement



→ A (formative/summative) assessment approach on the basis of artefacts

→ Focus on production and reflection

A game approach to arts

8

Cultural Awareness and expression

ePortfolios
(FS)

Virtual worlds / games (FS)

http://pheon.org
;
http://pheon.org/intro.html
;
http://americanart.si.edu/multimedia/games/pheon.cfm.
Pheon

is a multimedia scavenger hunt that tests players


ingenuity through
exploring the collection of the Smithsonian American Art Museum, making
objects, and texting through the galleries

ePortfolios meet

Virtual reality

Science:

Molecular Workbench

3b

Basic competences in science and technology

CBA (S)

Virtual Worlds / games (FS)

Simulations

(FDS)

http://mw2.concord.org/public/part2/diffat/page3.cml

3a

Mathematical competence

CBA (DS)

Quizzes / simple games

(FD)

Simulations (FDS)

Inteligent tutoring systems (FDS)

Simulations for

mathematics

Sample task from

US future New State Tests

(2014/15)

http://www.k12center.org/rsc/pdf/a
-
sea
-
change
-
in
-
assessment
-
letter
-
size.pdf

Cognitive Tutor

(Carnegie)

3a

Mathematical competence

CBA (DS)

Quizzes / simple games

(FD)

Simulations (FDS)

Inteligent tutoring systems (FDS)

http://www.youtube.com/watch?v=RR4HPrGlUz
8&feature=related

Science:

Quest Atlantis

3b

Basic competences in science and technology

CBA (S)

Virtual Worlds / games (FS)

Simulations

(FDS)

Example:

Quest Alantis


Quest Atlantis (QA) is an
international learning and
teaching project that uses a 3D
multi
-
user environment to
immerse children, ages 9
-
16, in
educational tasks. QA combines
strategies used in the
commercial gaming
environment with lessons from
educational research on learning
and motivation.



Citizenship

6

Social and civic competences

CBA (DS)

Peer/Self Assessment (FDS)

Virtual Games / Simulations
(FDS)

Educational game, designed to help teach concepts in citizenship, geography, and media.

Developed by Serious Games International, it has detailed lesson plans and assignments for students.

Example: Child labour in
Bangladesh:

http://www.youtube.com/wat
ch?v=mf9Ju86IVYw

Global Conflicts

http://www.globalconflicts.eu/


Examples


PART 2



CRITICAL INSPECTION

Digital Competence

4

Digital competence

CBA (S)

Simulations (FDS)

Internet and Communication Core Certification (IC3) exam sample question

http://www.certiport.com/Portal/common/htmllibrary/Sales-Video-Demos/demo_ic3.html
→ Focus on knowledge

CBA for

Digital Competence

Digital Competence: IC3

4

Digital competence

CBA (S)

Simulations (FDS)

Internet and Communication Core Certification (IC3) exam
sample question

http://www.certiport.com/Portal/common/htmllibrary/Sales
-
Video
-
Demos/demo_ic3.html


→ Focus on operational skills

CBA for

Digital Competence

Entrepreneurship Questionnaire

7

Sense of initiative and entrepreneurship

CBA (DS)

Peer/Self Assessment (FDS)

Virtual Games / Simulations
(FDS)

?

https://www.careeringahead.
com.au/survey/Entrepreneurs
hip/Generic/questions.asp

→ Focus on personality/disposition as a static fact

Entrepreneurship

Questionnaire

Conclusions

ICT offers powerful, engaging assessment formats, in particular:

CBA

ePortfolios

Self / Peer
Assessment

Quizzes /

simple games

Virtual worlds/ games

Simulations

Basic literacy

(foreign/mother tongue)

Basic mathematical
competence

Learning

to Learn

Creative writing

(foreign/mother tongue)

Cultural awareness and
expression

Scientific competence

Digital
competence

Social and civic
competences

Educational
software

Mathematical
competence

CBA lends itself to simplified, knowledge
-
based assessment

→ Pedagogy comes first! Careful selection of question formats

ICT Assessment approaches are underdeveloped for:

Learning to Learn

Sense of initiative and entrepreneurship

Social and civic competences

Caveat:

The Paradigm Shift

Technology

1990 1995 2000 2005 2010 2015 2020

2
025


Focus on: Efficient testing

Focus on: Personalised learning

Era of Computer
-
Based Assessment (CBA)

Era of Technology
-
enhanced learning

Pedagogy

Efficient

Testing

Validity of

Scores

Quick

Feedback

Authentic

Contexts

CBA

(Gen.1)

Adaptive

Tests

(Gen.2)

Automatic

Scoring

Gen. Re
-

Invention

Learning

2.0

Immersive

Learning

Learning

Analytics

Intelligent

Tutors

?

?

?

?

→ Primacy of Pedagogy:

P
edagogical guidelines for intergrated assessment






Thank you very much for your attention!





Christine Redecker


Yves Punie



Christine.Redecker@ec.europa.eu


Yves.Punie@ec.europa.eu





http://ipts.jrc.ec.europa.eu/public
ations/pub.cfm?id=5719


further information ……


http://is.jrc.ec.europa.eu


Newsletters:

http://is.jrc.es/pages/newsletter.html


Publications:

http://is.jrc.ec.europa.eu/pages/Publications.html


Contact:


jrc
-
ipts
-
is
-
NEWS
@ec.europa.eu