Dunwei Wen_paper_1 - AUSpace - Athabasca University

obtainablerabbiData Management

Jan 31, 2013 (4 years and 9 months ago)

142 views

ADAPTIVE ASSESSMENT IN WEB
-
BASED
LEARNING



By

Dunwei Wen
(1)
, Sabine Graf
(2)
, Chung
Hsien Lan
(3)
, Terry Anderson
(1)
, Kinshuk
(1)
,
Ken Dickson(1)


(1) Athabasca University, Alberta, Canada

(2) Women's Postgraduate College for Internet Technologies,
Vienna University of Technology, Austria

(3) Nanya Institute of Technology, Jhongli, Taiwan


Presenter:



Dunwei Wen


IEEE ICME 2007
-

International Conference on Multimedia & Expo, July 2
-
5,
2006, Beijing

IEEE International Conference on
Multimedia & Expo, Beijing, 2007

2

OUTLINE


I. Am I Ready


An Adaptive Readiness Self
-
assessment
System


1. Introduction


2. Knowledge Modeling


2.1. User Model


2.2. Counseling Model


2.3. Process Model


2.4. Assessment Model


3. System Structure And Realization


4.
Conclusion of Am I Ready


II. Performance Self
-
Assessment


III. Peer Assessment



IEEE International Conference on
Multimedia & Expo, Beijing, 2007

3

I.
AM I READY for Athabasca Unv.

A DISTANCE EDUCATION READINESS SELF
-
ASSESSMENT SYSTEM



Features


Is an
adaptive online

readiness

self
-
assessment system


Helps prospective students understand their


requirements


readiness


Provides
information

of distance learning at Athabasca University.


Based on integrated knowledge models


dynamically adjusts the contents

of the self
-
assessment
according to the interaction between


the knowledge models


the user’s responses.

IEEE International Conference on
Multimedia & Expo, Beijing, 2007

4

1. Introduction


Online readiness self
-
assessments are
widely
used

in universities that provide distance
education services.


Most current readiness self
-
assessment tools
are
online questionnaires
.


The shortcoming of those systems:


The
same question set

for different users


The
same question sequence

for every user


Less remedial information

to help users

IEEE International Conference on
Multimedia & Expo, Beijing, 2007

5

BASIC IDEAS
-

features


User
-
oriented


Questions are filtered for different users


Dynamic and adaptive


Next questions rely on users’ previous answers


Remedial


Instant feedback and remedial assessment
information

IEEE International Conference on
Multimedia & Expo, Beijing, 2007

6

BASIC IDEAS



methods


Technologies to model counselling knowledge


User Model


Counselling Model


Process Model


Assessment Model.



Operators adopted according to the above models
of a user:


Enabling some questions


Disabling some questions


Sorting questions by priorities of questions


Checking contradictory answers


Showing real time information


IEEE International Conference on
Multimedia & Expo, Beijing, 2007

7

2. Knowledge Modeling And Architecture

IEEE International Conference on
Multimedia & Expo, Beijing, 2007

8

2.1 USER MODEL


(1)

STATIC MODEL





Acting as the first assessment filter:


Highest level of education


Employment status


Financial capability


Computer skills


Disability


Full or partial assessment

IEEE International Conference on
Multimedia & Expo, Beijing, 2007

9

STATIC

USER

MODEL


IEEE International Conference on
Multimedia & Expo, Beijing, 2007

10

(2)

DYNAMIC MODEL



Records of a user’s
response history

are dynamically modified in real time as
the assessment proceeds.



The answer to each question


The records of questions that are disabled, enabled,
or changed to higher priority

IEEE International Conference on
Multimedia & Expo, Beijing, 2007

11

2.2. COUNSELLING MODEL


Modeling questions


Dividing counselling information into fields and their sub
-
fields


Assigning questions to each sub
-
field


Setting up relations between questions


Authoring instant information


Instant information (including descriptions, links to Web pages with
multimedia resources etc) can be assigned to each choice of a question.
They will show whenever you make a choice.


Pre
-
defined answer types


Yes and No


Yes, No and Not Clear


Grades


Multi
-
Checks

IEEE International Conference on
Multimedia & Expo, Beijing, 2007

12

Questions


&

Instant

Information

IEEE International Conference on
Multimedia & Expo, Beijing, 2007

13

FOUR KINDS OF RELATIONS


Enable


Disable


Plus


Contradictory




Examples:


(
Enable

(
answer
(
i
) of
question
(
j
),




(
question list
) (
sub
-
field list
) ) )


(
Contradict

(
answer
(
i
)

of

question
(
j
),





(list

of

answers

of

some

questions)))



IEEE International Conference on
Multimedia & Expo, Beijing, 2007

14

2.3 PROCESS MODEL

Modeling the behavior that controls the
counseling process



Actions:



Utilizing user model and counseling model


Changing question priority in real time


Sorting questions by priority


Filtering questions



IEEE International Conference on
Multimedia & Expo, Beijing, 2007

15

NEXT PAGE

IEEE International Conference on
Multimedia & Expo, Beijing, 2007

16

2.4 ASSESSMENT

Provides assessment information in response to user’s answers



Two kinds of assessment information:



Question related


Question
-
group related


Example:

(
Assessment
(
i
)






(
answer
(
i
1
)

of

question
(
j
1
)
,






(
answer
(
i
2
)

of

question
(
j
2
)
,











) )

IEEE International Conference on
Multimedia & Expo, Beijing, 2007

18

3. SYSTEM STRUCTURE

Assessme
nt Model
User Model
Process
Model
Process Logic
JavaBean, Servlet or
JSP in Webserver
Browser
Internet
Counseling
Model
Data Access
Interface Flow
Data &
knowledge
Layer
Process &
Interface
Layer
Database
IEEE International Conference on
Multimedia & Expo, Beijing, 2007

19

Process

Interaction

Welcome
User Model
user
Build
Usermodel
Questions &
Instant Info
Treat
Answers
Contradictory
Questions
Consistency
Check
Assessments
start
self-assessment
answer questions
confirm
user information
start first
question page
answers
more questions
contradictories
confirm
if no
contradictories
access
final report
IEEE International Conference on
Multimedia & Expo, Beijing, 2007

20

REALIZATION


Putting the knowledge into PostgreSQL database
(21 tables)


including rules, relations, facts and text information



Determining the question sequences



Reasoning by the support of SQL


Interacting with users


Java/JSP based HTML Webpage Dialog

IEEE International Conference on
Multimedia & Expo, Beijing, 2007

21

4. CONCLUSION

Am I Ready


User oriented & user specific self
-
assessment


Dynamic & adaptive self
-
assessment process


Real time guidance and remedial information to
improve the readiness of students for distance
education


More effective and more natural self
-
assessment
process


Flexible design for more general use



IEEE International Conference on
Multimedia & Expo, Beijing, 2007

22

II. Performance Self
-
Assessment


How can performance self
-
assessment help to
address the different learning styles of students
in learning management systems?



For providing adaptive courses



For improving adaptivity by gathering additional
information about the students


IEEE International Conference on
Multimedia & Expo, Beijing, 2007

23

Motivation


Learning Management Systems (LMS) are often
and successfully used in e
-
education but provide
little or in the most cases no adaptivity



Learners have different needs



Considering learning styles makes learning
easier and increases the learning progress



IEEE International Conference on
Multimedia & Expo, Beijing, 2007

24

Felder
-
Silverman Learning Style Model


Richard M. Felder and Linda K. Silverman, 1988


Each learner has a preference on each of the four
dimensions:


Active


Reflective


learning by doing


learning by thinking things through


learning by discussing & group work


work alone


Sensing


Intuitive


concrete material


abstract material


more practical


more innovative and creative


patient and careful


not patient and careful with details


standard procedures


challenges


Visual


Verbal

learning from pictures


learning from words


Sequential


Global


learn in linear steps


learn in large leaps


good in using partial knowledge


need “big picture”

interested in details


interested in the overview


IEEE International Conference on
Multimedia & Expo, Beijing, 2007

25

Adaptive Features for Self
-
Assessments


Courses can be adapted with respect to the order and
number of included learning objects


Regarding self
-
assessments we consider:


Theoretical tests


Practical exercises


Adaptation features for
theoretical tests


Presentation at the beginning of each chapter


Presentation at the end of each chapter


Presentation at the end of the course


Adaptation features for
practical exercises


Number of presented exercises


Presentation at the beginning of each chapter


Presentation at the end of each chapter

IEEE International Conference on
Multimedia & Expo, Beijing, 2007

26

Adaptation with respect to learning styles


Active learning style:


Theoretical tests and exercises at the beginning and end of each
chapter (due to the preference for active learning)


High number of exercises


Reflective learning style:


First present learning material, then self
-
assessments


students
have time to reflect about the material


Low number of exercises


Sensing learning style:


First present learning material, then self
-
assessments


students
can use the learned material for performing the self
-
assessments


High number of exercises (due to the preference for problem solving)


Intuitive learning styles:


Present self
-
assessments first


provide learners with challenges


Low number of exercises

IEEE International Conference on
Multimedia & Expo, Beijing, 2007

27

Adaptation with respect to learning styles


Sequential learning style:


Avoid theoretical tests at the end of the course (since
sequential learners prefer to test their knowledge in
short intervals)


Theoretical tests at the end of each chapter are more
suitable


Global learning style:


Avoid self
-
assessments at the beginning of the
chapter (since global learners need the big picture for
performing self
-
assessments)


Theoretical tests can be provided at the end of the
course


IEEE International Conference on
Multimedia & Expo, Beijing, 2007

28

Self
-
Assessments for Improving Adaptivity


Providing adaptivity requires knowing the students’
learning styles


Self
-
assessments can be used to gather additional
information from learners


This information can help to identify the learning styles of
students more accurately and therefore enable the
system to provide more suitable adaptivity


Examples for patterns:


Time

students spend on self
-
assessments


sensing/intuitive


Number

of performed exercises


active/reflective


Performance on questions about
multimedia content



visual/verbal


Performance on questions about
details and overview



sequential/global

IEEE International Conference on
Multimedia & Expo, Beijing, 2007

29

III. Peer Assessment


Peer assessment is one form of group
assessment


The issue of fairness has to be concerned in
group assessment


The proposed methodology can aggregate
students’ marks and consider individual learning
styles to


Reduce personal bias



Enhance the accuracy of the assessment

IEEE International Conference on
Multimedia & Expo, Beijing, 2007

30

Adaptive Peer Assessment


The process of adaptive peer assessment


Detect reviewers’ learning styles


Learning Styles
: Active/reflective, sensing/intuitive, visual/verbal, and
sequential/global


Consider assessment criteria and learning styles


Assessment criteria:

Creativity, Completeness, Execution, and
Security


Aggregate all marks and sends the result to the original author


Assessment feedback =
w
1
x
1
+
w
2
x
2
+…
w
i
x
i


Assessment criteria

Weight of learning styles

IEEE International Conference on
Multimedia & Expo, Beijing, 2007

31

Relations between Assessment Criteria and
Learning Styles


Execution

and
Security



Active students tend to be experimentalists


Completeness


Sensing students like solving problems by standard
methods and are patient with detail


Creativity


Intuitive students like innovation

IEEE International Conference on
Multimedia & Expo, Beijing, 2007

32

FUTURE WORK


A knowledge editor to enhance the speed and accuracy of knowledge
modeling, modification and upgrade of AM I READY.



Graphical statistical information representation of readiness in self
-
assessment report.



Data mining for analyzing the question
-
answer pairs and their relations,
and expanding the knowledge base.



A formal knowledge description and a SQL
-
based reasoning method.



Distance education readiness self
-
assessment is only a rudimentary
step to general dynamic and adaptive counseling systems.



More multimedia resources.

IEEE International Conference on
Multimedia & Expo, Beijing, 2007

33

THANK YOU!



Questions?