Development of an Assessment System for

concretecakeUrban and Civil

Nov 29, 2013 (3 years and 11 months ago)

68 views

Development of an Assessment System for
Engineering Courses Using Petri
-
Net

Tung
-
Nan Institute of Technology
, Taiwan*

Tamkang University , Taiwan**


Yu
-
Hur Chou*

Hsin
-
Yih SHYU**




Outline

1.
Objectives

2.
System Architecture


3.
System Analyses and Design


4.
System Demonstrations (Test Demo.)


5.
Conclusions




Objectives

1.
To specify how to design and develop a partial
scoring assessment system.

2.
This assessment system can
make inference
diagnosis

in order to
investigate examinee’s
misconceptions

and
make the reasonable scoring for
engineering courses
.

3.
Reinforced Concrete Design is the objective course.


System Architecture

Test Generator
Test
Database
Subject (Z)
Step 1. Select Test Subject
Test
Database
Subject (M)
Test
Database
Subject (A)
Active Database
Select Test
Concepts
Step 2. Select Test Concepts and style.
Test Items
Answering
Evaluator
Correct
Answers
Concept-Mapping
Reply sub-system
Examinees’
Answers
Diagnostic
Evaluator
Diagnosis
Results
Step 3. User Reply
Step 4. Assessment results
Database System
: Database
: Internal Information
: Sub-system
System Analyses and Design (1/7)

1.
Defining Subject Concepts.


2.
Defining Subject Calculations.

3.
Evaluation between Concepts and Calculations.

4.
Construct the Logic Structure of Correct Answer.

5.
Partial Scoring Function.

6.
Misconception Evaluation
.

RC Beam Analysis (with Single Layer Bar) as example

Procedures:

System Analyses and Design (2/7)

1. Defining Subject Concepts:

Item


Concept name


Item


Concept name


1


Required Resisting Strength


12


Parameter of Equivalent Stress
Block


2


Stress
-
Strain Diagram
(Reinforcement)


13


Equivalent Stress Block


3


Modulus of Elasticity
(Reinforcement)


14


Concept of Under reinforced
Case


4


No. Of Reinforcing Steel


15


Concept of Over reinforced
Case


5


Area of Reinforcing Steel


16


Concept of Balanced Case


6


Modulus of Elasticity (Concrete)


17


Concept of Proportional Ratio


7


Ultimate Strain of Concrete


18


Equilibrium of Forces


8


Compressive Strength of
Concrete


19


Solution of Linear Equations


9


Capacity Reduction Factor


20


Central of Cross
-
Section


10


Nominal Resisting Strength


21


Solution of second order
equation


11


Designed Resisting Strength


System Analyses and Design (3/7)

2. Defining Subject Calculations:

Item


Calculation Name


Item


Calculation Name


1


Balanced Depth of Neutral Axis


12


Depth of Equivalent Stress Block


2


Modulus of Elasticity (Concrete)


13


Depth of Neutral Axis (Balance)


3


Area of Bar


14


Stress of Reinforcement (Balance)


4


Depth of Equivalent Stress Block
(Balance)


15


Stress of Reinforcement (Under
reinforcement)


5


Parameter of Equivalent Stress
Block


16


Nominal Resisting Moment (Tension)


6


Under reinforced Case


17


Nominal Resisting Moment
(Compression)


7


Area of Bar (Balance)


18


Total Tensile Force


8


Balanced Reinforcement Ratio


19


Depth of Neutral Axis (Under
reinforcement)


9


Over reinforced Case


20


Depth of Neutral Axis (Over
reinforcement)


10


Balanced Case


21


Stress of Reinforcement (Over
reinforcement)


11


Total Compressive Force


22


Ultimate Resisting Moment


System Analyses and Design (4/7)

3. Evaluation between Concepts and Calculations:

Relationship between

Q
i

and C
j


Concept C
j


C
1


C
2


C
3


…….


C
n




Calculation

Item

Q
i


Q
1


e
11


e
12


e
13


…….


e
1n


Q
2


e
21


e
22


e
23


…….


e
2n


Q
3


e
31


e
32


e
33


…….


e
3n


….


….


….


….


…….


….


Q
m


e
m1


e
m2


e
m3


…….


e
mn


e
ij

= 1 (related ) , e
ij

= 0 (irrelated )


System Analyses and Design (5/7)

4. Construct the Logic Structure of Correct Answer:

Total Tensile Force
T
A
s
f
s
s
s
A
f
T


Parameter
2
Par amet er Set
Calculation 1
Calculation 2
Calculation 3
Calculation m
Calculation Set
Input Arcs
Output Arcs
Parameter
1
Parameter
3
Parameter
4
Parameter
N

An independent unit

The Rule
-
based Petri
-
Net


The model of the Rule
-
based Petri
-
Net

System Analyses and Design (6/7)

5. Partial Scoring Function:

Score = WC
x

GCI + WS

x

CRC

Where

Score: Final score.

GCI: The value of
G
oldsmith’s
C
loseness
I
ndex (Similarity between correct and


examiner's concept map).

CRC: The value of
C
orrect
R
ate of
C
alculations.

WC: Weight of GCI (0~1).

WS: Weight of CRC (1
-
WC).


CRC =
Σ(
P
i

x D
i
) /
Σ
D
i

( i = 1 ~ m )

Where

CRC: The value of Correct Rate of Calculations.

P
i
: The value of each calculation (0 or 1).

D
i
: The difficulty value of each calculation (1(easy) ~9(hard)).

m: The number of calculations within a test item.

System Analyses and Design (7/7)

6. Misconception Evaluation:

(1) Complete correctness:

(2) No reply, but system infers complete correctness:

(3) Correct calculation (wrong answer from mistake inheritance):

(4) No reply, but system infers correct calculation:


(5) Wrong answer:

(6) No reply:


Calculation judgment


CRV
(i)

= WC
x

C
(i)

+ WS
x

P
i

( i = 1 ~ m )


Where

CRV
(i)
: The
C
omplete
R
ating
V
alue of calculation i.

C
(i)
: The closeness coefficient of calculation i.


System Demonstrations (1/8)

Entry Screen (Select the Test Subject)


Test Generator
Step 1. Select Test Subject
Test
Database
Subject (M)
Active Database
Select Test
Concepts
Step 2. Select Test Concepts and style.
Test Items
Answering
Evaluator
Correct
Answers
Concept-Mapping
Reply sub-system
Examinees’
Answers
Diagnostic
Evaluator
Diagnosis
Results
Step 3. User Reply
Step 4. Assessment results
System Demonstrations (2/8)

Select Concepts and Test Style


Test Generator
Step 1. Select Test Subject
Test
Database
Subject (M)
Active Database
Select Test
Concepts
Step 2. Select Test Concepts and style.
Test Items
Answering
Evaluator
Correct
Answers
Concept-Mapping
Reply sub-system
Examinees’
Answers
Diagnostic
Evaluator
Diagnosis
Results
Step 3. User Reply
Step 4. Assessment results
System Demonstrations (3/8)

Problem’s Description

Test Generator
Step 1. Select Test Subject
Test
Database
Subject (M)
Active Database
Select Test
Concepts
Step 2. Select Test Concepts and style.
Test Items
Answering
Evaluator
Correct
Answers
Concept-Mapping
Reply sub-system
Examinees’
Answers
Diagnostic
Evaluator
Diagnosis
Results
Step 3. User Reply
Step 4. Assessment results
System Demonstrations (4/8)

Answer Evaluator

Test Generator
Step 1. Select Test Subject
Test
Database
Subject (M)
Active Database
Select Test
Concepts
Step 2. Select Test Concepts and style.
Test Items
Answering
Evaluator
Correct
Answers
Concept-Mapping
Reply sub-system
Examinees’
Answers
Diagnostic
Evaluator
Diagnosis
Results
Step 3. User Reply
Step 4. Assessment results
System Demonstrations (5/8)

Concept Mapping Reply System

Test Generator
Step 1. Select Test Subject
Test
Database
Subject (M)
Active Database
Select Test
Concepts
Step 2. Select Test Concepts and style.
Test Items
Answering
Evaluator
Correct
Answers
Concept-Mapping
Reply sub-system
Examinees’
Answers
Diagnostic
Evaluator
Diagnosis
Results
Step 3. User Reply
Step 4. Assessment results
System Demonstrations (6/8)

Assessment Results

Test Generator
Step 1. Select Test Subject
Test
Database
Subject (M)
Active Database
Select Test
Concepts
Step 2. Select Test Concepts and style.
Test Items
Answering
Evaluator
Correct
Answers
Concept-Mapping
Reply sub-system
Examinees’
Answers
Diagnostic
Evaluator
Diagnosis
Results
Step 3. User Reply
Step 4. Assessment results
System Demonstrations (7/8)

Scoring Instruction

Test Generator
Step 1. Select Test Subject
Test
Database
Subject (M)
Active Database
Select Test
Concepts
Step 2. Select Test Concepts and style.
Test Items
Answering
Evaluator
Correct
Answers
Concept-Mapping
Reply sub-system
Examinees’
Answers
Diagnostic
Evaluator
Diagnosis
Results
Step 3. User Reply
Step 4. Assessment results
System Demonstrations (8/8)

Misconception Evaluation

Test Demonstration (1/6)

Step 1: Select Test Subject.

Step 2: Select Test Concepts and Style.

Test Demonstration (2/6)

Step 3 : Problem description (generated by system)

Correct Answer

Test Demonstration (3/6)

Step 4: User’s Reply Answer

36.43 cm

0.85

30.97 cm

86.99 cm
2

21.85 cm

159.18 T

1(True)

36.42 cm

0.8

29.14 cm

81.87 cm
2

23.21 cm

159.21 T

Step 5: Assessment Begin

Test Demonstration (4/6)

Step 6: Scoring Instruction

Test Demonstration (5/6)

Step 7: Diagnosis Instruction

Test Demonstration (6/6)

Conclusions

1.
This assessment system adopts the dynamic state to make
test items.
The values of the
test variables are generated
randomly
. Therefore, the same test problem will be shown
within different variable values for different tests, to prevent
students from memorizing the answers.


2.
We also purpose a
Rule
-
based Petri
-
Net model

for building
the logic structure of correct answer. This logic structure
can match the dynamic pattern and random parameters
completely.


3.
This system also adopts
concept
-
mapping

environment as
examiner’s reply system, can acquire enough information to
investigate examinee’s misconceptions and measure final
score by using
partial scoring strategy
.


The End

Thanks