# Development of an Assessment System for

Urban and Civil

Nov 29, 2013 (4 years and 7 months ago)

80 views

Development of an Assessment System for
Engineering Courses Using Petri
-
Net

Tung
-
Nan Institute of Technology
, Taiwan*

Tamkang University , Taiwan**

Yu
-
Hur Chou*

Hsin
-
Yih SHYU**

Outline

1.
Objectives

2.
System Architecture

3.
System Analyses and Design

4.
System Demonstrations (Test Demo.)

5.
Conclusions

Objectives

1.
To specify how to design and develop a partial
scoring assessment system.

2.
This assessment system can
make inference
diagnosis

in order to
investigate examinee’s
misconceptions

and
make the reasonable scoring for
engineering courses
.

3.
Reinforced Concrete Design is the objective course.

System Architecture

Test Generator
Test
Database
Subject (Z)
Step 1. Select Test Subject
Test
Database
Subject (M)
Test
Database
Subject (A)
Active Database
Select Test
Concepts
Step 2. Select Test Concepts and style.
Test Items
Evaluator
Correct
Concept-Mapping
Examinees’
Diagnostic
Evaluator
Diagnosis
Results
Step 4. Assessment results
Database System
: Database
: Internal Information
: Sub-system
System Analyses and Design (1/7)

1.
Defining Subject Concepts.

2.
Defining Subject Calculations.

3.
Evaluation between Concepts and Calculations.

4.
Construct the Logic Structure of Correct Answer.

5.
Partial Scoring Function.

6.
Misconception Evaluation
.

RC Beam Analysis (with Single Layer Bar) as example

Procedures:

System Analyses and Design (2/7)

1. Defining Subject Concepts:

Item

Concept name

Item

Concept name

1

Required Resisting Strength

12

Parameter of Equivalent Stress
Block

2

Stress
-
Strain Diagram
(Reinforcement)

13

Equivalent Stress Block

3

Modulus of Elasticity
(Reinforcement)

14

Concept of Under reinforced
Case

4

No. Of Reinforcing Steel

15

Concept of Over reinforced
Case

5

Area of Reinforcing Steel

16

Concept of Balanced Case

6

Modulus of Elasticity (Concrete)

17

Concept of Proportional Ratio

7

Ultimate Strain of Concrete

18

Equilibrium of Forces

8

Compressive Strength of
Concrete

19

Solution of Linear Equations

9

Capacity Reduction Factor

20

Central of Cross
-
Section

10

Nominal Resisting Strength

21

Solution of second order
equation

11

Designed Resisting Strength

System Analyses and Design (3/7)

2. Defining Subject Calculations:

Item

Calculation Name

Item

Calculation Name

1

Balanced Depth of Neutral Axis

12

Depth of Equivalent Stress Block

2

Modulus of Elasticity (Concrete)

13

Depth of Neutral Axis (Balance)

3

Area of Bar

14

Stress of Reinforcement (Balance)

4

Depth of Equivalent Stress Block
(Balance)

15

Stress of Reinforcement (Under
reinforcement)

5

Parameter of Equivalent Stress
Block

16

Nominal Resisting Moment (Tension)

6

Under reinforced Case

17

Nominal Resisting Moment
(Compression)

7

Area of Bar (Balance)

18

Total Tensile Force

8

Balanced Reinforcement Ratio

19

Depth of Neutral Axis (Under
reinforcement)

9

Over reinforced Case

20

Depth of Neutral Axis (Over
reinforcement)

10

Balanced Case

21

Stress of Reinforcement (Over
reinforcement)

11

Total Compressive Force

22

Ultimate Resisting Moment

System Analyses and Design (4/7)

3. Evaluation between Concepts and Calculations:

Relationship between

Q
i

and C
j

Concept C
j

C
1

C
2

C
3

…….

C
n

Calculation

Item

Q
i

Q
1

e
11

e
12

e
13

…….

e
1n

Q
2

e
21

e
22

e
23

…….

e
2n

Q
3

e
31

e
32

e
33

…….

e
3n

….

….

….

….

…….

….

Q
m

e
m1

e
m2

e
m3

…….

e
mn

e
ij

= 1 (related ) , e
ij

= 0 (irrelated )

System Analyses and Design (5/7)

4. Construct the Logic Structure of Correct Answer:

Total Tensile Force
T
A
s
f
s
s
s
A
f
T

Parameter
2
Par amet er Set
Calculation 1
Calculation 2
Calculation 3
Calculation m
Calculation Set
Input Arcs
Output Arcs
Parameter
1
Parameter
3
Parameter
4
Parameter
N

An independent unit

The Rule
-
based Petri
-
Net

The model of the Rule
-
based Petri
-
Net

System Analyses and Design (6/7)

5. Partial Scoring Function:

Score = WC
x

GCI + WS

x

CRC

Where

Score: Final score.

GCI: The value of
G
oldsmith’s
C
loseness
I
ndex (Similarity between correct and

examiner's concept map).

CRC: The value of
C
orrect
R
ate of
C
alculations.

WC: Weight of GCI (0~1).

WS: Weight of CRC (1
-
WC).

CRC =
Σ(
P
i

x D
i
) /
Σ
D
i

( i = 1 ~ m )

Where

CRC: The value of Correct Rate of Calculations.

P
i
: The value of each calculation (0 or 1).

D
i
: The difficulty value of each calculation (1(easy) ~9(hard)).

m: The number of calculations within a test item.

System Analyses and Design (7/7)

6. Misconception Evaluation:

(1) Complete correctness:

(2) No reply, but system infers complete correctness:

(3) Correct calculation (wrong answer from mistake inheritance):

(4) No reply, but system infers correct calculation:

Calculation judgment

CRV
(i)

= WC
x

C
(i)

+ WS
x

P
i

( i = 1 ~ m )

Where

CRV
(i)
: The
C
omplete
R
ating
V
alue of calculation i.

C
(i)
: The closeness coefficient of calculation i.

System Demonstrations (1/8)

Entry Screen (Select the Test Subject)

Test Generator
Step 1. Select Test Subject
Test
Database
Subject (M)
Active Database
Select Test
Concepts
Step 2. Select Test Concepts and style.
Test Items
Evaluator
Correct
Concept-Mapping
Examinees’
Diagnostic
Evaluator
Diagnosis
Results
Step 4. Assessment results
System Demonstrations (2/8)

Select Concepts and Test Style

Test Generator
Step 1. Select Test Subject
Test
Database
Subject (M)
Active Database
Select Test
Concepts
Step 2. Select Test Concepts and style.
Test Items
Evaluator
Correct
Concept-Mapping
Examinees’
Diagnostic
Evaluator
Diagnosis
Results
Step 4. Assessment results
System Demonstrations (3/8)

Problem’s Description

Test Generator
Step 1. Select Test Subject
Test
Database
Subject (M)
Active Database
Select Test
Concepts
Step 2. Select Test Concepts and style.
Test Items
Evaluator
Correct
Concept-Mapping
Examinees’
Diagnostic
Evaluator
Diagnosis
Results
Step 4. Assessment results
System Demonstrations (4/8)

Test Generator
Step 1. Select Test Subject
Test
Database
Subject (M)
Active Database
Select Test
Concepts
Step 2. Select Test Concepts and style.
Test Items
Evaluator
Correct
Concept-Mapping
Examinees’
Diagnostic
Evaluator
Diagnosis
Results
Step 4. Assessment results
System Demonstrations (5/8)

Test Generator
Step 1. Select Test Subject
Test
Database
Subject (M)
Active Database
Select Test
Concepts
Step 2. Select Test Concepts and style.
Test Items
Evaluator
Correct
Concept-Mapping
Examinees’
Diagnostic
Evaluator
Diagnosis
Results
Step 4. Assessment results
System Demonstrations (6/8)

Assessment Results

Test Generator
Step 1. Select Test Subject
Test
Database
Subject (M)
Active Database
Select Test
Concepts
Step 2. Select Test Concepts and style.
Test Items
Evaluator
Correct
Concept-Mapping
Examinees’
Diagnostic
Evaluator
Diagnosis
Results
Step 4. Assessment results
System Demonstrations (7/8)

Scoring Instruction

Test Generator
Step 1. Select Test Subject
Test
Database
Subject (M)
Active Database
Select Test
Concepts
Step 2. Select Test Concepts and style.
Test Items
Evaluator
Correct
Concept-Mapping
Examinees’
Diagnostic
Evaluator
Diagnosis
Results
Step 4. Assessment results
System Demonstrations (8/8)

Misconception Evaluation

Test Demonstration (1/6)

Step 1: Select Test Subject.

Step 2: Select Test Concepts and Style.

Test Demonstration (2/6)

Step 3 : Problem description (generated by system)

Test Demonstration (3/6)

36.43 cm

0.85

30.97 cm

86.99 cm
2

21.85 cm

159.18 T

1(True)

36.42 cm

0.8

29.14 cm

81.87 cm
2

23.21 cm

159.21 T

Step 5: Assessment Begin

Test Demonstration (4/6)

Step 6: Scoring Instruction

Test Demonstration (5/6)

Step 7: Diagnosis Instruction

Test Demonstration (6/6)

Conclusions

1.
This assessment system adopts the dynamic state to make
test items.
The values of the
test variables are generated
randomly
. Therefore, the same test problem will be shown
within different variable values for different tests, to prevent

2.
We also purpose a
Rule
-
based Petri
-
Net model

for building
the logic structure of correct answer. This logic structure
can match the dynamic pattern and random parameters
completely.

3.
concept
-
mapping

environment as
examiner’s reply system, can acquire enough information to
investigate examinee’s misconceptions and measure final
score by using
partial scoring strategy
.

The End

Thanks