Methodological strategies integrated to computational tools: Improvement of management of the teaching-learning process

fancyfantasicΤεχνίτη Νοημοσύνη και Ρομποτική

7 Νοε 2013 (πριν από 3 χρόνια και 8 μήνες)

70 εμφανίσεις




Methodological strategies integrated to computational tools:
Improvement of management of the teaching
-
learning process



Dilermando Piva Jr.
1
, Rosana G. S. Miskulin
2


1
Centro Estadual de E
ducação Tecnológica Paula Souza
, São Paulo


SP
-

Brazil


2
IGC
E
-
UNESP
-
RIO CLARO, São Paulo


SP


Brazil


pivajr@gmail.com, misk@rc.unesp.br



Abstract:
This work presents part of a result of

the doctoral thesis that has been
developed

a computational tool, called AUXILIAR, designed to
i
mprovement of
management of th
e teaching
-
learning process, to monitor courses, and to redirect the
students learning process in online courses. The system shall decrease the teachers’ time
and effort to manage the classes and the students learning process, measuring the students
learni
ng levels to assure the teaching quality. The system AUXILIAR is composed by
two modules. The first one contains an edition, diagramming, and publication tool for
online courses, called CONSTRUCTOR. The second one, called REASONING, manages
the path follow
ed by each student when exploring the courses contents. It is also
responsible to automatically detect and redirect the student, depending on his/her
deficiencies, detected by the followed path and by formative evaluations. To achieve this,

it uses a Case
-
Based System

which stores, adapts and reuses previous experiences to new
situations
.




1. Introduction


In the last decade, the use of the internet and wide band access have supported the spread of distance learning
through the web, which we will call Onl
ine Teaching. Schools and universities have been rethinking their
teaching practices and educational policies in establishing online teaching programs. Online Teaching
becomes not only a new pedagogical model or an educational technology, but also a new so
cial model and
technology, gathering each time more students at decreasing costs.


Nevertheless, Online teaching is far from achieving its maximum potentialities. Personal, methodological,
technological, and institutional restrictions are usually associat
ed with important limitations.


As an example, the number of students that can be attended by a teacher/mediator is severely restricted by
his/her capacity of mediation, orientation, and conduction of the teaching proposed practices in an online
environmen
t. Clearly, this may impose serious restrictions to a major growth of the number of students
attended and/or the quality of the learning process.


Among the studies that relate obstacles to the effective use and spread of technology in online courses, we
c
ould include Pajo and Wallace (2001) that pointed as the main restrictions: 1) the time required to learn how
to use the technology; 2) the time required to develop and to implement web
-
based courses; and 3) the time
required to use the Online Teaching env
ironment, and to monitor the course, e.g. giving feedback to the
students.


At a similar perspective, James and Beattie (1996) had pointed as the main obstacles: 1) the time required to
the management of the class and the students; 2) the time required to
produce good quality teaching
materials; and 3) the comparative reward in teaching at distance when compared to traditional teaching.




Similar results are also present in (
D
a
ugherty & F
unke
, 1998), (
Metcalf
, 1997), (
Hare &
Mccartan
, 1996),
(Piva Jr et al.
,

2002), among others.


The minimization of such obstacles when using computational tools becomes of major concern. Moreover,
methodological strategies that could be integrated to computational tools focusing on a more efficient
management of the teaching
-
l
earning process, minimizing the needed effort of the teacher/mediator without
worsening the learning levels, would also be highly desirable.


This work presents a computational tool, called AUXILIAR, designed to aid the teacher to manage the
classes, to m
onitor courses, and to conduct and redirect the students learning process in online courses. The
system shall decrease the teachers’ time and effort to manage the classes and the students learning process,
measuring the students learning levels to assure t
he teaching quality. Two modules compose the system
AUXILIAR. The first one contains an edition, diagramming, and publication tool for online courses, called
CONSTRUCTOR. The second one, called REASONING, manages the path followed by each student when
expl
oring the courses contents. It is also responsible to automatically detect and redirect the student,
depending on his/her deficiencies, detected by the followed path and by formative evaluations. To achieve
this, it uses a Case
-
Based System
(Kolodner, 1993
)
, which stores, adapts and reuses previous experiences to
new situations
.


2. The Teaching Model


In AUXILIAR, concepts may be exposed through different instructional materials, and have their own
evaluation method. A simplified model of contents organiza
tion is shown in Figure 1. It presents
synthetically how the system inferences kernel conducts the evaluation process for each studied concept. As
shown, each concept is composed by three basic parts: the Pedagogical Proposal; the Pedagogical Contents
and
Media; and Questions for Apprenticeship Checking.






Concept
n















Media:



Texts

Voice

Animations

Images

Pictures,
etc.



Pedagogical Content



Pedagogical Proposal

Questions



Knowledge / Compre
hension



Q1



Application / Analysis



Q2



Synthesis / Evaluation



Q3



Questions B
ase



Pedagogical Contents B
ase






Pedagogical Objectives

e

Abilities and Competences


Evaluation Criteria



Ex
ample
:

3

quest
ions of t
ype
Q1



2

questions of type
Q2



2


q
uest
ions of type
Q3





Figure 1


Pedagogical contents and questions for each concept.


All concepts are stored in Knowledge Bases, making their maintenance and retrieval easier. The retrieval
proces
s, based on the student profile, uses a Case
-
Based System. This facilitates the symmetric migration
among concepts, in a given apprenticeship problem (Piva Jr., 2006).


The teacher provides what he/she considers the most adequate evaluation criteria for ea
ch concept, which
will be used to automatically generate the evaluation. To better understand how it works, we present below a
brief description of the system modules.





2.1 The CONSTRUCTOR Module


The CONSTRUCTOR Module helps to better organize the course

contents, leading to more interactivity
between the student and the system. It avoids static contents, which Khalifa and Lam (2002) called DPL


distributed passive learning



in favor more efficient dynamic contents, called DIL


distributed interactive
learning
, such as hypertext structures.


One of the principles that conducted the development of this module was the semiotic language, which is
implicit in the online teaching process. There are several definitions to the term
semiotic
. In the AUXILIAR
sy
stem, we used the one proposed by
Charles Sanders Peirce (1839
-
1914), which is a very comprehensive
one, and directed to our work. For Peirce, semiotic is a formal doctrine of the signs, and a sign is something
that represents something to someone, under s
pecific aspects or capacities. Since knowing and knowledge
are inseparable, semiotic tries to explain how humans construct meaning from their interactions with the
signs available in the world. The need to build meaning is inherent to all human beings. And

man learns
through their interaction with the world (i.e. with the signs) (Nöth, 1998).


This way, concepts from various modes of signs


images, sounds, gestures, intonations and other non
-
verbal
manifestations


have fundamental importance in online tea
ching, because they are also constructors of
superior and cultural mental functions. According to Pierre Lévy (1993), “the interface contributes to define
the way of the information capturing offered to the communication actors. It opens, closes, and guide
s the
signification domains, of the possible uses of the media”.


Thus, different kinds of didactic materials are allowed to be included by the teacher in the CONSTRUCTOR
Module. Following the proposed Teaching Methodology, each course’s content is divided

into modules.
Each module has a series of concepts and, at the end of each concept there is an evaluation to check the
apprenticeship.


After entering a concept, the teacher will enter different questions regarding that concept, and how the
concept will b
e evaluated. He/she will include all the didactic material, and then just chose the option to end
the course production, so that the system will generate the web pages. The course edition ends when the
teacher liberates the material for internet publicatio
n.



2.2 The REASONING Module


Figure 2 illustrates, in a simplified way, the functioning of the REASONING Module. The students start the
Instruction Module at Level 1, developing Concept 1. After the development of this concept, a formative
evaluation is
performed, regarding the central issues of the concept. This evaluation is retrieved from the
questions base, following the validation criteria entered by the teacher. If the student does not succeed, he
will be conducted to Level 2. On Level 2, the same c
oncept will be presented in a more detailed way,
through sub
-
concepts. This way, Level 1 represents the student starting point, and the levels above contain
the same concept in more detail. If the student succeeds with Concept 1, he will be conducted to Co
ncept 2.


During this redirecting process, the Student Model starts to be established to provide ways to individualize
the teaching
-
learning process. The system registers the student’s learning path, his knowledge level (which
concept he is studying), his

capacities (which questions he solved), his attitudes (which questions were
solved through adaptations or other kinds of help asked), and his limitations (questions and types of
questions he had problems to solve). These attributes are part of the student

profile, and will be used, for
example, when the instruction modules are not sufficient to a specific content learning. In this case, they are
sent to the REASONING Module Modules. This module will look for a case, in the case
-
base, that fits the
student
profile. If there is such a case, it will be presented to the student and, after this a new evaluation will
be conducted. If the student succeeds, he/she will be directed to the next concept, in the first level. If not, he
will be sent to a direct teacher
intervention.




All the teacher interventions are also registered. They are used to establish new pedagogical cases, being
stored in the Case Base. During this process, the system tries to redirect the student to finish the Instruction
Module through the sh
ortest path.




Figure 2: Relationship among the Course, Case Base, and the Teacher.



2.3 The Student Model


Several techniques to establish a student model have been presented, but most of them are computationally
complex. As an example, we can cite the

numerical techniques (Jameson 1996), where there are three main
paradigms: the Bayesian Networks (Villano 1992), the Theory of Evidence of Dempster
-
Shafer (Bauer
1996), and the fuzzy approach (Hawkes et al, 199). Some computationally simpler approaches, s
uch as
modeling through approximation trace (Anderson et al
.,

1995), show important restrictions, such as not
including the student behavior or characteristics.


We propose a Student Model where the student information, gathered during all the learning pro
cess, is
considered to characterize his/her profile. As cited before, a Case
-
Based System carries out this task,
managing the student data. This feature enables more individual interventions by the system and the
teacher/monitor. Since the information rega
rding the students profiles are stored and retrieved from a case
-
base, and reused in similar future cases, complex inference algorithms are not necessary.


2.4 Elements of the Student Model




The Student Model holds the knowledge about the student, allowing

the case
-
based system to use it and to
provide individual feedback to each student profile. The representation of the Student Model, adapted to a
Problem
-
Solution model as often used in case
-
based reasoning, is illustrated in Table 1.


Table 1: Case struc
ture


C

A

S

E

n

Problem (symptoms)



Knowledge level
: {course, module, and concept code}



Capacitie
s
: {correctly answered exercises}



Limitation
s
: {wrongly answered exercises or difficulties observed}



Apprenticeship Path
: {concepts and sub
-
concepts codes studi
ed in each instruction
module}



Attitudes
: {solved exercises using adaptations or help}

Solution



Teacher/monitor intervention
: {video, chat, e
-
mail, etc.}


The
Problem

(symptoms) is composed by a series of attributes used as index information for a futur
e use of
the
Solution.
The Solution contains the material (automatically) sent and used to improve the concept
learning by the student. The cases are structured hierarchically in a case
-
base, allowing faster data access.
Figure 3 illustrates its structure.




Figure 3: Hierarchical Case
-
Based Structure



2.5 The Case Retrieval Process


When a student presents a learning deficiency, the system looks for a similar past situation, in which the
system was able to suggest successful teaching alternatives for t
hat student profile. The retrieval of a similar
case occurs in three steps. In the first step, the system establishes the student knowledge level, through the
set of attributes (Course + Module + Concept), which we call segment. The segment is associated w
ith a set
of relevant cases for the present situation, and only the cases stored in that segment will be considered,
improving retrieval performance. In the second step, the segment cases are checked, considering the
attributes
Limitations
and
Capacities
.
In
Limitations
, a list of wrongly answered questions in the evaluations
is stored, and in
Capacities

there is a list of correctly answered exercises. For a better understanding of an
evaluation, Figure 4 brings an example. The evaluation showed is composed

by three sets (T1, T2, and T3),
each one containing five questions. In order to automatically compose each set, the system will take one


question of each of five different groups of questions (Q1
-
Q5), at random. For each group, the teacher has
already ent
ered several equivalent questions. Question in set T1 are multiple
-
choice, and will be corrected by
the system. Questions in set T2 are conceptual and discursive, being corrected by the teacher. Questions in
set T3 are analytical and discursive, being also

corrected by the teacher. A
Capacities
list
{T1Q1,T1Q3,T2Q4} indicates that the student answered correctly questions of groups 1 and 3 of set T1, and
a question of group 4 of set T2. The
Limitations
list is constructed similarly, with the incorrectly answ
ered
questions.




Figure 4


Evaluation composing in system AUXILIAR


The association between
Capacities

and
Limitations
conducts to a specific cases space, where there can be
zero or more cases. Figure 5 illustrates these situations. When there is more
than one case in the case space,
the “best” case should be chosen. Its choice is performed using a very well
-
known similarity method
(“nearest neighbor”). To accomplish this, a similarity degree is calculated between each case in the case
space and the cur
rent situation. The attributed considered are the ones contained in the
Apprenticeship Path
and
Attitudes
. The similarity degree is obtained as follows:


n


Similarity(C,B) =


[

i

-
B
i
)
x
w
i

]/W



i=1

where:
C

is the current case


B

is the case being considered

w

is the weight of the attribute



i

is the index of each attribute




W

is the sum of all attributes weights





Capacities



Limitations



C1



Cn



C4



C5



C8



Cn



C7



C6



C2



Cn



C5



C

n



C1



C3



C9




Figure 5


Cases Retrieval in syst
em AUXILIAR





As a result, we have a list of all cases in the case space with their corresponding similarity degrees when
compared to the current case. The retrieved case is the one with highest similarity degree.


3. Empirical Tests and Results


Two diff
erent groups of tests were performed to verify the adequacy of the system to minimize existing
issues in online courses. In particular, the tests aimed at verifying the impacts on learning results from the
time reduction in the teacher/mediator efforts obt
ained by the use of AUXILIAR.


The first experience was performed in the first period of 2011, involving 48 students of a class of
Introduction to Computer Architectures

from the Computer Science Course at the Faculty of Technology of
Indaiatuba. We had pr
evious scores of the students from two prior classes (2009 and 2010) of the same
course.


The course was divided in two parts, each one with approximately two and a half months. Table 2 presents
the average scores of the students for each part.


Table 2:
Average scores of the students: first experience

Year

Number of students

Average score in
the first part

Average score in
the second part

Difference
(%)

2009

67

5.92

7.15

20.77

2010

52

5.81

6.96

19.79

2011

48

5.60

8.25

47.55


In the three classes, th
e average scores were higher in the second part of the course. In the class of 2011,
when the system AUXILIAR was applied to support the traditional classes, a greater increase in the students’
performance could be noticed. The average score of 8.25 is sig
nificantly higher than 7.15 and 6.96, the
scores obtained by the previous classes. Moreover, if we had a hypothetical increase of 20% in the scores (as
observed in the previous classes), we would have noticed an average score of 6.72 in the second part of
the
course, very far from the 8.25 observed.


The second experience was conducted in the second period of 2011. It was carried out with two classes of
Introduction to Computer Science
, one with Computer Engineering students in the Catholic University of
Ca
mpinas, and the other with Technology in Informatics students, at the Faculty of Technology of
Indaiatuba. The course content was also divided in two parts.


In this experience, we divided each class in four groups. Group I took both parts of the course o
nline. Group
II took only part one online. Group III took only part two online. Finally, Group IV did not had access to the
online tool. The students of all groups were also present to traditional classes.


Table 3 summarizes the results obtained in this e
xperience.



Table 3: Average scores of the students: second experience

Group

Number of
students

Average score in
the first part

Average score in
the second part

Students
average score

I

13

5.3

8.1

6.7

II

11

6.0

6.4

6.2

III

26

3.7

7.3

5.5

IV

21

3.1

5
.6

4.3


The results obtained by the students who also took the online course were significantly better than the
students who took only the traditional classes. Figure 6 also presents the results.




Figure 6: Results of the second experience


Students of
group I (who took both parts of the course online) obtained the best results in both parts of the
course. Students of group IV (who did not take the course online) had the lowest scores in both parts of the
course. Students of groups II and III had a much
better comparative performance in the course part in which
they participated in the online course, independently of being in the first or in the second part of the course.


The analysis of cases retrieval also brought valuable evidences. The case
-
based to
ol retrieved pertinent past
cases for the students with difficulties in learning the concepts in 93.18% of all situations. Moreover, in
75.61% of the cases, the retrieved cases were effective in helping the students, i.e., they contributed for the
concept
learning and the student answered correctly the question in the following level. In all these cases
(75.61%), the teacher/monitor intervention was not necessary. Figure 7 details the system case retrieval.


0,00%
20,00%
40,00%
60,00%
80,00%
100,00%
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
Dias
Cases Retrieval - 2o. Sem. 2006 - AUXILIAR System
% Retrieval
% Retrieval Effectiveness

Figure 7: Cases retrieval in the second experi
ence.


At the end of each course, the students were asked about the adequacy of the system AUXILIAR to improve
the teaching
-
learning process. In the first experience, 93% of the students considered positive and relevant
the tool use, and 97% of the student
s of the second experience had the same opinion.


4. Final Remarks


Improving the teaching
-
learning process simultaneously with saving the teacher/monitor time are very
desirable characteristics of an online teaching system. These characteristics may help
spreading the use of
such systems without too much increase of human resources, keeping or improving the apprenticeship levels.


The system AUXILIAR contribution resides in conciliating these characteristics, reducing the
teacher/monitor effort, allowing a

better student attendance and, at the same time, providing an adequate
scenario for learning and contents exploiting.


The system was able to improve teaching processes efficiency in Computer Science courses, evidenced by
increases in average students’ sc
ores and satisfaction. This improvement was due not only to the structure of
the online teaching method itself, but also to the performance of the cases retrieval, which raised from
23.47% of cases retrieval efficiency in the first week to 82.22% in the l
ast week, with an average


performance of 75.61%. The increased system performance over time through continuous training allows the
diminishing of teacher efforts in assisting students and in redirecting their learning paths.


5. Future Work


Implementing l
arge scale online teaching is a challenge which will not be surpassed only with the
development of computational systems that support the classes’ management. Many questions regarding
human and technological aspects related to teaching
-
learning processes i
n virtual contexts need to be
answered.



6. Referências


Anderson
, J.R.

&

Corbett
, A.T.
&

Koedinger
, K

&

Pelletier
, R.

(1995).

Cognitive tutors: Lessons learned.
The Journal of the Learning Sciences

v. 4, n.2, p. 167
-
207
.

Bauer
, M.
(1996).
A dempster
-
shafe
r approach to modeling agent references for plan recognition.
User
Modeling and User
-
Adapted Interaction
, n.5, p. 317
-
348
.

Daugherty
, M.

&

Funke
, B.L.

(1998).

University Faculty and Student Perceptions of Web
-
Based Instruction.
Journal of Distance Educatio
n
, v. 13, n.1, p. 21
-
39
.

Freitas
, R.L.

(1996).

Um Sistema de Planejamento de Ações Baseado em Casos para uma Célula
Flexível de Manufatura

(An Action Case
-
Based Planning System used in a Flexible Manufacturing
Cell).
Doctor Dissertation, University of São
Pa
ulo, São Paulo, SP, Brazil
.

Hare
, C.

&

Mccartan
, A.

(1996).

Maximizing resources in search of quality: Identifying factors to enable
thee integrative use of IT in teaching and learning.
Innovations in Education and Training
International
,
v. 33, n.4, p.
178
-
184
.

Hawkes
, L.W.

&

Derry
, S.J.;
Rundensteiner
, E.A.

(1990).

Individualized tutoring using an intelligent fuzzy
temporal relational database.
International Journal of Man
-
Machine Studies
, n.33, p. 409
-
429
.

James
, R.

&

Beattie
, K.

(1996).

Postgraduate c
oursework beyond the classroom: Issues in implementing
flexible delivery
.

Distance Education
, v. 17, n.2, p. 355
-
368
.

Jameson
, A.
(1996).
Numerical uncertainty management in user and student modeling: An overview of
systems and issues.
User Modeling and Us
er Adapter Interaction
, n.5, p. 193
-
251
.

Khalifa
, M.

&

Lam
, R.

(2002).

Web
-
Based Learning: effects on learning process and outcomes.
IEEE
Transactions on Education
. v. 45, n.4
.

Kolodner
, J.L.

(1993).

Case
-
Based Reasoning
. San Mateo, CA: Morgan Kaufmann
Pu
blishers
.

Lèvy
, P.

(1993).

As tecnologias da inteligência: o futuro do pensamento na era da informática
(Technologies of Intelligence: the future of thinking in the information era)
.
Rio de J
aneiro, RJ: Ed.
34, Brazil
.

Metcalf
, T.

(1997).
Distance educati
on: the issue of faculty time.
5
th

Annual Distance Education
Conference: 1997 Conference Proceedings
.
T
exas A&M, Center f
or Distance Education Research
.

Nöth
, W. (1998)
Panorama da Semiótica de Platão a Peirce (A Panorama from Plato to Peirce)
.
2ª. Ed.,

o Paulo:Annablumme, Brazil, pp. 149.

Pajo
, K. &

Wallace
, C.

(2001).

Barriers to the uptake of web
-
based technology by university teachers.
Journal of Distance Education
, v. 16, n.1, p. 70
-
84
.

Petrushin
, V.A.

&

Sinista
, K.M.

(1993).

Using probabilistic reas
oning techniques for leaner modeling.
World Conference on AI in Education
, Edinb
urgh, Scotland, p. 418
-
425
.

Piva Jr
., D.

(2006).

AUXILIAR: Uma Ferramenta Computacional Inteligente que Potencializa a Ação
Docente em Módulos de Ensino de Engenharia em Cursos

Online (An Intelligent Computational
Tool to Improve Teachers Actions in Online Engineering Courses)
.

Doctor Dissertation, Univ
ersity
of Campinas, Brazil
.

Piva Jr
., D.;
Gonçalves Jr
., G.
(2002).
Obstáculos encontrados pelos Professores da Área de Engenhar
ia na
Condução de Processos de Ensino Online (Obstacles experienced by teachers when conducting Online
Teaching Processes).
Revista de Informática
, v. 5, n.6, Brazil.

Villano
, M.
(
1992
).
Probabilistic student’s models: Bayesian belief networks and knowled
ge space theory. In
Second International Conference on Intelligent Tutoring System
, p. 491
-
498, Montreal, Canada.