Facilitating Students' Understanding of Simple DC Circuits - Celebrate

tangybuyerElectronics - Devices

Oct 7, 2013 (3 years and 10 months ago)

103 views

Jaakkola & Nurmi

Academic impact of learning objects

1





FINAL REPORT ON CELEBRATE EXPERIMENTAL STUDIES




LEARNING OBJECTS


A LOT OF SMOKE BUT IS THERE A FIRE?
Academic Impact of Using Learning Objects in Different Pedagogical Settings






By

Tomi Jaakkola & Sami Nurmi

tomi.jaakkola@utu.fi

sami.nurmi@utu.fi

Educational Technology Unit

University of Turku

Finland









Jaakkola & Nurmi

Academic impact of learning objects

2

Introduction


Although high expectation have been projected in to the new technologies throughout

the history
of the educational technology (see Kearsley, 1998; Velleman & Moore, 1996, Watson &
Downes, 2000), the level of worldwide enthusiasm that is now directed to the notion of learning
object (LO) in various fields of education and training is uniq
ue. An LO is defined as any entity
-
digital or non
-
digital
-

that may be used for learning, education or training in the IEEE Learning
Technology Standard (IEEE, 2002), but
there are almost as many definitions of LOs as there are
people offering them. Howev
er, LOs are generally understood as digital learning resources that
can be shared and accessed via Internet and used in multiple contexts.
Since the book
The
Instructional Use of Learning Objects

that was edited by David Wiley (Wiley, 2000a), thousands
of
pages in books, articles and white papers have been dedicated to promote the LO approach.
Many authors, from researchers
(Urdan & Weggen, 2000; Gibbons, Nelson & Richards, 2000)

to
corporate leaders (Hodgins, 2000) believe that the LO approach bears the po
tential of
transforming education into a new level. In the most enthusiastic beliefs, LOs are said to fulfil the
long promised rewards of computer
-
based learning by offering scalable (mass instruction in a
cost
-
effective way) and individually adaptive inst
ruction, which
-

at its the most extreme form
-

can even be generated on the fly according to the learner needs by “intelligent” semantic
technologies (Gibbons, Nelson & Richards, 2000). As a consequence, huge amount financial and
human resources have been

and are invested in developing digital learning materials and
eLearning systems. For example, many influential private companies (e.g. CISCO, 2000) and
public organisations (e.g. the ADL project of the US department of defence; ADL, 2001) are
spending mil
lions of Dollars in building and standardising educational systems that are based on
the LO approach. In the Europe, the Commission has funded the CELEBRATE project that aims
to build trans
-
European material bank with 7 million euros. Individual countries
such as Great
Britain has invested 150 million £ in developing and distributing digital learning materials for the
UK schools.



What is it that makes LO approach so attractive? As indicated by the definition, t
he basic idea is
to create learning materia
ls that can be easily delivered over the Internet and effectively used, and
more importantly, reused in multiple contexts. One

fundamental principle behind the LO
Jaakkola & Nurmi

Academic impact of learning objects

3

approach is the cost saving principle, known as LO economy, which refers to maximal utilizing

of existing learning resources. According to Downes (2000), the economic benefits offered by
increased scalability and reusability of LO economy have been essential for many learning object
advocates. It may be possible for a single, large
-
scale project t
o produce mass of high
-
quality
learning materials for the needs of the project, but the longer term use of learning materials needs
more sustainable approach. In the LO economy individuals are part of a larger community where
participants exchange with eac
h other the materials they have produced. By sharing their own
materials with the community, an individual participant gets return for her/his investment as s/he
is rewarded with the access to much larger pool of materials than what s/he could ever produce

alone
1

(e.g. Duncan, 2003). Sharing of materials also gradually increases the total number of
learning resources in the repository, and therefore LOs can be comprised as a new currency of
exchange within learning communities (Littlejohn, 2003). From the r
euse point of view a
fundamental property of an LO is that although it can stand on its own it may also be combined
with other materials (Koper, 2001; Campbell, 2003). Reusability also requires that the LOs are
more or less context
-
free, so that one LO and

its content could be transferred as such to other
teaching
-
learning situation. Is has been also argued that once the LOs are relative
small (relative
to the size of an entire course), it is easier to (re)use them a number of times in a different
learning
contexts and with a different learning materials (Wiley, 2000b; Littlejohn, 2003). By
standardising the description, structure, and the way LOs are transferred, we can maximise the
economy principle, since interoperability ensures
that users are not trappe
d by a vendor's
proprietary learning technology (ISO, 2003), but that they can fully utilise the existing resources
(Campbell, 2003).


In addition to the obvious benefits that learning object approach offers for content developers and
providers, LOs seem f
easible from the teacher’s and learners’ point of view, as LOs can provide
more flexibility for teaching and learning. A teacher can select specific material pieces from the
LO repositories and assemble them the way that match the learning objectives and n
eeds of
individual students at all levels of abilities. As Reigeluth and Nelson (1997) point out, such
tendency to split materials is already evident in teachers’ everyday use of “traditional” learning
materials: when teachers first access learning materia
ls, they tend to split them into smaller pieces



1

The material production and s
haring is desirable but not a
prerequisite

for the participation

Jaakkola & Nurmi

Academic impact of learning objects

4

which can then be more easily fitted to match particular learning objectives. From the learners
point of view, as the traditional schoolwork has been dominated by the tradition of working alone
which has hind
ered student collaboration and discouraged students to build upon the work done
by other students, within online environments learners can collaborate with one another fluently
and they have instant (often high
-
speed) access to a variety of materials, and
more freedom to
study at their own pace (Scardamalia & Bereiter, 1996). When students are encouraged to use
available resources as building blocks for their learning, they don’t have to build everything from
the scratch, but they have a better chance to fo
cus on deeper
-
level learning (Koschmann, in
press).


Although the above discussed benefits of the LO approach sound promising, there are still many
barriers, debates, and open issues that need to be addressed and solved. Recently there has also
been rather

substantial critique on the LO approach (e.g. Parrish, 2004; Lambe, 2002; Butson,
2003; Collis & Strijker, 2004), and mostly this critique concerns the flawed and reductionist
assumptions of knowledge and learning under
lying in many of the current LO syst
ems
. At its
worst, LOs and LO systems are designed
only
for knowledge transmission
, and LOs are
considered as context

and pedagogy
-
free. This objectivist idea is that

the
se small
de
contextualized

pieces of fragmented content can be reused in variety of dif
ferent learning
situations.

LOs
and
the LO systems as such are regarded as complete and sufficient context for
learning
. Butson (2003), when criticiz
ing

this reductionist view,
has
felicitously

called
LOs as
weapons of mass instruction, which foster knowle
dge transmission from pre
-
packaged learning
materials and their sequenced entities. Suc
h simplistic

assumptions are ignoring essential aspects
of human learning, and are strongly contrary to the current theories of learning, e.g.
constructivism and situate
d cognition.


One of the open issues is concerned with LOs’ impact on students’ learning outcomes as there is
thus far no empirical evidence to support claims such as how learning objects could revolutionize
schooling and improve learning outcomes. When c
onsidering the effect of learning objects on
students’ learning performance, it is important to understand that i
t is impossible, and irrelevant,
to separate the learner and the learning content to be learned from the context in which learning
occurs (Lave
, 1988; Lave & Wenger, 1991; Duffy & Cunningham, 1996).

Although LOs may
Jaakkola & Nurmi

Academic impact of learning objects

5

provide new possibilities for learning, LOs are only one part of the whole learning context
(Jaakkola, Nirhamo, Nurmi & Lehtinen, 2004). According to the contemporary models of
learni
ng, learning is not about knowledge transmission as learners do not just ‘acquire
knowledge’, but they actively construct their own knowledge through the dialogue with learning
materials and other learners (e.g. Palincsar, 1998; Duffy & Cunningham, 1996; J
onassen, Peck &
Wilson, 1999). In this sense LOs contain only information, not knowledge. The
information
within an LO itself is meaningless, but it becomes meaningful knowledge when it is interpreted,
elaborated and applied by the individual (Sveiby, 1997
, 42).

No

matter how large and how well
organized a corpus of information is, it doesn't yet ensure that people will necessarily see or use
the information.

The same assemble of information can evoke different responses from different
people at different p
oints in time or place (Stamper, 1987).
It is the learner, her/his past
experiences, existing (prior) knowledge schemas, and learning strategies as well as the context
s/he uses the information in, who defines how the information will be interpreted and us
ed.
By
applying different pedagogical methods a s
killed teacher can help students to use learning
strategies effectively, and interpret and apply the information in a meaningful way.
Knowledge
and learning pro
cesses have context, history
, meaning and emoti
on far beyond its informational
content.

Therefore, it’s important to understand that LOs itself and the information inside them
can’t be regarded as adequate for learning and knowledge construction (Lambe, 2002).
As a
conclusion of

the
LO debate, it must
stressed that LOs, despite of their deficiencies, have
potential (and indeed they can be used) flexibly to support widely varying instructional strategies,
theories and philosophies


not just reductionist learning theories (Parrish, 2004). It’s the contex
t
and the arrangements of their use in teaching and learning that defines the educational value of
LOs and LO systems.


By comparing the learning outcomes of users that use LOs to the learning outcomes of users that
use traditional” (non
-
digital) learning
materials, this study examines if learning objects can
enhance students learning outcomes in different pedagogical settings and with different types of
LOs. In order to test our hypothesis, we have set out three independent experimental studies with
differ
ent kind of structuring and pedagogical approach. In all these studies the participants are
from the same sample. In the first study students are learning cases of Finnish language in two
different learning environments. Half of the students are studying c
ases with LOs and the other
Jaakkola & Nurmi

Academic impact of learning objects

6

half is working with traditional textbook assignments. In the second study the experimental
design is the same, but students are learning mathematics, namely fractions and mixed numbers,
with LOs and traditional paper
-
and
-
pencil

assignments in normal classroom situation. In the third
study, three groups are learning simple electrics. The first group is working with traditional
hands
-
on laboratory methods; the second group uses an electricity simulation LO, and the third
group is
using a combination of both of these methods (laboratory and simulation LO). Method
and the results of each study are described separately, but the results are discussed simultaneously
in the final chapter.


STUDY 1


FINNISH LANGUAGE: CASES/GRAMMAR


Metho
ds


The aim of the first study was to compare students’ learning outcomes in two learning conditions:
1) with learning objects and 2) with traditional teacher
-
led instruction. The content of this study
was Finnish language grammar, more closely cases
. Case
s are considered to be

ver
y difficult in
Finnish language, and t
here are fourteen basic cases in Finnish language.

The participants of the
study were 37 11 year old (5
th

grade) students from
a
Finnish elementary school. Students in both
groups worked for t
w
o hours (two one hour sessions). I
n order to compare achieved learning
outcomes between the two learning conditions,

pre
-

and post
-
tests were administered before and
after the working phase.
The pre
-
test measured students’ prior knowledge of cases and par
ts of
speech of Finnish language as well as their reading comprehension skills. The post
-
test focused
only on cases, and students were required to identify the cases and inflect different words
according to different cases

In addition to the subject knowle
dge tests, the Raven’s (1958)
Standard Progressive Matrices (Sets A
-
E) test
of

general educative ability was also administered.
Based on the pre
-
test and Raven’s test scores, students were matched into two learning
cond
itions: 1) LO group and
2
) Traditiona
l group. In both of the

conditions

students’
were taught
by the same teacher in order to control
the possible effect of teacher. After the selection of the
exact content of the study, the used LOs and paper
-
and
-
pencil tasks were carefully selected in
order

to cover the same topics in both learning conditions.


Jaakkola & Nurmi

Academic impact of learning objects

7

In the first learning condition (LO group) students (n

=

19) used several learning objects
(altogether 10) independently and completing the LO
-
included
-
tasks at their own pace. Therefore
this working

method was very close to the ideal of individualized and programmed instruction

(c.f. Computer
-
Based Instruction/Computer
-
Assisted Instruction)
, the mainstream of using
computers in education during the 80’s and early 90’s
. All of the used materials were
drill
-
and
-
practice type LOs, and most of them there were simple games or drag
-
and
-
drop applications
giving instant feedback or points for students’ actions

(see figure 1 below for example LO)
. Both
lessons started with teacher
-
led introduction part, which
was also identical in traditional teaching
group: fill
-
in task that was done in teacher
-
led way. After the introduction students were assigned
to computers to complete a sequenced set of LOs. The rest of the both lessons were spent by
complet
ing assignment
s inside the LOs. Although there was no strictly structured, teacher
-
led
instruction, the pedagogical structure was set by a sequenced proceeding through selected LOs
and the teacher monitored students’ progression all the time in a computer suite.

In othe
r words,
the instruction didn’t focus on the subject knowledge or LO content
per se
, but purely sequenced
the order of the LOs.


Jaakkola & Nurmi

Academic impact of learning objects

8


Figure 1
.

Example of Finnish language LO
-

Identify genitives LO (by Sanoma eWSOY).


In the traditional teaching group (n

=

1
8), the lessons started with teacher
-
led introduction part
which was sequenced by fill
-
in tasks. Fill
-
in tasks were completed together with teacher and
students in a way that teacher asked students to propose what cases should be used in the fill
-
in
text.
After this students were assigned to completed similar fill
-
in and cases identification tasks
(in paper and pencil format) individually. By their content these assignments were almost
identical to the tasks completed with the learning objects in LO group.
In the end of the lessons
the tasks were jointly checked by the teacher (using the overhead projector) and the students. The
working of

the traditional group resembles

very closely to typical mixture of expository
instruction and students’ individual task
activities.


Jaakkola & Nurmi

Academic impact of learning objects

9

Results


Overall development


As students were matched into different conditions, there was no difference between the groups
in total pre
-
test scores or in any

of the

four section scores, t (36) = .
039, p = .969.

Despite the
matching procedur
e,
in order to minimize the effect of
the grammar

pre
-
test and Raven scores, a
between
-
subjects
statistical test of
ANCOVA was chosen to compare students’ post
-
test scores in
different condition. Although both the
grammar

pre
-
test score and the Raven score

correlated
significantly with the post
-
test score (r =
560
, p = .000; r =
329
, p =
.046
, respectively), only the
pre
-
test score was selected as a covariate for the actual model, since there was significant (
r =
.364, p = .024
) cor
relation between the

cova
riates
,
and analysis of regression revealed that merely
grammar pre
-
test score significantly adjusted pre
-
test scores, β = .507, p = .002. No univariate
(Cook’s distance < 1 and standardized residuals with p < .001) or multivariate (Mahalanobis
distance an
d leverage values with p < .001) outliers were detected in and among the DV or CVs.
The homogeneity of regression was reached sin
ce the effect of the grammar

test was equal for
both conditions as there was no interaction between the grammar pre
-
test and th
e tr
eatment
variable
, F (1,32) = .312, p = .580. Assumptions of normality of sampling distribution
was not
fulfilled within the post test scores of the traditional group (Shapiro
-
Wilk, p = .003)
and
nor was
the assumption of
homogeneity of variance
, F (1,
34) = 6.389, p = .016. Therefore, although
ANCOVA is a robust test, following result should be dealt with some precaution.


After adjustment by the
grammar

pre
-
test score,
even t
hough traditional group (M = 11.84, SD =
8.
66) achieved slightly higher post
-
t
est
scores that the LO group (M = 9.95, Std = 4.
71),
ANCOVA
showed

no
significant main effect of the treatment on

the post
-
test performance, F (1,
34) = .
894
, p =
.351
.

Nor was there a significant difference between conditions within low and
high prior kno
wledge students (p > .05).


Jaakkola & Nurmi

Academic impact of learning objects

10

Observation of student activities


These results are ba
sed on preliminary observations of videotaped

data
.
In the first session
students completed six LOs, which focused on identification of different cases. During the second
s
ession students worked with
four

LOs.
Observations

revealed

that while working with LOs,
students working resembled mainly trial
-
and
-
error behaviour.
Since

the LOs or/and the
instruction did not tightly structure

the students’ working
, s
tudents were really

not concentrating
on content but
they
were more interested in solving the logic or algorithm behind the LO

(how
the LO works)

as they
seemed to be
in hurry

to “click their way through


to the next assignment
ahead of their classmates.

Thus,

the task orien
tation of the students remained rather high during
the sessions,
but
the
depth

of orientation stayed
on a superficial level.

Although the teacher
monitored students’ working, t
he attitude in the classroom was rather restless. Students’
expressions such as
“I have completed already five assignments, how many have you?” were
typical in the computer suite.

In one of
the used LOs (see figure 1
), the students were asked to
identify genitives by proceeding through the table from one cell to another and selecting
the
correct cells which included a genitive. When the teacher noticed that students were hurrying
through the LOs


even without thinking, he stressed the need to focus on the tasks

and on the
content

and required students to work through the LOs for addit
ional times. Teacher started also
to control the pace of students’ work by making them to proceed from one LO to another at the
same time
. This calmed down the atmosphere
at the computer suite and slowed down the pace of
shifting from one LO to another
.


I
n traditional settings learni
ng activities were more teacher
-
led and
tightly
structured. Students
were focusing better on content than students working with LOs, since teacher was constantly
monitoring their progress.

The atmosphere in the classroom was al
so more peace
ful than in the
computer suite and the students were able to focus on completing the tasks during the assignment
periods.


Jaakkola & Nurmi

Academic impact of learning objects

11

STUDY 2


MATHEMATICS: FRACTIONS AND MIXED NUMBERS


Methods


The objective of the second study was to verify the
obtain
ed results from the study one
. In the
second study the effectiveness of LO and traditional learning conditions were compared in
mathematics. The subject matters were fractions and mixed numbers, and the study was
conducted with 35 10 year old (4
th

grade) F
innish elementary school students. The
study
consisted of two hours of concrete working (both learning condition groups worked two one hour
sessions) and administration of various tests. Before the actual working phase, participating
students
complete
d

pre
-
test of the subject matter and the Raven’s (1958) Standard Progressive
Matrices (Sets A
-
E) test. After the working phase, the mathematics post
-
test

w
as

administered.
The pre
-
test measured students’ prior understanding on basics of fractions, and in the po
st
-
test the
tasks included both fractions and mixed numbers.
Based on the
mathematics
pre
-
test and Raven’s
test scores, students were matched into two learning conditions.

Again, the selection of used LOs
and paper
-
and
-
pencil assignments were carefully cho
sen to cover the same topics.


In the LO group (n

=

19) teacher started both of the sessions with an introductory instruction in
which he presented the subject content to the students. In the first session he used chalk board
and in the second pie charts
clipped from paper
to

illustrat
e

the fractions and mixed numbers.
After the introduction, students were assigned individually to the computers and the
y were given
a

list of LOs that they ought to study with
. For the rest of the sessions students complete L
O
assignment individually at their own pace. In both sessions students used three different LOs, and
the sessions were arranged in school’s ICT classroom. In the first session the used LOs included
1) the identification of fractions, 2) the writing of frac
tions numerically, and 3) memory game
with fractions (see figure
2
). In the second session the used LOs were dealing with 4)
identification of mixed number and their relat
ion to number line (see figure
3

below), 5) forming
of and simple additions with mixe
d numbers, and 6) battle ships game with mixed numbers. The
selected LOs by their type were mainly quite simple drill
-
and
-
practice materials which were
designed to be game
-
like and provide instant feedback for students’ input/answers.

The way of
working wa
s rather student
-
led
because there was no expository teaching nor teacher
-
controlled
Jaakkola & Nurmi

Academic impact of learning objects

12

tasks during the assignment phase.
I
nstruction didn’t focus on the subject knowledge or LO
content
per se
, but purely sequenced the order of the LOs. In order to be succes
sful,
the working
supposedly required

higher

level of
self
-
regulation
(e.g. Zimmerman & Schunk, 1989)
and
metacognitive skills

(self
-
monitoring and

controlling, maintenance of task orientation etc.)

than
working in the traditional condition
.



Figure
2
.

Mathematics example LO
-

Memory game with fractions (by Sanoma eWSOY)

Jaakkola & Nurmi

Academic impact of learning objects

13


Figure
3
.

Mathematics example LO


Identification of mixed numbers (by Sanoma eWSOY)


Traditional group (n

=

16) was taught in normal classroom. The teaching method resembled
normal cl
assroom instruction with teacher
-
led introduction followed by assignment phase when
students completed different paper
-
and
-
pencil tasks individually. The tasks were similar
assignments completed with learning objects in the LO group including identificatio
n and writing
of fractions and mixed numbers, and simple additions with them. At the end of the sessions tasks
were checked jointly between the teacher and the students. These kinds of arrangements were
rather strictly controlled by the teacher and there w
ere no big requirements for students’ self
-
directed activities.


Results


Overall development


As students were matched into different conditions, there was no difference between the groups
in total pre
-
test scores or in any of the

four section scores, t (
36) = .
039, p = .969. Despite the
Jaakkola & Nurmi

Academic impact of learning objects

14

matching procedure,
in order to minimize the effect of the
mathematics

pre
-
test and Raven
scores, a between
-
subjects
statistical test of
ANCOVA was chosen to compare students’ post
-
test
scores in different condition.
Altho
ugh both the mathematics

pre
-
test score and the Raven score
correlated significantly
with the post
-
test score (r

= .
6
30, p = .000; r =
.
502 p = .002
,
respectively), only the pre
-
test score was selected as a covariate for the actual model, since there
was s
ignificant (
r = .495,
p
= .002
) correlation between the
covariates, and analysis of regression
revealed that merely mathematics pre
-
test score significantly adjusted pre
-
test scores, β = .634, p
=
.
000.
One
univariate
outlier, that exceeded the critical va
lue of 1 in Cook’s distance, was
removed from the traditional group.
No

other

univariate (Cook’s distance < 1 and standardized
residuals with p < .001) or multivariate (Mahalanobis distance and leverage values with p < .001)
outliers were detected in and a
mong the DV or CVs.
The homogeneity of regression was reached
since the effect of the mathematics test was equal for both conditions as there was no interaction
between the mathematics pre
-
test and the treatment variables, F (1,30
) = .
399, p = .533.
Assump
tions of normality of sampling distribution and homogeneity of variance were also
satisfactory at the α level of .05.


After adjustment by the
mathematics

pre
-
test score, even tho
ugh traditional group (M = 7.04,

SD
= 1.89
) achieved slightly higher post
-
tes
t sc
ores
that the LO group (M = 6.03,

SD = 2.16
),
ANCOVA showed no significant main effect of the treatment on the post
-
test performance, F
(1,
30
) = 2
,
370, p = .134
. Nor was there a significant difference between
the
conditions within low
and high prior k
nowledge students (p > .05).


Observation of student activities


These results are based on preliminary observations of videotaped data.
In the LO group, s
tudent
completed six LOs during two
working sessions
, and various paper
-
and
-
pencil assignments in the

traditional group
.
Observational results
here
are
following the

same
trend than in

the study 1:
in
LO group
students worked mainly in trial
-
and
-
error fashion,
and they focused on procedural
aspects of LOs
. It can be concluded that

working with LOs didn’t
engage students with

thinking
the content being learned
.

Instead, t
he work of traditional group was much more focused

on the
learning tasks. This higher task
-
orientation of the traditional group was due to larger amount of
Jaakkola & Nurmi

Academic impact of learning objects

15

external control imposed by teach
er
-
led activities and monitoring
. It may be that the LO
condition requirements on students’ self
-
regulation were overwhelming for students in LO group
.


STUDY 3


ELECTRICITY: SIMPLE DC CIRCUITS


M
ethod


After obtaining such poor results from the study one

and two, the third study was designed with
much more tightly controlled assignments.
The aim of the study was to teach to the
elementary
school students basic

electricity by introducing them
with
the concepts of a closed circuit and
a
current division in
a DC
circuit. Both of the topics are very challenging for students at all ages.
There is
a
large body of evidence which show that even high school and university students have
severe difficulties in understanding these two concept
s

still

after formal instr
uction in the relevant
material
s (Shipstone, 1984; McDermott & Shaffer, 1992).

U
nderstanding of the concept of
current division seems to be especially challenging task (McDermott & Shaffer, 1992).



Learning conditions and arrangements


The study was cond
ucted with 66 10
-
11 year old Finnish elementary school students.
One day
bef
ore the training sessions a pre
-
test was administered.
Based on
the
pre
-
test scores, students
were matched into three different conditions.


1) In the LABORATORY GROUP (n = 24) st
udents worked in a classroom and built real circuits
with batteries, bulbs, wires and switches, and measured current with a multimeter.

2) In the SIMULATION GROUP (n = 20) students worked in a computer suite and used an
online simulation, the “Electri
city

Exploration Tool” (Figure
4
), which allows them to build
simple DC circuits with batteries, bulbs and wires on a
diagram

level. The simulation also allows
students to observe the
behaviour

with their circuits by running the model they have built and
measu
re voltage with a multimeter.

3) In the MIXED GROUP (n = 22) students worked in a computer suite and used the Electricity
Exploration Tool and the real circuits both. In this group they were first asked to complete the
Jaakkola & Nurmi

Academic impact of learning objects

16

assignment using the simulation and
then, after succeeding with the simulation, repeat the
assignment with the real circuits that were located right next to the computer.



Figure
4
.

The Electricity Exploration Tool

(by DigitalBrain)
.


In each condition students were divided into two sub
-
gr
oups (10
-
12 students in each sub
-
group).
In these sub
-
groups students worked in pairs. The same teacher was teaching each group. Pairs in
each condition received exactly same instruction, which was given in specially designed
assignment cards.

Assignment c
ards introduced basic electricity concepts, addressed students’
common difficulties and misconceptions found at various studies, structured student work, and
provided metacognitive aid.

The contents of the assignments proceeded from a very simple DC
circu
it with one batte
ry, wires and a bulb towards

more challenging tasks where they had to, for
example, construct a circuit in which the brightness of four bulbs is A>B>C=D.


Opposite to study 1 and 2, a
ssignments in these cards focused on
scaffolding the l
earning of the
content

and
increasing students’ metacognitive awareness
, not just sequencing the proceeding
order.
For example, the assignments especially urged students to concentrate on topics that have
Jaakkola & Nurmi

Academic impact of learning objects

17

found to be difficult in previous studies. Moreover
, in each assignment

students were required to
measure the voltages of the bulbs in different circuits and explicitly prompted to compare the
characteristics of different types of circuits. Each card consisted of one assignment.
In contrary
to the study 1

and 2, assignment progression was tightly controlled:
Once students had completed
the assignment
, in order to proceed into the next assignment,

they
had to have teacher’s approval
to carry on
. Once the assignment was passed, student pair received next lev
el assignment card.
There were 12 assignments in total. Each group had two hours (one hour a day) to complete as
many assignments as they could.
In order to find out students’ learning outcomes in different
conditions, a post
-
test was administered one day
after the training session.



Tests


As a pre
-
test
, students were asked to fill in the Raven’s (1958) Standard Progressive Matrices
(Sets A
-
E) test that measures general educative ability and an electricity questionnaire that
consisted of four questions t
hat
concentrated on two topics:
1) the first two questions measured
students’ understanding of the concept of a closed circuit. In the first question students had to
evaluate in which circuit a bulb would be lit, and in the second question they had to draw

how
electric current flows between a battery and bulb. 2) The two remaining questions measured the
understanding of the current division in a DC circuit. In the first of these questions students had
to infer in which circuit a bulb is the dimmest, and in
the second question they had to calculate
the voltage of each bulb in different circuits (single bulb circuit, series circuit and parallel circuit)
when the power source stays constant.


The post
-
test consisted of an electricity questionnaire that concent
rated on the same two topics as
the pre
-
test. Besides the same four questions

as
in
the pre
-
test,
the post
-
test

had

four more
d
ifficult questions. All the questions in test
s

were designed to be equally fair for each condition.
Although students worked in
pairs, they completed all the tests individually.
Students’ answers in
the electricity questionnaires were scored against model answer template. Raven’s test was
scored by using Raven’s scoring key.


Jaakkola & Nurmi

Academic impact of learning objects

18

Results


Overall development


Although there was no dif
ference between the groups in total pre
-
test scores or in any of the four
section scores (ANOVA p > .05) a
s students were matched into different conditions,
in order to
minimize the effect of
the
electricity
pre
-
test a
nd Raven scores,
a between
-
subjects
st
atistical test
of
ANCOVA was chosen

to compare student
s’

post
-
test scores in different condition
s
.
Although
both the electricity pre
-
test score and the Raven score correlated significantly with the post
-
test
score (r

=

.
643, p

=

.000; r

=

.
356
, p

=

.003
, r
espectively
), only the
electricity
pre
-
test score was
selected as
a
c
ovariate for the actual model
, si
nce
there was significant correlation
(r = .389, p =
.001)
between the
covariates
, and
analysis of regression revealed that
merely
electricity pre
-
test
sc
ore

significantly adjusted pre
-
test scores
, β = .583
, p = .000
.

One multivariate outlier, that
exceeded the critical value of Cook’s distance at p < .001, was removed from the simulation
group.
N
o
other multivariate (Mahalanobis distance and leverage values with p < .001) or
univariate

(Cook’s dis
tance < 1 and standard
ized residuals with p < .001)

outliers were detected
in
and among
the DV or CVs
. T
he
homogeneity
of regression was

reached as the
effect of the
electricity tes
t was equal in all conditions since

there was no interaction between the el
ectricity
pre
-
test and the treatm
ent

variables
, F (2,62) = .043, p = .958
.

A
ssumptions of nor
mality of
sampling distribution and

homogeneity of variance
were a
lso satisfactory at

the
α level o
f .05.


In the post
-
test, the Laboratory group

scored the mean of 9.98 (S
D = 3.83), the Simulation group

13
.13 (4.00), and the Mixed group

12,63 (3,87). After adjustment by the electricity pre
-
test score,
ANCOVA alerted significant main effect of
the treatment on the post
-
test
2

performance, F (2, 61)
= 5.298, p = .008. As already indicated by the mean differences between the groups, the
Bon
ferroni’s post
-
hoc test confirmed

that
both groups using the simulation significantly
out
performed the traditi
onal group. The risk level of type

1 error was p = .013 between the mixed
group and the laboratory group, and p = .038 between simulation group and laboratory group
.

Between the groups th
at used the simulation

there was no difference (p = 1.00). The same
t
endency occurred between
the c
onditions in every post
-
test sub
-
section.




2

Adjuste
d mean for the laboratory group is 10.22, for the simulation group 12.60, and for the mixed group 12.
83
.

Jaakkola & Nurmi

Academic impact of learning objects

19


When investigating the effects of training on students’ with low and high pre
-
test scores, results
in Figure 4

show that students with high prior knowledge outperformed students with

low prior
knowledge in a post
-
test scores in every condition. ANCOVA revealed that there was statistically
significant difference between
the
conditions among low prior knowledge student
s (F (2, 25
) =
4.
564, p = .019
). Th
e difference was due to because th
e mixed group

significantly outp
erformed
the laboratory

group
(Bonferroni p = .026
). Between

the simulation group

and
the laboratory
group
, as

well as between the mixed group and the simulation group

the

differences were not
significant

(Bonferroni
p = .08
6
; p = 1.000
, respectively)
.
Although the same tendency between
the
conditions was also found among high prior knowledge students, the difference between the
groups was not statistic
ally significant, F (2, 32
) = 2.
225, p = .125.



Figure 4
.

Post
-
test scor
es of students with low and high prior knowledge in different conditions.

SG=simulation group, LG=laboratory group, MG=mixed group.


Examination

(ANCOVA)
of
the
post
-
test scores of low and high educative ability (Raven test
scores; MD = 43,5) students reve
aled same trend as comparison among low and high prior
knowledge students: Among
the
students with high educative ability
the simulatio
n users
(simulation group M = 15,05, SD = 4,11
; mixed group

M = 14
,
17, SD = 3,68
) succeeded slightly
Jaakkola & Nurmi

Academic impact of learning objects

20

better th
an the labo
ratory group

(M = 12,04, SD = 3,97
), but the difference between
the
conditions wa
s not significant, F (2, 33) = 2,225, p = .125
. Among
the
students with low
educative a
bility, the simulatio
n users (simulation group M = 10,28, SD = 1,87
; mixed group

M
= 1
0,80, SD = 3,38
) outperformed significantly the laboratory group

(M = 7,55
, SD =

1,69
), F
(2,27) = 4,564
, p = .020.
The difference was due to because the mixed group significantly
outperformed the laboratory group (Bonferroni p = .0
24
). Between the simulat
ion group and the
laboratory group, as well as between the mixed group and the simulation group the differences
were not significant (Bonferroni
p = .110
; p = 1.000, respectively).



Conceptua
l change in understanding
the current flow in a
DC

circu
it


In

o
rder to gain deeper insight into

students’ conceptual change process in understanding the
concept of current flow in a DC ci
rcuit, students’ answers in the

first
two
sections
of the
electricity questionnaires
(identical in pre
-
test and post
-
test) were rean
alyzed by two independent
ra
ters (inter
-
rater reliability 0.
92, disagreements were negotiated).
Based on students’ answers to
these two questions, their conceptions
of
the current flow in a DC circuit

were classified into
three

developmental categories
:


1.

I
nconsistent model. In this most primitive category, students ‘answers were inconsistent


the logic between different sub
-
questions didn’t follow any particular principle. With
some questions, answers followed the ‘sink’ principle (electricity can sink alo
ng a single
wire connection from a power source to an electrical device), whereas in others, students’
answers indicated the clashing current principle (below) or were somehow inconsistent.

In
the pre
-
test 15/66 and in the post
-
test 7/66 student answers we
re classified into this
category.

2.

Clashing current model. In this category, students reasoning followed consistently the
clashing current principle. In this model learner believes that positive electricity flows
from a power source’s positive terminal, and

negative electricity from negative terminal.
Then the opposing currents meet at the electrical device where they clash. This clashing
causes the device to work.

Pre
-
test 22/66 and in the post
-
test 8/66 student answers were
classified into this category.

Jaakkola & Nurmi

Academic impact of learning objects

21

3.

C
orrect circular model. Current flow is unidirectional and continuous. The scientifically
accepted model of DC circuit.

Pre
-
test 29/66 and in the post
-
test 51/66 student answers
were classified into this category.


Table 1 presents the frequencies of pre
-
te
st and post
-
test response ca
tegories in understanding the
current flow in a DC circuit

by the students’ working in different conditions.
According to the
results of the Chi
-
square test, a
lthough the proportional amount of correct responses is slightly
high
er among the simulation and the mixed groups compared to the l
aboratory group

in the pre
-
test
,
the difference
in distributions
between the groups was not statistically significa
nt in
the
pre
-
test or post
-
test, p >
.05
. However, the Wilcoxon Signed Ranks te
st showed, that the pre
-
test
-
post
-
test change was signific
ant in the simulation group (Z = 2,111, p = .035) and the mixed
group

(Z = 2,443, p = .0
15),
but not in
the
laboratory

group
(Z = 1,814, p = .0
70).


Table1.
Pre
-
test and post
-
test r
esponse cat
egories

in understanding the current

flow in a DC
circuit
.



Laboratory group
(n=24)

Simulation group

(n=20)

Mixed group

(n=22)


Pre
-
test


Post
-
test

Pre
-
test

Post
-
test

Pre
-
test

Post
-
test

1. Circular
(correct)
model


42% (10)

66% (
16
)

50% (
10
)

85% (
17
)

41
% (9)

82% (
18
)


2. Clashing model


25% (6)

17% (4)

45% (
9
)

10% (
2
)

32% (7)

9% (2)

3. Inconsistent
model

33% (8)

17% (4)

5% (
1
)

5% (
1
)

27% (6)

9% (2)


In order to see in more detail how students with different preconception levels
developed

from
the
pre
-
test to post
-
test in different conditions, inconsistent and clashing models we
re

merged as
incorrect respons
es. As can be seen from table 2,

in the mixed and simulation groups

more
students than

in
the laboratory group

improved from
the
incorrect
conceptua
l level in the pre
-
test

to
the
correct
conceptual level in the post
-
test
. In
the laboratory group

there are more students
who don’t deve
lop between the tests than in the simulation and mixed group
. In
the laboratory
group
a
nd in the mixed group
, there is o
ne student who
s
e

answer drops from correct to incorrect

category
. The McNemar test confirms the results obtained from the Wilcoxon Signed Ranks test:
Jaakkola & Nurmi

Academic impact of learning objects

22

pre
-
test
-
post
-
test developm
ent is si
gnificant in the simulation (p = .016) and mixed group

(p =
.0
12), but

not

in the laboratory group

(p = .0
70).


Table2. Pre
-
post
-
test change in understanding closed circuit


Pre
-
test

Post
-
test





Incorrect

Correct

Laboratory


G
roup

Incorrect

29% (
7
)

29% (
7
)

Correct

4% (
1
)

38% (
9
)

Simulation

G
roup

Incorrect

15% (
3
)

35%

(
7
)

Correct

0% (
0
)

50% (
10
)

Mixed group



Incorrect

14% (
3
)

45% (
10
)

Correct

5% (
1
)

36% (
8
)



Conceptual cha
nge in understanding the division

of current in a DC circuit


In order to gain deeper insight into students’ conceptual change process in und
erstanding the
concept of current division in a DC circuit, students’ answers to the third and fourth question in
the electricity questionnaires (identical in pre
-
test and post
-
test) were reanalyzed by two
independent raters (inter
-
rater reliability 1.00).

Based on students’ answers to these two
questions, their conceptions of the current division
were classified into two
categories
:

1.

Correct solution. All answers are correct, which requires understanding of current
distribution in both series and parallel c
ircuit.
In the pre
-
test 8 students out of 66 arrived at
correct solution whereas in the post test
the number of correct answers

was

21
.

2.

Wrong solution. One or more answers are incorrect.

In the pre
-
test 58/66
and in the post
-
test 45/66
students belonged to

this category.


Such tight criteria were chosen in order to avoid the effect of guessing.


Table 3

presents the frequencies of
the students’
correct and incorrect pre
-
test and post
-
test
responses
in understanding the concept of current sharing in a DC

circuit.
As can be seen from
the table, more students in groups that used simulation advanced from incorrect solution to
correct solution, but the overall development in all groups was small.

The C
hi
-
square test showed
Jaakkola & Nurmi

Academic impact of learning objects

23

no difference between the groups in p
re
-
test a
nd post
-
test (p >
.05)
. Accordi
ng to the McNemar
test only the mixed group’s

pre
-
test
-
post
-
test
development
was statistically significant (
p =
.016)
.





Table 3.

Pre
-
test and post
-
test response categories in
understanding the division

of current
in a
DC circuit


Observation of student activities


These results are based on preliminary observations of videotaped data. Students that worked
only with
the
simulation were focused on learning content and tasks and concentrated o
n solving
tasks successfully. Although the simulation LO has rather modest layout, students were still
highly motivated in solving the assignments with the LO. Thus they used the simulation as a
problem solving tool
, and it engaged them at a sufficiently d
eep level for meaningful learning to
take place
. The atmosphere in the computer suite was
enthusiastic

and students’ working was
focused

on

completing
and solving
the tasks
.



Above applies also to the mixed group: they were extremely focused and motivated
. Moreover,
mixed students seemed eager on experimenting their own theories and hypothesis with real
circuits, since after solving the assignments with the LO, they were keen on to try to solve the
same assignment with the real bulbs and wires as well. The
y seemed to succeed pretty well in
transforming their theoretical (or simulation
-
level
) understanding in to real circuits.

When
compared to traditional laboratory group, the use of simulation before the construction of real
circuits seemed to make hands
-
on

circuit activities more rational and
intentional.




Laboratory group
(n=24)

Simulation group
(n=20)

Mixed group (n=22)


Pre
-
test

Post
-
test

Pre
-
test

Post
-
test

Pre
-
test

Post
-
test

1. Correct
solution

13% (3)

21% (5)

15% (3)

35% (7)

9% (2)

41% (9)

2. Incorrect
solution

87% (21)

79% (19)

85%
(17)

65% (13)

91% (20)

59% (13)








Jaakkola & Nurmi

Academic impact of learning objects

24

Students of the traditional group that build
only
real circuit
were also motivated and excited to
complete the hands
-
on activities, but at the same time, they
had hard time in working with bulbs
and wire
s.
It was also evident that students working only with real stuff had difficulties in going
beyond to
mechanical (superficial circuit functioning)

level into understanding the principles of
DC circuits in a theoretical level.
The wires were often disorgan
ised (although instructed
in
keeping them straight and in order)

and as a consequence, students, especially the ones with
lower level, had constant difficulties in getting their circuit work. After series of failures, some
students got clearly frustrated
w
hich
occasionally
caused

lost
of
the motivation

and

off
-
task
behaviour
.


In this study in all of the conditions, teacher’s role was rather passive, as he monitored student
work from the distance giving only occasional hints.


Conclusions


Aim of the pres
ent study was to inspect the impact of learning objects on students learning
outcomes.
In order to test the hypothesis, three independent experimental studies were conducted.
In the first two studies students that used learning objects performed equally we
ll as the students
that worked with traditional materials
,
but in the third study students that used learning objects
outperformed students in traditional learning condition.
Results
of the st
udy three also indicate
that

combining learning objects with tra
ditional learning materials can be especially fruitful.



Results of the study three provide encouraging evidence on the effectiveness and usefulness of
learning objects. In this study
students that used the simulation LO were able to improve their

overal
l learning outcomes compared to
those students that worked in laboratory environment
with real bulbs and wires
.
Simulation
-
laboratory
-
combination seemed to be even more effective.
Analysis of students’ conceptual understanding showed
that the simulation en
vironments helped
the students to change their conceptions of current flow from incorrect model to scientifically
accepted model, but in the laboratory environment there was no statistically significant change.
Between the pre
-
test and the post
-
test, only
the simulation
-
laboratory group’s comprehension of
current division in a DC circuit advanced statistically significantly from the incorrect model to
Jaakkola & Nurmi

Academic impact of learning objects

25

the correct model.

Moreover, e
xamination of ability groups revealed that the use of simulation
-
laboratory
can be particularly beneficial for students with lower prior knowledge and educative
ability. This result is little surprising since in previous studies simulations are considered as rather
challenging environments for learners (de Jong & van Joolingen, 19
98). One explanation for the
observed result may be that assignments cards may have helped weaker students by structuring
simulation environment and guiding students to focus on relevant issues. Findings of Veenman
and Elshout (1995) on structuredness of l
earning environments support this explanation. It seems
that the success of high ability students is not so dependent on learning environments because
they seem to cope with every studied learning condition
and settings as

they systematically
outperform lo
wer ability and lower prior knowledge students.


One explanation for the success of the
simulation

users
could be that
the
simulation
LO
helps
students to understand the theor
etical principles

by revealing the behaviour of DC circuit and
visualizing the cu
rrent flow in the circuit.
Simulation
also
provides unique opportunities for
students to interact with a knowledge domain (
Gredler, 1996).
Interactive simulation can be
especially effective

because it

require
s

users to make explicit their implicit reasonin
g

and it
allows

them to visualise the
consequences of their reasoning
(Hennesy & O’Shea, 1993).
Sim
ulation LO can also provide immediate feedback to students about their actions and errors
(McDermott, 1990; Ronen & Eliahu, 2000).
For example, students can

easily construct their own
circuits and obs
erve the effect of manipulating

the characters o
f the circuits.


The argument for the simulation
-
laboratory
-
combination is that
a
fter understanding the basics of
electricity on a theoretical (simulation) level m
akes it easier for a student to transfer acquired
knowledge into the laboratory exercises with real circuits, and as a consequence acquire more
coherent and holistic comprehension of the topic. Thus the combination of laboratory and
simulation work can bri
dge the gap between theory and reality (Ronen & Eliahu, 2000). This
combination use can also increase the credibility of simulation because students can perceive that
the laws and principles of electricity are functioning in the same way both in the simula
tion and
the real circuits (c.f. Hennessy & O’Shea, 1993).


Jaakkola & Nurmi

Academic impact of learning objects

26

Let us
next
consider

why learning objects were not so effective in the first two studies, but in the
third study LOs clearly provided additional help.
Observation data from the
three
studies reve
aled
that in the language and mathematics examples, instead of concentrating on content and learning
tasks, students concentrated only on surface and procedural issues by focusing on “solving the
LO”, whereas in the electricity

study students were able to
stay on the task and concentrate on
important issues
study the
. In the electricity study

the
LO itself was secondary (but important) as
students used the LO as a tool to solve the problem (see also Lajoie, 1993; Jonassen & Reeves,
1996)
.


There are co
uple
of important differences between studies one and two compared to study three

that may explain the difference in the student behaviour and learning outcomes
.
First of all,
because students are used to solve classical text book assign
ment rather that work wi
th LOs, it
may be, that in the beginning, successful LO work requires more structuring than traditional
work, or at least little more time to acquaint students with the new way of working.
Because of
this “LO novelty”

effect
, i
n the language and mathematic
s

studies using of

multiple LOs during
the lessons

may have caused restlessness, since students may have been overly eager to just
explore different LOs and not to concentrate in learning with them. In the electricity study
students were working with the s
ame LO
whole the time. They had thus enough time to
familiarise themselves with the LO, and the presence of upcoming LOs didn’t disturb their
concentration.
LO type may also be an issue.
Some LOs also provide more and different
affordances than the others,

and therefore different types of LOs can vary in their probabilities to
produce or arouse certain type of learning and working behaviour. For example, simulations and
other exploration LOs typically afford possibilities to discovery or problem
-
based learn
ing,
whereas games and drill
-
and
-
practice LOs are strong in immersing learners into competition or
highly motivating and entertaining learning endeavours where learning can happen by accident.
Thus, i
n the first two studies
where students
used
game type LO
s
,
they

may have
had
difficulties
in associating game type LO into learning.


The interaction between the level of instructional support and students’ abilities is also an
important factor. It may be

that the difficulties are more connected to
the used pe
dagogy
than to
the characters of the used LOs.

It has been found in previous studies (de Jong & van Joolingen,
Jaakkola & Nurmi

Academic impact of learning objects

27

1998) that students’ difficulties in using digital materials may be related closely
to
the problems
that they have with the working method (e.g.
scientific discovery learning) the new materials
require. Unstructured tasks can cause frustrations and hinder learning, as unstructured and more
open tasks require more elaborated skills in task
-
setting and self
-
monitoring. This is especially
true, if the

requirements of the task on students’ self
-
regulation are too high compared to
students’ readiness to self
-
regulation (Vermunt & Verloop, 1999; Olkinuora, Mikkilä
-
Erdmann &
Nurmi, 2004).
I
n order to be successful
,

t
he level of external instructional suppo
rt (structuring,
scaffolding, monitoring etc.) and the participating students’ metacognitive abilities need to be
balanced in any learning environment.
In the language and mathematics examples the teacher, the
assignments, or the LOs did not tightly struct
ure student work
, but students were more on their
own. I
n the electricity example,
although students were also in charge of their own learning
process, the
tightly struc
tured tasks played an important role in improving students learning
performance by guid
ing their work and helping

them to stay on task and to concentrate on
important issues.

This highlights the importance of the pedagogy and level of scaffolding.


Although this paper has provided some inspiring and encouraging results on behalf of learning
objects,

in order that

LOs
could
have real impact on education and
become part of everyday
teaching and learning practice
s
, there are some critical issues that need to be addressed. T
he poor
availability of computers in schools is the greatest barrier that

prevents the welding of traditional
learning materials and LOs together. If every student had a laptop, everyday use of LOs
and their
integration into traditional materials
could become
reality.


To summarise:

1.

Learning objects can en
hance learning but th
ey do not guarantee

learning outcomes.

2.

Applied pedagogy
and contextualizing play

key role
s

in successful implementation of
learning objects.


3.

The educational value of learning objects is dependent on and is defined by the whole
learning environment / cont
ext where they are used.
It’s important to bear in mind the
complex nature of learning. As Parrish (2004) said it, the problems of education are
always more complex than technology alone can solve.
LOs are only tools to stimulate
learning processes, and hu
man processes involving
,

for example
,

communication,
Jaakkola & Nurmi

Academic impact of learning objects

28

collaboration, meaning making, inferring are more important (c.f. Collis & Strijker, 2004;
Jonassen et al., 1999).


4.

Different kind
s

of learning materials

provide specific affordances to support learning.

From the point of view of
flexible,
meaningful and effective teaching and learning,
combining LOs with traditional learning materials could be the most appropriate
approach.
Different kind of materials
-

whether they are digital or non
-
digital


should be

used flexibly when they best fit to the particular learning situation.




Jaakkola & Nurmi

Academic impact of learning objects

29

References


ADL. (2001).
Sharable content objects reference model.

Version 1.2. The SCORM overview.
Available:
http://www.adlnet.org/

But
son, R. (2003). Colloquium. Learning objects: Weapons of mass instruction.
British journal of
educational technology, 34(5)
, 667
-
669.

Campbell, L. (2003). Engaging with the learning object economy. In
A. Littlejohn (Ed.)

Reusing
Online Resources: A Sustain
able Approach to e
-
Learning.

London: Kogan Page.

CISCO systems Inc. (2000).
Reusable learning object strategy. Definition, creation process, and
guidelines for building.

Version 3.1.

Collis, B. & Strijker, A. (2004). Technology and human issues in reusing
learning objects.
Journal of interactive media in education, 4
. Available:
http://www
-
jime.open.ac.uk/4


Downes, S. (2000). Learning objects [Essay]. Available:
http://www.atl.ualberta.ca/downes/naweb/Learning_Objects.htm


Duffy, T. & Cunningham, D. (1996). Constructivism: Implications for the design and delivery of
instruction. In D. Jonassen (Ed.)
Handbook of research for education
al communications
and technology.
New York: Macmillan.

Duncan, C. (2003) Conceptions of Learning Objects: Social and Educational Issues. In A.
Littlejohn (Ed.)
Reusing Online Resources: A Sustainable Approach to e
-
Learning.

London:
Kogan Page.

Gibbons,
A.
S., Nelson, J. & Richards, R. (2000). The Nature and Origin of Instructional Objects.
In D. Wiley (Ed.)
The Instructional Use of Learning Objects
. Bloomington: Association for
Educational Communications and Technology.

Gredler, M. (1996). Educational games

and simulations: A technology in search of a (research)
paradigm. In D. Jonassen (Ed.) Handbook of research for educational communications and
technology. New York: Macmillan.

Hodgins,
H. W. (2000). The Future of Learning Objects. In D. Wiley (Ed.)
The In
structional Use
of Learning Objects
. Bloomington, IN: Association for Educational Communications and
Technology.

Hennessy, S. & O’Shea, T. (1993). Learner perceptions of realism and magic in computer
simulations.
British journal of educational technology,
24(2)
, 125
-
138.

Jaakkola & Nurmi

Academic impact of learning objects

30

IEEE. 2002.
IEEE P1484.12.1 Learning Object Metadata Standard.

New York: Institute of
Electrical and Electronics Engineers, Inc.

At: http://ltsc.ieee.org/wg12/

ISO
. (2003).
Business Plan for JTC1/SC36
(Standards for Information Technology f
or Learning,
Education, and Training).
Available:
http://jtc1sc36.org/doc/36N0651.pdf

Jaakkola, T., Nirhamo, L., Nurmi, S. & Lehtinen, E. (2004).
Erilaiset oppimisaihiot osana
joustavaa kokonaisuutta [Di
fferent types of learning objects as a part of flexible complex].

In L. Ilomäki (Ed.)
Opi ja onnistu verkossa


Aihiot avuksi
.
Hakapaino, Helsinki:
Opetushallitus.

Jonassen, D., Peck, K. & Wilson, B. (1999)
Learning with Technology. A Constructivist
Perspe
ctive.

Upper Saddle River, NJ: Prentice Hall.

Jonassen, D. & Reeves, T. (1996). Learning with technology: Using computers as cognitive tools.
In D. Jonassen (Ed.)
Handbook of research for educational communications and
technology.
New York: Macmillan.

de J
ong, T. & van Joolingen, W.R. (1998). Scientific discovery learning with computer
simulations of conceptual domains.
Review of educational research,
68(2), 179
-
201.

Kearsley,

G. (1998).
Educational Technology: A Critique
. Educational Technology, March

Apri
l, 47

51.

Koper, R. (2003) Combining reusable learning resources and services with pedagogical
purposeful units of learning. In A. Littlejohn (Ed.)
Reusing Online Resources: A
Sustainable Approach to e
-
Learning.

London: Kogan Page.

Koschmann,
T. (In press)
. Tools of Termlessness: Technology, educational reform, and Deweyan
inquiry. In T. O’Shea (Ed.)
Virtual Learning Environments.

Mahwah, NJ: Erlbaum.

Lajoie, S. P., & Derry, S. J. (Eds.). (1993).
Computers as cognitive tools

(pp. 1
-
401). Hillsdale,
NJ: Lawr
ence Erlbaum Associates.

Lambe, P. (2002).

The autism of knowledge management.

Available:
http://www.straitsknowledge.com


Lave, J. 1988.
Cognition in Practice.

Cambridge, MA: Cambridge University Press.

La
ve, J. & Wenger, E. (1991).
Situated learning.

New York, NY: Cambridge university press.

Littlejohn, A. (2003) Issues in Reusing Online Resources. In A. Littlejohn (Ed.)
Reusing Online
Resources: A Sustainable Approach to e
-
Learning.

London: Kogan Page.

Jaakkola & Nurmi

Academic impact of learning objects

31

McCormick, R. (2003). Keeping the Pedagogy out of Learning Objects.
Paper Presented in the
Symposium
Designing Virtual Learning Material

EARLI 10th Biennial Conference
Improving Learning, Fostering the Will to Learn
, 26
-
30 September, 2003.

McDermott, L. &
Shaffer, P. (1992). Research as a guide for curriculum development: An
example from introductory electricity.
American journal of physics,
60(11), 994
-
1013.

Morrison, D. Goldberg, B. 1996. New actors, new connections: The role of local information
infrastr
uctures in school reform. In T. Koschmann (Ed.)
CSCL: Theory and Practice of an
Emerging Paradigm
.
Mahwah, NJ: Erlbaum.

Olkinuora, E., Mikkilä
-
Erdmann, M. & Nurmi, S. (2004).
Evaluating the pedagogical value of
multimedia learning material: An experimental

study in primary school. In N. Seel & S.
Dijkstra (Eds.).

Curriculum, plans, and processes in instructional design. International
perspectives.

Mahwah, NJ: Lawrence Erlbaum.

Palincsar, A. S. (1998). Social constructivist perspectives on teaching and learn
ing.
Annual
Review of Psychology
, (49), 345
-
375.

Parrish, P.E. (2004). The trouble with learning objects.
Educational technology, research &
development, 52(1)
, 49
-
67.

Raven, J.C. (1958). Standard progressive matrices. Sets A, B, C, D and E. Cambridge: H.K
.
Lewis & Co. Ltd.

Reigeluth
, C. M. & Nelson, L. M. (1997). A new paradigm of ISD? In R. C. Branch & B. B.
Minor (Eds.)

Educational media and technology yearbook

(Vol. 22). Englewood, CO:
Libraries Unlimited.

Ronen, M. & Eliahu, M. (2000). Simulation


A bridge between theory and reality: The case of
electric circuits.
Journal of computer assisted learning, 16
, 14
-
26.

Scardamalia, M. & Bereiter, C. 1996. Computer Support for knowledge building communities. In
T. Koschmann (Ed.)
CSCL: Theory and Practice
of an Emerging Paradigm,
(pp. 249
-
268).
Mahwah, NJ: Erlbaum.

Shipstone, D.M. (1984). A study of children’s understanding of electricity in simple DC circuits.
European journal of science education,
6, 185
-
198.

Stamper, R. "Semantics," In R.J. Boland and R
. Hirschheim (Eds.),
Critical Issues in Information
Systems Research
, pp. 43
-
78, Wiley, Chichester, 1987.

Jaakkola & Nurmi

Academic impact of learning objects

32

Sveiby, K.E. 1997.
The New Organizational Wealth: Managing and Measuring Knowledge
-
Based Assets.

Berrett
-
Koehler Publishers, Inc. San Francisco.

Urdan
, T. A. & Weggen, C. C. (2000).
Corporate e
-
learning: Exploring a new frontier
.
Available:
http://wrhambrecht.com/research/coverage/elearning/ir/ir_explore.pdf

Veenman, M
.V.J. & Elshout, J.J. (1995). Differential effects of instructional support on learning
in simulation environments.
Instructional science,
22, 363
-
383.

Velleman, P.F. & Moore, D.S. (1996). Multimedia for Teaching Statistics: Promises and Pitfalls.
The Amer
ican Statistician
, 50, 217
-
225.

Vermunt, J. D. & Verloop, N. (1999). Congruence and friction between learning and teaching.
Learning and instruction, 9
, 257
-
280.

Watson
, D. & Downes, T. (2000). Communications in an era of networks. Projects, models and
v
isions chalenged by complex reality. In D. Watson & T. Downes (Eds.) Communications
and Networking in Education. Learning in a Networked Society, (pp. 3

8). Boston, BA:
Kluwer.

Wiley, D. (Ed.) (2000a).
The Instructional Use of Learning Objects
. Bloomington
, IN:
Association for Educational Communications and Technology.

Wiley, D. (2000b). Connecting learning objects to instructional design theory: A definition, a
metaphor, and a taxonomy. In D. Wiley (Ed.)
The Instructional Use of Learning Objects
.
Bloomingt
on, IN: Association for Educational Communications and Technology.

Zimmerman, B. J. & Schunk, D. H. (Eds.) (1989).
Self
-
regulated learning and academic
achievement: Theory, practice and research.

New York: Springer
-
Verlag.