Pen-based interactions to support interactive teaching and learning of Computer Science topics

walkingceilInternet και Εφαρμογές Web

22 Οκτ 2013 (πριν από 3 χρόνια και 10 μήνες)

93 εμφανίσεις

I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M












Pen
-
based interactions to support interactive teaching and learning of
Computer Science topics


ITrace Final Report for ACPP


University “Politehnica” of Bucharest


223434
-
CP
-
I
-
2005
-
IT
-
Minerva
-
M




December 2007








Adina Magda Florea, Eugeni
a Kalisz, Serban Radu, Irina Mocanu

University Politehnica of Bucharest

I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


Pen
-
based interactions to support interactive teaching and learning of
Computer Science topics


ITrace Final Report for ACPP


University “Politehnica” of
Bucharest


Adina Magda Flore
a, Eugenia Kalisz, Serban Radu, Irina Mocanu

University Politehnica of Bucharest

adina@cs.pub.ro
, ekalisz@cs.pub.ro,
sradu@cs.pub.ro
,
irinag.mocanu@gmail.com



Executive summary

The recent availability of pen
-
enabled devices such as Tablet PCs, pen
-
based (mobile)
devices, or interactive whiteboards fosters the opportunity of a new generation of natural
interfaces between the user a
nd the computing device. The new types of interfaces supported
by pen interaction allows novel approaches to human usage of computers, allowing the
development of both novel methods of problem solving and enhanced applications in various
areas., including
education. The pen enabled devices have the potential to significantly
change the educational process by providing new dimensions of classroom interactions based
on digital ink and drawing tools for writing, sketching, and drawing, and for real
-
time
collab
oration.

The pen technology can offer significant improvements in computerized learning
environments through the development of systems that support participative and collaborative
learning. Such systems encourage an active type of learning, the interacti
on between students
and instructors, new possibilities for electronic assignments, and better motivate students in
their learning endeavor. Therefore, it becomes important to study how digital ink can be used
in the learning process, how it can support dif
ferent learning and teaching styles, and which
pedagogical approaches can benefit from the use of pen
-
based techniques. When integrating
digital pen technologies, educators must re
-
think their teaching approach, must understand
how to produce and best take

advantage of new teaching resources and must be able to
develop and follow new pedagogical approaches.

The ITrace project gave us the opportunity to study and try to give some answers to the
above mentioned challenges by gaining experience in using pen
-
ba
sed interactions in several
learning contexts while teaching computer science topics, developing enhanced learning
scenarios and studying the impact of graphical interaction on students’ learning
performances, learning evaluation, and learning styles.

Spec
ifically, we have focused our work on several issues, namely: initial studies and
surveys on how computer science teaching and learning can benefit of pen interactions, on
one side, and on available software to support these interactions, on the other side
; study of
the impact of hand
-
written note
-
taking, sketching, and graphical annotation on learner's
preferences, learning styles, and the provided added value; use of pen based input and
graphical interaction for creating cognitive maps; development of an
interactive course and
assessment module on Data Structures and Algorithm; development of an interactive course
and assessment module on selected topics on Artificial Intelligence; integration of learning
I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


materials in a Learning Management System; study of

the impact of graphical interaction on
students’ learning and assessment; development of an interactive environment allowing pen
annotations and sketching, and the construction of concept maps; development of relevant
good practices of how to use pen
-
base
d interaction to increase the effectiveness of learning.

As a conclusion of our work, we think that pen based interactions and digital ink
technology have the potential to significantly change pedagogical approaches, production and
delivery of learning con
tent, and the quality of student teacher interaction. We also think that
our pedagogical experiences and developed learning resources are contributing to the current
knowledge of using this new technology in education.

During our work, we found that the ov
erall response to our pedagogical experiences was
positive and that the students enjoyed being part of the experiments, obtained improved
learning performances, were more motivated during the class and even, some of them, were
planning to buy digital ink e
nhanced devices of their own.

However, we share the view that there are some concerns related to: the deployment of the
technology, in particular connected to the availability of associated devices on a large scale;
effectiveness of the new pedagogical app
roach on different aspects of teaching computer
science; and commitment of the instructors to efforts required in adequately changing the
already available learning resources.



1. Who we are and what we aimed at

Founded in 1818, University ″POLITEHNICA″ of Bucharest (UPB,
http://www.pub.ro
) is
the oldest and most prestigious technical university in Romania. The foremost mission of UPB
is to educate students in science and technol
ogy by imparting knowledge and practical skills,
developing their creative thinking, and preparing them to address the demands of today
economy.

At UPB, 24,000 students are studying at Bachelor, Master, and Ph.D. levels in the
following fields: electrical
engineering, power engineering, automatic control and computer
science, electronics, mechanical engineering, system management, aerospace engineering,
transports engineering, material science engineering, industrial chemistry, economics
engineering, enviro
nmental engineering, mechatronics applied sciences.

The University houses 37 Research Centres, among which 4 were recognized as Centres of
Excellence at national level and 8 grew into Multi
-
User Research Infrastructures with the
support of the Romania
-

Wo
rld Bank Program. UPB has a comprehensive infrastructure with
modern research and teaching laboratories and an Intranet/Internet communication network. A
Scientific & Technological Park is currently under development at UPB, to bring real
-
world
technology
and management issues into its research laboratories and teaching.

The strategic lines of development are in fundamental research: micro and nanotehnology,
non
-
conventional technologies, modeling of biochemistry processes, signal & image
processing, intell
igent robots, cognitive systems; and in applied research: tribology,
environment engineering, clean transport systems, electrical vehicles, composite and
“intelligent” materials, wasted water treatment, ionic interchange process, biodegradable
polymers.

I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


UP
B is firmly integrated in the international academic community and shares the same
moral, educational, scientific, and cultural values. Due to its prestige, UPB has bilateral co
-
operation agreements with 74 universities from Europe, America, Asia and Afric
a, and is
member of international academic organizations such as: CESAER, EUA, IAU, AUPELF
-
UREF. UPB actively participates in R&D international programmes like: COST, FP5, FP6,
CORINT, NATO, etc. Amongst its current priorities, UPB aims at valorizing its h
uman
potential and logistic possibilities towards the full integration in the European Research Area.

The Faculty of Automatic Control and Computer Science (
www.acs.pub.ro
), one of the
biggest in UPB, offers undergrad
uate and graduate programmes in “Computer Science” and
“Automatic Control and Applied Informatics.” With over 3400 undergraduate, M.Sc. and
Ph.D. students, and 270 faculty members, the Faculty is consistently among the top
-
ranked in
Romania. Recognizing th
at research is on the critical path to Romania’s integration in the EU,
the main goal of the Faculty is to maintain an outstanding record in teaching, research, and
innovation in IT and advanced control. Its research excellence has been confirmed at both
n
ational and international levels.

Areas of faculty expertise include: high performance computing, distributed systems, VLSI,
artificial intelligence, relational databases, graphics, computer networks, human
-
computer
interaction, intelligent control systems
, bioengineering, dynamic and real
-
time systems,
industrial process control, signal processing and communication, discrete event systems,
diagnosis. The Faculty houses modern laboratories with significant computing resources and
advanced technological plat
forms. Its future research priorities are in intelligent autonomous
systems, adaptive enterprises, Grid computing and services, ubiquitous computing and
communication.

APCC is the Excellence in Research Center for Automatics, Process Control and
Computers
and includes several well established research domains and laboratories, both in
System Control and in Computer Science.

Founded in 1997, AI
-
MAS Laboratory (
http://turing.cs.pub.ro/ai_mas
) focuses its
researc
h on multi
-
agent systems, with special interest in coordination mechanisms, automated
negotiation, multi
-
agent learning, MAS architectures and autonomy. Members are also
involved in researches related to models of affective computing, evolutionary agents,
intelligent agents in e
-
learning, and intelligent agents in CSCW.

The AI
-
MAS laboratory has been involved in the development of several national and
international R&D programmes and grants, and maintains cooperation relationships with
similar laboratories
and computer science departments in European universities, such as Ecole
Polytechnique de l’Université de Nantes, Ecole Nationale Superieure des Mines de Saint
-
Etienne, Université Paris 13, Free Univerity of Amsterdam.

The AI
-
MAS Laboratory has been
member

no
. 21

“AgentLink, Network of Excellence
for Agent
-
Based Computing”, EU FP5 and member no. 117 of AgentLink III: EU FP6
Co
-
ordination Action for Agent Based Computing, EU FP7.

Members of AI
-
MAS Laboratory are professors, associate professor, lecturers and

assistants. Among the courses they are teaching, we can mention: Programming Languages,
Data Structures and Algorithms, Artificial Intelligence, Multi
-
agent Systems, and others.
I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


Some of the AI
-
MAS members are also teaching course at the Faculty of Enginee
ring in
Foreign languages of University “Politehnica” of Bucharest, at the Electrical and Computer
Science Department, both in the English Stream and in the French stream.

Several AI
-
MAS members were involved in the ITrace project and aimed at gaining
expe
rience in using pen
-
based interactions in several learning contexts while teaching
computer science topics, developing enhanced learning scenarios and studying the impact of
graphical interaction on students’ learning performances, learning evaluation, and

learning
styles.

In the framework of the ITrace project, we have investigated and try to prove how the new
technology of digital ink offers flexibility and a range of pedagogical expressions that can
achie
ve several educational goals,

support a more parti
cipative att
itude of students in learning,
a
nd how this attitude impacts the amount and quality of acquired knowledge.

Among our specific goals, we can indicate: the investigation of pen
-
based input and
graphical interaction in teaching and learning to und
erstand how annotational capabilities may
be used in educational activities; the impact of using pen based input and graphical interaction
for creating cognitive maps; the investigation of the impact of graphical interaction on
students’ learning and asses
sment; the development and deployment of an interactive course
and assessment module on Data Structures and Algorithm and its integration into a LMS; the
development and deployment of an interactive course and assessment module on Artificial
Intelligence;
the study of the impact of hand
-
written note
-
taking, sketching, and graphical
annotation on learner's preferences, learning styles, and the provided added value; and
contributions to the development of a reference model on how to use pen
-
based interaction
to
increase the effectiveness of learning.


2. Interactive learning: how to learn and teach using the digital pen

Interactivity in learning is generally recognized as one of the key aspects of improving
students’ performances and motivation during learning
. Several papers in existing literature
report on the benefits of using pen
-
based interaction to support interactivity in learning.

2.1 Teaching using the digital pen

The first goal of our work was to investigate the use of pen
-
based input and graphical
in
teraction to understand how annotational capabilities may be used for educational activities.
In order to achieve this goal we have conducted a study of the current experience of using pen
based interaction in teaching Computer Science, a study of the exis
ting techniques and
pedagogical approaches that try to exploit pen
-
based interaction to improve learning outcomes
and to stimulate a participative learning style, but also a study of the existing software tools
that allow pen based interaction. This last s
tudy included open source and/or free products and
proprietary software products.

Our study was based on several available on
-
line articles that are presented and
summarized at (ITrace: Web: Pen papers, 2007; ITrace: Pen Review).

Slide
-
based annotations sy
stems are quite wide spread (Anderson et. al., 2004; Simon et.
al., 2004). Most of them were born from research projects at different universities cross over
I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


the world, and had the goal to integrate new digital ink technologies in systems that are used
to
deliver lectures. Two relevant examples, in this sense, are:



Classroom Presenter, a system developed at the University of Washington and that can be
freely (for educational use) downloaded from the Internet



DyKnow, a project started at the DePauw Univers
ity and which nowadays is a widespread
commercial product, used in active and collaborative learning

These systems, which run on a Tablet PC or a pen
-
based mobile computer allow the
instructor to handwrite over computer
-
projected slides. The slides and in
k are then multicast to
other machines for students’ use. Students have the ability to take notes with the pen and to
offer real
-
time anonymous feedback to the instructor. The ink annotated slides can be saved
for review after lecture, or made available el
ectronically to the students.

Although digital slide projection is controversial it has a number of advantages, including
the ability to structure material in advance, prepare high quality examples and illustrations,
easily share and reuse material and fac
ilitates distance learning. These kind of systems seemed
to be well received by the students. Their general opinion was that such systems can
considerably increase spontaneity in lecture presentation. They also feel that they are
encouraged to engage in cl
assroom activities and to work in teams. Exchanging ideas with
their colleagues on a particular topic help them to construct new knowledge.

There are no or very few reported experiences on pen
-
based annotations systems for web
pages. There are several exam
ples of textual and graphical annotations systems for web, but
none of them had support for digital ink. The best known tools for web annotation are listed at
(Web annotation, 2007). It seems that most of them accept textual not graphical annotations.

Two
representative examples of such web technologies are:
Annotea

and
OntoMat
. These
are Sema
n
tic Web based projects which use RDF (Resource Descript
ion Framework)
metadata and OWL(Web Ontology Language)
-
markups. One big benefit using Semantic Web
technologies and metadata is that user generated metadata can be easily combined and reused
in many other applications.

The technologies mentioned above kee
p annotations and bookmarks in specialized objects
which can be extended. These objects are web resources that have a URI, contain some RDF
metadata and normally include a property referring to some other Web resource. Annotations
are used for sharing comm
ents, notes, questions, explanations, discussion threads and so on
and provide better collaboration over the web. One main advantage provided by these systems
is that all the data can be structured on topics and categories and therefore can be easy
manipul
ated.

The study on existing products dedicated to digital ink annotation included the following
products: SATIN


A Toolkit for Informal Ink
-
based Applications; kAWT; JTablet
SketchStudio; OpenOffice SDK; Classroom Presenter; Microsoft OneNote 2003; Corel
Grafigo 2; DyKnow Vision; DyKnow Monitor; Adobe Acrobat Standard 7.0; Waba; Ewe;
riteForm Local SDK; Groove Workspace. The review of each of the products is available at
(ITrace: Pen Software, 2007).


I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


2.2 Pen Annotator

We have developed a software module,
called Pen Annotator that takes advantage of pen
-
based technology to encourage active and collaborative learning
.

It
s

main aim is

to annotate
documents using a digital pen. These annotations can be graphical (underlying lines , circles
around a portion of

text, bullets, checkmarks or other types of graphical
objects
), or textual,
permitting the user to add hand
-
written text to the image.

This software is intended to improve natural interaction with electronic documents through
the

use of pen and digital in
k. It
s goal is to facilitate annotations on documents allowing
handwritten complex notes taken on documents as well as creation of notes from blank pages.
Particularly interesting is the “movie mode” feature which allows the user to see how a
complex drawi
ng had evolved when it was created.

The following features describe the application:



create a blank page where annotations can be added



load a locally stored image for annotation



annotate the loaded image with different colors and line widths



save the ann
otated image locally and make it available through the web



easily switch to next image in the folder or project



easily switch back to previous image in the folder or project



page erase, stroke erase functionality



ask the user if he wants to save a yet unsa
ved annotation when a switch to another image
is being made



allow the user to define a project in which to include as many images as he likes and load
them as a whole just by loading the project



allow the user to optionally enter a username used to “sign”
his own projects



movie mode

-

the user has the ability to see the annotations as they progressed when they were made

-

for this he chooses to enter movie mode when annotating an image, chooses
“automatic save every x seconds” with the ability to set x to any
value he wants or
“save manually when asked” and then starts annotating the image

-

the system will record the progress every x seconds as specified or when the user
chooses if manual save is selected

-

then, at a later time he can review the image with annota
tions in steps: first what has
been annotated after x seconds, then what the image looked like after another x
seconds, etc.

Using the Project menu the user has the ability to define a project in which to include as
many slides or images as he likes and lo
ad them as a whole just by loading the project. In the
right part of the graphical user interface is a Project view window where the user can visualize
existing slides/images in the project, can add new images, can remove selected images , can
move a prev
iously selected image one position up or one position down in the image
hierarchy, all these by clicking the associated buttons.

The application was developed in Java (the user has to have JRE 5.0 installed) and was
integrated in a “2in1” application with

the concept map builder @Graph presented in Section
3.3.


I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M




Figure 1. Use of Pen Annotator



Figure 2. Importing a directory in Pen Annotator



3. Meaningful learning: building concept maps using the pen

Concept maps offer a method to represent inform
ation visually. Concept maps harness the
power of our vision to understand complex information “at
-
a
-
glance.” Our aim was to
investigate the impact of using concept maps in teaching computer science and how this
impact can be influenced by the fact that co
ncepts maps are drawn with a pen instead of using
“classical” computer supported drawing tools (mouse, drag and drop figures, etc.)

3.1 Concept maps in learning

Concept maps are represented by diagrams that contain nodes and labeled arrows (links
between n
odes). The nodes correspond to important concepts in a domain, denoted by one or
more words, and are enclosed by a circular or rectangular border but other forms are also
possible. The labeled arrows denote a relation between a pair of concepts (nodes) and

the label
I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


on the link state the relationship between the concepts. The arrow describes the direction of
the relationship and reads like a sentence. The combination of two nodes and a labeled arrow
is called a proposition. A proposition is the basic unit o
f meaning in a concept map and the
smallest unit that can be used to judge the validity of the relationship drawn between two
concepts (Dochy, 1996).

Originally, concept mapping was developed by Joseph D. Novak at Cornell University in
the 1980s, as a way
to increase meaningful learning in the sciences. Novak based his work on
David Ausubel’s theory of meaningful learning that stated that prior knowledge is used as a
framework for understanding and learning new knowledge, in other words learning new
knowled
ge is dependent on what is already known (Novak,1991). More specifically, new
knowledge gains meaning when it can be substantively related to a framework of existing
knowledge rather than being "processed and filed" in isolation according to more or less
a
rbitrary criteria. Concept mapping supports the visualization of such conceptual frameworks
and “stimulates prior knowledge by making it explicit and requiring the learner to pay
attention to the relationship between concepts” (Jonassen, 1993).

Nowadays c
oncept maps are used in education but also in business, management, and
research. Examples of concept maps usage are : note taking and summarizing, knowledge
elicitation for individual expert knowledge and team knowledge, knowledge capture, new
knowledge c
reation: e.g., transforming tacit knowledge into an organizational resource,
mapping team knowledge, collaborative knowledge modeling and the transfer of expert
knowledge, facilitating the creation of shared vision and shared understanding within a team
or

organization, trainings, increasing meaningful learning, communicating complex ideas and
arguments, strategic planning, product development, market analysis, decision making,
measurement development, tools to support the interviewing process in knowledge
acquisition
from experts.

Concept mapping is the strategy employed to develop a concept map. There are many
approaches regarding the development of a concept map. Some sustain a hierarchical
downward structure with the most enclosing concept at the top of
the map and the more
specific concepts at the bottom. Other approaches are less restrictive and present a variety of
structures, as presented in the category classification of concept maps further on.

Another encountered rule states that general concepts
should be enclosed in circles, while
the particular instances of objects should be enclosed in rectangles. Another version says that
specific examples of objects are not to be represented except for the cases in which there
presence is relevant as an examp
le of a given concept that helps clarify its meaning. In this
case these are not included in ovals or boxes, since they are specific events or objects and do
not represent concepts. As a general rule to embrace most of this approaches, the layout is not
as

important in constructing a concept map as long as it is hierarchal structured and concepts
are represented from general to particular and the relationship between concepts is marked and
labeled.

We can identify four categories of concept maps. These are
distinguished by their different
format for representing information, as follows:

I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M




Spider concept map
. The “spider” concept map is organized by placing the central theme
or unifying factor in the center of the map. Outwardly radiating sub
-
themes surround th
e
center of the map.



Hierarchy concept map
. The hierarchy concept map presents information in a descending
order of importance. The most important information is placed on the top. Distinguishing
factors determine the placement of the information.



Flowchar
t concept map
. The flowchart concept map organizes information in a linear
format.



Systems concept map
. The systems concept map organizes information in a format which
is similar to a flowchart with the addition of “inputs” and “outputs”.

3.2 Meaningful le
arning through concept maps

Assuming that knowledge within a content domain is organized around central concepts, to
be knowledgeable in the domain implies a highly integrated conceptual structure. Concept
maps, then, represent some important aspect of a s
tudent’s declarative knowledge in a content
domain (Jonassen, Beissner,
and Yacci, 1993; White and

Gunstone ,1992).

As a learning tool,

concept maps can contribute to meaningful learning by knowledge
capture, knowledge representation, integration of new kn
owledge with existing one and even
knowledge elicitation (Bareholz and

Tamir, 1992; Novak, 1990).

Being a form of visual
representation, concept mapping has several advantages: visual symbols are quickly
recognized, minimum use of text makes it easy to sca
n for a word, phrase, or the general idea,
and visual representation allows for development of a holistic understanding that words alone
cannot convey.

The stages to build a concept map can be briefly summarized as follows:



Selecting the subject referred t
o by the concept map.



Defining the context for the concept map in the form of a concept or a question
called Focus Question, that specifies the problem or issue the concept map should
have to resolve.



Extracting the key concepts in the form of a list. Depe
nding on the concept maps
purpose this can be done either from memory or from reviewing the material.



The list of concepts resulted is then ranked from the most general concept to the
most specific ones. An intermediary ordered list can be used for this p
urpose.



After that the concept can be placed on the map respecting the hierarchical order no
matter of the chosen layout.



Concepts have to be linked together using meaningful linking words. Cross
-
references can be added in they exist.

This process results

in a primary concept map that is revised afterwards by adding new
concepts, links or by moving around existing ones. The final concept map is ready after a
number of such revisions.

I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


Intuitively, the use of concept maps to evaluate students’ declarative kn
owledge structure is
appealing. A student’s map construction directly reflects, to some degree, her or his
understanding in a domain.

However some barriers prevent their widespread use in teaching and evaluation of students’
knowledge. Some barriers are re
lated to the difficulty of building such a map on a computer
by using traditional interfaces, especially if the instructor or the student wants to do this
interactively, during the class. Drawing a concept map with a pen helps the person focus on
the core
of the process, makes the drawing easier and prevents being disrupted by the details
of drawing with classical drawing tools.

3.3 The Concept map builder

We have developed a software tool named @Graph that allows the development and
management of concept m
aps and can be run on either classical PCs, for which concept maps
are drawn with a mouse, or on Tablet
-
PC or PCs equipped with a pen, for which concept maps
are drawn with the pen. Moreover, the software allows the possibility to combine a concept
map des
igned with pre
-
defined forms (mouse
-
based) with pen
-
based annotation of the map.
One of the main advantages of our tools is the possibility to use files in both an internal
specific format and in JPEG or GIF formats. This ensures the interoperability of th
e didactic
material with other available software.

When creating a concept map, one can either create a concept map from scratch, or load
and enhance an already existing concept map (by adding nodes, links, and/or naming nodes
and links), either by using a
vailable commands in the upper menu or by using pen
-
based
interaction. When using menu commands, a map can be drawn either from scratch or from a
given lists of concepts that can be loaded for the description of a certain key topic.



Figure 3. Menu based

built concept map



I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M




Figure 4. Pen
-
based enhanced concept map

Figure 5. Free
-
hand drawn concept map


Among the many functionalities of the software we can note: drawing nodes with different
shapes and colors, drawing labeled arrows between nodes, add
ing keyed text with formatting
or free text on nodes and links, selecting, copying and pasting parts of a map or the entire
concept map, saving and retrieving concepts maps. Besides these basic functionalities, the
software allows also to insert pictures i
n nodes, expand and collapse nodes, auto
-
arrange
nodes and links in a page, and use template concept maps in designing more complex ones.
Figure 3 shows a concept map drawn with menu commands and a map, Figure 4 shows an
enhanced concept map with free hand

sketching, and Figure 5 shows a concept map drawn
entirely in free hand.


4. Pedagogical scenarios: coordinating interactive engagement

We have conceived our experiment to cover two aspects of teaching: lecture teaching and
laboratory work (Florea, Radu,
2007). In both cases, the instructor has a tablet PC connected
to a video projector and to the local area network. At the beginning of the class, the students
are equipped with tablet PCs and receive the slides of the lecture or the slides corresponding to

the laboratory topic, i.e., what assignments they have to do during the laboratory. Usually, the
assignments are organized in what we call a project


an ordered collection of slides relevant
to the topic.

We have developed scenarios for students’ assignm
ents and assessment in which we
included both programming and non
-
programming exercises. Programming exercises were
developed in the normal mode during assessment (i.e., using the associated programming
environment), while non
-
programming exercises implied

several activities, among which
many were requiring hand
-
written responses that were input using the digital pen.

Because of the limited number of Tablet PCs that we possessed, we have scheduled that
during each lecture, every student has at least three t
imes access to Tablet PC
s during the
entire
lecture. During laboratory assignments and assessments, because of mixing
I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


programming and non
-
programming exercises, we have scheduled the work in such a way so
that all students have access to pen
-
based interact
ions.

We have tested our scenarios in two different settings: the first setting was the Data
Structures and Algorithm course and the second one was a part of the Artificial Intelligence
course. The courses were given at the undergraduate level for the stud
ents at the Department
of Engineering Science (a department of University “Politehnica” of Bucharest with all
teaching given in foreign languages, namely English, French and German), the English
stream.

4.1 Pedagogical scenarios

Pedagogical scenarios for l
ectures

The instructor starts by presenting the learning goals of the class either by using a concept
map (previously built) or by using bullet text. Then, she presents the lecture topics by going
through the prepared slides and annotating them, when neces
sary, in different manners:



selection of text by underling or circling group of words, or drawing catch attention
symbols in the margin of certain sections of the text;



entering short text as group of words to further explain a concept;



building associatio
ns by making links to other items of the presentation, including nodes
in the learning goals concept map;



drawing new diagrams or drawing additional elements on diagrams already existing on
the slides;



writing or drawing examples or completing partially fi
lled examples on the slides.

During the lecture, the instructor may ask the students to:



develop short exercises in order to practice the acquired knowledge, for example “draw
the binary tree obtained from a sequence of keys” or “show how an element will b
e
removed from a linked list by emphasizing pointer modification”;



answer 2
-
3 short questions to catch misconceptions, for example “when can we remove an
element from a stack?”

By the end of the presentation, the instructor may:



draw, interactively, a gene
ral concept map of the concepts presented during the class;



mark on that concept map the associated relevant slides in the lecture;



ask the students to draw a concept map for a particular concept taught during the class,
discuss and modify with the class o
ne such map;



save the annotated slides together with the topics concept map, to be made available to the
students.

According to the teacher preference, the slides may be saved with all drawn annotations
(end
-
project) or gradually, one different slide for e
very annotation, so as to keep in the slides
the sequence in which additions were made (step
-
by
-
step project). During the lecture, the
I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


students may take their own notes, including slide annotation. In the end of the lecture, they
can save their own annotat
ed version of the slides.

Pedagogical scenarios for laboratory work

Before starting the work, we have organized a preparatory laboratory during which one
hour was dedicated to expose the students to the use of pen on the tablet PC, to let them
practice wit
h the new device, and to explain to the students different approaches of building
concept maps. We have found that it is not always obvious,
f
rom t
h
e first
exposure
, how to
use the pen and how develop a concept map. Therefore, preparatory exercises and exa
mples
greatly enhance the desired result of pen
-
based interactions and the use of maps as an
instructional tool. However, getting acquainted with the pen usage required less time and
effort than learning how to properly build a concept map.

During the labo
ratory, students were asked to perform both programming exercises and
exercises that required use of the pen (non
-
programming exercises), such as:



draw flowcharts or pseudocode of the program they are going to implement;



solve exercises that require depict
ion of data structures and show each step of its building
according to a given algorithm;



illustrate the functioning of an algorithm on a particular instance of input data


step by
step following the algorithm;



point out errors in a solution of an assignm
ent acting as a teacher who corrects a paper,
namely correct the errors in red and try to explain why the solution is erroneous;



justify a formula presented at the lecture;



draw proof trees;



draw concept maps for topics covered by laboratory work or annota
te existing concept
maps, for example further develop concepts and relationships;



write the solution of a programming exercise instead of keying and running it.

Pedagogical scenarios for students’ assessment

Student assessment took place either interactive
ly during a limited time frame during
laboratory or by giving homework assignments to students and evaluated afterwards by the
teacher. To this last aim, access of the students to the laboratory where pen
-
based enhanced
computing devices were available was

granted.

Every assignment had an associated number of points and, for the non
-
programming
exercises, corrections were made using digital ink. Assignment feed
-
back and corrected
assignments were given to the students.

The assessment scenario for non
-
progra
mming exercises mimicked in fact the one that was
classically performed with pen and paper for student work or exam papers. The basic
advantage of using digital ink was obviously that both students and teacher can keep copies of
corrected assignments and r
eview them when necessary. Novelty of having electronic
corrections was also an attracting factor.

I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


Pedagogical scenarios for using concept maps

One of the important aspects of our scenario was to have an introductory laboratory (in fact
a part of a laborat
ory) in which to present to the students what a concept map is and how one
can develop such a concept map according to one or several structured methods. The stages
that one has to follow in order to build a concept map were presented in Section 3.2. We fo
und
out that this introduction to concept mapping was essential for the proper exploitation of this
pedagogical tool by the students.

We have employed concepts maps in different ways and also the construction of concept
maps was different, according to the

students’ preferences but also according to what the
teacher asked at a particular moment.

The specific contexts in which we employed concept maps were the following:



By the teacher when introducing course goals;



By the teacher when summarizing important
concepts;



By the students when asked to present acquired concepts;



By the students in their own learning process (the teacher asked them to use
concept maps on their own learning process but this activity was not supervised or
evaluated).

The method for cr
eating concept maps by the student varied also. Therefore, we have asked
the students or let them choose among several methods, namely:



Draw a concept map from scratch to show a given concept or topic;



Refine and/or complete a concept map from a partially
built and/or filled concept
map;



Draw a concept map by selecting nodes from a list of important concepts given by
the teacher; in this case the students are require to organize and link the concepts
presented in the list.

Moreover, the students were free t
o choose if to draw the concept map using menu
commands or freely using pen input.

4.2 Data Structures and Algorithms Course

We have developed and deployed (starting from our previous materials on the subject) an
interactive course and assessment module on

Data Structures and Algorithms, in which we
have applied the scenarios described above for lecture teaching, laboratory work and
assessment. The topics covered by the course are presented in what follo
ws. We have made
our experiments

with a group of 25 pe
ople for the first course and with a group of 20 students
for the second.

Course content



Algorithms complexity



Vectors and arrays



Linked lists: simple linked lists, double linked lists, operations on lists

I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M




Trees: tree traversal, binary trees, binary search

trees, operations on trees



Hash tables: construction, search



Graphs: directed, undirected, graph traversal

For the laboratory work and assessment exercises, we have conceived two types of
exercises:

-

programming exercises


not developed using pen (a)

-

non
-
programming exercises (b)

The goals of non
-
programming work exercises during laboratory/assessment were:



Information visualization



Data structure recognition



Understanding of data structures representation in memory



Understanding the concept of an abst
ract data type regardless of the representation



Incremental building of data structures for particular cases



Discrimination abilities to highlight relevant parts of data structures



Illustration of functioning of presented algorithms on particular cases



Dev
elopment of algorithms in pseudo
-
code



Understanding of algorithms analysis by formula justification



Understanding the conceptual relationships among presented concepts



Understand conceptual variation in organizing concepts














Figure 6. Examples of DSA exercises




I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M









Figure 7. Example of DSA learning goals

Assessment feed
-
back for non
-
programming exercises (b) was given using digital ink
correction, while assessment feed
-
back for programming exercises (a) was given
in the
traditional way.

4.3 Artificial Intelligence Course

We have developed and deployed (starting from our previous materials on the subject) an
interactive course and assessment module on Artificial Intelligence, in which we have applied
the scenarios d
escribed in Section 4.1 for lecture teaching, laboratory work and assessment.
The topics covered by the course are presented in what follows. We have made our
experimented with a group of 15 people.

Course content (the selected chapters)



Basic search techn
iques



Informed search techniques



Propositional logic



Predicate logic



Resolution and theorem proving

For the laboratory work and assessment exercises, we have conceived three types of
exercises:


(a1)
-

multiple choice exercises to test understanding of the

presented concepts and
techniques


not developed using the pen

(a2)
-

programming exercises


not developed using pen

(b)
-

non
-
programming exercises

The goals of non
-
programming exercises during laboratory/assessment were:



Information visualization of t
he problem search space



Understanding of problem representation and solution representation



Illustration of functioning of presented algorithms on particular cases



Understanding proof tree of Prolog programs



Understanding unification of expressions


I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M




Underst
anding theorem proving using refutation resolution



Understanding the conceptual relationships among presented concepts



Understand conceptual variation in organizing concepts








Figure 9. Examples of AI exercises solutions given by students









Figure 10. Examples from an AI lecture

Assessment feed
-
back for non
-
programming exercises (b) was given using digital ink
correction, while assessment feed
-
back for programming exercises (a1, a2) was given in the
traditional way.

4.4 Integration in a Learn
ing Management System

We have set up an integration of the DSA course in Moodle (2007). Moodle is a course
management system, more precisely a free open source software package designed using
sound pedagogical principles, to help educators create effective

online learning communities.
Moodle has a large and diverse user community with over 330,000 registered users. We use
Moodle in the Department of Computer Science as an on
-
line repository and interaction
environment for our taught classes.

Moodle has a la
rge number of features and offers the possibility to develop different
instructional management strategies. There are several roles in the system (and one can define
new roles), among which the most relevant ones are: administrator, course creator, teacher
,
non
-
editing teacher, student, etc.





I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


There are a number of interactive learning activities, among which we can quote:
assignments, chat, forum, lesson, quiz, forum, survey, and feedback questionnaires (this last
one has to be specifically added as a plug
-
i
n). There is also the possibility to add a number of
resources, such as files, web pages, text pages, link to files or web pages, etc. ach course
homepage generally contains blocks on the left and right with the centre column containing
the course content.

Blocks may be added, hidden, deleted, and moved up, down and left/right
when editing is turned on. Latest news, blogs, upcoming events, and recent activity are a few
examples.

Students may be enrolled in a course (or several courses, in fact) and, once en
rolled, they
have the possibility to access the course resources, post assignments, participate in forums,
chats and blogs. The teacher managing a course has the possibility to view students’ posted
assignments, grade assignments, post news in forums, answ
er questions asked by the students,
view results of surveys and feed
-
back questions.

We have defined the following sections for our course:



news forum


place for the teacher and assistants to insert announcements about the
course, laboratory and assessmen
ts;



course description


description of the course, its aims and its syllabus, including
references and grading rules



course feedback


a questionnaire requesting students to send general feedback about the
course, how useful it has been, quality if the co
urse, etc.



pen
-
based enhanced learning feedback


the pen
-
based enhanced learning questionnaire,
second version (see Annex 2) where students were asked to answer the questions
regarding the use of pen, their appreciation of the technique, and their learnin
g style;



learning style questionnaire


a link to the Felder and Silverman learning style
questionnaire (Section 5.2) where students can take the test in order to find out about their
learning style;



questions forum


where students can ask questions about

the course, about the
assignments, and where both other students and teacher/teaching assistants can respond to
these questions, the answer being available to all enrolled students.

We have posted on the LMS the slides of the course, both
t
he initial vers
ion of the slides
and the annotated version. In time, we have found out, based on student reactions, that only
the annotated version was consulted by the students.

The environment permits also the management of student assignments. The teacher or the
teach
ing assistant
uploads

assignments on the site, with an associated due date. Then students
have to submit the answers, in particular they can load a file with the answers. In our case, the
students have to submit a file that was previously develop either in

a programming
environment, for programming exercises, or a file developed using digital ink in which they
can give answers to non
-
programming exercises (Sections 4.2 and 4.3). The environment
allows the teacher or the teaching assistant to grade the assig
nments on
-
line, to assign grades
to assignments (as a number of points from 100), to write general comments about the
I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


assignment and to upload a file with corrections to exercises. Students can then view their
grades and associated corrections.

A very usef
ul facility of the LMS was the Feedback questionnaires (Figure 11) that can be
defined, which allows the definition of questions containing several types of answers, the
marking of questions which have required answers or optional answers, and provides
ano
nymous feedback from the students. The answers are automatically collected and counted
by the system, according to the different choices specified in each questions, and the results
can be exported as an Excel file.



Figure 11. LMS questionnaire


ques
tions marked in red have required answers


5. Supporting differences: influence of learning styles on teaching and
learning

People perceive information differently. Students learn in many ways, either by seeing and
hearing, or thinking logically and acting
, drawing analogies or reflecting on the overall
information that they received while being taught, and so on. Teaching methods also vary as
some instructors lecture, others like discuss ideas and approaches, some emphasize memory
and others understanding.

Ideally, a student’s learning style has to be matched by the teaching
style. However, we face two problems: how to find out a student’s learning style and whose
responsibility is to best fit a learning style with a learning approach, namely is the teacher

responsible for matching different learning styles or is the student responsible to learn
according to his/her learning style. Moreover, we should try to investigate if these two
problems can be tackled also by use of pen
-
based input and feed
-
back.

5.1 Le
arning styles

A learning style is a student’s consistent way of responding to and using stimuli in the
context of learning. Learning styles are “characteristic cognitive, affective, and psychological
behaviors that serve as relatively stable indicators of
how learners perceive, interact with, and
respond to the learning environment” (Keefe, 1979). Learning styles tend to be relatively
stable over time, in other words are predictable, but are not static in that there can be some
variation from day
-
to
-
day, we
ek
-
to
-
week, and as one ages (Price, 2004).

Students are characterized by different learning styles, respond well to different types of
information and favor different ways of acquiring new knowledge. Teaching methods also
I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


vary (Felder, Brent, 2005). Some i
nstructors lecture, others demonstrate or lead students to
self
-
discovery; some focus on principles and others on applications; some emphasize memory
and others understanding. A student learns best when his/her learning style is matched by the
teaching sty
le and tend to be frustrated and have poor performances otherwise. To maximize
student learning, it is important for instructors to address the variety of learning styles when
designing and delivering their lessons.

Although its origins have been traced ba
ck much further, research in the area of learning
style has been active for around four decades. During that period the intensity of activity has
varied, with recent years seeing a particularly marked upturn in the number of researchers
working in the area
. Also of note is the variety of disciplines from which the research is
emerging. Increasingly, research in the area of learning style is being conducted in domains
outside psychology
--
the discipline from which many of the central concepts and theories
ori
ginate. These domains include medical and health care training, management, industry,
vocational training and a vast range of settings and levels in the field of education.

We have investigated several learning style models, among which the VARK model
(Fle
mming and Mills, 1992; Flemming, 2001; Web on VARK, 2006), the Kolb learning styles
inventory (Kolb, 1984, Web on Kolb, 2007), the Honey and Mumford Learning Styles (Honey
and Mumford, 1986), and the Felder and Silverman learning style model (Felder and
Si
lverman, 1988).

The
VARK model

identifies four different media and, respectively, four distinct learning
styles. These styles are visual, aural, reading/writing, and kinesthetic. The name of the theory
is the acronym of these four terms. For people with au
ral preference speech is the preferred
and most efficient way of receiving information. Students with this preference learn best from
lectures, discussions etc. People from the reading/writing group prefer to receive information
from written or printed wor
ds. Students with this preference learn best from textbooks, lecture
notes, handouts, etc. Members of the third group, visuals, like information to arrive in the
form of graphs, charts, various diagrams etc. They are particularly sensitive to matters like
color coding or spatial layout. The last group, kinesthetic, need
s

concrete, multi
-
sensory
experience. They learn by doing. Students with this preference learn best from practical
sessions, field trips, experiments, role playing or simulation, etc. In orde
r to acquire
conceptual and abstract material they need it to be accompanied by analogies, metaphors, and
real life examples. An immediate addition to this four groups classification are the groups of
various multi
-
mode preferences. People with multi
-
mode
preferences, “lucky ones”, can get
information using several ways equally well.

The
Kolb learning cycle

model of learning suggests that successful learning should pass
through the following stages: Concrete Experience (having an experience), Reflective
Obs
ervation (reviewing the experience), Abstract Conceptualisation (concluding from the
experience), and Active Experimentation (planning the next steps). Kolb classifies learners as
having a preference for concrete experience or abstract conceptualisation; a
nd for active
experimentation or reflective observation. According to Kolb, concrete perceivers absorb
information through direct experience
-

doing, acting, sensing, and feeling; abstract perceivers
take in information through analysis, observation, and t
hinking; active processors make sense
of an experience by immediately using the new information; reflective processors make sense
of an experience by reflecting on and thinking about it.

I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


The
Honey and Mumford learning styles

is developed from Kolb’s inven
tory and learning
cycle and has four components: activists; reflectors; pragmatists; and theorists. Activists learn
best from activities where they can throw themselves into a task. Reflectors learn best when
they can review what has happened. Theorists le
arn best when they can understand what they
have learned as part of a wider picture. Pragmatists learn best when an opportunity presents
itself to learn on the job.

The
Felder and Silverman learning style

model is the one we have chosen and which is
presen
ted in the following section.

5.2 Selected approach

We have selected the
Felder and Silverman model

(Felder and Silverman, 1988; Felder,
Felder and Dietz, 2002; Felder and Brent, 2005) for its accuracy and simplicity but also
because we have considered it
as being appropriate for Computer Science students. The model
was originally formulated by Dr. Richard Felder in collaboration with Dr. Linda Silverman, an
educational psychologist, for use by college instructors and students in engineering and the
science
s, although it has subsequently been applied in a broad range of disciplines.

According to this model, students may be classified along four dimensions:



Active learners

understand new information by doing something with it while
Reflective learners

prefer
to think about new information first before acting on it;



Sensing learners

like learning facts and solving problems by well established
methods while
Intuitive learners

prefer discovering new relationships and can be
innovative in their approach to problem

solving;



Visual learners

understand new information best by seeing it in the form of
pictures, demonstrations, diagrams, etc, while
Verbal learners

understand new
information best through written and spoken words;



Sequential learners

understand new inform
ation in linear steps where each step
follows logically from the previous one, while
Global learners

tend to learn in
large jumps by absorbing material in a random order without necessarily seeing any
connections until they have grasped the whole concept.

To asses the student's preferences for one style or another, we have used the Index of
Learning Styles Questionnaire, developed by Barbara Soloman and Richard Felder from North
Carolina State University.

http://www.engr.ncsu.edu/learningstyles/ilsweb.html

The questionnaire scores students on a scale from 0 to 11 in one direction or another along
the four dimensions of learning style mentioned above, according to their answers to a set of
4
4 questions.



If the score on a scale is 1
-
3, the learner is fairly well balanced on the two dimensions of
that scale.

I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M




If the score on a scale is 5
-
7, the learners have a moderate preference for one dimension of
the scale and will learn more easily in a te
aching environment which favors that
dimension.



If the score on a scale is 9
-
11, the learners have a very strong preference for one
dimension of the scale. You may have real difficulty learning in an environment which
does not support that preference.
















We have used these measures to cluster the results in 5 areas:



<X>strong = 9
-
11

<X>med = 5
-
7


Aver =1
-
3 on both sides

with <X> being one of the two values along a given dimension

For example, in the chart bellow
Sstrong

means “Sequential lea
rner” in 9
-
11,
Smed

means
Sequential in 5
-
7,
Aver

means Sequential or Global in 1
-
3,
Gmed

means “Global learner” in
5
-
7, and
Gstrong

means Global in 9
-
11 (see Figure 12)



Figure 12. Different responses according to learning style


Learning Styles Results


ACT

X REF


11 9 7 5 3 1 1 3 5 7 9 11


<
--

--
>



SEN X INT


11 9 7 5 3 1 1 3 5 7 9 11



<
--

--
>



VIS X VRB


11 9 7 5 3 1 1 3 5 7 9 11


<
--

--
>



SEQ X GLO


11 9 7 5 3 1 1 3 5 7 9 11


<
--

--
>


I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


In order to evaluate th
e students’ learning styles, we have asked the students to go to the
Index of Learning Styles Questionnaire, take the test by themselves, and record their results.
Moreover, we have also advised students to re
-
take the test at the end of our pedagogical
ex
perience and try to see if there were any changes in their learning styles (any of the four
dimensions). Most of the students took the test but some (rather few did not and were content
to auto
-
evaluate their learning styles along the four dimensions after

we have briefly explained
the characterizations.

Among the students who took the learning styles questionnaire of Soloman and Felder,
about 5% reported that the results obtained did not
adequately

described how they learn, while
the others agreed with the

obtained results.

5.3 Learning styles, teaching and learning

We share the view that a professor can not tailor the instructional process to fit every
student learning style in a class. However, teaching approaches should be adapted to include
elements tha
t cover all learning styles, a particular pedagogical element facilitating learning to
those who favor that element according to their learning style and exposing the others to
differences in knowledge acquisition.

Active learners are stimulated by program
ming exercises, as they are required to be
attentive to details, get practical experience and focus on experimental thoroughness, which
are the hallmarks of such type of learners. Both programming and non
-
programming exercises
are appealing to reflective l
earners as both types of tasks require creativity, theoretical ability,
and some kind of guesswork that characterize reflective learners. To be effective when
teaching lectures, the instructor should present a blend of concrete information and abstract
con
cepts, support every abstract idea with associated examples, stimulating thus both type of
learners. Sitting through lectures without getting to do anything practical but take notes or
listen to the teachers’ explanations is particularly hard for active le
arners. Interactivity while
lecturing, such as short required exercises or student feed
-
back during the class
,

help
s

motivate active learners.

Pen
-
based interaction is appealing in this context from at least two points of view:



allows a faster response tim
e and the implication of more students in interactivity during
classes;



satisfy better the necessity of effectively doing something during a computer science
lecture where students have no possibility to effectively develop programs.

Although intuitive lea
rning is a natural human learning process, most teaching styles are
deductive (which corresponds to sensing learners), in the sense that the teacher starts from
general rules and principles and works down towards examples, by organizing and presenting
a ma
terial that is already understood. One problem with deductive presentation is that it gives
a seriously misleading impression to both sensing and intuitive learners. When students see a
perfectly ordered and concise exposition of a relatively complex deriv
ation they tend to think
that the author/instructor originally came up with the material in the same neat fashion, which
they (the students) could never have done. In order to match both types of learners, a teacher
should start with the formulation of at
least one problem to be solved, and then present the
associated methods. Such an approach play to the intuitive learners strength and
it

also help
s

I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


sensing learners develop facility with their less preferred learning mode. We did not find any
aspect along
this learning dimension that can be really enhanced by pen
-
based interaction.

Most teaching styles involve the presentation of material in a logically ordered progression,
starting from basic facts and leading gradually the learner towards the big picture.

Some
students are comfortable with this approach; they learn sequentially, mastering the material
more or less as it is presented. Others, however, cannot learn in this manner. Sequential
learners follow linear reasoning processes when solving problems; g
lobal learners make
intuitive leaps and may be unable to explain how they came up with solutions. Sequential
learners can work with material when they understand it partially or superficially, while global
learners may have great difficulty doing so.

One g
ood strategy to reach global learners is to present the lecture’s goals before presenting
the steps, doing as much as possible to establish the context and relevance of the subject
matter and to relate it to the students’ experience. Concept maps proves to

be an invaluable
tool for global learners as they offer the students the possibility to see “the big picture”, to
make connections and to highlight what is the most important focus points.

Using digital ink in free hand drawing of concept maps is essentia
l to the acceptance of
concept maps both in lecturing and in exercises by both types of learners, sequential and
globals. Being able to highlight relevant keywords or giving non
-
programming exercises
which asks to show the functioning of an algorithm help
global learners stay more motivated
in the class.

Visual learners remember best what they see: pictures, diagrams, flow charts, time lines,
films, demonstrations. Verbal learners remember much of what they hear or what they read.
They get a lot out of disc
ussion, prefer verbal explanation to visual demonstration, and learn
effectively by explaining things to others. As most of the lecture is based on speaking and
partially
o
n showing written words, the teacher has to devise ways to stimulate visual learners
.

Currently, most teachers make an adequate mixture of diagrams, tables, charts and written
words in their slides. However, there is a lack of stimulation of visual learners during
laboratories taught in computer science.

From this point of view, we have d
iscovered that non
-
programming exercises based on pen
input had a good effect on simulating visual learners. Drawing concept maps also have a good
effect on the learning performance of global learners


6. Performance evaluation: pen
-
based interactions infl
uence on students’
learning

6.1 Approach to evaluation

We have based our evaluation on two aspects of students’ activities and results. One type of
evaluation was performed based on questionnaires to which the students answered. Another
type of evaluation
was based on grades of assessed work done by the students and also on the
overall grades obtained by the students.

We have developed a set of questionnaires: one first questionnaire which was used in
preliminary evaluations (see Annex 2) and a second one w
hich was used in evaluations and
I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


based on which we have developed our conclusions and interpretations. We shall concentrate
here on this second questionnaire. For each question, students have to choose among four
possible answers: SA
-

strong agree, A
-

ag
ree, D
-

disagree, SD
-

strong disagree.

We have divided the questions into several groups, each group being dedicated to a
particular targeted aspect of the evaluation. In order to link answers with learning styles we
have asked the students to make a sel
f test of their learning styles (Section 5.2) but we also
made provisions for the case in which students did not do that.

We have divided the questions in the following sections:



Questions for using digital ink in the class (course + lab)


6 questions



Que
stions on slide annotation and pen interaction during lectures


6 questions



Questions on using concept maps


7 questions



Questions on pen
-
based exercises

8 questions



General questions on how students feel about the digital ink


2 questions



1 question ab
out the student learning style


students have to select answers according to
the four dimensions of the Felder and Silverman model. They are suggested to first take
the Felder and Silverman test.

For collecting the answers to our pen
-
enhanced learning que
stionnaire, we have used,
during our several experiments, different ways of feed
-
back, namely: questionnaire answered
on papers (as the ones in Annex 2), on
-
line questionnaire on the ITrace APCC web site, as
presented in Figure 13 (a) and the Feedback faci
lity of the Moodle environment, as presented
in Figure 13 (b,c).



(a) Web questionnaire

(b) LMS questionnaire

Figure 13. Different ways of presenting the Pen
-
based enhanced learning questionnaire

I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M



(c) LMS questionnaire


on learning styles

Figure 1
3 (cont). Different ways of presenting the Pen
-
based enhanced learning questionnaire

We had a number of 25 students, and in the second instance 20 students in the Data
Structures and Algorithms course, for which we collected and summarized a number of 45
a
nswers and analyzed the answers according to students preferences and acceptance of the
new technology. We had a number of 15 students for the Artificial Intelligence course, for
which we collected, summarized and analyzed the answers according to students

preferences
and acceptance of the new technology. Some conclusions were drawn by comparing responses
on students enrolled in different courses, while some other conclusions were drawn based on
the collected answers of the 45 plus 15 students.

6.2 Performa
nce evaluation

Performance evaluation according to obtained answers to questionnaires

We have conducted several analyses of pen
-
based enhanced learning questionnaire results
and have grouped the answers according to 4 possible choices: SA, A, D, SD. We hav
e
focused on the following aspects that were covered by the different questions in the feedback
(by grouping answers to relevant questions to each of the bellow criteria):

(1)

General opinion about using the pen during teaching and learning

(2)

Slide annotation: t
eacher vs. student


preference towards slides that are annotated
by the teacher vs. slides that are annotated by the students themselves

(3)

Interaction in classroom


how much the pen and the associated interactions
supported by the pen contributed to increa
sing level of motivation, attention and
student understanding of the presented material

(4)

Use of concept maps in class


how much this instructional tool was valued by
students when used by the teacher

I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


(5)

Use of concept maps in exercises


how much this instruc
tional tool was valued by
students when used by themselves in assignments

(6)

Programming exercises


how did the students feel about writing programs (or
making corrections) with the pen vs. in an on
-
line development environment

(7)

Pen
-
based exercises


how did
the students appreciate the pen
-
based exercises

(8)

Feed
-
back on corrections


how did the students appreciate feed
-
back written in
free form (or corrections) on their assignment files

We have summarized the results for each group of DSA students and IA studen
ts and also
summarized the results for the entire poll of students who were involved in the experiments.

Most of the students had a general good opinion about using the pen during teaching and
learning, as presented in Figure 14. Although the percentage of

Agree

choices was lower in
the AI course, the combined percentage of
Agree

and
Strongly Agree

was higher for the
students in the AI course than for those in the DSA course.

Slide annotation was also preferred by a high percentage of questioned students, w
ith a
stronger
preference towards teacher’s annotations

as compared with sudents’ annotations
; in
fact this preference was higher in case of reflective learners and a bit lower in case of active
learners. However, active learners had still a higher prefere
nce towards the teacher annotating
slides than their own annotation.

Interaction in the classroom was definitely the highest rank category from the point of view
of stimulating motivation, attention and retention. Percentages for both disciplines were
comp
arable.

The use of concept maps in the classroom was also favorably appreciated although not as
high as the previous criteria, with a definite dominance of appreciation in the case of AI
students as compared to the DSA students. The use of concept maps in
exercises was less
favored by students enrolled in both disciplines. Visual learners, global learners, and intuitive
learners appreciated the use of concept maps when presenting the learning material and agreed
that concept map help them better understand
and retain the presented topics. Sensing learners
did not really valued concept maps and found this activity somehow boring. Active learners
preferred, in general, drawing a concept map with a pen while reflective learners preferred the
use of mouse and pr
edefined forms.

Programs written with pen was strongly rejected, which is understandable, but also
corrections to pieces of code was a less favored option, in particular among students in DSA
course and regardless of their learning style.

Pen based exercis
es were generally scored high (about 70% Agree plus Strongly Agree),
with a higher percentage for AI students, may be because the emphasis of the discipline is
more on mechanisms and techniques of problem solving, which are well supported by non
-
programmin
g exercises, as compared with the DSA discipline. Most of the students who were
in favor of pen based exercises were more oriented towards intuitive and visual learning
styles. Reflective learners were also more in favor than active learners, a result whic
h was
somehow surprising.

I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


We have somehow expected a positive result to pen
-
based annotation of assignments;
however we were surprised by the high percentage of acceptance of this correction method
among students in both disciplines. Still, we consider tha
t the strong appreciation obtained
was also because of the novelty of the approach and it is to be seen if, when currently exposed
to such an approach, the interest of the students will be still significant.

We are aware that the obtained results and evalu
ations represent the opinion of small poll
of students; results in the long run and taken from larger sets of students may vary. However,
we consider that the overall response of the students to the introduction of the new technology
and, especially, to th
e new interactions modalities that were triggered by the technology was
definitely positive.


Figure 14. General opinion about using the pen during teaching and learning

Performance evaluation according to obtained grades

Another way to evaluate learning

performances resulted from our experiment was to
compare grades. We have performed two comparisons. The first one was by comparing grades
of individual assignments of students during the first week with grades of individual
assignments in the last week. R
esults are presented in Figure 15 (a) for the data Structures and
Algorithms and in Figure 15 (b) for Artificial Intelligence. From the figure we can see that
bellow threshold grades (failures = between 0 and 50 points) has been significantly reduced,
with

0 failures in the DSA case, and the top grades (between 90
-
100 and 70
-
90) increased. In
the DSA case middle grades (50
-
7) decreased because of the increase in top grades.



Figure 15. Grade comparison


I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


The second comparison was between
the
number of st
udents which were placed on
different scales of top scores in previous classes with those that were involved in our
experiments. The results of this comparison were the following: top 10% increased slightly,
between top 10% and above passing threshold we n
otices a significant increase together with a
significant

decrease

of bellow threshold students. Although the results are highly positive, we
do not think that the definite improvement in learning performance is due
only

to the use of
the new technology, a
lthough that had a definite contribution. We think that the improved
results are also justified by the small number of students on which the teaching team was
m
ore
focused and spent more time in teaching and coaching activities.


7. Lessons learned: good p
ractices and impact of the new technology

We have found out that digital ink technology allows teachers to better exploit interactions
during the lectures and to broaden the r
ange and diversity of students’

assignments during
laboratory work and assessment
.

Learning by doing and active learning can be stimulated during lectures by interactive
refinement of presented slides but also by engaging students in short feedback activities, such
as short questions to which they can rapidly draw the answer or the dra
wing of a concept map.

We have already used for a number of years non
-
programming exercises in our classes,
such as the ones presented in Section 4, but obviously the solution was to use paper, as the
time necessary to develop such exercises using drawing
tools was rather too high. Pen
-
based
techniques allowed us to widen the types of these exercises, to integrate them in an LMS and
to significantly decrease to solution time.

The role of these non
-
programming exercises and of building concept maps in
consol
idating learning is, according to our opinion, significant, by enhancing skills such as:
information visualization, data structure recognition, discrimination abilities, understanding
functioning of algorithms, understanding the conceptual relationships am
ong concepts
presented in the class, understanding of problem representation and solution representation.

We also found out that digital corrections of assignments have a significant contribution to
raising students awareness of their misconceptions and mi
stakes and also stimulate students to
review their corrected assignments more often than in paper correction (in the traditional
way). Moreover, such a correcting style allows both the students and the teacher to have
access to the corrected material.

One
of the main advantages of using digital ink, recognized by both the teachers
participating to the experiment and by the students, is definitely the speeding up of a lot of
activities. e.g., lecture preparation, non
-
programming assignments solutions, feedba
ck.

Students and teacher can explore their learning styles and different intelligences. By using
on
-
line sources, each student can determine their individual learning styles and dominant
intelligences. In addition, they can gain an insight into how they an
d others learn. This is very
important in understanding why different instructional methods should be employed so that all
students can have the greatest opportunity for success. The teacher can then construct a
pedagogical approach that can exploit differ
ent learning styles but also an activity in which to
have students discuss how their learning styles can affect their learning performances.

I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


The timeless question posed by students, “Why do I need to know this stuff?,” needs to be
addressed in order to bui
ld a positive classroom environment. Students need to be convinced
or “sold” that the class will be meaningful to them. For example, in an assessment, the teacher
can stress that there is not an absolute answer and the true goal is the development of inqui
ry
skills that can be used throughout life. Having a wide range of exercises, that exploit several
student abilities
,

can contribute to “selling” the taught material.

Common to all types of learners is the ability to effectively communicate. Teachers shoul
d
be aware of the attributes of effective communication and constantly find ways to include
them in their activities. Aspects of effective communication must include different forms of
communication, including verbal, textual, and visual. Ideally, the teac
her should allow
students to select the ways to communicate that suit them best, in order for the students to be
able to effectively share their acquired knowledge.

When integrating digital ink technology in courses or laboratory work, computing
educators
must re
-
think what they teach students and how they enable students to learn, in
such a way as to take best advantage of the new communication resources.

We found out that adopting maps for assessment use needs a common understanding of
what a concept map

assessment is and whether it provides a reliable and valid measure of a
student’s cognitive structure. When used as an assessment tool, concept maps are very
appealing but without a precise measure of how “correct” a concept map is they can not be
accurat
ely used for grading the students’ knowledge.

When evaluating concepts maps developed by the students we have used both a qualitative
measure (how correct the concept map is to the teacher) but also some quantitative measures
that we intend to further deve
lop in what can be called a “precise” measure of how “correct” a
concept map is. The quantitative measures used were the following: the percentage of “right”
node concepts used in a CM drawn by the student, the percentage of “wrong” node concepts
used in a

CM drawn by the student, the percentage of “meaningful” labeled links among
concepts.


8. Conclusions and perspectives

8.1 Conclusions

The participation in the ITrace project was a very rewarding one. The project gave us the
opportunity both to perform ou
r own evaluations on a new technology and novel approaches,
but also to interact with the partners in the project, share ideas, experiences and results.

The aim of APCC
-
UPB partner was to design a set of pedagogical experiences in order to
evaluate the imp
act of using the pen technology on the effectiveness of the teaching process.
Moreover, we aimed at evaluating the impact of this technology on different student learning
styles and the role concept maps can play in facilitating student learning depending
on his/her
learning style. We think that our proposed pedagogical experiences can add to the range of
available teaching methods that use this new technology.

During our experiments, we found out that the overall response to our pedagogical
experiences was

positive and that the students enjoyed both being part of the classes that used
digital ink and practicing with the new approaches of input data into the computer. However,
I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


we share the view that there are some concerns related to the deployment of the te
chnology, in
particular connected to the availability of associated devices on a large scale (we have more
than 1000 students enrolled in Computer Science courses at the Department of Computer
Science) and potential downsides at pedagogical level. There ar
e also concerns of usability
and concerns from the point of view of the extra work that the instructors have to put in order
to rethink their classes for optimal use of pen technology.

One of the connected results of our work was the insight gained in valu
ating the students’
learning styles and pedagogical practices that are able to accommodate different learning
styles. Last but not least, we have found out one extra valuable teaching tool, namely the
concepts maps that facilitate meaningful learning and h
elp students in knowledge acquisition
and recollection.

8.2 Perspectives

Our participation in the ITrace project open up a lot of perspectives and we do not intend to
end our pedagogical experiences of using digital ink once the project is over. We have
id
entified several directions in which we shall continue our activity. One of the most
important aspects to sustain pen
-
based experience in learning is the available technology. As
prices are constantly getting down, we assume that Tablet PCs will become qui
te affordable to
get. We are planning to get more funding from local or national funding bodies in order to
equip an entire laboratory with 24 tablet PCs, a laboratory that can be used both during
laboratory work, and during assignment problem solving by t
he students.

We are also planning to equip a lecture room with an interactive digital board, which will
also facilitate digital ink experiences. However, we consider that a teacher equipped with a
Tablet PC and a video projector is in general quite adequat
e to deliver interactive experiences.

We are also planning to transform the Data Structures and Algorithms course and the
Artificial Intelligence course in permanent pen
-
based enhanced learning experiences in the
future. Moreover, we plan to involve other
members of our department in our experiments and
stimulate the teachers to develop their own course and assignments that includes pen
-
based
facilities. Last but not least, we plan to improve our own developed software in order to allow
more interactivity a
nd groupware interaction.

We have also a list of open questions that needs to be further explored and investigated.
The basic question, to which this study tried to contribute, is “How pen
-
based interaction can
transform the way we, teachers, teach, and th
e way students learn?” If in the future (which is
in fact the near future) most students will have PCs, with a significant fraction of these being
Tablet PCs; therefore we need to take advantage of this technology to create new learning
environments, and t
o exploit the fact that (mobile) Tablet PCs allows a new range of
communication modes besides the text based one, both in the classroom and outside of it.


References

Anderson, A. et. al. (2004).

Experiences with a Tablet PC Based Lecture Presentation Syst
em in Computer Science
Courses. SIGCSE 2004.

Retrieved December 2007 from
http://www.cs.washington.edu/research/edtech/publications/

Bareholz, H. and P. Tamir. (1992). A comprehens
ive use of concept mapping in design instruction and assessment,
Research in Science and Technology Education
, 10(1), pp. 37
-
52.

I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


Dochy, F. J. (1996). Assessment of domain
-
specific and domaintranscending prior knowledge: Entry assessment
and the use of prof
ile analysis. In M. Birenbaum & F. J. R. C. Dochy (Eds.),
Alternatives in assessment of
achievements, learning process and prior knowledge

(pp. 93
-
129). Boston, MA: Kluwer Academic Publishers.

Felder, R.M. and L.K. Silverman. (1988). Learning and Teaching
Styles in Engineering Education,
Engr.
Education, 78
(7), 674
-
681.

Felder, R.M. and R. Brent. (2005). Understanding Student Differences,
Journal of Engineering Education
, 94 (1),
2005, pp.57
-
72.

Felder, R.M., G.N. Felder, and E.J. Dietz. (2002) The Effects
of Personality Type on Engineering Student
Performance and Attitudes.
J. Engr. Education
, 91(1), 3
-
17.

Flemming, N. D. and C. Mills. (1992). Not another inventory, rather a catalyst for reflection. To Improve the
Academy, 11:137. Retrieved December 2007 f
rom
http://www.ntlf.com/html/lib/suppmat/74fleming.htm

Flemming., N. D. (2001).
Teaching and Learning Styles: VARK Strategies
, Christchurch, New Zealand, 5th edition,
2001.

Florea, A.M. a
nd S. Radu. (2007). Enhancing Pen
-
based Experiences with the Use of Concept Maps,
First
International Workshop on Pen
-
Based Learning Technologies

(PLT 2007), IEEE Computer Society
Conference Publishing Service, in print.

Honey, P. and A. Mumford. (1986).
A

Manual of Learning Styles
, Peter Honey, Maidenhead

ITrace APCC Web site. (2007). Retrieved December 2007 from
http://turing.cs.pub.ro/ITrace/index.html

ITrace: Concept Maps. (2007). Retrieved Dece
mber 2007 from
http://turing.cs.pub.ro/concept_maps/

ITrace: Pen Review. (2007). Retrieved December 2007 from
http://turing.cs.pub.ro/pen_te
ch/review.html

ITrace: Pen Software. (2007). Retrieved December 2007 from
http://turing.cs.pub.ro/pen_tech/software.html

ITrace: Pen Techniques. (2007). Retrieved December 2007 from
http://turing.cs.pub.ro/pen_tech

ITrace: Web: Pen papers. (2007). Retrieved December 2007 from
http://turing.cs.pub.ro/pen_tech/papers.html

Jonassen, D.
H., K. Beissner, and M. Yacci. (1993).
Structural knowledge: Techniques for representing, conveying,
and acquiring structural knowledge.

Hillsdale, NJ: Lawrence Erlbaum Associates.

Keefe, J.W. (1979). Learning Style: An Overview, in Keefe, J.W., ed.,
Stude
nt Learning Styles: Diagnosing and
Prescribing Programs
, Reston, Va.: National Association of Secondary School Principals, 1979.

Kolb, D.A. (1984) david a. kolb on experiential learning, Retrieved December 2007 from
http://www.infed.org/biblio/b
-
explrn.htm

Moodle. (2007). Retrieved December 2007 from
http://moodle.org/

Novak, J.D. (1990). Concept mapping: A usefull tool for science education,
Journal of Research i
n Science
Teaching
, 27(10), pp. 937
-
949.

Novak, J.D. (1991). Clarify with concept maps: Atool for students and teachers alike.
The Science Teacher
, 58 (7),
45
-
49.

Price, L. (2004). Individual differences in learning: Cognitive control, cognitive style, and

learning style.
Educational Psychology
, 24 (5), 681
-
698.

Ruiz
-
Primo, M. A., S.E. Schultz, and R. J. Shavelson. (1997). Concept Map
-
Based Assessment in Science: Two
Exploratory Studies,
CSE Technical Report 436
, CRESST/Stanford University.

Simon, B. et. al
. (2004). Preliminary Experiences with a Tablet PC Based System to Support Active Learning in
Computer Science Courses, ITICSE’04, June 28

30.

Soloman, B. and R. Felder. (2007). Index of Learning Styles Questionnaire, Retrieved December 2007 from
http://www.engr.ncsu.edu/learningstyles/ilsweb.html

Web annotation. (2007). Retrieved December 2007 from
http://annotation.seman
ticweb.org/annotationtool_view

I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


Web on Kolb (2007). The experimental learning cycle, Retrieved December 2007 from
http://www.learningandteaching.info/learning/experience.html

Web

on VARK. (2006). VARK: A guide to learning styles, Retrieved December 2007 from
http://www.vark
-
learn.com/english/index.asp

White, R. T. and R. Gunstone. (1992).
Probing understanding
. New York
: Falmer Press.




I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


Annex 1: Charts









I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


Annex 2: Questionnaires

A2.1 Feed
-
back questionnaire No.1


quality of pen interaction (
revised form
)


“Pen
-
based enhanced learning” Poll

The following set of questions are aimed to evaluate the impact of
using the digital ink on
enhancing the pedagogical experience, of using the pen on drawing concept maps, and of
using concept maps in teaching students characterized by different learning styles.

There are 30 questions. In questions number 1 to 29 you have

to choose one answer among
four possible ones: SA
-

strong agree, A
-

agree, D
-

disagree, SD
-

strong disagree.

In question 30 you are asked to select the learning style that describes you best. In order to be
answer this question you may want to first t
ake the Learning Style Test at
http://www.engr.ncsu.edu/learningstyles/ilsweb.html


Alternately, or you may read the materials on learning style and figure out by yourself which
learning
style describes you best. Please be aware that question 30 will require you to input a
score. The highest the score, the more dominant the feature is.

In the end you are free to input any of your comments regarding your pen
-
based learning
experience.

Good
luck!


Some information about you (your name is not asked!)

Age:




-





-





-


卥S:



F



M

B慣a杲潵湤:



St畤敮t⁩渠nom灵t敲 卣S敮ee




St畤敮t⁩渠慮⁅湧楮敥物湧⁄i獣i灬i湥



Gr慤a慴攠e渠nom灵te爠卣r敮捥



Gr慤a慴攠e渠慮⁅湧楮n敲i湧⁄is
ci灬i湥

H慶攠祯甠慮礠y牥vi潵猠數灥物敮捥 i渠nsi湧⁤i杩t慬 i湫



Y敳






䵯M桥爠r潮杵攺



Rom慮i慮



E湧li獨



䙲敮捨

却畤礠S慮杵慧g:



Rom慮i慮



E湧li獨



䙲敮捨


Now the questions

Questions for using digital ink in the class (course + lab)


1.
Using digital ink in the class is enjoyable








A




D







I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M



2. Using digital ink in the class is stressful








A




D







㌮⁉i步⁵獩湧nt桥⁰敮⁤eri湧⁴n攠el慳a








A




D







㐮⁉⁡mo牥⁡tt敮ei癥⁤v物湧nt桥⁣l慳猠w桥渠
䤠I獥 t桥⁰敮








A




D







㔮⁕獩湧nt桥⁰敮⁤楳t牡捴s me








A




D







㘮⁉⁡mo牥 m潴i癡t敤eto敡e渠ife慲湩n朠g湶潬v敳 灥渠楮p敲a捴i潮








A




D







Q略uti潮猠潮o獬i摥⁡湮otati潮⁡od⁰敮⁩湴n牡rti潮⁤畲u湧nl散t畲us


㜮⁈慶i湧nt桥⁰h獳ibilit礠y漠慮o潴at攠t桥⁳lid敳⁨ l灳攠e整ai渠n湯wl敤来








A




D







㠮⁉⁤漠湯ti步kt漠慮oot慴攠e桥h獬i摥d








A




D







㤮⁉i步kw桥渠t桥ht敡捨敲 m慫敳⁡湮潴慴i潮o⁳lide猠摵物湧⁴桥h捬a獳








A




D







㄰⸠䤠1敡e渠n整t敲 if 䤠I慮ar敶eew t敡eh敲'猠慮湯t慴敤⁳ id敳








A




D







ㄱ⸠䤠1敡e渠n整t敲 if 䤠I慮ar敶eew m礠yw渠慮湯tat敤e獬id敳








A




D







ㄲ⸠卨潲o⁰敮
-
b慳a搠數敲ci獥s⁡獫敤⁤畲un朠g桥hle捴畲攠桥l灳pm攠扥tt敲⁵
摥r獴慮a t桥h
灲敳敮t敤et潰o捳








A




D







Q略uti潮猠潮⁵獩湧n捯湣c灴慰s


ㄳ⸠䤠灲efe爠桡ri湧⁴n攠e散t畲攠u潡o猠灲s獥湴敤⁡猠愠捯湣cpt慰








A




D







ㄴ⸠䤠灲efe爠桡ri湧⁴n攠e散t畲攠u潡o猠w物tt敮⁩nt敲a捴i癥l礠慴 t桥⁢敧e湮i湧

of⁴桥⁣ha獳








A




D







ㄵ⸠1桥⁣潮捥灴pm慰⁨敬p猠s礠畮d敲st慮ain朠gf t桥⁴潰oc








A




D






I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


16. I prefer to build the concept map by selecting concepts from a given list








A




D







ㄷ⸠䤠灲efe爠扵rl摩d朠愠捯g捥灴

m慰af牯m⁳ 牡r捨








A




D







ㄸ⸠䤠灲efe爠摲rwi湧⁴n攠e潮捥灴 m慰awit栠愠h敮⁲et桥爠t桡h wit栠愠m潵oe








A




D







ㄹ⸠1uildi湧n捯湣cpt m慰猠s猠s潲ong








A




D







Q略uti潮猠潮⁰敮
-
扡獥搠dx敲捩獥s


㈰⸠䤠灲efe爠rx敲
ci獥猠whic栠楮癯h癥v⁰敮
-
扡獥搠i湰ut








A




D







㈱⸠䤠灲efe爠灲潧牡rmi湧⁥ 敲捩s敳⁵ i湧敹⁩湰ut








A




D







㈲⸠䤠2潮獩d敲⁴h慴⁰ n
-
扡獥搠數敲ci獥猠桥l瀠p攠扥et敲⁵ 摥d獴慮a t桥ht潰ic








A




D







㈳⸠䤠灲efe爠r

捯mbi湡ni潮f⁰敮
-
扡獥b⁥ 敲捩獥猠慮搠d牯杲ammi湧n數敲捩獥s








A




D







㈴⸠䤠灲efe爠w物tin朠愠灳敵摯捯摥dwit栠愠h敮⁲et桥爠t桡h⁵ i湧敹ei湰畴








A




D







㈵⸠䤠灲efe爠r慫楮朠捯cre捴i潮猠s漠愠灩散攠of⁣潤攠wit栠愠h敮⁲et桥爠
t桡h⁵獩n朠步礠g湰ut








A




D







㈶⸠2igital⁣ 牲散ti潮f m礠數敲ci獥猠s猠sl敡e敲 t桡渠瑥xt⁢慳敤爠潮⁰慰敲








A




D







㈷⸠䤠2敶eew mo牥 f牥煵敮tl礠m礠y潲o散t敤⁥e敲ci獥s⁩f t桥礠h牥 i渠nigit慬 f潲m
灥p
-
扡獥搠捯牲e捴i潮⁡od

灯inti湧f⁥牲潲猩








A




D







G敮敲慬ⁱ 敳ti潮猠潮⁨sw⁹潵 fe敬⁡ 潵琠t桥⁤i杩t慬 i湫


㈸⸠䤠2i步⁵獩湧nt桥⁰敮








A




D







㈹⸠2f 䤠扵礠m礠yw渠l慰瑯瀠䤠灲pf敲 it t漠扥⁡⁴慢let PC爠t漠桡o攠愠f潲mf⁧牡灨楣慬
i湴n牡rti潮








A




D






I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


Question about your learning style


30. My learning style is (select one answer in each group):


Active vs. Reflective
: Active learners understand new information by doing something
with it while Reflective learners prefer to think
about new information first before acting
on it.


Active learner




獣o牥‹
-
ㄱ†


獣o牥‵
-
㜠†


獣潲o‱
-
3

Refle捴i癥vl敡牮敲



獣o牥‹
-
ㄱ†


獣o牥‵
-
㜠†


獣潲o‱
-
3

䤠摯潴Iow






Deductive vs. Intuitive
: Deductive learners like learning facts an
d solving problems by
well established methods while Intuitive learners prefer discovering new relationships
and can be innovative in their approach to problem solving.


Deductive learner



獣o牥‹
-
ㄱ†


獣o牥‵
-
㜠†


獣潲o‱
-
3

䥮I畩ti癥敡en敲



獣or
攠e
-
ㄱ†


獣o牥‵
-
㜠†


獣潲o‱
-
3

䤠摯潴Iow






Visual vs. Verbal
: Visual learners understand new information best by seeing it in the
form of pictures, demonstrations, diagrams, etc, while Verbal learners understand new
information best through wri
tten and spoken words.



Visual learner




獣o牥‹
-
ㄱ†


獣o牥‵
-
㜠†


獣潲o‱
-
3

V敲扡氠l敡牮敲




獣o牥‹
-
ㄱ†


獣o牥‵
-
㜠†


獣潲o‱
-
3

䤠摯潴Iow





Sequential vs. Global
: Sequential learners understand new information in linear steps
where eac
h step follows logically from the previous one, while Global learners tend to
learn in large jumps by absorbing material in a random order without necessarily seeing
any connections until they have grasped the whole concept.


Sequential learner



獣o牥‹
-
ㄱ†


獣o牥‵
-
㜠†


獣潲o‱
-
3

Gl潢慬ol敡e湥n




獣o牥‹
-
ㄱ†


獣o牥‵
-
㜠†


獣潲o‱
-
3

䤠摯潴Iow





Note: On learning style is not better than another. It just explain the way you learn best


Input your free comments

………………………………………………………………………………
………………


………………………………………………………………………………………………


I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


………………………………………………………………………………………………


………………………………………………………………………………………………


………………………………………………………………………………………………



Thank you for taking this test


A2.2 Feed
-
back questionnaire No.1


quality of pen interaction
(
initial form
)


Pen
-
based enhanced learning Poll

The following set of questions are aimed to evaluate the impact of using the digital ink on
enhancing the pedagogical experience, of using the pen on drawing concept maps, and of
using concept maps in teachi
ng students characterized by different learning styles.

There are 25 questions. In questions number 1 to 23 you have to choose one answer among
four possible ones: SA
-

strong agree, A
-

agree, D
-

disagree, SD
-

strong disagree. In
question 24 you are ask
ed to select the learning style that describes you best. In order to be
able to answer this question you may want to first take the Learning Style Test at
http://www.engr.ncsu.edu/learning
styles/ilsweb.html

or you may read the available materials
on the site about learning style and figure out by yourself which learning style describes you
best.

In the last question you are able to input any free comments regarding your pen
-
based learning
experience.

Good luck!


Some information about you (your name is not asked!)

Age:




-





-





-


卥S:



F



M

B慣a杲潵湤:



St畤敮t⁩渠nom灵t敲 卣S敮ee




St畤敮t⁩渠慮⁅湧楮敥物湧⁄i獣i灬i湥



Gr慤a慴攠e渠nom灵te爠卣r敮捥



Gr慤a慴攠e渠n
n⁅湧楮n敲i湧⁄i獣i灬i湥

H慶攠祯甠慮礠y牥vi潵猠數灥物敮捥 i渠nsi湧⁤i杩t慬 i湫



Y敳






䵯M桥爠r潮杵攺



Rom慮i慮



E湧li獨



䙲敮捨

却畤礠S慮杵慧g:



Rom慮i慮



E湧li獨



䙲敮捨

I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


Now the questions

1. Using digital ink when presenting the class i
s enjoyable








A




D







㈮⁕獩湧⁤i杩tal⁩湫 w桥渠灲敳敮ti湧⁴桥h捬a獳 i猠stre獳ful








A




D







㌮⁉⁰牥f敲⁨慶楮朠g桥e捴畲攠杯ul猠灲敳敮t敤⁡猠愠s潮捥灴 m慰








A




D







㐮⁉⁰牥f敲⁨慶楮朠g桥e捴畲攠杯ul猠writt
敮ei湴n牡rti癥v礠yt⁴h攠扥ei湮in朠gf th攠ela獳








A




D







㔮⁔桥⁣潮h数e m慰⁨el灳pm礠畮摥牳t慮ai湧f t桥⁴潰楣








A




D







㘮⁉⁰牥f敲⁴漠扵il搠t桥h捯湣数n m慰⁢礠獥l散ti湧n捯湣数ns f牯m⁡ 杩v敮ist








A




D







㜮⁉⁰牥f敲⁢ il摩湧d愠捯湣数e m慰af牯r t桥⁰牥s敮e敤at敲ial








A




D







㠮⁉⁰牥f敲⁳ 慲ai湧nf牯m⁡⁧ 癥渠v慳a挠c潮捥灴pm慰a慮a⁡摤⁣潮 数es⁡湤in歳








A




D







㤮⁉⁰牥f敲⁤牡win朠g桥h捯湣数n m慰awit栠愠h敮⁲et桥爠h桡h wi
t栠愠a潵獥








A




D







㄰⸠1uildi湧n捯湣cpt m慰猠s猠s潲ong








A




D







ㄱ⸠䤠1i步⁵獩湧nt桥⁰敮⁤e物湧nt桥⁣l慳s








A




D







ㄲ⸠䤠1m潲o⁡tt敮ei癥⁤畲u湧nt桥⁣l慳猠w桥渠䤠h獥⁴桥⁰敮








A




D







ㄳ1

U獩湧nt桥⁰敮⁤e獴r慣a猠se








A




D







ㄴ⸠1a癩湧nt桥⁰h獳i扩lit礠t漠慮oot慴e⁴桥h獬i摥d⁨敬p猠s攠牥t慩渠nnowl敤ee








A




D







ㄵ⸠䤠摯1ti步kt漠慮ootat攠eh攠eli摥s








A




D







I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


16. I like when the teacher makes

annotation on slides during the class








A




D







ㄷ⸠䤠1敡e渠n整t敲 if 䤠I慮ar敶eew t敡eh敲'猠慮湯t慴敤⁳ id敳








A




D







ㄸ⸠䤠1敡e渠n整t敲 if 䤠I慮ar敶eew m礠yw渠慮湯tat敤e獬id敳








A




D







ㄹ⸠䤠灲efe爠rx敲c
i獥猠whic栠楮癯h癥v⁰敮
-
扡獥搠i湰ut








A




D







㈰⸠䤠灲efe爠灲潧牡rmi湧⁥ 敲捩s敳⁵ i湧敹⁩湰ut








A




D







㈱⸠䤠灲efe爠w物tin朠愠灳敵摯捯摥dwit栠愠h敮⁲et桥爠t桡h⁵ i湧敹ei湰畴








A




D







㈲⸠䤠灲efe爠r慫楮
朠捯cre捴i潮猠s漠愠灩散攠of⁣潤攠wit栠愠h敮⁲et桥爠t桡h⁵獩n朠步礠g湰ut








A




D







㈳⸠2f 䤠扵礠m礠yw渠l慰瑯瀠䤠灲pf敲 it t漠扥⁡⁴慢let PC








A




D







㈴⸠䤠2m潲o⁡ 
s敬e捴 o湥⁡湳w敲 i渠n慣栠牯w):




A捴i癥vl敡en敲





fl散tiv攠e敡e湥r




䤠摯It ow


(A捴i癥vl敡en敲猠畮摥牳ta湤 w info牭慴i潮⁢礠摯i湧⁳nmet桩湧 wit栠it whil攠剥fl散tiv攠
l敡en敲猠s牥f敲 t漠ohi湫⁡扯bt w⁩nf潲mati潮 fi牳t⁢ f潲o⁡捴i湧n it.)




D敤畣ei癥vl敡牮敲




䥮Iuiti癥vl敡牮敲




䤠摯It

歮ow


(D敤畣ei癥vl敡牮敲猠si步kle慲湩湧nf慣t猠慮搠獯s癩湧⁰牯扬em猠s礠w敬l⁥獴慢li獨s搠浥t桯h猠
w桩l攠䥮Iuiti癥vl敡e湥牳⁰牥fe爠摩獣潶敲in朠湥w⁲敬ati潮獨s灳p慮a⁣慮⁢攠e湮o癡viv攠en
t桥h爠慰灲r慣栠a漠灲潢lem⁳ l癩n朮g




Vi獵sl敡en敲




V敲扡le慲湥r




䤠摯It ow


(Vi獵sl敡en敲猠畮s敲sta湤 w info牭慴i潮⁢e獴 批⁳敥i湧nit i渠瑨攠fo牭f 灩ptu牥猬s
摥d潮ot牡ti潮猬⁤i慧牡msⰠ整挬cwhil攠噥牢el敡en敲猠畮摥牳t慮a w inf潲o慴i潮⁢敳t
t桲潵杨 writt敮⁡湤⁳灯 敮ewo牤献)




S敱略etial敡en敲





潢ole慲湥r




䤠摯It ow


(卥煵敮tial敡en敲猠畮se牳t慮aew⁩nf潲m慴i潮oi渠汩湥慲⁳t数e wh敲攠敡e栠獴数ef潬low猠
l潧o捡cl礠f牯m⁴h攠灲e癩潵猠潮攬sw桩l攠el潢慬ole慲湥牳⁴e湤⁴漠l敡牮⁩渠l慲来畭灳⁢礠
I
-
TRACE: Final Report
AP
CC


University Politehnica

________________________________________________________________________________

Grant N. 223434
-
CP
-
1
-
2005
-
IT
-
MINERVA
-
M


absorbing material in a random order without nece
ssarily seeing any connections until
they have grasped the whole concept.)


25. Input your free comments

………………………………………………………………………………………………

………………………………………………………………………………………………

………………………………………………………………………………………………

………………………………………………………………………………………………

………
………………………………………………………………………………………

………………………………………………………………………………………………

………………………………………………………………………………………………


Thank you for taking this test