A Tool Supporting Concept Map
Evaluation and Scoring
November 29, 2010
Master’s Thesis in Computing Science,30 ects credits
Supervisor at Cs
Umu : Jürgen Börstler
Examiner: Per Lindström
Department of Computing Science
901 87 ,
Concept maps are a form of graphical knowledge representation . Concept maps are
drawn as directed or undirected graphs. Concept maps may
carry a large and complex
structure where concepts are described as a form of relationships between the nodes and the
edges. Concept maps are widely used in teaching and learning where educational research
and assessment are necessary functions. The common
problem when dealing with concept
maps is evaluation. This usually involves comparing concept maps for similarity. When
developing a concept map the developer uses different ways and terminologies to describe
the same concepts and relationships. Furthermo
re there is no commonly accepted way of
scoring a concept map since not all scoring rules are suitable for all types of concept maps.
This thesis proposes a tool that reviews scoring techniques used for assessing the concept
maps. A tool was developed to
unify the terminology used in describing the concept and
relationship in a concept map. The tool maps these terminologies into a common language
and further scores it against a map master map. The tool also assesses concept map
on different approach
es of relational and structural scoring techniques as structure based
scoring has shown better comprehension than scores based on concept
alone . The tool
produces the final score by summing up the scores of all these approaches. The tool is also
supported and is automated by the input prepositions given from the user.
The results suggest that the evaluation techniques used by the tool helps in assessing the
concept map but the tool does not support all forms of concept map
. The tool shows
can accommodate various aspects of scoring and has shown capacity to integrate with other
tools like the word net and implement algorithms such as path finder. However the tool
needs to be further extended to support assessment of hand written con
cept maps and other
2 Literature R
eview and Background
Uses and Application..
Treatment by preposition……………
Master map vs student map…………………..8
of structure in Concept Map
Role of shapes in a concept map………………………………….12
Concept map scoring (in detail)…………………………………..14
the tool (actual methods)
Basic Design and Implementation 21
Design Overview and Architecture………………………………26
Application of scoring Algorithm………………………………..28
Input constraints & Front end of the tool…………………………38
Holistic Overall Impression………………………………………42
Manually Resolve Window………………………………………49
Path Finder Algorithm and Shapes of Concept maps…………….50
Final Summary……………………………………...…………… 53
Evaluation of t
6 Acknowledgment 58
List of Tables
2.1 Relational scoring …………………………………………………..
2.2 Structural scoring ………………………………………………….
4.10 Metric Results………………………………………………………….
Concept maps are graphical tools which are used for
organizing and representing
knowledge , mainly used in the areas of teaching. A concept within a concept map
is usually represented in boxes or labels and is connected with labeled arrows. The
label for the concept map can be a word or a + or % and some
times more than the
word itself. The relationship between concepts is represented as linking phrases
between two concepts. In the figure 1.1 we note that “Represent”, “begins with “ and
“is” are examples of relationships used to link concepts. The techniqu
e used in
comprehending these relationships among different concept is called as “Concept
Mapping”. A preposition contains two or more concepts connected using linking
words or phrases to form a meaningful statement . In the figure below” concept
> organized knowledge” is a meaningful statement which is a preposition
from the concept map
A classic concept map explaining notation and elements of a concept
map, see .
characteristic of a concept map is concepts within a concept map are
structured hierarchically with the most inclusive and general ones at the top of the
structure and less general concepts hierarchically below .In concept mapping the
concepts, hierarchically organizes these concepts, differentiates
between them and expresses more complex cross hierarchical relations. Cross links
are relationships or links between concepts in different parts of the same map.
However these characteristi
cs may not be true in reality for all concept maps in
general. These characteristics are true when we follow Nowak and his tools. In reality
not always a concept map may follow a complex structure and may not always be
hierarchical or possess cross links.
Concept maps are getting increasingly used in the areas of teaching and learning.
They are very similar to mind maps. Concept maps are also used in areas of
educational research, for capturing ideas and note making. Specifically in areas of
education a co
ncept map is usually used to explain a concept or a phenomenon for
students to easily understand it [1, 3].Educational fields use concept maps for
assessing the ability of student in understanding a topic .There are many resources to
develop a concept map.
The main ones are free mind, XMind, Compendium, IHMC
and VUE [5, 6]. These tools are basically used to represent knowledge in the form of
concept maps or mind maps. They have different features that help in drawing maps
such as graph based links , exporti
ng images to the maps ,icons on nodes , web and
file hyperlinks to nodes , export maps into various file formats such as XML, JPEG.
These concept mapping tools do not incorporate scoring of concept maps. They are
usually restricted to drawing concept maps.
When we consider a collection of concept maps the key operations is to retrieve the
maps, compare it for similarity and closeness with a master map . For example
instructors review student drawn maps by comparing them with expert knowledge
basically means comparing each concept map with other for similarity
(when same concepts are involved). This becomes a nontrivial issue since different
people use different ways to represent different structures when describing the same
concepts and their
relationships . There is no rule in the use of terminologies used
in the map and there is a clear distinction between one map builder with another in
expressing knowledge while developing a concept map [3, 27]. This allows the map
builder to build maps
which are quite distinct in nature. So a common method to
evaluate these maps is important since it helps in realizing one’s ability to understand
.In this thesis we develop a tool that evaluates concept maps by matching knowledge
ents between concept maps, unifies terminology used in the map into a common
language and then score the concept map based on certain scoring strategies .
This is interesting because different people exhibit different mental models in order to
ruct a map .
The tool is written completely in java. The main advantage of using java is ease of
integration with other systems, prepares the tool to be extended in the future changes
also is platform independent.
1.2 Problem Description
problems that will be addressed in this thesis are divided as below:
: While matching elements between two concept maps there are
obvious comparisons between concepts and relationship nodes directly. The
relationship names are usually
problematic as it is often repeated more than one time
in a concept map. A direct string matching algorithm between the 2 maps is not
enough as we are not going to compare the elements string wise but we are going to
consider the structure of the concept m
ap as a whole. This means that direct string
matching between concepts and relationship of two different maps may not yield
Variations in Terminology used by the map builder:
Map builders often use
different terminology to describe the s
ame concepts or relationships. The terminology
variations could pose as a serious problem since the map builder could express the
same concept using a different term but finally mean the same as mentioned in the
master map. The tool must look to map these
similarities in terminology when
matching the knowledge elements otherwise the comparison between the student map
and the master map may not lead to accurate results.
Structural Differences in a concept map: .
When the student is asked to draw a
ap on a particular phenomenon there are some complexities. Each student
draws a map explaining the same concept but with a difference in representing the
structure. The structure of the map varies from each student. A teacher could find it
hard in this cas
e to find out whether the student has understood the concept.
A concept map’s elements are represented in form of shapes that are graph based.
They usually reflect the map builders mental structure in building the concept .The
tool must find some ways to
find the substructures used within a map. Graph theory
algorithms that address the shape (list, tree, spoke, circular, network) of the concept
map . Since concept maps are graphs which represent knowledge in a unique way,
a problem arises when there is
a question about treating the structure of this map. To
be precise there is a question of finding the path or the structure of the map.
Hand written concept maps
: Concept maps that are drawn in a piece of paper
should also be accommodated for eva
luation by the tool. An approach to handle hand
drawn concept maps is needed since this tool is built for the purpose of handling
concept map of any form.
Scoring a concept map is assigning scores in the form of
numbers based on c
ertain scoring strategies. However there is no commonly accepted
way of scoring concept maps. The literature however agrees on some approaches that
are more or less subjective . These scoring strategies are considering the closeness
of a concept map wit
h a master map, number of concepts, number of relationships,
number of valid prepositions shape of concept maps and graph based measures. The
weighted sum of all these approaches could be used as the final score.
These approaches are practical and can be
coded for scoring these maps .However
not all scoring rules is suitable for all types of concept maps. Nowak and Gowin’s
scheme may be useful only when hierarchical levels are identified unambiguously.
But in general the score assigned based on the stru
cture of the concept map has given
accurate results .So the problem is to find a suitable scoring method which is reliable
and easy to use.
In the following sections of the chapter the problem of the thesis are discussed, goals
of the the
sis are formulated. In chapter 2, supporting literature, similar tools that
perform similar functions are reviewed and background of this thesis is furnished. It
covers few but relevant strategies regarding the concept map scoring. It also helps in
ating the reason behind scoring. In chapter 3 design and implementation of the
tool is provided. The core data structure and scoring algorithms that are applied in
building the tool is discussed. In chapter 4, we discuss the various test cases that were
plied on the application divided by parts fulfilling all the possible criteria for
testing. It also includes how each goal of the thesis was achieved reasonably. The
results and evaluation of the tool are also discussed in the same chapter .It also covers
the limitations of the tool and also future work. Chapter 5 consists of final conclusion
of the thesis.
The following list describes all the goals that were aimed when developing the tool:
To address differences in terminologies, match them clearly between
Implement scoring strategies that help in realizing the amount of how
well One understands the concept.
Tool must be flexible and extensible and must accommodate more
strategies in future.
Intuitive and easy to understand user interface.
The goal is to make the tool comprehensive as far as all scoring strategies are
concerned and automate it. The tool must also be extendable for changes and it
be able to integrate with other tools. The user could bring varieties of
prepositions which constitute the map builders knowledge. These varieties pose a
huge challenge since there is no standard way of organizing, computing and scoring a
concept map. The
tool must also be able to organize each preposition in a
standardized manner such that this tool processes them with less time. The next
chapter reviews the literature and background of this thesis.
All the concept maps drawn in this thesis are drawn using the concept mapping
Literature Review and Background
This chapter discusses the application of concept map, reviews the literature,
addresses the issues of matching
with supporting literature and analyzes the various
scoring schemes that are suitable for the tool. The next section discusses the
application of concept maps in common and specifically in the field of education.
2.1 Concept map
Uses and application
Concept mapping techniques and structural organization of the concept maps shows
meaningful learning, progressive difference in the core topic and integrative
reconciliation . Building concept maps try in identifying learners
spotting key concepts and relationships. One of the most important
uses of concept map in the field of education is to compare student map with the
teachers map graphically .
In the field of education, students draw maps to express a particular concep
phenomenon . The teacher further uses these maps to assess the student’s ability to
understand the concept. Assessing these maps have shown considerable amount of
success in understanding the ability of the student in expressing knowledge. However
a student differs from another student in drawing maps by structure of maps he or she
represents , by use of terminology in expressing concepts and also in shapes.
There must be a common method to assess all these maps taking into account the
ural and terminological variations. They should also act as a platform for
scoring these maps.
a cricket concept map built for Student A.
For this reason, we assume two hand drawn concept maps described below in
2.1 and figure 2,2 to be two maps drawn by student A and student B. They focus on
explaining the question “what are the important facts about cricket?” They differ in
structure, there is variation in terminology in expressing the same concept and a
differ in the shapes used in expressing the relationship.
: a cricket concept map built for Student B.
Considering the distinct nature of concept maps a common form of evaluation is
required to assess the concept maps drawn by students . Studies have shown that
evaluation of concept maps are very important since they reflect the map builders
understanding  .
2.2 Treatment by preposition
Prepositions contain two or more co
ncepts connected using linking words or phrases
to form a meaningful statement . In the figure 2.2 one of the preposition is cricket
>by 22 players. This represents one meaningful statement which goes to
explain that cricket is played by 22
The studies previously made indicate the necessity of a tool that maps knowledge
elements in concept maps. There is no written rule in naming the concepts and nodes
when developing a concept map and they allow the map builder to express his or h
own ideas on a wide variety of topics .Therefore tool must actually look to match
elements in the form of prepositions.
2.3 Terminology variations
However previous studies have aimed at focusing on the overall map similarity
without matching the
elements. . A concept builder usually differs from other concept
builders when he uses different terminologies in expressing the elements in a map
. For example in figure 2.1 we have the preposition cricket
batsmen and in figure 2.2 we
have a preposition cricket
>a batsmen. They
both actually mean the same but they are expressed in a different way.
If the focus is completely based on overall map similarity then the evaluation tends to
avoid the variations in terminology used b
y the map builder. Therefore this tool takes
into account the idea of element matching rather than the overall similarity of the
map. This feature however is observed to support assessment of concept map scoring
This lead to necessity of an onlin
e dictionary or a tool which sorts out the problem of
matching terminologies is very much required. An online dictionary like the word net
was included to the tool and it was plugged in.
Master Map v
s Student map
Previous studies ha
ve shown matching prepositions of a student map and a master or
expert map created by experts or teachers can achieve better results in evaluating the
maps. Chen , Lin and Chang evaluated student maps based on fuzzy algorithms and
fuzzy based matching tech
niques which enlightened the importance of ranking
concept nodes and relationship links between the concepts nodes. They constructed a
master map from maps constructed by 3 experts and matched it with student maps
using the fuzzy algorithm .
They observed some correlation between the performance of the student in a
handwritten test and similarity between the student map and the master map. The
correlation was more significant in the case of students who perform well and also for
ect matters that were difficult. The accuracy went down when the organizational
structure of these maps were distinct. They later concluded that matching techniques
address the actual contextual representations rather than the whole map as an
instance and also called for the need of preposition based matching mechanism .
For this reason we create a expert map for the concept map drawn in the figures 2.1
and figure 2.
2. This map explains the same question “ what are the important facts
about cricket?”. This map contains nearly perfect information about the question. We
also take student map A drawn in figure 1 for comparison. A master map is assumed
to have built with
complete or nearly perfect understanding of the concept. A
student’s map during the phase of analysis needs to require a map such as master map
to find how similar his map has been with respect to perfection..
: an expert map
created for comparing with student map 1 and student map 2
When comparing the expert map and the student map concept wise there is a issue
where one concept in the master map maybe similar to another concept in the student
map but when it is represented as preposition it may vary. Therefore it makes
sense to co
mpare each preposition from the master map with each preposition of the
student map and find out how many prepositions are similar between these 2 maps.
Based on this assumption the prepositions for both the maps where automated from 2
ds and they were made to compare and checked for terminology
variations. Studies seem to show that when concept map prepositions are taken and
compared with maps created by experts they seem to show better results. Further
studies show the importance of ma
tching these techniques since concept maps are
context based and they don’t have a restriction on the vocabulary used in building the
: preposition of expert map
prepositions of concept map developed by student A.
The prepositions of expert map and student map A were compared for similarity .
There are 6 prepositions that seem to match between the student map and the master
map. However, the similarity has to be further evaluated based on scores.
2.5 Role of structur
e in concept maps
Thomas Reichherzer and David leake (2006) 
 analyzed the role played by
structures in concept map. They developed models for analyzing concept maps. These
models were divided in the form of four candidate models representing concepts as
nodes in the concept map. The first 3 models consider map’
s topology while the
fourth model disregards the topology and considers each concept to be equally
important. The models are parameterized, to enable the actual contributions of
structures and hierarchies and connectivity to be determined empirically
A final conclusion based on this experiment notes that subjects rank concepts higher
if they are closer to the map’s root concept and if they have more outgoing
connections or incoming connections relative to other concepts. They also observed
ogy alone is a sufficient indicator to extract topic
relevant information from
Linkage analysis is a concept devised by Liu, Don and Tsai (2005) which helps to
identify misconceptions and false assumptions by students
. The whole idea of this
concept is to take individual concept and link it with the concept map of a student and
then to that of a teacher . This helps in finding out the flaws in developing a
concept map and also establishes ways to improve the map.
An example of how linkage analysis works:
We have a set of concepts C. We have teachers set of concepts C2 which has
most of concepts connected to C. We also have student map C1 which seems to be
connected with the concepts C. In this case he studen
t may end up in confusing C2
with C1. This case shows C1 to be a concept that is possibly confused.
So if a student wrongly connects C1 with concepts in C while most of the concepts in
C seem to agree with the C2 i.e. teacher’s concepts then it po
ints out that the student
is wrong and he can substitute C2 with C1 now.
2.6 Role of shapes in a concept map
From top to bottom
Tree, Spoke and net differentiation .
There are 3 types of
substructures from concept maps namely spoke, chain and nets
(Kinchen and Hay 2000) .
A chain represents single level hierarchy where concepts are arranged one
below the other from top to bottom.
A chain corresponds to a sequence of concepts.
A net represent a substructure where a pair of concepts can be related to one
another of different set of concept links.
The diagram above represent spoke, chain and a net substructure .The above
substructures represent how well concepts are
integrated into the mental models of
the learner. They also indicate how a concept map collapses when new information
contradicts the leaner while developing it.
A spoke substructure fails to unite a former concepts but helps in identifying certa
concepts that are related to the given core (or key organizing) concept. In the figure
we could find that number 5 concepts in the spoke diagram indicates that a learner
will be unable to specify the attributes of a given object without referring to the
object’s class . A chain substructure shows the order in which concepts were
introduced initially as it corresponds to a sequential pattern . They have chance of
getting broken when they are presented with new information. They simply cannot
so quickly and accommodate new information in their flow of representing
concepts. In net substructure concepts are integrated with one another strongly. The
net structure seems to handle new information more smoothly and represents a
regulated form of re
presenting information. Evidence suggests that net structures
indicate meaningful learning (Kinchin et al, 2005) .
Evaluation of concept maps depends on shapes of the concept map to an extent. The
scorer or a teacher could actually judge what a studen
t understood by looking at the
different shapes which the student has used in representing the relationship between
different concepts . However, depending on shapes used within a map for
evaluating the map has its own flaws. As, developing concept map
s are context based
a particular substructure used in representing a relationship need not be a good
indicator in all occasions. For example , a student may represent a particular concept
in a tree structure when the relationship can actually be represente
d only in a tree
structure. A master map preposition could represent a spoke substructure when
representing a relationship while the actual student map may represent a net
substructure . This case indicates that a shape may not be the right indicator in ce
For this reason a set of graph theory algorithm was applied which traverses the
prepositions and indicates the user about the type of shapes used in the map. The
graph theory algorithm can find out if it is a tree, spoke, list or net which ev
er the map
builder has used in exhibiting relationships between the concepts.
Figure 2.7: shapes of concept map developed for student A.
The graph theory algorithm indicates the shape from the preposition file leaving the
user with the convenience of knowing what shapes the map builder has used without
looking at the actual map. The shapes are also one of the approaches in scoring
however it is logical to leave it as an option for the user. This is because if the user
the shapes could be an important factor in scoring the concept maps based on
the context of the preposition that he has then the user could click continue to further
process the shapes for scores.
2.7 Concept map scoring (in detail)
concept maps manually is not a very easy task. It is tedious and a
difficult job. Scoring is method of assigning scores based on scoring strategies .
Scoring a concept map is important as concept mapping usually is troublesome for
many students as it te
sts the personal understanding of the student rather than
knowledge that was merely accumulated .
The strategies basically assign points for the map based on various approaches and
the final score is then computed. We need to find suitable scoring stra
evaluate the map. A scoring strategy gets more logical when we have a master map
since this map acts like a guide .
When building a tool that can evaluate maps one needs to know actual techniques
that could be useful and feasible. This secti
on studies these techniques which lay as
major foundations in building a tool.
Shavelson et al. has proposed some techniques which is useful in evaluating these
maps . They possibly provide different ways to generate and score t
Some of the techniques are:
Quantitative measurement of number of map characteristics such as counting
the number of prepositions.
The levels if hierarchy used in expressing relations within the map.
Assigning scores to reward the validity of prepositions used in the map.
Comparing the map with an expert map and observing the closeness that exist
between these maps.
Quantitative assessment method
The quantitative assessment method provid
es a means to calculate a numerical score
for a given concept as a measurement of a students understanding of a particular
domain . The various schemes in this assessment method are listed below:
Holistic overall scoring method
: This concept of s
coring was developed by
sonak and suen
(1999) which allows the scorer to usually score the map based on
what the scorer has actually understand about the domain on a scale of 0
scorer will have a look at the map and based on his or her observation scoring is made
from a scale betwe
en 0 and 10. This form f scoring seems to be easy and many
previous studies which were conducted on relational and structural also seems to
accept this scoring strategy to produce fruitful results .
The figure 2.8 is an example demonstrated in the tool based on the holistic overall
impression. The student concept map for the question “ what are the important facts
about cricket?” and the expert map for the same question appears in
the figure 2.6.
The user is allowed to rank the map based on a scale of 0
10. The rank value is then
multiplied by 1.
Weighted component scoring method
: This idea of scoring the concept map is to
assign partial points to each component of the map and / o
r links between them. The
total sum of all these points would make the final score of the map . The points are
given usually based on the type of structure they add to the concept map.
The closeness index
: This closeness index developed by Gol
dsmith, Johnson and
action (1991) compares the student map with the teachers map or master map to
find out how close both are. This scoring approach helps in finding the value of
similarity between actual concept and links in the student and teachers m
ap that are
common . This allows a student map which is assumed as imperfect to be
compared with a expert or master map. The value showing how close it is helps us in
finding how much the student has understood the concept. The closeness value is
computed based on the similarity between the expert map and student map.
The comparison for the maps in figure 2.1 and figure 2.3 was done earlier in section
2.4. There were 6 prepositions hat were fund to be similar between the maps. A score
of 3 is assi
gned to every preposition that seems to be similar. The final score is 18.
Qualitative assessment scoring
The Qualitative assessment methods are used to produce descriptive assessment of
the concept map. They usually make a synthesis of various features and provide a
descriptive diagnosis of the understanding . The following tables highlight the
structural and relational scoring the structural scoring method is based on
the hierarchies represented in the concept map. They are developed by Novak and
Gowin(1984). The relational scoring method usually gives points to each link
between concepts in
isolation. The scores r high when they are correctly labeled and
they represent foundational relationship of the domain such as taxonomical and
casual relationship . The relational scoring method was adapter from a technique
developed by Mclure and bel
Feature of concept map
Valid , but incorrectly labeled link between
1 point each
Valid and correctly labeled link between concepts
2 points each
that does not represent a hierarchical , casual or
Valid and correctly labeled link between concepts
3 points each
that does represent a hierarchical, casual Or
sequential relationship between concepts
Link between concepts where no relationship exists
0 points each
The relational scoring scheme has shown high reliability if master map which
measures plain relations between the maps for the area experts. Relational scoring
seems to be a useful method of scoring as shown by the r
esults. They seem to be
useful and easy to use and also easy to implement. Relational scoring also seems to be
valid and reliable in other studies too .
The table below shows a different pattern of scoring. The main focus of scoring is
on the structure and hierarchies. This leads to the idea of choosing each concept
one by one and analyzing each concept in relationship with the other from top to
bottom or vice versa. The idea of choosing structural method is because the concept
map is in itself made up of diverse structure. Each component and relationship from
and to affect the other component directly or indirectly in a concept map .
Features of concept maps
Valid hierarchical link between concepts
Valid link between concepts on different branches of
10 points each
Other valid links between concepts
2 points each
Examples of concepts
1 point each
Invalid concept or link
0 point each
Structural scoring .
Usually these structures start from the main concept that appears on the top of the
map and grow into a big tree like structure representing each concept. Therefore
choosing the structural method actually solves the
problem logically. When the scorer
actually has a control over the structure of the concept the scorer is now able to judge
how well the map builder has understood the concept  .
These methods however practically can be implemented as a software
application and automated easily than rest of the methods. Literature seems to agree
on most occasions that choosing the hierarchy of the concept can reach far more
The structural scoring method provides instructions to score which is ea
implement and has also shown great results. The instruction for structural scoring
scheme was applied on a concept map containing 9 concepts and 9 different links.
The hierarchies depend on the map builder’s interpretation of the concept. The map
builder’s interpretation can be judged by the number of valid prepositions he has
employed in building it. In this case it is 8 valid prepositions out of 9 concepts. The
number of hierarchies from the root is 5 and cross links if in case is valid will be
onsidered as 10 points. The examples mentioned in the map are allotted with 2
points if it is valid .
The final score based on assumption that they are valid comes to 30 which is not a
bad number. When we carefully analyze this method we deriv
e the fact that this
scoring method makes sure that deeper the link is deeper the understanding of the
concept is. It shows how the map builder has managed to break the topic in to levels
of hierarchies and elucidate these levels with corresponding cross l
inks and labels.
2.8 Scoring by the tool (actual methods)
Final summary of results for student concept map A.
The above figure shows the final summary window of the tool after scoring the
student map in the figure 2.1 with the expert map in the figure 2.2 .
The holistic overall impression is given to be 1 out of 10. The number of concepts
said to unique and valid
are 10 . So a score of 1 is assigned to it which makes it 10 in
The number of valid relationship is said to be 10. So the score is (10 * score 1) which
is 10 in total. The closeness value after comparing the prepositions of a
with the preposition of student concept map A is 18.
Closeness show the prepositions which are directly similar between the master map
and the student map . 6 prepositions seem to be similar between each other. So the
score for closeness is 6*
3 which is 18.
The number of valid preposition is the value of closeness plus the preposition which
are valid but don’t match with the prepositions of the expert map. The prepositions
which are student map which don’t match directly based on exac
t similarity and
terminology will be discarded. But however there are some prepositions which could
be unique and may not match directly with the prepositions in the expert map. The
value for these prepositions will be added to the closeness value. The sco
re for 4 valid
prepositions is 4* score of 2 i.e. 8. the closeness value is 18.The overall score for
number of valid preposition in this case would be 26 (18 + 8).
A score of 2 for tree , 3 for spoke , 4 for chain and 5 for net was assigned by the t
The map has 3 tree structures which gets a score of 6 and a score each of spoke and
chain which gets 3 and 4 respectively. The weighted sum of all the scores adds up to
80. As discussed earlier the shape of the concept map may not be always a good
icator of map builders understanding. This example demonstrates the scoring
based on shapes just in case the user wants to score for shapes.
The literature and analysis based on successful experiments goes to show that the
approaches derived from these me
thods could possibly lead to better results in
building a scoring application.
Studies also indicate the fact that by discarding a master map the scoring strategy
becomes vague and unfruitful. Also relational scoring method fills most of the gaps in
ng by considering the importance of cross links and graphical representations of
the map. Most of the other scoring methods are either incomplete or just basic.
The structural scoring method also considers important aspects such as hierarchy
s the map builders understanding from the root of the map.
Graph based measures
Concept maps are used as conceptual graphs in the field of textual mining . An
algorithm which produces the structure of the concept map diagrammatically to the
d aid in further restructuring of the map for its betterment. This however has
an effect of scoring the maps as well. A scorer can evaluate the map on looking at the
structure of the map.
A path finder algorithm that traverses a concept graph and
finds the relationship
between nodes would actually resemble the maps structure diagrammatically. A
scorer from his perspective can judge the map and also allow it to be restructured.
Basic Design and Implementation
chapter discusses the basic semantics and syntactic of the tool. This chapter also
deals with basic design, core data structures, architecture and implementation of the
The tool largely deals with prepositions which are in the form
of texts. The form of
inputs given to the tool will be in form of image files of the map and also a .txt file or
a excel file which contains set of prepositions as input. The image file is useful in a
way the user tends to score the map for overall impress
ion and closeness. The other
input should be an image file of both the concept maps. I.e. master and student map.
The image file could be a jpeg or a bitmap image. This image file is supposed to
allow the user to have a look at the student map and give an
overall rating in a scale
10 for it. To add to the purpose the user can compare the map with a master map
and also affect the rating. However the requirement to treat concepts and nodes within
the concept map is not the original goal of this tool
: Input sets for master map.
The above prepositions shows simple master map designed for testing. This contains
prepositions which have direct similarity with the student map. Then similar texts
which could map
prepositions with that in the student map. A general input form
have the text ‘staff’ followed by a comma then the name of the staff or expert
or a teacher. Then in the next line the focus question must be specified by mentioning
the word ‘Question’ followed by a comma then the topic which the map covers. Then
positions are arranged in the order Concept
A student map preposition
The figure 3.2 is a classic example of student map preposition which is automated
from a notepad. These
prepositions are compared with the master map for similarity
and scored based on the relational and structural scoring method discussed in the
previous chapter. In the next section the data structure and handling of prepositions
are discussed followed the
handling terminology variations between the prepositions
of master and student map.
The prepositions form the main crux of the tool. For the tool to unify all concept
maps it requires a standardization of representing
all prepositions. So while building
the tool it was made necessary to take the actual pattern of preposition as formulated
in a concept map drawing tool such as the IHMC. A preposition is made up of a
concept A , concept B and a relationship $ .
oncept map 1 drawn by the student could have prepositions such
as (A , $, B) , (A 1 , $ 1, B1) , A 2 , $ 2, B 2)… and so on.
Concept map 2 could have prepositions in the similar fashion. Namely,
(C1, $, C2) and (C 5, $ 5, C7).
The prepositions for exam in
one map could be A 1
class and $
“A class has objects “is the preposition in this case. Further in the next section of this
chapter we will see how the tool matches these prepositions for similarities.
basic data structure of a preposition
The table shows the arrangement of prepositions. A
B denotes one preposition.
The following standard is what is used throughout the tool for processing. The
scoring strategies are applied later on for processing. Manually developed concept
maps ‘s preposition can be entered into a notepad or a excel file a
nd fed to this tool
for getting processed.
Is divided into
Table 3.2: Basic data structure of a preposition (real examples).
>Cancer is a live example of preposition that will be used by the
tool for processing.
Hash maps for mapping similarities
Hash maps are basically a data structure which is used for comparing multiple
structures by assigning indices and keys .
The tool was coded using this concept.
The data structure created by the hash map is useful in realizing concept maps in the
same form. A master map’s prepositions and a student concept map’s prepositions are
taken in the form of a hash map for easy compar
ison of similarities. The hash table
concept if required can be represented in the form of arrays also.
The hash function used in the hash map is used to transform the key to the index of
the array element where the corresponding value is to be s
ought. The hash maps are
well dimensioned so the number of instructions for each look up is independent of the
number of elements stored. This function helps in mapping similarities irrespective of
the number of prepositions. The tool is now
capable of handling vast number of
Hash map that stores
master map (every
rows denote array
Figure 3.3: Hash Maps
The above diagram shows the list of prepositions of the student map being stored in a
array list further being compared to the list of preposition of the master map. The
value of the comparison is further computed based on the scoring strategies and the
nal score is listed as shown in the diagram. The prepositions are not compared
hierarchically in this tool or rather matched instantly. The every preposition in each
side is compared for similarity. This would show how close a student map is with
o a master map. The maps perform direct string matching. ‘Strcmp’ function
is used by the map to match strings that are same from both sides. However a concept
map as discussed earlier does not levy any norms and rules on the vocabulary for
building the to
The main advantage of using hash maps is for its speed at which it can locate
similarities . There are cases where the hash table allows constant look ups. This
function makes the tool to allow the users pick those prepositions which are valid yet
not found matching with the master map. Another classic usage of this hash map is
every location can be used as well . This enables the preposition in master map to
be compared with all prepositions in student map forming a hierarchical checking
. The keys used are ahead of time and helps in avoiding collision. It also
establishes neat and smooth comparison between the prepositions .
However the prepositions are texts and hash maps don’t support text match matching
Hash map that stores
Prepositions (rows denote
so well. This was
a major disadvantage which was later solved by concurrently
implementing word net.
Handling terminology variation
Word net is a large database of English. Synsets ,
a part of Word net is a cognitive
synonym which are formed by grouping adverbs , nouns , adjectives and explain a
distinct concept. The role of word net played in this tool is to map synonyms
between master map and student map. A case which occurs whi
le scoring is the use of
text by student. A student could actually be explaining the same concept as mentioned
in the concept map but the student might be explaining it using different words r
texts. The tool can handle different structure but when it come
s to different texts it
appeals to Word net. Word net is structured easily to suit computational linguistic and
natural language processing. In the case of similarity matching it plays a very
word net (score of
The prepositions Smoking
> cancer and Smoking
> source of
mean the same. The concepts smoking and cancer match directly in this case. But
causes and source of don’t match directly. Word net helps in matching
The tool takes each and every word of the prepositions separately and later on
performs the comparison. This helps in indicating matching synonyms between two
prepositions which are expressed in a sentence. The synonym matching
in this case
does occur between 2 words but they are picked anywhere within the sentence for
comparison. This restricts the fact that similar words with same meaning are treated
not only individually but also similar words in sentences are matched and lat
indicated to the user.
is for windows OS
alone). Download and install files from the above link. A jar file called jaw
added to the project folder to enable dictionary. Dictionary path to database files is
defined in the
path file in the configuration folder in the project folder
path is C:/Program Files/WordNet/2.1/dict. Change the root folder if the wor
d net is
installed in different folder accordingly. The word net automatically checks for
similarities in the prepositions and alerts the user. The user now can have all the
similarities at a glance and now realize how well the map builder has been able to
elucidate the concepts though with a different text.
The actual words found in jeopardy in the concept map is also checked . This
is an interesting case because a word which is found in the form of concept with no
connection i.e. no incoming and out
going links could be found and may match with
prepositions in the master map. For such cases hash maps aid in locating the
destination an word net helps in locating the meaning and thereby its similarity with
the master map. The hash maps do manual checkin
g of text and maps similarities.
However, it is restricted since it needs a lot of help from the user . The user
cannot be checking textual similarities as it may take time and make it tedious. Word
net is also user friendly. It is a more professional
tool to check synonyms and synsets
, which basically what the tool looks up to. The implementation part of word net was
successful since we can expect a lot of similarities within these maps. The similarities
usually appear in most cases as a single word.
The implementation of all the above
sections could be manually scene in the coming section of class diagrams.
3.3 Design overview and Architecture
Figure 3.5 :
Design overview of the tool.
The BO package contains the domain specific representation upon which the
application operates. BO package contains all classes which has the data structure
needed by the tool. The Util package contains all the necessary classes that define
ons carried out by the dictionary ,graph algorithms etc. The controller
package contains the main class which makes all the interactions between these
package possible. The GUI package contains the implementation of the user interface
and functions to rend
This application follows the MVC (Model View Controller) architecture closely. The
GUI is the front end, with which the user interacts. These files are packaged as GUI
together. Controller is the one that intercepts the requests from the user and de
legates it to
the appropriate BO class and finally renders the user interface back to the user. This
follows a front controller design pattern and is present in the controller package.
BO classes are the business rules that actually process the request pa
ssed on by the
controller and returns a result to the controller. These are the core classes and are
packaged as BO.
The user interacts with the user interface by clicking the mouse button. The controller
handles the input event from the user interface, b
y a registered handler and converts the
event into appropriate user action, understandable for the BO package. The controller
invokes the BO during a user action. This results to a state change in BO package (For
example, the controller updates the user's
rating which it gets from the user rating panel).
The GUI package queries the BO package in order to generate an appropriate user
interface. The GUI package gets its own data from the BO package.
The controller may issue a general instruction to
the GUI to render itself. The GUI is
automatically notified by the BO package of changes in state (Observer) which require a
screen update. When a BO package changes its state, it notifies its associated package so
they can refresh. The controller receives
input and initiates a response by making calls on
BO objects. The GUI renders the model into a form suitable for interaction. Multiple GUI
can exist when there is a real purpose.
3.4 Application of scoring algorithm
The following algorithm represents the basic relational scoring methodology used in
creating the tool. . Each preposition P in a concept map is given a score between o
and 3 according to the following protocol:
Structural scoring algorithm .
The scoring algorithm forms as the basic for the tool to choose before scoring
prepositions. The algorithm checks if there is a relationships of any sought (direct or
indirect) between 2 prepositions. In case there is a
relationship it gives a score of 1. If
not it gives a score of 0. The algorithm goes to check if the relationship is found in
the order of hierarchy or is it just casual or maybe sequential. A score of 2 is given in
case there is no such relationship stru
cture else a score of 3 is awarded in case the
relationship is any such order.
The tool uses a different algorithm which was derived from the algorithm mentioned
above. Before going into the actual scoring algorithm the tool checks if the questio
in both the preposition is same and proceeds only if they are same. Otherwise it
throws an ‘error’ message. This is to make sure that only two prepositions of the same
map and same question is being compared for similarity and late scored. After this
p the input file date are mapped to the array list in the order concept
Closeness to a master map
: This approach is the heart of relational scoring
technique. The tool checks each preposition in the master map with the
the student map and computes all or no similarities between the 2 maps. Every single
preposition in a master map is checked with all prepositions in a student map one by
one. Scores are assigned simultaneously the final score from these simi
shows the closeness index .
The algorithm for scoring for closeness is as follows:
Step 1: Compare student map preposition with each master map preposition.
Step 2: If there is a matching pattern found between the 2 files then incre
closeness value by 2.
Step 3: Else prompt the prepositions for manual resolving.
Step 4: If there is a matching pattern found after resolving manually then increment
Closeness value by 1; else stop scoring.
The scoring for other parameters or
approaches is done simultaneously while scoring
the closeness value. The closeness value is done only after comparison. Scoring for
other aspects like holistic over all impression, shapes, number of valid concepts,
relationships and valid prepositions are
calculated and displayed in the final summary
window once the prepositions are mapped from the input file.
The design is based on the ideas derived from structural scoring and relational scoring
discussed in the previous chapter .
The theme of this tool is to score a concept
map based on a number of approaches which were drafted from the structural and
relation scoring method. The approaches which we are going to consider for scoring
are discussed below:
Holistic overall impressio
: The overall impression of the concept map is given on a
scale of 0
10 . A score of 10 shows that the user has been able to understand the
concept well and structures of the concept has been strong and well correlated. A
score of 0 shows poor or no u
nderstanding of the concept and the structure of the map
seems to show bad coherence with respect to the concept. A score of 4 or 5 shows
average understanding and moderate approach to representing the structure.
Number of valid concepts
approach applied in this tool counts the number of
concepts which seem to be valid. However not all concepts that were used in drawing
the map would count. Those concepts which are supposedly similar to those concepts
in the master map will be taken
into consideration. There is a void that a concept
appearing in the student map maybe unique by itself and need not necessarily match
with the one in the master map , in that case the tool asks the user to validate the
concept. Such concepts are given mor
e marks since it represents the map builders
ability to illustrate a concept on his own.
Number of valid prepositions
: A preposition would be to a valid relationship
between one concept and another . These could be in spoke, chain and a net
]. A preposition has to be valid with respect to its counter part in the master
map. For this reason the tool checks each preposition appearing in the master map
with that of preposition in the student map starting from the top level of hierarchy.
osition that founds to be matching with the preposition in the master map
will be assigned a point of 2. This matching has to be direct i.e using similar texts or
same texts or seems to be explaining the same phenomenon. However there are cases
are prepositions that the student could illustrate that may not be
comparable with the ones in the master map; in that case separate prepositions will
have to be validated by the user. If found unique and non
matching then a score of 2
is assigned otherwi
se a score of 1 is assigned.
Number of Relationships
: This approach of scoring is basically to find how much
relationships have been exhibited by the student in his map . This helps in finding
the mental ability of the student to expresses the concept
in the form of different
structures in a well correlated fashion. The tool adds all the relationships that the
student has drawn in his map by assigning 1 point for each relationship.
Shape of the concept map
: The idea behind this approach is to know wha
t type of
structure the student has exhibited in the map to represent the map. The shapes could
be tree, spoke, chain and net differentiation . A map score of 3, 4 and 5 are
assigned to the preposition depending on the shape of the map. A graph theory
algorithm for finding the structure of the concept map through the preposition was
developed in java and added to this tool . This algorithm traverses through the
preposition and realizes the shape of concept map through the structure in which it is
In case of (a, b) (b,c) and (c,a) this preposition when drawn a structure represents a
chain structure whereas a (a,b) and (b,c) represents a simple tree. So in this order of
preposition the tool uses this algorithm to find out the
structure which the map
represents. A score of 2 is assigned for a tree, spoke and chain whereas a score of 4 is
assigned for a net differentiation . The reason behind the changes in scoring
pattern for different structure is explained in chapter 3. Ho
wever this structure is
solely based on the context at which the user develops the map. So this scoring
approach is optional for the user. It can be left to the user whether or not to choose
this approach in case the user thinks there is not much connectio
n for this approach in
that map. It can also be left to the user jus to know what type of the structure was
used to represent the map and it need not be scored further.
Graph based measures
: These measures could be useful in scoring the concept map
structure wise . A. This graph theory based algorithm is used to check the
relationship between the concepts of the map and finally derived by the proximities of
the pair of the entities . They express the pattern of relationships in the map.
er this tool tries to implement an algorithm which produces the relationship
pattern and giving them a suitable score.
A path finder algorithm as a part of graphical representation was developed in this
tool. The algorithm traverses from the root node to
the top. It traverses hierarchically
from the root to the top . In our case the inputs will be in the form of prepositions
that consists of data in the form of variety of texts. The texts are in the format
(Concept A, relationship $, Concept B). The p
repositions have to be in this order to
make the algorithm conduct a search. The search starts from Concept A and takes its
path through the relationship $ and finally meets Concept B. Then it moves onto the
next line that contains Concept A1.
ntation took place in a separate class that calls this algorithm on a set of
prepositions and this class was integrated with the rest of classes. The Breadth First
search traverses prepositions in the order of A, B, C, D, E, F. For example, let us
hat we have these prepositions:
(OOP, class, object, attribute, method, data type) then the root node here is OOP and
it starts its traversal from OOP and goes until data type which is the final preposition.
The BFS written in java for this tool computes
the prepositions in the same order i.e
(OOP , class , object , attribute , method , data type) and the DFS search traverses in
the order of ( OOP , class , object , method , attribute , inheritance , data type).
The idea behind traversing the prepositions
was to provide support for the tool to know
path of the graph. By knowing the graph it could help in restructuring the map.
Path finder algorithm is integrated in this tool and finally appears in the summary frame
that shows the overall output of the
tool. The algorithm was written in java and it
provides the path as the output in the final frame just under the summary. The scoring
scheme depends purely on the context of the user. So the path finder algorithm is kept
optional in the tool for the user a
s concept maps are build on contextual information. The
necessity of a path finder may or may not be suitable in some situations.
The user can activate it by choosing it and then rate it. The score from this will be added
to the summary. If the user finds
the algorithm not relevant to that context keeping in
mind the context in which the map builder has done the project then the user need not
Weights of all the scores:
The weighted sum of all these approaches will be the final
the map .
3.5 Package Diagrams
The tool has 4 packages namely package BO , package controller , package util and
package gui. In this section we will see the description of the class diagrams and the
implementation of these packages in
detail. The descriptions carry details about the
relationship between various classes within a package and also the significant
methods and functions that constitute the classes.
: class diagram of
The package.BO; contains the basic data structures of the tool. The methods, structures
and all functionalities are provided by this class. The classes and its main functions and
purposes in this package is listed below.
: This class gets the question for the comparison for the input files (student
map and master map prepositions). The Question.java class is named as a question since
this tool follows a standardized format in giving the input prepositions. It is indexed
the name question after which the array of prepositions is taken in.
: The class has the structure of the maps with created question given first
importance followed by the concepts and relationships involved in the prepositions of
input files. This class is used for adding and setting relationship between both the maps.
It contains all the basic data structure for the preposition file to be used in this tool. It is
basically done in hash maps. The mapping between the two classes
is done through the
hash maps called in this class.
: This class has methods like get() , add() and set() data for the student
map. This class is used for setting up the student map file.This also creates an array list
containing the student
map prepositions which is to be compared with the master map
: This class is used to set up the master map file, creating an array list with
which the comparison with the student map preposition will be made. The master
itions are taken as strings or texts in a huge array list
The package.util consists of classes that are considered for auxiliary purposes in this tool.
: This class is used to look
up synonyms for given word that appears
between the student map prepositions and master map prepositions.
: This class constructs the traversal of nodes . For this the input file
prepositions are considered and they are traversed from bottom to
top. This helps in
finding the path of the concept graph.
: This class defines the structure of the node.
: This class reads the configuration file for the path of dictionary files.
: This is sued to adjust
the size of the picture in order to fit into the
: This class reads the configuration file for the score defined by the user.
This class creates the data structures for the gui and its frames. It calls all
other Gui frames and panels as per the flow of processing of the class. MainGui is the
main class for this package. This acts as the Frame (or Parent Window) of this
on. Acts as another controller within the GUI classes.
This is a Jcomponent which is a base class for all Swing components
level containers JFrame, JDialog, and JApplet. This component displays
pictures to the user.
: This class is a model dialog box which throws error messages,
information to the user.
: This panel is one of the panel that is used when manually resolving
: This panel is used to
select the files that are given as input for marking.
: This displays rating option in order for the user to select. This aids the
holistic overall impression window for choosing the overall impression.
Controller.java is the main class which handles the all the events accordingly. Controller
is the one that intercepts the requests from the user and delegates it to the appropriate
and finally renders the view back to the user. This class imports data from
GUI package to setup the graphical user interface, display messages, upload messages,
imports data from BO package to get the array list prepared and sets the rules for
rison . While comparing this class imports data from the Util package mainly such
as the auxiliary data like dictionary, loading and scoring functions.
The controller.java class lists out the sequences or the order by which processing of the
classes based on various scenarios. The java.io.file is assigned by the controller class for
the input output stream through which the input file can be realized and
output can be spawn. Util.LoadData.java is used by the controller for uploading the data.
The master map prepositions and student map prepositions are handled directly and made
to run by the controller.java class. LoadStaff() method assigns th
e controller with
particular string that needs to be uploaded
The scoring involves the usage of the following methods.
when this method is called the controller sets the initial values which
stored which the input is loaded and
when this method is called the controller sets the value set by the thru the
user interface as the value or the score given by the user.
this will set the value which is the result of comparison of the two files after
the automatic and manual resolving functions are carried out.
final _score ()
this method when called will result in the summing up the scores which
have been collected.
this method will be responsible for the displaying of the summar
y of the
scores in the summary page on the user interface.
The scoring is done through comparison of the array list which have been created from
the files input by the user. Once the user uploads the file through the user interface the
uploaded and the array lists are prepared and then when the score option is
selected thru the user interface the controller handles this event by passing the array lists
for comparison and getting a value, based on the result of the comparison, the obtaine
value is again passed by the controller by a response event to the user interface which
results in the display of the score in the user interface.
This chapter furnishes the results achieved by the tool after it
was tested on various
scenarios. The results also discuss the problems that were faced and the required
solutions given to the problem. The problems are analyzed and it reveals on what
cases the tool works and what cases the tool cannot work. The tool was
for quality using SD metrics. The last section discusses the case of manually drawn
concept maps in detail and further enhancement which could be made to this tool to
solve the problems that were discussed earlier in this section.
constraints and front end of the tool
The input sets as discussed earlier in chapter 3 have been designed in such a way that
it should have a question. The topic of the concept map is supposed to be specified
question followed by a comma and then the topic name. The tool cannot match
the preposition of both the maps if the question is different in character. The diagram
below shows the front end of the tool. The friend end has the list of scoring strategies
described in the previous chapter just to show the user on what scoring approaches
does the tool score. The front end has a next button on clicking that it moves to the
upload window which uploads the student map ,
master map and prepositions of these
Figure 4.1 :
input files for wrong questions (master map)
Figure 4.2: input files for wrong questions (student map).
The tool cannot accept a
preposition which has questions that are different in text and
character used. This is because the tool used hash maps to represent a student concept
map and a master map for comparison. The hash maps as discussed earlier in chapter
3 takes the conce
pt maps in two different hash maps and compares the prepositions
for similarity. The matching process starts first by checking for the question. The tool
cannot check further if there is a change in the format of question. So this demands
the user to write
the question in the same format for both the maps. The above
example shows the difference in expressing the question by varying a word ‘rules’
and ‘rule’ will be considered as a mistake and tool cannot take it. This makes sure
that concept maps which belo
ng to the same topic question is only compared. On the
flip side this restricts users who want to directly enter prepositions and score it. This
issue can be later addressed in the future and required technical changes can be made
to counter this.
This case involves prepositions which repeat more than once. This could be an issue
since this preposition could be counted more than once for scoring. The tool basically
treats repeatable prepositions as ‘duplicate’ and discards the
These type of prepositions could affect the accuracy in giving correct scores if they
are treated. For this reason the tool takes the string just once and compares it with
master map. However the tool does not restrict the use of same preposit
ion again and
again. But when it is being scored these repeatable prepositions is matched using a
string algorithm which usually doesn’t take the same string again.
4.3 Upload window
Figure 4.3 :
The upload window takes the image and the text files containing the prepositions. The
staff input files stream must have the file path of the staff image file in .jpeg and the
input file for prepositions .txt. This holds good for th
e student files also. Then on
giving upload these files are uploaded. The student text file and staff text files are
then sent for scoring and the image file appears in the next window for allowing the
user to score based on holistic overall impression.
4.4 Holistic overall impression
Figure 4.4 :
Holistic overall impression window.
After the files are uploaded the user is asked to rate the map based on 0
window shows 2 maps in comparison.
The map created by the experts or the master
map comes on top while the student map comes in the bottom. The student map is the
map that is going to be scored. But the user is given an option to see both the maps so
that the opinion on the student map is a
ffected. On giving a score of 0
10 the score
gets recorded and then sent to the final summary window for publishing. It is also
added to other scores for computing the weighted score of the whole map. After
giving next the tool starts to match the preposit
ions between the master and student
maps through the text file which was uploaded before. The next section discusses the
matching in detail and also discusses the problems that were faced and the solutions
that were offered henceforth.
This section deals with preposition matching and various problems faced when
matching these prepositions for similarity and terminology differences.
Self calling prepositions
The preposition highlighted below is a self calling preposition.
>enzyme calls itself when drawn in a map. In terms of concept
mapping it means one of its outgoing link actually points to itself This preposition
could pose a problem when scoring since they are unique in nature. They may or