COMPARISON AND EVALUATION OF COURSE MANAGEMENT SYSTEMS FOR
EFFECTIVE LEARNING
By
Samuel Dominic Chukwuemeka
A THESIS
Submitted in partial
fulfilment
of the requirements
for the degree of Master of Science
in Computer Science
in the Graduate School of
Troy University
TROY, ALABAMA
MAY 2013
COMPARISON AND EVALUATION OF COURSE MANAGEMENT SYSTEMS FOR
EFFECTIVE LEARNING
Submitted by Samuel
Dominic Chukwuemeka
in partial
fulfillment
of the requirements
for the degree of Master of Science
in Computer Science
in the Graduate School of
Troy University
Accepted on behalf of the Faculty of the Graduate School by the thesis committee:
______
____________________________________
__________________________________
Irem Ozkarahan, Ph
.
D
.
Date
Chair
__________________________________________
Sunil R. Das, Ph
.
D.
__________________________________________
Matthew Mariano, Ph
.
D
.
_________________
__________________
_____________________________
Bill Zhong, Ph.D.
Date
Chair, Department of Computer Science
___________________________________
_____________________________
James Rinehart, Ph.D.
Date
Dean of the College of Arts and Sciences
__________________________________________
__________________________________
Dianne L. Barron, Ed.D.
Date
Associate Provost and
Dean of the Graduate School
MAY 2013
ABSTRACT
COMPARISON AND EVALUATION OF COURSE MANAGEMENT SYSTEMS FOR
EFFECTIVE LEARNIN
G
Samuel Dominic Chukwuemeka
Technology has had a global impact on almost every aspect of human lives. Of
particular importance is the impact of technology in education. Many secondary schools,
colleges, and universities now offer several courses and deg
rees online. Noteworthy to
mention is the offering of MOOCs (Massive Online Open Courses) by several
universities to the global populace. These online courses are administered using course
management systems. These systems have varying features which direc
tly or indirectly
affect the teaching and student learning experience of these courses.
Despite the numerous deliveries of online courses using these systems, the
effectiveness of online learning is still being questioned. Hence, there is the need to
eva
luate these course management systems for their effectiveness in maximizing student
learning. In this thesis, several course management systems and their features shall be
studied. Several surveys will be taken, and reports will be presented on the results
of
these surveys. Three major course management systems namely Blackboard/Pearson,
Moodle and Desire2Learn course management systems will be evaluated using
appropriate statistical techniques. An overall survey will be done using a random sample
of users
of these course management systems. One

Way Analysis of Variance will be
used to compare the equality of the means of these three systems. Results will be
interpreted. Possible alternatives and improvements shall be recommended.
HUMAN OR ANIMAL SUBJECTS
REVIEW
For
SAMUEL DOMINIC CHUKWUEMEKA
COMPARISON AND EVALUATION OF COURSE MANAGEMENT SYSTEMS FOR
EFFECTIVE LEARNING
This research project has been reviewed by the Research Review Board and approved as
follows:
□
Neither humans nor animals
will be used and this research is certified exempt from
Research Review Board review by the thesis committee chair.
□
Human participants
will be used and this research is certified exempt from Research
Review Board review by the thesis committee chair.
□
Human participants
will be used and this research was reviewed and is approved by
the Research Review Board.
□
Animal participants
will be used and this research was reviewed and is approved by
the Animal Research Review Board.
______________________
___________
________
Signature of Thesis Committee Chair
Date
_________________________________
________
Signature of Chair of Research Review Board
Date
Copyright by
SAMUEL
DOMINIC CHUKWUEMEKA
2013
DEDICATION
This thesis is dedicated to the Most Holy Trinity: God the Father, God the Son, and God
the Holy Spirit, my creator and helper.
vi
ACKNOWLEDGMENTS
My utmost thanks go to the
Almighty God. Without Him, this thesis would have
been nonexistent. I truly appreciate my mother, brothers, sisters, nephew and nieces.
They were also influential to my completing this thesis. I am very grateful to my
supervisor, Dr. Ire
m Ozkarahan for her
help
, supervision, encouragement, and guidance.
She was always there for me, to ensure that I completed this thesis. She
i
s a supervisor,
teacher, and mentor.
I sincerely thank my thesis committee members, Dr. Sunil Das, and Dr. Matthew
Mariano. They have
been very helpful towards my success. In addition, I am grateful to
Mr. Scott Morton for all his good advice and efforts towards the success of this thesis.
I appreciate the efforts of the Department Chair, Dr. Bill Zhong; the Dean of the
College of Arts
and Sciences, Dr. James Rinehart; and the Associate Provost and Dean of
the Graduate School, Dr. Diane L. Barron. You are all contributors to my successful
graduation from our prestigious institution, Troy University.
vii
TABLE OF CONTENTS
Page
LIST OF TABLES
………………………………………………….…….……………..ix
LIST OF FIGURES ………………………………………………………….…………
..x
LIST OF SYMBOLS AND ABBREVIATIONS
………………………………………
.xiii
CHAPTER 1: INTRODUCTION ………………………………………………
.
…….....1
Overview of Learning…………………………………………………….1
Overview of Cours
e Manageme
nt Systems………………………………
2
CHAPTER 2: LITERATURE REVIEW ………………………….…………….……...
..5
History of Online Learn
ing and
Course Management Systems………….
5
Features of Course
Management Systems………………………………..6
CHAPTER 3:
DATA METHODOLOGY
................................................................…...
.11
Data Col
lection ………………………………………………………….11
H
ypothesis and Hyp
othesis Testing
…
…………………………………..
.
15
Overvi
ew of ANOVA……………………
………………………………
17
CHAPTER 4:
DATA METHO
DOL
OGY USING TECHNO
LOGY…………………...21
Data Pre
sentation of
the
Survey Responses
...
……………………………25
CHAPTER
5
: DATA ANALYSIS….…………………………
………………………...42
Testing the Require
me
nts of One

Way ANOVA……………………….42
ANO
VA
by Hand………………………………………………………..47
ANOVA b
y T
echnology………………………………………………...47
Interpreting Results……………………
………………………………
...50
CHAPTER
6
: CONCLUSION………….
…………………….…
………………………51
LIST OF REFERENCES …………………………………………………………….…
55
APPENDICES
…
………………………………………………………………………...57
viii
LIST OF TABLES
Page
Table 2.1
…………………………………………………………………………………10
Comparing the Features of Three CMSs
Table 3.1…
………………………………………………………………………………12
Course Management Systems Survey
Table 3.2
…………………………………………………………………………………15
Evaluation of Course Management Systems Survey
Table 3.3
…
………………………………………………………………………………17
Four Outcomes of Hypothesis Testing
Table 4.1………………………………………………
……………………………..…..30
Quantitative Results of the Sample Survey
Tab
le 5
.1………………………………………………
……………………………..…..42
Overall Numeric
Results of the Survey
Table 5.2
…………
…………………………………………………………………..…..46
ANOVA Table for a Completely Randomized Design
ix
L
IST OF FIGURES
Page
Figure
1.1
…………………………………………………………………………………3
Some Course Management Systems
Figure 4
.1………
………………………………………………………………………...21
Using Excel to G
enerate
Random Numbers
–
1 of 4
Figure 4.2………………………………………………………………………………...
23
Using Excel to G
enerate Random Numbers
–
2 of 4
Figure 4.3………………………………………………………………………………...
23
Using Excel to G
enerate Random Numbers
–
3 of 4
Figure 4.4…………………………………………………………………………
……...
24
Using Excel to G
enerate Random Numbers
–
4 of 4
Figure 4.5………
………………………………………………………………………...25
Representing Survey Responses to Question 1 on a Multiple Bar Graph
Figure 4.6
………………………………………………………………………………...
26
Representing Survey Responses to Question
2
on a Multiple Bar Graph
Figure 4.
7
………………………………………………………………………………...
26
Representing Survey Re
sponses to Question 3
on a Multiple Bar Graph
Figure 4.8
………………………………………………………………………………...2
7
Representing Survey Re
sponses to Question 4
on a Multiple Bar Graph
Figure 4.
9
………………………………………………………………………………...2
7
Representing Survey Re
sponses to Question 5
on a Multiple Bar Graph
Figure 4.10
……………………………………………
..
………………………………...2
8
Representing Survey Responses to Question
6
on a Multiple Bar Graph
Figure 4.
11..
……………………………………………………………………………...
28
Representing Survey Responses to Question
7
on a Multiple Bar Graph
Figure 4.12……..
………………………………………………………………………...29
Representing Survey Responses to Question 8 on a Multiple Bar
Graph
x
Figure 4.13..
…
…
………………………………………………………………………...29
Representing Survey Re
sponses to Question 9
on a Multiple Bar Graph
Figure 4.14..……
………………………………………………………………………...31
Using Excel to Find
Descriptive Statistics
–
1
of 5
Figure 4.15..……
………………………………………………………………………...32
Using Excel to Find Descriptive Statistics
–
2
of 5
Figure 4.1
6
..……
………………………………………………………………………...32
Using Excel to Find Descriptive Statistics
–
3 of 5
Figure 4.17
..……
………………………………………………………………………...33
Using Excel to Find
Descriptive Statistics
–
4 of 5
Figure 4.18
..……
………………………………………………………………………...33
Using Excel to Find
Descriptive Statistics
–
5 of 5
Figure 4.19
..……
………………………………………………………………………...34
Using
Minitab to
Dra
w the Data Distribution
–
1
of 6
Figure 4.20..……
………………………………………………………………………...34
Using Minitab to Dra
w the Data Distribution
–
2
of 6
Figure 4.21..……
………………………………………………………………………...35
Using Minitab to Dra
w the Data Distribution
–
3
of 6
Figure
4.22..……
………………………………………………………………………...35
Using Minitab to Dra
w the Data Distribution
–
4
of 6
Figure 4.23..……
………………………………………………………………………...36
Using Minitab to
Draw the Data Distribution
–
5
of 6
Figure 4.24..……
………………………………………………………………………...36
Using Mi
nitab to Dra
w the Data Distribution
–
6
of 6
Figure 4.25..……
………………………………………………………………………...37
Using Minitab to Draw th
e Boxplots for the Data
–
1
of 5
Figure 4.26..……
………………………………………………………………………...38
Using Minitab to Draw th
e Boxplots for the Data
–
2
of 5
Figure 4.27..……
………………………………………………………………………...38
Using Minitab to Draw th
e Boxplots for the Data
–
3
of 5
xi
Figure 4.28
..……
………………………………………………………………………...39
Using Minitab to Draw th
e Boxplots for the Data
–
4
of 5
Figure 4.29..……
……………………………………………
…………………………...39
Using Minitab to Draw th
e Boxplots for the Data
–
5
of 5
Figure 4.30..……
………………………………………………………………………...40
Sample Mean and Sample Variance of Blackboard/Pearson System
Figure 4.31..……
………………………………………………………………………...41
Sample Mean and
Sample Variance of the Three CMSs
Figure 5.1....……
………………………………………………………………………...43
Using Minitab to Draw the N
ormal Probability Plots
–
1
of 6
Figure 5.2....………
……………………………………………………………………...44
Using Minitab to Draw the N
ormal Probability Plots
–
2
of 6
Figure 5.3....……
………………………………………………………………………...44
Using Minitab to Draw the N
ormal Probability Plots
–
3
of 6
Figure 5.4
....……
………………………………………………………………………...45
Using Minitab to Draw the N
ormal Probability Plots
–
4 of 6
Figure 5.5
....……
………………………………………………………………………...46
Using Minitab to Draw the Norma
l Probability Plots
–
5 of 6
Figure 5.6
....……
………………………………………………………………………...46
Using Minitab to Draw the N
ormal Probability Plots
–
6 of 6
Figure 5.8....……
………………………………………………………………………...48
Usi
ng Minitab to Run ANOVA
–
1
of 3
Figure 5.9....……………………………
………………………………………………...49
Usi
ng Minitab to Run ANOVA
–
2
of 3
Figure 5.10..……
………………………………………………………………………...49
Usi
ng Minitab to Run ANOVA
–
3
of 3
xii
LIST OF SYMBOLS AND ABBREVIATIONS
CD

ROM
–
Compact Disc
–
Read Only Memory
CMS
–
Course Management System
LMS
–
Learning Management System
VLE
–
Virtual Learning Environment
KMS
–
Knowledge Management System
ANOVA
–
Analysis of Variance
CALC
–
Computer Assisted Learning Center
HTML
–
Hyper
Text Markup Language
AICC
–
Aviation Industry Computer

Based Training Committee
IMS
–
Information Management Standard
SCORM
–
Shareable Content Object Reference Model
n
–
Size of the Sample
N
–
Size of the Population
H
0
–
Null Hypothesis
H
1
–
Alternative
Hypothesis
Α
–
α
–
Level of Significance
̅
–
Sample Mean
̿
–
Overall Sample Mean
s
–
Sample Standard Deviation
µ

Population Mean
σ
–
Population Standard Deviation
MSE
–
Mean
Square D
ue to Error
SSE
–
Sum of Squares
D
ue to Error
Σ
–
S
ummation
df
–
Degree of Freedom
MST
R
–
Mean Square
D
ue to Treatment
SST
R
–
Sum of Squares
D
ue to Treatment
WYSIWYG
–
What You See Is What You Get
xiii
1
CH
APTER 1: INTRODUCTION
OVERVIEW OF LEARNING
Teaching and Learning has been, is, and will continue to be an important part of
human history and education. Education seen as a critical aspect of the future of any
country, gives knowledge. Knowledge in turn, gives power.
Learning can
occur as:
Traditio
nal or Synchronous Learning, Distance Learning, Blended Learning, and Online
or Asynchronous Learning.
Traditional learning is the learning that typically takes place in the classroom
environment or brick

and

mortar schools. This learning is mainly a tea
cher

centered
approach, and students look at the teacher as the main source of knowledge. Assessments
and evaluations are usually done in class; the teacher grades them according to a rubric
and grading scale, and provides feedback to the students in class
. Majority of the teaching
and learning activities are done within the walls of a classroom.
Distance learning is the learning provided to students who are located at a
reasonable distance from the traditional classroom. The instructional delivery include
d an
instructor who was physically located in a different place from the learner, as well as
possibly providing the
instruction at disparate times
(Dede, 1996)
.
Blended learning is a mix of traditional learning and online learning. It combines
the learning activities done in traditional classrooms with the learning activities done
online. It is an integrated instructional strategy that is both teacher and student
centered.
“
Those who use blended learning environments are trying to maximize the benefits
of
both the face

to

face and online methods
[
or technology delivery methods
]
–
using the
Web for what it does best and using class time for what it does best” (Osgut
horpe &
2
Graham, 2033, p. 227)
.
The objective is to provide the best of both worlds
–
traditional and online learning
experiences. It is seen as meaning “to combine any form of instructional technology (e.g., videotape, CD

ROM, Web

based training, film) wi
th face

to

face instructor

led training
” (Driscoll, 2002). Blended
courses are sometimes referred to as “hybrid” courses or “mixed

mode” courses.
Online learning is the learning that occurs via an online or asynchronous medium. It typically
involves the u
se of computing and telecommunication devices to facilitate the acquisition of knowledge. It
is mainly a student

centered learning approach as the teacher is seen as a facilitator rather than a teacher or
instructor. Students acquire knowledge from many so
urces, unlike the traditional classroom where they
gain knowledge mainly from the teacher
, and books or teaching materials
. The use of technology and the
internet are usually required. Teachers and students also communicate but not face

to

face. Several st
udies
have shown that there is increased communication and collaboration among students in an online
classroom than the traditional classroom. Online learning is described by most authors as access to learning
experiences
via the use of some technology
(Be
nson, 2002; Carliner, 2004; Conrad, 2002). It is becoming
more popular locally and globally, and is often referred to as eLearning. Sometimes, online learning takes
place when both the students and the instructor are online at the same time. Most times, it
takes place
irrespective of whether the instructor and the students are online at the same time. Online learning typically
takes place via a course management system. The facilitator or instructor uses chat, video conferencing,
web conferencing, virtual w
orlds, and internet podcasts, among others to communicate learning objectives,
expectations, tasks, and evaluations to students.
OVERVIEW OF COURSE MANAGEMENT SYSTEMS
A Course Management System (CMS) is basically the platform via which an online course
is
delivered. It is sometimes referred to as a Learning Management System (LMS), Virtual Learning
Environment (VLE), or a Knowledge Management System (KMS). Though a Course management system
is sometimes referred to as a Learning management system, there i
s a key difference. Course management
3
systems are designed to support academic classroom courses while learning management systems are
designed to support corporate training.
A CMS
“provides an instructor with a
set of tools a
nd a framework
that allows the
relatively easy cre
ation of online course content
and the subs
equent teaching and
management
of that course
including various interactions
with student
s taking the course” (EDUCAUSE
Evolving Te
chnologies Committee, 2003, p.
1).
Examples of course managemen
t systems include
Blackboard Academic Suite, Pearson LearningStudio Campus, Moodle, Desire2Learn Learning
Environment, Angel Learning Management Suite, and Sakai, among others. A display of the logos of some
of the
se course managements systems can be
seen
in
Figure 1.1
Figure 1.1
Some Course Management Systems
In this thesis, Chapter One is the
Introduction
.
T
he various forms of learning; as well as course
management systems, the platforms via which online courses occur
are
briefly discussed
. Chapter Two
is
the
Literature Review. It
discusses the history of online learning, history of course management systems, as
well as the general features of course management systems. Chapter
Three
is
Data Methodology
.
This
4
explains the
sampling technique used to collect general
survey responses from students who have used, or
who use these three major course management systems
–
Blackboard/Pearson course management system,
Moodle course management system, and Desire2Learn course manageme
nt system. Pearson MyLabs
is
integrated with Blackboard
systems. The labs cover a wide array of academic disciplines with numerous
examples
. A lot of students have found it very useful in learning math and other subjects.
http://www.pearsonhighered.com/resources/Pearson_Global_Whitepaper.pdf
.
T
he data presentation,
explanation, and the computation of the descriptive statistics of the result
s
, are explained
.
The
concept of
One

Way ANOVA and its computations by Hand, and by Technology
are discussed
. In
C
hapter
F
our

Data Methodology Using Technology
, we shall use
the statistical software, Microsoft Excel and Minitab to
obtain the descriptive
statistics, histogram, an
d boxplots of the data we obtain
from our survey. We now
use the inferential technique, One

Way Analysis of Variance (ANOVA) to test the hypothesis
developed
from chapter four in Chapter F
ive
–
Data Analysis
.
We shall use
Microsoft
Office Excel
to
run
the
ANOVA test, and the Minitab to draw the normal probability plots of our quantitative data results.
Chapter
Six
is the
Conclusion and Recommendation
s. This
summarizes the results of
our evaluation
.
P
ossible
recommendations to the th
ree major systems, on some additions that would be useful for the overall
student learning experience
, are recommended
.
5
CHAPTER TWO: LITERATURE REVIEW
HISTORY OF ONLINE LEARNING
AND COURSE MANAGEMENT SYSTEMS
The history of online learning dates
back to the origin of the internet (formerly known as
ARPANET commissioned by the Department of Defense in 1969 (
Hobbes’ Internet Timeline
–
the
definitive ARPAnet and Internet history
).
Howeve
r, it was not until early 1995 that CALC Online Campus
moved on the internet as
CALCampus.com
CALCampus which means
Computer Assisted Learning
Center (CALC) was founded in 1982 in Rindge, N
ew Hampshire as a
small, offline computer

based adult
learning center.
The campus was actually the first to develop and implement a
fully online

based school, as well the course
management system, QuantumLink campus. This enabled real time communication among administrato
rs,
classroom instructors, and students.
Later in 1995,
the University of Illinois developed Mallard
and
CyberProf
web

ba
sed course management system (
University of Illinois
).
In the mid

1997, CourseInfo
founded by Dan Cane and Stephen Gilfus released the ILN
–
Interactive Learning Network.
The ILN was
the first eLearning system to levera
ge and install on a relational database system,
MySQL
.
Several other
course management systems were being developed and implemented during these years.
In 1997,
Blackboard
was
founded
by Michael Chasen and Matthew Pittinsky
.
That same year,
CourseInfo LLC was founded at Cornell University.
Blackboard and CourseInfo LLc later merged in 1998
and launched a free course system for faculty known as Course
S
ites.
(
Bla
ckboard  Our Story
).
Cisco
Networking Academy Management System, CNAMS was also released in 1998 to facilitate
communication and course management of Cisco Networking Academy.
In 1999, Desire2Learn was
founded in Canada.
In 2001, Moodle.com ran Moodle. S
akai project was formed from several college and
university projects and
was
released in 2004.
6
FEATURES OF COURSE MANAGEMENT SYSTEMS
Course management systems (CMSs) have become
a symbol of innovation at institutions of higher
education and in less th
an a decade, they have been rapidly adopted by a large number of colleges and
universities in many countries around the worl
d
(Coates, 2005; Dutton, Cheong, & Park, 2004;
Malikowski, Thompson, & Theis, 2007; Wise & Quealy, 2006)
.
While CMSs were initially
developed to
support distance education
and online courses, they are now used predominately to complement campus
based classroom courses (
Morgan, 2003; West et al., 2006; Wise & Quealy, 2006).
The number and
diversity of CMS features has led to their use
in offering distance learning course
s, offered over the Web
(Allen &
Seaman, 2004, 2005; Morgan 2003).
Hence, the increase in the use of CMSs is directly
proportional to the increase in the number of courses offered in online format, as well as the increas
e in the
number of users enrolled in these courses.
Enrollment in these courses has steadily increased in recent
years. In the fall of 2002, 1.6 million students enrolled in an online course. In the fall
of 2003
, enrollment
was 2 million, and in the fall o
f 2004, enrollment was 2.3 million (
Allen & Seaman, 2004, 2005).
The basic features of CMSs include the functionality of transmitting the course contents
, creating
class di
scussions, and evaluating students. Many CMSs have several other features that cove
r several other
functionalities.
Beginning from the first function, the most common feature of CMSs is the ability to share
files including submitting assignment files, post announcements to the class, post the class syllabus and
other course contents,
pos
t
grades and comments in the
grade book
,
among others.
Instructors most often
use a CMS
to transmit course content, such as syllabus, related reading, or assignment. This course content
is most often given to students
in the form of computer files uploaded
to a CMS. These typically include
word processor files, PowerPoint presentations, or HTML files
(Ansorge & Bendus, 2003; Dutton et al.,
2004; Morgan, 2003; Woods et al., 2004).
The second feature consists
of tools that enable students and/or
users to inte
ract asynchronously.
Regarding the third feature, there are several tools within the CMS for
evaluating students. Assessments include, but are not limited to assignments, discussions, and quizzes.
Quizzes can be instructor custom quizzes or generated from
a test bank. They can contain a variety of
7
question types including multiple choice, fill

in,
true or false,
matching
, short answer, long answer,
essay,
among others.
Majority of
these varieties
of quizzes are automatically graded and entered in the course
grade book.
Long answer, and essay assignments and quizzes are usually
submitted via the
drop box
in a
CMS.
A more detailed comparison of CMS features for the three course management systems:
B
lackboard/Pearson system, Moodle, and Desire2Learn learning environment
is shown in Table 2.1
(Taken from
http://lms.findthebest.com/
,
http://moodle.blogs.we
sleyan.edu/home/
,
http://www.elearninglearning.com/communication/desire2learn/engineering/
)
Features/CMS
Blackboard/Pearson
Moodle
Desire2Learn
Administration
Features
Administrative
Reporting
Authentication
Hosted Services
Registration
Management
User Access Controls
Administrative
Reporting
Authentication
Hosted Services
Registration
Management
User Access Controls
Administrative Reporting
Authentication
Hosted Services
Registration Management
User Access
Controls
Course
Development
Features
Blended Learning
Virtual Classroom
Course Catalog
Course Interactivity
eLearning
Exam Engine
Goal Setting
Individual

Based Plans
Blended Learning
Virtual Classroom
Course Catalog
Course Interactivity
eLearning
Exam Engine
Goal Setting
Individual

Based
Plans
Blended Learning
Virtual Classroom
Course Catalog
Course Interactivity
eLearning
Exam Engine
Goal Setting
Individual

Based Plans
Multimedia
Performance Assessment
8
Multimedia
Performance
Assessment
Self

Paced
Skills Assessment
Skills Tracking
Test Building
Test Scoring
Multimedia
Performance
Assessment
Self

Paced
Skills Assessment
Skills Tracking
Test Building
Test Scoring
Self

Paced
Skills Assessment
Skills Tracking
Test Building
Test Scoring
Productivity
Features
Generate Reports
Instructor Scheduling
Report Management
Work Offline
Work Online
Generate Reports
Instructor Scheduling
Report Management
Work Offline
Work
Online
Generate Reports
Instructor Scheduling
Report Management
Work Offline
Work Online
Integration
Features
Mobile Access
Third Party Access /
Guest Access
Third Party Access
integrations like Wimba
Collaboration Suite
Mobile Access
Third Party
Access /
Guest Access
Third Party CMS
integrations such as
Wimba Collaboration
Suite
Mobile Access
Third Party Access / Guest
Access
Third Party CMS integrations
like
Wimba Collaboration
Suite
Communication
Features
Discussion Forum
Live Chat
Drop box
Internal Messaging
Student Lounge
Wikis
Discussion Forum
Live Chat
Drop box
Internal Messaging
Student Lounge
Wikis
Discussion Forum
Live Chat
Drop box
Internal Messaging
Student Lounge
Wikis
9
Content
Features
Content Management
Custom Fields
Custom User
Interface
Data Import/Export
Data Management
Multiple Delivery
Formats such as
HTML, Plain text, etc.
Latex for writing and
displaying symbolic
notations
Content Management
Custom Fields
Custom
User
Interface
Data Import/Export
Data Management
Multiple
Delivery
Formats such as
HTML, Plain text,
etc.
Content Management
Custom Fields
Data Import/Export
Data Management
Multiple Delivery Formats
such as HTML, Plain text, etc.
Multilingual
Support besides
English
Language
15 languages
86 languages
10
languages
Accessibility
Compliance
AICC Compliant
IMS Compliant
SCORM Compliant
AICC Compliant
IMS Compliant
SCORM Compliant
AICC Compliant
IMS Compliant
SCORM Compliant
Support
Features
24/7
Forums
Online self

serve
Brochures
Remote Training
System
upgrades
Request Forms
24/7
Forums
Online self

serve
Brochures
Remote Training
System upgrades
Request Forms
24/7
Forums
Online self

serve
Brochures
Remote Training
System upgrades
Parent/Student co

enrollment
Request Forms
Cost
A lot
Free (open source)
A lot
Cross Platform
Compatibility
No
No
No
Table 2.1
Comparing the Features
of Three
CMSs
10
CHAPTER THREE: DATA METHODOLOGY
DATA COLLECTION
The evaluation used in this study was done by gathering data through surveys. The targeted
population is the
users of the three course managements systems: Blackboard with integrated Pearson
MyLabs., Moodle, and Desire2Learn learning environment in the United States. It is very difficult to
obtain such a huge data. Hence, there is the need to use a random sample
from these populations. Random
sampling is defined as the process of selecting individuals from a population to be in the sample. This is
necessary to eliminate any selection biases introduced during the selection process, intentionally or
unintentionally.
The goal of sampling is to obtain significant information about the population at the least
cost. The population considered in this study is residents in the United States who use or have used
Blackboard with integrated Person MyLabs., Moodle, or Desire2L
earn learning e
nvironment. Many of
them are
YouTube subscribers, several college students from numerous online colleges, and online general
public forums. Two forms of surveys were administered. The first survey was given to a much larger group
of individu
als. The second survey was given to individuals that were randomly selected using a random
number generator.
The first survey was given to 500 individuals online who use, or have used, any of the three systems. 100
individuals did not respond. 400 individ
uals responded. Among those who responded, 100 responses were
not applicable. 100 individuals indicated that they use or have used Blackboard/Pearson integrated system;
100 individuals responded that they use or have used Moodle; and 100 individuals indica
ted that they use
or have used Desire2Learn learning environment.
11
The first survey is shown in Table 3.1
Course Management Systems Survey
This survey is designed to find the number of students who use, or have used these course management
systems
namely Blackboard/Pearson system, Moodle, and Desire2Learn learning environment. Your
honest response is highly requested and appreciated. Thank you.
1.
Are you a student or have you been a student?
True
False
2.
Have you ever used a
course management system? Examples of course management
systems include Blackboard/Pearson, Moodle, Desire2Learn, Sakai, etc.?
True
False
3.
Which of the course management systems below have you used? If you have used more
than one, please
select only the first you used.
Blackboard/Pearson system
Moodle
Desire2Learn
None of the above
Table 3.1 Course Management Systems Survey
Next, a combination of stratified sampling and simple random sampling
was used
. A stratified sample is a
sample obtained by dividing the population into non

overlapping groups called strata, and then obtaining a
simple random sample from each stratum. A simple random sample is a sample of size,
n
selected from a
population of size,
N
when every possible sample of size,
n
has an equally likely c
hance of occurring. This
ensured
that the individuals within each stratum are homogenous or similar. So, the responses
were divided
into similar groups or stratum. This means that the responses
from users of Blackboard/Pearson system
were one group/stratum, the responses from Moodle users were another group/stratum, and the responses
from Desire2Learn learning environment were another group/stratum.
S
tratified sampling technique
was
chosen becau
se it allowed
the same or more information
to be obtained
by surveying fewer individuals. In
addition, because each stratum is homogenous or similar, opinions with
the stratum we
re less likely to vary
12
much from one individual to another. This technique als
o guarantees that each stratum is represented in the
sample.
Further,
the characteristics of each stratum
could be
easily
determined
. All in all,
t
he use of this
technique allows
an analysis to
be performed, in order to
compare and evaluate the means of
ea
ch stratum
to see if there were
any significant differ
ences among them which le
a
d
to the thesis.
After dividing the 300 responses into three strata of 100 individual similar responses each, a simple
random sample from each stratum
was used
. Each stratum
correlated with the responses from the users of a
particular course management system. Each 100 individuals from a strat
um were numbered and given a
three

digit numerical label corresponding to the number
.
T
he 100
th
individual
was labeled ‘
1
00’. The 54
th
individual was given the label ‘
0
54’; the 1
st
individual was given the label ‘
0
01’, and so on. Then, a simple
random sample of 30 individuals identified by thei
r labels, from each stratum was
formed and selected to
take a second survey. Microsoft Excel
wa
s used to generate the
simple random sample of 30.
T
he second
survey
, based on a
Likert scale is
shown in Table 3.2
Evaluation of Course Management Systems Survey
This survey is designed to partly assess the impact of some course management systems namely
Blackboard/Pearson system, Moodle, and Desire2Learn learning environment. You have been identified as
a user of one of these course management systems. Your honest response is highly requested and
appreciated. Thank you.
1.
Regarding the course managem
ent that I use or have used, there is easy access to the
course materials including assignments, and grades.
Strongly Disagree
Disagree
Neutral
Agree
Strongly Agree
2.
Regarding the course management that I use or have used, the
website navigation is easy
and straightforward.
Strongly Disagree
Disagree
Neutral
Agree
Strongly Agree
13
3.
Regarding the course management that I use or have used, there is/was good
communication with the instructor, and with the
students.
Strongly Disagree
Disagree
Neutral
Agree
Strongly Agree
4.
The course management system I use or have used, is/was free or open source.
Strongly Disagree
Disagree
Neutral
Agree
Strongly Agree
5.
I can
access the course management system on my laptop, desktop, iPad, Mac, and other
electronic devices with internet capabilities. It also works well across different internet
browsers.
Strongly Disagree
Disagree
Neutral
Agree
Strongly Agree
6.
The course management system I use/used is/was user friendly and aesthetically
pleasing. Course log in credentials and retrieval is/was easy. The system is/was also
secure.
Strongly Disagree
Disagree
Neutral
Agree
Strongly Agree
7.
The course management system I use or have used is/was easily accessible by students
with disabilities.
Strongly Disagree
Disagree
Neutral
Agree
Strongly Agree
14
8.
Some technical problems often occur within the course management
system. When they
occur, the downtime is/was usually four hours. In addition, early announcements are
made prior to most technical problems.
Strongly Disagree
Disagree
Neutral
Agree
Strongly Agree
9.
There is a high probability that I
would prefer to use/keep using my course management
system.
Strongly Disagree
Disagree
Neutral
Agree
Strongly Agree
10.
If I was/am to grade my course management system based on its overall effectiveness in
student learning,
communication, and success; what percent would I give?
_______________
Table 3.2 Evaluation of Course Management Systems Survey
It is best to represent the survey responses with a multiple bar graph
as can
be
seen in Chapter Four, pages
25 through
29
. This is done to clearly show the diagrammatic comparison of the responses of the different
users of the three systems. Microsoft Office Excel was used to obtain the multiple bar graphs.
Next, the descriptive statistics of the quantitative data was foun
d.
The descriptive statistics including the
sample mean, median, mode, among others can be found using
the appropriate
formulas, or by using
technology.
Next, we
find the shape of the distribution to see whether the distribution is skewed left, sym
metric
or
skewed right. T
he Minitab software
was used
for this task. This is to see the distribution clearly and the fit
as well.
Another good comparison of these three systems is the Boxplot often referred to as the box and whisker
plot. The boxplot presents a c
lear visual way of comparing data sets. In this regard, we shall use it on the
15
results we get after conducting our data analysis using One

Way Analysis of Variance (ANOVA) test. For
this thesis, we shall be using the Minitab software to draw the boxplot of
the data values of the three
systems.
HYPOTHESIS AND HYPOTHESIS TESTING
After the statistical analysis of the three systems, we can state the null hypothesis, represented by
H
0
as: H
0
: There is no significant difference among Blackboard/Pearson, Moodle,
and Desire2Learn course
management systems in ensuring effective learning of its users. The alternative hypothesis, H
1
is stated as:
H
1
: There is at least a significant difference among Blackboard/Pearson, Moodle, and Desire2Learn course
management
systems in ensuring effective learning of its users.
The sample data obtained, will be used to determine whether to reject or not to reject the null hypothesis.
However, there is always the possibility of making an incorrect decision because the informatio
n from the
sample data is not complete, as it is based on a sample, rather than a population. As such, hypothesis
testing results in four outcomes. These outcomes are:
We reject the null hypothesis when the alternative hypothesis is true. This decision wou
ld be
correct.
We do not reject the null hypothesis when the null hypothesis is true. This decision would be
correct.
We reject the null hypothesis when the null hypothesis is true. This decision would be incorrect,
resulting in a Type I error.
We do not r
eject the null hypothesis when the alternative hypothesis is true. This decision would be
incorrect, resulting in Type II error.
16
This
is shown
in Table 3.3
Outcomes of Hypothesis
Testing
Reality
H
0
is True H
1
is True
Do Not Reject H
0
Conclusion
Reject H
0
Correct Conclusion
Type II Error
Type I Error
Correct Conclusion
Table 3.3
Four Outcomes of Hypothesis Testing
The probability of making a Type I error is called
the level of significance, α (Greek letter pronounced as
“alpha”). As the probability of Type I error increases, the probability of Type II error decreases. Similarly,
as the probability of Type I error decreases, the probability of Type II error increases
.
OVERVIEW OF ANOVA (ANALYSIS OF VARIANCE)
Analysis of Variance is defined as an inferential method used to test the equality of three or more
population means. This is the reason for using ANOVA. When comparing the three systems, one way is to
conduct t
hree separate hypothesis tests. This way would result in a higher probability of making a Type I
error than the level of significance, α. Another way is to compare the population means two at a time. This
would also result in a high probability of making a
Type I than the level of significance. As the number of
populations to be compared increases, the probability of making a Type I error using multiple t

tests
(another type of test) for a given value of α also increases. Perhaps, we may try to decrease the
probability
of making a Type I error, but then, the probability of making a Type II error increases. So, conducting
multiple t tests ultimately leads to an increased chance of making a mistake no matter what level of
significance we use in the individual
tests. Hence, there is the need of ANOVA to conduct such tests.
There are several requirements to be met prior to using a One

Way ANOVA Test.
17
They are:
There are
x
simple random samples; one from each of
x
populations.
The
x
samples are independent
The
populations are normally distributed
The populations have the same variance. This requirement can also be met if the largest sample
standard deviation is no more than twice the smallest sample standard deviation.
The basic idea in a One

Way ANOVA is to det
ermine if the sample data could come from populations
with the same mean, µ or if there is evidence to suggest that at least one sample comes from a population
whose mean is different from the others. Given any data set, the variability among the sample me
ans is
called the between

sample variability. The between

sample variability is the variability due to differences
in the samples. The variability of each sample is called within

sample variability. The within

sample
variability is the variability within e
ach sample. The logic behind using ANOVA to test the equality of
means is that if the between

sample variability is large relative to the within

sample variability, then there
is evidence to suggest that the samples were taken from populations with differe
nt means. We use a test
statistic, known as ANOVA F

test Statistic denoted by F
0
to judge whether the population means might be
equal.
It is seen that the F

test statistic is a ratio of two estimates of the variance: the between

sample variability
and th
e within

sample variability.
In testing any hypothesis, the null hypothesis is assumed to be true until there is evidence to suggest
otherwise. So, in testing hypothesis of
k
population means, it is assumed that
H
0
:
µ
1
= µ
2
= µ
3
= ... = µ
k
H
a
: Not all
population means are equal
It is assumed that a simple random sample of size, n
j
has been selected from each of the
k
populations or
treatments. Let:
18
µ
k =
mean of the
k
th population
x
ij
=
x =
value of observation i for treatment
j
n
j
=
n =
number of observations for treatment
j
̅
j
=
̅
sample mean for treatment
j
s
2
j
=
s
2
=
sample variance for treatment
j
s
j
=
s =
sample standard deviation for treatment
j
̿
=
overall sample mean
nT = total number of observations = n
1
+ n
2
+ …+ n
k
The
formula for the sample mean for treatment
j
is:
̅
∑
The formula for the sample standard variance for treatment
j
is:
∑
̅
The overall sample mean is the sum of all
the observations divided by the total number of observation
s.
The formula for the overall sample mean is
:
̿
∑
∑
If the size of each sample is n, nT = kn;
̿
∑
∑
=
∑
∑
=
∑
̅
This means that whenever the sample sizes are the same, the overall sample means is the average of the
k
sample means.
Between

Treatments Estimate of Population Variance
The between

sample variability of σ
2
is also known as the mean square due to treatment (MSTR) because
any differences in the sample means could be attributed to the different levels of the sample. The MST
R
is
19
found by first computing the sum of squares due to treatment (or sample), written a
s SSTR
, and then
dividing it by the degrees of freedom associated with SSTR
.
∑
̅
̿
The SSTR is computed by determining the sum of the squared differences between each sample mean and
the overall mean, where each squar
ed difference is weighted by its sample size.
∑
̅
̿
k

1 represents the degrees of freedom
associated with SSTR.
The degree of freedom is a positive integer
that indicates the lack of restrictions in our calculations. It is the number o
f values in our calculation that
we can vary.
If
the null
hypothesis
is true,
MSTR provides an unbiased estimate of σ
2
. However, if the
alternative
hypothesis is true,
MSTR is not an unbiased estimate of σ
2
.
Within

Treatments Estimate of Population
Variance
The within

sample variability of σ
2
is also known as the mean square due to error (MSE).
It is based on the
variation within each of the treatments.
It is a ratio of the sum of squares due to error (SSE) to
the degrees
of freedom
associated with S
SE
.
∑
The SSE is the sum of all sample variances weighted by the degrees of freedom.
represents the degrees of freedom associated with SSE.
∑
The MSE
always provides an unbiased estimate
of σ
2
. It is not affected
by whether the null or alternative
hypothesis is true.
20
The computation of the F

test statistic is based on mean squares. A mean square is an average or mean of
squared values.
Therefore, the F

test statistic can also be written as
F
0
=
In general, the steps for computing the F

test statistic by hand are:
Find the sum of all the data values. Divide this sum by the sampl
e size. This gives the
overall
sample mean.
Find the sample mean of each sample.
Find the sample variance of each sample.
Determine the sum of squares due to treatments, SST; and the sum of squares due to error, SSE.
Determine the mean square due to treat
ments, MST; and the mean square due to error, MSE.
Compute the F

test statistic using the formula given above.
The ANOVA test is a right

tailed test, so the critical F

value is the F

value whose area in the right tail is α
with k

1 degrees of freedom in the numerator and n

k degrees of freedom in the denominator. A right

tailed test is a hypothesis test where the r
ejection region is located to the extreme right of the distribution.
21
C
HAPTER 4
DATA METHODOLOGY USING TECHNOLOGY
As mentioned previously in
Page 10 of
Chapter 3,
the random number generator of Microsoft Excel wa
s
used to generate these numbers
.
Figures 4.1
–
4.
4
shows the steps used to
generate random numbers
using Microsoft Office Excel.
Microsoft Excel add

in
was added to
the ‘Data Analysis Tool Pak’
and was
activated.
Click
on the ‘Data Analysis’ icon
A menu pops up as seen in Figure 4.1
Fig
ure
4
.1 Using Excel to G
enerate Random Numbers
–
1 of 4
We highlight ‘Random Number Generation’ from the list and click ‘Ok’
The Random Number Generation window menu pops up. In the:
‘Number
of Variables’ field,
we
enter the number, “1”
‘Number of
Random Numbers’
field;
we
enter the number,
“
50
”
. 30
numbers are actually needed
,
but the
number, “50” was entered
. This is done to avoid using repeated numbers as
the “without replacement”
condition of probability
is being used
. To avoid biases, it would
not be right for a label to be used twice.
22
This means that it would not be right for the same individual to fill the survey twice. In the event of using
technology to generate random numbers, it is possible to have the same random number twice or more. So,
the number,
“
50
”
wa
s entered
in order
to cancel any random number which may appear more than once.
‘Distribution’ field;
“Uniform”
was selected
because each label should have equal probability of occurring.
Uniform distribution ensures that each label ha
ve equal chances of being selected. This means that each
individual should have equal chances of participating in the survey. This is one of the conditions for
random sampling.
‘Parameters’ field; enter
“Between 1 and 100”which means 1 ≤ x ≤ 100 where x is
an individual. This is
because each stratum has 100 individuals. Any of those individuals can participate in the survey.
‘Random Seed’ field;
we
enter
“
12
”
. A random seed, also known as seed, is any nonzero number that
serves as an initial point for the random number generator to start generating random numbers.
‘Output options’ field;
we
leave the
“New Worksheet
P
ly”
as is
by default. This is in order for
the result to
be generated in a new worksheet.
This is shown in Figure 4.2
Figure
4
.2 Using Excel to G
enerate Random Numbers
–
2 of 4
23
The random numbers generated are shown in
Figure 4.3 and Figure 4.4
Figure
4
.3 Using Excel to
Generate Random Numbers
–
3
of 4
Figure 4
.4 Using Excel to
Generate Random Numbers
–
4
of 4
Usually, the numbers to the right of the decimal point are neglected. We only consider the integer values to
the left of the decimal point. Starting from the 1
st
number on the list, the surveys were given to individuals
24
with label numbers:
00
1,
0
18,
0
19,
0
88,
00
5,
0
80,
0
40,
0
91,
0
37,
0
68,
0
13,
0
87,
0
15,
00
7,
0
75,
0
83,
0
10,
0
55,
0
76,
0
82,
0
64,
0
16,
0
36,
00
9,
0
11,
0
95,
0
45,
0
98,
0
17, and
0
47. A sample size of 30 is only needed,
so the first 30 labels that were not repeated were used. The
numbers
, 91 appeared on the 8
th
and 25
th
position; 7 appeared on the 14
th
and 20
th
position; and 68 appeared on the 10
th
and 27
th
position. Those
labels we
re used only once, so the second occurrence was not used. This is to avoid bias by giving the
same individual two surveys. As mentioned earlier, the “without replacement” condition was used.
So, the survey was given to 30 individuals in each stratum, makin
g it a total of 90 individuals.
The
responses are
found
in the Appendix
B
.
DATA PRESENTATION OF THE SURVEY RESPONSES
As previously mentioned in Chapter 3, t
he steps
in representing the survey responses with a
multiple bar graph are
shown in Figures 4.5
–
4.13
Microsoft Excel application software
was started
The
Category, variables, and data values,
were entered,
and highlighted
On the menu bar,
we
select
Insert
Under the Bar icon, we
select
Clustered Bar in 3

D
. This bar compares values across categories
and
display clustered bars in 3

D format
25
The d
ata p
resentation of question 1 can be seen in Figure 4.5
Figure 4.5 Representing Survey Responses to Question 1 on a Multiple Bar Graph
The d
ata
presentation of question 2 can be seen in Figure 4.6
Figure 4.6 Representing Survey Responses to Question
2
on a Multiple Bar Graph
26
The d
ata
p
resentation of question 3 can be seen in Figure 4.7
Figure 4.7 Representing Survey Responses to Question 3 on a Multiple Bar Graph
The d
ata
presentation of
question 4 can be seen in Figure 4.8
Figure 4.8 Representing Survey Responses to Question 4 on a Multiple Bar Graph
27
The d
ata
presentation of question 5 can be seen in Figure 4.9
Figure 4.9 Representing Survey Responses to Question 5 on a Multiple Bar
Graph
The d
ata
presentation of question 6 can be seen in Figure
4.10
Figure 4.10 Representing Survey Responses to Question 6 on a Multiple Bar Graph
28
The d
ata
presentation of question 7 can be seen in Figure 4.11
Figure 4.11
Representing Survey Responses to Question
7
on a Multiple Bar Graph
The d
ata
presentation of question 8 can be seen in Figure 4.12
Figure 4.12 Representing Survey Responses to Question 8 on a Multiple Bar Graph
29
The d
ata p
resentation for question 9 ca
n be seen in Figure 4.13
Figure 4.
13
Representing Survey Responses to Question
9
on a Multiple Bar Graph
Question 10 gives a quantitative data result
. The responses are shown in Table 4.1
Overall Numeric
Results of the Survey
Blackboard/Pearson
Moodle
Desire2Learn
85
88
100
94
88
88
92
98
85
88
100
88
96
98
84
93
92
98
87
99
96
85
99
99
97
98
100
98
85
78
88
88
88
87
86
95
95
95
93
85
93
86
84
92
86
92
99
86
85
85
87
96
88
95
100
87
92
100
99
93
30
88
100
99
81
98
99
95
97
78
93
95
85
86
100
88
87
87
88
99
88
84
93
85
92
95
98
91
95
83
88
Table 4.1 Quantit
ative Results of the Sample Sur
vey
Microsoft Excel
was used
to find the descriptive statistics of the quantitative data above.
Figures 4.14
–
4.18 shows the steps used
to find the descriptive statistics using Microsoft Office Excel.
The
Microsoft Office Excel application
was started
and
the data
was entered.
T
he Excel Add

in, Data Analysis Toolpak
was added to the menu.
Then, we
highlight
our
data and click on the
Data
i
con in the menu bar
A pop

up menu appears, and we
select
Descriptive Statistics
from the list
. Th
e screen shot can be seen
in
Figure 4.14
Figure 4.14 Using Excel to Find
Descriptive Statistics
–
1 of 5
31
We
click Ok and another pop

up menu appears
In the
Input Range
field of the
Input
section,
we
select the first column of the survey results which
corresponds to the results of the Blackboard/Pearson.
We then
check the box next to
Summary Statistics
.
We
leave the rest as is
. The screen shot can be seen in
Figure 4.15
Figure 4.15 Using Excel to Find
Descriptive Statistics
–
2
of 5
We
click Ok
.
T
he summary statistics of the Blackboard/Pearson system is displayed on another worksh
eet
as can be seen in Figure 4.16
32
Figure 4.16 Using Excel to Find
Descriptive
Statistics
–
3
of 5
Using similar process, the summary stat
istics of Moodle can be seen in Figure 4.17
Figure 4.17 Using Excel to Find
Descriptive Statistics
–
4
of 5
The descriptive statistics of the Desire2Learn learni
ng environment can be seen in Fi
gure 4.18
Figure 4.18 Using Excel to Find Descriptive Statistics
–
5
of 5
33
To see the shape of th
e distribution, and to find out whether it is symmetric, skewed left, or skewed right
,
the Minitab software application
is used.
Figures 4.19
–
4.24 shows the steps in using the Minitab software
to draw the data distribution.
The survey results can be seen in Figure 4.19
Figure 4.1
9
Using Minitab
to Draw
the
Data
Distribution
–
1
of 6
Then,
we
click on the
Graph
l
ist on the menu
bar, and click
on
Histogram
. A menu bar appears and
we
click
With Fit
.
This
can be seen in Figure 4.20
Figure 4.20 Using Minitab to Draw the
Data
Distribution
–
2
of 6
34
We
click
Ok
and double click on the Blackboard/Pearson to insert it in the
Graph
variables section
The screen shot can be seen in Figure 4.21
Figure 4.21 Using Minitab to Dr
aw the Data Distribution
–
3
of 6
We
click
Ok
. T
he distribution shape of
the Blackboard/Pearson can be seen in Figure 4.22
Figure 4.22 Using Minitab to Dra
w t
he Data Distribution
–
4
of 6
35
Similarly, the distri
bution shape of Moodle can be seen in Figure 4.23
Figure 4.23 Using Minitab to Draw the Data Distribut
ion
–
5
of 6
The distribution shape of
the Desire2Learn can be seen in Figure 4.24
Figure 4.24
Using Minitab to Dr
aw the Data Distribution
–
6
of 6
Because the difference between the mean and median of these three systems are not substantial, their
distributions are roughly symmetric. This is also observed in the respective diagrams of their distrib
utions.
36
Computing the descriptive statistics especially the mean of these systems are important because much of
the inferential statistics performed are based on the mean.
As
mentioned in Chapter 3,
the boxplots of these systems
need to be drawn. Figures 4
.25
–
4.29 shows the
steps in using the Minitab software to draw the
boxplots of these systems
.
To start with the boxplot of these three systems,
We
start the Minitab software
Then,
we
enter the data values for the Blackboard/Pearson, Moodle, and Desire2Le
arn under the columns,
C1, C2, and C3 respectively
as seen in Table 4.1
We
click on the
Graph
menu and click on
Boxplot...
The screen shot can be seen in Figure 4.25
Figure 4.25 Using Minitab to Draw th
e Boxplots for the Data
–
1
of 5
Because we
need three boxplot corre
sponding to the three systems, we
selected
Multiple Y’s, simple
, and
click
Ok
.
The screen shot can be seen in Figure 4.26
37
Figure 4.26 Using Minitab to Draw th
e Boxplots for the Data
–
2
of 5
We
double

click on the three systems t
o place them under the
Graph variables
section
. The screen shot can
be seen in Figure 4.27
Figure 4.27 Using Minitab to Draw th
e Boxplots for the Data
–
3
of 5
Then,
we
click on
Scale
and check the box,
Transpose value and category scales
. This is done
so that the
boxplot would be displayed horizontally
. The screen shot can be seen in Figure 4.28
38
Figure 4.28 Using Minitab to Draw th
e Boxplots for the Data
–
4
of 5
Then,
we
click
Ok
to c
lose the Boxplot

Scale window. We
click Ok again to plot t
he
boxplot of the three
systems as can be seen in Figure 4.29
Figure 4.29 Using Minitab to Draw th
e Boxplots for the Data
–
5
of 5
The boxplot tells us that the median of the Desire2Learn learning environment is less than the median of
Blackboard/Pearson sy
stem, and that in turn is less than the median of the Moodle system.
39
Using the method explained previously,
we compute the descriptive statistic
s of the three systems. This
screen shot can be seen in Figure 4.30
Figure 4.30 Sample Mean and
Sample Varianc
e of Blackboard/Pearson System
ẋ = 91.51 (rounded to two decimal places)
All calculated values are rounded to the nearest hundredth or to two decimal places.
Next, the individual sample means are found.
Let the sample mean of the Blackboard/Pearson system
= x
1
Let the sample mean of Moodle = x
2
Let the sample mean of the Desire2Learn learning environment = x
3
40
The sample means and sample variances of the three systems can be seen in Figure 4.31
Figure 4.3
1
Sample Mean
s
and Sample Variance
s
of
the
Three CMSs
x
1
=91.30, x
2
= 92.93, x
3
=90.3
Next, the individual sample variances are found.
Let the sample mean of the Blackboard/Pearson system = s
1
2
Let the sample mean of Moodle = s
2
2
Let the sample mean of the Desire2Learn learning environment = s
3
2
s
1
2
= 28.91, s
2
2
= 33.79
, s
3
2
= 38.22
The rest of the computations namely the SST, MST, and the F
0
shall be computed in Chapter
5
using
technology specifically Microsoft Excel software.
41
CHAPTER FIVE
: DATA ANALYSIS
TESTING THE REQUIREMENTS OF ONE

WAY
ANOVA
As previously mentioned in Chapter Three, it is necessary to test the requirements of the overall
numeric quantitative/numeric results of our survey to ensure that we can analyze the results using One

Way ANOVA.
We begin with the t
able of our resul
ts as can be seen in Table 5.1
Overall Numeric
Results of the Survey
Blackboard/Pearson
Moodle
Desire2Learn
85
88
100
94
88
88
92
98
85
88
100
88
96
98
84
93
92
98
87
99
96
85
99
99
97
98
100
98
85
78
88
88
88
87
86
95
95
95
93
85
93
86
84
92
86
92
99
86
85
85
87
96
88
95
100
87
92
100
99
93
88
100
99
81
98
99
95
97
78
93
95
85
86
100
88
87
87
88
99
88
84
93
85
92
95
98
91
95
83
88
Table 5
.1 Overall Numeric Results of the Survey
Verifying these requirements gives:
42
Requirement 1:
The survey was randomly distributed to the participants, thereby minimizing bias. No one individual
completed two surveys.
Requirement 2:
The samples are independent. The probability of a response on one particular course management system
d
oes not in any way depend on the probability of a response on another course management system.
Requirement 3:
The normality requirement is also satisfied. This means that the normal probability plots of the three
systems are roughly linear.
We
use
the Min
itab statistical software to get the normal probability plots.
Figures 5.1
–
5.6
shows the steps in using the Minitab software to draw the normal probability plots. To
draw
the
normal probability plot of the
Blackboard/Pe
arson course management system, the
Minitab
software
is launched, and
raw data
is entered
in the first column, C1 and Row number 1
as can be seen in
Figure 5.1
Figure 5.1 Using Minitab to Draw the N
ormal Probability Plots
–
1
of 6
Then, from the
Graph
menu, we
select
Probability Plot
. The
screen shot can be seen in Figure 5.2
43
Figure 5.
2
Using Minitab to Draw the N
ormal Probability Plots
–
2
of 6
Then, we
select
Single
and click
Ok
.
The screen shot can be seen in Figure 5.3
Figure 5.3 Using Minitab to Draw the N
ormal Probability Plots
–
3
of 6
We
double

click on
C1
to place it in the
Graph variables
section
T
he
Distribution
is then
set to
Normal
44
Then,
we
click
Ok
The normal probability plot of the Blackboar
d/Pearson system can be seen in Figure
5.4
1
1
0
1
0
5
1
0
0
9
5
9
0
8
5
8
0
7
5
9
9
9
5
9
0
8
0
7
0
6
0
5
0
4
0
3
0
2
0
1
0
5
1
B
l
a
c
k
b
o
a
r
d
/
P
e
a
r
s
o
n
P
e
r
c
e
n
t
M
e
a
n
9
1
.
3
S
t
D
e
v
5
.
3
7
7
N
3
0
A
D
0
.
7
1
4
P

V
a
l
u
e
0
.
0
5
6
P
r
o
b
a
b
i
l
i
t
y
P
l
o
t
o
f
B
l
a
c
k
b
o
a
r
d
/
P
e
a
r
s
o
n
N
o
r
m
a
l

9
5
%
C
I
Figure 5.4
Using Minitab to Draw the Normal Pro
bability Plots
–
4 of 6
The normal probability plot of the Moodle course management system
can be seen in Figure 5.5
1
1
5
1
1
0
1
0
5
1
0
0
9
5
9
0
8
5
8
0
7
5
9
9
9
5
9
0
8
0
7
0
6
0
5
0
4
0
3
0
2
0
1
0
5
1
M
o
o
d
l
e
P
e
r
c
e
n
t
M
e
a
n
9
2
.
9
3
S
t
D
e
v
5
.
8
1
3
N
3
0
A
D
1
.
5
5
8
P

V
a
l
u
e
<
0
.
0
0
5
P
r
o
b
a
b
i
l
i
t
y
P
l
o
t
o
f
M
o
o
d
l
e
N
o
r
m
a
l

9
5
%
C
I
Figure 5.5
Using Minitab to Draw the No
rmal Probability Pl
ots
–
5 of 6
45
The normal probability plot of the Desire2Learn lear
ning environment can be seen in Figure 5.6
1
1
0
1
0
0
9
0
8
0
7
0
9
9
9
5
9
0
8
0
7
0
6
0
5
0
4
0
3
0
2
0
1
0
5
1
D
e
s
i
r
e
2
L
e
a
r
n
P
e
r
c
e
n
t
M
e
a
n
9
0
.
3
S
t
D
e
v
6
.
1
8
2
N
3
0
A
D
0
.
5
8
6
P

V
a
l
u
e
0
.
1
1
8
P
r
o
b
a
b
i
l
i
t
y
P
l
o
t
o
f
D
e
s
i
r
e
2
L
e
a
r
n
N
o
r
m
a
l

9
5
%
C
I
Figure 5.6
Using Minitab to Draw the No
rmal Probability Plots
–
6 of 6
The normal probability plots of the three
systems are roughly linear. This satisfies the normality
requirement.
Requirement 4:
The 4
th
requirement is that the populations have the same variance. This requirement can be satisfied if the
largest sample standard deviation of the variables is not mor
e than twice the smallest standard deviation.
From the normality plots of the three systems, the sample standard deviations of the Blackboard/Pearson,
Moodle, and Desire2Learn are 5.377, 5.813, and 6.182 respectively. Because the largest sample standard
de
viation, (6.182) is not more than twice the smallest sample standard deviation, (2* 5.377 = 10.754), the
requirement of equal population variances is satisfied.
Since all four requirements are satisfied, we can perform One

Way ANOVA.
46
In continuation of th
e ANOVA that was previously discussed in
Pages 17
–
20 of
Chapter 3, the ANOVA
Table for a Completely Randomized Design can be seen in Table 5.2
Source of Variation
Sum of Squares
Degrees of Freedom
Mean Square
F
p

value
Treatments
SSTR
k
–
=
N
=
䕲牯r
卓S
湔n
–
=
k
呯瑡l
卓S
湔n
–
=
N
=
=
=
=
=
呡扬攠㔮㈠b乏噁⁔=扬b潲⁃潭灬e瑥ty⁒a湤n浩ze搠䑥獩sn
=
=
䅎佖䄠B夠vA乄
=
t桥渠瑨攠䅎佖A
=
瑥獴ti猠摯湥=by=ha湤Ⱐwe=晩f獴s晩湤f瑨攠c

瑥獴t獴慴楳瑩cⰠc
0
. Then, we
compare F
0
with the critical F
Comments 0
Log in to post a comment