COMPARISON AND EVALUATION OF COURSE MANAGEMENT SYSTEMS FOR EFFECTIVE LEARNING

deliriousattackInternet and Web Development

Dec 4, 2013 (3 years and 9 months ago)

198 views

COMPARISON AND EVALUATION OF COURSE MANAGEMENT SYSTEMS FOR
EFFECTIVE LEARNING







By

Samuel Dominic Chukwuemeka








A THESIS








Submitted in partial
fulfilment

of the requirements


for the degree of Master of Science

in Computer Science

in the Graduate School of

Troy University









TROY, ALABAMA

MAY 2013





COMPARISON AND EVALUATION OF COURSE MANAGEMENT SYSTEMS FOR
EFFECTIVE LEARNING





Submitted by Samuel

Dominic Chukwuemeka

in partial
fulfillment

of the requirements

for the degree of Master of Science

in Computer Science

in the Graduate School of

Troy University




Accepted on behalf of the Faculty of the Graduate School by the thesis committee:




______
____________________________________

__________________________________

Irem Ozkarahan, Ph
.
D
.



Date

Chair


__________________________________________

Sunil R. Das, Ph
.
D.


__________________________________________

Matthew Mariano, Ph
.
D
.


_________________
__________________

_____________________________

Bill Zhong, Ph.D.




Date

Chair, Department of Computer Science


___________________________________

_____________________________

James Rinehart, Ph.D.




Date

Dean of the College of Arts and Sciences


__________________________________________

__________________________________

Dianne L. Barron, Ed.D.



Date

Associate Provost and

Dean of the Graduate School


MAY 2013

ABSTRACT

COMPARISON AND EVALUATION OF COURSE MANAGEMENT SYSTEMS FOR
EFFECTIVE LEARNIN
G


Samuel Dominic Chukwuemeka


Technology has had a global impact on almost every aspect of human lives. Of
particular importance is the impact of technology in education. Many secondary schools,
colleges, and universities now offer several courses and deg
rees online. Noteworthy to
mention is the offering of MOOCs (Massive Online Open Courses) by several
universities to the global populace. These online courses are administered using course
management systems. These systems have varying features which direc
tly or indirectly
affect the teaching and student learning experience of these courses.


Despite the numerous deliveries of online courses using these systems, the
effectiveness of online learning is still being questioned. Hence, there is the need to
eva
luate these course management systems for their effectiveness in maximizing student
learning. In this thesis, several course management systems and their features shall be
studied. Several surveys will be taken, and reports will be presented on the results

of
these surveys. Three major course management systems namely Blackboard/Pearson,
Moodle and Desire2Learn course management systems will be evaluated using
appropriate statistical techniques. An overall survey will be done using a random sample
of users
of these course management systems. One
-
Way Analysis of Variance will be
used to compare the equality of the means of these three systems. Results will be
interpreted. Possible alternatives and improvements shall be recommended.



HUMAN OR ANIMAL SUBJECTS
REVIEW


For


SAMUEL DOMINIC CHUKWUEMEKA


COMPARISON AND EVALUATION OF COURSE MANAGEMENT SYSTEMS FOR
EFFECTIVE LEARNING



This research project has been reviewed by the Research Review Board and approved as
follows:



Neither humans nor animals

will be used and this research is certified exempt from
Research Review Board review by the thesis committee chair.



Human participants

will be used and this research is certified exempt from Research
Review Board review by the thesis committee chair.



Human participants

will be used and this research was reviewed and is approved by
the Research Review Board.



Animal participants

will be used and this research was reviewed and is approved by
the Animal Research Review Board.




______________________
___________


________


Signature of Thesis Committee Chair




Date






_________________________________


________



Signature of Chair of Research Review Board


Date






























































Copyright by

SAMUEL
DOMINIC CHUKWUEMEKA

2013


DEDICATION


This thesis is dedicated to the Most Holy Trinity: God the Father, God the Son, and God
the Holy Spirit, my creator and helper.










































vi

ACKNOWLEDGMENTS



My utmost thanks go to the
Almighty God. Without Him, this thesis would have
been nonexistent. I truly appreciate my mother, brothers, sisters, nephew and nieces.
They were also influential to my completing this thesis. I am very grateful to my
supervisor, Dr. Ire
m Ozkarahan for her

help
, supervision, encouragement, and guidance.
She was always there for me, to ensure that I completed this thesis. She
i
s a supervisor,
teacher, and mentor.

I sincerely thank my thesis committee members, Dr. Sunil Das, and Dr. Matthew
Mariano. They have

been very helpful towards my success. In addition, I am grateful to
Mr. Scott Morton for all his good advice and efforts towards the success of this thesis.

I appreciate the efforts of the Department Chair, Dr. Bill Zhong; the Dean of the
College of Arts
and Sciences, Dr. James Rinehart; and the Associate Provost and Dean of
the Graduate School, Dr. Diane L. Barron. You are all contributors to my successful
graduation from our prestigious institution, Troy University.














vii




TABLE OF CONTENTS


Page

LIST OF TABLES
………………………………………………….…….……………..ix


LIST OF FIGURES ………………………………………………………….…………
..x


LIST OF SYMBOLS AND ABBREVIATIONS
………………………………………
.xiii


CHAPTER 1: INTRODUCTION ………………………………………………
.
…….....1



Overview of Learning…………………………………………………….1



Overview of Cours
e Manageme
nt Systems………………………………
2


CHAPTER 2: LITERATURE REVIEW ………………………….…………….……...
..5



History of Online Learn
ing and
Course Management Systems………….
5



Features of Course

Management Systems………………………………..6


CHAPTER 3:
DATA METHODOLOGY

................................................................…...
.11



Data Col
lection ………………………………………………………….11



H
ypothesis and Hyp
othesis Testing

…………………………………..
.
15



Overvi
ew of ANOVA……………………
………………………………
17


CHAPTER 4:

DATA METHO
DOL
OGY USING TECHNO
LOGY…………………...21



Data Pre
sentation of
the

Survey Responses
...
……………………………25





CHAPTER
5
: DATA ANALYSIS….…………………………
………………………...42



Testing the Require
me
nts of One
-
Way ANOVA……………………….42



ANO
VA
by Hand………………………………………………………..47



ANOVA b
y T
echnology………………………………………………...47



Interpreting Results……………………
………………………………
...50


CHAPTER
6
: CONCLUSION………….
…………………….…
………………………51





LIST OF REFERENCES …………………………………………………………….…
55


APPENDICES

………………………………………………………………………...57
















viii



LIST OF TABLES













Page


Table 2.1
…………………………………………………………………………………10



Comparing the Features of Three CMSs


Table 3.1…
………………………………………………………………………………12


Course Management Systems Survey


Table 3.2
…………………………………………………………………………………15


Evaluation of Course Management Systems Survey


Table 3.3

………………………………………………………………………………17


Four Outcomes of Hypothesis Testing


Table 4.1………………………………………………
……………………………..…..30

Quantitative Results of the Sample Survey


Tab
le 5
.1………………………………………………
……………………………..…..42

Overall Numeric

Results of the Survey


Table 5.2
…………
…………………………………………………………………..…..46

ANOVA Table for a Completely Randomized Design























ix



L
IST OF FIGURES


Page


Figure

1.1
…………………………………………………………………………………3


Some Course Management Systems


Figure 4
.1………
………………………………………………………………………...21

Using Excel to G
enerate
Random Numbers


1 of 4


Figure 4.2………………………………………………………………………………...
23


Using Excel to G
enerate Random Numbers


2 of 4


Figure 4.3………………………………………………………………………………...
23



Using Excel to G
enerate Random Numbers


3 of 4


Figure 4.4…………………………………………………………………………
……...
24

Using Excel to G
enerate Random Numbers


4 of 4


Figure 4.5………
………………………………………………………………………...25


Representing Survey Responses to Question 1 on a Multiple Bar Graph


Figure 4.6
………………………………………………………………………………...
26


Representing Survey Responses to Question
2

on a Multiple Bar Graph


Figure 4.
7
………………………………………………………………………………...
26


Representing Survey Re
sponses to Question 3

on a Multiple Bar Graph


Figure 4.8
………………………………………………………………………………...2
7


Representing Survey Re
sponses to Question 4

on a Multiple Bar Graph


Figure 4.
9
………………………………………………………………………………...2
7


Representing Survey Re
sponses to Question 5

on a Multiple Bar Graph


Figure 4.10
……………………………………………
..
………………………………...2
8


Representing Survey Responses to Question
6

on a Multiple Bar Graph


Figure 4.
11..
……………………………………………………………………………...
28


Representing Survey Responses to Question
7

on a Multiple Bar Graph



Figure 4.12……..
………………………………………………………………………...29


Representing Survey Responses to Question 8 on a Multiple Bar
Graph






x



Figure 4.13..


………………………………………………………………………...29


Representing Survey Re
sponses to Question 9

on a Multiple Bar Graph


Figure 4.14..……
………………………………………………………………………...31


Using Excel to Find

Descriptive Statistics


1

of 5


Figure 4.15..……
………………………………………………………………………...32


Using Excel to Find Descriptive Statistics


2

of 5


Figure 4.1
6
..……
………………………………………………………………………...32


Using Excel to Find Descriptive Statistics


3 of 5


Figure 4.17
..……
………………………………………………………………………...33


Using Excel to Find
Descriptive Statistics


4 of 5


Figure 4.18
..……
………………………………………………………………………...33


Using Excel to Find
Descriptive Statistics


5 of 5


Figure 4.19
..……
………………………………………………………………………...34


Using
Minitab to
Dra
w the Data Distribution


1

of 6


Figure 4.20..……
………………………………………………………………………...34


Using Minitab to Dra
w the Data Distribution


2

of 6


Figure 4.21..……
………………………………………………………………………...35


Using Minitab to Dra
w the Data Distribution


3

of 6


Figure
4.22..……
………………………………………………………………………...35


Using Minitab to Dra
w the Data Distribution


4

of 6


Figure 4.23..……
………………………………………………………………………...36


Using Minitab to

Draw the Data Distribution


5

of 6


Figure 4.24..……
………………………………………………………………………...36


Using Mi
nitab to Dra
w the Data Distribution


6

of 6


Figure 4.25..……
………………………………………………………………………...37


Using Minitab to Draw th
e Boxplots for the Data


1

of 5


Figure 4.26..……
………………………………………………………………………...38


Using Minitab to Draw th
e Boxplots for the Data


2

of 5


Figure 4.27..……
………………………………………………………………………...38


Using Minitab to Draw th
e Boxplots for the Data


3

of 5


xi



Figure 4.28
..……
………………………………………………………………………...39


Using Minitab to Draw th
e Boxplots for the Data


4

of 5


Figure 4.29..……
……………………………………………
…………………………...39


Using Minitab to Draw th
e Boxplots for the Data


5

of 5


Figure 4.30..……
………………………………………………………………………...40


Sample Mean and Sample Variance of Blackboard/Pearson System


Figure 4.31..……
………………………………………………………………………...41


Sample Mean and
Sample Variance of the Three CMSs


Figure 5.1....……
………………………………………………………………………...43


Using Minitab to Draw the N
ormal Probability Plots


1

of 6


Figure 5.2....………
……………………………………………………………………...44


Using Minitab to Draw the N
ormal Probability Plots


2

of 6


Figure 5.3....……
………………………………………………………………………...44


Using Minitab to Draw the N
ormal Probability Plots


3

of 6


Figure 5.4
....……
………………………………………………………………………...45


Using Minitab to Draw the N
ormal Probability Plots


4 of 6


Figure 5.5
....……
………………………………………………………………………...46


Using Minitab to Draw the Norma
l Probability Plots


5 of 6


Figure 5.6
....……
………………………………………………………………………...46


Using Minitab to Draw the N
ormal Probability Plots


6 of 6


Figure 5.8....……
………………………………………………………………………...48


Usi
ng Minitab to Run ANOVA


1

of 3


Figure 5.9....……………………………
………………………………………………...49


Usi
ng Minitab to Run ANOVA


2

of 3


Figure 5.10..……
………………………………………………………………………...49


Usi
ng Minitab to Run ANOVA


3

of 3








xii



LIST OF SYMBOLS AND ABBREVIATIONS

CD
-
ROM


Compact Disc


Read Only Memory

CMS


Course Management System

LMS


Learning Management System

VLE


Virtual Learning Environment

KMS


Knowledge Management System

ANOVA


Analysis of Variance

CALC


Computer Assisted Learning Center

HTML


Hyper

Text Markup Language

AICC


Aviation Industry Computer
-
Based Training Committee

IMS


Information Management Standard

SCORM


Shareable Content Object Reference Model

n


Size of the Sample

N


Size of the Population

H
0



Null Hypothesis

H
1



Alternative
Hypothesis

Α


α


Level of Significance



̅



Sample Mean



̿



Overall Sample Mean

s


Sample Standard Deviation

µ
-

Population Mean

σ


Population Standard Deviation

MSE


Mean
Square D
ue to Error

SSE


Sum of Squares
D
ue to Error

Σ


S
ummation

df



Degree of Freedom

MST
R



Mean Square
D
ue to Treatment

SST
R



Sum of Squares
D
ue to Treatment

WYSIWYG


What You See Is What You Get


xiii
1


CH
APTER 1: INTRODUCTION


OVERVIEW OF LEARNING



Teaching and Learning has been, is, and will continue to be an important part of
human history and education. Education seen as a critical aspect of the future of any
country, gives knowledge. Knowledge in turn, gives power.
Learning can
occur as:
Traditio
nal or Synchronous Learning, Distance Learning, Blended Learning, and Online
or Asynchronous Learning.


Traditional learning is the learning that typically takes place in the classroom
environment or brick
-
and
-
mortar schools. This learning is mainly a tea
cher
-
centered
approach, and students look at the teacher as the main source of knowledge. Assessments
and evaluations are usually done in class; the teacher grades them according to a rubric
and grading scale, and provides feedback to the students in class
. Majority of the teaching
and learning activities are done within the walls of a classroom.


Distance learning is the learning provided to students who are located at a
reasonable distance from the traditional classroom. The instructional delivery include
d an
instructor who was physically located in a different place from the learner, as well as
possibly providing the

instruction at disparate times

(Dede, 1996)
.


Blended learning is a mix of traditional learning and online learning. It combines
the learning activities done in traditional classrooms with the learning activities done
online. It is an integrated instructional strategy that is both teacher and student
centered.

Those who use blended learning environments are trying to maximize the benefits
of
both the face
-
to
-
face and online methods
[
or technology delivery methods
]


using the
Web for what it does best and using class time for what it does best” (Osgut
horpe &
2


Graham, 2033, p. 227)
.

The objective is to provide the best of both worlds


traditional and online learning
experiences. It is seen as meaning “to combine any form of instructional technology (e.g., videotape, CD
-
ROM, Web
-
based training, film) wi
th face
-
to
-
face instructor
-
led training
” (Driscoll, 2002). Blended
courses are sometimes referred to as “hybrid” courses or “mixed
-
mode” courses.


Online learning is the learning that occurs via an online or asynchronous medium. It typically
involves the u
se of computing and telecommunication devices to facilitate the acquisition of knowledge. It
is mainly a student
-
centered learning approach as the teacher is seen as a facilitator rather than a teacher or
instructor. Students acquire knowledge from many so
urces, unlike the traditional classroom where they
gain knowledge mainly from the teacher
, and books or teaching materials
. The use of technology and the
internet are usually required. Teachers and students also communicate but not face
-
to
-
face. Several st
udies
have shown that there is increased communication and collaboration among students in an online
classroom than the traditional classroom. Online learning is described by most authors as access to learning
experiences
via the use of some technology
(Be
nson, 2002; Carliner, 2004; Conrad, 2002). It is becoming
more popular locally and globally, and is often referred to as eLearning. Sometimes, online learning takes
place when both the students and the instructor are online at the same time. Most times, it

takes place
irrespective of whether the instructor and the students are online at the same time. Online learning typically
takes place via a course management system. The facilitator or instructor uses chat, video conferencing,
web conferencing, virtual w
orlds, and internet podcasts, among others to communicate learning objectives,
expectations, tasks, and evaluations to students.


OVERVIEW OF COURSE MANAGEMENT SYSTEMS


A Course Management System (CMS) is basically the platform via which an online course
is
delivered. It is sometimes referred to as a Learning Management System (LMS), Virtual Learning
Environment (VLE), or a Knowledge Management System (KMS). Though a Course management system
is sometimes referred to as a Learning management system, there i
s a key difference. Course management
3


systems are designed to support academic classroom courses while learning management systems are
designed to support corporate training.
A CMS
“provides an instructor with a
set of tools a
nd a framework
that allows the

relatively easy cre
ation of online course content
and the subs
equent teaching and
management
of that course
including various interactions
with student
s taking the course” (EDUCAUSE
Evolving Te
chnologies Committee, 2003, p.
1).

Examples of course managemen
t systems include
Blackboard Academic Suite, Pearson LearningStudio Campus, Moodle, Desire2Learn Learning
Environment, Angel Learning Management Suite, and Sakai, among others. A display of the logos of some
of the
se course managements systems can be
seen
in
Figure 1.1








Figure 1.1

Some Course Management Systems



In this thesis, Chapter One is the
Introduction
.
T
he various forms of learning; as well as course
management systems, the platforms via which online courses occur

are

briefly discussed
. Chapter Two

is
the
Literature Review. It
discusses the history of online learning, history of course management systems, as
well as the general features of course management systems. Chapter
Three

is

Data Methodology
.
This
4


explains the
sampling technique used to collect general
survey responses from students who have used, or
who use these three major course management systems


Blackboard/Pearson course management system,
Moodle course management system, and Desire2Learn course manageme
nt system. Pearson MyLabs
is
integrated with Blackboard

systems. The labs cover a wide array of academic disciplines with numerous
examples
. A lot of students have found it very useful in learning math and other subjects.
http://www.pearsonhighered.com/resources/Pearson_Global_Whitepaper.pdf
.
T
he data presentation,
explanation, and the computation of the descriptive statistics of the result
s
, are explained
.
The

concept of
One
-
Way ANOVA and its computations by Hand, and by Technology

are discussed
. In
C
hapter
F
our

-

Data Methodology Using Technology
, we shall use

the statistical software, Microsoft Excel and Minitab to
obtain the descriptive
statistics, histogram, an
d boxplots of the data we obtain

from our survey. We now
use the inferential technique, One
-
Way Analysis of Variance (ANOVA) to test the hypothesis
developed
from chapter four in Chapter F
ive



Data Analysis
.
We shall use
Microsoft

Office Excel
to

run

the
ANOVA test, and the Minitab to draw the normal probability plots of our quantitative data results.
Chapter
Six

is the

Conclusion and Recommendation
s. This
summarizes the results of
our evaluation
.
P
ossible
recommendations to the th
ree major systems, on some additions that would be useful for the overall
student learning experience
, are recommended
.










5


CHAPTER TWO: LITERATURE REVIEW

HISTORY OF ONLINE LEARNING

AND COURSE MANAGEMENT SYSTEMS


The history of online learning dates
back to the origin of the internet (formerly known as
ARPANET commissioned by the Department of Defense in 1969 (
Hobbes’ Internet Timeline


the
definitive ARPAnet and Internet history
).
Howeve
r, it was not until early 1995 that CALC Online Campus
moved on the internet as
CALCampus.com

CALCampus which means
Computer Assisted Learning
Center (CALC) was founded in 1982 in Rindge, N
ew Hampshire as a

small, offline computer
-
based adult
learning center.

The campus was actually the first to develop and implement a
fully online
-
based school, as well the course
management system, QuantumLink campus. This enabled real time communication among administrato
rs,
classroom instructors, and students.

Later in 1995,
the University of Illinois developed Mallard
and
CyberProf

web
-
ba
sed course management system (
University of Illinois
).
In the mid
-
1997, CourseInfo
founded by Dan Cane and Stephen Gilfus released the ILN


Interactive Learning Network.
The ILN was
the first eLearning system to levera
ge and install on a relational database system,
MySQL
.

Several other
course management systems were being developed and implemented during these years.


In 1997,
Blackboard
was
founded

by Michael Chasen and Matthew Pittinsky
.

That same year,
CourseInfo LLC was founded at Cornell University.

Blackboard and CourseInfo LLc later merged in 1998
and launched a free course system for faculty known as Course
S
ites.
(
Bla
ckboard | Our Story
).

Cisco
Networking Academy Management System, CNAMS was also released in 1998 to facilitate
communication and course management of Cisco Networking Academy.
In 1999, Desire2Learn was
founded in Canada.
In 2001, Moodle.com ran Moodle. S
akai project was formed from several college and
university projects and
was
released in 2004.




6


FEATURES OF COURSE MANAGEMENT SYSTEMS


Course management systems (CMSs) have become
a symbol of innovation at institutions of higher
education and in less th
an a decade, they have been rapidly adopted by a large number of colleges and
universities in many countries around the worl
d

(Coates, 2005; Dutton, Cheong, & Park, 2004;
Malikowski, Thompson, & Theis, 2007; Wise & Quealy, 2006)
.
While CMSs were initially
developed to
support distance education
and online courses, they are now used predominately to complement campus
based classroom courses (
Morgan, 2003; West et al., 2006; Wise & Quealy, 2006).
The number and
diversity of CMS features has led to their use
in offering distance learning course
s, offered over the Web
(Allen &

Seaman, 2004, 2005; Morgan 2003).
Hence, the increase in the use of CMSs is directly
proportional to the increase in the number of courses offered in online format, as well as the increas
e in the
number of users enrolled in these courses.
Enrollment in these courses has steadily increased in recent
years. In the fall of 2002, 1.6 million students enrolled in an online course. In the fall
of 2003
, enrollment
was 2 million, and in the fall o
f 2004, enrollment was 2.3 million (
Allen & Seaman, 2004, 2005).


The basic features of CMSs include the functionality of transmitting the course contents
, creating
class di
scussions, and evaluating students. Many CMSs have several other features that cove
r several other
functionalities.
Beginning from the first function, the most common feature of CMSs is the ability to share
files including submitting assignment files, post announcements to the class, post the class syllabus and
other course contents,
pos
t

grades and comments in the
grade book
,
among others.
Instructors most often
use a CMS

to transmit course content, such as syllabus, related reading, or assignment. This course content
is most often given to students
in the form of computer files uploaded

to a CMS. These typically include
word processor files, PowerPoint presentations, or HTML files

(Ansorge & Bendus, 2003; Dutton et al.,
2004; Morgan, 2003; Woods et al., 2004).
The second feature consists
of tools that enable students and/or

users to inte
ract asynchronously.

Regarding the third feature, there are several tools within the CMS for
evaluating students. Assessments include, but are not limited to assignments, discussions, and quizzes.

Quizzes can be instructor custom quizzes or generated from
a test bank. They can contain a variety of
7


question types including multiple choice, fill
-
in,
true or false,
matching
, short answer, long answer,
essay,
among others.
Majority of
these varieties

of quizzes are automatically graded and entered in the course
grade book.

Long answer, and essay assignments and quizzes are usually

submitted via the
drop box

in a
CMS.

A more detailed comparison of CMS features for the three course management systems:
B
lackboard/Pearson system, Moodle, and Desire2Learn learning environment

is shown in Table 2.1

(Taken from
http://lms.findthebest.com/
,
http://moodle.blogs.we
sleyan.edu/home/
,
http://www.elearninglearning.com/communication/desire2learn/engineering/
)


Features/CMS

Blackboard/Pearson

Moodle

Desire2Learn

Administration
Features

Administrative
Reporting


Authentication

Hosted Services

Registration
Management


User Access Controls

Administrative
Reporting


Authentication

Hosted Services

Registration
Management


User Access Controls

Administrative Reporting


Authentication

Hosted Services

Registration Management


User Access

Controls

Course
Development
Features

Blended Learning

Virtual Classroom

Course Catalog

Course Interactivity

eLearning

Exam Engine

Goal Setting

Individual
-
Based Plans




Blended Learning

Virtual Classroom

Course Catalog

Course Interactivity

eLearning

Exam Engine

Goal Setting

Individual
-
Based
Plans




Blended Learning

Virtual Classroom

Course Catalog

Course Interactivity

eLearning

Exam Engine

Goal Setting

Individual
-
Based Plans


Multimedia

Performance Assessment


8


Multimedia

Performance
Assessment


Self
-
Paced

Skills Assessment

Skills Tracking

Test Building

Test Scoring



Multimedia

Performance
Assessment


Self
-
Paced

Skills Assessment

Skills Tracking

Test Building

Test Scoring



Self
-
Paced

Skills Assessment

Skills Tracking

Test Building

Test Scoring



Productivity
Features

Generate Reports

Instructor Scheduling

Report Management

Work Offline

Work Online

Generate Reports

Instructor Scheduling

Report Management

Work Offline

Work
Online

Generate Reports

Instructor Scheduling


Report Management


Work Offline

Work Online

Integration
Features

Mobile Access

Third Party Access /
Guest Access


Third Party Access

integrations like Wimba
Collaboration Suite

Mobile Access

Third Party
Access /
Guest Access


Third Party CMS
integrations such as
Wimba Collaboration
Suite

Mobile Access

Third Party Access / Guest
Access


Third Party CMS integrations

like
Wimba Collaboration
Suite

Communication
Features

Discussion Forum

Live Chat

Drop box

Internal Messaging

Student Lounge

Wikis

Discussion Forum

Live Chat

Drop box

Internal Messaging

Student Lounge

Wikis

Discussion Forum

Live Chat

Drop box

Internal Messaging

Student Lounge

Wikis

9


Content
Features

Content Management

Custom Fields

Custom User
Interface

Data Import/Export

Data Management

Multiple Delivery
Formats such as
HTML, Plain text, etc.

Latex for writing and
displaying symbolic
notations


Content Management

Custom Fields

Custom

User

Interface


Data Import/Export

Data Management

Multiple
Delivery
Formats such as
HTML, Plain text,
etc.

Content Management


Custom Fields

Data Import/Export


Data Management

Multiple Delivery Formats
such as HTML, Plain text, etc.

Multilingual
Support besides
English
Language

15 languages

86 languages

10

languages

Accessibility
Compliance

AICC Compliant

IMS Compliant

SCORM Compliant

AICC Compliant

IMS Compliant

SCORM Compliant

AICC Compliant

IMS Compliant

SCORM Compliant

Support
Features

24/7

Forums

Online self
-
serve

Brochures

Remote Training

System
upgrades

Request Forms

24/7

Forums

Online self
-
serve

Brochures

Remote Training

System upgrades

Request Forms

24/7

Forums

Online self
-
serve

Brochures

Remote Training

System upgrades

Parent/Student co
-
enrollment

Request Forms

Cost

A lot

Free (open source)

A lot

Cross Platform
Compatibility

No

No

No


Table 2.1

Comparing the Features
of Three

CMSs

10


CHAPTER THREE: DATA METHODOLOGY

DATA COLLECTION


The evaluation used in this study was done by gathering data through surveys. The targeted
population is the
users of the three course managements systems: Blackboard with integrated Pearson
MyLabs., Moodle, and Desire2Learn learning environment in the United States. It is very difficult to
obtain such a huge data. Hence, there is the need to use a random sample
from these populations. Random
sampling is defined as the process of selecting individuals from a population to be in the sample. This is
necessary to eliminate any selection biases introduced during the selection process, intentionally or
unintentionally.

The goal of sampling is to obtain significant information about the population at the least
cost. The population considered in this study is residents in the United States who use or have used
Blackboard with integrated Person MyLabs., Moodle, or Desire2L
earn learning e
nvironment. Many of
them are
YouTube subscribers, several college students from numerous online colleges, and online general
public forums. Two forms of surveys were administered. The first survey was given to a much larger group
of individu
als. The second survey was given to individuals that were randomly selected using a random
number generator.

The first survey was given to 500 individuals online who use, or have used, any of the three systems. 100
individuals did not respond. 400 individ
uals responded. Among those who responded, 100 responses were
not applicable. 100 individuals indicated that they use or have used Blackboard/Pearson integrated system;
100 individuals responded that they use or have used Moodle; and 100 individuals indica
ted that they use
or have used Desire2Learn learning environment.






11


The first survey is shown in Table 3.1

Course Management Systems Survey


This survey is designed to find the number of students who use, or have used these course management
systems
namely Blackboard/Pearson system, Moodle, and Desire2Learn learning environment. Your
honest response is highly requested and appreciated. Thank you.



1.


Are you a student or have you been a student?




True


False


2.


Have you ever used a
course management system? Examples of course management
systems include Blackboard/Pearson, Moodle, Desire2Learn, Sakai, etc.?




True


False


3.


Which of the course management systems below have you used? If you have used more
than one, please
select only the first you used.




Blackboard/Pearson system


Moodle


Desire2Learn


None of the above




Table 3.1 Course Management Systems Survey


Next, a combination of stratified sampling and simple random sampling

was used
. A stratified sample is a
sample obtained by dividing the population into non
-
overlapping groups called strata, and then obtaining a
simple random sample from each stratum. A simple random sample is a sample of size,
n

selected from a
population of size,
N

when every possible sample of size,
n

has an equally likely c
hance of occurring. This
ensured

that the individuals within each stratum are homogenous or similar. So, the responses
were divided
into similar groups or stratum. This means that the responses

from users of Blackboard/Pearson system
were one group/stratum, the responses from Moodle users were another group/stratum, and the responses
from Desire2Learn learning environment were another group/stratum.
S
tratified sampling technique
was
chosen becau
se it allowed
the same or more information
to be obtained
by surveying fewer individuals. In
addition, because each stratum is homogenous or similar, opinions with
the stratum we
re less likely to vary
12


much from one individual to another. This technique als
o guarantees that each stratum is represented in the
sample.
Further,
the characteristics of each stratum

could be

easily

determined
. All in all,
t
he use of this
technique allows
an analysis to
be performed, in order to
compare and evaluate the means of
ea
ch stratum
to see if there were

any significant differ
ences among them which le
a
d
to the thesis.


After dividing the 300 responses into three strata of 100 individual similar responses each, a simple
random sample from each stratum

was used
. Each stratum

correlated with the responses from the users of a
particular course management system. Each 100 individuals from a strat
um were numbered and given a
three
-
digit numerical label corresponding to the number
.

T
he 100
th

individual
was labeled ‘
1
00’. The 54
th

individual was given the label ‘
0
54’; the 1
st

individual was given the label ‘
0
01’, and so on. Then, a simple
random sample of 30 individuals identified by thei
r labels, from each stratum was

formed and selected to
take a second survey. Microsoft Excel
wa
s used to generate the

simple random sample of 30.
T
he second
survey
, based on a

Likert scale is

shown in Table 3.2

Evaluation of Course Management Systems Survey


This survey is designed to partly assess the impact of some course management systems namely

Blackboard/Pearson system, Moodle, and Desire2Learn learning environment. You have been identified as
a user of one of these course management systems. Your honest response is highly requested and
appreciated. Thank you.


1.


Regarding the course managem
ent that I use or have used, there is easy access to the
course materials including assignments, and grades.




Strongly Disagree


Disagree


Neutral


Agree


Strongly Agree


2.


Regarding the course management that I use or have used, the
website navigation is easy
and straightforward.




Strongly Disagree


Disagree


Neutral


Agree


Strongly Agree


13


3.


Regarding the course management that I use or have used, there is/was good
communication with the instructor, and with the
students.




Strongly Disagree


Disagree


Neutral


Agree


Strongly Agree


4.


The course management system I use or have used, is/was free or open source.




Strongly Disagree


Disagree


Neutral


Agree


Strongly Agree


5.


I can
access the course management system on my laptop, desktop, iPad, Mac, and other
electronic devices with internet capabilities. It also works well across different internet
browsers.




Strongly Disagree


Disagree


Neutral


Agree


Strongly Agree


6.


The course management system I use/used is/was user friendly and aesthetically
pleasing. Course log in credentials and retrieval is/was easy. The system is/was also
secure.




Strongly Disagree


Disagree


Neutral


Agree


Strongly Agree


7.


The course management system I use or have used is/was easily accessible by students
with disabilities.




Strongly Disagree


Disagree


Neutral


Agree


Strongly Agree


14


8.


Some technical problems often occur within the course management
system. When they
occur, the downtime is/was usually four hours. In addition, early announcements are
made prior to most technical problems.




Strongly Disagree


Disagree


Neutral


Agree


Strongly Agree


9.


There is a high probability that I

would prefer to use/keep using my course management
system.




Strongly Disagree


Disagree


Neutral


Agree


Strongly Agree


10.


If I was/am to grade my course management system based on its overall effectiveness in
student learning,
communication, and success; what percent would I give?


_______________




Table 3.2 Evaluation of Course Management Systems Survey


It is best to represent the survey responses with a multiple bar graph

as can

be

seen in Chapter Four, pages
25 through
29
. This is done to clearly show the diagrammatic comparison of the responses of the different
users of the three systems. Microsoft Office Excel was used to obtain the multiple bar graphs.

Next, the descriptive statistics of the quantitative data was foun
d.
The descriptive statistics including the
sample mean, median, mode, among others can be found using
the appropriate
formulas, or by using
technology.

Next, we

find the shape of the distribution to see whether the distribution is skewed left, sym
metric
or
skewed right. T
he Minitab software
was used
for this task. This is to see the distribution clearly and the fit
as well.

Another good comparison of these three systems is the Boxplot often referred to as the box and whisker
plot. The boxplot presents a c
lear visual way of comparing data sets. In this regard, we shall use it on the
15


results we get after conducting our data analysis using One
-
Way Analysis of Variance (ANOVA) test. For
this thesis, we shall be using the Minitab software to draw the boxplot of

the data values of the three
systems.


HYPOTHESIS AND HYPOTHESIS TESTING


After the statistical analysis of the three systems, we can state the null hypothesis, represented by
H
0

as: H
0
: There is no significant difference among Blackboard/Pearson, Moodle,

and Desire2Learn course
management systems in ensuring effective learning of its users. The alternative hypothesis, H
1

is stated as:
H
1
: There is at least a significant difference among Blackboard/Pearson, Moodle, and Desire2Learn course
management
systems in ensuring effective learning of its users.

The sample data obtained, will be used to determine whether to reject or not to reject the null hypothesis.
However, there is always the possibility of making an incorrect decision because the informatio
n from the
sample data is not complete, as it is based on a sample, rather than a population. As such, hypothesis
testing results in four outcomes. These outcomes are:



We reject the null hypothesis when the alternative hypothesis is true. This decision wou
ld be
correct.



We do not reject the null hypothesis when the null hypothesis is true. This decision would be
correct.



We reject the null hypothesis when the null hypothesis is true. This decision would be incorrect,
resulting in a Type I error.



We do not r
eject the null hypothesis when the alternative hypothesis is true. This decision would be
incorrect, resulting in Type II error.





16


This
is shown

in Table 3.3

Outcomes of Hypothesis
Testing

Reality


H
0

is True H
1

is True


Do Not Reject H
0

Conclusion


Reject H
0


Correct Conclusion

Type II Error

Type I Error

Correct Conclusion


Table 3.3

Four Outcomes of Hypothesis Testing

The probability of making a Type I error is called
the level of significance, α (Greek letter pronounced as
“alpha”). As the probability of Type I error increases, the probability of Type II error decreases. Similarly,
as the probability of Type I error decreases, the probability of Type II error increases
.


OVERVIEW OF ANOVA (ANALYSIS OF VARIANCE)


Analysis of Variance is defined as an inferential method used to test the equality of three or more
population means. This is the reason for using ANOVA. When comparing the three systems, one way is to
conduct t
hree separate hypothesis tests. This way would result in a higher probability of making a Type I
error than the level of significance, α. Another way is to compare the population means two at a time. This
would also result in a high probability of making a

Type I than the level of significance. As the number of
populations to be compared increases, the probability of making a Type I error using multiple t
-
tests
(another type of test) for a given value of α also increases. Perhaps, we may try to decrease the

probability
of making a Type I error, but then, the probability of making a Type II error increases. So, conducting
multiple t tests ultimately leads to an increased chance of making a mistake no matter what level of
significance we use in the individual
tests. Hence, there is the need of ANOVA to conduct such tests.

There are several requirements to be met prior to using a One
-
Way ANOVA Test.


17


They are:



There are
x

simple random samples; one from each of
x

populations.



The
x

samples are independent



The
populations are normally distributed



The populations have the same variance. This requirement can also be met if the largest sample
standard deviation is no more than twice the smallest sample standard deviation.

The basic idea in a One
-
Way ANOVA is to det
ermine if the sample data could come from populations
with the same mean, µ or if there is evidence to suggest that at least one sample comes from a population
whose mean is different from the others. Given any data set, the variability among the sample me
ans is
called the between
-
sample variability. The between
-
sample variability is the variability due to differences
in the samples. The variability of each sample is called within
-
sample variability. The within
-
sample
variability is the variability within e
ach sample. The logic behind using ANOVA to test the equality of
means is that if the between
-
sample variability is large relative to the within
-
sample variability, then there
is evidence to suggest that the samples were taken from populations with differe
nt means. We use a test
statistic, known as ANOVA F
-
test Statistic denoted by F
0

to judge whether the population means might be
equal.


It is seen that the F
-
test statistic is a ratio of two estimates of the variance: the between
-
sample variability
and th
e within
-
sample variability.

In testing any hypothesis, the null hypothesis is assumed to be true until there is evidence to suggest
otherwise. So, in testing hypothesis of
k

population means, it is assumed that

H
0
:
µ
1

= µ
2

= µ
3

= ... = µ
k

H
a
: Not all
population means are equal

It is assumed that a simple random sample of size, n
j

has been selected from each of the
k

populations or
treatments. Let:

18


µ
k =
mean of the
k
th population

x
ij

=
x =
value of observation i for treatment
j

n
j

=
n =
number of observations for treatment
j


̅
j

=


̅


sample mean for treatment
j

s
2
j
=
s
2

=
sample variance for treatment
j

s
j

=
s =
sample standard deviation for treatment
j


̿

=

overall sample mean

nT = total number of observations = n
1

+ n
2

+ …+ n
k

The
formula for the sample mean for treatment

j

is:


̅











The formula for the sample standard variance for treatment

j

is:









̅











The overall sample mean is the sum of all
the observations divided by the total number of observation
s.
The formula for the overall sample mean is
:


̿















If the size of each sample is n, nT = kn;


̿














=















=


̅







This means that whenever the sample sizes are the same, the overall sample means is the average of the
k

sample means.


Between
-
Treatments Estimate of Population Variance

The between
-
sample variability of σ
2

is also known as the mean square due to treatment (MSTR) because
any differences in the sample means could be attributed to the different levels of the sample. The MST
R

is
19


found by first computing the sum of squares due to treatment (or sample), written a
s SSTR
, and then
dividing it by the degrees of freedom associated with SSTR
.







̅


̿
















The SSTR is computed by determining the sum of the squared differences between each sample mean and
the overall mean, where each squar
ed difference is weighted by its sample size.







̅


̿








k
-
1 represents the degrees of freedom
associated with SSTR.

The degree of freedom is a positive integer
that indicates the lack of restrictions in our calculations. It is the number o
f values in our calculation that
we can vary.

If
the null
hypothesis

is true,
MSTR provides an unbiased estimate of σ
2
. However, if the
alternative
hypothesis is true,

MSTR is not an unbiased estimate of σ
2
.


Within
-
Treatments Estimate of Population
Variance

The within
-
sample variability of σ
2

is also known as the mean square due to error (MSE).
It is based on the
variation within each of the treatments.
It is a ratio of the sum of squares due to error (SSE) to
the degrees
of freedom
associated with S
SE
.
























The SSE is the sum of all sample variances weighted by the degrees of freedom.





represents the degrees of freedom associated with SSE.
















The MSE
always provides an unbiased estimate
of σ
2
. It is not affected
by whether the null or alternative
hypothesis is true.

20


The computation of the F
-
test statistic is based on mean squares. A mean square is an average or mean of
squared values.

Therefore, the F
-
test statistic can also be written as


F
0
=






















In general, the steps for computing the F
-
test statistic by hand are:



Find the sum of all the data values. Divide this sum by the sampl
e size. This gives the
overall
sample mean.



Find the sample mean of each sample.



Find the sample variance of each sample.



Determine the sum of squares due to treatments, SST; and the sum of squares due to error, SSE.



Determine the mean square due to treat
ments, MST; and the mean square due to error, MSE.



Compute the F
-
test statistic using the formula given above.


The ANOVA test is a right
-
tailed test, so the critical F
-
value is the F
-
value whose area in the right tail is α
with k
-
1 degrees of freedom in the numerator and n
-
k degrees of freedom in the denominator. A right
-
tailed test is a hypothesis test where the r
ejection region is located to the extreme right of the distribution.










21


C
HAPTER 4

DATA METHODOLOGY USING TECHNOLOGY

As mentioned previously in
Page 10 of
Chapter 3,

the random number generator of Microsoft Excel wa
s
used to generate these numbers
.

Figures 4.1



4.
4

shows the steps used to
generate random numbers

using Microsoft Office Excel.

Microsoft Excel add
-
in

was added to
the ‘Data Analysis Tool Pak’
and was

activated.

Click

on the ‘Data Analysis’ icon

A menu pops up as seen in Figure 4.1


Fig
ure
4
.1 Using Excel to G
enerate Random Numbers


1 of 4

We highlight ‘Random Number Generation’ from the list and click ‘Ok’

The Random Number Generation window menu pops up. In the:

‘Number
of Variables’ field,
we
enter the number, “1”

‘Number of
Random Numbers’
field;

we
enter the number,


50

. 30

numbers are actually needed
,
but the
number, “50” was entered
. This is done to avoid using repeated numbers as
the “without replacement”
condition of probability

is being used
. To avoid biases, it would
not be right for a label to be used twice.
22


This means that it would not be right for the same individual to fill the survey twice. In the event of using
technology to generate random numbers, it is possible to have the same random number twice or more. So,

the number,


50


wa
s entered
in order
to cancel any random number which may appear more than once.

‘Distribution’ field;

“Uniform”
was selected
because each label should have equal probability of occurring.
Uniform distribution ensures that each label ha
ve equal chances of being selected. This means that each
individual should have equal chances of participating in the survey. This is one of the conditions for
random sampling.

‘Parameters’ field; enter

“Between 1 and 100”which means 1 ≤ x ≤ 100 where x is

an individual. This is
because each stratum has 100 individuals. Any of those individuals can participate in the survey.

‘Random Seed’ field;

we
enter

12

. A random seed, also known as seed, is any nonzero number that
serves as an initial point for the random number generator to start generating random numbers.

‘Output options’ field;

we
leave the

“New Worksheet
P
ly”
as is
by default. This is in order for
the result to
be generated in a new worksheet.

This is shown in Figure 4.2


Figure
4
.2 Using Excel to G
enerate Random Numbers


2 of 4

23


The random numbers generated are shown in

Figure 4.3 and Figure 4.4


Figure
4
.3 Using Excel to
Generate Random Numbers


3

of 4


Figure 4
.4 Using Excel to
Generate Random Numbers


4

of 4


Usually, the numbers to the right of the decimal point are neglected. We only consider the integer values to
the left of the decimal point. Starting from the 1
st

number on the list, the surveys were given to individuals
24


with label numbers:
00
1,
0
18,
0
19,
0
88,
00
5,
0
80,
0
40,
0
91,
0
37,
0
68,
0
13,
0
87,
0
15,
00
7,
0
75,
0
83,
0
10,
0
55,
0
76,
0
82,
0
64,
0
16,
0
36,
00
9,
0
11,
0
95,
0
45,
0
98,
0
17, and
0
47. A sample size of 30 is only needed,
so the first 30 labels that were not repeated were used. The
numbers
, 91 appeared on the 8
th

and 25
th

position; 7 appeared on the 14
th

and 20
th

position; and 68 appeared on the 10
th

and 27
th

position. Those
labels we
re used only once, so the second occurrence was not used. This is to avoid bias by giving the
same individual two surveys. As mentioned earlier, the “without replacement” condition was used.

So, the survey was given to 30 individuals in each stratum, makin
g it a total of 90 individuals.

The
responses are

found

in the Appendix

B
.


DATA PRESENTATION OF THE SURVEY RESPONSES


As previously mentioned in Chapter 3, t
he steps
in representing the survey responses with a
multiple bar graph are
shown in Figures 4.5


4.13

Microsoft Excel application software

was started

The
Category, variables, and data values,
were entered,
and highlighted

On the menu bar,
we

select
Insert

Under the Bar icon, we

select
Clustered Bar in 3
-
D
. This bar compares values across categories

and
display clustered bars in 3
-
D format









25


The d
ata p
resentation of question 1 can be seen in Figure 4.5


Figure 4.5 Representing Survey Responses to Question 1 on a Multiple Bar Graph


The d
ata
presentation of question 2 can be seen in Figure 4.6


Figure 4.6 Representing Survey Responses to Question
2

on a Multiple Bar Graph



26


The d
ata
p
resentation of question 3 can be seen in Figure 4.7


Figure 4.7 Representing Survey Responses to Question 3 on a Multiple Bar Graph


The d
ata
presentation of
question 4 can be seen in Figure 4.8


Figure 4.8 Representing Survey Responses to Question 4 on a Multiple Bar Graph



27


The d
ata
presentation of question 5 can be seen in Figure 4.9


Figure 4.9 Representing Survey Responses to Question 5 on a Multiple Bar

Graph


The d
ata
presentation of question 6 can be seen in Figure
4.10


Figure 4.10 Representing Survey Responses to Question 6 on a Multiple Bar Graph



28


The d
ata
presentation of question 7 can be seen in Figure 4.11


Figure 4.11

Representing Survey Responses to Question
7

on a Multiple Bar Graph



The d
ata
presentation of question 8 can be seen in Figure 4.12


Figure 4.12 Representing Survey Responses to Question 8 on a Multiple Bar Graph


29


The d
ata p
resentation for question 9 ca
n be seen in Figure 4.13


Figure 4.
13

Representing Survey Responses to Question
9

on a Multiple Bar Graph


Question 10 gives a quantitative data result
. The responses are shown in Table 4.1

Overall Numeric

Results of the Survey

Blackboard/Pearson

Moodle

Desire2Learn




85

88

100

94

88

88

92

98

85

88

100

88

96

98

84

93

92

98

87

99

96

85

99

99

97

98

100

98

85

78

88

88

88

87

86

95

95

95

93

85

93

86

84

92

86

92

99

86

85

85

87

96

88

95

100

87

92

100

99

93

30


88

100

99

81

98

99

95

97

78

93

95

85

86

100

88

87

87

88

99

88

84

93

85

92

95

98

91

95

83

88


Table 4.1 Quantit
ative Results of the Sample Sur
vey

Microsoft Excel
was used
to find the descriptive statistics of the quantitative data above.
Figures 4.14


4.18 shows the steps used
to find the descriptive statistics using Microsoft Office Excel.

The
Microsoft Office Excel application

was started

and
the data
was entered.

T
he Excel Add
-
in, Data Analysis Toolpak

was added to the menu.

Then, we

highlight

our

data and click on the
Data

i
con in the menu bar

A pop
-
up menu appears, and we

select
Descriptive Statistics

from the list
. Th
e screen shot can be seen

in
Figure 4.14


Figure 4.14 Using Excel to Find
Descriptive Statistics


1 of 5


31


We
click Ok and another pop
-
up menu appears

In the
Input Range

field of the
Input

section,
we

select the first column of the survey results which
corresponds to the results of the Blackboard/Pearson.
We then

check the box next to
Summary Statistics
.
We
leave the rest as is
. The screen shot can be seen in
Figure 4.15


Figure 4.15 Using Excel to Find

Descriptive Statistics


2

of 5

We
click Ok
.

T
he summary statistics of the Blackboard/Pearson system is displayed on another worksh
eet
as can be seen in Figure 4.16


32


Figure 4.16 Using Excel to Find
Descriptive

Statistics


3

of 5

Using similar process, the summary stat
istics of Moodle can be seen in Figure 4.17


Figure 4.17 Using Excel to Find

Descriptive Statistics


4

of 5


The descriptive statistics of the Desire2Learn learni
ng environment can be seen in Fi
gure 4.18


Figure 4.18 Using Excel to Find Descriptive Statistics


5

of 5

33


To see the shape of th
e distribution, and to find out whether it is symmetric, skewed left, or skewed right
,
the Minitab software application

is used.
Figures 4.19


4.24 shows the steps in using the Minitab software
to draw the data distribution.
The survey results can be seen in Figure 4.19


Figure 4.1
9

Using Minitab

to Draw
the

Data
Distribution


1

of 6

Then,
we

click on the
Graph

l
ist on the menu
bar, and click

on
Histogram
. A menu bar appears and
we
click

With Fit
.
This
can be seen in Figure 4.20


Figure 4.20 Using Minitab to Draw the
Data
Distribution


2

of 6

34


We
click
Ok

and double click on the Blackboard/Pearson to insert it in the
Graph

variables section

The screen shot can be seen in Figure 4.21


Figure 4.21 Using Minitab to Dr
aw the Data Distribution


3

of 6


We
click
Ok
. T
he distribution shape of
the Blackboard/Pearson can be seen in Figure 4.22


Figure 4.22 Using Minitab to Dra
w t
he Data Distribution


4

of 6


35


Similarly, the distri
bution shape of Moodle can be seen in Figure 4.23


Figure 4.23 Using Minitab to Draw the Data Distribut
ion


5

of 6


The distribution shape of
the Desire2Learn can be seen in Figure 4.24


Figure 4.24
Using Minitab to Dr
aw the Data Distribution


6

of 6

Because the difference between the mean and median of these three systems are not substantial, their
distributions are roughly symmetric. This is also observed in the respective diagrams of their distrib
utions.
36


Computing the descriptive statistics especially the mean of these systems are important because much of
the inferential statistics performed are based on the mean.

As
mentioned in Chapter 3,
the boxplots of these systems

need to be drawn. Figures 4
.25


4.29 shows the
steps in using the Minitab software to draw the
boxplots of these systems
.

To start with the boxplot of these three systems,

We
start the Minitab software

Then,
we

enter the data values for the Blackboard/Pearson, Moodle, and Desire2Le
arn under the columns,
C1, C2, and C3 respectively

as seen in Table 4.1

We
click on the
Graph
menu and click on
Boxplot...
The screen shot can be seen in Figure 4.25


Figure 4.25 Using Minitab to Draw th
e Boxplots for the Data


1

of 5

Because we

need three boxplot corre
sponding to the three systems, we

selected
Multiple Y’s, simple
, and
click
Ok
.
The screen shot can be seen in Figure 4.26

37



Figure 4.26 Using Minitab to Draw th
e Boxplots for the Data


2

of 5

We
double
-
click on the three systems t
o place them under the
Graph variables

section
. The screen shot can
be seen in Figure 4.27


Figure 4.27 Using Minitab to Draw th
e Boxplots for the Data


3

of 5


Then,
we

click on
Scale

and check the box,
Transpose value and category scales
. This is done
so that the
boxplot would be displayed horizontally
. The screen shot can be seen in Figure 4.28

38



Figure 4.28 Using Minitab to Draw th
e Boxplots for the Data


4

of 5

Then,
we

click
Ok

to c
lose the Boxplot
-
Scale window. We
click Ok again to plot t
he
boxplot of the three
systems as can be seen in Figure 4.29


Figure 4.29 Using Minitab to Draw th
e Boxplots for the Data


5

of 5

The boxplot tells us that the median of the Desire2Learn learning environment is less than the median of
Blackboard/Pearson sy
stem, and that in turn is less than the median of the Moodle system.

39


Using the method explained previously,

we compute the descriptive statistic
s of the three systems. This
screen shot can be seen in Figure 4.30


Figure 4.30 Sample Mean and
Sample Varianc
e of Blackboard/Pearson System


ẋ = 91.51 (rounded to two decimal places)

All calculated values are rounded to the nearest hundredth or to two decimal places.

Next, the individual sample means are found.

Let the sample mean of the Blackboard/Pearson system

= x
1

Let the sample mean of Moodle = x
2

Let the sample mean of the Desire2Learn learning environment = x
3







40



The sample means and sample variances of the three systems can be seen in Figure 4.31


Figure 4.3
1

Sample Mean
s

and Sample Variance
s

of
the

Three CMSs


x
1
=91.30, x
2

= 92.93, x
3
=90.3

Next, the individual sample variances are found.

Let the sample mean of the Blackboard/Pearson system = s
1
2

Let the sample mean of Moodle = s
2
2

Let the sample mean of the Desire2Learn learning environment = s
3
2



s
1
2

= 28.91, s
2
2

= 33.79
, s
3
2

= 38.22

The rest of the computations namely the SST, MST, and the F
0

shall be computed in Chapter
5

using
technology specifically Microsoft Excel software.






41


CHAPTER FIVE
: DATA ANALYSIS

TESTING THE REQUIREMENTS OF ONE
-
WAY
ANOVA


As previously mentioned in Chapter Three, it is necessary to test the requirements of the overall
numeric quantitative/numeric results of our survey to ensure that we can analyze the results using One
-
Way ANOVA.

We begin with the t
able of our resul
ts as can be seen in Table 5.1

Overall Numeric

Results of the Survey

Blackboard/Pearson

Moodle

Desire2Learn




85

88

100

94

88

88

92

98

85

88

100

88

96

98

84

93

92

98

87

99

96

85

99

99

97

98

100

98

85

78

88

88

88

87

86

95

95

95

93

85

93

86

84

92

86

92

99

86

85

85

87

96

88

95

100

87

92

100

99

93

88

100

99

81

98

99

95

97

78

93

95

85

86

100

88

87

87

88

99

88

84

93

85

92

95

98

91

95

83

88

Table 5
.1 Overall Numeric Results of the Survey

Verifying these requirements gives:

42


Requirement 1:

The survey was randomly distributed to the participants, thereby minimizing bias. No one individual
completed two surveys.

Requirement 2:

The samples are independent. The probability of a response on one particular course management system
d
oes not in any way depend on the probability of a response on another course management system.

Requirement 3:

The normality requirement is also satisfied. This means that the normal probability plots of the three
systems are roughly linear.
We

use

the Min
itab statistical software to get the normal probability plots.
Figures 5.1


5.6

shows the steps in using the Minitab software to draw the normal probability plots. To
draw
the
normal probability plot of the
Blackboard/Pe
arson course management system, the

Minitab
software
is launched, and
raw data
is entered
in the first column, C1 and Row number 1

as can be seen in
Figure 5.1


Figure 5.1 Using Minitab to Draw the N
ormal Probability Plots


1

of 6

Then, from the
Graph

menu, we

select
Probability Plot
. The

screen shot can be seen in Figure 5.2

43



Figure 5.
2

Using Minitab to Draw the N
ormal Probability Plots


2

of 6


Then, we

select
Single

and click
Ok
.

The screen shot can be seen in Figure 5.3


Figure 5.3 Using Minitab to Draw the N
ormal Probability Plots


3

of 6


We
double
-
click on
C1

to place it in the
Graph variables

section

T
he
Distribution

is then
set to
Normal

44


Then,
we

click
Ok

The normal probability plot of the Blackboar
d/Pearson system can be seen in Figure

5.4

1
1
0
1
0
5
1
0
0
9
5
9
0
8
5
8
0
7
5
9
9
9
5
9
0
8
0
7
0
6
0
5
0
4
0
3
0
2
0
1
0
5
1
B
l
a
c
k
b
o
a
r
d
/
P
e
a
r
s
o
n
P
e
r
c
e
n
t
M
e
a
n
9
1
.
3
S
t
D
e
v
5
.
3
7
7
N
3
0
A
D
0
.
7
1
4
P
-
V
a
l
u
e
0
.
0
5
6
P
r
o
b
a
b
i
l
i
t
y

P
l
o
t

o
f

B
l
a
c
k
b
o
a
r
d
/
P
e
a
r
s
o
n
N
o
r
m
a
l

-

9
5
%

C
I

Figure 5.4

Using Minitab to Draw the Normal Pro
bability Plots


4 of 6


The normal probability plot of the Moodle course management system
can be seen in Figure 5.5

1
1
5
1
1
0
1
0
5
1
0
0
9
5
9
0
8
5
8
0
7
5
9
9
9
5
9
0
8
0
7
0
6
0
5
0
4
0
3
0
2
0
1
0
5
1
M
o
o
d
l
e
P
e
r
c
e
n
t
M
e
a
n
9
2
.
9
3
S
t
D
e
v
5
.
8
1
3
N
3
0
A
D
1
.
5
5
8
P
-
V
a
l
u
e
<
0
.
0
0
5
P
r
o
b
a
b
i
l
i
t
y

P
l
o
t

o
f

M
o
o
d
l
e
N
o
r
m
a
l

-

9
5
%

C
I

Figure 5.5

Using Minitab to Draw the No
rmal Probability Pl
ots


5 of 6


45


The normal probability plot of the Desire2Learn lear
ning environment can be seen in Figure 5.6

1
1
0
1
0
0
9
0
8
0
7
0
9
9
9
5
9
0
8
0
7
0
6
0
5
0
4
0
3
0
2
0
1
0
5
1
D
e
s
i
r
e
2
L
e
a
r
n
P
e
r
c
e
n
t
M
e
a
n
9
0
.
3
S
t
D
e
v
6
.
1
8
2
N
3
0
A
D
0
.
5
8
6
P
-
V
a
l
u
e
0
.
1
1
8
P
r
o
b
a
b
i
l
i
t
y

P
l
o
t

o
f

D
e
s
i
r
e
2
L
e
a
r
n
N
o
r
m
a
l

-

9
5
%

C
I

Figure 5.6

Using Minitab to Draw the No
rmal Probability Plots


6 of 6


The normal probability plots of the three
systems are roughly linear. This satisfies the normality
requirement.


Requirement 4:

The 4
th

requirement is that the populations have the same variance. This requirement can be satisfied if the
largest sample standard deviation of the variables is not mor
e than twice the smallest standard deviation.
From the normality plots of the three systems, the sample standard deviations of the Blackboard/Pearson,
Moodle, and Desire2Learn are 5.377, 5.813, and 6.182 respectively. Because the largest sample standard
de
viation, (6.182) is not more than twice the smallest sample standard deviation, (2* 5.377 = 10.754), the
requirement of equal population variances is satisfied.

Since all four requirements are satisfied, we can perform One
-
Way ANOVA.

46


In continuation of th
e ANOVA that was previously discussed in
Pages 17


20 of
Chapter 3, the ANOVA
Table for a Completely Randomized Design can be seen in Table 5.2


Source of Variation

Sum of Squares

Degrees of Freedom

Mean Square

F

p
-
value

Treatments

SSTR

k


=
N
=











䕲牯r

卓S

湔n

=
k










呯瑡l

卓S

湔n

=
N
=
=
=
=
=
呡扬攠㔮㈠b乏噁⁔=扬b⁦潲⁡⁃潭灬e瑥ty⁒a湤n浩ze搠䑥獩sn
=
=
䅎佖䄠B夠vA乄
=
t桥渠瑨攠䅎佖A
=
瑥獴ti猠摯湥=by=ha湤Ⱐwe=晩f獴s晩湤f瑨攠c
-
瑥獴t獴慴楳瑩cⰠc
0
. Then, we
compare F
0

with the critical F