Learning Content Management Systems

cameramanfrictionInternet και Εφαρμογές Web

8 Δεκ 2013 (πριν από 3 χρόνια και 11 μήνες)

798 εμφανίσεις




Learning Content
Management Systems
Comparative Analysis of Systems Used to
Construct, Organize and Reuse “Learning Objects”
By Bryan Chapman and Brandon Hall, Ph.D.

Web site: www.brandon-hall.com

E-mail: brandon@brandon-hall.com

690 W. Fremont Ave., Suite 10
Sunnyvale, CA 94087
Tel.: 408.736.2335






Table of Contents

Part I – Introduction..............................................................................1
Why This Report?......................................................................................2
Research Process.......................................................................................2
How to Use This Report..............................................................................4
Part II – Definitions................................................................................7
Authoring Tool...........................................................................................8
Learning Infrastructure...............................................................................8
Learning Object.........................................................................................9
Knowledge Management.............................................................................9
CMS – Traditional Content Management System.........................................11
LMS – Learning Management System.........................................................11
LCMS – Learning Content Management System..........................................13
Part III — Anatomy of an LCMS............................................................15
Part IV — Comparison Points: Primary Functionality...........................25
At-a-Glance Grid Information....................................................................26
Critiques..................................................................................................31
Part V — LCMS Product Offerings.........................................................33
Adaptive Tutoring System.........................................................................36
Aspen Content Development Server...........................................................48
Aspen Learning Experience Server.............................................................64
Centra Knowledge Server.........................................................................77
Docent Outliner/Content Delivery Server....................................................95
ePath Learning......................................................................................113
Evolution...............................................................................................129
f(2).......................................................................................................149
iAuthor..................................................................................................164
ibtraining.com Adaptive Learning Framework...........................................179
Intellinex LEAP Learning Development System.........................................191
iperformance.........................................................................................208
IPRESS/KBRIDGE...................................................................................224
Jupiter..................................................................................................236
Knowledge Mechanics Studio..................................................................252
Knowledge Pathways..............................................................................266
Knowledge Producer..............................................................................284
Knowledgelinx 2000...............................................................................295
KnowledgePlanet Content.......................................................................308
LeadingWay KnowledgeOne Content Manager..........................................322



Lightspeed OmniSite..............................................................................334
LogicBuilder 3.0.....................................................................................351
nogginware...........................................................................................367
SmartBuilder..........................................................................................383
Gemini SWIFT 5.....................................................................................395
THEORIX...............................................................................................411
TopClass...............................................................................................427
Total Knowledge Management System.....................................................443
Vuepoint Learning System......................................................................459
Part VI—Other Products Using “Learning Object” Technology...........474
ADAM...................................................................................................475
Anlon....................................................................................................476
e3.........................................................................................................477
Hyperwave............................................................................................478
How To Master Learning Management System.........................................479
Obtree Learning Center..........................................................................481
Plateau.................................................................................................482
Saba Content.........................................................................................483
University 360.......................................................................................484
Vitalect.................................................................................................485
Part VII—Comparative Grids..............................................................486
Overview Comparison of At-a-Glance Tables............................................487
Levels of Reusability: Support Functionality for Reusable Objects...............504
Support for Multiple Output Types...........................................................505
Built-in Authoring: Question and Interaction Types Built Into the System....506
Extensibility of the Authoring Environment...............................................508
Test/Exam Delivery Features...................................................................511
Collaborative, Team Development Tools..................................................514
Revision Controls, Archiving and File Management....................................515
Part VIII—Short List Picks..................................................................516
Brandon-hall.com Short List Picks............................................................517
Appendix............................................................................................524
Feature Lists..........................................................................................525
Summary of Statistics About LCMS Products (meta-analysis)....................533
List of Vendors and Contact Information (alphabetical by product).............539
Glossary................................................................................................543


Do Not Reproduce, v1.0  2001 brandon-hall.com 1
Part I – Introduction
ƒ Why This Report?
ƒ Research Process
ƒ How to Use This Report



Do Not Reproduce, v1.0  2001 brandon-hall.com 2
Why This Report?
As an industry, we’ve been talking about the concept of “learning objects” for many years
now. Early in the game, it seemed that the only real implementations of learning object
technology were found inside large organizations that invested heavily in creating innovative,
enterprise-level, in-house projects, designed to support multi-developer publishing models for
e-Learning courseware. The rest of us had to settle for creating e-Learning in the same
fashion as we created CBT; namely, creating learning content piece by piece using desktop
applications such as off-the-shelf authoring tools, general purpose Web development tools
(like FrontPage and Dreamweaver), or even standard office tools like Microsoft Word and
PowerPoint. The net result is that, as an industry, we continue to create learning content as
large monolithic courses with content locked inside proprietary file formats or even as HTML
pages with limited reusability of the content for other purposes.
In this report, we provide comparative information on 29 commercially available learning
content management systems (referred to hereafter as an LCMS) that provide the foundation
technology for creating, managing and reusing learning content (or learning objects) across
courses, curricula and even across an entire enterprise and supply chain.
At brandon-hall.com, our goal is to help demystify the technologies available for deploying e-
Learning inside your organization. We realize that learning content management systems are
not suited to everyone’s needs. However, if you work in an organization that produces large
volumes of e-Learning content with larger groups of content contributors, then you should at
least be aware of what’s available in the new LCMS space.
Everyone is talking about the LCMS trend. We’ve seen recent reports from industry analysts
such as IDC, WR Hambrecht +Co and Gartner Group, and by all indications, learning content
management systems are the next big wave in e-Learning, and rightly so. The products
attempt to solve some fundamental business issues that are only partially addressed by other
e-Learning offerings, such as:
How do we efficiently produce hundreds of e-Learning courses inside our organization
(rather than just a few)?
How do we manage large teams of developers working on e-Learning projects?
How do we leverage content that we’ve already created in future courses or derivative
versions of the same course (e.g., multilingual courses or courses re-branded and adapted
for different customers)?
How do we use the same content for e-Learning and simultaneously use it for print-
based learning or to automatically re-publish the content as an electronic performance
support system (EPSS)?
In this report, we’ll tell you how the LCMS vendors are attacking these issues and how you
can match the functionality of these systems to your specific needs.
Research Process
For this report, we cast a large net to identify products that utilized learning object
technology by getting in touch with companies who have already caught on to the vision of
managing content at the enterprise level. We asked them what tools they’re using and how
the process is working for them. We searched for articles and white papers on innovative
learning object technologies. We sat through endless demonstrations and presentations at e-
Learning and knowledge management trade shows to identify key players and also innovative
smaller players in this space.
The research project has spanned a six-month time period as we analyzed over 50
companies that offer products allowing content reuse and delivery of granular modules of


Do Not Reproduce, v1.0  2001 brandon-hall.com 3
instruction. The name “learning content management systems” was literally born during our
research process. We began to see the LCMS moniker appear on banners at trade shows,
used extensively on e-Learning discussion groups, and even the formation of an LCMS
vendor council comprised of several key LCMS companies. It has been exciting to watch a
new industry sector emerge right before our eyes and to be at ground zero reporting on the
innovation that is taking place inside these companies.
As part of our research, we identified key criteria for evaluating an LCMS system and
generated a comprehensive survey to find out how companies are addressing each of these
dimensions, which include:
How learning content is structured and organized as “learning objects;”
Levels of reusability;
Database support;
Ability to generate content into multiple output formats (such as e-Learning, CD-ROM, print-
based instruction, EPSS format, PALM Pilot, help files, access to individual objects for use
in instructor-led training, etc.);
Built-in, rapid content creation and authoring utilities;
Test and assessment capabilities;
Support for content created using third-party tools (such as Dreamweaver, Flash, Word,
PowerPoint, etc.);
Support for adaptive learning and dynamic assembly/delivery of content (such as dynamically
generating a pre-test that recommends specific learning based on a learner’s
performance);
Built-in tools to manage the workflow of content development and facilitate collaboration
among the entire development team;
Content management capabilities such as revision control, archiving and file management;
Advanced search engine and metadata tagging functionality;
Use of XML for interoperability with other systems and future-proofing of content;
Adherence to industry standards and specifications (such as AICC and SCORM);
Interoperability with third-party learning management systems;
Flexibility and performance of the delivery platform (the ability to wrap global navigation
controls around content and change the look and feel without re-authoring the content);
System requirements;
Cross platform support;
Product support offered by the company.
Note: Several of the vendors commented that the survey was an extremely useful self-
assessment of their own technology and will provide a road map for future enhancements to
their LCMS offering.
We spent several months collecting data in these critical areas and also performed live
evaluations of each of the 29 LCMS products so that we could see the systems in action and
evaluate their performance.
We summarized the data collected to provide meta-analysis data on the new, emerging LCMS
industry, and we provide that information throughout the report. This allows us to tell you
what is common among the various systems and also what we found to be unique about
each product.


Do Not Reproduce, v1.0  2001 brandon-hall.com 4
How to Use This Report
If you are already familiar with the basic concepts of learning object technology, you can use
this report as a resource for finding information on each of the products we reviewed. What
you might find most useful is to read the “Critique” section for each product to find the
results of our evaluation. You can also use the comparative grids and the “At a Glance”
section for each product to find LCMS tools that match your specific needs. You may also be
interested in reviewing the section entitled “Comparison Points – Primary Functionality” to
learn the advantages and disadvantages of various approaches we discovered while
reviewing the systems. This will help you focus your review on the factors most important to
you in selecting an LCMS system.
If you are not familiar with learning objects or the make-up of an LCMS system, we
recommend that you read this report in the order it was written. We have organized the
report to bring you quickly up to speed on the concept of creating, managing and delivering
learning content from a central repository. This is a much different concept than the previous
model of creating courses one at a time at the atomic level.
Here are the major sections you’ll find in this report and how each can help you:
Definitions: As you read LCMS advertising literature and listen to the vendors, you’ll often
hear terminology used interchangeably. For example, some of the vendors call themselves
“content management systems” rather than “learning content management systems.”
However, there is already a well-established group of content management system providers
that have very little to do with e-Learning applications. Tools such as Documentum and
Vignette, which are from the content management space, provide organizational
infrastructure for all types of content and digital assets which can be found in a company. It
is helpful to know the difference in philosophy and functionality between traditional content
management systems and learning content management systems.
Still other vendors refer to their systems as enterprise-level authoring tools, which leads to a
confusing comparison between an LCMS and tools like Authorware and ToolBook.
This section will provide a definition for each of the common terms often confused with (or
sometimes associated with) learning content management systems.
Anatomy of an LCMS: In this section, you’ll find a general description of what makes up a
learning content management system. We’ll outline the domain of an LCMS and how it fits
into the practice of creating content as a team, organizing content in a central database
repository, and publishing the content for use in a variety of formats.
Comparison Points – Primary Functionality: This section is a walk-through of the critical
components of LCMS functionality. You can read about the common characteristics we
observed in the systems evaluated, and learn about the advantages and disadvantages of
each approach.
LCMS Products: In this section, you will find detailed information about and a review of
each of the 29 LCMS products. Each product review contains an At-a-Glance grid so that you
can quickly find the type of product you are looking for. Read the Critique to read comments
from our review of the product with a listing of what we liked and didn’t like about each tool.
You’ll also find recommendations concerning the most appropriate context for each particular
LCMS. Each product review also contains an overview of the company’s philosophy on
learning objects and detailed information on each of the critical comparison points. Watch
also for checklists to help you find out if the tool has specific functionality (such as spell
checking or the ability to check files in and out).
Other Products Using “Learning Object” Technology: In this section, you’ll find a
listing of other products that serve as learning object repositories. Often the tools in this


Do Not Reproduce, v1.0  2001 brandon-hall.com 5
section have a twist, such as one product that focuses exclusively on PALM delivery of
instruction, or another product that embeds learning content management system
functionality into its primary LMS product, etc.
Brandon-hall.com Short List Picks: In this section, you’ll find our picks for the best LCMS
products in several different categories. The picks are based on our live reviews of each
system.
Appendix: The Appendix provides additional resources, such as a sample questionnaire you
can use to evaluate LCMS products that may not be covered in this report. You can also find
a summary of the LCMS meta-analysis data here for quick review. Also, with LCMS
technology comes a slew of new terminology. You can find out what these words mean by
looking in the Glossary.


Do Not Reproduce, v1.0  2001 brandon-hall.com 6


Do Not Reproduce, v1.0  2001 brandon-hall.com 7

Part II – Definitions
ƒ Authoring Tool
ƒ Learning Infrastructure
ƒ Learning Object
ƒ Knowledge Management
ƒ CMS — Traditional Content Management System
ƒ LMS — Learning Management System
ƒ LCMS—Learning Content Management System



Do Not Reproduce, v1.0  2001 brandon-hall.com 8
Introduction
The following definitions are provided to help you differentiate among the variety of
technological offerings in the e-Learning market. The fact that the industry now uses both
the LMS and LCMS acronym makes it difficult to tell what functionality one is talking about.
This is especially difficult for e-Learning novices attempting to compare and contrast the
various offerings.
Authoring Tool
Authoring tools are generally desktop, single-user applications used to construct learning
content by assembling and combining text, graphics, audio, video and animations into e-
Learning courses. Such tools include:
Authorware
Flash
Director
ToolBook
Quest
You can also consider general-purpose Web development tools (visual HTML editors) as
fitting into the authoring category. These tools are often used to create standard Web pages,
but they can also be used for creating learning applications. This category includes tools like:
FrontPage
Dreamweaver
In addition, there are a growing number of authoring tools focused on rapid, template-based
development of learning content, sometimes with a specific focus on a particular type of
learning application, such as software simulations. This category includes tools like:
TrainerSoft
Lectora Publisher (Trivantis)
Ready Go Web Course Builder
Tactic!
DazzlerMax
All of these are considered authoring tools even though they offer a rather broad range of
solutions.
Some people make unnatural comparisons between authoring tools and learning content
management systems, mostly because both have content creation capabilities. Ninety
percent of LCMS products in this report have some form of content authoring built into the
system. On the other hand, some LCMS suppliers have chosen to focus their efforts on the
assembly and management of content. These systems provide for flexible integration of
content created with third-party tools, and in some cases come bundled with a popular
authoring tool.
Learning Infrastructure
Learning infrastructure is a less known term used to describe turnkey, integrated solutions
containing both content authoring capability and learning management system functionality
in a single product offering. A good indication that a system is a learning infrastructure and
not an authoring tool can be determined by asking the question, “Would I ever create
content in this system and use it with a third-party learning management system?” If the
answer is no, it’s probably a learning infrastructure. We find that most learning


Do Not Reproduce, v1.0  2001 brandon-hall.com 9
infrastructures are also Web-based, meaning that most content is created through a browser
interface.
Learning infrastructures are most often confused with learning content management systems
because, like LCMS products, they are also often multi-user development environments.
However, here are some of the characteristics of an LCMS that are generally missing in a
learning infrastructure:
Shared media repository;
Ability to reuse learning objects across courses;
Advanced searchability of learning content using metadata tags;
Content is stored in a central database repository;
Content can be exported into non-proprietary formats like XML for use in other systems;
Dynamic pre-testing;
Collaborative/workflow tools to manage the content development process;
Version control mechanisms;
Interoperability with third-party learning management systems.
Learning Object
A learning object is not really a set technology; rather, it is a philosophy for how content can
be created and deployed. There are no current standards for what constitutes an exact
learning object, although some have provided simplistic models for learning object design.
For example, one company defines a learning object as presentation content with embedded
test questions and objectives wrapped in metadata tags. This oversimplifies the learning
object model we saw in the LCMS products reviewed. The SCORM specification probably
comes the closest to defining learning objects by providing standards for tagging learning
content and methods for storing course hierarchy (See www.adlnet.org
for more
information). The specification is currently evolving further to include definitions for how to
store actual content, such as the content items that make up a drag-and-drop question,
discovery exercise, simulation, etc. The industry will greatly benefit as these specifications
become true standards.
A higher level, but more accurate description of a learning object, is a granular learning topic,
anywhere from five to 15 minutes in length that can be reused with all of its associated
relationships, which may include links to objectives, competencies, test questions, associated
properties and metadata tags. However, we also discovered that in some of the LCMS
learning object models, even an individual, tagged media object could be considered a
learning object.
We found that each of the companies using the learning object metaphor has their own
defined relationships and characteristics for what constitutes a learning object. Therefore,
providing a precise definition of a learning object would be a moot point.
We hope to see better definitions and common standards for learning objects in the future.
Knowledge Management
By simple definition, knowledge management is the practice of organizing information in a
central repository, representing the best practices of an organization, and serving up that
information in a timely fashion to those who need it. For some types of knowledge and some
types of work, however, it is best to focus on supporting group interactions rather than
structuring knowledge into databases and documents. This highlights the two sometimes
divergent views of knowledge management: the “content” perspective and the “community”


Do Not Reproduce, v1.0  2001 brandon-hall.com 10
perspective. Ideally, both are applied in various combinations suitable to each particular
context.
The “content” approach to knowledge management has most often been applied to large
collections of digital assets such as documents, memos, charts, graphs, schematic drawings,
flowcharts, video clips, digital photographs, and even things like important e-mail
communication. The greatest difficulties for those establishing a robust knowledge
management practice has been keeping all the information current and providing access to
the right information at the right time without burdening viewers with “information overload.”
In contrast, some have described learning content management systems as the convergence
between traditional e-Learning and knowledge management. This oversimplifies the issue.
We see this convergence as just starting to take place for the following reasons:
Both learning content management and knowledge management share a similar purpose,
namely to increase knowledge, learning and skills within the organization.
Both share similar technologies. The database technology for both knowledge
management and LCMSs are fundamentally the same. Both allow for advanced searching of
content. The primary difference is that learning content generally includes opportunities for
measuring performance through tests, quizzes and interactions such as simulations, while a
knowledge base generally serves up only critical information needed to perform one’s job.
Both represent expert knowledge and best-practice information to achieve organizational
goals.
There is significant overlap between the types of content managed by both systems. For
example, a process flow chart showing the procedure for how to maintain a piece of
equipment is just as useful in a knowledge management system used on the job as it is for
training and learning purposes. In fact, most organizations would be delighted if changes
reflected on the process flow chart would automatically be reflected for both uses.
On the other hand, there are some significant barriers to this convergence. The obstacles are
more structural and historical than they are technological barriers:
Structurally, the management of learning almost always resides within the training
department as part of human resources.
Knowledge management, however, in most cases, does not reside within the training
department.
Knowledge management often resides at a very high level in the organization, sometimes
directly below the executive level or is dispersed among several departments. Very often,
knowledge management is married to the establishment of the corporate intranet.
Complex and ambiguous concepts: Knowledge management concepts are among the
most ambiguous and misunderstood. There is only partial agreement on definitions of
knowledge, content, intellectual capital and other basic concepts, leading to confusion and
missed opportunities for collaboration.
Divergent communities of practice: While this has begun to change, knowledge
management and e-Learning/training people rarely attend or speak at the same conferences.
Divergent technologies: Knowledge management and e-Learning have spawned totally
different software sub-industries, with very few firms trying to serve both markets. Content
and document management systems include virtually no learning or competency
management functions, and learning management systems offer little support for content
which is not structured training.



Do Not Reproduce, v1.0  2001 brandon-hall.com 11
CMS – Traditional Content Management System
Content management systems are designed for a much broader purpose than learning
content management systems. They are usually used to create information portals for
organizations and can serve as the foundation for the practice of knowledge management,
but they can also be used for simply organizing documents and media assets. For example, a
newspaper agency may use a content management system to provide an archive of every
story ever written for the paper. Likewise, they might use the CMS to provide an extensive
library of photographs that are reusable for future stories.
Content management systems have many uses beyond providing information for learning
purposes. Some of the leading content management system providers include:
Company Web Site
Documentum www.documentum.com

Interwoven (TeamSite) www.interwoven.com

Vignette www.vignette.com

Macromedia (Spectra) www.macromedia.com

BroadVision (Bladerunner) www.broadvision.com

OpenMarket www.openmarket.com

Percussion (Rhythmx) www.percussion.com

Infosquare.com (Openshare) www.infosquare.com

E-Business Technologies (DynaBase) www.ebt.com

Note: Throughout our research in LCMS technology, we continued to wonder why more
mainstream content management system providers are not participating in the e-Learning
space. We did, however, discover two LCMS products that are based on traditional content
management platforms: F(2), from Interactive Media, is based on Interwoven and a
discontinued product called E-Learn was built on Bladerunner, from Broadvison.
LMS – Learning Management System
Learning management systems (LMS) and learning content management systems (LCMS)
really have two very different functions. It’s unfortunate that both have such similar names
and a shared acronym, which only serves to confuse e-Learning buyers even more.
The primary objective of a learning management system (LMS) is to manage learners,
keeping track of their progress and performance across all types of training activities. By
contrast, a learning content management system (LCMS) manages content or learning
objects that are served up to the right learner at the right time.
Understanding the difference can be very confusing because most of the LCMS systems also
have built-in LMS functionality. In fact, 81% of the systems in this report include LMS
functionality as part of their system. Interestingly enough, 100% of the LCMS systems list
themselves as being interoperable with third-party learning management systems. More than
half (54%) have actually performed interoperability tests with leading LMS products. The
most frequently listed LMS tools that have performed interoperability tests with LCMS tools,
and good examples of products that clearly fit in the LMS space include:


Do Not Reproduce, v1.0  2001 brandon-hall.com 12
Saba
Docent
Ingenium (Click2learn)
THINQ TrainingServer
LearningSpace (Lotus)
LEAP (Intellinex)
LearnFrame
TopClass (WBT Systems)
Besides the embedded learning management system functionality, there can also be
significant overlap between LCMS and LMS capabilities and purpose. The following chart is
based on what we’ve observed by analyzing both LMS and LCMS products. While some
products have functionality that crosses the boundaries, we found most systems generally
focused on their own domain as follows:


LMS
LCMS
Primary target users Training managers,
instructors, administrators
Content developers,
instructional designers,
project managers
Provides primary management of… Learners Learning content
Management of classroom,
instructor-led training
Yes (but not always) No
Performance reporting of training
results
Primary focus Secondary focus
Learner collaboration Yes Yes
Keeping learner profile data Yes No
Sharing learner data with an ERP
system
Yes No
Event scheduling Yes No
Competency mapping – skill gap
analysis
Yes Yes (in some cases)
Content creation capabilities No Yes
Organizing reusable content No Yes
Creation of test questions and test
administration
Yes (73% of all LMS tools
have this capability)
Yes (92% of all LCMS tools
have this capability)
Dynamic pre-testing and adaptive
learning
No Yes
Workflow tools to manage the
content development process
No Yes
Delivery of content by providing
navigational controls and learner
interface
No Yes


Do Not Reproduce, v1.0  2001 brandon-hall.com 13

LCMS – Learning Content Management System
A learning content management system is a multi-developer environment where developers
can create, store, reuse, manage and deliver learning content from a central object
repository.
An LCMS will generally have a majority of the following characteristics. You can use this
checklist to determine if a software application could be called a learning content
management system.
Note: Not all of the products in this report have all of these characteristics. The primary
differentiator to determine if a product is an LCMS is if it offers reusability of learning content
and is generally constructed using a learning object model.
Common Characteristics Checklist:

ƒ Based on a learning object model.
ƒ Content is reusable across courses, curricula or across the entire enterprise.
ƒ Content is not tightly bound to a specific template and can be re-deployed in a variety of
formats such as e-Learning, CD-ROM, print-based learning, PALM, EPSS, etc.
ƒ Navigational controls are not hard coded at the content (or page) level.
ƒ There is a complete separation of content and presentation logic.
ƒ Content is stored in a central database repository.
ƒ Content can be represented as XML or is stored as XML.
ƒ Content can be tagged for advanced searchability (both at the media and the topic level).
ƒ Pre-tests and post-tests can be automatically aggregated from test questions written for
the primary instruction. In addition, the system can deliver the test and prescribe
learning based on performance.
ƒ The system manages the development process by providing some level of workflow tools
to manage a multi-developer, team environment.
ƒ Version controls and archiving capabilities to store previous versions of content.
ƒ Advanced searching capabilities across all objects in the repository.
ƒ Interoperability with third-party learning management systems.
ƒ Includes a delivery engine for serving up content, automatically adapting to user or
group profiles, adding navigation controls, collaboration tools, utilities, and look and feel
(skins).

In the next section, you’ll learn more about how a typical LCMS works.


Do Not Reproduce, v1.0  2001 brandon-hall.com 14


Do Not Reproduce, v1.0  2001 brandon-hall.com 15

Part III — Anatomy of an LCMS
ƒ Rapid Content Development Tools
ƒ Content Assembly Interface
ƒ Database Storage
ƒ Delivery Engine
ƒ Multiple Output Formats


Do Not Reproduce, v1.0  2001 brandon-hall.com 16
Introduction
Not all LCMS products are alike. However, there seem to be some common themes we
encountered as we reviewed the systems covered in this report. We were surprised to see so
many similarities among products, where it was obvious that vendors who hadn’t seen each
other’s tools before nevertheless created very similar functionality.
In this section, we’ll create a composite of the typical learning content management system
and include core functionality that we saw time and time again in most of the LCMS products.
We’ll also walk through the process of how someone might create, manage, reuse and
deliver a learning object in a typical system. Again, you will find differences in the systems; in
fact, most of the systems in this report have very unique functionality that make them
noteworthy.
Here is a diagram of what we saw as the core functionality of an LCMS:


Rapid Content Development

Rapid Content Development Tools
In reviewing the LCMS product offerings, we found that most of the systems (90%) have
built-in content authoring capabilities; however, most of the systems are also open to
accepting content from third-party authoring tools like Flash, Authorware, ToolBook,
Dreamweaver, etc. The basic idea is that content can be placed in the learning object
repository from a wide variety of content data sources.
Do these systems have to include content authoring capabilities to be considered an LCMS?
The answer is “no.” The primary focus of an LCMS is its ability to assemble content from the


Do Not Reproduce, v1.0  2001 brandon-hall.com 17
repository for multiple uses regardless of how the content was created. Rapid content
development tools are simply utilities that minimize the need to create all content externally
and organize it in the repository. We reviewed a few systems in which all content is created
externally and organized in the repository. This approach is not wrong, it just depends on
how much authoring capability you want included in your system.
As you begin to assess and compare LCMS systems, you will see a wide variety of
sophistication in the content authoring capabilities among products. While most systems have
built-in authoring, most have only very simple capabilities with limited layout functionality
and limited question types. For example, only 54% of LCMS offerings have built-in
capabilities to create more advanced interactions like drag and drop (See an example of a
drag-and-drop editor below).

Example of a content authoring interface (drag-and-drop question from Evolution LCMS from Outstart).

existing HTML editor such as FrontPage or Dreamweaver, or allow you to enter HTML at the
HTML tag level. In addition, many of the systems allow you to add advanced HTML pages
containing JavaScript to extend the authoring functionality and to create advanced interactive
exercises.
Most of the systems are designed for use by novice content developers such as subject
matter experts and classroom instructors. With minimal learning, novices can often create
basic content, classify the content with descriptive information called metadata tags, and
organize the content in the master learning object repository.
Content Assembly Interface
Almost all of the systems provide a visual interface for viewing the hierarchical structure of
the learning object repository and its constituent courses, lessons, knowledge objects and so
on. The most common metaphor found in LCMS products for representing this hierarchy is a
standard Windows tree view (See the examples below).


Do Not Reproduce, v1.0  2001 brandon-hall.com 18

Tree View from Knowledge Pathways (Global Knowledge).


Tree View from Jupiter (Avaltus).



Do Not Reproduce, v1.0  2001 brandon-hall.com 19

Tree View from a browser-based LCMS, Theorix (Theorix).

What we found common in most systems is that content can be reused from other courses in
the curricula or enterprise (through linking) without copying the content. Often visual
indicators show what content is native to that part of the hierarchy and what content has
been linked from another location.
The content assembly interface is used to sequence and organize content into courses,
modules, topics, individual exercises and even media objects (such as graphics, animations,
audio, video, etc.). Although the terminology is often quite different from system to system,
we found many similarities and equivalent levels among most systems. You should be aware
that some of the systems allow you to have as many nested sub-levels as needed, while
other systems have a very structured, well defined taxonomy of levels in the course
hierarchy. At the same time, some systems allow you to change the terminology to describe
different levels, while others require you to learn and use their defined terminology (course,
lesson, unit, etc.). None of the approaches is necessarily right or wrong. It’s just a matter of
what will work to communicate the appropriate organizational metaphor among members of
your development team.
In terms of technology used to display the content assembly interface, you will find that
systems are divided into two main groups – browser-based content development systems
and ones that require a locally installed application. There are advantages and disadvantages
to each approach as follows:
Browser-Based Content Assembly Interface
Advantages
Disadvantages
Can create and modify learning objects from
anywhere.
Slower performance (especially when
accessing from remote location without
fast connection speeds).
Novice developers are often more familiar with a
browser environment. Subsequently, there’s less
need to learn a proprietary application interface.
Limitations of browser can sometimes
reduce the level of functionality available
for creating and assembling content (i.e.,
dragging and dropping learning content
into the course hierarch
y
is less often


Do Not Reproduce, v1.0  2001 brandon-hall.com 20
found in a browser interface).
Can be easily integrated into a hosted environment
(up and running in minutes).

Locally Installed Client
Advantages
Disadvantages
Often much faster performance. You have to install an application on your
hard drive.
More powerful functionality for managing large
repository of learning objects.
You can’t create and modify content
anytime anywhere.
Easier to interact with other desktop applications
(e.g., Internet Explorer, Word, PowerPoint).
Learning curve associated with learning a
desktop application.

Sixty-five percent of the systems in this report use a locally installed application for
assembling, organizing and sequencing learning content.
Forty-one percent use a browser-based interface.
You may be asking, “Why do the numbers total more than 100%?” The answer is because
7% (two systems) allow developers to choose either a browser-based interface or a locally
installed client.
Here is a list of the systems that use a local application:
Centra Knowledge Server (Centra) *
Docent Outiner/Content Delivery Server (Docent)
Evolution (Outstart) *
IAuthor (NYUOnline)
IPRESS/KBRIDGE (KnowledgeXtensions)
Jupiter (Avaltus)
Knowledge Mechanics (Knowledge Mechanics)
Knowledge Pathways (Global Knowledge)
Knowledge Producer (IBM Mindspan Solutions)
KnowledgePlanet Content (KnowledgePlanet)
LeadingWay KnowledgeOne Content Manager (LeadingWay Knowledge Systems)
Lightspeed OmniSite (LightSpeed Interactive)
LogicBuilder (LogicBay)
Nogginware (Handshaw Inc.)
SWIFT (Gemini Learning Systems)
TopClass (WBT Systems)
Total Knowledge Management (TKM) System (Generation21)
VuePoint (VuePoint)
Here is a list of the systems that use a browser:
Aspen Content Development Server (Click2learn)
Aspen Learning Experience Server (Click2learn)


Do Not Reproduce, v1.0  2001 brandon-hall.com 21
Centra Knowledge Server (Centra) *
EPath Learning (ePath Learning)
Evolution (Outstart) *
F(2) (Interactive Media)
Ibtraining.com Adaptive Learning Framework (ibtraining.com)
Intellinex LEAP Learning Development System (Intellinex)
IPerformance (Online Courseware Factory)
Knowledgelinx 2000 (Knowledgelinx)
SmartBuilder (Suddenly Smart)
THEORIX (e-com Inc.)
* Uses both local application or browser-based content assembly.
The content assembly interface is also where content developers provide descriptive
metadata tags for content objects. Metadata tags can be used to provide advanced searching
capabilities for easily locating and reusing content in another course. For example, you may
want to find a specific schematic drawing that was used in a previous course nine months
ago. To find the drawing, you simply type in keyword descriptions, media type and/or the
name of the content creator. Most systems have excellent searching capabilities, which is one
of the fundamental characteristics of a good LCMS tool.
Finally, the content assembly interface (or middleware layer) is often responsible for
communicating with the central database and loading and retrieving objects from the
database as requested.
Database Storage
We found in almost every system that content resides inside a master database located
centrally so that every team member has access to shared, reusable content. Media files are
most often not stored inside the database (although there are some exceptions). Rather,
they are referenced from inside the database and reside in a media folder on the central
server, most often organized into a folder structure to store different types of media.
The rest of the content in the course, such as course organization structure, metadata
tagging information, on-screen text, question stems, question distractors, feedback text, etc.,
typically resides inside the database in one of three primary formats: (1) as ASCII text inside
the database, (2) as HTML text, or (3) as XML text.
Our first discovery was that exactly half of the LCMS systems we reviewed use multiple
database formats (50%). Here are the most common database formats used:
57% Oracle
57% SQL
11% Microsoft Access
14% Use some other type of database as their primary format *
* Other databases listed include the following:
ƒ Interwoven
ƒ Objectivity
ƒ Datastore (ODI)
ƒ Tamino
ƒ Linter


Do Not Reproduce, v1.0  2001 brandon-hall.com 22
ƒ MYSQL
ƒ SyBase
ƒ DB2
ƒ PostgreSQL
Delivery Engine
What differentiates an LCMS from a simple “media repository” is that an LCMS always has a
delivery engine for serving up content to the learner. This typically consists of querying the
learning object repository and assembling content for use by the learner. (Note: Some
systems draw content from the learning object repository at run time, while others require
you to pre-publish content prior to delivery.)
The LCMS delivery engine typically wraps robust navigational controls around the learning
content, automatically generates tables of contents from the data, and applies look-and-feel
elements (skins) to the learning content. Some LCMSs also add additional functionality, such
as the ability to take notes, discussion groups, bookmark utilities, ability to e-mail other class
members, etc. Compare and contrast the examples below. Note the persistent navigational
controls and other functionality.

Delivery engine in Aspen Learning Experience Server (Click2learn).



Do Not Reproduce, v1.0  2001 brandon-hall.com 23

Delivery engine in VuePoint VLS (VuePoint).



Delivery engine in Knowledge Producer (IBM Mindspan Solutions).


The LCMS delivery engine usually consists of a player to present learning content to the
learner in a browser window (when used as e-Learning) or in other formats such as print-
based instruction, CD-ROM (sometimes with an offline viewer other than a browser), and
even PALM Pilot distribution.
Most of the delivery engines (72%) offer dynamic, prescriptive pre-testing where the system
tests the learners up front and creates a tailored learning plan for each learner based on pre-


Do Not Reproduce, v1.0  2001 brandon-hall.com 24
test performance. Some systems are so advanced that they can even adapt learning to
match a learner’s learning style preference, serving up content that yields the best results.
Multiple Output Formats
As you begin evaluating LCMS products, you will find some that are targeted specifically for
e-Learning, while others provide a wide range of output formats such as e-Learning, CD-
ROM, print-based training, slide shows, resource repository for classroom instructors, EPSS
(electronic performance support systems), job aids, help files, PALM output, etc. The promise
of LCMS platforms is that content doesn’t have just one use. If LCMS is the beginning of the
convergence of traditional e-Learning and knowledge management, this multiple use of
content is key.
In most of the systems we reviewed that support multiple output formats, individual learning
objects are tagged for their intended output type or multiple types. When content is
published and a format selected, the system typically assembles the learning material and
prepares it for the proper distribution. It should be noted again that some systems actually
leave content in the central database repository and serve it up live at run time when
creating e-Learning output, while others pre-compile content into such formats as (1) HTML
pages, (2) ASP or JSP pages, (3) XML with a run time DTD/XSL, (4) compressed, run time
databases, etc.
We also observed some systems capable of serving up content to remote learners not
continuously connected to the Internet, allowing them to download and play content offline
and then reconnect and record their performance upon their next connection with the
Internet.
Here are some statistics about the systems covered in this report:
100% Output for e-Learning
72% Output for CD-ROM
48% Output for print-based learning
28% Output for PALM distribution
17% Creates a separate mechanism for delivering an EPSS system (beyond just
generating topic-based e-Learning)



Do Not Reproduce, v1.0  2001 brandon-hall.com 25

Part IV — Comparison Points: Primary Functionality
ƒ At-a-Glance Grid Information
ƒ Critiques


Do Not Reproduce, v1.0  2001 brandon-hall.com 26
Introduction
In this section, we’ll walk through each type of information you’ll find for each of the
products we reviewed. We’ll tell you the issues for each of the headings you’ll find on the
reviews and give some guidance about how to evaluate the information.
At-a-Glance Grid Information
In the At-a-Glance grid for each product, you’ll find 21 product characteristics that you can
use to quickly identify which products meet your specific needs, or more importantly, rule out
ones that don’t meet your needs. As the name implies, this information can be browsed
quickly, and the fields contain minimal information to make the assessment. You will find
more detailed information in the remainder of the product review.
Database Support

Use this field to quickly find products that support existing database formats that you might
already use inside your organization such as Oracle, SQL, SyBase, Access or other unique
database format. You can also keep an eye out for products using new innovative
technologies such as true object-oriented databases to support robust learning object
repositories.
AICC-Compliant/Certified

AICC is a specification that allows AICC-compliant content to communicate learner
performance data to AICC-compliant (or certified) learning management systems. The
strongest products in this class are those that have gone through the formal AICC
certification process and been tested by an independent lab. There are only a handful of
LCMS products that have received full certification. They are marked in this category. For a
current and complete list of AICC-certified products, see www.aicc.org.
Of the LCMS systems reviewed:
7% of all systems in this report have received AICC certification through an independent
testing lab.
76% of the systems are AICC-compliant.
SCORM Specification Support

SCORM is a newer, more comprehensive specification for interoperability between an LCMS
and LMS product to pass course structuring information and learner performance data among
the systems. If you’d like more information about the SCORM specification, see
www.adlnet.org .
83% of all systems in this report use the SCORM specification.
90% will follow SCORM by the end of 2001.
Use this section of the grid to locate products that follow this standard. This may be of
special interest for those working on U.S. government contracts, as many of these types of
projects require SCORM.
Built-in Authoring

If you want your learning object repository to have some form of content authoring
capability, use this section to find systems with built-in capability. A “Yes” in this category


Do Not Reproduce, v1.0  2001 brandon-hall.com 27
doesn’t automatically mean that you have to create content in the system, it simply means
that the feature is available for your use.
90% of systems in this report have built-in authoring capability.
Authoring Method

This category will help you find systems that are either 100% browser-based or ones that
require a local application to be installed in order to create content. Note: Some systems
allow you to do either or both.
Please be aware that this has nothing to do with the content output the learner will see. All
of the systems will playback content through a standard Web browser. This is only a
requirement for those who will create, store and reuse content in the LCMS system.
58% of all the systems require you to install a local application to create content.
34% only use a browser for creating content.
8% offer both options.
# of Authoring Templates

This category provides a quick run down of interaction types available in the LCMS, along
with a total number of templates available. More detailed information appears later in the
review. Please be aware that this is not an “apples to apples” comparison, as the word
“template” has different meanings in different LCMS products. Use this section with
discretion.
Just for your information: The average number of templates available per system in this
report is 37.
Built-in Assessment Tool for Creating Tests

Is writing test questions a function of an LCMS, an LMS or an external authoring tool? The
choice is entirely yours, but this category will help you identify which of the LCMS products
have built-in assessment generation capabilities. Because most of the systems also have
dynamic pre-testing capabilities, we suspect that you will most likely want to create test
questions in this environment.
The assessment tools allow you to create simple test questions like multiple choice,
true/false, fill-in-the-blank, matching, etc. And, on the high end, some systems allow
advanced question types like hot spots, drag-and-drop, and even performance based testing
such as role-play scenarios.
ƒ 93% of the products reviewed in this report have built-in assessment capabilities.
Output Types Currently Supported

This category will help you locate systems capable of generating multiple output types. All of
the systems output content in e-Learning format; however, fewer of the systems can create
output content for PALM delivery.
100% of the systems in this report output content for e-Learning delivery.
72% output to CD-ROM delivery.
48% output content for print-based delivery.
28% output for PALM delivery.


Do Not Reproduce, v1.0  2001 brandon-hall.com 28
17% create a separate mechanism for delivering an EPSS system (beyond just generating
topic-based e-Learning).
Uses XML to Store Metadata Tags

If you are interested in XML, use this category to identify systems that create metadata tags
as XML-formatted files.
83% of the systems in this report use XML to tag content.
Uses XML to Store Actual Content

A majority of the systems are now also storing their actual content as XML. Use this category
to identify those systems.
76% of the systems in this report use XML to store the actual content.
Content Output Formats

This is a category where the shorter the list, the more efficient and less intrusive the system.
For some of the fastest delivery of learning content, watch for systems with only HTML and
JavaScript output. The output formats are listed for the native output of the system not
including content you might add. For example, a system may natively use HTML/JavaScript
for their delivery engine. However, if you add a Flash file or some Real Audio, the learner will
need the appropriate plug-in to view course material. This category includes only the native
output format for your review. This is most helpful in finding systems that fit in your IT needs
and limitations. For example, some government agencies don’t allow the use of Java applets
on their intranets for security reasons. If you fall into this group, you can use this At-a-
Glance rating to find systems that don’t use Java applets. However, Java applets can provide
some of the most engaging, interactive content. So, for the rest of you, this is not an issue.
Cross Platform Support for Content Delivery

Although a large majority of organizations use PCs to view e-Learning content, we are still
seeing some organizations that must deliver their training to MAC and UNIX as well. In a
recent international survey conducted by brandon-hall.com of e-Learning developers
(corporate trainers, academic and government), we identified the following percentage of
companies that must deliver e-Learning content on MAC and UNIX in addition to PC delivery
of content.


Do Not Reproduce, v1.0  2001 brandon-hall.com 29

Survey Results of cross-platform delivery needs for organizations when delivering e-Learning.
100% of the systems in this report deliver content for PC.
83% deliver content for use on a MAC.
76% deliver content for use on UNIX.

Interoperable With Third-Party LMS Products

One hundred percent of all the products listed in this report indicate that they are
interoperable with third-party learning management systems (LMS). However, this field is
most helpful to quickly identify systems that have been tested with specific LMS products.
You’ll find this information listed in the At-a-Glance grid in this category.
Interesting note: Although 100% of the LCMS products list themselves as being interoperable
with third-party LMS products, only 66% said that they have performed an interoperability
test with a product.
Here is a list of LMS products that have been tested with more than one LCMS:
Saba
Docent
Ingenium (Click2learn)
THINQ (TrainingServer)
Lotus LearningSpace
LEAP (Intellinex)
LearnFrame
TopClass
WebCT
A handful of other LMS products were also listed (with single listing only).
Company-Offered LMS

Use this category to find systems that also offer LMS functionality as part of their LCMS
solution. You will find that some of the companies consider these one and the same offering.
However, we thought it would be useful to separate learning object repository technology
from LMS functionality for the sake of comparison in this report.


Do Not Reproduce, v1.0  2001 brandon-hall.com 30
83% of the systems in this report have built-in LMS functionality.
Available as Hosted Solution (ASP)

Use this category to find solutions that also offering content hosting or hosting the LCMS
content production repository on their own Web servers or through partnerships with
standard ISP Internet service providers.
97% of the systems in this report offer their solutions in a hosted model.
Installed Locally on Your Server

Many companies prefer having the solution locally installed on their own Web servers inside
their organization for a variety of reasons such as security, control of environment, etc. Use
this category to find products that can be locally installed. The systems that list “No” here
offer their solution only as a hosted model.
93% of the systems in this report allow you to install their solution locally on your own
server.
# of Clients

The use of learning object technology is a rather new area. Many of the companies have a
very small number of existing clients, mostly because their products are so new. Use this
category to see how many organizations are already using this solution. Be open to
innovation from smaller companies with very few or no clients and also be wary of well
known larger companies that may be listing their entire client base rather than those using
only their LCMS solution.
The median number of clients for vendors listed in this report is 10 (for comparison’s sake).
Company Size

Some would rather work with a large company, while others like the attention from smaller
companies. Use this category to find the overall size of the LCMS vendor (by number of
employees).
The typical size of an LCMS company (median across all companies) is 70 employees.
Cost (based on a large scale scenario)

For many of the vendors listed in this report, they are still working out their pricing models
(as the technology is so new), and pricing is constantly changing. For finding a specific price
for your scenario, we recommend that you contact the companies directly.
However, for this report, we gave each of the companies the same, very large
implementation scenario and asked them to give us an estimate. Please note that the price
may seem very high when compared with your scenario, but it allowed us to derive some
comparative information about the relative cost of products. The scenario consisted of the
following conditions:
8,000 learners;
Over a five year time period (total cost reflects cost for all five years);
Five different servers (located in different regions – international); and
40 course authors.


Do Not Reproduce, v1.0  2001 brandon-hall.com 31
Here’s what we found (all prices are in U.S. dollars):
The range of prices in the scenario were from $150,000 to $1.9 million for the same
scenario.
The average price for the scenario was $537,410.
The median price for the scenario was $430,592.
FYI: We posed the same scenario for 62 learning management systems in our LMS report.
Here’s what we found:
The range of prices in the scenario (for LMS vendors) was $3,000 on the low end to $4.8
million on the high end.
The median price for the scenario (for LMS systems) was $200,000.
In this part of the grid, you’ll see a cost rating for each product, based on the scenario
showing where the price falls according to the average price. Here is the key for the cost
rating:
Key:
Low = < $270,000
Below Average = $270,000 to $400,000
Average = $400,000 to $600,000
Above Average = $600,000 to $800,000
High = $800,000+
Year Originally Released

This field shows you what year the product first appeared on the market (usually as a version
1.0 release). Use this category to find systems that have been around for a while.
Critiques
Our reviewers spent several hours with each tool creating learning objects, organizing them
in the central repository, reusing content from previous courses, designing new interactive
templates, creating dynamic pre-tests, importing content created in third-party authoring
tools, publishing content to a Web server, testing the playback of content through the
delivery engine, etc.
In this section, you can read our evaluation of each LCMS product offering. We’ll tell you
what we liked about the product and also what we didn’t like. Each critique also includes
some overall impressions and recommendations for the best use of each product. For the
most part, we tried to point out what was unique about each tool in this review as well.


Do Not Reproduce, v1.0  2001 brandon-hall.com 32


Do Not Reproduce, v1.0  2001 brandon-hall.com 33

Part V — LCMS Product Offerings
Adaptive Tutoring System
Company: Adaptive Tutoring System
Aspen Content Development Server
Company: Click2learn
Aspen Learning Experience Server
Company: Click2learn
Centra Knowledge Server
Company: Centra Software Inc.
Docent Outliner/Content Delivery Server
Company: Docent Inc.
ePath Learning
Company: ePath Learning Inc.
Evolution
Company: OutStart Inc.
f(2)
Company: Interactive Media Corp.
iAuthor
Company: NYUonline
ibtraining.com Adaptive Learning Framework
Company: ibtraining.com
Intellinex LEAP Learning Development System (LDS)
Company: Intellinex
iperformance
Company: Online Courseware Factory
IPRESS/KBRIDGE
Company: KnowledgeXtensions Inc.
Jupiter
Company: Avaltus Inc.
Knowledge Mechanics Studio
Company: Knowledge Mechanics
Knowledge Pathways
Company: Global Knowledge Inc.
Knowledge Producer
Company: IBM Mindspan Solutions


Do Not Reproduce, v1.0  2001 brandon-hall.com 34
knowledgelinx 2000
Company: Knowledgelinx Corp.
KnowledgePlanet Content
Company: KnowledgePlanet
LeadingWay KnowledgeOne Content Manager
Company: LeadingWay Knowledge Systems
Lightspeed OmniSite
Company: Lightspeed Interactive Inc.
LogicBuilder
Company: LogicBay Corp.
nogginware
Company: Handshaw Inc.
SmartBuilder
Company: Suddenly Smart
Gemini SWIFT 5
Company: Gemini Learning Systems Inc.
Theorix
Company: Theorix
TopClass
Company: WBT Systems
Total Knowledge Management (TKM) System
Company: Generation21 Learning Systems
Vuepoint Learning System (VLS)
Company: Vuepoint Corp.


Do Not Reproduce, v1.0  2001 brandon-hall.com 35


Do Not Reproduce, v1.0  2001 brandon-hall.com 36
Adaptive Tutoring System
Version: 1.0
Company: Adaptive Tutoring Systems Inc.

At a Glance
Database support PostgreSQL (V7.1) – primary
Oracle 8i and DB2 - secondary
AICC-compliant/certified
Certified – No
Compliant – No
SCORM specification support Yes
Built-in authoring
No
Authoring method: browsed-based and/or
locally installed application
Browser-based
# of authoring templates
Templates are determined by students as a
reflection of their learning style. This includes
combinations of the following:
1 – Video
1 – Audio
1 – Graphics
1 – Text
1 – Global index
For sole delivery elements the following are
employed:
4 – Experiential
1 – Detailed text reference
1 – Multiple choice competency assessment
Built-in assessment tool for creating tests Yes
Prescriptive pre-testing
No
Output types currently supported E-Learning
Uses XML to store information about the course
(metadata tags)
Yes
Uses XML to store the actual content (i.e.,
presentation text and questions)
Yes
Content output formats
XML, HTML, WML
Cross platform support for content delivery Unknown
Interoperable with third-party LMS products
Yes — Testing underway with Boniva software LMS.
Company offered LMS (or built-in LMS
functionality)
Yes
Available as a hosted solution (ASP)
Yes
Installed locally on your server Yes
# of clients
3
Company size 41 employees
Cost Rating (based on the following scenario):

• Five year implementation
• 8,000 learners
• Five different servers
• 40 course authors
Below Average

Key:

Low = < $270,000
Below Average = $270,000 to $400,000
Average = $400,000 to $600,000
Above Average = $600,000 to $800,000
High = $800,000+
Year originally released
2001


Do Not Reproduce, v1.0  2001 brandon-hall.com 37
Contact Information
Adaptive Tutoring Systems Inc.
32 Greenlaw Avenue
Toronto, ON M6H 3V5
Phone: (877) 990-9943
Fax: (416) 651-7821
www.adaptivetutoring.com

Product Overview
The Adaptive Tutoring System is a learning content management system that combines
complementary content elements in a synchronized presentation that adapts to the learning style
preferences of each individual.
The system consists of software that presents content to the user, assesses user mastery of this
content, tutors the user to increase level of mastery, and tracks user interactions and results.
Content elements that can be presented include video, audio, animation, graphics and text.
Mastery is demonstrated via incremental testing.
The business benefits of this software include the compression of time to competence as well as
increased content retention.
This will interoperate with existing software, is easily upgraded, and is standards-based. The
company also provides consulting and professional services.
The software is designed to operate on an open source infrastructure. Because of this, there are
no additional licensing fees associated with running the product.
The architecture is based on Enterprise Java Beans (EJB). The most compelling argument for its
use is that its modular design allows incremental system updates that promote longevity of the
platform (and content), thereby reducing expenditures.
This tutoring system is based on active recombination and synchronization of content elements
relying on individual learning style preference, initiated as a remedial response to incremental
assessment of level of content mastery and regulated by AI.
Critique
One thing you need to understand about this product before evaluating it is that the company
has put 100% of its focus and innovation on the delivery of learning objects and not on
managing learning objects during course development. This product was one of only a very few
of the learning content management systems in this report in which learning object content
adapts itself to the learning style of individual learners. This feature goes far beyond simple
prescriptive pre-testing. The system actually monitors learner performance and uses built-in
heuristics* to determine the most efficient learning style. We liked that the system didn’t require
us to duplicate content in creating the different delivery styles; rather, we simply added non-
formatted text, audio, graphics and video to the server, and the system made inferences about
what to show to the learner and when to show it. If learners miss a question, they have the
opportunity to review the course material using a different learning style (such as auditory or
visual learning). Behind the scenes, the system is keeping a running log of what types of learning
best suit the learner’s needs.
We liked the pre-assessment for determining the initial learning style of the learner. When you
finish the assessment, the system will actually diagnose and tell you your preferred learning style.
We found also that the system can accept parameters to set an initial learning style if you happen
to use a third-party assessment or even your own learning style assessment.


Do Not Reproduce, v1.0  2001 brandon-hall.com 38
Another thing we found intriguing about the system is that it uses almost the exact opposite
model for deploying learning content. Most systems in this report use a strong database
repository on the content development side. This system has you assemble content as small bits,
upload them to the server, and it deploys content as a run time database.
On the downside: The content creation process is rather cryptic. There is absolutely no visual
interface for content development or even for organizing or loading your files to the server.
Content developers work at the file level, organizing primitive media files and ASCII text files as
the core content and manually uploading their content directly to the Web server. Course
sequence is determined by following an assumed file naming convention. Even test questions are
formatted in an ASCII format. There is no guidance, wizards or even a dialog box to help you
format the questions.
Content reusability is possible but very difficult to manage because of a lack of a visual interface
to organize content. You can reference already created content by file name. There is also no
built-in search capability for locating existing content.
Because the intelligence of the system is in the adaptive delivery of primitive media assets, you
will find that you don’t have very much control about how the content is laid out on screen. The
default formats are hard coded in the system. When entering text, we found that we couldn’t
even enter formatted text.
We also didn’t like that we couldn’t change the rule base for how content is delivered. Again, this
is hard coded in the system and can only by changed by the engineers at Adaptive Tutoring
Systems.
On the whole, this is a very specialized system, targeted for those who are more interested in
providing an adaptive learning environment. The system is definitely not recommended for those
looking for a system to keep track of content for the purpose of reusability during content
development.
* heuristic, adj.

Of or constituting an educational method in which learning takes place through
discoveries that result from investigations made by the student.
Computer Science. Relating to or using a problem-solving technique in which the most
appropriate solution of several found by alternative methods is selected at successive
stages of a program for use in the next step of the program.
Company’s Learning Object Philosophy
The company originally called learning objects “knowledge bytes.” Learning objects are the
organizational focus of the system and are designed in such a way as to create an object that
exists independent of both the player and the media. The learning object provides the framework
for defining one or two simple, testable concepts. (The competency assessment is one of the
learning objects in this system).
Levels of Reusability – Learning Object Organizational Schema
Since the content that the learner experiences is rendered at the time of consumption from the
available media forms, the learning object can be continually recombined. During the playing of a
course, a single media server or a host of media servers can be employed to deliver the learning
objects.
Support Functionality for Reusable Objects:
Yes/No
Type
No Search engine for media objects (graphics, audio, video, animation)


Do Not Reproduce, v1.0  2001 brandon-hall.com 39
No Can tag media objects with keywords and other searchable information
No Search engine for text objects used in course
No Built-in model for creating “learning objects”
No Preview window when browsing media objects
Yes Built-in content repository for storing and reusing “learning objects” and media
assets
Database Support
Adaptive Tutoring uses PostgreSQL as the default database management system. It’s been
tested with both Oracle and DB2 to ensure cross database conformance. PostrgreSQL offers full
ANSI-standard relational interfaces, as well as ODBC and JDBC support.
The design uses the database for both content storage and content reference storage. As more
complex media forms are introduced, the references are managed as part of the
decision/intelligence of the system. This helps manage references to both native and third-party
learning objects.
The content is rendered in real time from the database. The system uses caching for higher
performance.
There are no license fees for the PostgreSQL (it is in the open source community), and it is
broadly accepted as a database product.
Support for Multiple Output Types
Output Types Supported:
Yes/No
Output
Yes Internet-delivered courses
Yes Intranet-delivered courses
No CD-ROM-based courses
No Paper-based courses (workbooks, printed lesson material)
No Can create optional help files



Do Not Reproduce, v1.0  2001 brandon-hall.com 40
Built-in Authoring (Rapid Content Creation)
Authoring Environment: The current release provides a set of utilities that allow the author to
stage content into the servers. Authors can use their favorite text and graphics tools to create the
content. A full internal authoring tool will not be available until the 1.2 release of the platform.
The 1.2 release authoring tool will be a timeline oriented tool that allows the author to assemble
the execution sequence of the various media.
Question and Interaction Types Built Into the System:
Yes/No
Type
No Presentation (layout) templates for media (text, graphics, audio, video and
animations)
No Electronic book: dynamic index to one side in a frame.
No Performance support: “show me,” “how to,” “how do I”, etc.
No Discovery exercise (e.g., click on various components of the system to learn more
about each part with pop-up information, etc.)
Yes Multiple choice questions
Yes True/false question
No Practice/drill
No Word response (key in the answer) question
No Fill-in-the-blanks question
No Matching exercise
No Drag and drop Question
No Hot spot
No Software application simulation
(a template used to create interactive exercises teaching
software tasks without having to build it from scratch)

No Instructional games
No Soft skills role play simulation
No Likkert scale (used to create surveys)
No Other unique interactive templates in system
High Level – Curriculum Management Functionality
The course staging utilities employ a simple, three-level course hierarchy view.

Extensibility of the Authoring Environment:
Yes/No
Type
No Scripting language (to go beyond built-in functionality)
Yes Can add complex interactions created in third-party applications


Do Not Reproduce, v1.0  2001 brandon-hall.com 41
No Extend tool by using JavaScript

Other Authoring Features Available:
Yes/No
Type
No Quick preview function (to see courses without compiling them first)
No Built-in spell checker
No Built-in thesaurus
No Alignment tools for screen objects
No Spacing tools (automatically provides even spacing between object; i.e., bulleted
list, graphics, etc.)
No Built-in formatted rich text (bold, underline, text color, etc.)
No Easy to create hyperlinks to other URLs
No Use of styles to keep things consistent in course design
No Library utility for storing reusable design templates
No Can create and save your own templates in a library
No Comes with library of look-and-feel elements: backgrounds, navigation buttons,
clip art, media samples, etc.
No Automated tool for creating online glossaries in courses
No Can directly import and use PowerPoint presentations
Support for Content Not Created in the System
Currently the system supports the use of text (served up as HTML), graphics (.gifs) structured
text (served up as HTML tables) and video (QuickTime served up by default from the Darwin
media server).
Test/Exam Delivery Features
Yes/No
Type
Yes Questions can be randomized in a quiz or test
No Test question answers (distractors) can be randomized for each question
Yes Test results can be automatically shown to learners at the end of a quiz
Yes Test questions can be drawn from a pool of questions
No Automatically stores performance data in flat file format
Yes Automatically stores performance data in a database
Yes Learner can see his/her progress report for all tests
No Number of attempts for test questions can be easily set


Do Not Reproduce, v1.0  2001 brandon-hall.com 42
Yes Questions can have immediate feedback
No Questions can have delayed feedback (at the end of the exam)
No Feedback can be turned off
No Timed test questions
No Timed tests
No Can create instructor-graded essay questions (Responses cue for instructor, who
grades them and then the score appears for learners)
Yes System records test item analysis data for each test question
No Built-in statistical analysis package
No Built in utility for creating surveys (“happy sheets”) for assessing Level I
effectiveness (Did they like the course?)
No Test questions can be imported from tab or comma delimited file formats (i.e.,
.csv file)
Adaptive Learning Model
The adaptive learning model rests upon an active recombination and synchronization of content
elements based on individual learning style preference. It is initiated as a remedial response to
incremental assessment of level of content mastery (student achievement) and regulated by AI.
The major components of this model include:
Adaptive, Dynamic Content Delivery: Content (video, audio, graphics, photos,
animation, bullet points) is dynamically assembled and presented to the user on demand, based
on: (1) user preference (based on learning style assessment) and/or (2) user achievement
(based on incremental content mastery assessment).
Synchronized, Multi-Modal Presentation: Multi-modal content (video, audio, graphics,
photos, animation, bullet points) is synchronized and presented simultaneously, allowing the user
to absorb information from several perspectives.
Active, Incremental Assessment: Assessments take place at the end of each learning
object. If the learner scores poorly and re-takes the object in their secondary learning style, the
user is then re-assessed using different questions.
Predictive Artificial Intelligence: Artificial Intelligence algorithms are used by the system
to determine content presentation based on preferred learning style. The AI will also determine
presentation based on previous performance with the type of content being presented.
Relevant, Personalized Information: Users can choose which segments they need on
demand. Users can also choose font, font color, background color and window size.
Learning Style-Based Content Adjustment: Users complete an assessment that
determines their preferred content absorption pattern based on their learning style. The results
of this assessment provide the basis for a unique user profile that is used by the system to
determine which complementary content elements are synchronized for presentation in the
learning object.
The system does not recommend the particular sections of the course the learner should take.


Do Not Reproduce, v1.0  2001 brandon-hall.com 43
Collaborative Team Development Tools
Team Collaboration Functionality:
Yes/No
Type
Yes
Multiple authors can work on the same course at any given time
No Security log-in for valid course developers
Yes
Tools for tracking what has been approved for release
No Tools for tracking copyright information
No Utility for tracking bugs during course development
Yes
Can create production notes associated with specific development activities or
course objects
No Utility for assigning production tasks
No Can assign ownership of course topics, media, etc., to individuals
No Internal e-mail system to communicate with others working on the project during
development
No Product development timeline tools
No Budget forecasting vs. actual course development costs
Revision Controls, Archiving and File Management
The system uses the version control enabled by CVS, which is a standard Unix-based concurrent
versioning system. The content promotion scripts use CVS as part of the tag/upgrade facility.
Yes/No
Type
No Mark-up and revision controls similar to those found in Microsoft Word
Yes Content can be checked out and checked in (e.g., SourceSafe)
Yes Built-in archiving and back-up system
Yes Tools for tracking original creation date, date of changes and reason for review
(audit trail)
Advanced Search Engine and Metadata Tagging
The metadata tagging facility is planned for the 1.1 release. The search at that point will be at an
item level (course, course section, playable learning object).
Use of XML (tagging, content, both)
Adaptive Tutoring utilizes XML extensively for internal tagging (current), content and internal data
exchange. XML helps manage relatively complex interfaces in a consistent manner. The XML
schema, as enabled recently by the W3C consortium, is a tool to be employed by the industry to
allow true content/metadata level interoperability to take place.


Do Not Reproduce, v1.0  2001 brandon-hall.com 44
Adherence to Standards and Specifications (AICC, SCORM, etc.)
This company attended PlugFest 4 in Virginia in 2001, and it is confident that the data model and
interfaces will meet SCORM 1.2. The formal publication of the SCORM 1.2 specification in
August 2001 will determine its compliance.
Interoperability With Learning Management Systems
The system is currently being integrated with the Boniva Software LMS as a trial interoperability
test. The primary interfaces use the standard Enterprise Java Beans for management and
definition. This enables other systems to find and invoke functional components of this system.
This can occur either at a simple access level—as would be the case from a Web-based/Web
server environment—or at a complex transactional level.
Flexibility and Performance of Delivery Platform

The system automatically generates navigation controls, a table of contents and menus.
Currently, content presentation is regulated by an AI algorithm that adapts presentation based on
user preferences.
Other Features Included in the Delivery Environment
Yes/No
Type
No Built-in synchronous tools (chat, discussion groups)
No Built-in e-mail support (teacher to student, student to student)
No Built-in personal notebook feature (so that learners can take notes as they go
through the course)


Do Not Reproduce, v1.0  2001 brandon-hall.com 45

System Requirements
For the Content Developer:

500 MHz Pentium III processor (or equivalent)
128 MB RAM
100 MB free hard drive space
Windows NT 4.1 (or higher) or 2000
For the Learner:

Internet Explorer 5.0+ (preferred)
Netscape Navigator 4.02+
800x600 resolution
Sound card and speakers
56K Internet connection (or higher—high speed access required for good video delivery)
Server Requirements:

As a general approach, this system was constructed with software that uses most
commonly available operating platforms. As a reference platform, it uses Linux on Intel.
However, it has also completed limited platform tests on Solaris and Windows 2000. The
server requirements are a function of the number of concurrent sessions being served
up. Here is an example:

3-4 concurrent sessions:
1x Intel Pentium III server (2x CPUs at 750 MHz)
512 MB RAM
Min 5GB of disk space(SCSI 2 if possible) – factor of amount of media
Linux Redhat 7.2
PostgreSQL 7.1
Darwin (Media Server)
Cocoon
Apache/JTTY
Jboss 2.1

20-50 concurrent sessions:
Application server:
1x Intel Pentium III server (2x CPUs at 750 MHz)
512 MB RAM
Min 5GB of disk space(SCSI 2 if possible) – factor of amount of media
Linux Redhat 7.2
Cocoon
Apache/JTTY
Jboss 2.1
Database server:
1x Intel Pentium III server (2x CPUs at 750 MHz)
512 MB RAM
Min 5GB of disk space(SCSI 2 if possible) – factor of amount of media
Linux Redhat 7.2
PostgreSQL 7.1
Media Server:
1x Intel Pentium III server (2x CPUs at 750 MHz)
512 MB RAM
Min 5GB of disk space (SCSI 2 if possible) – factor of amount of media
Linux Redhat 7.2
Darwin (Media Server)


Do Not Reproduce, v1.0  2001 brandon-hall.com 46
Cross-Platform Support (for content delivery)
E-Learning content from this system has been tested on the following platforms:
Yes/No
Type
Yes PC
Yes MAC
Yes UNIX
No PALM
No Other wireless devices
Product Support
Available
Type
Yes
Live via telephone and person
No Live, online chat person-to-person technical support
Yes
E-mail support
No Active user support groups
No Online newsgroups
No Getting Started tutorial comes with the tool
No Product conference (live event)
Yes
Vendor-offered training classes on the product available
No Third-party (training partners) classes available
No Third-party tutorials or books on how to use the tool
Yes
Vendor supplies project mentoring services (help with development)
Yes
Vendor offers full outsource courseware development services




Do Not Reproduce, v1