Higher Education Open and Distance Learning Knowledge Base for Decision Makers

kneewastefulΤεχνίτη Νοημοσύνη και Ρομποτική

29 Οκτ 2013 (πριν από 3 χρόνια και 10 μήνες)

140 εμφανίσεις




Higher Education

Open and Distance Learning
Knowledge Base

for Decision Makers







Study prepared by the

European Distance Education Network





for UNESCO,


Information Society Division











Final Report



Higher Education Open and Distance Learning Knowledge Base for Decision Makers


EDEN


UNESCO

Study

Final Report






2






Fee Contract No. 402.063.1


Terms

of Reference, Phase 2



WORK PLAN


for the development of a future prototype


ODL knowledge base and management system





The report was assembled by the EDEN Secretariat,


with the co
-
ordination of

Dr András Szűcs, Secretary General,


based on the contributions of

Dr Pál Vásárhelyi

Budapest University of Technology and Economics


and

Dr Nick Farnes, Director

International Centre for Distance Learning, (ICDL)

The Open University, UK





Budapest, Fe
bruary 2002

Higher Education Open and Distance Learning Knowledge Base for Decision Makers


EDEN


UNESCO

Study

Final Report






3




INTRODUCTION

................................
................................
.........................

7

Interpretation of work

................................
................................
................................
........

7

Relation to the Country Surveys

................................
................................
........................

7

1.

DEVELOPMENT OBJECTIV
E

................................
...............................

8

2.

IMMEDIATE
OBJECTIVE
................................
................................
.......

9

3.

EXPECTED OUTPUTS

................................
................................
.........

10

4.

ACTIVITIES

................................
................................
...........................

11

4.1

System analysis and design

................................
................................
........................

11

4.2

Development of data bases
................................
................................
.........................

11

4.2.1

Bibliographic data

................................
................................
................................

11

4.2.2

Full text

................................
................................
................................
................

11

4.2.3

Factual information

................................
................................
..............................

11

4.3

Implementation of information repackaging

................................
...........................

12

4.4

Implementat
ion of a semantic processing technology based tool for the three
selected domains

................................
................................
................................
.........

13

4.5

Implementation of an ontology driven document enrichment system

..................

13

Stages of work

................................
................................
................................
..............

13

Deliverables
................................
................................
................................
..................

14

4.6


Supporting the comparative analysis of options and evaluation of

developments by the means of using a Group Decision Support System

..............

15

4.6.1

The aim
................................
................................
................................
.................

15

4.6.2

Activities

................................
................................
................................
..............

15

4.7

Implementing an expert system regarding ODL policy and/or quality

assurance

................................
................................
................................
.....................

16

4.7.1

The aim
................................
................................
................................
.................

16

4.7.2

Basic definitions used for the implemen
tation of the expert system
....................

16

4.7.3

Activities

................................
................................
................................
..............

17

4.8

Implementation of a system providing controlled feedback and up
-
grading

.......

18

4.9.
Preparation of report

................................
................................
................................
.

18

4.10

Evaluation

................................
................................
................................
...............

18

5.

IDENTIFICATION OF TH
E TARGET AUDIENCE AN
D THEIR NEEDS
FOR KNOWLEDGE BASE S
ERVICES

................................
...............

18

Aspects for the definition of the elements of the expected user profile

........................

18

Higher Education Open and Distance Learning Knowledge Base for Decision Makers


EDEN


UNESCO

Study

Final Report






4



6.

DETAILS REGARDING IN
FORMATION REPACKAGIN
G
ACTIVITIES

................................
................................
...........................

20

6.1

Policy
................................
................................
................................
............................

20

6.2

Quality insurance mechanisms and accreditation procedures for ODL

...............

21

6.3

IT in ODL

................................
................................
................................
....................

21

7.

WORKLOAD

................................
................................
.........................

22

8.

COSTS

................................
................................
................................

23

8.1

Semantic processing technology

................................
................................
................

23

8.2 GDSS


Group Decision Support System
................................
................................
..

23

8.3

Expert system

................................
................................
................................
..............

24

9.

RECOMMENDATIONS ON T
HE FURTHER DEVELOPME
NT AND
INTERNATIONALISATION

OF THE KNOWLEDGE BAS
E AND
STRATEGIES FOR MOBI
LISATION AND TRAININ
G OF USERS

.

24

9.1. Survey

of motivations and requirements for successful international

partnerships in ODL

................................
................................
................................
..

24

9.2

Decisive factors of ODL policies, to be supported by the international

dimension
................................
................................
................................
.....................

25

9.3

Sustainibility and d
issemination

................................
................................
...............

25

9.4

Bottlenecks

................................
................................
................................
..................

27

ANNEX 1
-

QUALITY ASSURANCE

S
teps leading to the identification of existing quality assurance mechanisms and
accreditation procedures for ODL, as well as resources

for collecting information

on
ODL
provision

................................
................................
.......................

28

INTRODUCTION

................................
................................
.......................

28

1. METHODS, PROCEDUR
ES

................................
................................

28

1.1 Assessing the quality of institutions and courses

................................
......................

28

1.2

Assessing the quality of ODL information

................................
...............................

29

1.2.1

Quality of the source and content of information

................................
................

30

1.2.1.1.Quality of Electronic information

................................
................................
....

30

1.2.1.2 Assessing

the quality of printed ODL information

................................
..........

31

1.2.2

Quality issues related to information management

................................
..............

33

2.

RESOURCES

................................
................................
........................

36

2.1

Selected institutions

................................
................................
................................
....

36

Higher Education Open and Distance Learning Knowledge Base for Decision Makers


EDEN


UNESCO

Study

Final Report






5



2.2

Selected experts
................................
................................
................................
...........

38

2.3

Sources of information on ODL

................................
................................
................

38

2.3.1 Types of information to be covered

................................
................................
........

38

2.3.2 Data elements to be recorded

................................
................................
..................

38

Illustration 1
-

Description of a correspondence college
................................
..............

39

Illustration 2
-

Description of an MBA course
................................
.............................

39

Illustration 3
-

Description of an ODL research article
................................
................

39

2.3.3 Sources of information to be covered

................................
................................
.....

39

2.3.3.1 Selected Sources of information related to literature

................................
.......

39

2.3.3.2 Selected Sources of information related to institutions and courses

................

40

ANNEX 2
-

KNOWLEDGIST

................................
................................
..

42

1. SEMANTIC PROCESSI
NG

................................
................................
..

42

2. THE WAY IN WHICH
THE SYSTEM WORKS
................................
....

42

3. SELECTED DETAILS
FOR THE WAY IN WHICH

THE SYSTEM
WORKS

................................
................................
................................
.

44

3.1 Processing Local files

................................
................................
................................
..

44

3.2 Web Search

................................
................................
................................
..................

45

4.

DEMONSTRATION OF KNO
WLEDGIST APPLICATION

IN ODL
DOMAIN
................................
................................
................................
.

47

ANNEX 3
-

COBRAIN
................................
................................
.............

48

SEMANTIC PROCESSING
OVERVIEW

................................
.................

48

COBRAIN FEATURES

................................
................................
.............

49

END
-
USER MODULE

................................
................................
...............

49

Basic Search Mode

................................
................................
................................
............

49

Semantic Search Mode

................................
................................
................................
......

50

Advanced Boolean Search Mode
................................
................................
......................

51

Viewing and Selecting Knowledge Bases
................................
................................
.........

52

Any Field Search
................................
................................
................................
................

52

Expanded Search

................................
................................
................................
...............

52

Login for Confid
ential Knowledge Bases

................................
................................
........

52

Plug
-
In

................................
................................
................................
................................

52

Higher Education Open and Distance Learning Knowledge Base for Decision Makers


EDEN


UNESCO

Study

Final Report






6



Document Processing

................................
................................
................................
........

52

Lotus Notes Domino Server Processing
................................
................................
...........

53

ANNEX 4
-

ONTOLOGIES, KNOWLEDG
E MODELS, WE
BONTO
AND APPLICATION IN T
HE FIELD OF ODL

................................
.....

54

1.

DEFINITIONS

................................
................................
........................

54

1.1

Ontology

................................
................................
................................
......................

54

1.2

A knowledge model
................................
................................
................................
.....

54

1.3

Software tools

................................
................................
................................
..............

55

2.

APPLICATION OF THE B
ASIC CONCEPTS TO THE

FIELD OF ODL
POLICY

................................
................................
................................
..

55

3.

EXAMPLES OF QUESTION
S FOR A KNOWLEDGE MO
DEL IN THE
FIELD OF ODL POLICY

................................
................................
.......

56



Higher Education Open and Distance Learning Knowledge Base for Decision Makers


EDEN


UNESCO

Study

Final Report






7




Introduction

Interpretation of work

The
present Final Report follows the Interim Report submitted in December 2001 and
contains, in accordance with the Term of Reference, the
detailed
Work Plan
for the
development of a future prototype ODL knowledge base, which includes steps to:

a. identify res
ources for collecting information on ODL provision such as experts,
institutions and information centers;

b. identify existing quality assurance mechanisms and accreditation procedures for ODL;

c. identify the target audience and their needs for knowledge

base services, such as
relevant decision makers involved in the provision of tertiary ODL;

d. develop structure and functionality analysis for a future prototype knowledge base and
its associated support services, along with specifications of the platform

to host the
prototype, including recommendations on further development and internationalization
of the knowledge base and strategies for mobilization and training of users.


We have organised the different above components around the Work Plan, because i
n this
way we could make clear the role of all contributions in relation to the complete work
envisaged.


Some of the issues specified by the terms of the Contract individually were included in the
structure and text of the Work Plan itself, e.g.:

-

identifi
cation of the target audience and their needs for knowledge base services

-

recommendations on the further development and internationalization of the knowledge
base and strategies for mobilization and training of users


The topics related to Quality assuran
ce, including the steps leading to the identification of
exisiting quality assurance mechanisms and accreditation procedures for ODL, as well as the
resources for collecting information on ODL provision were dealt with in Annex 1, because
we considered th
is component of the material as partial implementation of the Information
Repackaging activity proposed in the Work Plan.


Relation to the Country Surveys

UNESCO has, in frames of its Higher Education ODL Knowledge Base Project, as part of the
Preparatory
Activities in 2001, initiated a series of needs' analysis in target regions in Africa,
Latin America and Caribbean, Asia and the Pacific, the Arab States Region and in CIS
countries. We have taken into consideration the experience and methodology reflected

in
these surveys.


From the reports, a certain variety of approaches and potential national implementation
scenarios can be seen. They are quite different in the extent and strategy of the public
involvment and also concerning the positions and business
approaches of the corporate sector,
and naturally also in the background in human capacities and traditions, further with regard to
access to the information technology infrastructure.


Higher Education Open and Distance Learning Knowledge Base for Decision Makers


EDEN


UNESCO

Study

Final Report






8



The country surveys are useful in identifying a number of bottlenecks
which are different and
characteristic country by country meanwhile have several things in common. They can serve
as good basis for the elaboration of recommendations and actions with joint relevance.


Common issues which show clear significance from the p
oint of view of the present project
are (i) the clarification concerning the understanding of open and distance learning and
information technologies, (ii) the identification of elements of reliable professional
information and databases further (iii) path
ways of appropriate decision making on strategic
levels.


The country studies meanwhile demonstrate a lot of information which are helpful in offering
the possibility of systematic interpretation and highlighting possible future scenarios. This
could be pa
rticularly efficiently used in the light of results with recent European Union
observatory and benchmarking programmes in the field of ODL and eLearning.


It is worth here making a reference to another valuable pool of experience, gained in
somewhat simila
r economical and political conditions, namely the Phare Multi
-
Country
Distance Education Programme of the European Union (1995
-
1999) which aimed the
implementation of open and distance learning in 12 countries, in a not very well developed
region of the wo
rld. EDEN was strongly involved in this programme exercise (cf: Keith Harr
y

(ed): Higher Education through Open and Distance Learning Routledge, London and New
York


The Commonwealth of Learning, 1999 A. Szűcs and J. Jenkins: p. 227 and ODL
networking in
Europe and the experience of the East
-
West co
-
operation. in: A.R. Trindade
(ed): New Learning Universidade Aberta, Lisbon 2000, Lajos, Szűcs, Grementieri and also:
Policies And Practice In Open And Distance Learning: The Phare Multi
-
Country Programme
For D
istance Education 1995
-
1999 Kay Mac Keogh, Hans
-
Peter Baumeister, in: Proceedings
of the EDEN Research Workshop, Prague, 2000
http://www.eden.bme.hu/papers/EDEN
-
Prague.pdf
)


It seems entirely sen
sible to ensure the synergy between the experience drawn from the
UNESCO country surveys and the achievements of the present project approach. In this
respect, it is highly relevant to rely on the support and legitimation potential of the
international pro
fessional community to verify the pertinency and reliability of the knowledge
base and its services.

1.

Development objective

The long term development objective is to create an IT
-
based system supporting the widest
range of activities related to the developm
ent of ODL including the provision of value
-
added
services.


The IT
-
based system will include different kinds and levels of data bases, such as
bibliographic, full text, numerical/statistical, etc, and different types and levels of information
processing
and management tools, such as information repackaging, semantic processing
technology, expert system, knowledge management system, etc.


In other words the long term objective is to create a support system, which:

-

enables searching using the key dimension
s in the field


agencies, institutions, people,
publications, courses, projects etc, and organised by region, country, level, subjects,
types etc. so that decision makers could for example, find out what is going on in ODL
Higher Education Open and Distance Learning Knowledge Base for Decision Makers


EDEN


UNESCO

Study

Final Report






9



in a particular country at vocat
ional level; or which institutions are experts in drop
-
out
research; or which teachers have experience in developing mathematics education at a
distance, etc.

-

guides various types of users, such as decision support staff in policy development,
ODL system
and content developers,etc. through the stages and steps leading to the
final product expected from them, e.g. a national or institutional ODL development
policy, providing simultaneously access to the information needed in order to complete
the task.


The

achievement of the development objective will be the result of a step
-
by
-
step process,
starting with the elaboration of a demo version, which will be limited both in coverage and IT
technologies used. The duration of the process is expected to be of 5,5 y
ears, covering the
Medium term programme period of UNESCO.

2.

Immediate objective

The immediate objective is to create a demo system:

-

allowing to discuss/evaluate the various approaches proposed

-

which is operational and can be filled with practical content

-

offering IT tools which can be both directly used and further developed, extended for
different purposes related to ODL.


It is understood, that the content, coverage, target audience, services, etc. of the demo system
will be proposed, nevertheless open

to further development. The duration of implementation
will be 1,5 years.


Regarding the target audience the demo version will be aimed at:

-

policy makers, in particular at national level

-

those involved in different aspects of quality assurance in the fiel
d of ODL

-

those involved in developing methods and tools for ODL, in particular IT applications
at institutional level


Regarding the services provided the demo version will be aimed at demonstrating and testing

selected data bases and tools, in particular:

a)

the creation of a bibliographic data base covering policy and methodology related
material

b)

the creation of a full
-
text data base to include most important, basic material selected
making use of the bibliographic data base

c)

the compilation of data on select
ed experts, institutions, tools, case studies, etc

d)

the demonstration of using information repackaging technics for the purpose of
implementing guidelines and support tools for three target groups in three specific
domains:

-

the preparation of guidelines ba
sed on a state
-
of
-
the
-
art report for those involved
in the elaboration of a policy regarding the development of ODL in their country,
including the presentation of the procedures to be followed, the methods to be
used

Higher Education Open and Distance Learning Knowledge Base for Decision Makers


EDEN


UNESCO

Study

Final Report






10



-

the data required with their potential

sources, resources of knowledge and
experience, which can support the policy development

-

the preparation of guidelines based on a state
-
of
-
the
-
art report for those in charge
of developing and applying quality assurance mechanisms

-

the preparation of a re
view with evaluation and examples regarding IT tools
applied in/recommended for ODL

In all above mentioned cases

-

the repackaged information will include practical elements, such as the
identification of selected experts, institutions, information resourc
es, cases
of application and practical experience, etc.

-

information repackaging will make use of the data bases established under
a), b), c) above.

e)

the demonstration of using semantic processing technology, which creates a structured
index of problems an
d solutions on the basis of reading and analysing various types of
documents, in particular those found in the above mentioned full
-
text data base and
specific Internet searches.

f)

the demonstration of using expert system technologies, including both freewar
e
-
and
commercial software based examples. These technologies will make use of the products
resulting from the above mentioned semantic processing.

g)

the demonstration of integrating feed
-
back in the use and further development of the
system


From the point
of view of content, the demo system will cover the three selected domains of

-

national policy development,

-

quality assurance mechanisms and accreditation procedures

-

methodology, in particular the use of IT
-
based methods and tools in ODL.


Regarding geogr
aphic coverage, the demo version will focus on, but will not be limited to
three countries representing typical levels of development, problems and solutions:

a./ UK

b./ South Africa

c./ Hungary

3.

Expected outputs

The Project is expected to produce the follo
wing outputs:

-

a system of operational software tools, allowing the provision of both "traditional" and
innovative, value
-
added information services

-

a content elaborated at a demonstration level in three selected domains of ODL
development, allowing

-

the un
derstanding the ways in which advanced information support can be
provided

-

the enriching of the system with additional knowledge

-

a system designed for the collaborative collection, evaluation and use of the
information for ODL development purposes

Higher Education Open and Distance Learning Knowledge Base for Decision Makers


EDEN


UNESCO

Study

Final Report






11



4.

Activiti
es

4.1

System analysis and design

This will include

-

in depth analysis of the ways in which the selected user groups can be supported in the
three selected domains with information

-

in depth analysis of the IT tools considered for application, including discussi
on
with/training by the software provider

-

the design of the subsystems, with their outputs, inputs and intellectual/computerised
processes,

-

the design of the integrated system, in which the subsystems co
-
operate in the most
efficient way, including any ne
cessary modification of the basic design of the
subsystems


4.2

Development of data bases

4.2.1

Bibliographic data

Identification of basic documents in the three domains of the system, on the basis of

-

personal experience

-

search on the Web

-

search in on
-
line da
ta bases

4.2.2

Full text

-

selection

-

copyright arrangements

-

scanning of selected parts of text from hard copies and downloading selected documents
in electronic form

4.2.3

Factual information

1.

Identification of resources for collecting information on ODL p
rovision, such as
experts, institutions and information centres

-

Experts

will be identified making use of

-

the knowledge and experience of the members and partners of EDEN as a most
comprehensive European open and distance learning association, with act
ive
database on members and relevant experts in the field. EDEN members and
partners (see also the European ODL Liaison Committee, www.odl
-
liaison.org)
will be requested to nominate one expert per domain covered by the system
and per country, known to have

practical experience and to be willing to
provide co
-
operation and assistance

-

the EDEN Network of Academics and Professionals (NAP) list and (screened)
conference speakers/resource persons databases (see also:
http://www.eden.bme.hu/contents/nap.html

and
http://www.eden.bme.hu/contents/publications/100.html
)

These data bases will be searched by author, and analyses will be undertaken

ranking authors by 'productivity' and 'popularity' ie the number of papers in the
database over a period, and the number of hits each entry receives.

Higher Education Open and Distance Learning Knowledge Base for Decision Makers


EDEN


UNESCO

Study

Final Report






12



Further analysis of keywords in the abstracts will give a profile of experts
based on publications and ar
eas of work.

-

the bibliographic data, which will be used to identify most relevant authors,

-

Institutions

will be identified making use of

-

the ICDL Providers data base

-

searching literature databases by Institutional Affiliation to locate institutio
ns
employing authors who have published in areas specified by search words.

-

creating a facility which will enable institutions to submit information about
themselves and update existing records, subject to quality control.

-

subject
-
oriented searches on

the Web, identifying the institutions most active in
the domains covered

-

Information centres

will be identified making use of

-

the ICDL data base

2.

Definition of the data elements to be collected

3.

Collecting information on

-

Experts
. Data will be co
llected by the means of

-

contacting them and obtaining data on their background/experience, as well as
statement regarding their availability to assist others

-

Institutions
. Data will be collected by the means of

-

contacting them

-

I
nformation resourc
es
. Data will be collected by the means of

-

further work on the information centres identified in the Interim Report,
including a careful evaluation and selection

4.

Organising information for use


4.3

Implementation of information repackaging

In the fields
of policy development, quality assurance and IT applications, the following steps
will be undertaken:

-

Analysis

-

in
-
depth study of the documents selected under 4.2

-

preliminary sorting on the basis of content and evaluative criteria

-

selection and extraction
of the most relevant elements of information

-

structuring the selected elements

-

Synthesis

-

comparative arrangement and merging of extracted information

-

comparative evaluation of extracts

-

presenting eventually conflicting information in synthesis

-

compression
of the information, if required

-

Preparation of state
-
of
-
the
-
art reports regarding policy development and quality
assurance, and evaluated review regarding the use of IT
-
based technologies, adding new
elements, such as evaluation, etc. as required


Higher Education Open and Distance Learning Knowledge Base for Decision Makers


EDEN


UNESCO

Study

Final Report






13



4.4

Impleme
ntation of a semantic processing technology based tool for the
three selected domains

(Note: this activity will be implemented making use of two software tools to be acquired
from the Invention Machine Co: Knowledgist and CoBrain)

-

acquisition of the softwa
re tools, including negotiations regarding the financial and
legal conditions, to be conducted by UNESCO

-

organisation of the information available in full
-
text form, including the output of the
information repackaging work, as required for the use of the C
oBrain and Knowledgist
software tools, paying attention to their requirements, such as eliminating question
forms, "bullets" and special signs for structuring texts, etc.

-

reading and analysing the information available, automatically creating a semantic
in
dex of documents presented in a problem
-
solution format

-

application of the software tools for processing information resulting from Web
-
searches

-

analysis of eventual problems arising from the use of synonyms and the up
-
grading of
synonym lists


Details on
Knowledgist, a demonstration of applying Knowledgist to two ODL related
documents and the characteristics of CoBrain are presented in Annexes 2, 2a and 3.


4.5

Implementation of an ontology driven document enrichment system

This will be done regarding the dom
ain(s) of IT applications in ODL, i.e. using an ontology to
represent (code) good practice in the above mentioned domain, using formatted databases of
case studies, etc.

(Note: this activity will be implemented making use of the WebOnto software to be
prov
ided free of charge by the Open University, UK.)


This sub
-
project is within the context of a broader project to create computer based services to
support all kinds of activities related to the development of ODL and provide value
-
added
services. The sub
-
p
roject involves the construction of a prototype knowledge base which will
demonstrate how enable users can acquire and apply knowledge in the domain of ODL
Policy. Knowledge and step
-
by
-
step guides will assist users in undertaking a range of tasks
and comp
leting products (eg. drafting ODL policy documents). The focus here is on
demonstrating how knowledge engineering can be applied to a particular domain of ODL.


Stages of work

1.

Arrangements for the acquisition of the software tool and obtaining practical
ex
perience

2.

Define and analyse user needs in terms of representative tasks and products in the
domain of ODL policy

3.

Review and select relevant and representative documents

4.

Analyse documents and derive main concepts, structure and procedures related to the
tas
ks

5.

Develop an ontology to represent the concepts, categories, procedures, relations and
structure

6.

Develop a template (ie. with defined fields) for the extraction and formatting of
content from the documents to assist in completing tasks

Higher Education Open and Distance Learning Knowledge Base for Decision Makers


EDEN


UNESCO

Study

Final Report






14



7.

Define categories/c
odes for tagging the elements in each field

8.

Create a repository/database for storage and access to templated information

9.

Extract information from documents, mark up content for each field, tag/code content
in each field, enter data into template, upload in
to the database

10.

Use an automatic coding tool based on the ontology to represent the content of each
entry in a knowledge base

11.

Develop step
-
by
-
step guides to assist users in applying knowledge to undertake tasks
and produce products

12.

Code guides and upload i
nto database

13.

Define and program algorithms for linking knowledge base to guides

14.

Create interface to present knowledge, guides, and space for users to undertake tasks
towards constructing their product

Deliverables

This work would lead to the production of
the following deliverables:

1.

A bibliographic database giving details of selected documents covering ODL policy

2.

A searchable database of formatted information derived from analysis of selected
documents

3.

An ontology in which issues of ODL policy can be articu
lated

4.

A knowledge base capable of supporting enquiries and providing knowledge to assist
with a range of tasks

5.

Step
-
by
-
step guidelines enabling users to apply knowledge in undertaking tasks

Optional


6.

Full
-
text versions of copyright approved documents

7.

Synth
esis and comparisons drawn from the knowledge base

8.

Listing and profiling of experts in domain of ODL policy

9.

Representative source documents (eg. examples of policies) from agencies and
institutions


The explanation of the way in which the concept and appr
oach of an ontology driven
document enrichment system can be applied to the field of ODL Policy is presented in

Annex 4.


Further development could include facilities for collaborative online working in the context of
knowledge and guides, and direct subm
ission and encoding of experience and expertise.


Parallel work would include the selection and indexing of online information sources, and the
application of agent technology to extract and synthesise information.


Ongoing evaluation can be made of the re
lative cost
-
benefit of search engines of varying
sophistication (eg. searchword, keyword, thesaurus, semantic and associative linking) applied
to unformatted or information formatted in different ways, compared to a carefully
constructed ontology and forma
l knowledge base. Also advances in the automatic encoding of
information in documents using an ontology to extend the knowledge base may lead to large
cost reductions.


Higher Education Open and Distance Learning Knowledge Base for Decision Makers


EDEN


UNESCO

Study

Final Report






15



4.6


Supporting the comparative analysis of options and evaluation of
developments by
the means of using a Group Decision Support System

All ODL related decisions involving the comparison of different options and /or the
evaluation of the progress achieved, requiring the co
-
operation of a certain number of experts,
can be supported by an ap
propriate Group Decision Support System (GDSS).

(Note: This activity will be implemented using the TenderExpert system to be provided by
the Hungarian copyright
-
owner. This system is Oracle
-
based, allowing several sessions to
be run simultaneously and sev
eral dozens of specialists to work on each of the problems. A
new version allowing access via Internet will be available during the Project period)


4.6.1

The aim

The aim of this sub
-
project is to help a team of decision
-
makers in solving problems and
maki
ng choices by the means of a software tool: a GDSS. The proposed domains of
application are: developing an ODL policy and developing quality assurance mechanisms.


This tool is targeted in particular to supporting groups of several dozens of persons in
performing group decision
-
making tasks. It is an interactive, computer
-
based system, using
heuristic and quantitative models to support decision
-
making. The analysis of problem
situations is based on the contribution of experts.


The application of a GDSS
is expected to improve group work, in particular in the fields of
policy development/implementation and quality assurance/accreditiation by the means of:

-

enabling all participants to work simultaneously (human parallel processing), thereby
promoting broad
er input into the meeting process and reducing dominance of the
meeting by a few people

-

providing equal opportunity for participation (through anonymity)

-

enabling larger group meetings which can effectively bring more information,
knowledge, and skills to
bear on the task

-

providing process structure to help focus the group on key issues and discourages
irrelevant digressions and nonproductive behaviors

-

supporting the development of an organizational memory from meeting to meeting

-

providing also an expandin
g knowledge base over time.


The application of the system can be characterised as follows:

-

a leader (faciliator) coordinates the meeting

-

each participant has a computer workstation

-

computers are networked and client/server architecture is used

-

specializ
ed software is available to all participants


4.6.2

Activities

The following activities will be implemented in order to judge the applicability of the GDSS
tool:

1.

Organisational arrangements for a demo application to be made in Hungary in the
framework of
the Project's Evaluation Meeting

2.

Preparation of the Demonstration version, including

Higher Education Open and Distance Learning Knowledge Base for Decision Makers


EDEN


UNESCO

Study

Final Report






16



-

making decisions on the

-

domain of application

-

the types of the decisions to be supported

-

definition of the appropriate attributes of the decision tree (set of cri
teria)

-

definition of the weights for the criteria

-


elaboration of the utility functions

3.

Implementing a demonstration session, presenting

-


the way in which the system with the pre
-
established decision tree, etc. can be
used during a decision support m
eeting, including the definition of the weight of
the experts and voting rights

-


the way in which the criteria, utility functions, etc. can be developed and/or
extended by the participants of the decision support meeting

4.

Evaluation, recommendations for t
he future continuation of the work

5.

Arrangements regarding the operational use of the system by UNESCO and its Member
States, for which several options are available, including an Internet
-
based solution.


4.7

Implementing an expert system regarding ODL pol
icy and/or quality
assurance

(Note: this activity will be implemented making use of the commercial tool GURU to be
acquired from Micro Data Base Systems, Inc. USA)


4.7.1

The aim

In the case of this sub
-
project the aim is to implement a
system, which can o
ffer intelligent
advice (but not take an intelligent decision) about the ways in which policy, and/or quality
assurance mechanisms for the development of open and distance learning can be elaborated,
leading decision makers and university staff in charge o
f this matter to the most appropriate
sources of information, methods, tools, experience gained, partners, etc.


A desirable additional characteristic would be the capability of the system, on demand, to
make explicit, explain its own line of reasoning in
a manner directly intelligible to the
inquirer.


4.7.2

Basic definitions used for the implementation of the expert system

The work will be based on the following definitions:

-

the
goal

is to model the reasoning processes of an expert in a well defined prob
lem
domain

-

the
domain
, is the definition of the specific activities/decisions the system is expected
to support, e.g. analysis of …. selection of…

-

the
domain expert

is a knowledgeable person within a specific problem domain.

-

the
knowledge engineer

acts a
s a consultant, who is in charge of capturing information
from the domain expert, applying the extracted information to build a prototype expert
system for the domain expert to test and critique and developing and refining the system
to completion

-

the
end
user

is the person who uses the expert system to obtain advice or to accomplish
a task.. He/she is also a data resource!

Higher Education Open and Distance Learning Knowledge Base for Decision Makers


EDEN


UNESCO

Study

Final Report






17



-

a
Rule
is a piece of reasoning knowledge that tells what action to take if certain
conditions are satisfied. A rule's conditions are re
ferred to as the rule's premise and its
actions are referred to as the conclusion. A rule consists, therefore, of a premise and a
conclusion.


Sets of IF
-
THEN rules define the ways in which the action to be undertaken by the user
can be specified depending

on the description of a specific problem, step by step.


4.7.3

Activities

The following activities need to be implemented:

1.

Making decisions on the

-

domain

-

role of the expert system, which may be an advisor, an assistant, a back
-
up or the
program taking fu
ll responsibility

-

the types of expected outputs, including in particular

-

questions requiring some answer from the user, defining the problem to be
solved, the goal to be achieved

-

pieces of useful information to be provided in answer to the questions

-

a list

of problems
,
the solution of which is to be supported by the system

-

the approach to be used between the goal and the raw data input, which may be
goal
-
driven, relationship
-
driven or data
-
driven.

2.

Acquisition of the appropriate software tool, including cons
ultation on its efficient use

3.

Acquisition of necessary elements of knowledge from human experts, by the means of

-

interviews

-

analysis of documentation on the ways in which information processing and
decision making is being performed by human experts

-

analy
sis of the steps undertaken during the performance of a task by those in
charge, by the means of verbalizing the steps needed to accomplish a specific task

-

questionnaires

-

formalizing the specific process by appropriate means, such as flow charting

4.

Defining

the problems and related solutions. This includes the identification of the
participants, resources and goals related to each of the problems to be solved

5.

Definition of concepts to represent knowledge, including the formal expression of the
elements assum
ed to exist in the problem area and their relations. Definition of sub
-
goals that must be reached in order to obtain the master goals related to each of the
problems to be solved

6.

Formalization of the key concepts outlined during the previous (conceptualiza
tion)
stage by the means of representing the knowledge in a dependency diagram (this will
serve as a guide for implementing the rules to describe the expert's knowledge). This
will include, among others

-

assigning variable names to represent components of t
he problem

-

resolving how data are acquired

-

formulating question the system will have to present to the user

-

establishing the certainty factors for the data (variables)

-

deciding on requirements for the results of the consultation by the user

Higher Education Open and Distance Learning Knowledge Base for Decision Makers


EDEN


UNESCO

Study

Final Report






18



7.

Building rule s
ets

-

associating rules with the relationships drawn in the previously described
formalization phase

-

specification of rule premises, including variables, functions, operators…

-

specification of desired actions, including external actions (e.g. consulting
other rule sets, statistical analysis, report generation, etc)

-

organisation of rules into small, easily managed modules: sets containing a
collection of rules that can be consulted to receive advice about various problems

-

creating as many rules and rule
sets as needed to represent each sub
-
goal of
the problem

-

connecting these related rule sets together

-

editing rule sets

-

debugging rule sets

8.

Tailoring the consultation environment for the expert system

9.

Verification, involving

-

testing the expert system aga
inst a standard data set

-

considering its reasoning behaviour


4.8

Implementation of a system providing controlled feedback and up
-
grading

-

designing a system for co
-
operative, distributed collection of data and contribution to
the development of content for

various subsystems

-

specification of a software tool for the management of input and processing of the
above mentioned contributions

-

implementation of programming work

-

testing with concrete inputs


4.9.

Preparation of report

-

presentation of the system

-

gui
de for users of the demo version

-

recommendations regarding the next stages of development


4.10

Evaluation

-

by the means of Internet
-
based testing and discussion

-

in the framework of an expert meeting

5.

Identifi
cation of the target audience and their needs

f
or knowledge base services

Aspects for the definition of the elements of the expected user profile

1. The the target group of users of (particularly electronic) open and distance learning and
their needs in general
-

but also equally that of related decis
ion makers
-

is extremely and
increasingly diverse. Therefore identification of users of the knowledge base and its related
services requests a comprehensive and complex approach.


Higher Education Open and Distance Learning Knowledge Base for Decision Makers


EDEN


UNESCO

Study

Final Report






19



2. When considering the approach of these decision makers, it is certainly
worth noting that
between the providers and users of education, the relations are increasingly based on
accountability of the institutions, including correct interest


cost


benefit relations, therefore
rather the pragmatic concerns than general mission
statements play significant role in the
decisions. ODL producers have to work with users (especially enterprises) in order to develop
learning materials which are suitable for contextualisation and correspond to perceived needs.


3. Besides providing effic
ient access to a relevant pool of appropriatel
y structured and reliable
knowledge, t
he task with the knowledge base and ODL decision making includes necessarily
a sort of strong communication and awareness raising element about electronic open and
distance

learning: what it can be used and also what and in which situations certailly not
-

and
the target groups have to be identified in accordance with this requirement. This element is
very clearly reflected in the country reviews prepared in frames of the UN
ESCO ODL
knowledge base project.


4. Due to the nature of the subject itself, there is a need for permanent re
-
positioning of
learning in the newly contextualised environment, including the increasing and always more
sophisticated use of information and co
mmunication technologies and the change of the social
context as well, such as lifelong learning, continuing education, etc.


5. There is a need for contribution to the establishment of a new "professional culture" for
people involved, from different persp
ectives and various institutional levels, in the definition,
implementation and assessment of public initiatives, measures and policies oriented towards
the development of ODL and its integration in mainstream education and training systems.


5. Experienc
e shows that besides the expressis verbis professional elements, an essential
dimension of appropriate implementation of open and distance learning


in all levels and
environments


is the increased emphasis on the sustainibility, management and economic
aspects of the development of education, including efficient international liaions, private
-
public partnerships, dissemination of systematic knowledge about issues of realistic and
sustainable operation, resourcing and planning. Managers, financial experts
, fund
-
raising and
corporate relations executives of universities therefore can well be in the potential target
group.


Policy
in this sense therefore includes a wide range of subjects of comprehensive nature and
needs proper understanding and competences
about issues/fields of ODL.


Policy makers
for the purpose of identification of the target group using the achievements of
the present project: the knowledge base and the associated services, have been understood as:


From the angle of field of activity:

d
ecision makers of different levels, dealing with the subjects of

-

higher education

-

public education

-

Open and Distance Learning

-

educational policy

-

employment

-

vocational training

-

adult education

Higher Education Open and Distance Learning Knowledge Base for Decision Makers


EDEN


UNESCO

Study

Final Report






20



-

corporate and professional training

-

home and lifelong learning

F
rom institutional angle

-

institutions of government, particularly Departments of Education, but also

-

the officials of increasing number of countries established national information
technology agencies and “e
-
Ministries”

-

national educational institutions

-

un
iversity leaders, including those responsible for institutional strategy development
and involved in decisions about comprehensive educational developments and
investments on institutional (University) or faculty but even on department level

-

professional b
odies and associations

-

national and international organisations and associations

-

Policy Research Institutes and University research groups

-

Standards and Technical Policy Organizations

-

educational accreditation bodies

-

decision makers in the corporate sect
or, particularly in the fields of information and
communication technologies but also leading officials and decision makers in human
resource development and company training


One further potential particular group in higher education may prospectively be
teachers
requiring staff development. Increasingly in Europe and later in developing countries,
emphasis is being/will be given to staff development, training and ultimately registration of
university teachers. Naturally the use of ICT and DE concepts and
quality feature
significantly in the content of this training. One crucial skill of competent staff need is being
able to use information sources to keep abreast of the field and of innovation in teaching.
Similar steps are being taken for administrative s
taff and their continued professional
development.

6.

Details regarding information repackaging activities

The activities will aim at creating documents covering the following elements:


6.1

Policy

(Note: This material will deal with various aspects of policy fo
rmulation and approval,
and will not go into details of implementation and operation)


Definition of ODL policy

Objectives of a national ODL policy

Need for a national ODL policy

Goal

Key issues

Features

Methodology and procedures

Main characteristics

Prec
onditions for the formulation

Major phases

Documents

Higher Education Open and Distance Learning Knowledge Base for Decision Makers


EDEN


UNESCO

Study

Final Report






21



Background document

Policy issues

Institutional profiles

Procedural documents

Official approval procedures

Implementation and operation

Examples


6.2

Quality insurance mechanisms and accreditation proce
dures for ODL

(Note: This material will focus on quality assurance and not go into details of
accreditation procedures)



It will cover the following:

Methods, procedures


Assessing quality



of institutions and courses



of information


Quality assurance

procedures



definition procedures




in public sector




in private sector



monitoring procedures




in public sector




in private sector


Schemes for accreditation


Evaluating and describing quality



regarding institutions and courses



regarding inf
ormation

Resources


Selected institutions

agencies assuring quality of ODL



recognition and examining bodies for ODL courses


Selected experts

Examples


Charters, quality contracts


Codes of practice, quality marks

ISO standards

Case studies


Elements for

illustrating the structure and content of the document to be produced as a result
of information repackaging are presented in Annex 1.


6.3

IT in ODL

This product will be a further developed version of the study prepared for the current, first
stage of the Pr
oject.

Higher Education Open and Distance Learning Knowledge Base for Decision Makers


EDEN


UNESCO

Study

Final Report






22



7.

Workload

Most of the tasks to be implemented are extremely labour
-
intensive and more or less time is
needed according to the information
-
richness, scope and coverage of the various subsystems
envisaged. Therefore, for some activities the time requ
ired for completion can be estimated,
for other activities we present an estimate regarding the minimum time needed in order to
develop a menaningful content (for demonstration, explanation, discussion purposes rather
than for operational use), understandi
ng, that the complete coverage of the topic would need
at least twice as much time. The estimates for the implementation of the nine groups of
tasks/activities as follows:



1.

System analysis and design, including the co
-
ordination

of the activities impleme
nted under the various subprojects

required

2 m/m

2.

Development of data bases, including bibliographic,

full
-
text and factual data bases in three subject fields:

mimimum

6 m/m

3.

Implementation of information repackaging regarding

three subject fields






mimimum

6 m/m

4.

Implementation of a semantic processing technology based

tool for the three selected domains:




required

2 m/m

5.

Implementation of an ontology driven document enrichment

system regarding the domain of information technology

applications







minimum

8 m/m


Estimations of workload in % for the stages leading to the development of a prototype knowledge
base on IT applications in ODL (understanding that the work will be spread over 12
-
18 months):


Work

%

Task definition and analysis

6

Docume
nt search and analysis

16

Development of ontology

16

Development of
template/database etc

10

Information extraction, formatting
etc

15

Writing and coding user guides

16

Programming

15

User interface development and
testing

6

Total

100


The ontology

for this particular domain will have to be designed as a subset of a larger one capable
of covering the whole field of ODL. To include further domains will require other areas of the
ontology to be extended and elaborated. Additional domains will also inv
olve further definition of
tasks; document search and analysis; information extraction, formatting and coding, preparation of
guides and so on. While there will be some savings as the knowledge base is extended, it is likely
that each additional domain wil
l involve around 70% of the resources required for the development
of the content of the first application.

6.

Implementing a GDSS application




required


2 m/m

7.

Implementing an expert system regarding one domain of

ODL








minimum

10 m/m

Higher Education Open and Distance Learning Knowledge Base for Decision Makers


EDEN


UNESCO

Study

Final Report






23



8.

Implementation

of a system providing controlled feedback

and up
-
grading, i
ncluding programming



required


4 m/m

9.

Preparation of report including user guide



required


2 m/m

10.

Evaluation, including preparation and holding of expert meeting



1 m/m

8.

Costs

Some of t
he software tools and related services (consultations) have to be acquired against
payment, noting that UNESCO can expect substantial reductions, in particular if solutions are
proposed allowing the vendor to benefit from the prestige of a UNESCO applicati
on


8.1

Semantic processing technology

Regarding Knowledgist and CoBrain:

-

one day training with reduced cost for educational institution


225 Euro

-

following preliminary discussions with the Invention Machine Corporation,

the acquisition of the software

tools could be envisaged in two stages

-

the first stage will cover the period of the Project for which the present Work Plan
has been elaborated (planned to cover the years 2002
-
2003), i.e. the development,
implementation and discussion/evaluation of a pil
ot version. The cost of using the
two software tools required for piloting the semantic processing technology could
be negotiated on the basis of initiating a joint project with the Invention Machine
Corporation

The indicative prices would be:

-

Knowledgist








8,000 USD

-

CoBrain: Here the Vendor would not apply its price list, and

would allow, in the context of the joint project, a limited number

of simultaneous users accessing the system from institutions

located all over the world, with a view to di
scussing and

evaluating the solutions proposed, rather than for

practical applications







50,000 USD

-

a second stage would cover the practical application in all institutions located in
all interested Member States, (planned to cover the years 2004
-
200
5)
understanding that the number of users benefiting from
simultaneous

access
would remain limited to 15, nevertheless they can be located in any institution of
the Member States. The price of this stage of application should be discussed
between UNESCO's

services and the Vendor.


8.2 GDSS


Group Decision Support System

The GDSS TenderExpert is available free of charge for demonstration, testing and evaluation
in Budapest, Hungary. It is recommended, that the demonstration and evaluation of the GDSS
Tende
rExpert be implemented in the framework of the evaluation meeting, when experts from
different regions are present. In this case the sub
-
project will have no software cost.

The ways in which the system

-

can be acquired by UNESCO for operational use at the
HQ and/or in Member States, or

-

can be used by the Member States through the computer facilities available in Budapest
should be negotiated by the partners after the demonstration and evaluation.


Higher Education Open and Distance Learning Knowledge Base for Decision Makers


EDEN


UNESCO

Study

Final Report






24



8.3

Expert system

Regarding GURU, as long as UNESCO can con
firm that the project is non
-
commercial, non
-
profit, MDBS is prepared to offer a 75% discount on list pricing and a 50% discount on
Runtime pricing.


The list price for a single user version of the GURU Development System is $7,000 and the
network version
is $15,000. UNESCO's prices would be $1,750 and $3,750 respectively. The
list price for GURU Runtimes is $1000; UNESCO's discounted price would be $500. It
should be noted, that MDBS does not extend discounted pricing to Service and Maintenance.
One year o
f Service and Maintenance costs 20 % of the list price of the Development System.

9.

Recommendations on the further development and
internationalisation of the knowledge base and strategies

for mobilisation and training of users

Regarding the successful u
se of the knowledge base, we largely recommend to consider the
assumptions emphasized under the description of the target group and their needs. Based on
systematic experience from similar projects, the following set of further aspects are proposed
to take

into account:


9.1. Survey of motivations and requirements for successful international
partnerships in ODL

Experience suggests a number of lessons have already been learned about what makes for
successful international partnerships:



common and shared nee
ds: the right course/program for each partner



academics that know and respect each other



a common language



complementarity: each partner has something to offer that the other partner does not
have and needs



start
-
up funding: making the initial contacts, ag
reeing on content, negotiating the
agreement



reliable common technology: this is needed both for negotiating and inter
-
institutional
administration, as well as for course delivery



strategic positioning: given the costs of developing partnerships, they need

to be
strategic rather than ad hoc, and incorporate a range of activities, rather than isolated
cases; while small
-
scale pilots may be necessary to convince other academics or the
university administration, they need to be planned as part of a larger sche
me.



institutional leadership, patience, individual motivation, and academic rewards


Resource: Dr. A.W. Bates:

Technology, distance education and national development

A paper presented at the ICDE World Conference in 1997

http://bates.cstudies.ubc.ca/icde/icde.html


Higher Education Open and Distance Learning Knowledge Base for Decision Makers


EDEN


UNESCO

Study

Final Report






25



9.2

Decisive factors of ODL policies, to be supported by the international
dimension:

Policy orientations influencing public initiatives to support ODL and learning technologies:



the c
oncern to innovate education and training systems by increasing accessibility and
flexibility of learning opportunities, to rationalise expenditure and to introduce an
investment perspective in the education/training expenditure;



the concern to add an int
ernational/European dimension to national education and
training provision;



the concern to guarantee quality of supply and to "protect consumers" against bad
practice and unreasonable expectations;



the concern to develop a market area for information and

communication technologies


Enhancing factors for the success of ODL policies:



political stability



positive attitudes towards innovation at large expressed by target populations



the consolidation of a habit to concertate policy decisions



the fact that
education/training organisations are exposed to a certain level of
competition and do not feel unconditionally guaranteed in their continuity



insecurity of education and training organisation, however, is both on enhancing factor
for the uptake of ODL and
an objective cause of difficulty in making long term plans
for development.


Approaches to policy assessment, in general term as well as in relationship to the specific
ODL domain:



to measure accountability/achievement of a public policy



to develop a proce
ss of collective learning through participated monitoring off all
activities, resources and outcomes. (NB: this is particularly appropriate in the specific
domain of ODL, that is characterised by a relative novelty, lack of consolidated
experience among in
volved actors, high fluidity of needs, behaviours and contexts)



to measure effectiveness of implementation and management approach


Resource: ODL Policy Seminar


http://www.policy.odl.org/index.html


9.3


Sustainibility and dissemination

Relying on the achievements of a recent, very relevant related European Union ODL
programme (SUSTAIN), which was implemented under the co
-
ordination of Scienter, Italy,
with the participation of EDEN, as essential constit
uents of further development and
internationalisation should be identified
sustainibility and dissemination.
The key elements
of this approach are illustrated below.

Higher Education Open and Distance Learning Knowledge Base for Decision Makers


EDEN


UNESCO

Study

Final Report






26



DISSEMINATION AND SUSTAINABILITY

























THE SUSTAIN
IMPLEMENTATION MODEL























Resource: The SUSTAIN Dissemination Guide:

http://www.sustain.odl.org/index.html


O

U

T

P

U

T

Diss
emination

SUSTAINABILITY:

Visibility

Networking

Policy consensus

User feedback

Funding (public or
external)

Official recognition

Competent staff

Human resources
commitment

Achieved
sustainability of
related output

etc..

Time

Dissemination

1

Awareness of Project

Identity and Outputs



4A

Assessment

of Action


2

Analysis of Surrounding
Environment (targets,
competitors, policy

developments, etc.)



4

Action


3

Feasibility

D & S

Actions



2F

Future Scenarios


Higher Education Open and Distance Learning Knowledge Base for Decision Makers


EDEN


UNESCO

Study

Final Report






27



9.4

Bottlenecks

A major obstacle to the use of available ODL resources is the

lack of a consolidated practice
in trans
-
national transfer and adaptation of ODL products, but also the lack of rules and
practical solutions to allow the re
-
use of products developed thanks to public funding, (the
property of which is mixed): they consti
tute a huge amount of learning resources that are not
systematically utilised just because no solution has been found for their distribution.



Higher Education Open and Distance Learning Knowledge Base for Decision Makers


EDEN


UNESCO

Study

Final Report






28




ANNEX 1

QUALITY ASSURANCE

Steps leading to the identification of existing quality assurance
mechanisms and ac
creditation procedures for ODL, as well as
resources for collecting information on ODL provision

from Dr. Nick Farnes, Director, ICDL

Introduction

Quality assurance mechanisms and accreditation procedures relate to:

-

institutions and courses

-

the informatio
n related to them


It should be noted, however, that the definition of an ODL course is problematic. Should
open learning packages without formal assessment, and courses involving flexible learning
with a mix of conventional and distance learning, be incl
uded? What about a publisher
providing materials for independent study towards a public examination? There has been an
explosion of online components being introduced into conventional courses


are these now
distance learning courses? How crucial is the '
open' component
-

are only courses open to
everyone irrespective of previous qualifications and experience to be included? Are courses
available to students in other countries? What about in
-
company open learning which may
only be open to employees? Some d
istance learning courses or modules may be part of a
programme and not available separately.


The definition of ODL
-
related information is similarly difficult. There are key books, articles,
journals, conferences etc which deal exclusively with ODL. Howeve
r, there are many others
which may touch on ODL related issues or issues which might be included under a broader
definition of ODL. For example a journal with articles purely about computers would be
outside the field, but another on computers and educatio
n might include some articles
specifically about ODL. Similarly journals on education might also have occasional articles
on ODL or which are relevant to ODL. Other books and conferences may also include ODL
issues or deal with new areas that are becoming
relevant. As with the difficulties of defining
an ODL course so accounts of these developments raise difficulties about their relevance to
the field and to a particular information service.

1. Methods, procedures

1.1 Assessing the quality of institutions
and courses

Information about accreditation and quality assurance related to particular institutions or
courses is of crucial importance for information managers wishing to maintain a quality
service. This information forms the basis for decisions about wh
ether to include institutions
and courses. Most information services avoid taking on direct responsibility for accreditation
and quality assurance of ODL itself which is best left to the relevant authorities. However, a
service can find itself in a positio
n where course providers demand to be included or ask why
Higher Education Open and Distance Learning Knowledge Base for Decision Makers


EDEN


UNESCO

Study

Final Report






29



they are not included. In these cases reference to selection criteria based on recognition by
relevant authorities reduces accusations of subjectivity or arbitrariness.


Providers of ODL include wel
l known institutions specialising in these kinds of courses (eg.
open universities, correspondence colleges), but also include other institutions which offer
mainly conventional teaching (eg. HE institutions, further education colleges), and
organisations
providing other services as well as ODL courses (eg. publishers, companies
with training departments, professional bodies, agencies).


Quality assurance procedures affecting ODL may be defined by law, monitored through
inspection and involve accreditation

and authorisation of course providers, and be carried out
by government, official agencies and professional bodies. Where quality assurance is the
responsibility of an official body or quality assurance agency, it might be run by government
ministries, st
atutory bodies, professional associations, training providers, or by groups of
employers. In the private sector where organisations are operating on a commercial basis or
providing in
-
company courses, government regulation may not apply. Quality assurance
in
this sector can be a private initiative and the procedures vary. They may include 'charters' or
'quality contracts', membership of coordinating organisations operating 'codes of practice' or
'quality marks', or adopt ISO
-
standards which assure quality
of provider and process, but not
of output (skills and qualifications).
In most countries higher education institutions are subject
to accreditation procedures and their courses quality assured. In which case the decision about
whether to include an HE ins
titution offering a ODL course in an information service is
relatively easy compared to other levels where regulation is more haphazard and diverse.


Various schemes for accreditation, recognition or membership require providers to apply
procedures for ass
uring the quality of their courses. Also particular courses may be recognised
or examined by external bodies who accept the learning outcomes as meeting their standards


1.2

Assessing the quality of ODL information

For information services their overall qualit
y depends on their comprehensiveness within their
defined field, the quality of their sources, the content of the information and information
management. An ODL information service would be poor quality if the field it covered was
unclear, its sources were

misleading and unreliable, it did not cover all relevant sources and
that the process of collection, editing, coding, formatting, checking and updating the
information was unsystematic, inconsistent, and prone to errors.