From Library Systems to Mainstream Software: How Web Technologies are Changing the Role of the Systems Librarian

elbowsspurgalledInternet and Web Development

Oct 21, 2013 (4 years and 6 months ago)


Revise: November 5, 2002

From Library Systems to Mainstream Software: How Web Technologies are
Changing the Role of the Systems Librarian

AUTHOR: Arthur Rhyno,

Leddy Library,
University of Windsor

401 Sunset Avenue

Windsor, ON

N9B 3P4

Tel: 519
3000 X. 3163

Fax: 519

ABSTRACT: With the advent of the Web, systems librarians now find themselves managing
or facilitating a wealth of mainstream technologies. Systems wor
k revolves around many
resources that live outside the library’s walls, securing access for our patrons and pushing
access to the desktop of the user. XML is the web technology that is tying together systems,
applications and formats and the possibilities
for component
based applications seem
revolutionary. The resulting need for mainstream IT Web
based skills in addition to traditional
specific technologies is expanding the role of systems librarians and offering a new
world of possibilities.

ORDS: Systems Librarian, Web technologies, XML, Integrated Library Systems, Skills

In a newsgroup posting on August 6, 1991, Tim Berners
Lee described a project called
the “WorldWideWeb (WWW)” that aimed to “allow links to be made to any

anywhere”. The associated prototype software was also offered for free:

If you’re interested in using the code, mail me. It’s very prototype, but available
by anonymous FTP from

Few could have predicted that a simple networked

hypertext linking system would soon
form the basis of much of the world’s online experience. By the end of the decade the
Web had become firmly established as the preferred computing environment for the
public face of most organizations.


For libraries,
the Web was one of several network
based systems to be pressed into
service at the same time as the Internet was changing the rules for widespread
communications. Libraries were utilizing Gopher

and WAIS

while the Web was taking
flight, and had also been

involved in earlier attempts to create public information
infrastructure through technologies such as Telidon
. Yet the Web was different from
these initiatives in that it did not stay on the fringes of the library computing landscape
for very long. A sys
tems librarian could probably be forgiven for not learning how to set
up a Gopher menu or a WAIS index, even at the height of these systems’ popularity, but
the technologies of the Web would become far more pervasive and central to the library’s


Early library web efforts usually consisted of several documents created in HTML
(HyperText Markup Language)
, a format based on SGML (Standard Generalized
Markup Language)

and an easy way to publish a system of “l
inks” known as hypertext.
These links were typically to information on library hours and services, and usually
offered information on how to access the OPAC (Online Public Access Catalogue). The
architecture for the first incarnation of the Web can be seen

in Figure 1.



A Web Server uses HTTP (HyperText Transfer Protocol) to serve HTML documents
located on disk to a web browser.

The simplicity of the web model was a big factor in its early success, but a major shift
ould occur in 1994. As part of the web server developed at the National Center for
Supercomputing Applications (NCSA), the CGI (Common Gateway Interface)

specification was introduced to allow HTML content to be created dynamically. The CGI
specification a
llowed external applications to be wired into the web as shown in Figure 2.
This was another key to the web’s success, CGI programs provided an interactive
experience, and just as importantly, opened the door to plugging in applications that
predated the w



An application is made

available through CGI to any web browser

Two of the earliest library
related CGI programs were CNIDR’s Z39.50 Gateway (Zgate)
and an interface called “WebCat” from the University of Waterlo

that provided a web


layer for the library’s GEAC system. In 1996, SIRSI introduced a web option for its
OPAC and eventually most other Integrated Library System (ILS) vendors would roll out
a web interface to the catalogue, a class of application now al
most universally referred to
as a WebCat.

These web
based interfaces did not initially replace platform
dependent programs in
library systems, and, for that matter, are still not necessarily the only option for accessing
the catalogue. But there is little

doubt that the web has become the primary vehicle for
allowing patrons to interact with the library’s ILS and generally with the library itself in
an online setting. From Interlibrary Loan forms and pathfinders, to virtual reference and
interactive tutori
als, keeping the library’s web offerings running has become an important
part of the library’s operations and, not surprisingly, a core function of the system
librarian’s workload.

Systems librarians have always been called on to gauge the potential of va
technologies for libraries but the web has made this task more complicated. The
technologies that fuel the web truly live in “Internet time”, and keeping well versed in the
latest trends can be a daunting challenge. Web initiatives like Hyper


failed on a grand scale, while others, like VRML (Virtual Reality Markup
, found niche applications. Systems librarians may often find themselves
weighing which web technology makes sense for the library on the basis of both
standing technical “rules of thumb”, such as long term viability and scalability, and more
“web aware” features like available bandwidth and accessibility factors.

The web has added a new emphasis to technology decision
making by bringing the
y’s network environment more firmly into the mix. Deciding how services should
be made available on the web and what is the best way to move content from in
authoring to a community, campus, or global audience, is now an almost intrinsic part of
ing a library’s technology needs.

Web Tools & Toolkits

In the early days of the web, memorizing about two dozen HTML tags and some
versatility with a text editor was about all that was required to establish a web presence,
provided you could download and

get a web server running. Today’s web sites require far
more elaborate building blocks, from bandwidth
friendly image formats to judicious use
of plug
ins, JavaScript, and animations. The good news is that the tools have evolved to
better support the comp
lexity of modern web services. Feature
rich HTML editors and
web editing environments, as well as a high level of web integration in many productivity
applications like word processors, have made these tasks somewhat more manageable.

It is also important

to note that many major aspects of the library’s web offerings are no
longer necessarily the domain of the systems librarian. In particular, there is a high level
of visual design that typically falls outside of technical training. Web managers and
ers now often come from outside of the library’s systems office. Yet if systems
librarians are not always the final arbitrators of web design, they are usually called upon


to help integrate a large and growing amount of local and remote resources into the
library’s web offerings.

This integration may be carried out using scripting languages such as Perl

and PHP
These tools provide the plumbing to tap into the Web for delivering services with “low
barrier” adoption requirements, either by CGI programmin
g or supplying functionality
for processing content like large contents of text to make it more suitable for web

An example of this is setting up a “new book” request form or feedback mechanism on
the library’s web site. This is the type of ta
sk that a scripting language like Perl shines at,
and it is no surprise that libraries often turn to scripting languages for web solutions

In general, web technologies are typically much easier to publicize and share than the
workarounds and tools buil
t prior to the web. In part, this is due to more ubiquitous
networking, it is easier to put out tools and have them discovered and utilized when they
are more accessible. But more importantly, a global community now has much more
commonality when using sof
tware tools for the web. The Perl module for processing
based resources is equally useful to the corporation selling products and may
originate with developers who would not have dreamed that their creations would be
useful to libraries. As well, the r
ise and popularity of Open Source Software (OSS)
which drives much of the web, has also given impetus for developers to “share” more
than ever before.


One of the consequences of both the web and the Internet is that
many of the library’s
resources may now live outside of the library’s walls, and “linking”, as well as other
technologies that minimize the number of hoops required for users to get to a resource,
have become of great interest to libraries. Linking issues
for libraries tend to fall into two
major areas,
, i.e., verifying the user’s digital identity so that they can be
given access, and

link resolving
, where a mechanism is put in place to
ensure that users are matched to the reso
urces that are appropriate to the context of the
link itself.

One of the most common methods of ensuring that users are given access to a remote
resource is via IP authentication, which uses the address or range of addresses associated
with the library. S
ystems librarians now often find themselves managing or facilitating
the use of a “proxy”

and/or “reverse proxy”

mechanism to allow users of different
networks to access content that has been licensed for the library’s user community based
on IP addresse
s or ranges. Proxies are systems that retrieve content from the supplying
vendor or service and hand it over to the user’s application, most commonly a browser.

Another common method of authentication is “referrer” programs, where the library
es a patron and then “passes” them on to the remote service. The server, in
turn, checks that the patron is coming from the “referrer” address, and if so, allows access


to the licensed resources
. Shibboleth
, the authorization framework from the
e Architecture Committee for Education (MACE)

of the Internet2
, represents the next generation of authentication and is a good example of
the Open Source collaborative environment that the Internet has fostered. Internet2 is a
consortium being

led by 200 universities and Shibboleth has been developed to work like
a referrer system, but with much more granularity built
in for access control. Shibboleth
and other initiatives associated with Internet2 could result in a widely deployed toolkit
tioned by the academic and research communities for tackling problems like digital
identity and user rights in an online environment.


and link resolver

systems also have a found role in helping patrons get to
the content they need by making the
process more seamless. With these technologies,
users have a better chance of connecting to the resources that are appropriate to the range
of the library’s licensed content. OpenURL is a standard designed to enable the transfer
of the metadata from a remo
te service to a resolver service. The resolver, in turn,
provides context
sensitive services for the transferred metadata. Resolver systems are a
key piece in solving the “appropriate copy” problem, where a user must be directed
towards the resource that m
akes the most sense for the particular link

represent another important type of linking system , but in this case, help with the
library’s management of the links. For systems librarians, these technologies represent
some critical plumbing
to make sure the library’s user community can benefit from
electronic resources.

In addition to the links themselves, there are numerous systems that have appeared to
improve, stabilize, speed up and lock down the web servers that provide the links, so th
patrons are more likely to retrieve resources without the dreaded “page can not be
displayed” or “404 error” messages appearing
. Caching mechanisms and firewalls, load
balancers and search engines, there are just a few of the many options for enhancing

securing a web site. Depending on the level of IT support available to the library, the
systems librarian will either mediate the deployment of these tools in response to demand
or projected service use, or be knee
deep in rolling out these technologie
s on their own.

One type of web application that deserves special mention is the
application server
Application servers are a type of "middleware" and typically connect databases and other
sources of information in a consistent manner. Middleware, as t
he name implies, sits
between systems and facilitates and/or helps manage their interaction. Although scripting
languages can work real magic in hooking systems into the web, it is often desirable to
have a middle level framework for plugging various types

of content together, and to be
able to provide a consistent means of assigning security levels and other types of
management functions.

One of the most popular application servers is Zope
, which can function as a web server
and has a built
in object da
tabase, as well as supporting document workflow functions for
publishing web sites. Application servers can also have their own scripting environments,
and may be more comparable to operating systems than web servers. Despite the


vagueness in definitions,
application servers represent one of the most promising
solutions for dealing with a multitude of content sources.


If one web technology needed to be emphasized above any other for systems librarians,
that technology would be XML
Every substantive web initiative is now expected to
demonstrate its relationship to XML and the library community has utilized or is
investigating XML for many of its data formats, including MARC and other forms of
, as well as using it for passin
g information back and forth for protocols such
as Z39.50

and NCIP

XML is also an important enabling technology for sharing library information with other
applications. If the library wants to pass circulation information into a patron’s
calendaring s
ystem, for example, XML is most likely to be the exchange format to make
this happen.

Nowhere is the impact of XML more evident than in delivering content on the web. XML
has been employed to define almost every interaction on the web, from how content i
stored and presented to defining formats for publishing web site information. For systems
librarians, XML represents a lingua franca for tying together systems, applications, and

The current and potential uses of XML for libraries requires far
more space that is
available here, but the good news is that XML is probably the most documented
technology on earth. The OASIS Cover Pages

is a good starting point for seeing how
many initiatives are based on XML, but almost any general XML reference wil
l be
enough to gain an appreciation for the number of initiatives that are based on XML, and
will give a sense of the sheer quantity of XML tools that are being developed to facilitate
processing XML content. The table in Figure 3 gives an overview of some

of the most
important XML technologies of interest to systems librarians.

take in “Table I”


based WebCats represent the main intersection between the web and library
systems, but a new
model of delivering applications on the Internet is appearing that
could make the web even more integrated into library operations. This new model is
Web Services
. Although the definitions are not always consistent, a web service
is most often defi
ned as a programmable

that provides a service and is
accessible over the Internet. A component is any piece of pre
written code that defines
interfaces that can be called to provide the functionality that the component encapsulates,
much like a
Lego building block can plug into another to build almost anything.
based software is also very familiar to anyone who has developed Windows


applications. Visual Basic Extensions (VBXs)

are components that are used extensively
to put together Wi
ndows applications.

Of course, there are very different ideas about how this should work on the web between
the big players in the software industry. Each of the major platform vendors has a web
service offering, notably Microsoft's .Net
, Sun's J2EE
, a
nd IBM's Websphere
, but
some common ground is emerging. XML is playing a key role in defining requirements,
behavior, and mediation formats, so that XML toolkits are increasingly being used to tell
systems what to do and to provide a conduit between diffe
rent computer tasks.

In a typical Web services scenario, an application sends a request to a service at a given
URL using SOAP (Simple Object Access Protocol ) over HTTP. The service receives the
request, processes it, and returns a response, with XML pr
oviding the format of the
request (see Fig. 4).

Figure 3

Web Services "plug" into the Web's existing architecture

The most common example of a Web service is that of a stock quote application, in
which the request asks for the current price of a spec
ified stock, and the response gives
the stock price. The requester program uses standard HTTP for sending and receiving this
information. To understand the potential for libraries, consider a SOAP request to the
Barnes and Noble book
pricing engine
. One c
ould envisage this happening from the
Acquisitions module in an Integrated Library System for example. A pricing request is
automatically made at the time of the order and is received without going through the
steps necessary to connect to the Barnes and N
oble service and perform a manual search

An existing web service of interest to libraries comes from Google’s application
development support
. Google offers SOAP access to many of its services, including
direct search, pulling pages from its cache, and t
he ever
popular “Did you mean”
function for suggestions on better search terms. A library could use SOAP to pass terms
from a patron’s search to Google for suggestions, or utilize Google’s cache for pages that
have expired or are unavailable when the patro
n tries to access them from the catalogue.

The library system itself may make a desirable target for Web Service requests. A third
party application could check for a title’s availability in the library or confirm patron
authentication details. Web Servic
es involve directory mechanisms and other
technologies that won’t be touched on here, but systems librarians may soon link to
remote Web Services in the same way that remote web sites are currently linked by



The ILS has been
one of the great success stories of library technology and one of the
main sources of job security for systems librarians. Putting together an ILS has
traditionally required a common code base and a high level of consistency in software
development tools.
The core technologies that built the OPAC usually provide the
circulation system, underline acquisitions, and are often stretched to the limit to provide
extended functions like serials and authority control. For a vendor or in
house systems
staff to maint
ain it all, the ILS can require the care and feeding of many thousands of
lines of code, and walls full of flowcharts and database table layouts. The systems
librarian has traditionally been the ILS vendors’ primary contact in the library, routing
ion between the organization and the ILS technical support group.

At one time, this information transfer was completely within the organization. Libraries
in the 1960s and 1970s often had an “in
house” system that was built using local
programming staff.
Most libraries have long since given up on building or running the
ILS on their own. Although modest in software terms, the ILS market supports a small
but dedicated group of competing vendors. The vendor provides the stability of a
controlled software dev
elopment environment to support and enhance the ILS, while
libraries have the flexibility to pick from several options when purchasing an ILS, and
can define a suitable service and support contract.

Web Services and other web technologies may represent a
benevolent shakeup to this
arrangement. Because of the scope and richness of web technologies, and the
development advantages of component software, big software companies are building
smaller and smaller components that can plug into the web and each othe
r for producing
large and completely “web
enabled” systems. Web Services are often tied in with
software architectures like Enterprise Java Beans (EJB)

that supply even more hooks
for tying components together in a highly scalable and robust environment.
For the ILS
vendor, a new market may emerge which puts a premium on components that supply
specific functionality.

Imagine using a general ledger from a company specializing in business systems for
acquisitions while plugging into a general calen
dar component to handle circulation
duties. The ILS may be viewed less as an integrated system than as an application
framework for bringing together building blocks from both library
aware communities
and developers who have never heard of MARC or Z39.50.

Whether the ILS will become know by a new acronym, such as LAF (Library
Application Framework), is still unclear, and it’s too soon to tell if moving component
based software to the level of such large systems in the library world will be a success,
the results could greatly change how systems librarians work with the library’s
processing needs. Web Services and component software offer the enticing possibility of
mapping more than catalogue and the web publishing side of the library into web
gies, these new approaches may expose all of the library’s processes to the web.



Library systems represent a fairly small marketplace in software terms, and system
librarians have spent much of their time learning how to

manage and, in some cases,
extend applications created by each library vendor’s development team. As a result of
this, systems librarians have often dealt with bizarre terminal emulations, numerous
networking environments, and deploying and maintaining sp
ecialized software across
different platforms for the library’s user community. The web and its associated tools
represent a much
preferred method of delivering and customizing library applications.

Systems librarians can now utilize “general” web traini
ng materials and opportunities,
and find plenty of occasions to apply these skills to their day
day activities. This
mainstreaming of technical skills does not mean that libraries will not continue to have
unique automation needs, but it seems likely t
hat the web will continue to foster more
broadly supported technologies for tackling library problems. This is no way will narrow
the role of the systems librarian, and it will continue to be necessary for systems
librarians to have a deep understanding of

the library’s activities in order to apply web
technologies effectively to the library’s operations and services. The difference is in the
range of software solutions that systems librarians have at their disposal. The web has
changed the role of the syst
ems librarian from dealing with a preponderance of niche
technologies for niche tasks to working with a huge palette of largely mainstream

Yet the web has also raised user expectations. The instant gratification that comes with
web browsing
and the high level of web
design skills and constant learning to maintain
savvy skills has required that systems librarians closely follow the latest web trends.
The web may not have made the life of systems librarians easier, but it has moved them
ser to the experiences of IT workers in other organizations by allowing the use of a
globally built and commonly used toolkit. The possibilities for innovation in this
environment seem endless, and thanks to the web, there has probably never been a more
citing time to be a systems librarian.


XML Technology


XSLT (Extensible Stylesheet
Transformation Language)

Separating content from presentation is an essential step in
dealing with widely competing opinions on web page design.
XSLT brings i
n a suite of related technologies, including
stylesheets and XSL
FO (Extensible Stylesheet Language
Formatting Objects). Taken together, these technologies lay the
groundwork for providing flexible output to the meet the needs of
a variety of audiences.

OAP (Simple Object Access

This XML
based protocol allows various content types to be
passed between systems. It deserves special mention not only
because of its role in Web Services (see below) but also because it
greatly expands the range of co
ntent that can be passed between
systems on the web. If you have ever stuffed endless parameters
into a URL or dealt with complex CGI programs that utilize many
variables, you can appreciate the advantages of an XML packing
mechanism for passing informatio
n between programs.


Ant is an XML Java
based program building tool. If you are
lucky, all applications needed for your library will come with
seamless installation tools. For large systems that must be built
from source files, Ant can avoid the endle
ss grief associated with
other types of compilation tools. Ant delivers the flexibility XML
provides for documents to developers and is a good example of
the high quality, Open Source tools made available by the Apache

RDF (Resource Descriptio
/XTM (XML Topic

Few topics raise more debates around the W3C than the Semantic
, and RDF and XTM are just starting points for a multitude
of mapping and metadata initiatives and debates. Despite this,
libraries have the longest trac
k record on the planet for delivering
metadata and may be able to make good use of these tools when
the smoke clears. RDF can accommodate a variety of metadata
while XTM offers a mechanism for exploring “relationships”
between resources. If creating RDF ca
n be compared to MARC
cataloguing for the web, XTM is closer to authority work. Neither
is for the faint of heart when it comes to syntax and the tools are
still evolving, but if a common standard for encoding metadata
could be integrated into the web, it
would allow libraries to share
records and skills honed over decades of cataloguing.

XHTML (The Extensible HyperText
Markup Language)

Take all the presentation advantages of HTML and combine them
with the structure of XML, and you end up with XHTML. Even

with XML standards for every conceivable document type,
libraries will inevitably have lots of unstructured content that will
be well suited for HTML. With XHTML, this content will be
easier to manage and plug into XML tools.

Table I

XML technologies
of interest to systems librarians



Tim Berner
Lee's entire 1991 message can be seen at:



See the Webopedia definition of Gopher and introductory links at:



Background on WAIS can be found at the Webopedia entry:


Telidon was a Canadian precursor to the Internet that came to an end in

1985. A good overview is at <http://w>.


See the W3C HTML Page: <>.


A good starting point for SGML is <>.


The CGI specification is documented at <>.


These an
d other early attempts at web/library system interaction are listed

at <>.


An overview of Hyper
G written at the height of the system's popularity can

be seen at <>.


A good i
ntroduction to push technology from
Web Review

in 1996, courtesy of the Wayback

Machine can be found at


VRML and it's succesor technology, X3D, have a home
at the Web3D

Consortium: <>.


Perl has been called the “duck tape” of the web and there are numerous resources available to learn more
about it. O’Reilly has brought many of the sites together at <>.


The official
PHP site is <>.


One good example of the community that has grown up around scripting languages and libraries can be
witnessed on the Perl4lib list: <>.


There are many sources for Open Source information, b
ut a great starting point for OSS and libraries is
the oss4lib site: <>.


A quick overview of proxy servers can be found at


By far the most popular reverse proxy use
d by libraries is EZproxy, see


See Tim Kambitsch's excellent overview of referrer solutions at


The main site for Shibboleth is at <>.


See <>.


A brief overview of Internet2 can be found at <>.


OpenURL is documented at <>.


See Andy Powell’
s examples in “OpenResolver: a Simple OpenURL Resolver”, in


See the background links available at MARS Hot Topics Discussion Group “OpenURL and Link Servers
(SFX, Openly Jake) Take Aim at the ‘Approp
riate Copy’ Problem” at


PURL information and a PURL server can be found at <>.


For a unique view of the 404 error messages, see <>.


An overview from

can be found at


The Zope home page is at <>.


The W3C site for XML is <>.


See the new resources and the link to the “XML,

MARC and Digital Librarianship” bibliography at


For example, the XML support in the Bath Profile <http://www.nlc
current.htm> and the
"Z39.50 Next Generation" initiative at <


See <


Found at <>.


W3C Activity on Web Services can be found at <>.



An introduction to component software can be found at


The Webopedia definition of a VBX can be found at <>.


The main .Net page is at <>.


The main J2EE pag
e is at <>.


Websphere is targeted to developers and is described at <>.


The Barnes and Noble book pricing engine is described by Annrai O'Toole in "SOAP: Disrupting the
balance of power",

Web Service
s News & Analysis
, Oct. 2001:


See Google’s list of options at <>.


This section draws from an article I did for


The main EJB site is <>.


The W3C site for XSLT is <>.


The W3C site for SOAP is <>.


Ant is part of the Apache Jakar
ta project and is available at <>.


The W3C site for RDF is <>.


Version 1.0 of the XTM specification can be found at <>.


W3C activity on the Semantic Web starts at <h


The "Reformulation of HTML 4 in XML 1.0" can be found at <>.