A Fame interface for Perl and Tcl Fernando Trias August 9, 1995 A draft prepared for Fame derived from work performed at the

ugliestharrasΛογισμικό & κατασκευή λογ/κού

4 Νοε 2013 (πριν από 3 χρόνια και 1 μήνα)

83 εμφανίσεις

A Fame interface for Perl and Tcl

Fernando Trias

August 9, 1995

A draft prepared for Fame

derived from work performed at the

Federal Reserve Board, Washington, DC

from 1993 to 1995

Author's permanent address:

7151 SW 103 Ct. Cir.

Miami, FL 33173


Current address:

132 1/2 Oxford St Apt 6

Cambridge, MA 02140


Current email address:



This paper describes the design, implementation and use of a general

purpose interface between Fame and the widely available programming

languages Perl and Tcl. The interface was developed us
ing a macro

package which helps generates appropriate "glue" for Tcl or Perl from

the same source files. This paper also describes several applications

created using these interfaces.



This paper is divided into four parts. Th
e first is the introduction

which describes the justification for this paper. Then, the interface

is described. The third section shows various examples of the use

of this utility. Of special interest is the example of how this

technology was used in co
njunction with World
Web technology to

offer a uniform method for accessing macroeconomic data at the Federal

Reserve Board. After the conclusion, the apendicies give more

technical information about the interface and tools used to create it.


First, it is useful to ask what benefits Perl or Tcl will give you

beyond the Fame 4GL and the Fortran or C HLI.

Tcl is a general purpose, easy to learn, extensible scripting

language. Tcl was designed to be a central mechanism where differe

extensions or modules can be executed and share data. The most

important of these extensions is Tk, which provides a simple common

interface to various windowing environments such as Unix X
Windows or

Windows. There is also a BLT extension which pr
ovides numerous

facilities for generating charts and graphs dynamically.

Fame already provides its Fame
Windows package which, although not as

widely used or flexible as Tk, is sufficiently powerful for many

applications. However, many Tcl extensions exi
sts which access Sybase,

Oracle, Ingres and other database products, so that Tcl offers a

significant advantage when the program must manipulate data from

various source. Also, since Tcl is a scripting language it is very

efficient when the data must
be processed through various different

programs. Furthermore, the file and string processing capabilities of

Tcl are very powerful (though not as powerful as Perl's).

Perl is another general purpose extensible scripting language which

has more core fea
tures than Tcl and was designed, among other things,

for text processing. Many extensions are also available which allow

access to Informix, Ingres, Interbase, Oracle, Sybase and others. The

Tk extension is also available for Perl, but not as widely used
. Perl

has many facilities for sorting information and converting between

different binary and text data formats (see the tape loading example

far below).

It can be argued that Fame performs most of the functions of these

languages. However, it does so
with much difficulty and very slowly,

particularly reading files and manipulating strings. Fame's strength

lies in its representation of time
series data, its reporting

facility, the analytical tools the Fame 4GL provides, and many other

areas. However,
use of Perl or Tcl has the advantages described above

which I have sought to exploit through the various examples shown on

this paper.



For the purposes of this discussion, I will call the Perl interface

FamePerl and

the Tcl interface FameTcl.

Fame provides a way to access its data through a series of C or

Fortran libraries known as the HLI. FamePerl and FameTcl are

adaptations of the C HLI library for the calling conventions of these

languages. This adaptation is
generally performed by adding a

"wrapper" or "glue" around each of the C HLI functions which converts

values to and from the format of Perl or Tcl. This section describes

the interface is some detail.


1. All but 6 C HLI version 7.5 functions a
re fully implemented in

native Perl and Tcl code. Not implemented: cfmlsts, cfmrdfa,

cfmrrng, cfmrsts, cfmwrng, cfmwsts.

2. For implementations with the FRB extensions (various

utilites created at the Federal Reserve), all hli_* functions

are implemented fully. Famedbpath is also implemented.

3. New Perl
like and Tcl
like functions have been added (see Fame


4. Values of hli.h are available.


The typical C HLI function is translated directly into Perl with

le modification. For example, to call cfmddes from C HLI, you

would use:


Where "status" and "desc" are defined according to the manual as

"output" variables, and "dbkey" is an input variable.

In FamePerl, you would use:


Perl makes no distinction between pointers and values. Thus, all

variables (whether output or input) are specified the same way. Perl

takes care of modifing those values it has to modify. The data type

is not importa
nt as this is handled dynamically by the language.

In FameTcl, you would use

cfmddes status $dbkey desc

Since Tcl does make a distinction between pointers and values, you

must pass the name of the "output" variables "status" and "desc" and the

f "input" variable "dbkey".


There are exceptions to this rule. Functions which use a range or

translation table (or other array with a limited and fixed length) are

called with one variable for every array element. Thus, cfmsrng woul

be called from C as:


where range is a three element array. From Perl, each element of range

must be specified. Thus, you would use the following:



The functions which require this type of array expansion are:

cfmrstr, cfmsbm, cfmsfis, cfmsnm, cfmspm, cfmwstr.

Functions which access unlimited length arrays such as cfmrrng

cannot be implemented using this scheme an
d therefore are

unavailable in both FamePerl and FameTcl. However, replacement

functions which are functionally equivalent, but called differently,

have been added.


FamePerl and FameTcl provide utilities for converting the values of a

C header fi
le into an appropriate header file for the language, so

that the values of all the constants are available.


There are several functions to assist Perl users in rapidly retrieving

and updating Fame data. Since most of the C HLI functions

that write

or read data are not implemented directly, programs should use the

functions described in this section.

When missing value translation is not specified, missing values are

returned as string "NA", "ND", "NC" if the object is numeric; this


evaluate to a zero in a numeric context. For strings, "" is

returned. Likewise, writing a value of "NA" to a numeric series will

store an actual NA. However, there is no way to store NA strings.

The list of functions below is ordered in a special way.

The first

line is an example of the FamePerl format, the second line is FameTcl

format, and the indented lines that follow are a summary description.



Initialize the Fame hli



Terminate the Fame hli

bkey = &fameopen($name);

set dbkey [ fameopen $name ]

$dbkey = &fameopen($name,$mode);

set dbkey [ fameopen $name $mode ]

Open a Fame database. If the database is not there, then try

to open it in the current directory. Returns
1 on error.


fameclose $dbkey

Close database.

$type = &famegettype($dbkey,$objnam);

set type [ famegettype $dbkey $objnam ]

Get an object's type

$freq = &famegetfreq($dbkey,$objnam);

set freq [ famegetfreq $dbkey $objnam ]

Get an object's f

@list = $famegetinfo($dbkey,$objnam);

set list [ famegetinfo $dbkey $objnam ]

Get object information as in cfmwhats. The following table

describes the order of the elements returned by famegetinfo:

Name #



class 0

type 1

freq 2

basis 3

observed 4

first year 5

first period 6

last year 7

last period 8

reate year 9

create month 10

create day 11

modify year 12

modify month 13

modify day 14

description 15

documentation 16

@data = &fameread($dbkey,$objnam,$syear,$sprd,$eyear,$

set data [ fameread $dbkey $objnam $syear $sprd $eyear $eprd ]

Read data for the given range of dates.

@data = &famereadn($dbkey,$objnam,$numobs,$range1,$range2,$range3


set data [ famereadn $d
bkey $objnam $numobs $range1 $range2

$range3 $tmiss $mistt1 $mistt2 $mistt3 ]

Read a given number of value starting with the values of a

given range. Get the rangex values with the functions

cfmsrng, cfmsfix. This function wil
l read the whole range of

values, but only store numobs of them. You can get the

value of numobs when setting the range. The missing value

variables must contain data even if no translation is being



famewrite $dbkey $objnam $year $prd $list

Writes out all the elements of @list into the object staring

with the given date.




FamePerl was used to develop programs internally at the Federal

eserve Board (Fed) for the World
Web (WWW) which allow users to

search for or browse through virtually all of the public macroeconomic

data at the Fed. All of this data resides on Unix servers, and until

this project, could therefore only be accessed

from Unix. Using

FamePerl and WWW technology, users on PCs, Macintoshes, and other Unix

computers can now access all this data easily through the graphical

interfaces of a WWW Browser such as Mosaic or Netscape.

Because the WWW has become such a popular

and widely discussed

subject, I will assume the reader has a rudimentary understanding of

WWW concepts.

How does it work?

Without going into much detail, I will try to explain the components

of the system and how they are integrated. The project has t
wo simple

goals. First, users must search for data. Then, they must be able to

view the data and extract it.

There is only one way to extract data, but there are two principal

ways of searching for data: through tables and by searching for



Tables are text files which contain lines of text and pointers to

specific Fame object names. These tables are modeled after existing

printed tables which are produced by the various agencies or academic

institutions. Users can either perform
a string search through the

titles of all the tables, or browse through one of three indexes.

The first index displays all tables according to the organization

within the Fed which is responsible for gathering and maintaining each

table. Each table has o
nly one maintaining organization so that there

is a one
many relationship between each index entry and each table.

The second index displays all tables according to general economic

concept. For example, "Interest rates" contains all the tables which

have any data pertaining to interest rates. Clearly, a table may fall

into more than one category, so that there exists a many


The last index shows tables according to country of origin. Again, a

table may have data from different
countries, so the relationship is


Users may begin by choosing one of the three indexes, and then one of

the many index entries. Then, a list of the titles of all applicable

tables is shown. Each table title is a link to the actual table, s

that the table contents are displayed if a user selects the title.

Users can also perform string searches on the titles.

If the user displays and browses a table by clicking on the title,

each Fame object reference in the table is turned into a link to

object data (see Object data).

All these indexes, hypertext links, and files are generated using



There is also a form which will allow users to enter complex search

queries which will be applied to the documentation and description

attributes of all public Fame objects. (For the technically curious,

this data is extracted daily into a flat file using FamePerl; also, a

simple query cache mechanism was implemented to speed up multiple

similar queries.)

Once the search is applied, all

the matching objects are displayed.

Users may perform further subqueries on this list. Each data object

is shown by its object name, description and documentation. The

object name is a link to a the object data.

Object data

If a user selects a link t
o an object's data, the server executes an

extraction program written in FamePerl which quickly displays the

object's system attributes and user attributes. On the same page the

user can extract the data values by clicking on a button after

optionally ent
ering a start date, end date and output format.

This data can then be extracted from the browser program and

manipulated on a spreadsheet, file, or other program using the system

specific features, such as Cut/Past, OLE, DDE, etc.


Another co
mmon application is the use of the text
processing power and

speed of Perl to extract data from tapes, disks or files, and convert

it into a form suitable for importing into Fame, Sybase, Oracle, or

another database for which a Perl interface exists. Perl


several advantages over C or Fortran. First, Perl is an interpreted

language which means that it can be easily transported from one

platform to another. Second, Perl provides arbitrary length strings

as well as many simple facilities for conver
ting data in binary format

(numbers, dates) of one computer into the format of another. Along

with the Unix utility dd, byte swapping, EBCDIC conversion, blocking

and case conversion are very simple. There are many other technical

reasons which are beyon
d the scope of this paper.


Many database extensions have been written for Perl. The most

prominent of these are Informix, Ingres, Interbase, Oracle and Sybase.

With the Fame extension, Perl can be used to easil
y extract data from

any source and save it into any target, including, of course, text file

and the screen using Perl's report generation facilities.

A lot of database extensions have also been written in Tcl. Tcl could

be used to create a general data b
rowser or updater.

Third party vendors also provide gateway products through common

workstation databases for DB2, IMS and other Mainframe databases.



Both FamePerl and FameTcl were developed at the Federal Reserve Board

in Was
hington, DC by the author over the course of a year. FameTcl

uses Tcl version 7.4; FamePerl is available for version 4.036 and

version 5. Information about these programs, as well as the source

code, may be obtained from the Freedom of Information Office

of the

Fed. I am also working on a new version for Perl version 5 which may

be available shortly from me.



Fame provides excellent support for time
series data, analysis, report

generation, graphing, and many other areas. However
, its text

capabilities, scripting support and Windowing system are very limited.

FamePerl and FameTcl provide significant advantages in these areas.




This is a technical section describing th
e interfaces. It will

include information from a paper I wrote, but never published because

it is mildly obsolete. I will fax you a copy since I no longer

have an electronic version [insert paper 1 here]


The following C HLI fu
nctions have not been implemented: cfmlsts,

cfmrdfa, cfmrrng, cfmrsts, cfmwrng, and cfmwsts.

The following C HLI functions have arrays which are flattened (that

is, a different variable must be used for each array element), or

functions with slightly modi
fied calling convention.

Name Parameter



cfmgatt value

cfmrstr range

cfmsatt value

cfmsbm range

cfmsdm range

cfmsfis range

cfmsnm nmistt

cfmspm range

cfmsrng range

cfmwstr range, strary

The f
ollowing C HLI functions are version dependent and may not be

available when linking with older versions of Fame.

Name Version



cfmawl 7.5

cfmopre 7.5

cfmopwk 7.0

cfmrdfr 7.0

cfmrdsr 7.5

cfmrdst 7.0



cfmrng 7.0

cfmsinp 7.0

cfmsopt 7.0

cfmssln 7.5

cfmwtfr 7.0

cfmwtsr 7.0

cfmwtst 7.0