Hiding legacy software using Perl and SOAP as glue

Hiding legacy software

using Perl and SOAP as glue

Alasdair Allan

University of Exeter, Exeter, U.K.

Abstract

The

worst

nightmare

a

software

developer

can

face

is

a

major

re
-
engineering

of

a

mature

system
.

We'd

all

much

rather

sit

down

and

write

new

code

than

carry

out

software

archaeology

on

a

half

understood

system

written

by

a

dozen

different

people
.

This

is

where

Perl

can

come

in,

wrapping

legacy

code

in

XS

and

SOAP

means

that

you

can

hide

the

horror

a

mature

system

behind

a

clean

interface
.

Perl

makes

an

excellent

glue

language,

and

implementing

new

systems

on

top

of

your

legacy

code

no

longer

means

that

you

have

to

know

the

exotic

internals

of

the

legacy

system
.

By

using

web

services

as

building

blocks

new

functionality

can

be

written

quickly

and

efficiently
.

Old

code

never

dies,

it

just

gets

hidden
.

Saturday 11th Dec. 2004

London Perl Workshop

1

Encapsulating knowledge

•
Your code base is the fossil record of
your organisation

•
The worst mistake you can make is to
try and re
-
implement your existing code
base from scratch

Saturday 11th Dec. 2004

London Perl Workshop

2

Throwing code away…

•
A very painful example from my own
experience is “Chunking”

•
This wasn’t even because we thought
we could do it better, we simply thought
it wasn’t relevant to the modern world

•
We should have modified, not discarded

Saturday 11th Dec. 2004

London Perl Workshop

3

Second system effect

•
Re
-
implementing things can be a lot more
costly than you would expect



Second System Effect

n. (sometimes, more
euphoniously, `second
-
system syndrome')

When one is designing the successor to a
relatively small, elegant, and successful
system, there is a tendency to become
grandiose in one's success and design an
elephantine feature
-
laden monstrosity.

Saturday 11th Dec. 2004

London Perl Workshop

4

Software archaeology

•
Software naturally
forms layers

•
Underneath the
convenience layer
you’ll normally find a
more powerful layer
which is usually
much harder to
understand

Saturday 11th Dec. 2004

London Perl Workshop

5

Code reuse

There are two main approaches,


•
Wrapping code

•
Software as services


Both you and the code will benefit from,


•
Refactoring

Saturday 11th Dec. 2004

London Perl Workshop

6

Refactoring

Why should you refactor?


•
To fix architectural problems

•
To remove inefficiency


Just plain ugly is not a good reason.

Saturday 11th Dec. 2004

London Perl Workshop

7

Just plain ugly

•
Code is often ugly for a reason

•
Every bug fix makes the code harder to
understand and much uglier

•
Refactoring just to make the code pretty
could remove added goodness

Saturday 11th Dec. 2004

London Perl Workshop

8

Wrapping code

•
The traditional way to hide legacy code
is to wrap it up behind a convenience
layer

•
In Perl this is usually done using XS and
lately the Inline::* modules

Saturday 11th Dec. 2004

London Perl Workshop

9

Refactoring by stages

•
Once you wrap components you can
remove them in a piece meal fashion

•
This is especially true if you use the
“software as services” paradigm

Saturday 11th Dec. 2004

London Perl Workshop

10

The Book…

Extending and Embedding Perl

Tim Jenness & Simon Cozens

Manning, ISBN 1930110820

Costs
£21.17
at Amazon.co.uk


“…the canonical book for this type of
programming”
--

Alasdair Allan

Saturday 11th Dec. 2004

London Perl Workshop

11

Perl XS

•
It looks like a couple of talks on the
Advanced track will be discussion XS in
depth, so I’m not going to bother

•
Thank goodness for that…

Saturday 11th Dec. 2004

London Perl Workshop

12

Inline Module

•
Inline lets you write Perl subroutines in
other programming languages like C,
C++, Java, Python, Tcl and even
Assembly.

•
You don't need to compile anything. All
the details are handled transparently so
you can just run your Perl program like
normal.

Saturday 11th Dec. 2004

London Perl Workshop

13

What’s going on?

1.
Your Perl module reads the code from
the appropriate place usually below
the DATA handle

2.
An MD5 checksum is calculated for
the code in question

3.
This checksum and other information
are compared with the XS modules
previously generated by Inline.

Saturday 11th Dec. 2004

London Perl Workshop

14

What’s going on?

4.
If the checksum does not match, an
XS module is generated based on the
functions and arguments in the inlined
code.

5.
The module is built and and installed
into a local directory

6.
If the checksum matched, the relevant
module is loaded

Saturday 11th Dec. 2004

London Perl Workshop

15

Inline::C

use Inline C;

hello_world(’Your Name Here');


__END__

__C__

void hello_world(char* name) {


printf("Hello %s!
\
n", name);

}

Saturday 11th Dec. 2004

London Perl Workshop

16

Inline::Python

use Inline Python;

print "9 + 16 = ", add(9, 16), "
\
n";


__END__

__Python__


def add(x,y):


return x + y


Saturday 11th Dec. 2004

London Perl Workshop

17

Inline::Python and Objects

use Inline Python;

my $obj = new Myclass();


__END__

__Python__


from mylibrary import myclass as Myclass

Saturday 11th Dec. 2004

London Perl Workshop

18

Inline::Java

use Inline Java;

my $obj = new Example( ‘some data’ );

print $obj
-
>get_data() . "
\
n”;


__END__

__Java__


public class Example {


private String data = null;



public Example( String s){


data = s;


}



public String get_data(){


return data ;


}

}

Saturday 11th Dec. 2004

London Perl Workshop

19

Inline::Java and STUDY

use Inline (


Java => 'STUDY',


STUDY => ['java.util.HashMap']

);


my $hm = new java::util::HashMap();

$hm
-
>put("key", "value");

my $val = $hm
-
>get("key");


print $val . "
\
n";

Saturday 11th Dec. 2004

London Perl Workshop

20

Inline::*

•
Inline::C

•
Inline::Java

•
Inline::Python

•
Inline::Tcl

•
and others…

Saturday 11th Dec. 2004

London Perl Workshop

21

Software as services

•
A different approach to hiding legacy
code is the “software as services”
paradigm

•
Add yet another layer on top of the
wrapped legacy code so that the
interface to the outside world becomes
language neutral

•
Perl is good at this…

Saturday 11th Dec. 2004

London Perl Workshop

22

SOAP::Lite

See
www.soaplite.com
for details


The latest release is
V0.65 Beta 2
, although

this is not yet on CPAN. Adds, amongst other

things,



•
MIME and DIME support



Saturday 11th Dec. 2004

London Perl Workshop

23

Google Web Services

See
www.google.com/apis/
,


use SOAP::Lite;


my $key = "000000000000000000000000";

my $wsdl = "http://api.google.com/GoogleSearch.wsdl";

my $query = "foo";


my $google = SOAP::Lite
-
>service( $wsdl );

my $result = $google
-
>doGoogleSearch(


$key, $query, 0, 10, "false", "",


"false", "", "latin1", "latin1");


my $results = $result
-
>{'estimatedTotalResultsCount’};

print “About $results returned
\
n”;

Saturday 11th Dec. 2004

London Perl Workshop

24

The Book…

Programming Web Services with Perl

Randy Ray & Pavel Kulchenko

O’Reilly, ISBN 0596002068

Costs
£19.95
at Amazon.co.uk


Now slightly out of date, but still a good source

of information about the SOAP::Lite modules

Saturday 11th Dec. 2004

London Perl Workshop

25

The Mailing Lists…

See
www.soaplite.com

for details,


•
Main list is on Yahoo! Groups, see
groups.yahoo.com/group/soaplite/

•
But there is also
soaplite
-
announce

and
soaplite
-
devel

lists hosted at
SourceForge

Saturday 11th Dec. 2004

London Perl Workshop

26

Amazon Web Services

•
A good example of the software as
services paradigm

Saturday 11th Dec. 2004

London Perl Workshop

27

ItemSearch Service

sub make_request {


my $keywords = shift;


my $request_type =
\
SOAP::Data
-
>value(


SOAP::Data
-
>name('Keywords')
-
>value($keywords),


SOAP::Data
-
>name('SearchIndex')
-
>value('Books'));



my $itemsearch_request = SOAP::Data
-
>value(


SOAP::Data
-
>name('SubscriptionId’)
-
>value($subs_id),


SOAP::Data
-
>name('Request')
-
>value($request_type));



my $aws_handle = SOAP::Lite
-
>service("$aws_wsdl");


$aws_handle
-
>ItemSearch($itemsearch_request);



my $som = $aws_handle
-
>call();


return $som;

}

Saturday 11th Dec. 2004

London Perl Workshop

28

Building our Complex Type

my $request_type =
\
SOAP::Data
-
>value(


SOAP::Data
-
>name('Keywords')
-
>value($keywords),


SOAP::Data
-
>name('SearchIndex')
-
>value('Books'));


my $itemsearch_request = SOAP::Data
-
>value(


SOAP::Data
-
>name('SubscriptionId’)
-
>value($subs_id),


SOAP::Data
-
>name('Request')
-
>value($request_type));


Builds the following complex type


<SubscriptionId>$subs_id</SubscriptionId>

<Request>


<Keywords>$keywords</Keywords>


<SearchIndex>Books</SearchIndex>

</Request>

Saturday 11th Dec. 2004

London Perl Workshop

29

Calling the Service

my $aws_handle = SOAP::Lite
-
>service("$aws_wsdl");

$aws_handle
-
>ItemSearch($itemsearch_request);


my $som = $aws_handle
-
>call();

return $som;


Creates a SOAP object calls the Amazon

service using the WSDL found at the URL


Returns a SOAP::SOM object containing

the results of the query

Saturday 11th Dec. 2004

London Perl Workshop

30

A simple SOAP Server

The simplest way is to use an existing web server,

the following would end up as cgi
-
bin/service.cgi


#!/usr/bin/perl


use SOAP::Lite;

use SOAP::Transport::HTTP;


use ANY_OTHER_MODULE_NEEDED;


SOAP::Transport::HTTP::CGI


-
>dispatch_to(‘/path/to/perl/modules/’)


-
>handle;

Saturday 11th Dec. 2004

London Perl Workshop

31

dispatch_to( )

A collection of Perl modules,


dastardly
{
aa
}: ls modules/

drwxr
-
xr
-
x

2 nobody

users

4.0K Sep 16 2003
.
/

drwxr
-
xr
-
x

6 root

root

4.0K Oct 11 21:56
..
/

-
rw
-
rw
-
r
--

1 nobody

users


118 Sep 16 2003 Echo.pm

-
rw
-
rw
-
r
--

1 nobody

users


85 Sep 16 2003 Ping.pm


package Ping;


sub ping {


$class = shift;


return “ACK”;

}

Saturday 11th Dec. 2004

London Perl Workshop

32

Easily consumed…

You don’t need to write WSDL to use the service,


use SOAP::Lite;


my $soap = new SOAP::Lite();

$soap
-
>uri(‘http://www.company.com/Ping/);

$soap
-
>proxy(‘http://www.company.com/cgi
-
bin/service.cgi’);


my $result;

eval { $result = $soap
-
>ping(); };

if ( $@ ) {


print $result
-
>faultstring();


exit;

}


print $result
-
>result();

Saturday 11th Dec. 2004

London Perl Workshop

33

Asynchronous web services

•
SOAP::Lite does not support .NET
asynchronous callbacks

•
But you can implement asynchronous
web services using the module

•
Use contextual web services and
persistent state to keep track of things
by hand

•
This isn’t as hard as it sounds…

Saturday 11th Dec. 2004

London Perl Workshop

34

Authentication

•
See WebService::TicketAuth for a full
blown solution

•
Alternatively you could do a light weight
implement of authentication using HTTP
cookies by sub
-
classing the relevant
SOAP::Transport module.

•
Look at
www.astro.ex.ac.uk/people/aa/
for an implementation of the later

Saturday 11th Dec. 2004

London Perl Workshop

35

No more language neutrality

•
If you can serialise it you can send it over a
wire. Data::Dumper can be a powerful tool



my $dumper =


new Data::Dumper([$object], [qw($object)]);


my $serialised = $dumper
-
>Dump();
\



and at the far end,



my $object = eval $serialised;


if ( $@ ) {


print “Warning: Cannot de
-
serialise object
\
n”;


}

Saturday 11th Dec. 2004

London Perl Workshop

36

SOAP::Lite has problems

•
Interoperability
-

Although have a look
at the SOAP::WSDL module if you need
.NET interoperability

•
Complex types
-

You’ll find that the
SOAP::Data::Builder should simplify
things

•
WSDL
-

Perl is loosely typed, this
means no automatic WSDL generation

Saturday 11th Dec. 2004

London Perl Workshop

37

So why did I talk about Inline?

•
Because of the lack of “proper” WSDL
support I’m currently using a mixture of
Java and Perl services to hide legacy
Fortran (and C)

•
Perl serves as an excellent glue
language for this task

Saturday 11th Dec. 2004

London Perl Workshop

38

Grid Services?

See
www.sve.man.ac.uk/Research/AtoZ/ILCT


WSRF::Lite is the follow on work from OGSI::Lite,

it implements the Web Service Resource

Framework which has effectively replaced OGSI,

for details on WSRF visit
www.globus.org/wsrf

Saturday 11th Dec. 2004

London Perl Workshop

39

Lots of code, broken compiler?

If you have a large mature code base but the

limits of the underlying infrastructure are starting

to cause problems, then perhaps you should


•
Re
-
implement the compiler, not your code

•
No, seriously…

Saturday 11th Dec. 2004

London Perl Workshop

40

Re
-
implement the system?

The alternative is to rewrite the entire system in


a new language but,


•
Experience shows that such projects often
(almost always?) fail

•
You loose years (decades?) of customised
business logic

•
You make a lot of people very unhappy


Saturday 11th Dec. 2004

London Perl Workshop

41

Use Parrot?

•
Implementing functional compilers for 4GL
languages on top of Parrot has been done
before (err, once)

•
See “
Building a Parrot Compiler
” by Dan
Sugalski at the O’Reilly’s OnLamp.com

•
For simpler languages, or problems, you
could target the compiler for Perl 5 as Dan
did initially

Saturday 11th Dec. 2004

London Perl Workshop

42

Problems with Legacy Code

•
Layering
-

the higher level routines will
believe the output of the lower level
ones.

•
Bugs
-

You still have all the old bugs,
although on the bright side you, you
don’t have any new ones.

•
Backdoors
-

The possibility of
backdoors in the lower level code.

Saturday 11th Dec. 2004

London Perl Workshop

43

Conclusions

•
Perl is an excellent glue language

•
You can use it to hold together the
unlikeliest collection of different pieces
of code

•
It may be ugly but this is better than
loosing decades of “smartness”

•
Your code has evolved, don’t lose that