Adapters and EAI

Copyright iConexio Technologies Canada Inc.

Adapters and EAI

Technology, Tools and Trends


Presenter: Atul Apte

President and CEO, iConexio Technologies Canada Inc.

Agenda

•
Session 1:

–
Background on software integration

–
Background on adapters

–
Overview of adapter technology

•
Session 2:

–
Overview of trends in adapter technology

–
J2CA, Web Services, Semantic Adapters

–
Overview of adapter development tools

•
E
-
Fusion product architecture

•
Q&A

Background On Software Integration

•
Software integration is the #1 IT priority today

•
It is also the most complex and expensive problem to solve

•
About 30%
-

40% of all IT costs are integration
-
related

•
Approximately 1/2 of all integration costs are adapter
-
related

•
Adapters represent the highest incremental cost of software
integration

•
Each software system that needs to be integrated requires
an adapter

Background On Software Integration

•
The primary focus of software integration is
information systems (or business applications)

–
The two main areas of software integration are:

•
Data

•
Function (Web Services)

–
The two main types of software integration environments are:

•
Point to Point (Application Server based)

•
Brokered (Integration Platform based)

–
The extended focus of software integration is business processes

•
Integrating information systems with business processes

•
Automating the integrated business processes

Background On Software Integration

•
Components of Software Integration Platform

–
Foundation components

•
Host (Application Server, Integration Server, others)

•
Messaging bus (JMS, others)

•
Object brokers (OMG)

•
Database engines (RDBMS, ODBMS, others)

•
Transaction engines (XA compliant, others)

–
Value add integration components

•
Transformer

–
Mapping, Object aggregation, Object decomposition

•
Broker

–
Routing, Filters

•
Adapters

Current Perspective on EAI

•
EAI is still a big, complex, expensive, tedious and
repeating problem faced universally by companies
of all sizes

–
Approximately 1/3 of all IT costs are EAI related

–
Approximately 1/2 of all EAI costs are adapter
-
related

•
The definition of EAI is much broader today and includes
software systems within and outside the companies
firewalls

–
The term “EAI” has been changed and misunderstood with the continuing
evolution of EAI technologies

Current Perspective on EAI

•
EAI is an ongoing initiative and requires strategic
thinking, strong risk management, and realistic
goal oriented implementation plans

•
Industry studies have reported that the success
rates of EAI project is very low < 5%

•
Existing methodologies, project management
practices & processes are not enough to deal with
the unique and specific risk management
requirements of EAI

Current Perspective on EAI

•
iConexio has a strong commitment to solve one of
the toughest problems of EAI namely
development of custom/customized adapters and
lowering the total cost of ownership

•
The first step is to define EAI in a clear and
simple form easy to understand, comprehend, and
implement

Definition of EAI

EAI is the process of integrating applications and
database systems into a unified software system
capable of supporting:

–
On
-
demand data exchange

–
Collaboration of shared functions

–
Consolidation & Transformation of data into information

–
Automating data synchronization

–
Automating business process

EAI Domains

•
The EAI processes are grouped into a set of
logical processes (“Domains”) which simplifies
the definition of specific EAI processes

–
Application To Application Domain (A2A
-

Domain)

•
The process of integrating two or more business applications

–
Application To Information Domain (A2I
-

Domain)

•
The process of integrating a business application with information
producers (data warehouses), information access portals, etc.

–
Application To Process Domain (A2P
-

Domain)

•
The process of integrating a business application with one or more
work flows, EDI gateways, supply chain automation networks, mobile
workforce networks

EAI Domain Reference Models

•
Every EAI Domain is supported by a context
sensitive reference model that can be used to
define, design, deploy a customized EAI solution

•
Each EAI domain reference model contains:

–
A logical architecture that defines the components and
relationships of the EAI process

–
A set of integration patterns that define the different behavior and
interactions of the EAI process

–
A technology stack containing layers of technologies and standards
that enable the EAI process

Business Application

Business Application

Application to Application (A2A) EAI

Database

Database

Function

Function

Data

Integration

Broker

Adapter

Adapter

Web

Services

Broker

Adapter

Adapter

SOAP

XML

Transformer

UDDI

Information Portal

Business Application

Application to Information (A2I) EAI

Database

Database

Function

Function

Adapter

EII

Server

Adapter

ETL

Data

Warehouse

Portal

Server

Business Application

Partner / Supplier

Systems

Business Application

Application to Process (A2P) EAI

Database

Database

Function

Function

Adapter

Workflow

Engine

Adapter

EDI

Gateway

XML

Database

Function

Adapter

XML / EDIFACT

Firewall

WFMC

Data Replication Pattern

A2A Patterns

Data

Change

Detector

Data

Extractor

Data

Filter

Data Source

Data Target

Data

Loader

Other Patterns


Data Synchronization (Bi
-
directional Data Replication)


Composite Data Extraction


Collaboration of Shared Services

Information Creation Pattern

A2I Patterns

Data

Retriever

Meta
-
Data

Creator

Info

Aggregator

Data Source

Information Sink

Info

Presenter

Other Patterns


Extract Transform Load (ETL)


Data Staging

Task Delegation Pattern

A2P Patterns

Task

Assignment

Agent

Task to

Function

Map

Workflow

Manager

Business
Application

Function

Manager

Other Patterns


Business Process Management



Generic EAI Solution Architecture

Foundation (Host, JMS, CORBA, Transaction and Database Engines)

Broker

Transformer

Adapter

Information

System

Adapter

Information

System

Adapter

Information

System

Adapter

Information

System

Adapter

Information

System

Adapter Technology Status Quo

•
Adapters represent the highest incremental cost of
integration

–
Each applications that needs to be integrated requires a minimum
of one to three adapters (A2A, A2I, A2P)

•
The average Fortune 1000 company
1

has 48
applications and 14 databases for a total of 62
potential adapters

•
There are few specific tools and platforms
available for development and maintenance of
adapters

1
ID
C

Definition of an Adapter

•
There is no Single Definition of an Adapter

•
But there is a common understanding of the
adapters behavior and role

•
Adapters, Connectors, Plug
-
ins, Web Services,
Integration Components and other proprietary
terms are used in different context to mean
“Adapter Type Of Functionality”

•
Adapter definitions tend to be context sensitive

Generic Definition Of Adapter

•
An Adapter is a software component which
integrates two different instances of software
systems

–
Typically the software systems integrated are business applications
(information systems)

•
In a brokered integration environment a business application is
integrated with the broker by using an adapter

–
Adapter facilitates integration between the integrated software
systems

•
Provide connectivity between the systems

•
Manage the interactions between the systems

•
Enforce transformation and other business rules

Background On Adapters

•
The concept of adapters was first introduced by
the Enterprise Application Integration (EAI)
vendors in the later half of 1990’s

•
Adapters have become the most important aspect
of software integration

–
Every software that needs to be integrated requires an adapter

–
The average Fortune 1000 company has 48 different business
applications and 14 different databases¹

–
Adapters represent the highest incremental cost of software
integration

¹
IDC

Background On Adapters

•
Until recently there were no standards for
developing and supporting adapters

–
J2EE:JCA is the first system level standard designed to support
adapters on the Java Application Server

•
Major vendors are now supporting J2EE:JCA in their Java application
servers

•
Adapters are becoming a commodity

–
System standards for adapters will result in more focus on
integration functionality

–
Multiple providers of the same adapter will give users opportunity
to compare adapter features

Background On Adapters

•
Generally adapters are custom developed from
scratch

–
Most information systems are proprietary or heavily customized

–
The average cost of developing and maintaining an adapter for
packaged applications like SAP, PeopleSoft, etc. over a five year
period is over U$ 1 Million

•
The adapter market has few tools available for
developers

–
The biggest challenge in creating adapter tools is the expertise and
experience required to design, develop, and maintain complex
adapters

Adapter Interactions

•
An adapter use case defines a specific interaction
between the two software systems integrated by
the adapter

–
An adapter use case is a sequence of one or more functional steps

•
There are two types of steps in an adapter use case

–
Integration function (Connection, Parsing, Transformation, etc.)

–
Support function (System Logging, Error handling, Audit Trail, etc.)

–
Two types of use cases

•
Simple Use Case

–
Involves only one adapter (two software systems)

•
Complex Use Case

–
Involves two or more adapters (four or more software systems)


Adapter Scenarios

•
An adapter scenario is a specific instance of an
adapter use case and is bound to a specific runtime
environment

–
Atomic Scenario (Derived from simple use case)

–
Collaborative Scenario (Derived from complex use case)

•
It is possible for adapter scenarios to change
between atomic and collaborative scenarios
depending on the runtime environment


Simple Use Case (Atomic Scenario)

Simple Adapter Use Case

SAP

Adapter

SAP

API

Client

•
Check customer

•
Add new order

Complex Use Case (Collaborative Scenario)

Simple Use Case

Complex Adapter Use Case

SAP

Adapter

SAP

Siebel

Adapter

Siebel

API

API

Client

Add new

order

Check

customer

Adapter Technical Architecture

Technology Stack

Client Interface

Service Gateway

Document Manager

Transformation Manager

Scenario Manager

Information System Manager

Host Environment Manager

Responsible for providing unidirectional or
bidirectional access to adapter services

Responsible for parsing, filtering, and formatting data
and documents

Responsible for managing single/multiple adapter
scenarios involving one or more information systems

Responsible for defining and implementing service
contracts and access control

Responsible for data type conversion, data mapping,
object aggregation and object decomposition

Responsible for managing connectivity to the
information systems

Responsible for managing the lifecycle and
collaborating with the operating platform

Adapter Technical Architecture

Technology Stack

Client Interface

Service Gateway

Document Manager

Transformation Manager

Scenario Manager

Information System Manager

Host Environment Manager

Optional Layer

Optional Layer

Adapter Technical Architecture

Technology Stack

Client Interface

Service Gateway

Document Manager

Transformation Manager

Scenario Manager

Information System Manager

Host Environment Manager

SOAP/UDDI

J2EE:JCA

XML / XSLT

Simplifying Adapter Complexity

•
How to manage the technical complexity of
adapters?

–
Use model based software development techniques

•
Pre
-
defined software models based on integration and design patterns
will provide a robust framework for developing adapters

–
Developing multiple adapters for one application

•
Encapsulate fewer interfaces into one adapter instance

–
Collaborate with external transformers instead of developing
transformer functionality in each adapter

Implementation


Analysis, Architecture and Design

Adapter Models

Abstract Adapter
Model

Structura
l

Patterns

Functional

Patterns

Conceptual

Patterns

Integration

Patterns

Concrete Adapter
Model

Trends In Adapter Technologies

•
Adapters are more sophisticated in functionality

–
Web services enable application function level collaboration

–
J2EE Connector Architecture enables plug
-
and
-
play support for
adapters inside application servers

–
Adapters can function in a point
-
to
-
point or broker based
integration environment

•
Standard adapter models will provide a consistent
architecture and design for adapters

•
Adapter tools will enable faster, cheaper, and
easier development and maintenance

Impact of Web Services

•
The most significant impact of web services is the
shift in software application architecture

•
Service Oriented Architecture “SOA” is becoming
the foundation for new internet based business
applications

–
The concept of “Composing” business applications from existing
software services

–
Web services enables development of “SOA” compliant
applications

–
Adapter have an even more critical role in fulfilling “Services”

Impact of Web Services

•
Where is the Synergy between adapters and web
services?

–
Web services provide a standardized platform for defining,
locating, and invoking business services

•
WSDL, UDDI, SOAP

–
Adapter provide a standardized component mechanism for
fulfilling the web services by accessing external software systems

•
J2CA

–
Both web services and adapters are moving towards a common
semantic environment enabling rapid “composition”

•
UDEF, RDF

What is J2EE Connector Architecture?

•
J2EE Connector Architecture (J2CA) is a sub
-
specification of J2EE

•
J2CA is a system level interface for hosting and
accessing connectors to non
-
J2EE applications

–
J2CA connectors extend the J2EE servers ability to interface with
non
-
J2EE applications

–
Most J2EE servers support J2CA specification

•
IBM WebSphere, BEA WebLogic, Jboss, Oracle AS

–
J2CA compliant adapters for popular applications like SAP,
PeopleSoft, are now available in the market place

What is J2EE Connector Architecture?

•
J2CA Specification (Ver 1.5) defines system
contracts for managing security, transactions, and
connections between J2EE application
components and legacy systems

•
Support meta
-
data interfaces and bi
-
directional
access between adapters and legacy systems

•
Message driven access to J2CA connectors is now
possible

Moving Towards Semantic Adapters

•
What is a Semantic Adapter?

–
A Semantic Adapter provides an interface to the semantics of the
adapter

•
Adapter Behavior (Services provided by the adapter)

•
Meta
-
data of the adapted systems data structures

•
Meta
-
data of the adapted systems service interfaces

•
Configuration properties of the adapters runtime environment

–
A Semantic Adapter uses an Ontology based approach to define
the semantics

•
This involves associating commonly understood meaning to the
definition of adapter properties, behavior, configuration, and
associated meta
-
data

Adapter Development Tools

•
Fundamental Requirements

–
Enable just in time adapter development and maintenance

•
Customers do not purchase adapters ahead of requirement

•
End points (information systems) change and therefore pre
-
built
adapters need customization too

–
Extend the model based design (MBD) paradigm to adapters

•
Model based development significantly reduces development time
and cost, and increases adapter quality

–
Differentiate and support abstract adapter models and concrete
adapter models

–
Enables multiple implementations of a common abstract adapter model

Adapter Development Tools

•
Fundamental Requirements

–
Enable adapter developers to focus on the business requirements of
integrating two software systems

•
Develop and maintain platform controllers which encapsulates the

operating platform system interfaces from adapter developers

–
Eliminates the burden of learning and programming system level
interfaces and standards

–
E.g: J2EE:JCA controller which encapsulates J2EE:JCA system
contracts from the adapter developer

•
Generate adapter framework and component shell from pre
-
defined
adapter models

–
Ensures design consistency, reduces bugs and enhances the quality of
adapters

E
-
Fusion Adapter Technology

•
Objectives:

–
Solve the root cause of high adapter development and maintenance
costs

•
Provide the best adapter development tools and runtime platform

–
Define the core technology architecture for adapters

–
Define new adapter standards and methodologies

•
Support relevant industry standards applicable to adapters

–
Develop a single adapter technology framework capable of
supporting different adapter interactions, programming languages
and operating platforms

•
Reduces the total cost of ownership for adapters

Overview of E
-
Fusion Adapter Tool

•
iConexio has developed E
-
Fusion, a break
-
through, patent pending technology for Adapter
development and maintenance

•
E
-
Fusion reduces the time & cost to develop and
maintain software adapters from 41% to 75%

•
Rapid progression towards holistic integration of
information systems for improved ROI

–
A2A

–
A2I

–
A2P

Benefits of E
-
Fusion

•
E
-
Fusion extends the Model Based Development
(MBD) software paradigm and applies is to
adapters

•
E
-
Fusion provides specific platform controllers
which eliminates 90% of the server/platform
coding

•
E
-
Fusion’s Development and Testing
methodology significantly improves adapter
quality

Benefits of E
-
Fusion

•
E
-
Fusion sets new standards for adapters

–
First to provide developers with design and implementation
standards for the business functions supported by adapters

–
First full life cycle adapter development methodology

•
E
-
Fusion provides a Adapter Platform which
includes all of the common adapter infrastructure

•
E
-
Fusion enables developers to concentrate on the
elements unique to each adapter project,
independent of platform

Positioning E
-
Fusion

•
E
-
Fusion provides a common adapter tooling
environment across application platforms

–
IBM platforms

•
WebSphere Application Server

•
WebSphere Business Integration Server (Crossworlds)

•
MQSeries (Integrator, Workflow, Messaging)

•
Mainframe and AS400

–
J2EE Application Servers

–
CORBA ORB’s

–
.NET Platform

–
Legacy Operating Systems (UNIX, Linux)

Positioning E
-
Fusion

•
E
-
Fusion provides a universal adapter hub across
enterprise integration environments

–

Business Integration

•
Process Integration

•
Application Connectivity

–
Enterprise Information Portals

•
Information Integration

–
EAI

•
Data Integration

–
Web Services

•
Application Collaboration

Integrated Collaboration Environment (
ICE
)

Adapter Development Environment (
ADE
)

Adapter Runtime Environment (
ARE
)

E
-
Fusion Product Architecture

Adapter

Runtime Platform

Adapter

Deployment Tool

Adapter

Development Tool

Adapter

Repository

Java

IDE

Generate Adapter
Component Shell
From Models

Assemble & Test
Adapter

Define Scenarios

Configure Adapters

Run Adapter Scenarios

Adapter

Models

Extends

Code

Applicatio
n

Platform

Extends

Register Components
To E
-
Fusion

Monitor Adapter

E
-
Fusion Adapter Technology

•
Architecture:

–
Adapter architecture based on the concept of functional layers

•
Each layer encapsulates specific adapter functionality accessible as
well defined services

•
Each layer can be implemented as an isolated/distributed component
of the adapter

–
Architecture supports three types of interactions for an adapter

•
Synchronization

–
Keeping distributed data models in sync with each other

•
Collaboration

–
Enabling software systems to collaborate functions

•
Delegation

–
Enabling software systems to execute tasks on behalf of other systems

E
-
Fusion Adapter Technology

•
Adapter Model

–
Is a collection of one or more adapter patterns managed by the E
-
Fusion framework Integrated Collaboration Environment (ICE)

–
E
-
Fusion supports two types of Adapter Models

•
Abstract adapter model

–
Represents the architecture and design of an adapter

–
Adapters cannot be created directly from abstract adapter models as
there are no implementation details defined

•
Concrete adapter model

–
Represents the implementation reference model of an adapter

–
Includes one or more run
-
time scenarios

–
Adapters can be created from concrete adapter models

Adapter Development Environment

•
Adapter Development Environment (ADE)
supports developers throughout the full lifecycle
of adapter development and maintenance

–
From design to maintenance

•
Design, Develop, Assemble, Configure, Deploy, Maintain

•
Develop pre
-
built adapter models and support custom adapter models

–
Enable higher efficiencies in adapter development without
changing existing development workflow and processes

•
Integrate E
-
Fusion with leading IDE tools and extend the tools
capabilities to support adapter development

•
Define new adapter engineering and development methodologies
which can be applied with generic software development tools

Adapter Runtime Environment

•
Adapter Runtime Environment (ARE) supports
distributed and centralized adapter deployments
across multiple platforms

–
Support dynamic distribution of adapter components across the
platforms in the adapter runtime environment

–
Support centralized administration, monitoring, and debugging of
adapters at runtime

–
Support dynamic language (locale) translation for multi
-
lingual
runtime environments

–
Support dynamic component code download and side
-
by
-
side
runtime environments (production and QA)

E
-
Fusion Adapter Runtime Platform

CPU/OS

CPU/OS

Application Server

(J2EE / CORBA / Other)

E
-
Fusion Adapter
Runtime Platform

E
-
Fusion Adapter
Runtime Platform

Repository

Execution

Manager

Execution

Manager

Controller

Controller

Summary

•
Adapters are complex pieces of software
performing critical integration tasks

•
The definition of adapters continues to expand and
support new application architectures (“SOA”)

•
Adapter tools are now available for enabling rapid
adapter development and deployment

•
The trend is towards developing semantically and
functionally rich adapters will continue to
strengthen the role of adapters

Q&A

Thank You