IBM and AOSD

decorumgroveInternet and Web Development

Aug 7, 2012 (5 years and 2 months ago)

562 views

IBM Software Group

®


Aspect Oriented Software Development


from Promise

to Reality

Dr. Daniel Sabbah

Vice President,

IBM Software Group Strategy and Technology

IBM Software Group

Outline


Industry trend toward on demand Business


IBM’s Customers and their Changing Requirements


IBM’s Own Transformation



Aspect Oriented Software Development



Driving AOSD Technology within IBM



Future Activities around AOSD



Challenges



Conclusion & Questions

IBM Software Group

Traditional

The Internet

On Demand

Structured

Calculations

Data Processing

Transactions

Open Standards

Connectivity

Flexibility

Simplicity

Modular Components

easily defined and manipulated


Dynamic definition and
operations

Deepening Integration of IT with Business

Emerging On Demand Computing Model

IBM Software Group

An (inter)enterprise whose

business
processes integrated end
-
to
-
end


across the company and with key partners,
suppliers and customers

can


respond with speed

to any customer demand, market opportunity
or external threat.

On Demand Business…

is Not Just About Getting Better…

IBM Software Group

Bridging the gap between business transformation and IT.

Partner
Relationship
Mgmt.

Product Lifecycle
Management

Category
Management/

Merchandising

SCM / Retail
Operations

Procurement

Business


Processes


IT


Sophistication

The Need to Become More Horizontal

IBM Software Group

Process Layer (modeling, execution, management)

Dealer Integration,
Warranty Claims
Management

Collaborative
Engineering

Direct and
Indirect
Procurement

Sequencing,
Order
Management

Inventory Control,
Sourcing, Supply
Chain visibility

Application Integration Layer

Connectors/Collaboration/Flows

Systems

Environment

Application

Environment

Human Interaction

and Collaboration

Transactions
& Messaging

Information
Management

Systems
Management
& Security

Software
Development

Middleware Platform

Business efficiency

BPM

BAM

Business Process

Integration Evolution

Autonomic

IT Efficiency

Resource optimization

Distributed OS Evolution

Linux

Unix

Windows

OS/400

z/OS

Servers

Storage

Evolution towards Middleware

IBM Software Group


Executing strategy to repurpose WAS as the kernel
for IBM’s On Demand offerings


Leveraging a very large investment in WAS


WAS technology has been
reused

and/or bundled by
129 product offerings

across IBM Software Group:


WebSphere


Commerce


WSAD


WSSD





Tivoli


Policy Director


Privacy Manager





Lotus


Portal


Domino


Workplace Messaging


Learning Management System


Pervasive


Voice Systems


Data Management


UDB Universal Developers
Edition





Growing reuse

of Application Server technology


GRID computing


On Demand


Additional SWG and Server Group products


Leveraging another very large investment in UDB


UDB technology has been
reused

and/or bundled by
90+ product offerings

across IBM Software Group:


WebSphere


WAS


Commerce


WSAD, WSADIE


WSSD


MQ Series Extended Security Edition


MQ Workflow


Business Integration Connect, Message Broker, Server


Everyplace Access


Tivoli


Policy Director


Configuration Manager





Lotus


Portal


Domino




Reuse Across the WebSphere Platform:

WebSphere Application Server (WAS) and UDB



Current approach gives reuse value, but is:



too expensive because components are too large, monolithic;



and results in high service and support expenses



A more cost
-
effective solution is needed


IBM Software Group

Reuse Across the WebSphere Platform
:

Portal Server

Key Pressures on Development/Test:


Marketplace demands for achieving aggressive release cycles


Currency with versions of O/S, Application Servers, Web Servers, Databases, Directory Servers, Browsers, Security, etc.


Breadth of O/S platforms: Windows, AIX, Solaris, zLinux, Linux/Intel, iSeries, zOS and OS390, etc.


Version currency of WAS, Domino, HTTP Server, DB2, Oracle, Informix, SQL, iPlanet, Active Directory, etc.


Integration with Tivoli Access Manager, Site Analyzer, SiteMinder, Lotus Sametime, Lotus QuickPlace


Supporting multiple, concurrent service streams for Portal Server V4.1.x, V4.2.x, V5.0.x


Increased customer support involvement for service, and Q&A

Portal Server

(V4 and V5)

WSSD

WAS EE

Sametime

QuickPlace

Domino

Site Analyzer

IBM Directory

Server

TAM and CM

(V4 only)

WEA

WVAA

IBM
Content
Mgr

Commerce

Lotus

Workplace

UDB

Transcoding

Publisher


Machine Translation

WPCP

3
rd

party:

LDAP Directories,

Security Subsystems,

Databases

IBM Software Group

Case Study of a Cross
-
Platform Offering:


Portal Server


Re
-
use of Platform products enabled faster delivery of function



But encountered some

inconsistencies and lack of interoperability of the parts


Development / technical hurdles


Smooth installation procedure very difficult due to complex pre
-
reqs


Complex interactions between Portal, Application Server, Tivoli Access Manager, Lotus / Domino
and IBM Directory all part of the integrated offering


No full binary compatibility for runtime APIs


Product testing vs. Platform solution testing



Platform hurdles



Plan prioritization of requirements in other products’ plans (Application Server, Tivoli, Lotus, etc.)


End of Support mismatches between all dependencies


Support measurements: number of PMRs rises in base products due to additional usage as a
bundled component



Sales and Marketing hurdles


Lag between availability dates of Portal and base products causes sales conflict


Need to cross
-
sell to address total solution needs, which compounds our current complexities




IBM Software Group


Changes the Software Industry must make to facilitate
customer transformation to on demand:


Products must become more flexible, more modular, more
adaptive, higher quality, and simpler for the customer


The way the software industry creates, extends and services
software must undergo a similar transformation.



AOSD enables both of these transformations


Introducing on demand characteristics into IBM’s software


E.g., quality, reuse, componentization, reengineering


Accelerating IBM as an on demand company.


E.g., organization matches architecture



The rest of the talk will describe our initial steps.


We know intuitively that AOSD can help

IBM Software Group

AOSD is the next step in the series of advances in

Software Engineering modularity

Structured Programming

Modular Programming

Data Abstraction

Object Oriented Programming

Aspect Oriented Software Development

Compiled Programming Languages

IBM Software Group

Gartner Hype Cycle for Application Development, 2003

“Hype Cycle for Application Development”,
J. Duggan, M. Blechar, M. Driver, J.Feiman, M. Light, T.
Lanowitz, J. Sinur, D.Vecchio, D. Stang, J. Fenn, A. Linden, Y.Natis, W. Andrews. 30 May 2003

© Copyright Gartner Research 2003. Used by permission.

IBM Software Group

It took a while to get here

Visibility

Maturity

Insight

Incubation

As of May 2003

Trigger

Peak

Trough

Slope

Plateau

Aspect
-
Oriented

Software

Development

There have been many

years of incubation and

innovation

IBM Software Group

AOSD within IBM

IBM Somers, NY, SWG Strategy

TJ Watson Research, NY. Core Technology Research

Hursley SW Laboratory, UK. Core Tech. Dev.

Raleigh, NC. WebSphere Development


Software Group System House.


Serviceability Team

Tech Focus Group,

Toronto lab

China Research Lab, Business AO

Approximately 75 people

Focused on AOSD across IBM


AOSD is a core IBM Software

Architectural Activity

IBM Software Group

IBM Roadmap

Core Technology



2003


2004


2005


2006



Driving Organizational Acceptance/Adoption

Reengineering Software

Enhancing the quality of software

Reengineering Software Engineering

NOW

New Services Offerings

IBM Software Group


Investing in core technologies


AspectJ™


AO technology for the Java™ language


AspectJ 1.1 recently awarded a Software Development Magazine Jolt
Productivity Award


AJDT


Development environment for AspectJ


CME


Cross
-
artefact, cross life
-
cycle capability



To provide the underpinning capability for internal, and external
exploitation


All are Eclipse based, Open Source projects

Core Technology Investment

IBM Software Group

Core

Technology

Application within IBM,

and with the community

Improvements

Performance

Weaving flexibility

Load Time weaving

Scalability

Robustness

Removal of adoption barriers




JDT Restructure

Cross
-
artefact capability


… more

Core Technology Investment

IBM Software Group

Delivering new Qualities of Service (QoS) into a WebSphere product


Adding problem diagnostic support into large software stack (a
WebSphere component


WSIF (web services invocation
framework)


110 source files


17,210 lines of code



FFDC = First Failure Data Capture.


Captures contextual information within exception catch blocks


Used for post
-
failure diagnosis


FFDC is a cross cutting concern




A key proof
-
point for broader application across IBM Software
products

Enhancing the quality and serviceability of software

IBM Software Group

First
-
failure data
capture in a
WebSphere
component.


Each bar is a .java
file. A red bar
indicates an
explicit catch block
and FFDC site.

Only a subset of
the110 source files
shown

Modularizing Cross Cutting Concerns

Enhancing the quality and serviceability of software

Delivering new Qualities of Service (QoS) into a WebSphere product

IBM Software Group

The AOSD
solution.


The FFDC policy
and logic have
been modularized
into an aspect.

The new FFDC
aspect applies to
241 exception
catch blocks.

Modularizing Cross Cutting Concerns

Enhancing the quality and serviceability of software

Delivering new Qualities of Service (QoS) into a WebSphere product

IBM Software Group

Re
-
engineering of existing QoS solutions to improve on manual/automated
techniques


Intuition, and now, experience, tells us that logging and FFDC can
be more reliably applied with AO technology



But


perhaps it is worth re
-
engineering existing solutions?



Using a custom Eclipse plugin, we did some analysis


SM (admin) Component of WebSphere


Total of 355 errors found


missing trace points


33% FFDC entries are incorrect


Runtime component of WebSphere


missing trace points


17% FFDC entries are incorrect



None of these errors would have occurred with an AO approach

Enhancing the quality and serviceability of software

IBM Software Group

org.apache.wsif

WebSphere RAS

WebSphere FFDC

WebSphere PMI

+

WSIF for Open
Source Community

Composition

WSIF for
WebSphere

Exploring Re
-
Use and Product Line Implications: The WSIF Story

Enhancing the quality and serviceability of software

IBM Software Group

Exploring Re
-
Use and Product Line Implications: The WSIF Story

Enhancing the quality and serviceability of software

241

FFDC join points

1060

RAS (Log4j) join points

46

PMI join points

IBM Software Group

Delivering Technologies to enhance problem diagnosis and serviceability


HyperProbe:


Java instrumentation technology based on Hyper/J



Effective at enhancing IBM J2EE field service situations, e.g.,


HTTP Session state data ballooning in production systems


JDBC connection deadlock due to mismanagement in application code


Misuse of Finalize() to free system resources




Next Steps


Developing robust/reusable diagnostics around AspectJ



Developing new AOSD
-
based Serviceability technologies coupled with FFDC to
enhance service processing.


Targeting multi
-
component interactions and problem isolation

Enhancing the quality and serviceability of software

IBM Software Group

Delivering higher quality code through capturing and enforcing architectural
standards and best practices: API Scanner Application

Product Deployment Tools (version N)

Scanner Report

210

deprecation


warnings

5

obsoletion


errors

3

contraventions

Aspects

Package/Class Map

Rules & Filters

Messages

Conversion scripts

Scanner rules

database

(Deprecated, Deleted,

Illegal Interfaces)

API Contravention

Scanner

Deployed

Application

Unit (EAR)

Deployer /

System Administrator

Application


Developer

WebSphere

Platform
-

supplied

Product
-

supplied

legend

Contravention Data

-
in human
-
readable form

-
for development tools

Enhancing the quality and serviceability of software

IBM Software Group


Results


Scanner being used with products under development, e.g.,


WebSphere Portal


WebSphere Commerce


Tivoli products



Goal: identify use of private APIs by products that run on top of the
WebSphere Application Server



Result:
50,000+

violations found



Application was very easily constructed



Positive contribution to goals of clean separation for:
componentization


Delivering higher quality code through capturing and enforcing
architectural standards and best practices: API Scanner Application

Enhancing the quality and serviceability of software

IBM Software Group

Business Investment


IBM Software Group System House


Focused on platform integration challenges


Cross Cutting Concerns:


Capability
: integration & interoperability


Usability
: enhancing ease of learning & understanding


Reliability
: addressing high customer Problem rates


Installability
: better out
-
of
-
box experience


Maintainability
: improving problem determination


Documentation
: improving documentation


Service
: support and maintenance challenges


consistency and
effectiveness



Across all IBM Software brands


Rational/Tivoli/DB2/Lotus and
WebSphere



AOSD delivers tools/technology that facilitates
Business
goals

Driving Organizational Change

IBM Software Group

Technology Enablement and buy
-
in


Enabled use of AspectJ in a newly
-
unified WebSphere product build
environment


All IBM Middleware Developers will be able to create, integrate and deploy AO
-
based solutions.



Products will be able to leverage AOSD in their construction


WebSphere Application Server


WebSphere Portal Server


Lotus Workplace


WebSphere Business Integration


WebSphere Messaging



Developers, and Customers, will be able to use AO technology with
IBM Middleware
, e.g.,


Create AspectJ aspects that add value to J2EE artefacts created by IBM Rational
tools


Deploy these composed J2EE artefacts (e.g., Servlets, EJBs)


Driving Organizational Change

IBM Software Group

Run

Query

Analyse

Query

Report

Alter

EJB Support

definition?

Refactor a

component

Componentization Investigation: Refactoring the WebSphere Container


Concern Modelling



Visualization



Concern
-
based
queries


At one point, Query
capability reported
> 1000 links to
resolve



Refactoring using
OO and AspectJ


Reengineering software


IBM’s and IBM’s customers

IBM Software Group



Scale of the exercise


15000 java source files. Around 1500 packages.

90 components, largest components around 250kloc.


Substantial entanglement complexity



The tools stood up to the test


compiled > 20,000 files with AspectJ


build time ok


queries ran fast


Was an early use of CME query capabilities


Success!!

Reengineering software


IBM’s and IBM’s customers

Componentization Investigation: Refactoring the WebSphere Container

IBM Software Group

Componentization


Realizing the Shared Capabilities of IBM’s Software
Portfolio

Lotus

WebSphere

DB2

Tivoli

Re
-
factor to SWG Product Offerings

Componentization

Lotus

Tivoli

WebSphere

DB2

New or


Enhanced

Capabilities

New or


Enhanced

Capabilities

New or


Enhanced

Capabilities

New or


Enhanced

Capabilities

New or


Enhanced

Capabilities

Shared

Components

Product Offerings

Product Specific

Investment

Shared

Capabilities

Initial

Base

Product

Reengineering software


IBM’s and IBM’s customers

IBM Software Group


Programming
-
level aspects (such as AspectJ) are now becoming
more common, and are easier to develop and use



Benefits of AOSD can be applied to entire solutions
:


A solution may have elements in multiple programming languages
(such as Java or C), or in XML (such as BPEL)



Considering entire solutions is becoming technologically easier
(SMD, EAR etc.)



Applying aspects can be ‘taken up a level’ from direct coding

Re
-
engineering Software Engineering

Solution Level Aspects

IBM Software Group

Re
-
engineering Software Engineering

Solution Level Aspects

Examples of Solution
Level Monitoring and
Measurement Aspects:


“Generate a business
event every time a
customer requests a
price quote over $500”


“Measure how long it
takes to update customer
details in the database”

Consider this Insurance Broker application dependent on Insurance company Web
Services. Many distinct artefacts, e.g., Web Service can be called from BPEL and EJB.

CME will provide the underpinning cross
-
artefact capability

IBM Software Group

CME


Concern Manipulation Environment


Broad View of Software Engineering


There's more to software engineering than just writing code


There are more artefact types than just code (descriptors, UML designs, …)


There are many phases in the engineering lifecycle


Cross
-
cutting concerns also apply at these phases, and


Integration needs are just as profound at design time, or deployment time, as development



CME


Is an open source IDE, integrated with Eclipse


Novel capabilities, tools to aid in full
-
lifecycle engineering



Enables consistent manipulation across multiple artefact types in an integrating
platform:


With control over everything that goes into a product
-

e.g. end user docs,
build scripts, deployment descriptors, WSDL files, …


Supporting multiple AOSD approaches (e.g., AspectJ, Hyper/J)


With components, frameworks for tool builders


With integration points on many levels



A unifying (meta
-
)model based on
concerns


Integrating different AO approaches


Re
-
engineering Software Engineering

IBM Software Group

IBM Roadmap

Core Technology



2003


2004


2005


2006



Driving Organizational Acceptance/Adoption

Reengineering Software

Enhancing the quality of software

Reengineering Software Engineering

FUTURE

New Services Offerings

IBM Software Group


Continuous improvement in core technologies


Focus on extending cross artefact capability


Through CME


Drive use up the software stack


With Solution Aspects; with integration into Rational Tools



Broader technology exploitation across and within the
products


For critical Qualities of Service


To
enable

componentization needed for customer (and IBM)
flexibility


Future

IBM Software Group

Future


Bringing the next level of AO value and capability to
customers requires:


first class support in design and development tools


E.g., Rational Development tools


first class support in the core runtime servers


E.g., WebSphere Application Server, Portal Server, BI Server, etc.


first class representation in the programming model.


E.g., Rational XDE Developer



And


bringing value to the level of Business Modelling


IBM Software Group

Challenges for the AOSD Research Community


Scalability through ‘complexity reduction’


Commercial software is large and complex


Experience with Container refactoring, and with legacy re
-
engineering provide some experience and challenges in tool
scaling


But future (and legacy) applications may well be even larger



Cross Artefact Querying and Composition


Essential for robust, full
-
solution integration


CME is an important start



Organizational Flexibility


“Organizational aspects” (e.g., Problem Determination, or
Serviceability, organizations) are assisted with AOSD technology


It is a transformational technology


What is the right organizational structure?


Who owns cross
-
cutting code?

IBM Software Group

Challenges for the AOSD Research Community


Standards: Do we need them?


For commercial adoption at the end
-
user level


Yes.


Standards will be important to allow customers, ISV’s to have flexibility
and to preserve investment



Complexity versus Simplification.


Does AOSD really help reduce complexity?


Need work toward gaining understanding of this question


But clearly
WE

think it
DOES



AOSD introduces its own learning curve


Consequences for industrial adoption?


Who will be the practitioners?

IBM Software Group

Our Conclusions


AOSD’s time has come.


The Software Industry needs it, and IBM is using it now.


Our customers stand to benefit significantly.



IBM is taking AOSD very seriously


From a technical and business perspective


AOSD has development impact today across all major IBM
brands



Tivoli, WebSphere, DB2, Lotus, Rational


Takeup in IBM is growing


no longer a “push”; there is now a
lot of pull from across IBM’s development teams


Future impact will become more visible in IBM’s runtimes and
in development tools

IBM Software Group

Trademarks


AspectJ is a trademark of Palo Alto Research Center Incorporated



Java and all Java
-
based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries,
or both.



Microsoft, Windows, Windows NT, BizTalk, and the Windows logo are trademarks of Microsoft Corporation in the
United States, other countries, or both.



Gartner is a registered trademark of Gartner, inc., or its Affiliates



Solaris is a trademark of Sun Microsystems, Inc. in the United States, other countries, or both.



UNIX is a registered trademark of The Open Group in the United States and other countries.



Intel, Pentium, Pentium Pro, Pentium II, Pentium III are trademarks or registered trademarks of Intel Corporation or
its subsidiaries in the US and other countries.



HP
-
UX is a registered trademark of Hewlett Packard Company.



Linux is a registered trademark of William R. Della Croce, Jr. (last listed previous owner was Linus Torvalds)



"SAP is the trademark of SAP AG in Germany and in several other countries.



AIX, AS/400, Blue Gene, BlueDrekar, Lotus, Tivoli, Rational, XDE, Z/OS, DB2, Deep Blue, Deskstar, Discoverylink,
IBM, Microdrive, OS/390, Scrollpoint, ServeRAID, Thinkpad, TransNote, Travelstar, Ultrastar, Websphere, Workpad,
are all trademarks and registered trademarks of International Business Machines Corporation in the United States
and/or other countries.



IBM Software Group

Thank You!


Questions?