ODI ST Partnership - NoCOUG

gruesomebugscuffleΛογισμικό & κατασκευή λογ/κού

25 Νοε 2013 (πριν από 3 χρόνια και 4 μήνες)

92 εμφανίσεις

<Insert Picture Here>

Oracle Data Integration

An Overview with Emphasis in DW Appliances



Where Does Data Integration Fit?

Essential Ingredient for Information Agility

SOA


Data Services


Information
services


Process
Integration


Event driven
data integration


Governance
and Impact
Analysis


Business


Intelligence


Data Integration for
BI applications


Event driven BI


Heterogeneous Data
Access


Information
-
based
analytics


Report to Source
Data Lineage

Enterprise Data

Warehousing


Extract, Transform, Load


Data Migration, Bulk Data


Data Quality, Profiling

Master Data
Management

Data

Integration

Oracle Data Warehouse

Breaks Data Bandwidth Bottlenecks


More Pipes


Modular storage “cell” building blocks
organized into Massively Parallel Grid


Bandwidth scales with capacity


Bigger Pipes


InfiniBand interconnect transfers data 5x
faster than Fibre Channel


Ships Less Data Through Pipes


Query processing is moved into storage
to dramatically reduce data sent to
servers while offloading server CPUs


Runs the Oracle Database


Database and options run unchanged

Exadata Moves a Lot
Less Data a Lot Faster

Oracle Master Data Management

Comprehensive MDM Application Solutions

Data Governance

Oracle Fusion Middleware

Analytical
MDM Apps

Operational MDM Apps

Customer

Supplier

Product

Application Integration Architecture

Enterprise schema & Common
services

Install Base

Employee

Financial

Oracle / Hyperion
Data Relationship
Management

Analytical

Site

Operational

Systems

EBS

Siebel

SAP

Custom Apps

External


Apps

PeopleSoft

JDE

Budgeting

Analytical

Systems

DW

BI &Datamarts

Planning

Dashboards

Financial
Consolidation

Use Cases for Oracle Data Integration

Unified Data Integration Platform for Enterprise Projects


Data Integration


Bulk
-
load historical data to new application


Synchronize new and old applications


Complex transformations, CDC


Ensure database
-
level consistency across
applications bidirectional

BI & Data Warehouse


Real
-
time Data Warehouse for BI


Populate Warehouse with High Performance ODI


Aggregates and aligns data for operational
analytics, performance management, etc

Modernization


Mainframe /
iSeries

sources to Oracle RAC


Re
-
architect legacy batch processing to SOA and
ODI
-
EE


Unification of structured and unstructured data

Data Migrations & Consolidation


Upgrade Applications or Migrate to New Schema


Single
-
time Bulk load and/or keep in Sync with
ongoing translation, delivery of data.


Applications merge support


SOA Initiatives


Establish Messaging Architecture for Integration


Incorporate Efficient Bulk Data Processing with ODI


Providing data access, transformation etc. as
services within SOA


Master Data Management


Create Single View of the Truth in real time


Aggregating multiple operational sources


Synchronize Data with ODI
-
EE


Supports master data stores, publishes master
data changes to all consumers

Introducing ODI
-
EE

Key Data Integration Products



Comprehensive Integration


ELT/ETL for Bulk Data


Data Delivery Services


Process Orchestration


Service Bus


Human Workflow


Data Federation


Replication & Migration


Data Governance


Data Grid


ELT/ETL for any DB


Changed Data Capture


Native SQL Code Gen


Declarative Design


ELT for Oracle DB


Database Modeling


PL/SQL Code Gen


Flow
-
based Design


Business Data / Metadata


Statistical Analysis


Time Series Reporting


Integrated Data Quality


Cleansing & Parsing


De
-
duplication


High Performance


Integrated w/ODI

New

Oracle Core Features

Comprehensive Data Integration Platform


E
-
LT Data Processing


Set
-
based Transformation


Configurable Deployment (E
-
LT, E
-
T
-
L, or others)


Native Code Generation


Declarative Integration Designer


Reverse Engineering of Sources


Intuitive Data Mapping


Built
-
in Reporting


Metadata Lineage


Graphical Impact Analysis


Native SOA Interoperability


SOA Sources, Targets, or Invoke
-
able E
-
LT Jobs, SOA Workflow, etc.


Automatic Error Handling with Data Recycling


Changed Data Capture


Built
-
in Support for Realtime Data Capture and Streaming


Log
-
based or Trigger
-
based


Inline Data Quality


Extensible Knowledge Module Framework


Data Integration Core Use Case

Comprehensive, Lightweight Data Integration


Key Architecture Benefits: 100% Java, Open APIs, very fast E
-
LT


Embeddable Java Agent consumes very little CPU or RAM


E
-
LT Architecture uses DBMS CPUs, not the Application CPUs


Open APIs, Open Metadata, Open XML Knowledge Modules

D

A

B

File

C

C$_0

C$_1

LKM

LKM

IKM

I$

E$ (Errors)

CKM

IKM

RKM

JKM


Check
-
Load


Transform

Extract
-
Load

ODI
Agent

ERP

Application

Business Intelligence

& Data Warehouse

ODI Agent may be
deployed in any part
of the architecture


Move and transform data. Mixed
sources and targets.

BENEFITS

KEY DIFFERENTIATED FEATURES

1.
Performance

>>>

Heterogeneous “E
-
LT”

2.
Flexibility


>>>

Event
-
Driven Platform


3.
Productivity


>>>

Declarative Design

4.
Open


>>>

100% Java and SOA Native

5.
Hot
-
Pluggable

>>>

Knowledge Modules

ODI
-
EE Value Proposition

Agile Data Integration

11

12

12

Differentiator
: E
-
LT Architecture

High Performance

Conventional ETL Architecture


E
xtract

L
oad

T
ransform

Next Generation Architecture


E
-
LT


L
oad

E
xtract

Transform

T
ransform

Conventional: Separate ETL Server


Proprietary ETL Engine


Poor Performance


High Costs for Separate Standalone Server


IBM & Informatica’s approach

Oracle: No New Servers


Lower Cost: Leverage Compute Resources &
Partition Workload efficiently


Efficient: Exploits Database Optimizer


Fast: Exploits Native Bulk Load & Other
Database Interfaces


Scalable: Scales as you add Processors to
Source or Target

Benefits


Optimal Performance & Scalability


Better Hardware Leverage


Easier to Manage & Lower Cost

Real Time Data Warehouse

Logs

Logs

Logs

CDC

Records

CDC

Records

CDC

Records

CDC

Records

CDC

Records

CDC

Records

CDC

Records

CDC

Records

CDC

Records

Transactional

RDBMS

Transactional

RDBMS

Transactional

RDBMS

Source Systems

Data Warehouse

Source DB’s

Redo Logs /

Triggers

Change
Tables

Subscription

Views

Staging
Area

Data
Warehouse

Differentiator
: Changed Data Capture

Event
-
Driven Framework for
Realtime

Data Warehouse

14

Focus on
Oracle BIEE Suite Plus

Unify Data Integration with Business Intelligence


Integrated with Oracle BIEE
Plus


Support for relational, R
-
OLAP,
OLAP sources & targets



Report
-
to
-
source lineage


Drill
-
in from a Report


Get query, DW schema, source
tables, transforms


Gain confidence in analytics



Unified infrastructure


Unified metadata & lineage


Integrated Data Quality


Unified data access,


Common administration &
monitoring


Integrated scheduling & security


Common auditing & tracing


Common error handling

Oracle BIEE Suite Plus

Oracle
EBS

PeopleSoft

SAP/R3

Other
Sources

Data
Warehouse

CDC

Message

Queues

Enterprise Data
Warehouse

Design
& Drill

Data
Flow

Bulk and Real
-
Time
Data Processing

Oracle Data Integration Suite

Metadata
Lineage

Data Distribution & Delivery APIs

Bulk/Trickle
Loading

Changed Data

Capture

Master

Data

Data Quality

& Profiling

ODI Knowledge Module Framework

Information
Assets

Oracle Business Intelligence Server

Common Enterprise Information Model

Interactive

Dashboards

Ad hoc

Analysis

Proactive

Alerts

Microsoft

Office

Reporting

& Publishing


Oracle RAC


Teradata


Netezza…etc

Report to Source Data Lineage

Explore Lineage from Target Columns to Source Columns

Focus on
OBI Analytic Applications

Overview


Pre
-
Built BI Analytic Applications


Pre
-
built Metrics, 5000+ Dashboards, Data
Warehouse 350+ Star Schema


Implementation Time: 3
-
4 Weeks


Works with EBS, PSFT, SAP, SEBL, JD
-
Edwards Sources


Biggest Savings: Pre
-
built ETL



Faster & Lower Cost to Build & Maintain
ETLs


Knowledge Module Architecture Enables Efficient
Development of Packaged ETL


Reduce Complexity and Size of Code


Reuse DW Creation, Update, Load Times


Value
-
add Modules (e.g. SOX)



Extensible by Customers & System
Integrators

Oracle BI Analytic Applications

Oracle BI Server and Semantic Layer

Oracle
EBS

People
Soft

SAP/R
3

Other
Sources

Data
Warehouse

CDC

M
es
sa
g
e
Q
u
e
u
es

Analytics Data
Warehouse

Informatio
n Assets

Sales

MKting

Order
Mgt

Supply
Chain

Finance

HR

Contact
Center

Data
Flow

Bulk and Real
-
Time
Data Processing

Oracle Data Integrator

Metadata
Repository

ODI Knowledge
Modules for BI
Analytic
Applications

Bulk
Transform.

Changed
Data
Capture

Data
Integrity

Data
Quality

ODI Connectivity Framework

Pre
-
Built ODI Interfaces and
Transformation Packages for
Analytic Applications

Weeks or months

Pre
-
built BI Apps

ETL
Mapping

DW Design

Training

Years or quarters

Metrics &
Dashbrds

Build from Scratch

50%

Oracle Hyperion
Planning

Oracle Hyperion
Financial Mgt

Oracle Hyperion
Essbase

Planning API

HFM API

Essbase API

Oracle | Hyperion Data Access

Data Services

Authentication

Logging Services

API Layer

Hyperion
Planning

Hyperion
Financial
Management

Hyperion
Essbase

Metadata Discovery

& Model Creation










Extract Data

Use

Essbase KM







Extracts Dimension
Members

Use

Essbase KM







Loads Data










Loads Dimension
Members










Other Features

Cube Refresh

Consolidate

Calculate

Oracle Hyperion Application Adapters

Focus on
Oracle | Hyperion Applications

Leveraging EPM with Embedded Data Integration

Oracle Data Integration Suite

Metadata
Lineage

Data Distribution & Delivery APIs

Bulk/Trickle
Loading

Changed Data

Capture

Master

Data

Data Quality

& Profiling

ODI Knowledge Module Framework

Oracle
EBS

PeopleSoft

SAP/R3

Other
Sources

Data
Warehouse

CDC

Message

Queues

Information
Assets

Bulk and Real
-
Time
Data Processing

Focus on
Hyperion MDM

Foundation for Data Relationship Management (DRM)


Hierarchy management


Derivation/inheritance


Classification, categorization


Attribute management


Change management


Streamlined, automated


Valid
-
from, Valid
-
to ranges


Compare any 2 versions


Business
-
user driven


Easy user declarative tool


Specify business rules


Multi
-
user collaboration


What
-
if & historical analysis


Meets IT requirements


Synchronize with operational systems


Complete auditing, fine
-
grained
security


Configurable, no coding

Hyperion Data
Relationship
Management

Oracle
EBS

People
Soft

SAP/R
3

Other
Sources

Data
Warehouse

CDC

M
es
sa
g
e
Q
u
e
u
es

Oracle Data Integrator

DRM
Import/Export
Profiles

Bulk and Real
-
Time Data Processing

Metadata
Repository

ODI Knowledge Modules for DRM

Bulk
Transform.

Changed
Data
Capture

Data
Integrity

Data
Quality

ODI Connectivity Framework

Information
Assets

18



Oracle
Data
Integrator



&


Oracle
Data
Quality



Oracle
Data
Integrator



&


Oracle
Data
Quality

Hyperion Data Relationship Management

Closed
-
loop Master Data Management

Operational

E
-
Business Suite

JD Edwards

HR

Legacy

SAP

Siebel CRM

People Soft

Hyperion Data Relationship Management

Hierarchy Governance

Master Hierarchies and Versions

Cost
Centers

Entities

Accounts

Org Struct

Geography

Analytic

Financial Apps

Budgeting

Planning &
Forecasting

Analytics

Key Performance

Query / Reporting

Enterprise
Reporting

19

ODI
-
EE A Look Ahead

ODI for Oracle Fusion

Data Integration as a Key Fusion Differentiator


Fusion Applications


Built
-
in, Automated Data Replication among ERP Applications


Fusion Business Intelligence


Pre
-
built Analytics from Fusion ERP Applications


Automatic Mapping and Model Updates (Application
-
driven)


Fusion Master Data Management


Data Replication, Conflict Detection and Conflict Resolution


Fusion w/Essbase


Fusion Financials (Automatic Financial Consolidation)


Pre
-
built Essbase Content with Apps and Business Intelligence

ODI
10gR3

ODI
11gR1

ODI
11gR2

2009

2010

2011

Unified

Team

Unified

Platform

OWB
10gR1

OWB
10gR2

OWB
11gR1

OWB
11gR2

High
-
Level Data Integration Roadmap

Towards a Unified ETL Platform


OWB/ODI Investments are
Fully Protected


No Forced Migrations


Natural Upgrade Path


Unified Platform aims to be
a Superset of Existing
Products


no regression

2009

ODI J2EE Deployment Architecture

Logical Architecture Overview

JDeveloper

ODI in JDev

Designer

Operator

Security

Topology

WLS

Data Sources Connection Pool

Repository Data Sources

Sources Data Sources

Target Data Sources

Repository Data Sources

Sources Data Sources

Target Data Sources

Repository Data Sources

Sources Data Sources

Target Data Sources

Servlet Container

ODI Agent

ODI Thin Client

Web Service Container

ODI Transform.
Service

ODI Data
Services

Any Application Container

Any Application

ODI SDK APIs

MBeans Server Registry

Any MBeans App

ODI MBeans
for ODI Agent

Sources and Targets

ERP

---

---

---

---

---

---

CRM

-

-

-

Legacy

---

---

---

---

---

---

---

---

---

---

---

---

Best
-
of
-
Breed Applications

PLM

RDBMS


ODI Repositories

ODI Master
Repository

ODI Work
Repository

ODI Work
Repository

ODI Work
Repository

Any Java App

Any Java App

Any Application

ODI SDK APIs

Any Web App

ODI 11gR1 New Features

Availability Planned for 1HCY10 Timeframe



JDeveloper
-
based Tooling


J2EE deployment option (using Weblogic Server)


Standalone Option is Preserved


using Jetty


JPS
-
based Security (external password management)


Enterprise Scheduler (ESS) Integration


Enterprise Manager Integration


New Thin Client (consolidated + more mgmt features)


Set
-
based Operators and Temporary Index Management


Java SDK APIs for Improved Embedability


JDE Enterprise One Knowledge Modules (v8.x and higher)


SAP ERP Knowledge Modules (v4.6 and ECC 6.0)


SAP iDocs Knowledge Modules (v4.6 and ECC 6.0)


SAP BW Knowledge Modules (v3.5 and v7.0)

24

Customer Examples

Enterprise Data Warehouse

Raiffeisen Bank


Meets legal requirements for data consolidation in the head office

(Basel II, Investor reporting)


Seamless and realtime Capital Market reporting to investors and stakeholders


Single
-
source of truth for business information


supports Business Intelligence and EDW


One standardized enterprise data model over the whole RZB Group


Reduction of core integration/development costs through central development practice


Reduced time to market for new Group applications through central interface development


Performance Issues with existing ETL Tool
(IBM DataStage)


New functions required for the next DWH
releases (realtime, replication, etc)


Business Challenges

Oracle Data Integration Solution Benefits

IT Challenges


C
osts

for Enterprise Data Warehouse
infrastructure were growing too fast


EDW processes not keeping up with 24x7
operations, ETL batches too slow


Order Replenishment, Online Catalog

J. Crew


Simplified the end
-
to
-
end data integration with all their core IT systems and database
platforms to reduce the total cost of ownership of their order replenishment and cataloging


Benefited from the Oracle advantage in batch processing, bulk data transformation for
increased performance


Successfully leveraged SAP and Teradata integration to phase out legacy systems and
ultimately reduce costs


Order replenishment for online catalog system


Define new corporate standard for data integration


Better integrate data across 175 retail stores and 52
outlet chains


Business Challenges

Oracle Data Integration Solution Benefits

IT Challenges


Replace antiquated home
-
grown solution of scripts


Teradata bulk data support, and help integrate data
across SAP, Red Prairie Inventory systems


Financial Services, Reinsurance

New Reinsurance, Division of the Munich Re Group


Seamlessly integrated across (Oracle applications (E
-
business Suite, Siebel CRM) SAP,
IBM, etc) to manage the regeneration of all code, scripts, and metadata across the
different environments.


Lowered the cost of development by taking advantage of ODI’s declarative model, ease of
use, and team collaboration capabilities.


NRC benefited from the flexibility and standards based approach of ODI, helped leverage
the existing environment.


Needed quality of data but uncompromising on the
cost to meet that objective.


Data interactions between multiple complex
systems, including both SAP and homegrown
insurance applications

Business Challenges

Oracle Data Integration Solution Benefits

IT Challenges


Hard to modify and maintain manually
-
developed
data transfer programs.


Data transfers as part of reinsurance, risk
transactions

Improve Inventory Control

Ross


Seamless data integration from heterogeneous sources leveraging SOA helped to create
a more flexible, standards based integration approach


73% Bulk data transfer performance improvement


Together with Oracle SOA Suite (Oracle BPEL PM) provided business optimization,
process visibility, exception handling


Closed loop processing using BAM and Data Integration combined eliminated
ordering/replenishing inventory errors by reducing inaccurate data


Hard to consolidate consistent, accurate inventory
information across over 800 multiple store chains


Lack of visibility to errors to the business


Higher cost and time to delivery of new value

Business Challenges

Oracle Data Integration Solution Benefits

IT Challenges


Current state was complex, didn’t scale, and difficult
to manage


Non
-
standard approach required coding paradigm,
inconsistent error handling

Dain Hansen

Shared Data Services

Verizon


ODI + SOA helped to build out a shared services layer which helped provide greater
flexibility


These shared services consolidate database (DB2, Sybase, Informix to Oracle),
consolidate ERP (PeopleSoft HR/Payroll, Decommissioning SAP), consolidate middleware
(Oracle FMW) and consolidate BI/ETL


In addition the program helped to decommission Cystal and Cognos and adopt Oracle
BIEE


Streamline and automate business processes
around new hire on boarding


Provide better business insight and manage FiOS
supply chain

Business Challenges

Oracle Data Integration Solution Benefits

IT Challenges


Lower the cost of rewriting SAP, PSFT & Legacy
interfaces (about 209) through custom code


Lower the cost of IT infrastructure from multiple
systems


Dain Hansen

Sales and Marketing Data Warehouse

Nestle


Replaced weekly insertion with daily recording of change in status with time imprint for all
clients

allowing for more effective client follow
-
up, for example


Improved response times by transferring reporting processes to the data warehouse


Improved reactivity for restocking requirements thanks to daily updates on stock quantity
and value for each store


Provided overall view of the status of business processes, resulting in improved user
communication


Hard to consolidate consistent, accurate inventory
information across over 800 multiple store chains


Lack of visibility to errors to the business


Higher cost and time to delivery of new value

Business Challenges

Oracle Data Integration Solution Benefits

IT Challenges


Manage a data warehouse for operational
transactions and marketing information


Scale available for high
-
value
-
added actionable data
on clients, orders, stock, etc.

Dain Hansen


Enable the marketing and CRM team to better target
its campaigns and activities across 50 countries


Anticipate future reporting challenges, in light of the
strong increase in the volume of production data

Questions

The preceding is intended to outline our general
product direction. It is intended for information
purposes only, and may not be incorporated into any
contract. It is not a commitment to deliver any
material, code, or functionality, and should not be
relied upon in making purchasing decisions.

The development, release, and timing of any
features or functionality described for Oracle’s
products remains at the sole discretion of Oracle.