Metadata and Metadata and Data Standards Data Standards

© Tata Consultancy Services ltd.
October 21, 2013
1
Metadata and
Metadata and
Data Standards
Data Standards
Levels of Metadata
Levels of Metadata
C. Anantaram
C. Anantaram
Innovation Lab
Innovation Lab
2
Business today
•
Data collected at various touch points
•
Critical business decisions made based on their output
Multiple Data Entry Points
Customer touch point
Enterprise-wide

data
Legacy Web
Mergers &
acquisitions
Central Database /
Data Warehouse
Customer support
& service
Sales force
automation
Marketing automation
Mining servers
Multiple Data Exit Points
3
Levels of Metadata
•
Ontology Level
–
Basic domain entities and their relationships
–
Dublin core, RDF-schema, OWL
•
ETL level
–
Mechanisms for describing data for cleansing and standarizing
–
Various tools exist; define data formats, define rules
•
Datawarehouse level
–
Mechanisms to describe Business Metadata, Technical Metadata
–
Data dictionary
•
Legacy Application level
–
Input and output from a legacy system
•
Components level
–
Consistent, comprehensive, core data across enterprise
–
Metadata about enterprise information and common business
processes
4
Ontology Level- RDF schema
•
Describe domain ontology (terms and relationships)
using various semantic web formats – Dublin Core,
RDF, OWL etc.
•
Has a set of pre-defined ontology that can be used
–
Class, Resource, Property
•
Can define our own classes and sub-classes
•
Has domains and ranges to specify the classes for
the subject and object
–
Example
:Book
rdf:type
rdfs:Class .
:bookTitle
rdf:type
rdf:Property .
:bookTitle
rdfs:domain
:Book .
:bookTitle
rdfs:range
rdfs:Literal .
:MyBook
:bookTitle
“My Book” .
5
Web Ontology Language
Web Ontology Language (OWL)
•
An ontology formally defines relations among terms
•
Enables definition of domain ontologies and sharing
of domain vocabularies
•
Modeled thru an object-oriented approach
•
Structure of a domain described as classes and
properties
•
Equivalent to Description Logic (DL)
•
Provides more vocabulary than RDF Schema
–
Relations between classes : disjointness, intersection etc.
–
Cardinality : exactly one, min, max etc.
–
Equality : equivalent class, equivalent property, sameAs
–
Characteristics of property : symmetry, transitive, etc.
6
ETL level
Cleansing ensures clean, de-duplicated data
•
Duplicate data
–
Robert Smith, 100 E Johnson Street and Dr. Robert J
Smith, 100E Johnson St.
•
Data integration
–
Mumbai 21 and Mumbai 400021
Customer support
& service
Sales force
automation
Marketing automation
Mining servers
Multiple Data Exit Points
E-T-L
Central Database /
Data Warehouse
Multiple Data Entry Points
Customer touch point
Enterprise-wide

data
Legacy Web
Mergers &
acquisitions
7
Datawarehouse level
8
Datawarehouse level
•
Business Metadata is the business terminology used by
the business users
–
for example, Bankers conversing on EOD FX open positions in
the Treasury Foreign Exchange Trades for various currencies.
•
Technical Metadata is the technical terminology used
by the BI System implementation / support team
–
for example, a ‘support’ person talking about the D_FX_POS
data load routine failed which was populating the report
FX_OPEN_POS.rep in the reports category ‘Treasury Reports’
•
Data Source Metadata is primarily the data dictionary of
the transaction processing systems that are used as
Data Sources for the BI system
–
for example, the list and descriptions of data fields in the FX
TRADE system in the Treasury Department.
9
Legacy Application level
•
Data as part of input output of legacy applications
•
Different users have different views and needs
Multiple Legacy Systems
Legacy App 1
Legacy App 2
Legacy App3
Metadata
Definition,
Access and
interface
User 1
User 2
User 3
User 4
Different types of users
10
Components level
Core Business Information
•
Focus on Core information for Business applications and encapsulate
them into components. Examples are:
–
Organization structure, Operating units, Distributor network / Demand
chain, Supply chain, Manufacturing units, Currencies, Financial Periods…
–
Static Information – Territories, Countries, States, Cities, Transaction
types, Product Brands, Product Categories, ……
–
Payment methods, Credit terms, Discount schemes, Incentives
–
Taxes – Sales Tax, Excise and Customs, Octroi, …
–
Manufacturing operations, Standard Costs
11
Components level
•
Example: Payment Term Object:
•
Standards for Calculation of Payment Due Date on a given
sale / purchase
•
Standards for Calculation of interest and penalty, etc.
•
Example: Currency Object:
•
Standards for Computation of currency rate as of a
particular date / time
•
Currency rate history
•
Standards for Calculation of base currency amount
•
Conversion of amounts between currencies
•
Calculation of currency gain/loss due to sale amount in a
given currency
12
Thank you !