Data Quality Management

triparkansasData Management

Oct 31, 2013 (4 years and 7 days ago)

107 views

Data Quality Management

Data quality management (DQM) is the pipeline process that checks the data for required values, valid
data types, and valid codes. You can also configure DQM to correct the data by providing default values,
formatting numbers and
dates, and adding new codes.

To apply data quality management to the data loaded into the system, you configure data quality
management rules (or DQM rules). DQM rules can perform a variety of repair, clean up, and
standardization functions on incoming ide
ntity data values, such as properly formatting numbers,
identifying and correcting clerical or transposition errors, and identifying and correcting intentional
inaccuracies introduced by those intent on trying to conceal their identities.

For example, the
date format for your system is DD/MM/YYYY. But in several of your data sources, the
date values are formatted as MM
-
DD
-
YYYY. You can add the DQM rule 204 to the

<NUMBER>

UMF
segment, configuring it to fix all incoming dates formatted as MM
-
DD
-
YYYY to the d
ate format of
DD/MM/YYYY.

Table 1. Examples of some possible derivatives for the root names of Richard and Mohammad

Derivatives

Root

Dick, Dickie, Ricardo

Rich, Richie, Rick

Rickey, Ricki, Rickie

Ricky, Rikki, Ritchie

Richard

Mohamad, Mohammad

Mohamed,
Mohammed

Mohammad





Service Offering:

Today, more than ever, organizations realize the importance of data quality. By ensuring that
quality data is stored in your data warehouse or business intelligence application, you also
ensure the quality of
information for dependent applications and analytics.

Service Benefits:



Through the chain of accountability, data quality management increases the confidence the organization has in the
data it manages and the use of that data for accurate decision
-
making.

Making decisions based upon timely and
accurate data can enable an organization to realize both tangible and intangible benefits for increased value throughout
the enterprise.



Managing data quality across the enterprise provides one point of accountabilit
y for the control and use of all
information assets in an organization. Establishing and maintaining a data quality practice offers the opportunity for an
organization to set the focus on managing their crucial information assets.



Data stewardship and gove
rnance are part of a functioning data quality program and provide a common set of
reference data, reduced data duplication and a common vocabulary for all of the enterprise’s information assets. These
benefits can be translated into savings of money and ef
fort across the organization.


Process:

Quality Assessment

In the quality assessment phase, you determine the quality of the source data. The first step in
this phase is to load the source data, which could be stored in different sources, into
Warehouse Bu
ilder. You can import metadata and data from both Oracle and non
-
Oracle
sources.

Quality Design

The quality design phase consists designing your quality processes. You can specify the legal
data within a data object or legal relationships between data
objects using data rules. For more
information about data rules, see

"About Data Rules"
.

You also correct and augment your data. You can use data
quality operators to correct and
augment data. For more information, see

"About Data Quality"
.

As part of the quality design phase, you also desig
n the transformations that ensure data
quality. These transformations could be mappings that are generated by Warehouse Builder as
a result of data profiling or mappings you create.

Quality Transformation

The quality transformation phase consists of runnin
g the correction mappings that are used to
correct the source data.

Quality Monitoring

Data monitoring is the process of examining your data over time and alerting you when the
data violates any business rules that are set. For more information about data
monitoring,
see
"About Quality Monitoring"
.




Within a business intelligence environment, there are several roles that are involved in data quality
management:

-

Program Manager and Project Leader

-

Organization Change Agent

-

Business Analyst

-

Data Analyst

-

Data Steward


Tools, Resources:

-

Mostly paid t
ools from Oracle, IBM and SAP

-

SAP’s data quality solutions


SAP Data Quality Management, SAP Information Steward, SAP
Data Services, SAP Sybase PowerDesigner

-

Oracle Warehouse builder

-

IBM Infosphere

-

IB
M PureData System for Analytics

-

TeraData and Trillium

Products