Release Management for the UBS Data Warehouse Project

separatesnottyΛογισμικό & κατασκευή λογ/κού

25 Νοε 2013 (πριν από 3 χρόνια και 7 μήνες)

59 εμφανίσεις

©1998, 1999, 2000, 2001 Rational Software
-

All rights reserved

Session CM11

Release Management

for the UBS Data Warehouse Project


Friedrich Lehn

UBS AG, Switzerland

friedrich.lehn@ubs.com

Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Agenda



Project Overview


Project Infrastructure


Change Management


Release Process


Summary


Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Project Overview

UBS



Global, integrated investment services firm

and leading bank in Switzerland


World’s largest private bank


Total client assets over US$ 1.5 trillion


Acquired in 2000

Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Project Overview

Data Warehouse Program (DWP)



Establish common infrastructure for analytical data
processing


Provide a business oriented set of data warehouse

and data mart services


Standardized business data model


Align the bank’s data mart portfolio


Improve flexibility, time
-
to
-
market and data quality

Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Project Overview

Data Flow

System of Records

Common Data / Business Warehouse

Data Marts

extract, condition & load

data mart sourcing

visualization

business user

Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Project Overview

Delivery Streams


Main organizational element:

team working on a subject data area / data mart


Three letter acronym as base for naming standards


Standardized infrastructure:

UNIX directories, access group, meta data area, ...


Each delivery stream has business responsible, data
modeler, database administrator, delivery stream manager


Team size typically between 1 and 5


Delivery streams release independently from each other

Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Project Overview

Application Structure


<MAR>

<MAR>

...

RCL

MDR

FDS

System of Records

DSF

DSF

Sourcing Framework

<SDA>

<SDA>

...

RCL

release control tools

MDR

meta data repository

DSF

DWP sourcing framework

FDS

feed configuration files

<SDA>

subject data area

<MAR>

data mart

Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Project Infrastructure

System Environments



IBM AIX SP2 cluster


DB2 UDB EEE


PowerCenter V1.7


DWP Sourcing Framework


Cognos / Business Objects


ClearCase V3.2



Development

Production

Test

Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Project Infrastructure

Logical Environments and Release Structure


Development

Test

Production

D

A

P

delivery stream development

T

framework development

framework test / delivery
stream migration to new
framework releases

X

F

V

E

emergency releases

mandatory

optional

Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Project Infrastructure

Release Cycles


delivery stream development:

D





P


framework development:

T








V

after sign
-
off:

X








P


migration to new framework

releases:

[DAP]





F


emergency releases:

[DAP]





P

Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Project Infrastructure



Two areas:

/dwp_root

release area, version controlled

/dwp_data

dynamic data, archival on demand


Additional directory level in order to support more than one
logical environment on one system


/dwp_root is organized by delivery streams, e. g.:

/dwp_root/d/streams/rcl/bin


~<user>/dwp_root

user specific development area

Directory Structure


Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Project Infrastructure

Directory Structure (continued)


/dwp_data

is organized by logical processing steps, e. g.:

/dwp_data/p/data/landing

(landing area)

/dwp_data/p/data/tgtfiles

(target files)

/dwp_data/p/logs/system

(framework log area)


Tool support for generation of directories in source
environments (delta processing)


Automatic creation of missing directories in target
environments by release procedures

Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Change Management

Design Principles


Support different, clearly separated environments with
different responsibilities


All environments have identical structure (products,
databases, server configurations)


All program changes are done on the development system


All changes on test and production systems go through the
release process and are clearly tracked


Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Change Management

ClearCase Set
-
up


One single VOB
/vobs/dwp


Fully automated access layer (
freeze

and
deliver

routines)


Same directory structure as below
/dwp_root
, directories are
automatically created


In general: only linear version trees (
important
:
synchronization with database change management)


Branch support planned for emergency releases and
migration path only


Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Change Management

Release Naming


<delivery stream>_<major>.<minor>.<patch>

e. g.:

RCL_1.1.0


<major>

major release number


(high level “wave” planning)


<minor>

minor release number


(delivery stream development plan)


<patch>

patch level (bug fixes)



Emergency releases:

RCL_1.1.0_sos_<#>

Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Change Management

Versioning Example


RCL_1.0.0

RCL_1.0.1

RCL_1.0.2

RCL_2.0.0

RLS_P

RLS_A

common.pm


/main/1

/main/2

/main/4

/main/5

/main/3

RLS_A

current version in


user acceptance test

RLS_P

current version in production

/main/sos/1

RCL_1.0.0_sos_1

Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Release Process

Change Control Board


Responsible for high level planning and impact analysis


Defines release scope and release numbers on base

of delivery streams


Assigns responsibilities (delivery stream manager, data
modeler, database administrator, business responsible)


Result is documented in “wave plan” document

Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Release Process

Roles & Responsibilities

developer

development, unit and integration testing


delivery stream manager

manager, release planning


database administrator

database change control


release manager

deployment, tracking, configuration control,


administration

Role

Responsibility

Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Release Process

Process Overview


Development

Production

Test

ClearCase

Freeze

Deliver

Receive

Deployment Package

Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Release Process

Release Objects


PowerCenter mappings


scripts / SQLs


Job dependency data


Uniserv jobs (address conversions)


Database objects (see below)

not included: documentation (intranet database)

Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Release Process

Database Objects


tables (regular, summary)


views (business, security)


keys (unique, primary, foreign)


indexes


triggers


aliases


check constraints

not included: table spaces (system dependent)

Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Release Process

Release Procedure

Responsible

1. Submit database change request *)

delivery stream manager

2. Implement database changes *)

database administrator

3. Prepare release area

delivery stream manager

(UNIX, PowerCenter, job dependencies, Uniserv)


4. Submit release request

delivery stream manager

5. Prepare release area (DDLs) *)

database administrator

6. Create new release (
Freeze
)

release manager

7. Create deployment package (
Deliver
)

release manager

8. Apply database changes to target system *)

database administrator

9. Install release in daily deployment window

release manager

(
Receive
)

(IT integration / IT operation)

*) in case of database changes only

Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Release Process

Freeze Process

freeze <delivery stream> [
-
patch |
-
minor |
-
major |
-
sos <release> ]


1. Retrieve previous release

2. Compare with release area (check for new, changed, deleted files)

3. Display results and ask for confirmation

4. Apply changes in ClearCase

5. Create and attach release label

Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Release Process

Deliver Process

deliver <delivery stream>
-
t <target env.> [
-
r <release> ] [
-
a ]


1. Retrieve specified / latest release in ClearCase

2. Retrieve target environment file versions and create delta

3. Use
-
a(ll) for initialization / synchronization

4. Create deployment package (tar file + control file)

5. Update target labels

6. Lock release label

Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Release Process

Receive Process

receive


1. Check hand
-
over area for pending releases

2. Remote copy deployment package to target system

3. Install it

4. Standardized post
-
installation steps: e. g. access permissions

5. Delivery stream defined post
-
installation steps (
PostInstall.ksh

file):

e. g. for non
-
standard path names, generators, setuid bits

Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Release Process

Release

Request (1)

Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Release Process

Release

Request (2)

Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Release Process

Release

Request (3)

Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Release Process

Database Change Management


PATROL DB
-
Change Manager by
bmc

software


Scope filter
: assign database objects to delivery stream

(


view in ClearCase, name equal to delivery stream)


Apply database changes to development database


Create

release baseline
:

freeze all database object versions for a delivery stream

(


label in ClearCase, name equal to release label)


Export DDL to release area

(for documentation & change detection)

Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Release Process

Database Change Management (target system)


Target baseline
: target database version

(delivery stream plus timestamp as name)


create delta DDL depending on release baseline

and latest target baseline (PATROL)


apply delta DDL


create new target baseline

Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Release Process

Database Retrofitting Process


Rationale: certain database changes have to be applied
and tested directly on the Production system (load
performance optimization: indexes, summary tables, ...)


In order to include target system changes into next regular
release, changes are promoted back to Development
using the
Retrofitting Process


In principle: new, database administrator driven release in
ClearCase and PATROL that is not delivered

Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Release Process

Releasing Meta Data


PowerCenter and JDM data are stored in the database


Uniserv jobs are stored in vendor specific directories


Unload utilities to export the corresponding data

and store it in the delivery stream
´
s release area (tar file)


Meta data is frozen and delivered together with all other

file system objects


Load utility for loading objects in target environment


e. g.:
meta_jdm <delivery stream> [ unload | load ]

Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Release Process

Release

Database


Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Release Process

Release Database

(continued)


Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Summary

Experiences


Over 1500 releases since May 2000


Effort for creation and installation of new release:

5
-

60 minutes depending on amount of meta data

(without database changes)


Main effort necessary for handling of meta data


No ClearCase problem encountered so far

Color Palette

To use or remove

these color palettes, go to

View/Master/Slide Master

Optional logo for your

notes/handouts slides

APPLYING THESE COLORS

1
.

2
.

3
.

Click on the desired color

Click on the paintbrush
tool located on your
toolbar

Click on the object you
want to colorize

Helpful tip: Double click the
paintbrush tool to apply

color to more than one

object at a time.

Summary

Wish List


Transaction concept for set of cleartool commands:

What happens if the 199th check
-
in of 200 fails? Roll back?


Signal handling option for cleartool:

Today it is not possible to ignore SIGINT in cleartool

(


manual clean up)

Questions?

Thank You!

Friedrich H. Lehn

friedrich.lehn@fhlConsult.com

www.fhlConsult.com


This presentation will be posted by tomorrow at:
http://www.rational.com/ruc