IBM Presentations: Blue Pearl Deluxe template

honorableclunkSoftware and s/w Development

Oct 30, 2013 (3 years and 7 months ago)

60 views

IBM India Research Lab

© 2004 IBM Corporation

ProACT: A Solution for Contact Center Analytics

Unstructured Information Management Group

IBM India Research Lab


Shourya Roy
<rshourya@in.ibm.com>

Behind the Scene: Raghu, Sree, Diwakar, Rahul,
Shantanu, Sumeet, Venkat…

Unstructured Information Management Group

IBM India Research Lab

2

I have been associated with IBM Research since 2002

Brief Intro

Working in the area of text analytics

Prior to that between 2000
-
02, I used to be seen
mostly in H1 Mess, TT Room and TV Room
-

sometimes in CSE Dept. classrooms and rarely in my
advisor Prof. Soumen Chakrabarti’s office


Unstructured Information Management Group

IBM India Research Lab

3

Contact Center : Application of Structured
-
Unstructured
Data Integration









Contact Points

Branch
office

Web

IVR

Call
Center

Customer

Enterprise

Products

& Services

Integrate & Analyze Structured

& Unstructured Data

Unstructured

Call logs & transcripts

Emails, Surveys

Self Service

Agent

Structured

Customer/Product

Transaction Data


Instant Market Intelligence


Customer preferences


Dissatisfaction Drivers


Lifetime Value Management



Analyze Agent Performance


Improve C
-
Sat, Upsell Rate

Analyze Contact Drivers


Improve FAQs, Web pages

Structured

Agent

Data


Automation of

C
-
Sat analysis


Analytics for

Agent Performance

Customer Preferences


Unstructured Information Management Group

IBM India Research Lab

4

I will be Talking About


Customer Satisfaction Analysis


Scenario and Importance


Solution and derived BI


Issues


Agent Performance Analysis


Analysis of telephonic transcriptions to identify scope of
improvement in a contact center


Automatically Building Domain Models


Automatically building
Domain Models
from noisy
telephonic transcriptions


Possible applications

Unstructured Information Management Group

IBM India Research Lab

5

I will be Talking About


Customer Satisfaction Analysis


Agent Performance Analysis


Automatically Building Domain Models

Unstructured Information Management Group

IBM India Research Lab

6

Customer Satisfaction (C
-
Sat)


Wikipedia says


Customer satisfaction is a business
term which is used to capture the idea of measuring
how satisfied an enterprise's customers are with the
organization's efforts in a marketplace.



In BPO scenario, it is crucial from client’s point of
view to monitor QoS provided by Contact Center


C
-
Sat analysis is mostly a part of the agreement
between Contact Center and client


C
-
Sat is different from SLAs

Unstructured Information Management Group

IBM India Research Lab

7

C
-
Sat Scenario

Email DB

C
-
Sat DB

Report

1

9

9

7

Query

Response

Feedback
Request

Feedback

Customer

Specialist

Analysts

Domain


Knowledge

Immediate and


helpful response

Unstructured Information Management Group

IBM India Research Lab

9

Sample Verbatims with Labels

Unstructured Information Management Group

IBM India Research Lab

12

Architecture

Integrate & Analyze Structured

& Unstructured Data

Transcribe

Speech

Emails, Surveys

logs

Language Skill &

Cust
-
Sat annotators

Upsell/Product Sentiment

Annotators

Business Intelligence

Explore/ View/Report

Self
-
Service


Speech


Web

Application

Specific Data

Integrated Views of

Agent Performance

Summary of customer

Views on products

Identification of Cross

Up Sell opportunities

Agent Training, Deployment

Market Intelligence

for

Enterprise Clients

Unstructured Information Management Group

IBM India Research Lab

13

Demonstration

ProACT

BI Tool

BI

Unstructured Information Management Group

IBM India Research Lab

14

Unstructured Information Management Group

IBM India Research Lab

15

Unstructured Information Management Group

IBM India Research Lab

16

Unstructured Information Management Group

IBM India Research Lab

17

Unstructured Information Management Group

IBM India Research Lab

18

Challenges


Technical Challenges


Too many class labels(>35) and insufficient data(<10 cases
for some of the categories). Examples are
IncompleteResolution
,
CannedResponse
,
PolicyIssues
.


Grouped into a higher level categories. Examples are
Resolution,
Communication, Uncontrollables


Short, poorly written text


Noisy Data


No fixed rule for manual labelling leading to inconsistencies


Same/similar verbatims being assigned different labels by human
labellers


Changing labels


Labels tend to change over time



Business Challenges


Smooth transition from exisiting manual C
-
Sat analysis process to a
complete automated one



Unstructured Information Management Group

IBM India Research Lab

19

Summary


Accuracy


Subjective as manual labelling is not accurate


Ballpark accuracy figures range from 60
-
75%


Going forward


Real
-
life deployment in different contact centers


Insightful Business Intelligence (BI) Tool


Can we introduce C
-
Sat analysis in a new process
without requiring any training data?

Unstructured Information Management Group

IBM India Research Lab

20

I will be Talking About


Customer Satisfaction Analysis


Agent Performance Analysis


Automatically Building Domain Models

Unstructured Information Management Group

IBM India Research Lab

21

Call Analysis to Improve Sales and Agent
Performance


Scenario


A car rental process outsourced to a call center


people calling up to rent cars


Objective


Call centers want maximum number of car bookings as
well as car pick ups


INCREASE agent
conversion rate


Approach


Analyse transcriptions of telephonic conversation and
find out the key
actionable

and
differentiating

insights

Architecture

Unstructured Information Management Group

IBM India Research Lab

22

Unstructured Information Management Group

IBM India Research Lab

23

Application of TAKMI for CRM

Customer Contact Center

Textual Data

Calls are

transcribed

Analysis of reasons

for agents making

Customers rent cars

Agent
Language

(choice of

phrases etc.)
and
compliance to
guidelines

AAA Member

Segment

Characteristic in

a Customer

Segment

Agent

Contact Center

Textual Data

Enter Bookings

information

Customer

Evaluate Effects

of different

segments

Analyze Reasons

and Retry

Investigate for
Enhancements


(to other agents


and cust segments)

Knowledge

Campaign

Customer

Models

Application of TAKMI to Customer Relationship Management in Car Rental Process

Evaluation

Unstructured Information Management Group

IBM India Research Lab

24

Highlights : Identifying Actionable
Insights


Detect customer intent at start of call and suggest
actions.


Weak start (
“I would like to know the rate”, ”I just want to
get a price on midsize car”
)


Strong start (“
Hey, I would like to pick up a car”, “I need to
make a reservation please”
)


In weak start case, “pick up” is
improved

by
mentioning
discount phrases
.


In strong start case, “pick up” is
concretized

by
mentioning
value selling phrases

and
discount phrases
.


Asking for
clean driving record

decreases

“pick up” in
strong start case


Unstructured Information Management Group

IBM India Research Lab

25

Highlights :
Call Flow and Compliance


Mandatory questions for call process checking


1. Brand name in opening



“welcome to Alamo”


2. Proper opening




“My name is”


3. Confirms age 25




“age 25”


4. Confirms check/debit card in their own names

“check card in your name”


5. Confirms clean driving record and license

“you need clean driving record”


6. Ask for future reservations



“anything else “


7. Brand name in conclusion



“thank you for calling National”


In 137 reservation calls…


Agents are not confirming “age over 25” in 36% calls.


Agents are not confirming “clean driving record” in 44%
calls.

In total 936 calls…


Agents are not starting with brand name in 11% calls.

Unstructured Information Management Group

IBM India Research Lab

26

Difference between Strong and Weak Start

pick up information

Customer intent at start of call

Based on the customer’s start, “not picked up (NS or CC)” is predictable.

65%

35%

23%

9%

49%

8%

pick up

not picked up

strong

weak

Unstructured Information Management Group

IBM India Research Lab

27

Difference between
pick up

and
not picked up

in Weak Start

10/13 * 100 = 76.9 %

9/21 * 100 = 42.9 %

Discount relating phrases

are mentioned by the agent
more frequently

in
“pick up” data.

pick up

not picked up

21

13

10

9

Number of calls containing
discount relating phrases

Unstructured Information Management Group

IBM India Research Lab

28

Detection of Improper Call Process (cont)

In 16 reservation calls, only less than 3 questions are mentioned.

How many mandatory questions are mentioned by the agent ?

Unstructured Information Management Group

IBM India Research Lab

29

Detection of Improper Call Process (cont)

In these 16 reservation calls, 2 questions are not mentioned at all.

Unstructured Information Management Group

IBM India Research Lab

30

I will be Talking About


Customer Satisfaction Analysis


Agent Performance Analysis


Automatically Building Domain Models

Unstructured Information Management Group

IBM India Research Lab

32

Scenario


Call centers handle customer complaints, issues
for computer sales to mobile phones to apparels


Typically domains have manually created domain
models which contain types of problems solved in
each category, solutions library, typical question
-
answers, appropriate call opening and closing
styles etc


Each instance in a domain requires separate
domain model


These models are dynamic in nature and change
over time

Our objective is “automatic generation of domain models from
largely available
noisy
transcriptions of telephonic conversations
between call center agents and customers”

Unstructured Information Management Group

IBM India Research Lab

34

Example: Snippet of Automatic Transcription

SPEAKER 1: windows thanks for calling and you can

learn yes i don't mind it so then i went to

SPEAKER 2: well and ok bring the machine front

end loaded with a standard um and that's um it's

a desktop machine and i did that everything was

working wonderfully um I went ahead connected

into my my network um so i i changed my network

settings to um to my home network so i i can you

know it's showing me for my workroom um and then

it is said it had to reboot in order for changes

to take effect so i rebooted and now it's asking

me for a password which i never i never said

anything up

SPEAKER 1: ok just press the escape key i can

doesn't do anything can you pull up so that i mean

Unstructured Information Management Group

IBM India Research Lab

35

Domain Model


We define the Domain Model as a Topic
Taxonomy where every
node

is
characterized by


Topics


Typical Question
-
Answers (QAs)


Typical Actions


Call Statistics


Unstructured Information Management Group

IBM India Research Lab

36

Block Diagram

Stopword

Removal

N
-
gram

Extraction

Feature Engineering

ASR

Clusterer

Taxonomy

Builder

Model

Builder

Component

Clusters of different
granularity

Voice help
-
desk data

1

2

3

4

5

Unstructured Information Management Group

IBM India Research Lab

37

replicate error unable to find path to server

go to the workspace

on left hand side look under server icon

………..



are you using lotus notes six

do you have the lotus notes closed

…………………..

avg. transcription length = 1214.540984 words

avg. call duration = 712.7395 secs

Unstructured Information Management Group

IBM India Research Lab

38

Unstructured Information Management Group

IBM India Research Lab

39

Unstructured Information Management Group

IBM India Research Lab

41

Conclusion


Huge amount of unstructured data is being produced
everyday in contact centers


Analysis can help to improve customer satisfaction, agent
productivity, call handling time


Opportunity to play with real “real
-
life data”


Learning experience


Importance of handling noise in unstructured data


Workshop on Analytics for Noisy Unstructured Text
Data (at IJCAI 07)
[http://research.ihost.com/and2007/]


deadline 25 Sep (day after tomorrow!!)

Unstructured Information Management Group

IBM India Research Lab

42

Thanks!!

Unstructured Information Management Group

IBM India Research Lab

43

BACKUP

Unstructured Information Management Group

IBM India Research Lab

44

Bharti PoC


Demonstrate how text analytics can add value
to the existing Complaint Management
Systems and make it more efficient


Demonstration of the software


Possible ways to extend this work


Discussion



Unstructured Information Management Group

IBM India Research Lab

45

How Does The Data Look Like?

Unstructured Information Management Group

IBM India Research Lab

46

Some Relevant Text Analytics Techniques


Cleaning up of data



Spelling correction


irregular dial, iregular dialtone,irregular dt fone,irrgular dt
psl,irregular dt so plss,irregular dt due
are grouped together



Abbreviation expansion



….


Annotators


Extracted problem areas such as
Intermittent Dial Tone, Rosette
Issue etc.
Hints taken from questions provided by Bharti


Address segmentation such as
Subhash Nagar, Bhopal, M.P.
etc.


Sentiment, Product, Services


Application specific


….

Unstructured Information Management Group

IBM India Research Lab

47

Then?


Relevant structured and extracted annotated
data is loaded into star
-
schema and a Business
Intelligence (BI) application is developed on
top of that


The BI application is capable of showing
different views of the data by doing slice
-
and
-
dice, rollup
-
drilldown, association, comparison
etc. operations

Lets see the demo of BI application developed
on hard
-
faults data collected from M.P.

Unstructured Information Management Group

IBM India Research Lab

48