Aggregating Support Vector Models for

chardfriendlyΤεχνίτη Νοημοσύνη και Ρομποτική

16 Οκτ 2013 (πριν από 3 χρόνια και 9 μήνες)

75 εμφανίσεις

Aggregating Support Vector Models for
Prediction of
Cross
-
Selling Problem


Si Jie Phua

School of Computer Engineering, Nanyang Technological University, Singapore


1 Introduction

This paper aims to demonstrate a methodology to find better solutions for a c
ross
-
selling
business problem [1].

The dataset is obtained from PAKDD competition 2007. I have
used an ensemble classifier method
with
Support Vector Machine (SVM) [2
, 3, 4
]
as base
classifier to obtain

faster training speed and
accurate

results.

This p
aper is organized as follows: Section 2 lists the modifications made on the dataset
before applying them for training and prediction.

Section 3 briefly explains the support
vector machine (SVM) and ensemble classifier.

Section 4
gives the parameter
s

used

to
train the model. Section 5 concludes the results with discussion on the business insight.

2

Data Preparation

M
odifications are done on
following attributes

before training and prediction

in order to
fit the data mining technique used
:



ANNUAL_INCOME_RA
NGE: “0K
-
< 30K”, “30K
-
< 90K”, “90K
-
< 150K”,
“150K
-
< 240K”, “240K
-
< 360K”, “360K+” is changed to 1, 2, 3, 4, 5, 6
correspondingly. This modification intends to represent the ordi
nal property of
this attribute
.



DISP_INCOME_CODE: A
-
E is changed to 1
-
5 a
ccordingly to represent the
ordinal property of the attribute.



All attributes related to bureau except “B_DEF_PAID_IND” and
“B_DEF_UNPD_IND”: 98 is changed to
-
1 and 99 is changed to
-
2 so that the
contact frequencies of customer contact with bureau are be
tter represented.

3

Modeling Technique

The training dataset is large and the class distribution is unbalanced. Thus, we propose to
train a few models

of Support Vector Machine (SVM)

where each model is trained by a
portion of training dataset, and then cl
assify new instances using the trained models with
voting scheme.

SVM is developed

on the concept of decision planes that define decision boundaries.

A
decision plane separates a set of objects

that have

different class memberships.
SVM

constructs

a hype
rplane as the decision plane, which separates the positive and negative
samples with the maximized margin.

Among the training technique of support vector
machine, sequential minimal optimization (SMO)

[5
]

algorithm in YALE [6
] is used as it
can handle bot
h numerical and nominal attributes.

SVM usually has nice generalization
abilities due to its margin maximization strategy. However, the training speed of SVM
usually increases tremendously with the size of dataset.

Thus, we partition the

training

dataset

to train a few models of SVM so that the training
speed is faster.

We have constructed 9 training datasets from the original training dataset
to train 9 models of SVM. Each training dataset contains all the positive instances and
equal amount of negativ
e instances. By this way, we can reduce the effect of unbalanced
class distribution.

For the purpose of prediction, we

combine the outputs of several classifiers
with voting
scheme. This type of classifier
is
popularly
known as ensemble classifier [7].

This
classifier generally has low variance and thus has better generalization abilities than
individual classifiers.

With the strategies of sampling, ensemble classifier and SVM, we aim to provide a good
prediction on cross
-
selling problems.

4

Summary of
Scoring Model Results

The only important parameter is the cost parameter of SVM. We have set it to 1 for
faster training speed.

5 Discussion


By applying the trained model on the
test
dataset to be predicted, we get the

following
results listed in Table 1
:

Score

Number of Instances

Percentage

0

704

8.8%

1

1811

22.7%

2

2733

34.2%

3

910

11.4%

4

543

6.8%

5

375

4.7%

6

305

3.8%

7

379

4.7%

8

168

2.1%

9

72

0.9%

Table 1: Prediction of test dataset

The higher the score, the more likely the customer will
sign up for a new home loan with
company within 12 months of opening the credit card account. As a result, the company
should focus more on the customers that have high scores in order to maximize their
profits.

References:

[1] PAKDD Competition 2007,
ht
tp://levis.shu.edu.cn/pakdd2007/competition/download
.

[2] C. C. Burges, “A tutorial on support vector machines for pattern recognition,” Proc.
Int. Conf. Data Mining and Knowledge Discovery, Vol.2, 1998, pp. 121
-
167.

[
3
] Hyeran Byun, Seong
-
Whan Lee, “A sur
vey on pattern recognition applications of
support vector machines,” International Journal of Pattern Recognition and Artificial
Intelligence, Vol. 17, No. 3, 2003, pp. 459
-
486.

[
4
] Marti A. Hearst, “Support vector machines,” IEEE Intelligent Systems, July
/August
1998,
pp.
18
-
21
.

[
5
]

J. Platt,

Sequential Minimal Optimization: A Fast Algorithm for Training Support
Vector Machines,


Microsoft Research Technical Report MSR
-
TR
-
98
-
14, (1998).
[4]

[
6
]
YALE (
Yet Another Learning Environment
),

http://www
-
ai.cs.uni
-
dortmund.de/SOFTWARE/YALE/intro.html

.

[7] Classifier Ensembles,
http://www.cs.cmu.ed
u/afs/cs/project/jair/pub/volume11/opitz99a
-
html/node2.html

.