Predixion Greenplum Stored Procedures

piloturuguayanΤεχνίτη Νοημοσύνη και Ρομποτική

15 Οκτ 2013 (πριν από 3 χρόνια και 9 μήνες)

76 εμφανίσεις


Predixion

Software

|

31910

Del

Obispo

#120,

San

Juan

Capistrano,

CA

926
75
|

+1.949.373.4900

|

www.PredixionSoftware.com


Predixion Greenplum Stored Procedures

Predixion Insight In
-
Database Scoring Overview

Starting with version Predixion Insight 3.0, the Machine Learning Semantic Model ™ (MLSM) allows
defining and executing complex predictive workflows on top of large amounts of data. The Deployment
Guide below is a step by step procedure intended for datab
ase administrators that want to enable
Predixion in
-
database scoring inside Greenplum. These workflows can be executed by Predixion Insight
server fetching the data, operating on that data, and then pushing the results to a destination. If the data
source

and destination are the same (for example, tables in the same Greenplum Server database or
HDFS files on the same Hadoop cluster), then it makes sense to push the workflow execution to the place
where data lives. The actual execution engine needs only be
deployed once, while actual workflows are
relatively small and are usually represented in the order of tens or hundreds of kilobytes.

Predixion Greenplum Stored Procedure is java
-
based. The stored procedure allows Predixion Insight to
score data inside th
ose databases without having to move the data out to the Predixion server. The
scored data is written back as a new table into the same database. This strategy minimizes data transfer
while scoring and allows large quantities of data to be scored very fast
. Predixion Greenplum Stored
Procedures Deployment Guide instructs you on how to deploy the Predixion Insight code component
required for executing an MLSM inside your database. The Predixion Insight execution component
deploys as a PL/Java stored procedu
re. Consequently, the execution of PL/Java needs to be enabled at
the server level. The stored procedure needs to be deployed in each Greenplum Server database which
contains data. The source data will be processed through the MLSM independent of the sourc
e data
origin. The in
-
database execution will only occur when the source data and the destination for the
processed data are inside the same Greenplum database. Alternately, data will be streamed for
processing through the Predixion server.

Deployment of P
redixion Greenplum Stored Procedures

Prerequisites:


The following are prerequisites for installing the Predixion CLR stored procedures:



Greenplum Database installed on every host machine.



User with superuser rights to the database to be able to deploy PL/
Java and stored procedures.



User with root rights to the machine to be able to install PL/Java.



Java JDK 1.6 on every Greenplum host machine.



JDK 1.6 is in the library path. You can ensure that JDK 1.6 is in the library path by entering the
following into

a command window:

# echo "$JAVA_HOME/jre/lib/amd64/server" > /etc/ld.so.conf.d/libjdk.conf

# ldconfig



A list of host names in a text file, with one host name per line, an example host file, called
HostFile, is included in the accompanying
Predixion.Greenplum.tar.gz archive.

How do I deploy the Predixion Stored Procedure
?

1.

Copy Predixion.Greenplum.tar.gz from the Predixion server location to the machine where you’ll
run the install and expand it to a folder. The package contains the following

files:

s
etup.sh

u
ninstall.sh

Shell script for installing/uninstalling.

bin/
g
uava
-
12.0.jar

Predixion binaries which are deployed as

Predixion

Software

|

31910

Del

Obispo

#120,

San

Juan

Capistrano,

CA

926
75
|

+1.949.373.4900

|

www.PredixionSoftware.com

bin/
PXGreenplum.jar

bin/
s
imple
-
xml
-
2.6.2.jar

bin/
postgresql
-
8.3
-
605.jdbc2ee.jar

managed assemblies to the target
Greenplum
server database

HostFile

Example host file of list of host name

i
nstall.sql

Installation SQL script

u
ninstall.sql

Uninstall SQL script


2.

Open a command window.

3.

Modify

HostFile.txt

to include all the Greenplum hosts, one host name per line.

4.

Run the
following command for each database where you want to install the Predixion Package:

./setup.sh <databaseName>

5.

Restart Greenplum to load the options.


The above steps will enable Predixion stored procedures and hence In
-
Database scoring from Predixion
Insight for that database. Please note that the input data must be in the same database for In
-
database
scoring to work.


Note: If you already have existing PL/Java libraries that you are using, please add the following line to
setup.sh to include your .ja
r files as well:


gpconfig
-
c pljava_classpath
-
v
\
'guava
-
12.0.jar:simple
-
xml
-
2.6.2.jar:PXGreenplum.jar:postgresql
-
8.3
-
605.jdbc2ee.jar
\
'

What does the Install script do
?

The following section describes the different sections of the install script, called
setup.sh. Understanding
these commands will allow an advanced user to modify the script manually, if required:


1.

Setting Greenplum Configuration Parameters:


#Setting Greenplum Configs

gpconfig
-
c pljava_classpath

-
v
\
'guava
-
12.0.jar:simple
-
xml
-
2.6.2.jar:PXGreenplum.jar:postgresql
-
8.3
-
605.jdbc2ee.jar
\
'

gpconfig
-
c pljava_vmoptions
-
v
\
'
-
Xmx128m
\
'


2.

Copying the required Jars to the Greenplum hosts as specified by HostFile:


#Copying Required Jars (Requires hostlist

file name)

gpscp
-
f HostFile /bin/guava
-
12.0.jar /bin/simple
-
xml
-
2.6.2.jar /bin/PXGreenplum.jar /bin/postgresql
-
8.3
-
605.jdbc2ee.jar =:$GPHOME/lib/postgresql/java


3.

Install PL/Java in Greenplum


#Setup PL/Java in Greenplum (Require DB Name)

psql
-
f install
.sql $dbname

How do I uninstall the Predixion Stored Procedure
?

To uninstall the Predixion stored procedure, run the following command from a command window:

./
uninstall
.sh <databaseName>

Note: If you have existing PL/Java scripts, do not use this
uninstall script, since it does destructive
uninstall to PL/Java.

Additional Resources for Installing PL/Java



http://blog.2ndquadrant.com/using_pljava_with_greenplum/



©201
3

Predixion

Software.

All

Rights

Reserved.

All

trademarks

and

registered

trademarks

are

the

property

of

their

respective

owners.