otfm_admin_guide.doc

mexicanmorningData Management

Dec 16, 2012 (4 years and 10 months ago)

184 views

(c)
Open Source Strategies, Inc
, 2007 Author: Cameron Smith,
Database, Lda

Av. Vladimir Lenine nº 288


17º Andar Dir
eito


Maputo


Moçambique
-

Tel/Fax.: + 258 21 310166



1


Open Source Strategies: OpenTAPS Funambol Module

Administration Guide


0.1 02/07/07

Cameron Smith

Complete Draft, ready for use with prototype version

0.2 03/7/07

Cameron Smith

Correct syncsource name, add note about case
-
sensitivity

0.3 06/07/07

Camer
on Smith

Remove obsoleted steps, explain more about classpath

0.4 11/07/07

Cameron Smith

Include Transaction Management Configuration

0.5 30/07/07

Cameron Smith

Include Merge configuration

0.6 31/07/07

Si Chen

updated with directions for enabling funamb
ol now that the component is by
default commented out

0.7 01/08/07

Cameron Smith

Correct Merge configuration

0.8 08/08/07

Cameron Smith

Include Twin regex configuration; Note that funambol.ds.home by default
should work OOTB

0.9 02/09/07

Cameron Smith

R
emove Transaction Management Configuration; Add SyncHandler suffix
configuration

0.9.1 05/09/07

Cameron Smith

Include new “config” build task

0.9.2 24/10/07

Eurico da Silva

Include Task setup instructions

1.0 27/10/07

Cameron Smith

Troubleshooting t
ips for addresses and phone numbers

1.1 03/11/07

Cameron Smith

Include a note about time settings

1.2 10/11/07

Cameron Smith

Include a note about geo settings

1.3 15/11/07

Cameron Smith

Document improved geo detection algorithm, as requested by

Si Chen's
email of 13/11/07

1.4 24/11/07

Cameron Smith

Include Calendar setup instructions

1.5 07/12/07

Cameron Smith

Notes about Calendar date/time handling

1.6 19/10/09

Spark Sun

Update the document after upgrade funambol to 8.0.0

1.7
27/10/09

Si Chen

Change configuration parameters for default country and state


This document is for systems administrators at OSSI, or their clients, who wish to install and maintain
the OpenTAPS Funambol Module (OTFM) inside a new or existing OpenTAPS s
erver installation.


Writing Conventions

Please note that, throughout this document, unless otherwise stated, all text strings, commands and
aother configuration values are
case
-
sensitive
and should be used exactly as written in the text.


Prerequisites

................................
................................
................................
................................
.....
3

Web Setup

................................
................................
................................
................................
........
3

DB Setup

................................
................................
................................
................................
...........
4

Starting and Stopping

................................
................................
................................
...........................
5

Setting up a new user for sync

................................
................................
................................
..............
5

Prerequisites

................................
................................
................................
................................
.....
5

Server Account Setup
................................
................................
................................
........................
5

FOPn Setup

................................
................................
................................
................................
......
6

Monitoring

................................
................................
................................
................................
.............
6

Sync Behaviour Configuration

................................
................................
................................
...............
6

Twin Detection
................................
................................
................................
................................
...
6

Merge Strategy/Confli
ct Resolution

................................
................................
................................
...
7

State & Country Recognition

................................
................................
................................
.............
7

Calendar Events


all day

................................
................................
................................
..................
8

(c)
Open Source Strategies, Inc
, 2007 Author: Cameron Smith,
Database, Lda

Av. Vladimir Lenine nº 288


17º Andar Dir
eito


Maputo


Moçambique
-

Tel/Fax.: + 258 21 310166



2

Troubleshooting

................................
................................
................................
................................
....
8

Address Handling

................................
................................
................................
..............................
8

Configuration

................................
................................
................................
................................
.....
8

Phone number handling

................................
................................
................................
....................
9

SyncSource
-
> SyncHandler mapping

................................
................................
...............................
9

Transaction Management

................................
................................
................................
..................
9

Not all of my items are synched!

................................
................................
................................
........
9

Reference
s

................................
................................
................................
................................
...........
9
(c)
Open Source Strategies, Inc
, 2007 Author: Cameron Smith,
Database, Lda

Av. Vladimir Lenine nº 288


17º Andar Dir
eito


Maputo


Moçambique
-

Tel/Fax.: + 258 21 310166



3

Initial Setup
This section describes how to setup the OpenTAPS
-
Funambol Sync Module (henceforth OTFM)
inside an existing OpenTAPS installation. The steps are specifically designed for the Fedora Core 6
+ PostgreSQL 8.2 + OpenTAPS 1.0 platform, but i
t is likely that the vast majority of them will work on
any platform.

Prerequisites

Before proceeding, make sure that you have met the following conditions:

1.

JDK 1.5 or above installed in a directory we will refer to as JAVA_HOME. Note that it must be
Sun
JDK and not a JRE or other version of Java.

2.

OpenTAPS 1.0 installed, with seed/demo/other data loaded, and working normally. We will
call the installation directory OT_HOME throughout this document, and call the OpenTAPS
server process “OT”.

3.

A PostgreSQL s
erver process available on your LAN, for which you have the
username/password for the administrator account. For the purposes of this document we will
assume that this is postgres/postgres. Note that a typical scenario is to have the Funambol
schema (whi
ch we will create below), installed in the same PostgreSQL cluster as the
OpenTAPs schema but this is
not a requirement

for OSFM. In fact, OpenTAPs could even be
using a different database product!

4.

The PostgreSQL server process must be configured to liste
n on the appropriate network
interface via which the machine on which OT is installed, will connect.

5.

The PostgreSQL command
-
line client
psql

installed somewhere. We will assume it is on the
same machine as OT_HOME.

6.

An operating system account with full (r
wx) access to OT_HOME. The setup does
not
require
you to use the
root

account therefore we recommend you do not use this account! We will
refer to this account as
user
.

Web Setup

First of all we will configure the web application part of OSFM. Login as
user

and carry out the steps
below.

1.

Stop OT.

2.

Edit OT_HOME/hot
-
deploy/component
-
load.xml and un
-
comment the line for


<load
-
component component
-
location="${ofbiz.home}/hot
-
deploy/funambol"/>



This will enable the Funambol DS to be loaded into opentaps.

3.

[
This step only necessary if you are have a non
-
standard directory structure for your OT
installation.
] Open
OTFM_HOME/webapp/WEB
-
INF/opentaps
-
sync
-
config.xml
in a
text editor and look for the section 1.1. Beneath this, find the
<entry
key=”funambol.ds.ho
me”…/>

tag and set its value attribute to the absolute or relative path
of OTFM_HOME. For instance something like: “
/opt/opentaps/hot
-
deploy/funambol
”.
Save your changes! Note that for most OOTB OT installations, the default value (relative
path:
hot
-
deploy/funambol
) should be sufficient.

4.

If your PostgreSQL cluster is on a different machine than the current one, then, still in
opentaps
-
sync
-
config.xm
l
, alter the value of
<property name=”url” />

in section
1.3, accordingly. You will also need to alter

the value of the
jdbc.url

property in
OTFM_HOME/install/install.properties

5.

Configure an HTTP (non
-
secure) connector on port 8080 in
OT_HOME/framework/base/config/ofbiz
-
container.xml
. It could be on any port
but for this manual we will assume it is 8080
.

(c)
Open Source Strategies, Inc
, 2007 Author: Cameron Smith,
Database, Lda

Av. Vladimir Lenine nº 288


17º Andar Dir
eito


Maputo


Moçambique
-

Tel/Fax.: + 258 21 310166



4

6.

From OT_HOME execute the following command, to prepare the default config tree. At the
prompt type 'y':

./ant
-
f hot
-
deploy/funambol/build.xml config


DB Setup

Now we must initialize the RDBMS schema which Funambol uses to manage its synchronization
ope
rations. We will also populate it with the relevant configuration to plug the contact SyncSource
into OTFM.


Execute the following commands at the command
-
line:

1.
psql
-
U
superuser

#<password>

2.
create user funambol password 'funambol';

3.
create databa
se funambol owner funambol;

4.
\
q

5.
psql funambol funambol

#<password>

6. List all tables
-

you should see "No relations found":

\
dt


Leave your psql client connected and open a separate terminal in the OT_HOME directory…


7. Now we will create the schema
. If the script asks you any questions, just type ‘y’. After each step,
execute the given verification command in your psql console.

Command to execute

Verification in psql...

…correct result

./ant
-
f hot
-
deploy/funambol/install/install.xml install

\
dt

List of tables appears,
ending with “26 relations
found”

./ant
-
f hot
-
deploy/funambol/install/install.xml
install
-
sync
-
module

SELECT * FROM
fnbl_module;

One row should be
listed.

./ant
-
f hot
-
deploy/funambol/install/install.xml
install
-
sync
-
source

SELECT

* FROM
fnbl_sync_source;

Three rows should be
listed.


7.

Start OT. Check OT/runtime/logs/ofbiz.log for errors. If there are no errors, check
OTFM_HOME/logs/funambol_ds.log. It should have something like the following at the end.
The most important lin
es are the ones marked in bold.


[2007
-
06
-
29 18:35:45,944] [funambol.transport.http] [INFO] [] [] [] [] ===============================================================
=================


[2007
-
06
-
29 18:35:45,944] [funambol.transport.http] [INFO] [] [] [] []

Funambol Data Synchronization Server v. 6.0.6 engine started.

Configuration object found.


[2007
-
06
-
29 18:35:45,944] [funambol.transport.http] [INFO] [] [] [] []

. . .


[2007
-
06
-
29 18:35:45,944] [funambol.transport.http] [INFO] [] [] [] [] ==============
==================================================================

[2007
-
06
-
29 18:35:46,384] [funambol.auth] [INFO] [] [] [] [] CREATED OULO with this ps: null

[2007
-
06
-
29 18:35:46,735] [funambol.transport.http] [INFO] [] [] [] [] Engine configuration:

[20
07
-
06
-
29 18:35:46,735] [funambol.transport.http] [INFO] [] [] [] []
-

store: com/funambol/server/store/PersistentStoreManager.xml (com.funambol.server.store.PersistentStoreManager
-

{stores:
com.funambol.server.store.ClientMappingPersistentStore@196eed5,c
om.funambol.server.store.IDPersistentStore@1a1b2f,com.funambol.server.store.LastTimestampPersistentStore@1eeba19,com.funambol
.server.store.ModulesPersistentStore@1fbe226,com.funambol.server.store.PrincipalPersistentStore@1bc6533,com.funambol.server.s
tore.S
yncSour
cePersistentStore@19811ce)

[2007
-
06
-
29 18:35:46,735] [funambol.transport.http] [INFO] [] [] [] []
-

officer: com/funambol/server/security/OFBizUserLoginOfficer.xml

(org.opentaps.funambol.security.OFBizUserLoginOfficer@b8f675)

[2007
-
06
-
29 18:35:46,7
35] [funambol.transport.http] [INFO] [] [] [] []
-

strategy: com/funambol/server/engine/Strategy.xml (com.funambol.server.engine.Sync4jStrategy@13c9557)

[2007
-
06
-
29 18:35:46,735] [funambol.transport.http] [INFO] [] [] [] [] Default encoding: Cp1252

[2007
-
06
-
29 18:35:46,735] [funambol.transport.http] [INFO] [] [] [] [] Server URI not set.

(c)
Open Source Strategies, Inc
, 2007 Author: Cameron Smith,
Database, Lda

Av. Vladimir Lenine nº 288


17º Andar Dir
eito


Maputo


Moçambique
-

Tel/Fax.: + 258 21 310166



5


9. If the logs look OK from the previous step, open a browser and navigate to the following URL
(if your browser is on a different machine, replace localhost with the h
ost in question):

https://localhost:8443/funambol


You should see a page which has the Funambol logo and “
Funambol DS Server: 6.0.6
”. Do NOT
click on the Web Demo Client link as this has been deactivated in
OTFM.


To verify the installation, please follow the steps in the section “Setting up a new user for sync” at
least once.

Starting and Stopping

The Module is started and stopped automatically by OT, via the normal module load/unload process.
No special ac
tion is necessary.

Setting up a new user for sync

The steps in this section should be followed for each new OT user who wishes to sync with OT/crmsfa
via the OTFM.

Prerequisites

Before proceeding, make sure that you have met the following conditions:

1.

The u
ser has, on their chosen PC, installed the following software:



Outlook XP or above.



Funambol Outlook Plugin (FOPn) installed, as per [Funambol Outlook Plugin Guide]

2.

The user has an IP address or hostname (which we will refer to as OT_HOST) via which
their
PC can establish a TCP connection with the OT server on port 8080. Make sure any
firewalls on the PC, the server, or any intermediate routers are appropriately configured.

3.

The user has an existing “normal” UserLogin for CRMSFA, with which he can at least
see
the “My Contacts” and “Create Contact” screens. We will refer to this UserLogin as
weblogin
.

4.

The user's PC and OT server are set to the same time and timezone. This can be done
manually but the best way to ensure it is for both computers to get their

time settings from
the same NTP server.

Server Account Setup

1.

Login to OT/partymgr with an OT user who has PARTYMGR_CREATE permissions.

2.

Find the profile page for
weblogin

and choose the option to create a new UserLogin. This
new UserLogin, who we will ref
er to as
synclogin

should have:



A different username/password (for instance
firstname.surname.sync

)



The same Role(s) as
weblogin



The same SecurityGroup memberships as
weblogin

3.

Login to OT/webtools and via the Entity Data Maintenance screen for UserLogin,
set
enabled = N, system = Y for
synclogin
.

FOPn Setup

1.

Open Microsoft Outlook and select Funambol


Options from the menu.

2.

The Funambol mini
-
panel should open, here select Tools


Options

3.

Click the “Account” tab and enter the following details:

Field

Value

Example

Location

http://
OT_HOST
:8080/funambol/ds

http://extranet.myfirm.com:8080/funambol/ds

Us
ername

synclogin.userLoginId

john.brown.sync

Password

synclogin.password

John123

(c)
Open Source Strategies, Inc
, 2007 Author: Cameron Smith,
Database,
Av. Vladimir Lenine nº 288


17º Andar Dir
eito


Maputo


Moçambique
-

Tel/Fax.: + 258 21 310166



6


4.

Now click the Sync tab, then deactivate every checkbox EXCEPT Contacts and Task

5.

Now click the “Details” button for Contacts, Tasks and Calendar in turns and fill in the
det
ails as follows (most of them should already be OK). Then click “OK”

Field

Contacts

Calendar

Tasks

Sync Direction

Two
-
way

Two
-
way

Two
-
way

Contacts Folder

\
\
Personal

Folders
\
Contacts

\
\
Personal

Fold
ers
\
Calendar

\
\
Personal

Folders
\
Tasks

Include Sub
Folders

Yes (checked)

Yes (checked)

Yes (checked)

Remote name

otcontact

otcalendar

ottask

Data format

vCard

SIF

SIF


6.

To check that your sync is working properly, click the “S
ync All” section on the Funambol
mini
-
panel. It should start the sync and after a moment say “The server has requested a
full sync”. Say “Yes” and after about 30s everything should be synced. Look at your
Outlook and OT pages to see the results!

Monito
ring

While the application is running, there are three sources of information about what is going on:



Main opentaps log: OT_HOME/runtime/logs/ofbiz.log


will have very basic startup information
about OT trying to start the OTFM. If it contains no errors

on startup, there is probably not
much else it. Any others that come out here probably mean errors in the code delivered by
DBL.



DSS log: OTFM_HOME/logs/funambol_ds.log


the most interesting. If, for example,
someone has an authentication failure or p
ermissions problem, it will probably come out here.

Sync Behaviour Configuration

You can control the following aspects of OTFM behaviour via making the appropriate alteration in the

opentaps
-
sync
-
config.xml
configuration file and restarting OT...

Twin Dete
ction

“Twin Detection” is a step which the DSS performs during a sync for every NEW record it detects
coming from Outlook. Before it definitively tells OTFM to
add
this record, it will give OTFM a chance
to check whether or not this record is a “twin”
-

t
hat is, actually corresponds to an existing record
already stored in OT. If OTFM responds “yes” at this point, then the DSS will subsequently issue a

merge

command to OTFM instead of
add
.

You can read more about twin detection in [Funambol
DSS Arch.], se
c. 2.5.1 “Twins Handling and Merging”. OTFM only uses twin detection for Contacts
as it does not make sense for other record types.


The general algorithm which OTFM performs when asked to detect a twin is the following:

1.

Find all the known records which c
ould possibly correspond to the new record, given its
content (i.e., if it has no person name, then it could only be an Account)

2.

For each record R in this set of records:

2.1 get a field which in human semantics should be distinguishing (ex. person name)
f
rom both the existing and new records

2.2 Strip all leading and trailing whitespace from both fields

2.3 Uppercase both fields

2.4 Remove any characters in a given regex string from both fields


2.5 If the fields now match, then inform DSS that R is a
twin


(c)
Open Source Strategies, Inc
, 2007 Author: Cameron Smith,
Database,
Av. Vladimir Lenine nº 288


17º Andar Dir
eito


Maputo


Moçambique
-

Tel/Fax.: + 258 21 310166



7

You can configure OTFM to use a different regex string for step 2.4 (the step in bold). To do this,
alter the node
<bean id="otcontactHandler
\
<property name="normalizeRemoveRegex":
any character which
matches the regex you put here will be removed i
n step 2.4, before comparison. The default regex
removes all non
-
alphanumeric characters.


Warning:

this file is XML, not Java source


therefore special characters should NOT be double
-
escaped, for instance, to indicate a literal dot:


\
.
-

CORRECT


\
\
.
-

INCORRECT

Merge Strategy/Conflict Resolution

To set the strategy used when the DSS detects that a certain record has been updated by both
Outlook and OT since the last sync, you must configure the node:

<bean
id="otcontactHandler
\
<property name="merg
eStrategy":


Strategy

What happens

Value

Server wins

OT record completely substitutes Outlook record

server_wins

Client wins

Outlook record completely substitutes OT record

client_wins

Merge

Field
-
level merging occurs: Server subrecord substitutes Outlo
ok's
IF AND ONLY IF they were updated in OT since last sync.
Otherwise Outlook subrecord substitutes OT.

merge


Attention:
Merge is an invalid strategy for Tasks and Events. All fields of Tasks and Event are in only
one Entity, it’s not possible to detec
t, for example, if “Subject” only has changed. So Merge is only
valid for objects represented by more than one Entity, for example, Contacts.

State & Country Recognition

OT stores state and country information for a postal address in a normalized fashion,
whereas Outlook
stores this information as simple strings (in the case of states), or enumerated list of country names (in
the case of countries). This creates the possibility that spelling errors or different spelling
conventions on the part of the Outl
ook user, could cause an address not to be recognized on the OT
side. OTFM deals with synching of these two fields in the following fashion:

1.

It first tries a reverse lookup from the country/state name to the corresponding geoId

2.

If this lookup returns no
country geoId, it will use the default country id defined in
opentaps.properties#defaultCountryGeoId
.
The administrator may configure this to any valid
value from

Geo.geoId where geoTypeId=COUNTRY.

3.

If this lookup was for a state and nothing was returned,

then OTFM will also try the geoCode,
abbreviation and geoId (note that in OT these three fields tend to all have the same value). If
OTFM still finds nothing, it simply uses a blank string “” for the state, EXCEPT where
geoId=USA. In this case, it will

use the default US State id defined in

opentaps.properties#
state.geo.id.default.us.
The administrator may configure this to any valid
value from

Geo.geoId where geoTypeId=STATE.


For example, if you want to indicate the golden state in Outlook, you coul
d put “California” OR “CA” in the
state

field. Note that after synching, your Outlook field will always change to the full name of the state which was
defined in OT


in this example “California”.


Note also that OTFM does NOT try to be intelligent and ma
ke sure that the state/province in question is in fact a
regional subunit of the given country!

(c)
Open Source Strategies, Inc
, 2007 Author: Cameron Smith,
Database,
Av. Vladimir Lenine nº 288


17º Andar Dir
eito


Maputo


Moçambique
-

Tel/Fax.: + 258 21 310166



8

Calendar Events


all day

Outlook has the concept of an “all
-
day event”, which you can set via a checkbox. Such an event has
no start or end time. However OT
does not have this concept. Therefore, when OTFM detects an
all
-
day event being synched from Outlook, it will set the corresponding Event (WorkEffort) in OT to
have the start and end time defined in crmsfa.properties:
crmsfa.calendar.startHour
and
crmsfa.
calendar.endHour
respectively. These values should be set in 24
-
hr clock format, without
leading zeros. Be aware that these properties are also used by CRMSFA, you should consult
CRMSFA documentation to understand any side
-
effects of changing them.

Troub
leshooting

This section describes common problems and known workarounds/configuration issues.

Address Handling

If you create an address in OT, and then synch, the address should be passed to Outlook. However
it may not be immediately visible there, becaus
e Outlook only shows one address at any one time.
After synching, make sure that beside the address field, you choose "Business/Home/Other" from the
dropdown, and the address should appear in the main field.

Configuration

If you see this message:

========
========================================================================

2007
-
07
-
13 19:15:06,598 (main) [ Sync4jServlet.java:564:ERROR] Error bootstrapping Sync4jServlet

java.lang.IllegalArgumentException: The given file is not a directory


at
com.funambol.server.tools.directorymonitor.DirectoryMonitor.<init>(DirectoryMonitor.java:82)


at com.funambol.server.tools.directorymonitor.DirectoryMonitor.<init>(DirectoryMonitor.java:102)


at com.funambol.server.config.Configuration.<init>
(Configuration.java:185)


at com.funambol.server.config.Configuration.getConfiguration(Configuration.java:204)


at com.funambol.transport.http.server.Sync4jServlet.bootstrap(Sync4jServlet.java:484)


at com.funambol.transport.http.serve
r.Sync4jServlet.init(Sync4jServlet.java:181)


at javax.servlet.GenericServlet.init(GenericServlet.java:168)


You have misconfigured the location of the funambol component in opentaps. Go back and edit hot
-
deploy/funambol/WEB
-
INF/opentaps
-
sync
-
confi
g.xml and change the “funambol.ds.home” to the right
location.

Phone number handling

The Funambol Outlook Plugin is slightly deficient in its handling of composite phone numbers (in part
due to the limitations of the VCARD format itself). So please follo
w these rules when entering phone
numbers:



Do NOT have any spaces between the parts of the number: for instance enter 824062 and not
82 40 62



Whether in Outlook or OT, make sure you put the right parts of the number in the right fields
(area code, phone nu
mber)



Extensions entered in OT will not be detected by Outlook.

SyncSource
-
> SyncHandler mapping

The SyncSource URI (ex.
otcontact
) is used to map to a specific “SyncHandler” (a Java object which
knows how to convert a specific PIM record type to an OT En
tity). This mapping is performed by
appending the value of the
<bean id="syncSourceConfig"/<property name="handlerSuffix"

property to the URI. The resulting string should correspond to the id of a bean defined in opentaps
-
sync
-
config.xml. By default all

this is set correctly, but if you for some reason alter the URI of your
Sync Source, you must also alter the ids of the “xxxHandler” beans accordingly.

(c)
Open Source Strategies, Inc
, 2007 Author: Cameron Smith,
Database,
Av. Vladimir Lenine nº 288


17º Andar Dir
eito


Maputo


Moçambique
-

Tel/Fax.: + 258 21 310166



9

Transaction Management

Some potential problems with Transaction Management have been reported, in that
OTFM may be
interfering with OT Transactions, when OT is run against Derby (No problems have been reported
when both OTFM and OT are run against Postgresql).


Within OTFM, there are two levels at which transaction management is performed:



The DSS. No tran
saction management is performed or possible here, because the DSS is
hardcoded to assume autocommit=true and cannot be configured to perform any transaction
management



The OTFM EntitySyncSource. By default, this treats each update to a top
-
level record,
mandated by the DSS (for instance an Add, Update or Delete of a Contact), as a single
transaction, including all Entities, sub
-
entities and service calls

Not all of my items are synched!

In rare cases, if a user performs several updates both in OT and Out
look within a short space of time,
some of the updates may not appear. This is most likely because the Outlook PC and the OT server
have different time settings (see “Setting up a new user for synch” above).


In any case, if your Outlook gets out of sync
h in this fashion, you can always “clean it” back to synch
with your OT account via using the Funambol
-
> Options
-
> Tools
-
> Recover
-
> “Replace all Outlook
data with data from server” option.

References

[Funambol DSS Arch.] Funambol DS Server Architectur
e & Design Document, rev. 1.28, Stefano
Fornari & Stefano Nichele, Funambol Team.


[Funambol Outlook Plugin Guide]
http://download.forge.objectweb.org/sync4j/Outlook_Plugin_UG.
pdf