Building a Reliable Remote

enginestagNetworking and Communications

Oct 26, 2013 (3 years and 5 months ago)

68 views

© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

Building a Reliable Remote
Communication Device with
Java
ME

JavaOne

San Francisco 2013
-

CON8004


Leonardo Lima


V2COM (
llima@v2com.mobi
)

Tiago Padovani


V2COM (tpadovani@v2com.mobi)

© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

Introduction

© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

About

the

presenter

(Leonardo)


Software Development Manager at V2COM, leading the
development of both embedded software for edge/field devices
and server
-
side software that receives field data and enables
fast communication with remote devices.


Eight years of development
and architectural experience on
mobile, embedded and server platforms to enable management
and operation of highly scalable and fast telemetry and network
control systems.

© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

About

the

presenter

(Tiago)


Lead Software Developer for Embedded at
V2COM, leading the
development of
embedded
software for edge/field
devices.


Six years of development
and architectural experience on
embedded Java platforms, building reliable communication
devices that stays on the field for years.

© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

About

the

presentation


Explore
how to build reliable remote communication
devices that must stay online, connected, and transmitting field
data without interruptions. Using Java ME, a solid foundation was
built that can be used to deploy many kinds of applications in the
field, with fast, reliable development. In this session, see what the
difficulties, ideas, and insights were that led to this foundation,
which is currently in use in more than 50,000 devices in Brazil
and Latin America to connect more than 800,000 field devices.

© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved


V2COM is a leading Latin American provider of
Smart Grid technologies and Advanced Metering
Infrastructure (AMI). Our offer includes hardware,
software and services that can reduce losses and
increase water and energy efficiency, currently
connecting more than 700.000 devices.


Besides the economic gains for our customers, we
develop technologies with significant impact on the
environment (avoiding losses of important natural
resources) and with social responsibility (our Smart
Grid systems can be used to offer free internet
access in challenging areas)

© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved


Awarded 2013 Java Business
Innovation Award


JCP member, nominated for EC election


More interaction with community to foster
Java ME development in this age of
IoT
!

© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

V2COM
clients

Meter Reading

Network Control

© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

The
scenario

© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

IP Network

Local Network

Legacy Systems

Intelligenceware

Suite

WAN

Terminal

LAN

Devices

Wireless Network

© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

Use cases


Automated

Meter Reading (AMR)


Read


dumb

meters

and

send

data over
to

Utility


Both
energy

and

water

meters


Can

be

any

meter,
really
!


Like

our

Zigbier

refrigerator’s

temperature

sensor,
with

alarms

for too
cold

or

hot!


Network
control


Remotely

connect

SCADA systems
to

field

devices

like

reclosures



transparent

pipe

between

SCADA
and

Device


DNP3/IEC104
protocols

and

proprietary

as
well


© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

Use cases

1

3

6

4

2

5b

5a

© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

The
techonology

used

Gateway:


Oracle
Java Micro
Edition

1 (Java profile IMP
-
NG
version

2 &
CLDC 1.1 Hl),
embedded

in a
Gemalto

TC65i module
.


Started

with

Siemens TC45, 6
years

ago
!


Intelligenceware

Suite
:


Oracle Java Enterprise
Edition

5


Oracle Java Standard
Edition

6


Oracle
Utilities

Meter Data Management 2.0

© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

Living
an

embedded

life

© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

The 3
commandments



If

the

device

has

a
power

from

source
, it must
be

on



Our

first

commandment

is

to

keep

the

device

always

on
.


If

it
can

be

on
, it must
be

on
, no
matter

what
.


“The firmware must
be

remotely

upgradable



One

of

few

certainties

in software
developments

is

that

bugs
always

happen
!


So
,
we

must
be

able

to

OTAP (over
-
the
-
air

provision
)
the

device



If

it’s

on
, it must
be

online
and

connected

(
or

trying

to

be
...)”


It
doesn’t

suffice

to

be

on
. It must
be

also

be

avaiable

so

that

communication
can

happen


© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

Hardware
choices


WatchDog


One

of

the

few

always
-
must
-
have

circuit

for
any

device

is

a
watchdog

to

turn

it
on

in case it
turns

off
or

there’s

a
lock
-
up


Dual Sim
Card


GPRS
infrastructure

is

not

very

reliable
,
so

it
should

be

able

to

have

option

in case
of

carrier

failure


Dual Serial


Many

field

devices

have

two

serial
ports
,
so

being

able

to

connect

to

both

at

the

same

time
is

very

important


Java ME


Being

able

to

run

Java ME
allows

many

advantages

over C

© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

Software
choices

-

S
eparation

of concerns



Hardcore



Abstracts
board

specifics

details

(
like

GPIO)



Zion



Handles

common
infrastructure

(
backend

connection,
message

queues
,
locks

and

so

on
)


Abstracts Modem
commands
,
so

there’s

one

for
each

avaiable

module
(
Cinterion
/
Gemalto
; Motorola;
Telit
)



App



Holds

application

logic

-

As
tailored

as
clients

need

it
to

be


Ranges
from

a
simple

Transparent

(TN)
to

Automated

Meter Reading
(AMR)


© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

Hardcore


Handles

board

specifics


Modem/AT Connection


SMS/GPRS
logic


GPIO


Watchdog




© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

Zion


Defines
command

shell


Handle

TCP communications


Kitchen

sink

utils

(
math
,
string
,
locks
,
queue
)


Event

Bus


Extendable

to

handle


fatter

functionalities


CSD


SNMP
monitoring

© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

Application


Handles

only

business
logic

and

protocol


Fast

changes

to

acommodate

business
logic


After

the

first

few

cicles
, it
stabilizes

© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

Package

Diagram


mobi.v2com.zion...


Application


Control


Hardcore


Communication


Event


Protocol


Tools


Utilities


...
and

application

specifics

as
well

© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

Challenges

Problems

faced
,
solutions

provided
!

© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

How

to

keep

the

device

on
?


Problem
:


Module
can

throw

Exceptions

(EXIT
CODEs
)
that

shuts

down

the

application


Solution
:


Hardware
Watchdog
!


Dedicated

QG8
to

handle

only

this

(
power

on
/off)


10 minutes timeout:
either

the

firmware “swings”
the

GPIO,
or

it
gets

powered

off
then

on


Many

other

software
watchdogs
...


Many

other

conditions

can

mean

a “
lock

up




© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

How

to

keep

the

device

online?


Problem
:


GPRS connections
can

be

unstable
,
specially

when

needed

most
:
thunderstorms

and

harsh

weather

condition


Solution
:


Dual
-
SIM
cards
,
so

if

a network
fails
,
there

can

be

another


Many

soft
watchdogs

to

retry

attachments


Turn

on
/off modem,
like

our

phones


Attach
/
Deattach



© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

How

to

get

behind

public

APN firewall
?


Problem
:


Can’t

use a server socket
while

communicating

on

public

networks!


Solutions
:


Device

polls

server,
asking

for new
commands


This

doesn’t

work

for
always
-
on

connection
needs


Expensive

on

resources


Software Switch (server
side
)!


Meet
-
in
-
the
-
middle

approach


© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

How

to

keep

the

TCP connection
up
?


Problem
:


You

don’t

know

the

connection
is

broken

until

you

have

a
blocking

read
()
or

write
()


TCP
keep
-
alive

is

not

enough


No module
support

for
this
...


Solution
:


Keep
-
Alive

packets


Application
-
level

bytes
sent

to

make

sure

connection
is

not

off

© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

How

to

keep

the

firmware
updated
?


Problem
:


There

are MANY
devices

deployed
,
each

with

a software
version


No
one

wants

to

mass
-
deploy
...


Not

all

updates

are
required

for
all

devices


Solution
:


Axon

has

control

of

each

version


Updates

are
dispatched

as
needed

and
/
or

cleared

by

client

© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

How

to

debug
in
-
device
?


Problem
:


No
simple

logging

framework
to

write

log
to


Log
redirection

works

for serial
port

only
,
not

remote

connections


Solution
:


Built

a
simple

logging

framework
that

handles

debug
level
,
message
,
threads


© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

How

to

debug off
-
device
?


Problem
:


How

can

we

run
/
test

outside

the

framework?


Many

classes
you

depend

on

are hardware
specific

(Files,
Connections)


Solution
:


Built

a
simple


java

abstraction

layer

that

implements

javax.microedition

classes (
and

proprietary

ones

too)
that

runs
on

Java
SE
on

the

desktop.


© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

How

to

keep

historical
,
debugging

info
?


Problem
:


No
long
-
term

storage

on

devices

to

keep

logs


Flash
memory

can

hold

at

most

30 minutes
of

logs


The logs
you

need

are
the

ones

when

device

is

unreachable
,
but

alas,
you

can’t

download it.


Solution
:


Have

a “
black

box”
flight

recorder


Small
,
coded

files
that

can

hold

critical

information

for
weeks


© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

How

to

synchronize

access

to

resources
?


Problem
:


Access
to

system
resources

must
be

synchronized


Serial
device
, system files


synchronized

can

lead
to

unknown

lock

ups


Some
race

conditions

hard
to

track

ando
also

some bugs...


Solution
:


Custom

implementation

of

Java
SE’s

Lock


ReentrantLock

inspired


Added

few

methods

for
convenience


tryLock
,
so

that

you

lock

if

you

can
,
but

don’t

fail

if

you

can’t


fastLock
, for
locks

that

should

last

than

2
secods


© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

How

to

default
parameters
?


Problem
:


Same

firmware for
different

clients

with

different

needs
,
lots

of

parameters


Parameters

for APN, Serial, Timings
and

such


Flash
memory

can

get

corrupt

and

you

lose

your

parameter

files!


Solution
:


Use JAD
parameters


JAD
can

be

custom

tailored

to

each

client


Can

provide

good

defaults



© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

Pending

challenges
...

...
because

you

can’t

win

them

all
...

© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

Many

pain

points are still
there


Make

smaller

binary

fragments
?


And

small

updates

over
the

air
.


Run

multiple

applications

at

the

same

time?


More “
permanent

and
/
or

remote

logging


Security
issues


Better

management
of

keys


Anti
-
tampering

measures



© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

Incoming

changes

Java ME (r)
evolution

© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

Java ME 8
has

much

new
features
!


Java ME 8
seems

to

have

addressed

some,
if

not

most
,
this

problems

and

abstractions


Device

I/O


Access Point


NIO


...
And

many

others
!


Some still
need

works


No
concurrency

support

might

mean

no
locks
...


We’ll

work

with

JSRs

to

try

and

address

the

rest

of

them


V2COM
is

joining

the

JCP EC
to

help
foster

Java ME
evolution


© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

Q?

© 2002
-

2013 V2COM
-

Hardware + Software. All rights reserved

Thanks
!

llima@v2com.mobi

www.v2com.mobi