3. Management - FER-a

bossprettyingΔιαχείριση Δεδομένων

28 Νοε 2012 (πριν από 4 χρόνια και 9 μήνες)

274 εμφανίσεις





















ColdWatch

User manual


Version
1.0


ColdWatch


Version:
1.0

User manual


Date: 2012
-
01
-
12





Page
3



Revision History


Date

Version

Description

Author

2012
-
01
-
06

0.1

Made a template for user manual

Luka Postružin

㈰ㄲ
-

-


0.2

fns瑡汬慴楯n 慮d sys瑥m 捯nf楧ur慴楯n 慤d敤

Luka Postružin

㈰ㄲ
-

-
N
N


P

䵡nua氠lor us楮g th攠sys瑥m m慤e

Luka Postružin

㈰ㄲ
-

-


N.0

䙩c慬av敲s楯n

Luka Postružin


ColdWatch


Version:
1.0

User manual


Date: 2012
-
01
-
12





Page
4




1.

Introduction

................................
................................
................................
............

6

1.1

Purpose of this document

................................
................................
...............................

6

1.2

Intended Audience

................................
................................
................................
.........

6

1.3

Scope

................................
................................
................................
.............................

6

1.4

References

................................
................................
................................
.....................

6

2.

System installation

................................
................................
................................
..

7

2.1

GSN server installation

................................
................................
................................
...

7

2.1.1

Installation of GSN o
n Debian server

................................
................................
..............................

7

2.1.1.1

Installing Java on Debian
................................
................................
................................
.............

7

2.1.1.2

Installing ant on Debian

................................
................................
................................
..............

7

2.1.1.3

Installing screen

................................
................................
................................
..........................

7

2.1.1.4

Installing
SVN on Debian

................................
................................
................................
.............

7

2.1.1.5

Downloading GSN from SVN

................................
................................
................................
.......

8

2.1.1.6

Managing GSN server

................................
................................
................................
.................

8

2.1.2

GSN server configuration

................................
................................
................................
................

8

2.1.2.1

Security in GS
N
................................
................................
................................
............................

8

2.1.2.2

HTTP Basic Authentication Mode

................................
................................
...............................

8

2.1.2.3

Configuring SSL on GSN server
................................
................................
................................
....

9

2.1.2.4

Configuring storage in GSN:

................................
................................
................................
........

9

2.1.2.5

Insta
llation and configuration of exim4 SMTP server

................................
................................
.

9

2.1.2.6

Configuration of GSN for sending emails

................................
................................
..................

10

2.2

Database installation

................................
................................
................................
....

10

2.2.1

Database configuration
................................
................................
................................
.................

10

2.2.2

Database managing

................................
................................
................................
......................

11

2.2.3

Cronjob


periodical data acquisition

................................
................................
...........................

13

2.3

Web application installation

................................
................................
.........................

14

2.3.1

SSH2 installation

................................
................................
................................
...........................

14

2.3.2

Application deployment

................................
................................
................................
...............

15

3.

Management

................................
................................
................................
........

15

3.1

GSN management

................................
................................
................................
.........

15

3.1.1

KeyboardTesting wrapper

................................
................................
................................
.............

15

3.1.2

SMS service and SmsService class
................................
................................
................................
.

15

3.1.3

YahooWeather wrapper

................................
................................
................................
...............

15

3.2

Database management

................................
................................
................................
.

15

3.2.1

ETL process

................................
................................
................................
................................
...

16

3.2.2

Data marts

................................
................................
................................
................................
....

16

3.3

Web interface management

................................
................................
..........................

16

3.3.1

Web interface development guide

................................
................................
...............................

16

3.3.1.1

Application

................................
................................
................................
................................

17

3.3.1.2

Config

................................
................................
................................
................................
........

17

3.3.1.3

Views

................................
................................
................................
................................
.........

17

3.3.1.4

Controllers

................................
................................
................................
................................

18

3.3.1.5

Model

................................
................................
................................
................................
........

19

3.3.1.6

Helpers

................................
................................
................................
................................
......

19

3.3.1.7

Error handling

................................
................................
................................
...........................

19

3.3.2

Web application user guide

................................
................................
................................
..........

20

ColdWatch


Version:
1.0

User manual


Date: 2012
-
01
-
12





Page
5



3.3.2.1

Admin guide

................................
................................
................................
..............................

20

3.3.2.2

User guide

................................
................................
................................
................................
.

26

4.

Conclusion

................................
................................
................................
.............

32



ColdWatch


Version:
1.0

User manual


Date: 2012
-
01
-
12





Page
6



1.

Introduction


1.1

Purpose of this do
cument

This document will be used as a full user manual. We will describe the process of installation and configuration
of the system, as well as
how to use it. Special sections

will
focus on

administration actions and end user manual

respectively
.


1.2

Intend
ed

Audience

This document is written for end user and for person who will administrate the system once it is deployed on the
server.

Also basic steps for deploying the application are also provided.


1.3

Scope

This document will focus only on how to manage ov
er system, without further explanations on possible
improvements. Main focus will be getting our application up and running. After that all of the necessary actions
concerning managing this system, will also be written in the following sections.


1.4

Reference
s

Reference to GSN application that was used for data collection:



http://sourceforge.net/apps/trac/gsn/

Reference to project site that consist of all the necessary data you could possibly think of:



http://www.fer.unizg.hr/rasip/dsd/projects/coldwatch


ColdWatch


Version:
1.0

User manual


Date: 2012
-
01
-
12





Page
7



2.

System installation

2.1

GSN server installation

First, we will let you in on how to install GSN server. If you will find this not sufficient
, or you possibly have
some other problems we have not encountered, please refer to references site.


2.1.1

I
nstallati
on

of GSN on Debian server


Assumption is that GSN source code is in: svn://lapis.rasip.fer.hr/svn/dsd11/WSN/GSN%20server/Trunk
. This
SVN reposi
tory was provided to us by FER for usage on DSD project only. This repository will be the key in
the following sections because it consists of all the code we have implemented. Also, please be aware that there
are multiple folders with similar names, so tr
y to use the ones we provided to you here. If you have some
problem with names of each folder, consult yourself with our structure policy for SVN on the main project site
(references).

We will now guide you through the process of installing GSN on Debian s
erver. One thing worth mentioning is
that you will need to know a thing or two of working in
UNIX

environment.


2.1.1.1

Installing Java on Debian

First off, you will need to install java on your Debian server. Why Debian? Because we used it on our project
and it p
roved to be sufficient for our needs. You will need to open the terminal (if you have graphical system)
and simply input the following statements.


sudo su


//switch to root

# apt
-
get update

#apt
-
get install openjdk
-
6
-
jdk


Be aware that you will be asked

to provide valid password for the root user. If you do not know what it is,
contact your server administrator.


2.1.1.2

Installing ant on Debian

Second program you need to install is Ant. This is used to bind our GSN application after making some
drastically chan
ges. Actions are similar to building the whole project. So you will need to input the following
statement (we assume you are still in root mode):


#apt
-
get install ant


2.1.1.3

Installing screen

You still need to install one more application. This application will

be used for running your GSN in the
background so you can run it through Putty and you do not need to keep your connection running. GSN will
then be run after the connection with the server is disconnected.


#apt
-
install screen


2.1.1.4

Installing SVN on Debian

S
VN is a repository that will gather the whole code of GSN. This will be used because in that way you can build
the application directly from an internet source, to have up to date version of the application.


# apt
-
get update

#apt
-
install subversion

# apt
-
get install libapache2
-
svn

# /etc/init.d/apache2 restart

ColdWatch


Version:
1.0

User manual


Date: 2012
-
01
-
12





Page
8



sudo su username


2.1.1.5

Downloading GSN from SVN

After installing all the prerequisite programs, you can now collect your GSN code from the internet by typing in:


svn co svn://lapis.rasip.fer.hr/svn/dsd1
1/WSN/GSN%20server/Trunk


Please make sure that SVN repository is up and running so you do no
t get any errors in the process.


2.1.1.6

Managing GSN server

Here are couples of crucial actions you can do with GSN after making your installation. One of them is runnin
g
your GSN, what can be done by following statement.


cd user/Trunk

sudo ant gsn


Also, as mentioned previously, if you want GSN to run even after you close your session, you need to input the
following instead of the last line in the previous section.


sc
reen

sudo ant gsn


Now you can close your session and be certain that GSN is running.

2.1.2

GSN server configuration

2.1.2.1

Security in GSN

Regarding the security, GSN can run in the three following modes.

Mode

SSL
(HTTPS)

Description

Open

Optional

The default mode. A
ll the virtual sensors are available to any client

HTTP Basic
Authentication

Optional

All the virtual sensors are available to any authenticated client. The SSL
support should be enabled.

In order to get data out of GSN, whenever it is through the web in
terface, the rest wrappers or a web service, we
are always using a servlet handled by the Jetty Container. The table below lists the differents PATH in GSN and
how they are securised regarding the security mode.

Path

Description

Open Mode

HTTP Basic Mode

/gsn, /field, /data, /multidata,
/geodata

Basic GSN URLs

Open

HTTP Basic

/streaming

Remote Wrapper
Implementation

Open

HTTP Basic

/gsn/My*

Access Control web interface

Form

Form

/services/GSNWebService

GSN Standard Web Service

Open

HTTP Basic

/services
/Service

Microsoft

SensorMap

Web
Service

Open (protected by
IP source)

Open (protected by
IP source)

/services/A3DWebService

Alpine3D Web Service

Open

HTTP Basic


2.1.2.2

HTTP Basic Authentication Mode

GSN uses the Basic

Access Authentication scheme for HTTP authentication. By default the authentication is not
activated. To activate the authentication, you should edit
conf/realm.properties

and add one or more line like the
ColdWatch


Version:
1.0

User manual


Date: 2012
-
01
-
12





Page
9



following:

username:password,rolename


The rolena
me you must provide is

gsnuser
. Once you provided at least one such line in
the

conf/realm.properties

and started GSN, the HTTP authentication is activated and everything
under

http://host:port/

path is protected. Any acce
ss to the web interface needs a username/password that
matches one of the lines you provided in the configuration file. Also, remote wrappers (Push and Rest) need to
have the username/password predicates provided to them to be able to connect to the secure
d remote instance.

2.1.2.3

Configuring SSL on GSN server

In conf/gsn.xml add following:


<ssl
-
port>8443</ssl
-
port>

<ssl
-
key
-
store
-
password>changeit</ssl
-
key
-
store
-
password>


<ssl
-
key
-
password>changeit</ssl
-
key
-
password>


Where

<ssl
-
port> tag refers to port for ss
l connection. Tags <ssl
-
key
-
store
-
possword> and <ssl
-
key
-
p
assword>
you can fill in with t
h
e

keys you get with your SSL Certificate.


2.1.2.4

Configuring storage in GSN:

Before configuring the storage you need to install PostgreSQL database. This procedure will be

explained in the
following section so please look it up before continuing.

GSN uses a database for processing (windows, user queries, etc.) and for storing/retrieving the processed data.
We will use
split

data storage. T
his mode splits the processing and
the storage in two different databases. With
this mode, the performances can be improved. In order to use this mode we have to configure the
file

conf/gsn.xml

like the one below.


...

<!
--

Main Storage Database, Mandatory
--
>

<storage user="dbuser" passwor
d="" driver="org.postgresql.Driver"
url="jdbc:postgresql://localhost/gsn:port" >...

<!
--

Processing Database, Optional. If not specified, the processing is
implemented using the Main Storage Database specified above
--
>

<sliding>


<storage user="sa" pa
ssword="" driver="org.h2.Driver"
url="jdbc:h2:mem:sliding;DB_CLOSE_DELAY=
-
1" />

</sliding>

...


2.1.2.5

Installation and configuration of exim4 SMTP server

These are

configuration

steps for mail
-
sending with Exim and

Googlemail account
.

This tutorial has been made

for our account, what you can see in the last section, but in the beginning we will write more general
configuration information.

Installation of exim4, application for email sending on Debian servers is done easily with the following
statements.


$ apt
-
g
et update

$ apt
-
get install exim4


Start configuration wizard from root:


# dpkg
-
reconfigure exim4
-
config


Answer the questions of configuration wizard. The important one is

a general type of mail configuration.
ColdWatch


Version:
1.0

User manual


Date: 2012
-
01
-
12





Page
10



Choose “sent by smarthost”
. A
nswer other que
stions
:



System Mail Name: e.g. mycompany.com



Provide IP addresses to listen on for incoming SMTP connections



User 127.0.0.1 if you don’t want only send mail from local machine and nothing for all IP’s.



Machine handling outgoing mail for this host (smarthos
t): smtp.gmail.com::587

Other parameter
s are not important

C
redentials information of your gma
il account needs to be set in

/etc/exim4/passwd.client

file with text editor.


gmail
-
smtp.l.google.com:yourYourMail@googlemail.com:yourPass

*.google.com:yourYourM
ail@googlemail.com:yourPass

smtp.gmail.com:yourYourMail@googlemail.com:yourPass


Make sure /etc/exim4/passwd.client belongs to user root and group Debian
-
exim
:


$ chown root:Debian
-
exim /etc/exim4/passwd.client


Actualize the whole configuration with:


$ u
pdate
-
exim4.conf


You can test your mail server with:


echo

"Server Mail Test Message "

| mail
-
s "Just Test"

SomeMail@someDomain.org


2.1.2.6

Configuration of GSN for sending emails

You need to change some things in the conf/emails.properties file to enable conne
ction of the GSN server with
your email account.


mail.smtp.from=
ColdWatch

mail.smtp.password=
********

mail.smtp.host=
smtp.gmail.com

mail.smtp.user=
coldwatch2011@gmail.com

mail.smtp.starttls.enable=
true

mail.smtp.auth=
true

mail.smtp.port=
587


What is still

left is installing SSH2 on the server. This will be explained later in the process, as the same package
will be used on application side to tra
nsfer notification files to GSN

2.2

Database
installation

As you would expect, we now need to install our database f
or data storage. This database does not need to be on
the same server as the GSN (only for the case of GSN storage option!) so you can choose another server you
find convenient. We assume that you already have PostgreSQL database installed on the server (a
s we have
had). In case of some problems, contact your server administrator.

The following steps will provide installation
steps for database on GSN server. Only thing that should be different for our other server is username.


2.2.1

Database configuration

First
, we need to go into root user mode, since that is the only way to go to postgres user, which is crucial in
PostgreSQL database. This user is considered as super user for database.


sudo bash

su postgres


ColdWatch


Version:
1.0

User manual


Date: 2012
-
01
-
12





Page
11



A
fter this we need to create the user with the same

name as our user name on Debian so we can connect to the
database with this user.


createuser mrenic


Y
ou will be prompt to answer a question.


Shall the new user be a superuser? (y/n)


Y
ou must answer this question with 'y' as this user will then be all
owed proper
privileges
. If you are making
another user, you can answer this question with 'n' and you will then be asked to answer two more questions
regarding user
privileges

(cas user create new roles and possibly make databases).


When we have created t
he user, we need to create the database for that user on which he can
operate
.

First, we need to go into our user mode. That can be done with 2
commands.


exit

su mrenic


We have closed our postgres user and connected as mrenic user.


createdb mrenic


Th
is command creates our database named 'mrenic'. We still need to do one thing, that is adding password for
our mrenic user.


psql

alter role mrenic with encrypted password 'matijagsn';


Psql command gets us into script mode for our database, from which we
can alter our role on the system.

What we need to do yet is add our user possibility to connect to this database.


sudo nano /etc/postgresql/8.4/main/pg_hba.conf


Path to this file is something that depends on your installation, so this should be adjusted
if you are performing
these steps

from installation on your own server.

At the
e
nd of the document you can add a line similar like this


local mrenic mrenic ident


This will create a possibility for a local connection to mrenic database from user mrenic w
ith his credentials.

If user wants to be able to establish remote connection to the database he/she should write similar line to the
previous:


host mrenic mrenic IP_address_of_the_user md5


After all this we need to restart our PostgreSQL server with foll
owing command:


sudo /etc/init.d/postgresql restart


2.2.2

Database managing

What is important to know for database administration is how our system works eventually. Following image
will show you how our database and its constraints look in the real situation.
You should take time and look into
it with much patience as this is not another simple relational database. We have used a combination of relational
database for production purposes, and data warehouse modeling for data archiving. This implies we are using

a
ColdWatch


Version:
1.0

User manual


Date: 2012
-
01
-
12





Page
12



dimensional schema for this purpose.


ColdWatch


Version:
1.0

User manual


Date: 2012
-
01
-
12





Page
13




For managing our database and data archiving process, you need to provide a cronjob on the server. This means
you will need to use couple of scripts that we have developed and let them
run in the background periodically to
allow them to collect the data from GSN and store it in our data archive.


2.2.3

Cronjob


periodical data acquisition

How to set up cronjob is pretty much easy. All you need is to deploy all the scripts you can find in our
ETL
process folder, and change the cronjobs.txt file with appropriate information about database administrator (ETL
process owner). This person will then get email on every run of the scripts so that you can have
valid

information on the process. We have m
inimized a possibility of error, and you will get a log similar to the
following:


2012
-
01
-
12 22
:05:02
---
> adding sensors started

2012
-
01
-
12 22
:05:02
--
> adding sensors ended


2012
-
01
-
12 22
:05:03
--
> extracting data started


There is no sensor with name "
n_mrenic_a_temperaturekeyboardvs1_1326102375"


There is no sensor with name "n_mrenic_a_temperaturekeyboardvs2_1326109317"


There is no sensor with name "n_mzagar_a_temperaturekeyboardvs1_1326111107"


There is no sensor with name "n_mzagar_a_temperaturekey
boardvs3_1326111051"


There is no sensor with name "n_ssentilles_a_temperaturekeyboardvs1_1326116487"


There is no sensor with name "n_jfeljan_a_temperaturekeyboardvs1_1326116294"


There is no sensor with name "n_ibosnic_a_temperaturekeyboardvs1_1326116162
"


There is no sensor with name "n_mrenic_a_temperaturekeyboardvs3_1326109204"


There is no sensor with name "n_aneta_a_temperaturekeyboardvs1_1326116578"


There is no sensor with name "n_morlic_a_temperaturekeyboardvs1_1326116078"


There is no sensor with

name "n_sreeharivallu_a_rabbitrasip1_1325870339"


There is no sensor with name "n_mrenic_a_temperaturekeyboardvs1_1326154392"


There is no sensor with name "n_icavrak_a_temperaturekeyboardvs1_1326116021"


There is no sensor with name "n_icrnkovic_a_temper
aturekeyboardvs1_1326115822"


2012
-
01
-
12 22
:05:12
--
> extracting data ended


Output of the script is sent to your email as we have mentioned earlier, but we would like to clear some things
about the log. First, you have a start and end time of each section

of the process. This means you know exactly
how much time did each script take in execution. You can find here some sensors for which is said that you do
not have them in the database. This information is sent more as a measure of precaution. All the sens
ors that are
really measuring something have a starting name like “a_”. These sensors with prefix “n_” are notification
sensors that are currently in the system, and it is normal that they are not collected in the process. Here you can
also double check if

all the notifications are really active in the system (or have they been created in some
point), and you have information on when they have been introduced in the system. Timestamp is in integer
format (UNIX).


MAILTO=ante.ivankovic@gmail.com

5 */2 * * *
php
-
q
/var/www/TestingLuka/cronjobs/cron_job_ETL_sensors_readings.php

30 1 * * * php
-
q /var/www/TestingLuka/cronjobs/cron_fact_table_filler.php

40 1 * * * php
-
q /var/www/TestingLuka/cronjobs/cron_aggregation.php

45 2 1 * * php
-
q /var/www/TestingLuka/cr
onjobs/cron_agg_month_day_part.php

0 3 * * * pg_dumpall > db_backup.sql

0 4 * * 0 mutt
-
s "Database backup" hyracoidea@gmail.com
-
a
/var/www/TestingLuka/db.sql <message_body.txt


This is the cronjobs.txt file. On the beginning you can see email address whi
ch you can fill with the email of
your choice. Other lines are calling appropriate scripts at the appropriate time. For example, main data
acquisition script (first cron job) is run on fifth minute of every second hour. This proved to be the best practice
ColdWatch


Version:
1.0

User manual


Date: 2012
-
01
-
12





Page
14



on given/taken principle.

Cronjob can be set up with the following set of commands.

crontab cronjobs.txt


This will set up the cron job to execute items from the script. Following two statements will do the following:



stop cron script from executing



show
which the active cron script is


crontab

r

crontab
-
l


2.3

Web application installation

This is the final part of our product. As mentioned earlier, this server can be the same one which has database on
it. What we still have to install are couple of essentia
ls for our application to run smoothly.


2.3.1

SSH2 installation

What we needed to do is install SSH2, which we will use for storing XML files on GSN server. These files will
be used as sensors that send notifications to user via email or SMS. This installation
is used on Debian server
that we have on virtual computers for DSD. First of all, we need to acquire 2 files that will be used as libraries.
We can easily fetch them with following statement:


aptitude install libssh2
-
1
-
dev libssh2
-
php


You can now test if

PHP recognizes its new ssh2 extension by running:


php
-
m |grep ssh2


It should return: 'ssh2'. Possibly this is all you need for the compiling process, but to be sure, there are few steps
in between. First let's install the following packages that we wil
l use, with using simple statement:


aptitude install php5
-
dev php5
-
cli php
-
pear build
-
essential openssl
-
dev
zlib1g
-
dev


After this, we need to get libssh2 (ssh2 program that we will use in the process). You can get the file needed
with following statemen
t:


wget http://www.libssh2.org/download/libssh2
-
1.3.0.tar.gz


Be aware that this will be downloaded in the same folder in which you are in the moment.

After downloading, we need to unpack the file, what we can do with:


tar zxvf libssh2
-
1.3.0.tar.gz


When

this has been done, go in the new folder with:


cd libssh2
-
1.3.0


and run configuration for the ssh2:


./configure


After this is done, simply run:


make all install


ColdWatch


Version:
1.0

User manual


Date: 2012
-
01
-
12





Page
15



We are done installing. Next we need to connect ssh2 with php so we can execute all stat
ements regarding ssh2.
First we need to run:


pecl install
-
f ssh2


The
-
f makes sure ssh2 is installed even though there's not a stable candidate. You could also use the package
name: ssh2
-
beta to overrule this, but it was not necessary

in our situation.
Now you need to make sure our new
ssh2.so module is loaded by PHP. Edit a php.ini file, and add a new line where all the extensions are:


extension=ssh2.so


After this, you are good to go. Now we only need to deploy our software.


2.3.2

Application deployment

As

we already have Apache 2 installed on our server (so we expect you to have one as well) we will in short
explain what you need to do. Application should be transferred to default folder for web application. If nothing
is changed on the configuration of Ap
ache, this folder can be found in var/www. You can now make a new
folder or overwrite everything you find there and you are ready to work.


3.

Management

3.1

GSN management

If you have done all the previous actions properly, you have a valid GSN server deployed a
nd ready for action.
If this is not the case, please consult the website mentioned on the reference section.

As all the server applications that work in real time, GSN is also the one you need to take care of in the future. If
you will have some new senso
rs that you will need to introduce in the system, you should place them in the
appropriate folder on GSN. This folder can be found by the name “virtual sensors” and it consists of the
configuration xml files of currently active sensors in the system. For a
ll the information on how to write this
from the beginning, let us know your problems so we can clarify them in short notice.


3.1.1

KeyboardTesting wrapper

Main purpose of this wrapper is to provide data source for developer while testing Virtual Sensors config
uration
files and Processing classes. KeyboardTesting wrapper collects integer value from testing/testFile.txt. Period of
readings is set to 1 second if not set differently in VS configuration file with rate attribute.


3.1.2


SMS service and SmsService class

In

order to use SMS

service it is necessary to create user account on www.smsmail.com and make
conf/sms.properties file similar to emails.properites but with email address that is registered on
www.smsmail.com
.


3.1.3

YahooW
eather wrapper

One of the most useful wrappers used in our system is the one that collects data from the internet source. This
one collects temperature and humidity for every town we provide him information to collect from. For now, our
system collects dat
a for Zagreb and Vasteras.


3.2

Database management

What you need to now to take care of the database can be found in the following section. We will in short
explain the process that happens in the background so you can fix any problems which could happen, or
improve our process if needed.


ColdWatch


Version:
1.0

User manual


Date: 2012
-
01
-
12





Page
16



3.2.1

ETL process


Our developed ETL process part consists of several different parts. First one is in charge of extracting data about
sensors that are on a certain GSN server, and the second one is in charge of extracting reading
s from sensor that
are gathered on GSN server.

These two processes are executed in the same time and every certain amount of time. A script that starts these
two processes is cron_job_ETL_sensors_readings.php. It gathers all information about GSN servers

that are in
the warehouse and starts chronological extraction.

Sensor data extraction consists of script adding_sensor.php. Addind_sensors.php receives as a parameter
GSN_ID and it builds a URL of that GSN server with the data that is available in the d
atabase. At this URL
GSN server lists information in xml structure about sensors from which it gathers data regarding location, name,
activation date, and units that it measures. This XML file is parsed and the data is stored in the dimension
di_sensors. T
his script writes its outputs to a log file that is named after the name of GSN server with
„_addSensorLog.txt“suffix. Since this script is run chronologically, it also takes care that the data about active
and inactive sensors in warehouse is regularly an
d consistently updated. Table f_sensor_type which has
information about what a certain sensor is measuring is also updated with data from this script.



Readings extraction process consist ETL_process.php. ETL_process.php receives as a parameter GSN_ID. T
han
it tries to connect to the local database of that GSN server. Each GSN server has for every sensor a table in its
database. In this table, sensor stores timestamp and the value of the reading. If connection to the database is
established extraction of
readings is started. Reading are than stored in warehouse in table l_readings along with
time_id and date_id of the timestamp when the reading was taken. This script writes it's outputs to a log file that
is named after the name of GSN server with "_ETLlog
.txt" suffix.



Cron_fact_table_filler.php is assigned to transfer the data from l_readings to f_readings. It will bu run as a
chronological job on Debian server exactly at every midnight. It only extracts the readings from previous days.


3.2.2

Data marts


D
ata in the warehouse is also aggregated on day part (morning, forenoon, afternoon, evening, night), on the
whole day, and also on monthly basis by part of the day.

Script cron_aggregation.php starts aggregation on daily basis and on part of the day. First

it checks when last
successful aggregation was and then it aggregates new values from f_readings. It only takes into account
readings from previous days. Than it fills newly aggregated values to table agg_day and agg_day_part.

Script cron_agg_month_day_p
art.php aggregates values on monthly basis on part of the day. It can be executed
at any time and stores data into table agg_month_day_part. If it is executed in current month it aggregates values
up to that point. Later these values are overwritten if agg
regation is done again in the same month.



System presumptions:

1) Sensors that are considered real and active start with prefix "a_".

2) Period between two executions of cron_job_ETL_sensor_readings has to be longer than the longest period
between to re
adings on sensors.


3.3

Web interface management

This section will be divided into two because of the complexity and differences on system specification and user
manual. First we will describe how we have developed the application. This part is concerning futu
re developers
and administrators of the system, so they can easily recover the system from errors. After that, we will in short
explain how to use our system, or where you can find information about that.

3.3.1

Web interface development guide

Web
-
application has

followed Model
-
view
-
controller architecture with Admin and user
-
interfaces. Code
-
igniter
framework is used to implement most of the code. Thus, Cold_watch is the main directory. It separates the main
application with other files and folders on the server.

ColdWatch


Version:
1.0

User manual


Date: 2012
-
01
-
12





Page
17



3.3.1.1

Application

Application directory contains our whole code which is organized in model

view
-
controller with some other
additional directories as support. Application directory contains following main subdirectories:

3.3.1.2

Config

The config
uration directory contains all the files necessary to configure our application on new server or
database.

3.3.1.3

Views

What user can see are in the views in the form of code. All the admin related view data is in admin directory
and different directories are p
laced for user related views. Most of the views code is in Html, php and Jquery.
All the view files are differentiated in such a way that, if the user is filling a form for a notification, then code
for that particular form will be in views/notification
s/form.php. If user is listing for email notifications, then the
code for that list is in views/notifications/email_list.php. The files are clearly organized in the form of forms and
lists for the purpose of traceability. There are two kind of user profile
s: user and administrator. Thus, there is a
distribution of css files and js with respect to each profile. The following table shows most important
corresponding css/JS (Java Script) files to both user and administrator profiles:


File Type

Administrator

USER

CSS

text.css

style.css

reset.css

loginAdmin.css

Jquery
-
ui.css

forms.css

buttons.css

loginUser.css

login.css

ColdWatch.css StyleTables.css
SpryValidationTextField.css
SpryValidationSelect.css
SpryMenuBarHorizontal.css
SpryValidation
Radio.css
SpryValidationSelect.css
SpryValidationCheckbox.css
SpryCollapsiblePanel.css
dhtmlxcalendar.css

ColdWatch


Version:
1.0

User manual


Date: 2012
-
01
-
12





Page
18



reset.css

JS


functions.js


jquery
-
ui.min.js



StyleTables.js
SpryValidationTextField.js
SpryValidationSelect.js
SpryMenuBarHorizontal.js
SpryValidationRadio.j
s
SpryValidationSelect.js
SpryValidationCheckbox.js
SpryCollapsiblePanel.js
dhtmlxcalendar.js



3.3.1.4

Controllers

Controller is the heart our web
-
application. It is responsible for input and output validations. The core logic
implementation is performed by cont
roller only. When an input is received from the view controller validates
inputs, performs the logical operation as per request and sends a request to models. Then the response from the
model is directed towards the views.


In certain cases where the user

has to receive emails, Controller takes the responsibility of formatting emails and
sending email to the user. Moreover, for all the notifications approvals controllers upload the Xml file to GSN.
In the same way, controller deletes the associated Xml fil
e from GSN. Controllers for admin and user are
differentiated for better maintenance of code. All the administrator controller classes look like
admin_<operation>. All the user controller classes simply look with the name of the corresponding operation.
T
his is described at schema below:












Admin:

admin c
ontroller manages users. It is responsible for adding, editing and deleting user data.

Admin_days:

daily requests.

Admin_email:
This is the smallest controller which is used to send emails to users. It implements the “email
users” function of admin.

Admin_
gsn:

This controller is responsible for managing GSN information. It adds new GSN, edit existing GSN
information, and delete GSN. It is also responsible for processing user requests to GSN. That means all the GSN
privilege management is performed by admin_
gsn only. If admin tries to disapprove the approved GSN request,
then the controller checks for the privileges of the sensors under that GSN, Notification requests in those sensors
and all those information will be deleted by the controller. In addition to

the deletion of the GSN privileges, the
controller sends GSN privilege related Emails to users. If the email sending is failed a pop
-
up will be provided
to admin saying that the email sending failed.

Admin_notifications:

All the email and SMS notification

requests are processed by this controller. On the
ColdWatch


Version:
1.0

User manual


Date: 2012
-
01
-
12





Page
19



approval of email or SMS notifications, the controller builds the xml file and uploads the associated xml file to
GSN in two different locations. Once the uploading of the xml file is successful, controlle
r sends email to user
regarding the status of notification. In the similar fashion, on the disapproval of email or SMS notifications,
controller delete the associated xml file from GSN(if exists) and sends an email to user.

Admin_sensors:

Similar to admi
n_gsn, admin_sensors manage the sensor information and user privileges to
sensors.

Admin_units:

In order to maintain the scalability admin have the option to add new units like wind
-
speed.

Gsn:

This controller contains the most key features of the user
-
in
terface. This shows the available GSN’s and
their privileges to the context user, in the coldwatch system. In addition this controller shows the sensors data in
Graphical and tabular forms to the user.

Help:

This controller does not have much logic but it

simply calls the help menu of the user
-
interface.

Members:

This controller is used to update the user profile.

Notifications:

User notification requests are handled by this controller. On the other it is also responsible for
showing the notification statu
s to the context user. User can also remove his notification whenever he wants. On
removal of notification, the controller removes the corresponding tuple from the database using model and also
removes associated xml
-
file from GSN.

Reports:

All the monthly

reports, weekly reports and daily reports are handled by this controller. This controller
also converts the report data to pdf format.

Sensors:

This shows the available sensors under privileged GSN and handles user sensor requests. If user do not
have any

GSN privilege, controller do not show any sensors.

Users:

User login, logout and password management are controlled by this sensor.


3.3.1.5

Model

The only one thing that can directly interact with database is model. All the queries are in code
-
igniter form.
T
here are 8 models in the system. Each model has its in importance in communicating with the database.
Example: Model gsn_m main aim is to handle the gsn requests either from user or admin. Model
notifications_m main aim is to handle the notification reques
ts either from user or admin. Similarly all the
remaining models communicate with database with the associated tables.

In addition to these directories all the third party libraries are placed in libraries directory. The libraries used are

Library

Utili
zation

Fpdf

Used for generating pdf files

Graph

User for generating graph on user
interface

Sftp

Used for to connect with Sftp server
and help to uploading files to gsn
server.


3.3.1.6

Helpers

This part of our application was intended to simplify our applicat
ion managing and to provide useful actions that
are not considered a real part of MVC architecture.

Helpers

Utilization

xml_helper

Helper file is using for formatting
xml files for notifications and helper
is simply a collection of functions.



3.3.1.7

Error han
dling

Errors are made as informative to help the user and handled in a specific format. All the error files are placed in
errors directory of application.

Error

File handling

Origination

Database error

Error_db.php

Errors for database

Php error

Error_php
.php

All php errors.

Page not found error(404)

Error_404.php

Errors if the requested page does not
found

ColdWatch


Version:
1.0

User manual


Date: 2012
-
01
-
12





Page
20



General Error

Error_general.php

Any other additional errors.


3.3.2

Web application user guide

Our application consists of two separate systems. One is ad
min panel which provides actions to the
administrator that will allow him proper system managing. Other one is the user side, which allows our users to
use the system easily and effectively.

3.3.2.1

Admin guide

This guide will be divided into parts depending on ta
b name. Every tab consists of couple of trivial actions
admin can do.



Dashboard


main screen, has some quick links on other parts of the system




Users

H
ere admin can approve user requests to have privilege and log into page. Admin can also revoke this a
nd edit
each user’s information. Admin information can be changed on the “edit profile” button at the top of the screen.
Admin can here

add a new user to the system. In that process user will get his credentials on the email admin
provided.

ColdWatch


Version:
1.0

User manual


Date: 2012
-
01
-
12





Page
21




Figure
1

List users tab

As mentioned, we can edit the data about users easily. Simply click on the “pencil” button seen on the previous
screen and you are good to go. You can also change the password of the user, what can be seen on the picture

below.


Figure
2

Edit user tab



GSN servers

A
dmin can add new GSN server to the system, as well as delete the old ones. Also, he can edit the data
regarding each GSN, what can be useful if something changed on other servers as th
is action cannot be collected
automatically. Also, admin can approve or disapprove user gsn privilege here.

ColdWatch


Version:
1.0

User manual


Date: 2012
-
01
-
12





Page
22




Figure
3

Add new sensor

Admin can also manage user subscriptions for GSN server. This can easily be done on the tab appro
ved
requests. Tab pending requests serves for the ones that are first time in the system, and are waiting for
confirmation.


Figure
4

GSN request managing



Sensors

A
dmin can review all the sensors that our system currently support
s. This is very useful to see if we have some
sensors that are missing from the gsn or some other problems. Also, admin can again approve or disapprove user
requests for sensor privilege this time.

ColdWatch


Version:
1.0

User manual


Date: 2012
-
01
-
12





Page
23




Figure
5

Listing sensors accord
ing to GSN

Admin can also review requests as for GSN servers that is explained above.


Figure
6

Pending sensor privilege requests tab



Notifications

T
his tab gives admin possibility to approve or disapprove a notification request.

SMS and email notifications are
divided under different tabs as they are considered to be different from administration aspect.

ColdWatch


Version:
1.0

User manual


Date: 2012
-
01
-
12





Page
24




Figure
7

Listing email notification that are waiting for approval

Admin can also view
SMS

request th
at are currently in the system. If none, he gets appropriate message.


Figure
8

List SMS users tab with no data to present



Data

A
dmin can add a new unit for new sensors that will be introduced in the system. If new sensor comes u
p during
ETL with a unit that is not specified here, it won’t be added to our system. Also, admin can expand our date
dimension. This will start to be necessary after 2020.

ColdWatch


Version:
1.0

User manual


Date: 2012
-
01
-
12





Page
25




Figure
9

Units currently in the system

Also, he can revi
ew if the dates in the system correspond to our needs.


Figure
10

Days dimension listing





Email users

Admin also has a console for emailing users with some useful information. He can as well send newsletters to
some people that a
re not in the system, so you can contact your potential customers in the future from the
application.

ColdWatch


Version:
1.0

User manual


Date: 2012
-
01
-
12





Page
26




Figure
11

Sending emails can be done to multiple users at the same time

Pretty much this is it from the admin site. Every actio
n is rather trivial so we won’t explain it more thoroughly.
Naming of the options is done in a way that it matches the action behind the button.

3.3.2.2

User guide

As for the end user, situation is somehow different. All the things administrator can manage, user
can choose
from and request for.



Edit profile

User can edit the data about himself that is saved in the database. He can change his password, email or
anything else he desires at the moment.


Figure
12

User profile editing



Notific
ations

User can do two things on the notifications tab. He can review his current notifications that are already in the
system and see the status of the request. If no request have been found he gets the appropriate message.

ColdWatch


Version:
1.0

User manual


Date: 2012
-
01
-
12





Page
27




Figure
13

No notifications for the user

User can as well make a new notifications request. He can choose from the variety of options, from which
crucial are: critical value, notifications type and resending interval. Resending interval should be set to
somethi
ng useful as user will get alarm again the specified time (only way this would be prolonged is if sensor
reads the data in larger time span).


Figure
14

New notification request



Reports

One of the main features of our system is re
port generating. Reports are made on three basis: daily, weekly and
monthly reports. User can choose one and select sensor and data span to see the report online.

ColdWatch


Version:
1.0

User manual


Date: 2012
-
01
-
12





Page
28




Figure
15

Online report viewing

What is not shown on the previous
image is the fall down menu where user chooses its action but this is pretty
trivial and intuitive so user should not have problems with this.

User can also download report in the pdf format pressing the appropriate button showed below.


Figure
16

Download button on the report

All the things concerning reports are the same for all three versions.



Sensors

Users can do three things with sensors. First off is managing sensor subscription. Users are presented with a list
of sensors the
y can see according to GSN privileges they have. After that, they can ask for a privilege for some
sensors, or see the status of some.

ColdWatch


Version:
1.0

User manual


Date: 2012
-
01
-
12





Page
29




Figure
17

Sensor privilege requests

Except for this, user can view the data of the sensors the
y have privilege to see in two ways.


Figure
18

Form for graphical data view

Users choose multiple sensors and units from this window, and provide valid date span from which they request
the data.

ColdWatch


Version:
1.0

User manual


Date: 2012
-
01
-
12





Page
30




Figure
19

Graphical presentation example

As for the last option, users are provided with bar chart data presentation that is gathered from similar form.



GSN server

User can view all the GSN servers from the system and request for the privilege to see one. This p
rivilege is
vital for seeing any sensor in that GSN’s network. User can request for the privilege or see the status of
previously requested ones.


Figure
20

GSN Server requests



Help

If somehow user forgets how to deliver some acti
ons, there is no place for worrying. We have added to our
system help screen. It consists of all the options our system has, and it describes how to manage those options in
easy and structured way.

ColdWatch


Version:
1.0

User manual


Date: 2012
-
01
-
12





Page
31




Figure
21

User help screen

ColdWatch


Version:
1.0

User manual


Date: 2012
-
01
-
12





Page
32



4.

Con
clusion

This document has described all the vital things necessary to know on how to deploy and manage our
application. All of the things this actions require are held on the webpage of the project (references) for any
customer to see.