NOWCASTING WITH NEURAL NETWORK

apricotpigletAI and Robotics

Oct 19, 2013 (3 years and 9 months ago)

57 views

INSTITUTO DE PESQUISAS METEOROLÓGICAS

NOWCASTING WITH NEURAL NETWORK



USING REFLECTIVITY IMAGES OF



METEOROLOGICAL RADAR

R. Machado (1,2)

C. A. Thompson (2)

R. V. Calheiros (1)

(1) Meteorological Research Institute (IPMet)/UNESP, Bauru, SP, 17033
-
360, Brazil



(2) Polytechnic Institute (IPRJ)/UERJ, Nova Friburgo, RJ, 28601
-
970, Brazil

Lençóis Paulista/SP, may,25
-
04, 20UTC


F2

Palmital/SP, may,25
-
04, 17UTC


F3

Indaiatuba/SP, may,24
-
05, 20:30UTC


F3

INSTITUTO DE PESQUISAS METEOROLÓGICAS

OBJECTIVES & STATUS


GENERAL

:

SUPPORT

TO

OPERATIONAL

NOWCASTING

IN

CENTRAL

SÃO

PAULO



SPECIFIC

:

IMPLEMENT

A

NEURAL

NETWORK

APPROACH

TO

IPMET’S

OPERATIONAL

FORECASTING

PRACTICES



AS

OF

NOW
:

PRELIMINARY

TESTS

OF

NETWORK


PERFORMANCE

RUN

FOR

DISTINCT

COMPUTATION

OF

AVERAGES

ON

STATISTICAL

TEXTURE

DESCRIPTORS

INSTITUTO DE PESQUISAS METEOROLÓGICAS

DATA & AREA


PRODUCT
:

REFLECTIVITY

CAPPIS

AT

3
,
5

KM

HEIGHT

AGL,

TO

A

240

KM

RANGE

FROM

THE

BAURU

RADAR

(BRU)


PERIOD
:

SUMMERS

OF

2002
/
2003

&

2003
/
2004


DATA

SET
:

300

IMAGES

GATHERED

IN

INTERVALS

OF

2

HOURS

EACH,

COMPOSING

TWO

SUB

SETS
:

CHARACTERIZED

BY

(
1
)

RAIN

AND

(
2
)

NORAIN

SITUATION

AT

THE

END

OF

THE

TIME

INTERNAL


DATA

SAMPLE
:


TARGET

AREA
:

15

KM

RADIUS

CIRCLE

AROUND

THE

RADAR

INSTITUTO DE PESQUISAS METEOROLÓGICAS


PROCESSING

A)


STATISTICAL

TEXTURE

DESCRIPTORS

(ATTRIBUTES),

I
.
E
.

MEAN,

STD

DEVIATION,

SKEWNESS

AND

KURTOSIS

WERE

COMPUTED

FOR

EACH

IMAGE



AVERAGES

OF

THE

ATTRIBUTES

WERE

CALCULATED

FOR

ALL

IMAGES

WITHIN

EACH

2

H

INTERVAL



RESULTED

TWO

SETS

OF

ATRIBUTE

VECTORS
:

ONE

CORRESPONDING

TO

THE

PROCESSING

OF

EACH

IMAGE,

AND

THE

OTHER

FOR

THE

AVERAGE

VALUES

OF

EACH

ATTRIBUTE

(SEE

TABLE

1
)

INSTITUTO DE PESQUISAS METEOROLÓGICAS

TABLE 1 : ATTRIBUTES FOR A SAMPLE
EVENT 30
-
JAN
-
2004 FROM THE DATA
BANK ( ALL TIMES LT = UTC


3)

OBS.: STATISTICS WERE COMPUTED ON THE IMAGE PIXELS IN mm.h
-
1
DERIVED WITH Z = 300R
1,4

TIME EVENT


ATTRIBUTE


12:53


13:23


13:53


14:23


AVERAGES


MEAN


0.1529


0.2059


0.2629


0.2675


0.2223

STD

DEVIATION


-
1.5224


1.8106


1.9487


2.0532


1.8337


SKEWNESS


-
0.1604


-
1.3625


-
1.9867


-
1.8567


-
1.3415


KURTOSIS


9.1118


8.1791


6.2289


5.2513


7.1927

INSTITUTO DE PESQUISAS METEOROLÓGICAS



B)


ATRIBUTE

VECTORS

WERE

USED

AS

INPUTS

TO

A

NEURAL

NETWORK

CONSTITUTED

OF

2

LAYERS,

WITH

4

NEURONS

IN

THE

HIDDEN

LAYER

AND

1

NEURON

IN

THE

OUTPUT

LAYER

LINE COMMAND WAS:
net2=newff(minmax(p),[4,1],{‘logsig’,’logsig’},’traingda’);

Newff = NETWORK WITH BACK
-
PROPAGATION

[
4
,
1
]

=

TWO

LAYERS,

4

NEURONS

IN

THE

HIDDEN

LAYER

AND

1

NEURON

IN

THE

OUTPUT

LAYER
;

AND


logsig

=

TRANSFER

FUNCTION

OF

EACH

NEURON,

DIFFERENTIABLE,

WITH

OUTPUT

BETWEEN

0

AND

1
.


RESULTS

OF

RUNS

WITH

DIFFERENT

NEURAL

NETWORKS

ARE

DEMONSTRATED

FOR

TWO

OF

THEM


TRAINNING

WAS

EFFECTED

FOR

OUTPUT

VALUES

BETWEEN

O

AND

1
,

AS
:
IF

OUTPUT



0
.
5



RAIN,

OUTPUT<

0
.
5



NO
-
RAIN

(OBS
.

AN

AREA

PROBABILITY

OF

RAIN,

a,

IF

δi

IS

AN

INDICATOR

VARIABLE

EQUAL

TO

1

WHEN

RAINS

OCCURS

AT

A

POINT

1
,

AND

ZERO,

IS


FOR

BRU,

ECHO

STATISTICS

INDICATES

a

~

0
.
55

FOR

SUMMER)
.








=




=
=
>

S

0

i

m

=
=
1

i

P

a

d

INSTITUTO DE PESQUISAS METEOROLÓGICAS



FIRST NETWORK

1.
TRAINING: PERFORMANCE RESULT

INSTITUTO DE PESQUISAS METEOROLÓGICAS

2.
SIMULATION:
ATTRIBUTES FOR EACH
IMAGE WERE USED. THE FIRST 15 IMAGES
WERE KNOWN TO RESULT IN RAIN, AND THE
LAST 15 IMAGES IN NO
-
RAIN,

FIRST 15 IMAGES (VALUES SHOULD BE ≥ 0.5)

0.8305


0.7978


0.789


0.7882


0.2381

0.2316


0.3901


0.421


0.897


0.7415

0.5778


0.6096


0.895


0.8603


0.8477


LAST 15 IMAGES (VALUES SHOULD BE < 0.5)

0.1595


0.1132


0.24


0.322


0.2745

0.2205


0.2051


0.193


0.2726


0.3201

0.303


0.2559


0.3032


0.3114


0.2616

TABLE 2


RESULTS AT THE OUTPUT OF THE FIRST NETWORK

ERRORS (VALUE IN
RED
) = 4/30


ㄳ%

=
㠷%=但=单䍃䕓E
=
INSTITUTO DE PESQUISAS METEOROLÓGICAS


SECOND NETWORK (TRAINING NOT SHOWN)

ERRORS (VALUES IN
RED
) = 1/30




=
㤷%=但=单䍃䕓E
=
FIRST 15 IMAGES (VALUES SHOULD BE ≥ 0.5

0.7375


0.6675


0.8093


0.8564


0.4218

0.6752


0.8411


0.9048


0.566


0.8166

0.972


0.8733


0.9503


0.9561


0.9485


LAST 15 IMAGES (VALUES SHOULD BE < 0.5

0.3749


0.348


0.0304


0.166


0.0223

0.0519


0.2077


0.1444


0.054


0.0261

0.2231


0.108


0.081


0.2816


0.263

SAME

ATTRIBUTES

AND

CRITERIA

FOR

RAIN

(


0
.
5
)

AND

NO
-
RAIN

(<

0
.
5

)

AS

THE

FIRST

NETWORK,

BUT

USING

AVERAGES

OF

THE

ATTRIBUTES

TAKEN

OVER

EACH

2
H

INTERVAL
.

TABLE 3


RESULTS AT THE OUTPUT OF THE SECOND NETWORK

INSTITUTO DE PESQUISAS METEOROLÓGICAS

CHI


SQUARE TEST


TWO SAMPLES (2 H INTERVALS) TAKEN. FOR THE FIRST x
1
=22, n
I

=30 AND
FOR THE SECOND: x
2
=79, n
2
=90, WHERE x
i=1,2

= RAIN, n
i=1,2

= SAMPLE SIZE.


NULL HYPOTHESIS IS FORMULATED, I. E. RAIN/ NO
-

RAIN RELATIONS ARE
TRUE. x
i = 1,2
IS A RANDOM VARIABLE. MODELING ITS BINOMIAL
DISTRIBUTION BY A NORMAL DISTRIBUTION, THE PROPORTION OF THE
SAMPLE TAKEN BY x

(
θ

=
X

/
n
)

IF UNKNOWN, IS




FOR THE MODELED NORMAL DISTRIBUTION ,



IF

THE

RANDOM

INDEPENDENT

VARIABLES

z
1

AND

z
2

HAVE

STANDARD

NORMAL

DISTRIBUTION,

THEN

(y=

z
2

+

z
2
)

HAS

A

CHI
-
SQUARE

DISTRIBUTION

WITH

m

DEGREES

OF

FREEDOM
.


COMPUTING Y WITH THE ABOVE NUMBERS :
θ

= 0.84 y = 3.495


FOR
α
= 0.05 AND
υ
(
m
) = 2 DEGREES OF FREEDOM


= 5.991(>3.495)


NULL

HYPOTHESIS

IS

SATISFIED

TO

95
%

OF

CONFIDENCE,

I
.

E
.
,

FORECASTS

ARE

NOT

BIASED
.

^

^

^

^

INSTITUTO DE PESQUISAS METEOROLÓGICAS


NEXT STEPS

A)
IMPROVEMENT

OF

PERFORMANCE


A

.

1
)

ADD

NEW

TECHNIQUES,

E
.

G
.


GABOR

FILTERING

AS

A

FIRST

LAYER

IN

THE

NETWORK

SYSTEM

TO

EXTRACT

TEXTURAL

FEATURES,

WHICH

WILL

FEED

THE

INPUT

LAYER

OF

THE

FORECASTING

NETWORK


(GABOR

FILTERING

HAS

SHOWN

TO

IMPROVE

THE

PERFORMANCE

OF

NEURAL

NETWORKS)


FUZZY

LOGIC,

DUE

TO

THE

FACT

THAT

THERE

IS

NO

CLEAR

SEPARATION

BETWEEN

SEASONS,

DAILY

INTERVALS,

AND

OTHER

STRAFICATION

FACTORS
.


GENETIC

ALGORITHMS,

WHICH

USE

TECHNIQUES

OF

BIOLOGICAL

DERIVATION

THAT

COULD

BE

APPLIED

TO

RAINFALL

CONFIGURATIONS

SUCH

AS

:

HERITAGE

(

RAIN

AT

T
0

IS

RELATED

TO

T
0



1
),

MUTATION

(

RAIN

PATTERNS

CHANGE

STRUCTURE

IN

TIME),

NATURAL

SELECTION

(PREFERENTIAL

DEVELOPMENT

CONDITIONS

EXIST),

AND

RECOMBINATIONS

(

RAIN

CELLS

SPLIT

AND

MERGE

IN

TIME)

INSTITUTO DE PESQUISAS METEOROLÓGICAS


A
.
2
)

ADD

NEW

(METEOROLOGICAL)

ATTRIBUTES,

E
.

G
.


DOPPLER

RADAR

WINDS


SATELLITE

IMAGES

(VIS,

IR,

WV

&

MW)

INDIVIDUALLY

OR

IN

COMBINATIONS

TO

INFER,

E
.

G
.

RAIN/NO

-

RAIN

THRESHOLD
.


VARIABLES,

LIKE

TEMPERATURE,

PRESSURE,

HUMIDITY
.


INSTITUTO DE PESQUISAS METEOROLÓGICAS

B)

VERIFICATION/VALIDATION

COMPARISONS

WITH

OTHER

NOWCASTING

TECHNIQUES

EITHER

IN

TESTS

OR

OPERATIONAL,

OR

IN

CONSIDERATION

FOR

OPERATIONAL

USE,

AT

IPMET

FORECASTING

SECTOR
.

B
.
1
)

TITAN

(THUNDERSTORM

IDENTIFICATION,

TRACKING,

ANALYSIS

AND

NOWCASTING)

PREDICTING

ECHO

CENTROID

POSITION

EVOLUTION
.

STATUS
:

UNDER

OPERATIONAL

EVALUATION

B
.
2
)

KAVVAS

(ADAPTIVE

EXPONENTIAL

METHOD)

PREDICTING

SHORT
-
TERM

EVOLUTION

(
15

MIN
.

TO

2

H)

OF

CENTROID,

BASED

ON

REFLECTIVITY

AND

VELOCITY

(DOPPLER)
.

STATUS
:

UNDER

STUDY

B
.
3
)

VIL

(VERTICALLY

INTEGRATED

LIQUID

WATER

CONTENT)

PREDICTOR

IS

WATER

COLUMN

FROM

GROUND

TO

12

KM

AGL

COMBINED

WITH

PRESENCE

OF

45

dBZ

ABOVE

3

KM
.

STATUS

:

OPERATIONAL

INSTITUTO DE PESQUISAS METEOROLÓGICAS

CONCLUSIONS


NEURAL

NETWORK

APPROCH

TO

RADAR

BASED

NOWCASTING

IN

CENTRAL

SÃO

PAULO

HAS

SHOWN

CLEAR

POTENTIAL
.


STATISTICAL

TEXTURE

DESCRIPTORS

HAVE

PROVEN

A

VALID

INPUT

TO

THE

NOWCASTING

WITH

NEURAL

NETWORK

IN

CENTRAL

SÃO

PAULO
.


IMPROVEMENTS

RESULTING

FROM

AVERAGING

DESCRIPTOR

VALUES

INDICATES

THAT

EVEN

RELATIVELY

MINOR

OPERATIONS

ON

IMAGE

CHARACTERISTICS

CAN

SIGNIFICANTLY

IMPACT

NETWORK

PERFORMANCE
.


FURTHER

IMPROVEMENTS

SHOULD

BE

PARTICULARLY

EXPECTED

FROM

TEXTURE

CLASSIFICATION

THROUGH

GABOR

FILTERING
.