Correction of the influence of the atmosphere using forward and inverse neural networks

appliancepartAI and Robotics

Oct 19, 2013 (3 years and 7 months ago)

61 views

Space Shuttle 335 km

20060720 , NASA


Correction of the influence of the atmosphere using

forward and inverse neural networks

Roland Doerffer

Retired from

Helmholtz Zentrum Geesthacht

Institute of Coastal Research

roland.doerffer@hzg.de

Atmospheric correction using models

Water constituents

Phytoplankton, SPM etc

Optical properties:

a(

)
,b
(

)
,Fl.
,

Water reflection Rw

Air/sea interface,
refractive index: Rfresnel

Atmosphere: molecule &
aerosol scattering, gas abs.:

Pathradiance,transmittance

Top of Atmosphere
(TOA) reflection

Bottom reflection

Forward

model

inverse

model

Replace a complex radiative transfer model by

a neural network, which is trained with
simulations using the RTF

The NN associates

a large number of
independent
components (IOPs)
with the dependent
reflectances

Type of Neural networks

Forward

NN

In

Independent:

IOPs, angles,

T, S,

out

Angular dependent

reflectances

Inverse

NN

Independent:

IOPs

Angular dependent

Reflectances + aux

angles,T, S,

in

out

Further NNs:

autoNN,

errorNN,

Normalisation NN

Very flexible, can be adapted

To bands, conditions

But: slow, more noise, ambiguities

During run

Fixed to bands, conditions,

Harder to train, incl.
ambiguities

But: very fast, less noise

Combination of

Inverse and forward

NN

Inverse Modellierung using NN in Optimization Procedures

Start values

Modell

Parameters

IOP / Konz

Radiative transfer

Model or

Forward NN

Reflexion
-

Spektra

simulated

Reflexion
-

Spektra

Satellite

Do spectra

agree ?

Parameters

are the

IOPs / Konz.

Change

Parameters

Determine

Search direction

Downhill in cost function

no

yes

Test = ∑
(Rsim(i)


Rsat(i))
2

Also measure of quality !

Start by

Inverse NN

Training of a neural network for atmospheric correction

MC

code

RLpath_noglint

RLpath_glint

Ed_boa

Tau_aerosole

NNforward

water

(

based

on

Hydrolight

simulations

)

RLw

Selection

Max

sunglint

Max

tau_aerosol

Min. Rlw(560)

Etc.

RLpath

Ed_boa

Tau_aerosole

RLtosa

RLpath

Ed_boa

RLw

Tau_aerosole

Bio

-

optical

model

Atmosphere

-

optical

model

Optional

Polarisation

correction

Transmittance

L_up

Training &

Test

data

set

1

2

4

6

5

7

8

9

10

11

12

13

Aerosol Optical Properties used for NN Training data set

Aerosol Optical Thickness (AOT) at 550 nm

Angström Coefficient

RTF models used for

simulation of training

data sets:



Monte Carlo photon tracing


6Sv


Aeronet based model by


R. Santer


Water: Hydrolight C. Mobley

Turbid water reflectances

Amazone: Rw 0.18 at 681nm

Plata: Rw 0.17 at 681nm

Yangtse: Rw 0.2 at 681nm

Cocco: Rw 0.12 at 560 nm

Simulations with Hydrolight


Hydrolight 5.1 for computation of bi
-
directional water leaving radiance
reflectance spectra (RLw)


Extension of Hydrolight with the pure water model of this project


Temperature


Salinity


Refractive index


Uncertainties


bio
-
optical model:



5 optical components


Absorption coefficients of pigment, detritus, yellow substance


Scattering coefficients of particles and white particles

Inverse NN for atmospheric correction


CC version

Neural Network


18
-
>25x30x40
-
>43

sun zenith

Path radiance reflectance

Ed_surface

RLw

RLtosa

12 bands

MERIS band 1
-
10, 12,13

view x

View y

Input (18)

Output (43)

RLw(

,

)
=Lw (

,

)
/Ed

View z

Tau_aerosol 412, 550, 778, 865

Sun_glint ratio

a_tot, b_tot

temperature

salinity

For each water type

Atmospheric Correction using NN

Scheme for forwardNN based procedure

forNNwater

n_lam

n_var (5)

Select

Bands

Variables

parameters

constraints

forNNatmo

3 par

RLtosa‘

MERIS

RLtoa

Compute

RLtosa

Compare

RLtosa

RLtosa‘

Results:

RLw

IOPs

uncertainty

Start values

Optim,

procedure

RLath,

trans

aaNN

OOS

Test and validation

Test

of NN 17x27x17


560 nm

Training with 5% random noise on rlw

Test of water algorithm forward NN







Relationship between the
measured and derived total
absorption coefficient a_total
at 443 nm using the neural
network algorithm. Red is the
1 by 1 line, blue the
regression. n=498 points

Test scene MERIS MER_RR__1PNPDK20080507

Separation atmosphere / water using inverse NN

MERIS band 5, 560 nm

Reflectance Spectra R_toa, R_path, Rw

Transect Test using forward NN

Forward NN

Inverse NN

Separation test: RLw vs RLpath

MERIS scene of
Hawaii 20040406
TOA radiance
reflectance band
13 with transect

Top of atmosphere, path
radiance and water
leaving radiance
reflectance along transect
of Hawaii scene

Band 5 560 nm


Rlw *10

Comparison as scatter plot

MOBY 73 no glint cases, log10 scale

Neural network AGC /C2R

Standard AC MERIS

Comparison as scatter plot

Neural network AGC /C2R

73 no glint cases

Neural network AGC /C2R

85 high glint cases

Identify spectra, which are out of scope of the training set

Detection of out of scope conditions


2 Procedures have been developed


Combination of an inverse and forward Neural Network


Use of an autoassociative Neural Network


Both produce a reflection spectrum, which is compared with the input
spectrum


Deviation between input and output spectrum can be computed as a chi2


A threshold can be used to trigger an out of scope warning flag

Inverse

NN

IOPs

Forward

NN

R Input

R output

autoassociative

NN

R Input

R output



Combination

of inverse

and forward NN



Auto
-
associative


NN

Detection of out of scope conditions (MERIS processor)

Top of atmosphere

radiance spectra

at normal and

critical locations

Detection of out of scope conditions (MERIS processor)

Exeptional bloom,

Indicated by high Chi_square value

Chi_square is computed by
comparing

The input reflectance spectrum
with the output of the forward
NN

Detection of out of scope conditions using an aaNN


Important to detect toa radiance specta which are not in the simulated
training data set


These are out of scope of the atmospheric correction algorithm


Autoassociative neural network with a bottle neck layer

Input

layer

Hidden 1

Bottle
-

neck

Hidden 3

output

layer

Functions also

as nonlinear PCA

i.e. bottle neck number of
neurons

Provide estimate of

Independent components


For the GAC training data

Set of ~ 1Mio. Cases

Bottleneck minimum was 4
-
5

Detection of out of scope conditions aaNN:

example for L1 (TOA) data

Transect

High
SPM

Sun

glint

Detection of out of scope conditions aaNN: example

significant deviation in area with

high SPM concentrations, but not in

sun glint area

rel. deviation

Histogram of deviations

shows 2 maxima,

around 1 in sun glint

0.9 in high SPM area,

which out of scope

TOSA, Path, Water Reflectance 708 nm

Inverse NN

Sun Glint correction

Sun glint problem: Hawai 20030705

Cross section Hawai scene

radiance_9 [mW/(m^2*sr*nm)]

0

20

40

60

80

100

120

140

160

180

200

-
160

-
158

-
156

-
154

-
152

-
150

-
148

-
146

longitude (deg)

radiance_9 [mW/(m^2*sr*nm)]

No glint and high glint TOA reflectance spectra

Simulated Rayleigh path radiance reflectance and sun glint radiance reflectance

400

450

500

550

600

650

700

750

800

850

900

0

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

wavelength [nm]

Radiance refelctance [sr
-
1
]

nadir view

sun zenith 20 deg

wind 3 m/s

sun glint

Rayleigh path radiance

Specular refleced radiance differences, ratio rel. to Lw

T5/S35

T25/S0

MERIS full resolution: Baltic and North Sea, 20080606

Spatial resolution: 300 m

Swath: 1200 km, 4800 pixel

Oslo

Stockhom

Copenhagen

Hamburg

Sun glint

Sun glint

Baltic Sea

North Sea

Water leaving radiance reflectance

Path radiance reflectance incl. Sun glint band 5 (560 nm)

Water leaving radiance reflectance band 5 (560 nm)

MERIS FR, Area of Gotland, TOA RLw RGB

sunglint

Gotland

Stockholm

Lettland

Ca. 100 km

Baltic Sea

Estonia

Water leaving radiance reflectance RGB

Gotland

Ca. 100 km

New York

MERIS 20070505: TOA reflectances RGB

Path radiance+ Fresnel
reflectance RLpath MERIS
band 5 (560 nm)

Water leaving radiance
reflectance RLw MERIS band 5
(560 nm)

Chlorophyll

MERIS FR USA East Coast 12.6.2008, Signal depth z90

Chesapeake Bay

North Atlantic

Washington

Philadelphia

Overview


Artificial neural networks (NN) can be used for atmospheric correction in
different ways


As a forward model to determine the path radiance and transmittance
as a function of aerosol optical properties, wind (sun glint), angles


As an inverse model, which determines water reflectances,
transmittances, path radiance from TOSA reflectance spectra


NN AC is based on association between water reflectance and top of
atmosphere reflectance


No negative reflectances


The relationship between TOSA reflectance, path radiance, transmittance
and the independent parameters (pressure, aerosols, wind/waves) must be
described with a radiative transfer model


NNs are then trained by a large number of simulated cases (> 1 Mio) by
minimizing the difference between the output of the NN and


For turbid coastal waters reflectance must include reflectance by water


For coastal waters include all bands for atmospheric corrections, no
extrapolation