exploitation of EO data

natureplaygroundΤεχνίτη Νοημοσύνη και Ρομποτική

14 Νοε 2013 (πριν από 3 χρόνια και 8 μήνες)

81 εμφανίσεις

Giancarlo Rivolta

giancarlo.rivolta@esa.int



ESRIN, 11 October 2012

RSS support to scientific
exploitation of EO data

No.
2

©
Logica

2012. All rights reserved

Summary


RSS Overview



RSS G
-
POD process



RSS flexible resources for on
-
demand processing



Success stories



Future directions



GPOD demo



Questions & Answers


©
Logica

2012. All rights reserved

No.
3

Research and Service Support



The ESA Research and Service Support (RSS) service provides resources for supporting Earth
Observation (EO) data exploitation



RSS users are EO Principal Investigators and EO Service Providers



The environments made available to RSS users are:


Grid Processing On
-
Demand (G
-
POD)


Service Support Environment (SSE)


E
-
collaboration environment (Join&Share)


OGC services (WMS, WFS, WCS)


Virtual Reality Theatre (VRT)


Knowledge
-
based Earth Observation (KEO)


Reference Data Sets (RDS)


EO Ontology search engine



More than 3500 people are registered as RSS users



On the industry side Logica delivers the RSS service as part of the ESRIN O&M Framecontract




More information available from the RSS Portal (
rssportal.esa.int

)


Let’s focus on G
-
POD

RSS Overview

©
Logica

2012. All rights reserved

No.
4

Research Process

©
Logica

2012. All rights reserved

No.
5


Principal Investigators



EO algorithms delivery


Data type and range indication












Output validation



On
-
demand EO data processing



Use of produced data (projects
delivery, publications,
etc
)



RSS






Data are made available in the RSS
catalogue



Algorithm porting



Integration design



Implementation



Test and validation (involving the PI)



On
-
demand EO data processing



Delivery



RSS G
-
POD process steps

©
Logica

2012. All rights reserved

No.
6

Flexible Infrastructure satisfies:

-
HW requirements

-
Connectivity requirements

-
SLA (HA, help desk, ticketing systems, etc.)

On
-
demand processing service:








Technology

Infrastructure

Application

ESRIN

-

172 cores

-

400 TB

UK
-
PAC

-

88 cores

-

300 TB

Flexible Infrastructure

-

10
-
200 cores

-

1
-
10 TB



EO Scientists

Principal
Investigators



delivery

Process

EO data

RSS Flexible Resources

©
Logica

2012. All rights reserved

No.
7

RSS G
-
POD Team

On
-
demand Processing: team composition and responsibility



The G
-
POD team is composed by 4 selected highly skilled resources



Key areas of responsibility are:



Interaction with Principal Investigators


Algorithm Understanding and proposal evaluation


Resources planning (data, storage, integration, capacity,
etc
)


Algorithm integration (porting, design, implementation, test)


Data management


Processing environment set
-
up


User support (for on
-
demand processing)


Bulk Processing management



Key skills are:



Remote sensing background / EO data exploitation expertise


Software development /software integration


Virtualization technology


Grid / Cloud technology


Operations and Maintenance


Scientific goal orientation


©
Logica

2012. All rights reserved

No.
8

Success Story: MGVI

MERIS Global Vegetation Index Cat
-
1 project



Objective:

Development of MERIS derived products based on the Fraction of Absorbed
Photosynthetically

Active Radiation (FAPAR) algorithm



IPR:
MGVI algorithm developed and owned by the Joint Research Centre



Evolution and time span:
Several GPOD
-
MGVI versions integrated since 2007, bulk re
-
processings

over the entire ENVISAT mission (2002
-
2012)



Outcomes/Benefits:


Access to online data for bulk re
-
processing


Systematic global products published on ESA
Earthnet

Portal


Regional products distributed via RSS SSE and RSS OGC service as well


On
-
demand MGVI regional available


MGVI Europe / Africa distributed to JRC for creating products for desertification
monitoring


Several scientific papers published


MGVI delivered as complementary information to Fire detection products over
Senegal


From JRC contribution to the paper “A Model for the Scientific Exploitation of Earth Observation Missions”:

RSS supported the process of algorithm development, test and deployment over a general purpose processing on demand
environment dedicated to the scientific exploitation of EO data generated by ESA missions, as well as the provision of tools
and

environments in support to that process.


©
Logica

2012. All rights reserved

No.
9

Success Story: KLIMA
-
IASI

The KLIMA
-
IASI ESA Cat
-
1 project was initially structured along 3 phases

1
-

The KLIMA
-
IASI application is integrated in G
-
POD


The data set used consists of: IASI data levels L1B, L1C and L2, in EPS format; for the month of July 2008;


The results of the application on this data set are used to determine if results produced from Level L1B data are better than

th
e
results produced from Level L1C;


The phase is concluded with the decision to use Level L1B or Level L1C data;


Also, during this phase, processing performance is measured and the processing time for the qualification campaign is estimat
ed

2
-

IASI Level L1 (B or C) and L2 are ingested for the whole year 2009, in EPS format, up to 5Tb


As a "qualification campaign", the KLIMA
-
IASI application is run upon the year 2009 data and results are used to validate the
quality of the algorithm in relation to
Gosat

measurements;


Also, during this phase processing performance is measured and the exploitation scenario defined

3
-

Full IASI Mission data and
EumetCast

NRT are systematically ingested in G
-
POD


The KLIMA
-
IASI application performance is optimized in line with NRT requirements;


The KLIMA
-
IASI application is enhanced to accept BUFR data format;


KLIMA
-
IASI application is provided as a service to the User Community



IPR:
KLIMA
-
IASI algorithm developed and owned by the IFAC
-
CNR, Florence



Evolution and time span:
Several GPOD
-
MGVI versions integrated since 2011, bulk processing
on dataset from 1 year IASI data (Mar 2010


Feb 2011)



Outcomes/Benefits:


On
-
demand KLIMA
-
IASI service available to PI for pre
-
analysis


Access to cloud resources for flexible capacity to meet the deadline


Access to online data for bulk re
-
processing (11 out of 12 months processed)


Several scientific papers published and presentations at conferences


Process speeded
-
up by 2 orders of magnitude (flexible)



©
Logica

2012. All rights reserved

No.
10

Success Story: SMOS

SMOS
testbed




Objective:
Provide a flexible test environment to support ESA GQ users for L1 calibration and
developers for L2 OS



Evolution and time span:
Several SMOS L1 and L2 OS re
-
processing campaigns with custom
configuration (recently processed sample SMOS L1 products over 25 months (from June 2010 to
June 2012, 1728 L1A/B + 6576 CRS products), SMOS L2 OS 1 month)



Outcomes/Benefits:



Fast integration of new versions (if ICD unchanged)


Access to online data for bulk re
-
processing


Access to flexible cloud resources for meeting deadlines


On
-
demand SMOS L1 available for GQ


On
-
demand SMOS L2 OS available for developer


SMOS L0 NRT ingestion chain set
-
up for L1 NRT custom re
-
processing


Testbeds

required for other sensors as well




Open call for proposal published on EOPI

©
Logica

2012. All rights reserved

No.
11

Future directions


In the light of the RSS success story the following should be
considered



RSS model as basis for the definition of a harmonized
model of research and service support



Federation of payload data ground segment services to
the
scientific community

©
Logica

2012. All rights reserved

No.
12

Contacts & further information

Giancarlo Rivolta

giancarlo.rivolta@esa.int

RSS Team


rss_team@esa.int

GPOD Team


eo
-
gpod@esa.int

G
-
POD Portal


gpod.eo.esa.int

Join&Share


wiki.services.eoportal.org




©
Logica

2012. All rights reserved

No.
13

GPOD demo

Now the GPOD Team will give a 15
-
minute GPOD demo

GPOD Portal:
gpod.eo.esa.int


©
Logica

2012. All rights reserved

No.
14

Questions and Answers






Q&A?

Logica is now part of CGI, a leading business and technology services company focused on helping clients achieve results. Sin
ce
our founding in 1976, we've
operated upon the principles of sharing in clients' challenges and delivering quality services to address them. With 72,000 p
rof
essionals in more than 40
countries, we have the presence, expertise and complete IT services to meet clients’ business needs anywhere, anytime. We pro
vid
e the responsiveness and
accountability of a true local partner while offering the global scale, talent and services needed to meet your evolving need
s.
Visit
www.cgi.com

and
www.logica.com

to learn more.

Thank you

Giancarlo Rivolta

Logica xxx Visitor address Street Postcode City Country www.logica.com

Contact: Name Function T: +XX (0) 000 00 000 000 F: +XX (0) 000 00 000 000 E: address@logica.com