CSA-CCRS_Session3_LTDP_Workshop_combined_final_v3 ... - ESA

arghtalentΔιαχείριση Δεδομένων

31 Ιαν 2013 (πριν από 4 χρόνια και 8 μήνες)

140 εμφανίσεις

CSA & CCRS LTDP Experience and
Approach


2
nd

Long Term Data Preservation Workshop

ESA/ESRIN, 30
-
31 May 2011


Yanmei Wu (CCRS) and Liang Ma (CSA)

2

CCRS introduction

Canada

Centre

for

Remote

Sensing

(CCRS)

ensures

that

satellite

data

is

accessible

and

usable

to

serve

the

Canadians

need

for

Natural

Resources

and

Landmass

Knowledge
.

Geodetic
:

spatial

referencing

information,

expertise,

and

services

.


Data

Acquisition

Division

(DAD)
:

reception,

processing,

archiving

and

dissemination

of

EO

data

in

Canada
;


Earth

Observation

Geosolution

Division

(EOGD)
:

R&D

supports

EO

satellite

sensors,

improves

EO

data

quality

and

data

handling

methodologies,

and

develops

value
-
added

EO
-
based

information,

applications

and

services
.





3

CSA
introduction

Established

in

March

1989
,

the

Canadian

Space

Agency

(CSA)

was

created

through

an

Act

of

Parliament
.

The

Agency

has

a

status

equivalent

to

that

of

a

Department

of

the

Government

of

Canada

and

reports

to

Parliament

through

the

Minister

of

Industry
.

The CSA has aligned its Programs and activities with
the Science and Technology Strategy and
reorganized its resources and activities through
four key Programs: Earth Observation, Space
Science and Exploration, Satellite
Communications, Space Awareness and Learning
since April 2011.





4

Mission and archive status(1/2)

Missions in operation


RADARSAT
-
1, launched 1995, archived in CCRS


RADARSAT
-
2, launched 2007, archived in CCRS


SCISAT, launched 2003, raw data in Mission Operations
Centre (MOC), CSA and higher levels archived in SOC
(Science Operations Centre) , U of Waterloo. About 10TB
in raw and 50TB in products. Public access (registered) to
Level 2, intranet access to Level 1, and access to raw is in
plan.

Missions in development


RADARSAT Constellation (RCM),
Routine generation of
data/products placed “immediately” in archive/catalogue
and accessible on
-
line and free of charge (government
and non
-
commercial unclassified data).


CASSIOPE, will be archived in its SOC in U of Calgary.


5

Mission and Archive status (2/2)


Current

Status

of

CCRS

archive

volume

(
AS

of

Feb

2011
,

~
800

TB
)


ERS
-
1

and

2

SAR

Image

mode
:

147

TB

(on
-
line

42
TB)


ENVISAT

ASAR
:

6
TB(on
-
line)


Radarsat
-
1

SAR
:

300

TB

(
100
TB

on
-
line,

transcription

on

going)


Radarsat
-
2

SAR
:

60

TB

(on
-
line)


Landsat

1
-
5

MSS
:

16
.
73

TB

(on
-
line

at

the

end

of

August)


Landsat

4
,
5

TM
:

121

TB

(on
-
line)


Landsat

ETM+
:

62

TB

(on
-
line)


ENVISAT

MERIS
:

40
TB

(on
-
line)


NOAA

8
-
19

2
.
5

(on
-
line)


RS
-
1

(
2
.
8
TB),

RS
-
2

(
15
.
5
TB),

MERIS(
16
TB),

MNSO

products


Trends
:

on
-
line

archive,

free

access

(eg
.
,NOAA

-
19
)




6

LTDP approach and activities (1/3)


CSA

is

responsible

for

the

overall

preservation

of

CSA

SAR

and

Space

Science

Missions

data
.

As

part

of

its

mandate,

the

CSA

shall

strive

to

make

the

data

accessible

to

Canadians

as

broadly

as

possible

taking

into

consideration

data

policies

relevant

to

each

category

of

data
.

However,

the

organization

responsible

for

the

maintenance

of

the

archive

may

not

be

the

CSA
.



CCRS

is

responsible

to



promote

the

development

and

use

of

remote

sensing

technology”

(Department

of

Natural

Resources

Act

1995

c
.
41
)
.

CCRS

supports

Canadian

missions

as

well

as

other

international

missions

that

meet

Canadian

user

needs,

providing

archive,

access,

and

long

term

preservation

services
.


7

LTDP approach and activities (2/3)


Consolidated

on
-
line

archive


Archive

security
:

controlled

access,

encrypted

data

transfer

(data,

network),

controlled

environment,

off

site

copy


Automated

data

ingest

from

data

capture

to

product

available

via

FTP

and

on
-
line



Leveraging

new

technologies

for

data

preservation


Service

Oriented

archive

operations



To

promote

the

use

of

archived

EO

data


To

provide

archive

service


8

LTDP approach and activities (3/3)


On
-
going

LTDP

related

activities


RS
-
1

transcription

project


RS
-
1

migration

from

portable

stations

and

network

stations


Landsat

Data

repatriation

(~
200

TB),

accessible

through

USGS

portal


Landsat

1
-
5

MSS
:

16
.
73

TB


Landsat

4
,
5

TM
:

121

TB


Landsat

ETM+
:

62

TB


To

develop

strategy

for

other

mission

data


To

promote

access

to

archive

data


9

CCRS Archive Architectures (1/2)


Architecture

and

Technology


In
-
house

designed

solution,

100
%

COTS,

multi
-
vendor,

open

standard

interface,

easy

future

migration


HSM

based

archive/storage

management

system

(SAM
-
QFS),

Equalogic

RAIDS,

Robotic

tape

library,

LTO
4

types


Configurable

water

mark

(

data

type,

client)

for

access


Front

end

support

RT/NRT

access


Back

end

support

historical

data

access


GB

network

from

receiving

stations

to

archive

center


Operational

aspects
:

Fault
-
tolerance,

light
-
out,

automated

operation


Leveraging

Open

Source

software

(PostgreSQL,

PostGIS,

Condor,

FTP


)

10

SCISAT Archive Architectures



Archiving

Centers

Raw

in

MOC,

CSA

and

products

in

SOC,

University

of

Waterloo




Data

Formats

CCSDS

packet

binary

(level

0
),

ABB
-
Bomen

proprietary

binary

level

1
,

and

ASCII

for

level

2
.



Technology

Used

Solaris,

ZFS,

RAID
5
,

FC,

NFS,

PostgreSQL
.




Data

Access

Public

web

interface

available

for

higher

level

products,

web

and

database

available

in

SOC

for

level

1
b,

raw

access

in

development

in

CSA
.




Architectures




11

Content of SCISAT Data Archive



Two instruments
A high spectral resolution Fourier Transform
Spectrometer (FTS) and a small spectrophotometer MAESTRO
(Measurement of Aerosol Extinction in the Stratosphere and Troposphere
Retrieved by Occultation)



The archive contains



Level 0 CCSDS packets with communication headers and transfer
frames removed



Level 1B
The solar spectra modified by the passage of sunlight
through the Earth's atmosphere.



Level 2 Images,
temperature, pressure and Volume Mixing Ratio
profiles as a function of altitude for the following trace gas molecules
H
2
O, O
3
, N
2
O, CO, CH
4
, NO, NO
2
, HNO
3
, HF, HCl, N
2
O
5
, ClONO
2
, CCl
2
F
2
,
CCl
3
F, COF
2
, CHF
2
Cl, HDO, SF
6
, OCS, HCN, CF
4
, CH
3
Cl, C
2
H
2
, C
2
H
6
, N
2
,
ClO, etc from FTS and NO
2

and O
3

profiles from MAESTRO.



Auxiliary

and

ancillary

Orbit

data,

time

bias

and

bus

telemetry



Documentation

All

available

mission

documentation




Software

Programs,

scripts

needed

to

generate

Level

1

and

Level

2

data



12

Content of data archive (1/1)




Preservation

Period

base

on

agreements,

MOUs


RS
-
1

and

RS
-
2
:

maintain

data

archives

for

a

period

of

15

years

following

the

cessation

of

spacecraft

operations
.



Archive

in

terms

of

Primary

data


Raw/level
0

data



Higher

level

products

(MERIS,

RS
-
1
,

RS
-
2
,

NMSO)


secondary

data,

documents,

etc


Working

on

to

include

Auxiliary,

ancillary

data

and

Documentation

into

the

scope

of

LTDP




13

European LTDP Guidelines Impact Analysis


Positive impact due to shared vision of Long
Term Data Preservation:


Operation concept


Evaluation of preservation dataset


Technical implementation options


Choice of data and metadata format


Interoperability & partnership with ESA


14

LTDP European Framework contribution and
participation


CSA, with the support and service provided by
CCRS, is
interested in discussion of contribution
and participation to the set
-
up of a European LTDP
Framework.


To evolve and develop the archive policies more
compatible with LTDP Common Guidelines.


To cooperate in standardization activities.


To study and test archive and data access
methodology and technology, and share the
knowledge and experience with other participants.


To support the creation of an interoperable network
of archive centers.

15




Thank you!


Questions?