Large-scale Integrated Project (IP)

idiotcanvasΑσφάλεια

17 Νοε 2013 (πριν από 3 χρόνια και 8 μήνες)

651 εμφανίσεις





Private Public Partnership Project (PPP)

Large
-
scale Integrated Project (IP)




D.6.1.1b: FI
-
WARE GE Open Specifications


(Data/Context Management Chapter)



Project acronym:

FI
-
WARE

Project full title:

Future Internet Core Platform

Contract No.:

285248

Strategic Objective:

FI.ICT
-
2011.1.7 Technology foundation: Future Internet Core
Platform

Project Document Number:

ICT
-
2011
-
FI
-
285248
-
WP6
-
D.6.1.1b

Project Document Date:

2
012
-
10
-
11

Deliverable Type and Security:

Public

Author:

FI
-
WARE Consortium

Contributors:

FI
-
WARE Consortium

Future Internet Core Platform



D.6.1.1b FI
-
WARE GE Open Specifications


Page
2





Table of Content


1

Introduction

................................
................................
................................
.....................
14

1.1

Executive Summary

................................
................................
................................
.
14

1.2

About This Document

................................
................................
...............................
14

1.3

Intended Audience

................................
................................
................................
...
14

1.4

Chapter Context

................................
................................
................................
.......
14

1.5

Structure of this Document

................................
................................
.......................
16

1.6

Typographical Conventions

................................
................................
......................
17

1.6.1

Links within this document

................................
................................
.................
17

1.6.2

Figures

................................
................................
................................
..............
18

1.6.3

Sample software code

................................
................................
.......................
18

1.7

Acknowledgements

................................
................................
................................
..
18

1.8

Keyword list

................................
................................
................................
..............
18

1.9

Changes History

................................
................................
................................
......
18

2

FIWARE OpenSpecification Data BigData

................................
................................
......
19

2.1

Preface

................................
................................
................................
....................
19

2.1.1

Copyright

................................
................................
................................
...........
19

2.1.2

Legal Notice

................................
................................
................................
......
19

2.2

Overview

................................
................................
................................
..................
19

2.2.1

Introduction to the BigData Analysis GE

................................
............................
19

2.2.2

Target Usage

................................
................................
................................
.....
20

2.2.3

Example Scenario

................................
................................
.............................
21

2.3

Basic Concepts

................................
................................
................................
........
22

2.3.1

MapReduce

................................
................................
................................
.......
22

2.3.2

NoSQL
................................
................................
................................
...............
25

2.4

Big Data Analysis Generic Architecture

................................
................................
....
26

2.5

Main Interactions

................................
................................
................................
......
29

2.5.1

Modules and Interfaces

................................
................................
.....................
29

2.5.2

Data Stream Interfaces

................................
................................
......................
29

2.5.3

Analysis Module Programming

................................
................................
..........
30

2.5.4

Analytical scripting

................................
................................
.............................
32

2.6

Basic Design Principles

................................
................................
............................
34

Future Internet Core Platform



D.6.1.1b FI
-
WARE GE Open Specifications


Page
3




2.7

References
................................
................................
................................
...............
35

2.8

Detailed Specifications

................................
................................
.............................
35

2.8.1

Open API Specifications

................................
................................
....................
35

2.9

Terms and definitions

................................
................................
...............................
35

3

BigData Analysis Open RESTful API Specification (PRELIMINARY)

..............................
38

3.1

Introduction to the
BigData GE RESTful

API

................................
............................
38

3.1.1

BigData GE RESTful API Core

................................
................................
..........
38

3.1.2

Intended Audience

................................
................................
.............................
38

3.1.3

API Change History

................................
................................
...........................
38

3.1.4

How to Read This Document

................................
................................
.............
39

3.1.5

Additional Resources

................................
................................
.........................
39

3.2

General
BigData GE RESTful

API Information

................................
.........................
40

3.2.1

Resources Summary

................................
................................
.........................
40

3.2.2

Authentication

................................
................................
................................
....
40

3.2.3

Representation Format

................................
................................
......................
40

3.2.4

Representation Transport

................................
................................
..................
40

3.2.5

Resource Identification

................................
................................
......................
41

3.2.6

Links and References

................................
................................
........................
41

3.2.7

Limits

................................
................................
................................
.................
41

3.2.8

Versions

................................
................................
................................
............
41

3.2.9

Faults

................................
................................
................................
................
41

3.3

API Operations

................................
................................
................................
.........
42

3.3.1

Platform

................................
................................
................................
.............
42

3.3.2

Queues

................................
................................
................................
..............
52

3.3.3

Modules

................................
................................
................................
.............
57

3.3.4

Operations

................................
................................
................................
.........
61

4

FIWARE OpenSpecification Data CEP

................................
................................
...........
66

4.1

Preface

................................
................................
................................
....................
66

4.1.1

Copyright

................................
................................
................................
...........
66

4.1.2

Legal Notice

................................
................................
................................
......
66

4.2

Overview

................................
................................
................................
..................
66

4.2.1

Introduction to the CEP GE

................................
................................
................
66

4.2.2

Target Usage

................................
................................
................................
.....
69

4.3

Basic Concepts

................................
................................
................................
........
70

4.3.1

Adapters design principles

................................
................................
.................
72

4.3.2

Adapters design

................................
................................
................................
.
73

Future Internet Core Platform



D.6.1.1b FI
-
WARE GE Open Specifications


Page
4




4.4

Main Interactions

................................
................................
................................
......
74

4.4.1

Definition of Input Adapters

................................
................................
................
74

4.4.2

Definition of Ouput Adapters

................................
................................
..............
80

4.4.3

Definition of CEP Application

................................
................................
.............
84

4.5

Basic Design Principles

................................
................................
............................
85

4.6

References
................................
................................
................................
...............
86

4.7

Detailed Specifications

................................
................................
.............................
86

4.7.1

Open API Specifications

................................
................................
....................
86

4.8

Re
-
utilised Technologies/Specifications

................................
................................
...
86

4.9

Terms and definitions

................................
................................
...............................
86

5

Complex Event Processing Open RESTful API Specification (PRELIMINARY)

..............
88

5.1

Introduction to the CEP GE REST API

................................
................................
.....
88

5.2

Introduction to the
CEP

API

................................
................................
.....................
88

5.2.1

CEP API

................................
................................
................................
............
88

5.2.2

Intended Audience

................................
................................
.............................
88

5.2.3

API Change History

................................
................................
...........................
88

5.2.4

How to Read This Document

................................
................................
.............
89

5.2.5

Additional Resources

................................
................................
.........................
89

5.3

General CEP API Information

................................
................................
..................
89

5.3.1

Resources Summary

................................
................................
.........................
89

5.3.2

Representation Format

................................
................................
......................
90

5.3.3

Representation Transport

................................
................................
..................
90

5.4

API Operations

................................
................................
................................
.........
90

5.4.1

Getting Events API

................................
................................
............................
90

5.4.2

Sending Events API

................................
................................
...........................
91

6

FIWARE OpenSpecification Data CompressedDomainVideoAnalysis

............................
92

6.1

Preface

................................
................................
................................
....................
92

6.1.1

Copyright

................................
................................
................................
...........
92

6.1.2

Legal Notice

................................
................................
................................
......
92

6.2

Overview

................................
................................
................................
..................
92

6.2.1

Introduction to the Compressed Domain Video Analysis GE

..............................
92

6.2.2

Target Usage

................................
................................
................................
.....
93

6.3

Basic Concepts

................................
................................
................................
........
93

6.3.1

Block
-
Based Hybrid Video Coding

................................
................................
.....
93

6.3.2

Compressed Domain Video Analysis

................................
................................
.
95

6.4

Architecture

................................
................................
................................
..............
96

Future Internet Core Platform



D.6.1.1b FI
-
WARE GE Open Specifications


Page
5




6.4.1

Media Interface

................................
................................
................................
..
97

6.4.2

Media (Stream) Analysis

................................
................................
....................
98

6.4.3

Metadata Interface

................................
................................
.............................
99

6.4.4

Control Interface

................................
................................
................................
99

6.5

Main Interactions

................................
................................
................................
......
99

6.5.1

Interfaces

................................
................................
................................
...........
99

6.5.2

Operations

................................
................................
................................
.......

102

6.6

Basic Design Principles

................................
................................
..........................

104

6.7

References
................................
................................
................................
.............

105

6.8

Detailed Specifications

................................
................................
...........................

105

6.8.1

Open API Specifications

................................
................................
..................

105

6.9

Re
-
utilised Technologies/Specifications

................................
................................
.

106

6.10

Terms and definitions

................................
................................
...........................

106

7

Compressed Domain Video Analysis Open RESTful API S
pecification (PRELIMINARY)

108

7.1

Introduction to the Compressed Domain Video Analysis GE API

...........................

108

7.1.1

Compressed Domain Video Analysis GE API Core
................................
..........

108

7.1.2

Intended Audience

................................
................................
...........................

108

7.1.3

API Change History

................................
................................
.........................

108

7.1.4

How to Read This Document

................................
................................
...........

109

7.1.5

Additional Resources

................................
................................
.......................

109

7.2

General Compressed Domain Video Analysis GE API Information

........................

109

7.2.1

Resources Summary

................................
................................
.......................

109

7.2.2

Representation Format

................................
................................
....................

110

7.2.3

Resource Identification

................................
................................
....................

110

7.2.4

Links and References

................................
................................
......................

110

7.2.5

Limits

................................
................................
................................
...............

110

7.2.6

Versions

................................
................................
................................
..........

110

7.2.7

Extensions

................................
................................
................................
.......

111

7.2.8

Faults

................................
................................
................................
..............

111

7.3

API Operations

................................
................................
................................
.......

112

7.3.1

/version

................................
................................
................................
............

112

7.3.2

/instances

................................
................................
................................
........

112

7.3.3

/{instanceID}

................................
................................
................................
....

114

7.3.4

/config

................................
................................
................................
..............

116

7.3.5

/sinks

................................
................................
................................
...............

118

Future Internet Core Platform



D.6.1.1b FI
-
WARE GE Open Specifications


Page
6




7.3.6

/{sinkID}

................................
................................
................................
...........

119

7.3.7

//{sinkNotificationURI}

................................
................................
......................

121

8

FIWARE OpenSpecification Data Location

................................
................................
...

123

8.1

Preface

................................
................................
................................
..................

123

8.1.1

Copyright

................................
................................
................................
.........

123

8.1.2

Legal Notice

................................
................................
................................
....

123

8.2

Overview

................................
................................
................................
................

123

8.2.1

Introduction to the Data Location GE

................................
...............................

123

8.2.2

Target usage

................................
................................
................................
...

124

8.3

Basic Concepts

................................
................................
................................
......

124

8.3.1

Third
-
party location services

................................
................................
............

124

8.3.2

Access control and privacy management

................................
.........................

125

8.3.3

Mobile end
-
user services

................................
................................
.................

125

8.3.4

Interfaces and data model

................................
................................
...............

126

8.4

Main Interactions

................................
................................
................................
....

127

8.4.1

MLP services

................................
................................
................................
...

127

8.4.2

NetAPI Terminal Location services

................................
................................
..

132

8.4.3

Positioning

................................
................................
................................
.......

136

8.5

Basic Design Principles

................................
................................
..........................

137

8.6

References
................................
................................
................................
.............

137

8.7

Detailed Specifications

................................
................................
...........................

138

8.7.1

Open API Specifications

................................
................................
..................

138

8.8

Terms and definitions

................................
................................
.............................

138

9

Location Server Open RESTful API Specification (PRELIMINARY)

..............................

140

9.1

Dedicated API Introduction

................................
................................
.....................

140

9.2

Introduction to the Restful Network API for Terminal Location

................................

140

9.2.1

Network API for Terminal Location

................................
................................
..

140

9.2.2

Intended Audience

................................
................................
...........................

140

9.2.3

API Change History

................................
................................
.........................

140

9.2.4

How to Read This Document

................................
................................
...........

141

9.2.5

Additional Resources

................................
................................
.......................

141

9.3

General Location Server REST API Information

................................
.....................

141

9.3.1

Resources Summary

................................
................................
.......................

141

9.3.2

Authentication

................................
................................
................................
..

142

9.3.3

Representation Format

................................
................................
....................

142

9.3.4

Representation Transport

................................
................................
................

143

Future Internet Core Platform



D.6.1.1b FI
-
WARE GE Open Specifications


Page
7




9.3.5

Resource Identification

................................
................................
....................

143

9.3.6

Links and References

................................
................................
......................

143

9.3.7

Limits

................................
................................
................................
...............

143

9.3.8

Versions

................................
................................
................................
..........

143

9.3.9

Faults

................................
................................
................................
..............

143

9.4

Data Types
................................
................................
................................
.............

144

9.4.1

XML NameSpaces

................................
................................
...........................

144

9.4.2

Requester

................................
................................
................................
........

144

9.4.3

Structures

................................
................................
................................
........

144

9.5

API Operations

................................
................................
................................
.......

149

9.5.1

Location Query

................................
................................
................................

149

9.5.2

Periodic Notification Subscription

................................
................................
.....

154

9.5.3

Area (Circle) Notification Subscription
................................
..............................

161

10

FIWARE OpenSpecification Data MetadataPrepro
cessing

................................
...........

169

10.1

Preface

................................
................................
................................
................

169

10.1.1

Copyright

................................
................................
................................
.......

169

10.1.2

Legal Notice
................................
................................
................................
...

169

10.2

Overview

................................
................................
................................
..............

169

10.2.1

Target usage

................................
................................
................................
.

169

10.2.2

Example scenarios and main services exported

................................
............

170

10.3

Basic Concepts

................................
................................
................................
....

170

10.3.1

Functional components of the Metadata Preprocessing GE

...........................

170

10.3.2

Realization by MetadataProcessor asset

................................
.......................

171

10.4

Main Interactions

................................
................................
................................
..

171

10.5

Basic Design Principles

................................
................................
........................

173

10.6

References

................................
................................
................................
...........

174

10.7

Detailed Specifications

................................
................................
.........................

174

10.7.1

Open API Specifications

................................
................................
................

174

10.8

Re
-
utilised Technologies/Specifications

................................
...............................

174

10.9

Terms and definitions

................................
................................
...........................

175

11

Metadata Preprocessing Open REST
ful API Specification (PRELIMINARY)

................

177

11.1

Introduction to the Metadata Preprocessing GE API

................................
............

177

11.1.1

Metadata Preprocessing GE API Core

................................
...........................

177

11.1.2

Intended Audience

................................
................................
.........................

177

11.1.3

API Change History

................................
................................
.......................

177

11.1.4

How to Read This Document

................................
................................
.........

178

Future Internet Core Platform



D.6.1.1b FI
-
WARE GE Open Specifications


Page
8




11.1.5

Additional Resources

................................
................................
.....................

178

11.2

General Metadata Preprocessing GE API Information
................................
..........

178

11.2.1

Resources Summary

................................
................................
.....................

178

11.2.2

Authentication

................................
................................
................................

179

11.2.3

Representation Format

................................
................................
..................

179

11.2.4

Representation Transport

................................
................................
..............

179

11.2.5

Resource Identification

................................
................................
..................

179

11.2.6

Links and References

................................
................................
....................

180

11.2.7

Limits

................................
................................
................................
.............

1
80

11.2.8

Versions

................................
................................
................................
........

180

11.2.9

Extensions

................................
................................
................................
.....

180

11.2.10

Faults

................................
................................
................................
...........

180

11.3

API Operations

................................
................................
................................
.....

181

11.3.1

Version

................................
................................
................................
..........

181

11.3.2

Management of instances

................................
................................
..............

181

11.3.3

Configuration of Instances

................................
................................
.............

183

12

FIWARE OpenSpecification Data PubSub Context Broker

................................
...........

188

12.1

Preface

................................
................................
................................
................

188

12.1.1

Copyright

................................
................................
................................
.......

188

12.1.2

Legal Notice
................................
................................
................................
...

188

12.2

Overview

................................
................................
................................
..............

188

12.2.1

Introduction to the (Publish/Subscribe) Context Broker GE

............................

188

12.2.2

Target usage

................................
................................
................................
.

189

12.2.3

Example Scenarios

................................
................................
........................

189

12.3

Basic Concepts

................................
................................
................................
....

191

12.3.1

Context Elements

................................
................................
..........................

191

12.3.2

Basic Entities of the GE Model
................................
................................
.......

192

12.3.3

Features and Functionalities

................................
................................
..........

194

12.3.4

Fi
-
WARE NGSI Specification

................................
................................
.........

195

12.4

Main Interactions using the FI
-
WARE NGSI Restful API

................................
......

195

12.4.1

OMA NGSI Basics

................................
................................
.........................

195

12.4.2

Basic Interactions and related Entities

................................
...........................

195

12.4.3

Registration of quer
y
-
able Context Producers (Context Providers)

................

196

12.4.4

Interactions to subscribe Context Consumers to specific notifications

............

197

12.4.5

Extended Operations: Registering Entities & Attributes availability

................

197

Future Internet Core Platform



D.6.1.1b FI
-
WARE GE Open Specifications


Page
9




12.4.6

Extended Operations: Applications subscription to Entities/Attributes registration

198

12.
5

Main interactions using ContextML/CQL

................................
..............................

198

12.5.2

ContextML API

................................
................................
..............................

201

1
2.5.3

ContextQL (CQL)

................................
................................
...........................

201

12.5.4

CQL API

................................
................................
................................
........

204

12.6

Basic Design
Principles

................................
................................
........................

204

12.6.1

Conceptual Decoupling

................................
................................
..................

204

12.7

References

................................
................................
................................
...........

205

12.8

Detailed Specifications

................................
................................
.........................

206

12.8.1

Open API Specifications

................................
................................
................

206

12.8.2

Other Specifications

................................
................................
.......................

206

12.9

Re
-
utilised
Technologies/Specifications

................................
...............................

206

12.10

Terms and definitions

................................
................................
.........................

207

13

FI
-
WARE NGSI Open RESTful API Specification (PRELIMINARY)

..............................

209

14

FI
-
WARE NGSI
-
9 Open RESTful API Specification (PRELIMINARY)

...........................

210

14.1

Introduction to the FI
-
WARE NGSI
-
9 API

................................
.............................

210

14.1.1

FI
-
WARE NGSI
-
9 API Core

................................
................................
...........

210

14.1.2

Intended Audience

................................
................................
.........................

210

14.1.3

Change history

................................
................................
..............................

210

14.1.4

Additional Resources

................................
................................
.....................

211

14.1.5

Legal Notice
................................
................................
................................
...

211

15

FI
-
WARE NGSI
-
9 Open RESTful API Specification (PRELIMINARY)

...........................

212

15.1

Introduction to the FI
-
WARE NGSI
-
9 API

................................
.............................

212

15.1.1

FI
-
WARE NGSI
-
9 API Core

................................
................................
...........

212

15.1.2

Intended Audience

................................
................................
.........................

212

15.1.3

Change history

................................
................................
..............................

212

15.1.4

Additional Resources

................................
................................
.....................

213

15.1.5

Legal Notice
................................
................................
................................
...

213

15.2

General NGSI
-
9 API information

................................
................................
..........

214

15.2.1

Resources Summary

................................
................................
.....................

214

15.2.2

Representation Format

................................
................................
..................

214

15.2.3

Representation Transport

................................
................................
..............

215

15.2.4

API Operations on Context Management Component

................................
...

215

15.2.5

API operation on Context Consume
r Component

................................
..........

218

16

FI
-
WARE NGSI
-
10 Open RESTful API Specification (PRELIMINARY)

.........................

219

Future Internet Core Platform



D.6.1.1b FI
-
WARE GE Open Specifications


Page
10




16.1

Introduction to the FI
-
WARE NGSI 10 API

................................
...........................

219

16.1.1

FI
-
WARE NGSI 10 API Core

................................
................................
.........

219

16.1.2

Intended Audience

................................
................................
.........................

219

16.1.3

Change history

................................
................................
..............................

219

16.1.4

Additional Resources

................................
................................
.....................

220

16.2

General NGSI 10 API information

................................
................................
........

220

16.2.1

Resources Summary

................................
................................
.....................

220

16.2.2

Representation Format

................................
................................
..................

221

16.2.3

Representation Transport

................................
................................
..............

221

16.2.4

API Operations on Context Management Component

................................
...

221

16.2.5

API operation on Context
Consumer Component

................................
..........

224

17

ContextML API

................................
................................
................................
.............

225

17.1

Using ContextML to
interact with the Publish/Subscribe GE

................................
.

225

17.2

ContextML Basics

................................
................................
................................

225

17.2.1

Context Data

................................
................................
................................
..

226

17.2.2

ContextML Naming Conventions

................................
................................
...

227

17.
3

ContextML API

................................
................................
................................
.....

228

17.3.1

Announcement of a Context Provider: providerAdvertising method

...............

228

17.3.2

Description of Context Providers: getContextProviders method

.....................

230

17.3.3

List of Available Context Scopes: getAvailableAtomicScopes method

...........

230

17.3.4

Context Update

................................
................................
..............................

231

17.3.5

Get context

................................
................................
................................
....

232

18

CQL API

................................
................................
................................
.......................

234

18.1

ContextQL (CQL)

................................
................................
................................
.

23
4

18.1.1

Context Query

................................
................................
...............................

234

18.2

CQL API

................................
................................
................................
...............

237

18.2.1

Examples of Context Queries

................................
................................
........

237

19

FIWARE OpenSpecification Data QueryBroker

................................
............................

242

19.1

Preface

................................
................................
................................
................

242

19.1.1

Copyright

................................
................................
................................
.......

242

19.1.2

Legal Notice
................................
................................
................................
...

242

19.2

Overview

................................
................................
................................
..............

242

19.2.1

Introduction to the Media
-
enhanced Query Broker GE

................................
...

242

19.2.2

Target usage

................................
................................
................................
.

243

19.2.3

Example Scenario
................................
................................
..........................

243

19.3

Basic Concepts

................................
................................
................................
....

245

Future Internet Core Platform



D.6.1.1b FI
-
WARE GE Open Specifications


Page
11




19.3.1

Design Principles

................................
................................
...........................

245

19.3.2

Query Processing Strategies

................................
................................
.........

246

19.3.3

MPEG Query Format (MPQF)
................................
................................
........

247

19.3.4

Federated Query Evaluation Workflow

................................
...........................

250

19.4

QueryBroker Architecture

................................
................................
.....................

252

19.5

Main Interactions

................................
................................
................................
..

255

19.5.1

Modules and Interfaces

................................
................................
.................

255

19.5.2

Architecture

................................
................................
................................
...

255

19.5.3

Backend Functionality

................................
................................
....................

256

19.5.4

Frontend Functionalities
................................
................................
.................

258

19.6

References

................................
................................
................................
...........

266

19.7

Detail Specifications

................................
................................
.............................

267

19.7.1

Open API Specifications

................................
................................
................

267

19.8

Re
-
utilised Technologies/Specifications

................................
...............................

267

19.9

Terms and definitions

................................
................................
...........................

268

20

Query Broker Open RESTful API Specification (PRELIMINARY)

................................
.

270

20.1

Introduction to th
e REST
-
Interface of the QueryBroker

................................
........

270

20.1.1

QueryBroker REST
-
API Core

................................
................................
........

270

20.1.2

Intended Audience

................................
................................
.........................

270

20.1.3

API Change History

................................
................................
.......................

270

20.1.4

How to Read This Document

................................
................................
.........

271

20.1.5

Additional Resources

................................
................................
.....................

271

20.2

General QueryBroker REST API Information

................................
.......................

271

20.2.1

Resources Summary

................................
................................
.....................

271

20.2.2

Authentication

................................
................................
................................

272

20.2.3

Representation Format

................................
................................
..................

272

20.2.4

Representation Transport

................................
................................
..............

272

20.2.5

Resource Identification

................................
................................
..................

272

20.2.6

Links and References

................................
................................
....................

272

20.2.7

Paginated Collections

................................
................................
....................

272

20.2.8

Limits

................................
................................
................................
.............

272

20.2.9

Versions

................................
................................
................................
........

273

20.2.10

Faults

................................
................................
................................
...........

273

20.3

API Operations

................................
................................
................................
.....

273

20.3.1

QueryBroker operations

................................
................................
.................

274

21

FIWARE OpenSpecification Data SemanticAnnotation

................................
.................

277

Future Internet Core Platform



D.6.1.1b FI
-
WARE GE Open Specifications


Page
12




21.1

Preface

................................
................................
................................
................

277

21.1.1

Copyright

................................
................................
................................
.......

277

21.1.2

Legal Notice
................................
................................
................................
...

277

21.2

Overview

................................
................................
................................
..............

277

21.2.1

Introduction to the Semantic Annotation GE

................................
..................

277

21.2.2

Target usage

................................
................................
................................
.

278

21.2.3

Basic Design Principles

................................
................................
.................

279

21.3

Basic Concepts

................................
................................
................................
....

279

21.4

Main Interactions

................................
................................
................................
..

280

21.5

Re
-
utilised Technologies/Specifications

................................
...............................

281

21.6

Terms and definitions

................................
................................
...........................

281

22

FIWARE OpenSpecification Data SemanticSupport

................................
.....................

283

22.1

Preface

................................
................................
................................
................

283

22.1.1

Copyright

................................
................................
................................
.......

283

22.1.2

Legal Notice
................................
................................
................................
...

283

22.2

Overview

................................
................................
................................
..............

283

22.2.1

Introduction to the Semantic Application Support GE

................................
.....

283

22.2.2

Target usage

................................
................................
................................
.

284

22.2.3

Example Scenario
................................
................................
..........................

284

22.3

Basic Concepts

................................
................................
................................
....

286

22.3.1

Ontologies

................................
................................
................................
.....

286

22.3.2

OWL
-
2

................................
................................
................................
...........

286

22.3.3

Ontology Engineering

................................
................................
....................

287

22.4

Semantic Application Support GE Architecture

................................
....................

288

22.5

Main Interactions

................................
................................
................................
..

291

22.5.1

Modules and Interfaces

................................
................................
.................

291

22.5.2

Backend Functionality

................................
................................
....................

291

22.5.3

Frontend Functionality

................................
................................
...................

293

22.6

Design Principles

................................
................................
................................
.

297

22.7

References

................................
................................
................................
...........

299

22.8

Detailed Specifications

................................
................................
.........................

300

22.8.1

Open API Specifications

................................
................................
................

301

22.8.2

Other Open Specifications

................................
................................
.............

301

22.9

Re
-
utilised Technologies/Specifications

................................
...............................

301

22.10

Terms and definitions

................................
................................
.........................

301

23

Semantic Support Open RESTful API Specification (PRELIMINARY)

..........................

303

Future Internet Core Platform



D.6.1.1b FI
-
WARE GE Open Specifications


Page
13




23.1

Introduction to the Ontology Registry API

................................
.............................

303

23.1.1

Ontology Registry API Core

................................
................................
...........

303

23.1.2

Intended Audience

................................
................................
.........................

303

23.1.3

API Change History

................................
................................
.......................

303

23.1.4

How to Read this Document

................................
................................
..........

303

23.1.5

Aditional Resources

................................
................................
.......................

303

23.2

General Ontology Registry API Information

................................
..........................

304

23.2.1

Resources Summary

................................
................................
.....................

304

23.2.2

Representation Format

................................
................................
..................

304

23.2.3

Representation Transport

................................
................................
..............

304

23.2.4

Resource Identification

................................
................................
..................

305

23.2.5

Links and References

................................
................................
....................

305

23.2.6

Limits

................................
................................
................................
.............

305

23.2.7

Versions

................................
................................
................................
........

305

23.2.8

Extensions

................................
................................
................................
.....

305

23.2.9

Faults
................................
................................
................................
.............

305

23.3

API Operations

................................
................................
................................
.....

305

23.3.1

Ontology Operations

................................
................................
......................

305

23.3.2

Management Operations

................................
................................
...............

314

23.3.3

Metadata Operations

................................
................................
.....................

315

24

FIWARE ArchitectureDescription Data SemanticSupport OMV Open Spe
cification
(DRAFT)

................................
................................
................................
.............................

323

25

FI
-
WARE_Open_Specifications_Legal_Notice

................................
.............................

324

26

Open Specifications Interim Legal Notice

................................
................................
.....

326



Future Internet Core Platform



D.6.1.1b FI
-
WARE GE Open Specifications


Page
14




1

Introduction

1.1

Executive Summary

This document
comprises

the
Open Specifications of the
Generic Enablers in the
FI
-
WARE
Data/Context Management chapter. These Generic Enablers provide advanced platform
functionalities dealing with gathering, processing, interchange and exploitation of data at
large scale, thus easing the development of intell
igent, customized, personalized, context
-
aware and enriched application and services beyond those available on the current Internet.

The functionality of the chapter is illustrated with several abstract use case diagrams, which
show how the individual GE
can be used to construct a domain
-
specific application
environment and system architecture. Each GE Open Specification is first described on a
generic level, describing the functional and non
-
functional properties and is supplemented by
a number of specifi
cations according to the interface protocols, API and data formats.

1.2

About This Document

FI
-
WARE GE Open Specifications describe the open specifications linked to Generic
Enablers GEs of the FI
-
WARE project (and their corresponding components) being
devel
oped in one particular chapter.

GE Open Specifications contain relevant information for users of FI
-
WARE to consume
related GE implementations and/or to build compliant products which can work as alternative
implementations of GEs developed in FI
-
WARE. Th
e later may even replace a GE
implementation developed in FI
-
WARE within a particular FI
-
WARE instance. GE Open
Specifications typically include, but not necessarily are limited to, information such as:



Description of the scope, behavior and intended use of the GE



Terminology, definitions and abbreviations to clarify the meanings of the specification



Signature and behavior of operations linked to APIs (Application Programming
Interfaces) that the GE sho
uld export. Signature may be specified in a particular
language binding or through a RESTful interface.



Description of protocols that support interoperability with other GE or third party
products



Description of non
-
functional features

1.3

Intended Audience


The document targets interested parties in architecture and API design, implementation and
usage of FI
-
WARE Generic Enablers from the FI
-
WARE project.

1.4

Chapter Context

FI
-
WARE will enable smarter, more customized/personalized and context
-
aware applicati
ons
and services by the means of a set of assets able to gather, exchange, process and analyze
massive data in a fast and efficient way. Nowadays, several well
-
known free Internet
services are based on business models that exploit massive data provided by
end users.
Future Internet Core Platform



D.6.1.1b FI
-
WARE GE Open Specifications


Page
15




This data is exploited in advertising or offered to 3rd parties so that they can build innovative
applications. Twitter, Facebook, Amazon, Google and many others are examples of this.

The "Data/Context Management" FI
-
WARE chapter aims at provid
ing outperforming and
platform
-
like GEs that ease development and provision of innovative Applications that require
management, processing and exploitation of context information as well as data streams in
real
-
time and at massive scale. Combined with enab
lers coming from the
Applications/Services Ecosystem and Delivery

chapters, application providers will be able to
build innovative business models such as the ones described above and beyond.

FI
-
WARE Data/Context Management GEs enables to:



Record, subscribe for being notifi
ed about and query for context information coming
from different sources.



Model changes in context as events that can be processed to detect complex
situations that will lead to generation of actions or the generation of new context
information (therefore, also treatable as events).



Processing large amounts of context informati
on in an aggregated way, using
map&reduce techniques, in order to generate knowledge that may also lead to
execution of actions and/or creation of new context information.



Process data streams (particularly, multimedia video streams) coming from different

sources in order to generate new data streams as well as context information that can
be further exploited.



Process metadata that may be linked to context information, using standard semantic
support technologies.



Manage some context information, such L
ocation information, in a standardized way.

A cornerstone concept within this chapter is the structural definition of Data Elements
enclosing its "Data Type", a number of "Data Element attributes" (which enclose the
following: Name, Type, Value) and, opti
onally, a set of "Metadata Elements" (which have also
in turn Data
-
like attributes: Name, Type, Value). However, this precise definition remains
unbound to any specific type of representation and is able to represent "Context Elements"
and "Events" as "Dat
a Element" structures. More comprehensive information is available at
Fi
-
WARE Data/Context Chapter vision.

"Data" in FI
-
WARE refers to information that is produced, generated, collected or observed
that may be relevant for processing, carrying out further

analysis and knowledge extraction.
A cornerstone concept in FI
-
WARE is that data elements are not bound to a specific format
representation.

The following diagram shows the main components (Generic Enablers) that comprise the
first release of FI
-
WARE Dat
a/Context chapter architecture.

Future Internet Core Platform



D.6.1.1b FI
-
WARE GE Open Specifications


Page
16






More information about the Data Chapter and FI
-
WARE in general can be found within the
following pages:

http://wiki.fi
-
ware.eu


Data/Context Management


1.5

Structure of this Document

The document is generated out of a set of documents provided in the

public FI
-
WARE wiki.
For the current version of the documents, please visit the public wiki at
http://wiki.fi
-
ware.eu/


FIWARE.OpenSpecification.Data.BigData


BigData Analysis Open RESTful API Specification (PRELIMINARY)


FIWARE.OpenSpecification.Data.CEP


Complex Event Processing Open RESTful API Specification (PRELIMINARY)


FIWARE.OpenSpecification.Data.CompressedDomainVideoAnalysis


Compressed Domai
n Video Analysis Open RESTful API Specification
(PRELIMINARY)


FIWARE.OpenSpecification.Data
.Location


Location Server Open RESTful API Spec
ification (PRELIMINARY)


FIWARE.OpenSpecification.Data.MetadataPre
processing


Metadata Preprocessing

Open RESTful API Specification (PRELIMINARY)


Future Internet Core Platform



D.6.1.1b FI
-
WARE GE Open Specifications


Page
17




FIWARE.OpenSpecification.Data.PubSub


FI
-
WARE NGSI Open RESTful API Specification (PRELIMINARY)


FI
-
WARE NGSI
-
9 Open RESTful API Specification (PRELIMINA
RY)


FI
-
WARE NGSI
-
10 Open RESTful API Specificat
ion (PRELIMINARY)


ContextML/C
QL over HTTP Open RESTlike API Specification (PRELIMINARY)


ContextML API


CQL API


FIWARE.OpenSpecification.Data.QueryBroker


Query Broker Open RESTful API Specification (PRELIMIN
ARY)


FIWARE.OpenSpecification.Data.SemanticAnnotation


FIWARE.OpenSpecification.Data.SemanticSupport


Semantic Support Open RESTful API Specification (PRELIMINARY)


FIWARE.ArchitectureD
escription.Data.SemanticSupport.OMV Open Specification
(DRAFT)


FI
-
WARE_Open_Specificati
ons_Legal_Notice


Open Specifications Interim Legal Notice


1.6

Typographical Conventions

S
tarting with October 2012 the FI
-
WARE project improved the quality and streamlined the
submission process for deliverables, generated out of the public and private FI
-
WARE wiki.
The project is currently working on the migration of as many deliverables as p
ossible towards
the new system.

This document is rendered with semi
-
automatic scripts out of a MediaWiki system operated
by the FI
-
WARE consortium.

1.6.1

Links within this document

The links within this document point towards the wiki where the content was rendered from.
You can browse these links in order to find the "current" status of the particular content.

Due to technical reasons not all pages that are part of this document ca
n be linked
document
-
local within the final document. For example, if an open specification references
and "links" an API specification within the page text, you will find this link firstly pointing to the
wiki, although the same content is usually integra
ted within the same submission as well.



Future Internet Core Platform



D.6.1.1b FI
-
WARE GE Open Specifications


Page
18




1.6.2

Figures

Figures are mainly inserted within the wiki as the following one:


[[Image:....|size|alignment|Caption]]

Only if the wiki
-
page uses this format, the related caption is applied on the printed document.
As

currently this format is not used consistently within the wiki, please understand that the
rendered pages have different caption layouts and different caption formats in general. Due
to technical reasons the caption can't be numbered automatically.

1.6.3

Sampl
e software code

Sample API
-
calls may be inserted like the following one.


http://[SERVER_URL]?filter=name:Simth*&index=20&limit=10

1.7

Acknowledgements

The current document has been elaborated using a number of collaborative tools, with the
participation of

Working Package Leaders and Architects as well as those partners in their
teams they have decided to involve.

1.8

Keyword list

FI
-
WARE, PPP, Architecture Board, Steering Board, Roadmap, Reference Architecture,
Generic Enabler, Open Specifications, I2ND, Clo
ud, IoT, Data/Context Management,
Applications/Services Ecosystem, Delivery Framework , Security, Developers Community
and Tools , ICT, es.Internet, Latin American Platforms, Cloud Edge, Cloud Proxy.

1.9

Changes History

Release

Major changes description

Da
te

Editor

v0

First review of deliverable submission

2012
-
08
-
31

TID


v1

Second review of deliverable submission


2012
-
10
-
31


TID

V2

Version

for submission

2012
-
11
-
08

TID

Future Internet Core Platform



D.6.1.1b FI
-
WARE GE Open Specifications


Page
19




2

FIWARE OpenSpecification Data BigData

You can find the content of this chapter as well in the
wiki

of fi
-
ware.

Name

FIWARE.OpenSpecification.Data.BigData

Chapter

Data/Context Management
,

Catalogue
-
Link to
Implementat
ion

<BigData Analysis>


Owner

FI
-
WARE Telefonica I+D
,
Andreu Urruela/Grant Croker



2.1

Preface

Within this document you find a self
-
contained open specification of a FI
-
WARE generic
enabler, please consult as well the
FI
-
WARE_Product_Vision
, the website on
http://www.fi
-
ware.eu

and similar pages in order to understand the complete context of the FI
-
WARE
project.

2.1.1

Copyright



Copyright © 2012 by
Telefonica I+D


2.1.2

Legal Notice

Please check the following
Legal Notice

to understand the rights to use these specifications.

2.2

Overview

2.2.1

Introduction
to the BigData Analysis GE

Big Data Analysis is the process of using new tools to provide insights in to

data previously
considered too big or complex, given the current state of technology. The Big Data Analysis
GE allows an end user or developer to unlock the information within the data.
The
components used in this GE are:



Data Stream Analysis



A high
-
per
formance distributed file system compliant with Apache Hadoop HDFS that
provides a query interface for querying files



A NoSQL document
-
orientated storage solution compatible with the MongoDB.

Future Internet Core Platform



D.6.1.1b FI
-
WARE GE Open Specifications


Page
20




2.2.1.1

Data Stream Analysis

The Data Stream Analysis platform brings

together a cluster of commodity computer servers
or nodes to divide up and process a pre
-
programmed task.

Each node in the cluster has a Worker process that divides up the task amongst the
available resources within the server. These tasks are programmed

in modules designed to
solve a specific problem. Modules are built as shared libraries allowing for changes to be
made to the module so that it can be reloaded at run
-
time without interrupting the operation
of the platform. These modules contain the neces
sary logic to process a given data stream
and to represent the data internally to the platform. The analytical process is made up of a
series of MapReduce steps, as outlined in
Basic Concepts
, to produce the desired result
.

Communication between the nodes is handled via the network layer that ensures the data
needed to process each task is available on every node. The network layer also compresses
and optimizes the data flow to reduce latency when transferring data.

Data
is uploaded into the cluster using the
delilah

client tool, provided by the platform, in
addition this tool can be used to launch processing tasks and download result data. This tool
is available in two forms, a command line client and Qt based graphical interface.

There is also a
delilah

client libr
ary (C++ API), also provided by the platform, which can be
used for communicating between an application and the Data Stream Analysis platform (for
streaming operations).

2.2.1.2

High
-
performance distributed file
-
system

This generic enabler requires the use of a high
-
performance distributed file system based on
Apache's Hadoop HDFS. This framework allows for the distributed processing of large data
sets (> 0.5 Terabytes) across clusters of computers using a simple program
ming model. It is
designed to scale up from single servers to thousands of machines, each offering local
computation and storage. In addition we will make use of an SQL
-
like interface what allows
data exploration of data stored within the file
-
system.

2.2.1.3

NoS
QL document
-
orientated storage

For data sets smaller than 0.5 Terabytes the GE will make use of a document orientated
storage system based on NoSQL storage techniques. See the NoSQL section for information
on how this differs with respect to Relational Da
tabase technologies. Due to there being no
standard interface to interact with NoSQL database the interface is to be compatible with
MongoDB. Additional interfaces to other NoSQL systems will be considered as needed. This
component will be used to provide
a high
-
availability store for querying data.

2.2.2

Target Usage

Big Data Crunching (also known as Big Data Batch Processing) is the technology used to
process huge amounts of previously stored data in order to get relevant
insights

in
scenarios where latency i
s not a highly relevant parameter. These insights take the form of
newly generated data, which will be at disposal of applications using the same mechanisms
through which initially stored data is available.