Do you see limitation of this model? - osiris

richessewoozyBiotechnology

Oct 1, 2013 (3 years and 8 months ago)

137 views


FP7
-
ICT
-
248295/EGI interview



Page
1

of
25




SEVENTH FRAMEWORK PROGRAMME


Report on
EGI

interview


FP7
-
ICT
-
248295/
EGI

interview



Project Number:

FP7
-
ICT
-
2007
-
4 248295

Project Title:

Towards an Open and Sustainable ICT
Research Infrastructure Strategy (OSIRIS)


Authors:

Tyndall


A污n 䵡瑨ewson
, Cian O’Murchu

Contributors

and
Participants
:

Steven Newhouse (EGI)

Rosette Vandenbro
u
c
ke

Piet Demeester

Antonio Candellio


Reviewed:

Colin Wright


Abstract:


This report is a result of the Osiris Information Gathering Meeting in
Lyon

on
September
1
9
th

2011








FP7
-
ICT
-
248295/EGI interview



Page
2

of
25


TABLE OF CONTENTS




INTRODUCTION

................................
................................
................................
........................

4



QUESTIONS ON GOVERNA
NCE

................................
................................
.............................

5



Mission and rationale for the RI

................................
................................
................................
..............

5



RI Stakeholders

................................
................................
................................
................................
.......

8



RI Governance Structure

................................
................................
................................
.........................

9



Do you get external advice on your strategic plans? How important is this to steer future
developments?

................................
................................
................................
................................
..............

10



QUESTIONS ON SUSTAIN
ABILITY

................................
................................
....................

11



RI future 10 years from now

................................
................................
................................
.................

11



RI funding major challenges

................................
................................
................................
..................

14



RI expenses’ major concerns and distribution

................................
................................
.......................

14



RI Usage and External Users

................................
................................
................................
..................

16



Opinion on EU policies

................................
................................
................................
..........................

17



Usefulness of an European Stakeholder Group

................................
................................
.....................

18



QUESTIONS ON POLICY

................................
................................
................................
.......

19



RI access policy

................................
................................
................................
................................
.....

19



Access granting mechanisms

................................
................................
................................
.................

19



Responsibility and care of output quality

................................
................................
..............................

19



QUESTIONS ON OPERATI
NG PRINCIPLES

................................
................................
.....

21



Links with other RIs

................................
................................
................................
...............................

21



RI confidentiality and data protection

................................
................................
................................
...

21



Obsolete equipment replacement policies

................................
................................
............................

21



RI policies in case of missing goals

................................
................................
................................
........

22


FP7
-
ICT
-
248295/EGI interview



Page
3

of
25




Most important RI KPIs

................................
................................
................................
.........................

22



OTHER ITEMS THAT WER
E DISCUSSED.

................................
................................
.......

23



RI’s major challenges, successes and failures

................................
................................
........................

23



Interaction with other organizations

................................
................................
................................
.....

24



Horizon 2020 workshop

................................
................................
................................
........................

24



KEY STATEMENTS FROM
INTERVIEW

................................
................................
...........

25




FP7
-
ICT
-
248295/EGI interview



Page
4

of
25




Introduction


A set of questions were prepared in advance and sent to EGI by the OSIRIS consortium to
enable the interview to proceed efficiently. An interview team from OSIRIS then travelled to
the
Palais De Congress in Lyon to meet
with Dr Ste
ven

Newhouse
,

who is the
director of
EGI
.eu
,

on the 19
th

of

September 2011

Piet Demeester presented what Osiris wanted to achieve with the meeting and described the
mind map representation of research infrastructure structures and sustenance models.

In the mind map four key areas
have been identified for elaboration through the interview
process. These are:




Governance



Sustainability



Policy



Operating Principles







FP7
-
ICT
-
248295/EGI interview



Page
5

of
25




Questions on Governance




Mission and rationale for the RI

Q1.

What was the rationale for your RI and how did it evolv
e over time?
What is the
mission?


The rationale for EGI is to p
rovide distributed computing
and
storage resources for people
who have European collaborative projects and programmes. As people collaborate

at the
research level

and share ideas and
research
strategies they need to share storage and
computing assets because most of their primary data is being stored in an electronic form and
being able to share and analyse this data is essential. Both of these things are getting more and
more difficult to do i
n single organisation. Hence, there has been an evolution of ESFR
I type
research infrastructures
where people get together to invest
in

a physical or distributed
infrastructure as part of their research areas activities.

There is also a need to distribute
the
computing resources because no single organisation will have the resources to support the

computing

requirements for all of this research in a single place. To take this to the next level
in terms of efficiency there is a need to support open access an
d knowledge movement across
national boundaries.

So
,

e
-
infrastructures are needed to support those research activities.

On

the network side there is

Dante

/

Géant

providing a high speed transnational network in
cooperation with

the NRENs.
Alongside t
here i
s
the

requirement for performing t
he
computing
services as
part of the system
. PRACE has one model for providing high
performance

computing
while

EGI brings together the National Grid
I
nitiatives in each
country to provide a distributed computing platform
for use by members of the research
community who have a need to use distributed computing resources.


EGI is a follow
-
up of a number of projects?

EGI has resulted from a continuation of existing infrastructure projects which started with
the
European
Data
Grid (EDG 2001)
. This

three year project progressed into
EGEE
1
,

EGEE
2
and
EGEE3
(which were two year projects). In parallel with EG
EE
3
,

the EGI
D
esign
S
tudy
project
was
carried out
which defined the structure and management requirements of the
infrastructure to ensure sustainability as part of its output.
This was done because i
n the
course of EDG and
EGEEx
,
it became apparent that there
was a clear
requirement for the kind
of access

supplied by EGI to have a much longer and more sustainable existence than that
provided by short duration projects.
An
example was used to illustrate the point.
A
n x
-
ray
instrument

or a telescope

could have an active lifetime of more than ten years and th
e data
that it supplies needs to be accessible for several decades. If this data w
ere

to be analyzed and
stored based
on
infrastructures that were in themselves based on three year (competitively
funded) projects
,

the longevity of the data and user access
to it could easily be called into
question. This ‘data lifecycle’ needs to be taken into account in any infrastructure that
supports such instruments and experiments
. The early adopters could not ensure such a
lifetime when
projects with
only a window of 2

year duration were available.

EGI
was

FP7
-
ICT
-
248295/EGI interview



Page
6

of
25


therefore challenged to provide
an infrastructure

on a decade timescale and the EGI
D
esign
S
tudy
proposed their solution based on the experience of the NRENs which have been in
existence
,

in different guises
,

for
more

or less

thirty years and
the
mapping
of
this experience
onto the ‘grid’ world.


Is EGI based on the NREN model?

Yes, it is a
s
imilar model
.

EGI is an independent legal
entity (
Stichting in the Netherlands)
and so is Dante
(Limited Company in the UK).
The Council participants are the National Grid
In
frastructures
/Initiatives

(NGIs) that represent each country and European Intergovernmental
Research Organisations (EIROs).
These are
respectively

the resource providers at the national
level and the resourc
e providers from user communities. Whether this model is the right
model we cannot yet know. EGI is only 15 months old and will certainly
evolve

in the next
decades
.


Was EGI s
et up in close collaboration with the EC?

EGI Design Study was an E
C project so the EC was certainly aware of the project results.
The
final blueprint document was also fundamental in preparing for what is
now
EGI
-
InSPIRE

(4
-
year project supporting EGI)
. The EC’s need will change, as will the needs of the EGI
community, t
ogether we will continue to iterate and converge.


Maybe the EC did cho
o
se the EG
I Design Study because of the model?

The EGI Design Study was a

project
in the “Support Actions” of the E
C

Framework
Programme
that was proposed in an open competition. The co
rresponding proposal did not
actually

propose a model but
rather
described the process
to obtain a model for a future
European Grid Infrastructure.


Perhaps

the EC wanted to have the model of the Design Study implemented because they
funded the study?

Not
necessarily
, even during the lifetime of a project the view of the EC or the reviewers can
change the focus of the project.

Part of what the EC asks us to do is to become sustainable. If
the EC is
asking

EGI to be sustainable,
then there will be a differen
t relationship if activities
are supported by community funding than if they are supported solely by EC funding.
The EC
needs to understand
that their

role in our governance
will change as the funding changes
.


Is there a decreasing contribution from the
commission as a function of your
project
’s
success
?

Do they expect you to get a certain level
of funding
so
that
when their funding goes
down
you
r

‘however you get it’ funding goes up?

For EGI.
eu
,

the coordinating organisation, we have chosen to profile ou
r EC payments for
some of our activities

precisely for that reason

to give an opportunity for the participants to
step

up

gradually their commitment

so that at the end of a 4 year project
we are
in
striking


FP7
-
ICT
-
248295/EGI interview



Page
7

of
25


distance of

that sustainability when the participants

take up their
full
commitment.

Having
s
aid that there
are

still a

lot of
things that EGI does as an organisation and a community

that
is
meaningful
for EC activity such as the events

to bring

the community togeth
er,
the
networking

functions
.
We think that the

EC does not want to pay for routine

operation of
infrastructure.

T
hey want n
ational governments
to
pay for
those routine operations as the EC
has put money to create them

through their support of innovative E
uropean technology.

A
nd
we suppose that
if
innovation

is needed in the routine framework

that
the EC would be willing
to pay
for
this innovation.


Does the commission

consider that say, someone gets a European project

based on your
infrastructure
,

that it
is

seen as

part of your sustainability?

If a
n

applied
research project
is funded that uses distributed computing
,

it uses resources from
EGI and that collaborative framework is exploited by the project
. The key question is how
does EGI.eu monetise this usa
ge?

H
ow
can the
organisational costs

that participants pass on
to EGI.
eu

be funded through the project structure
. Does EGI.
eu
come

in as a
partner,

as
a
service provider
, as a contractor, as a cloud provider where there is a pot of money to buy in
the
serv
ices

we provide
?
We have two challenges
.

We have

the restrictions
that

the EC

themselves
put
in place and t
hen we also have
the

challenge
s
that our

activity is built from
national
ly

or locally
contributed

resources
such as physical hardw
a
re
. The

a
dded value
that

EGI.
eu
provides

is
to bring
those

resources together
. Making

this work is part of
its role.


W
e are still
articulating
internally and in the community what these
valuable activities
are.


I think the
way in which

your infrastructure can int
erface to all of those European
communities is a very grey and ill defined area.

How does your organisation interface

with

all those routine requirements?

It is
.

O
ne area
where there will be change is our engagement with

ESFRI projects
and

EIR
O
FORUM

labs
.

They have legal

independent

entities
that are funded to deliver
s
ervices to
their community
.
One opportunity is to
become
a
reseller
of services
t
h
ro
ugh
t
hem to t
h
eir

research
communit
ies
.

If you are tal
k
ing about FP7 and FP8
applied
research
projects the
n I
find it very challenging to see how we

(EGI.eu)

fit in that model
, but easier for our partners in
the NGIs we can being explicit technical skills into these projects
.


People need to access your service so how can it be done?

Partly
it comes down

to

th
e natio
nal
funders.
The su
stainability

model

of EGI
.
eu
does depend
very much
on the sustainability of the

individual

NGIs.
Our business model
,

at the moment
,

is
to

bill

them for
participation
fees

in order to cover the cost of delivering these coordination
services
.
If the NGIs
are not able to generate an income themselves through national funding
,

or through whatever
usage
model they have in their own community
,

it is very hard for them
to support

us and that is one of the ongoing challenges.


FP7
-
ICT
-
248295/EGI interview



Page
8

of
25




RI Stakeholders

Q2.

Who are
your stakeholders

and what influence do they have on
decisions
?

NGIs from different EC states

and EIROs

are the stakeholders.
There is an
EGI.eu
E
xecuti
ve
B
oard which has six elected

repr
esentative
s

and an elected chair
.
This group

supervise
the
activities
of the director
on a regular basis.

Above them is the EGI
C
ouncil
which is the body
that elects the chairman of the
E
xecutive
B
oard and the members of the
E
xecutive
B
oard from
its
own membership
.
Members of the EGI
C
ouncil are the
representatives
of the participating
NGIs or
E
I
RO
s
.
There are n
o industrial representatives (they are not e
xcluded


participation
in EGI
is open to
anyone
who sub
scribes
to the goals of the organisation b
ut they have not
come forward yet
,

so far it is only NGIs and
E
IRO
s
).


The
re are two kinds of partici
patio
n;
participants

and
associated
participants
. To

be a
participant

it is necessary to be an
NGI of
a

country

which is

eligible for FP7 funding

(EU 27
+ associate member states). An
associated participant

can be anyone else who is prepared to
pay the required fees and has been
approved
by the
EGI
C
ouncil. There is no requirement on
associated participants to provide resources
,

they just have to p
ay the fee. This fee contributes
to the cost of running EGI.eu as well as paying for services that partners within EGI and
EGI.eu deliver at a global level to help with the coordination.


Q3.

How do you distribute the resources?


EGI.eu

do
es

not own
computing

and hardware
resources

nor can
it
control their use.
Furthermore, m
any NGI
s do not own physical resources
,

but
rely on resource centres in their
universities or research labs to provide the resources to fulfil the services they require
.
These
resource pro
viders make the decisions about which virtual organisation is enabled onto their
particular hardware.
The virtual organisation model relies on a virtual organisation manager
to allow people to enrol into that virtual organisation and once they are in that
group they are
eligible to use that particular hardware configuration.


EGI do
es

not take a stance on the value (or non value) of any particular activity. If a
researcher can get

membership of a
virtual organisation

and

agreed on hardware in any
particula
r country through their NGI or their research contacts
,

they can have access to the
resources. EGI provide
s

the
overall
infrastructure

management

and distribute
s

the list of
people within each virtual organisation and will help the resource providers to ensure that the
services provided by one resource provider are integrated with those of another. This situation
could change in the future
,

but this is the current

status.




Q4.

Do you see limitation of this model?



FP7
-
ICT
-
248295/EGI interview



Page
9

of
25


The
a
dvantage of this model is that it is very op
e
n
,

de
-
centralised and unc
ontr
ol
l
ed
and the
main disadvantage is that it is very difficult to achieve a specific strategic direction because
there are
no
r
esources available to directly achieve it. This can only be done through lobbying
the resource providers and trying to convince them of the merit of the strategic direction to be
taken. This makes it a ‘bottom up’ organisation rather than a ‘top down’ one.



Not all virtual organisations are enabled in all countries. The most widely enabled are those
from the high energy physics
virtual organisations
.
Countries have their favourite topics. For
example there are blocks of countries who are interested in life

sciences, other blocks are
interested in computational chemistry and these do not necessarily overlap
. Not all
researchers in a country are aware of the availabi
lity of the virtual organisation

i
n

their
domain and the corresponding services.

If the strate
gic planning part was more clearly
resolved then
e.g.

all computational chemists in Europe would be aware of the service and
should participate in this to strengthen their own research

area and be more efficient and
effective in the way they do their work. This is something that the ESFRIs are beginning to
change because
it

is a European level action operating in different multidisciplinary areas.
ESFRIs do not cover all disciplines

at
all levels

and as they come into
p
lace it is hoped that
there will be a trickle down into national organisations to generate more collaborations at
national and international levels. Thus

ultimately

implementing the ECs vis
i
on of f
r
ee
movement of
k
nowledge

across the whole ERA.
However, how quickly this
can
be achieved
and how much

political effort is needed and how much

it
will
cost is not clear at this moment.



RI Governance Structure

Q5.

How

did you come to the current
governance structure
? Would you
propose

changes

in the future?

How were you funded at the start and how did it
evolve? What are your major sources of funding?

Voting

Weighted voting system li
n
ked to the Ter
e
n
a
key puts each country into different bands
relating to the GDP. The number of votes
and fee levels are defined based on the band that
each country sits in. This is a well
-
established model in the NREN world and this is being
used to minimize the extensive debates seen in the networking world and is being used until
something better comes along. It isn’t a high priority to find something better and it seems to
be working ok
at the moment although it is being discussed.

Changes

There is a mixed level of happiness with the
current
voting structure



but no clear
alternatives are forthcoming
. The big countries are content because they feel that they have
enough control over the

vote. The smaller countries are not so happy because they feel they
can’t generate enough opposition to block a
vote from
large countr
ies
. The challenge about
composition is due to the fact that
EGI
inherited a national based structure
and
it is hard to
e
nvisage how the user communities fit into it. When the governance of EGI was being
established there was a perception to move away from the dominance the CERN had due to
their high energy physics activities. This stimulated an open call for the hosting of

the
organisation and where it would be based.
Holland (
Amsterdam
)

won the bid over six other

FP7
-
ICT
-
248295/EGI interview



Page
10

of
25


countries. EGI.eu is based there and
this

helps to
re
move the
perception of EGI being mainly
in support of the high energy physics community.
The establishment of

EGI.
eu

outside CERN
may

have worked too well because the high energy physicists feel they have less engagement
in what goes on and as a consequence
appear to be
less committed to the activity. The EGI
C
ouncil needs to think about its composition and to re
cognise that there are organizations

(
such as CERN) that have a
n
incredible commitment

from their own organizational needs for
this kind of infrastructure

and should be able to have this reflected in the governance model
.

CERN

In the past CERN led the ED
G and
EGEE

projects so they were intimately involved in this
infrastructure provision for a considerable time.

There are two computing components at CERN
.

Like every
one else
t
hey have their own

computing needs and operate a

very large computing facility on site
,

some

of which
is

for
administrative

and technical purposes.

Then there is t
he
LHC
data generated by CERN
that
is
analyzed using the world wide LHC computing grid the EU component of which comes
through EGI. The other

components of the
worldwide LHC structure
are in t
he US and the
As
ia
-
P
acific region
.


The challenge for
EGI
is to
d
eliver

a generic infrastructure
in partnership with national
activities
,

many of which are aligned

and committed

to
the high energy physics
activities
and
they also contribute to
a set of
services
that
run
on

top of EGI and other service

provider

throughout the world
,

through
Worldwide LHC Computing Grid (WLCG)

focusing on
delivering those
services to

their end users. This
,
results in a very
complex

multi
-
horizontal
and vertical

governance

delivery

structure
,

which is difficult to navigate and EGI
is
still
working out how to manage the various relationships inherent in the system
.

EGI infrastructure is running 24/7 delivering a million

jobs pe
r day into the worldwide
WLCG activities. There are
always
issues which crop up in the running of an infrastructure
.
However,

in general
,

the high energy phy
sics community

is
, so far

appears to be satisfied
with
the delivery of the service they get.




Do
you get external advice on your strategic plans? How important
is this to steer future developments?


From the EGI
-
I
n
SPIRE
project there is an
E
xternal
A
dvisory
C
ommittee which advises at a
user orientated level. The EC review in July pointed out a lack of

consistency in the way that
EGI
deliver
s

its
strategic planning function.
EGI is

currently trying to elaborate how to
respond to this criticism and solve it for the future. There was a major input from the EGI
D
esign
S
tudy as to what the strategic structu
re should be and the first year of the

EGI

existence has been to implement this rather than trying to reinvent themselves while they are
still setting up.


Do strategic plans relate to generating new user communities?


FP7
-
ICT
-
248295/EGI interview



Page
11

of
25


Yes, but
it is also about
sustainability
. Obviously this can easily be linked to gener
ating new
user communities
,

but
also to
the

services that are offered
--
which
is

another dimension in our
strategic discussion
.

The director needs to ‘unpick’ the multi
-
faceted problem of strategic

planning and make it
processable
for an

EGI
C
ouncil
meeting
to deliberate on. This needs to
be done every six months and is one of the challenges for the governance of the organisation.
This is an area that is being reviewed as part of our first EC review

in June 2011.



Questions on Sustainability





RI future 10 years from now

Q6.

How do you see your RI
in 10 years from now

and how will it be
funded? What do you consider the ideal mixture of funding for your RI today
and what do you consider
that
it

should be

in 10 years from now. (project /
program funding, industrial / government, regional / national / EU)

There is a long term need
(5
-
20 years
)
for something to be in existence to support the use of
distributed computing.


Does this move into a model within 5 to 20 years where EGI.
eu

is a
frontend and a contract negotiator for Amazon web services to deliver all the computing
requirements for people in the research area

or
sh
ould EGI
.eu

act as a broker between a large
number

of independent cloud providers who have agreed standards and quality with service
level agreement about operational delivery and protection of data
?



There
are
also people in the community who want to me
asure

the output of individual
resource centres and

possible sell capacity to external users.



However
,

with

al
l

of these discussions there is a need to identify an entity
which

would be
willing to pay for the service. Even in the case of an EU project


how do they pay
?

F
orget
even about paying

for EGI.e
u
,

each individual resource centre for the work
they do for a
given project?
How

can you define where to write down the charges? How can you claim
them? Is there a pot of money available from the
EC

(with minimal strings attached) to pay
for th
e funded

act
ivity. (
Possibly

earmarked for computing services??)


Q7.

The existing project for EGI is four years and at the end of it the
commission need
s

to be happy that progress has been made against the
deliverables in the project. But what happens to the sustainability model after
the four year period if the commission
is not

happy with progress in the
project?


One way to ensure that the
EC

is happy is to show some form of sustainability. Some parts of
the project could demonstrate sustainability over the four
year period
,

but not all of them
.

There are also areas where the EC
is
still willing to fund and there
are
activities that still need


FP7
-
ICT
-
248295/EGI interview



Page
12

of
25


to be done as a rollover from existing calls. All of these need to be considered as part of the
macroscopic strategy plan going forward.


However, the funding is not ‘out there’ to continue to deliver the level of service that has been
provided to date.
So
we and the EC
need to prioritise the services th
at can be

deliver
ed

or be
more imaginative as to who delivers them. It may get to the stage where communities will
have to allocate a portion of their research funding to sustain the distributed computing
requirements that their work requires.


These items have not been discussed in detail yet in the management
,

but they are on the
agenda and planning is required to discuss them and take action accordingly. There are no
legal entities that can deal with th
is kind of problem or participate in the discussions across
national boundaries and ma
ybe ERIC is a solution. This is a model that is interesting
,

but
there would need to be a number of
N
GIs
all collaborating together

with a common purpose
to achieve this
and in the short medium term it is difficult to see this happening.


Q8.

Do you think it is appropriate that the commission expect
s

you to
become self sustainable?


The
EC

has
said that EGI needs to become self sustainable because they can’t keep putting
money into the system at the same levels as they have

done

in the past. This is a reasonable
statement
:

however there should be consistency of delivery of this message (e.g. the NRENS
still get large amounts of money for dis
tributing

services in a monopoli
stic manner).
See
question 5 for more details on this.



Q9.

What is your view on the
sustainability
of software over the next ten


fifteen
years?



Software suppliers need to be considered in parallel with the
technologies used under the
umbrella of sustaina
bility.


Over the last ten years there has been money to support innovation. However
,
the
EC

is
now
saying that
there will be
fewer

resources to accommodate the
maintenance and development
of
middleware that is being used within the community
.
EGI
is

dependent on the
sustainability of the NGIs and the
technology providers

that are used to construct the
infr
astructure.
They need to be supported for the
maintenance

of the software they deliver
,

with new developments and innovations being supported throug
h research programmes.
However, t
his adds a software
maintenance burden to the coordination activities and that is
difficult to sustain from existing funds.


This
also
requires decisions on which technology
needs
to be supported
and
how much of the
existi
ng softw
are stack that is supported by
the
EC

is necessary
to
the infrastructure

provider
and how much of it is aimed at the end user? It is suggested that the cost of the end user

FP7
-
ICT
-
248295/EGI interview



Page
13

of
25


oriented software should be taken by user communities rather than
by
the in
frastructure
provider.


Part of the sustainability for software is to have less software that needs to be sustained. This
passes the burden onto somebody else but there are ongoing discussions about ‘whose burden
it
is
to sustain the infrastructure’. It
could be the state or the individual or some point in the
middle
,

but somehow there has to be a balance and this discussion needs to take place and
decisions

need to be

made.


Also virtualisation needs to be taken into account. New users want to deploy ex
isting software
that
they have

been us
ing

for years
and which is different from that which is currently being
run in EGI
on a European scale. EGI can help with deployment and operation but
EGI

will
seek more and more the technology of virtualisation to do
that. The challenge
becomes ...


how do we
,

as a
n

infrastructure
provider
deploy these
encapsulated virtual machine images’
?

This is different
from

the

current model where
distributed
software is

installed locally for
people to use across Europe. EGI
and
its resource centres
only
have the resources to
deploy
one software set and this restricts the number of people
that can be attracted
to use
that
software
. The more that open source or commercial software can be used
,

the more that EGI
can leverage on what

other people are doing and the bigger the market for EGI services. There
is
the

route forward for
us

as an
infrastructure

provider
.


H
ow
e
ver
,

the user community may not have this
possibility
because the user community
software has evolved when times were

good and ‘tinkering


with code was possible because
the resources were available to support it’
With the current funding environment t
his is no
longer the case and it may become necessary to compromise and reuse/reconfigure some
existing open source softw
are
,

instead of
using
the optimised codes
that were adopted

previously
,

that

will perform most
(~
90%)
of the user requirements. The only alternative to
this is to seek funding to either collaborate with others who can do the remaining parts of the
requirem
ents or support the custom construction of software specifically for purpose.


S
oftware providers need to start to look into this and they may end up having to stop working
on some aspects of their code and this will have an impact on the number of
projects they do,
funding levels, employment and a multitude of other aspects. It is very difficult to see a
sustainability route from the software providers
,

which doesn’t involve
more engagement of

the user communities.



EGI doe
s

not develop

software
so it
can get software from any provider
, commercial or not,
but the problem

is with depl
o
yment

of that software.
Currently,

it puts a high load on the local
system administrators to install and maintain that software and only a limited number of
packages
can be handled.


Virtualisation seems to be the way forward for the deployment of software to resource
centres. In this model specific software can be turned on in the local resource centre or not as
the need for it grows or diminishes. EGI provide
s

these
virtual images to resource centres and
this facilitates more user communities to get access to the infrastructure across Europe simply
by
activating the software they require as they need it. This can be deployed to resource
centres and end users transpare
ntly and this enables more flexibility on the way that software


FP7
-
ICT
-
248295/EGI interview



Page
14

of
25


is delivered
in the future.

T
he maintenance can then be done centrally on behalf of the users
or by the user communities themselves. Many such virtual images can co
-
exist.


EGI has established

a Federated Cloud Task Force, which is investigating exactly how and
where this transition will take place.




RI funding major challenges

Q10.

What are the major challenges to ensure the required funding? Is there a
need for a different funding model or are oth
er models more appropriate? Is
the funding model of your e
-
Infrastructure the right one to guarantee
sustainability?


The design study focussed on the creation of National Grid
In
frastructure
s
(NGIs) as national
entities. Within EGI they have had to come into existence. The current financial climate has
meant that these NGIs have not been sufficiently funded to really exist with the coordinating
power and force that
was

intended when they were c
onceived. The
EC

then, needs to decide
how
global

(European
) the
EC

wants the infrastructure to be. There are between five and ten
countries which are committed to this, at a national level and are willing to commit to it at a
European level. If EGI went a
head with just these countries it would very much bring out a
two tier Europe in terms of distributed computing activities. If the
EC

do
es
n’t want this to
happen, they will have to put some money into the system to stop it happening. If they don’t
want to
make this investment, this is what will happen.


The
EGI
-
In
SPIRE

reviewers said many times that EGI should adopt a more business oriented
approach. There are 350 resource centres in the EGI organisation.
On

the tail of the sizes of
these resource centres
distribution there are 150 resource centres that deliver
fewer

than 100
cores. A
strict
business
perspective

would indicate that these resource centres should be
dropped because they provide 20% of the capacity and draw down 80% of
our support

resources.
F
or this
distributed
model
encompassing resource centres of different sizes all over
Europe
to continue then the EC

needs to support this model directly.



EGI need
s

to decide on
its

comfort zones and present them to the
EC

with
a

view to getting
responses
that will define policy to be followed with respect to sustainability of EGI and the
levels
of
support that it provides across the EU.





RI expenses’ major concerns and distribution

Q11.

What do you see as the
major concerns

in terms of
expenses
? Capex,

Opex (personnel, energy, maintenance, consumables, R&D, …)



FP7
-
ICT
-
248295/EGI interview



Page
15

of
25


There
is

no capital expenditures from EGI
.
eu

as
all of the expenses are operational based and
efforts are being made to drive these costs down to fit within the funding available.

Most of the
ca
pital
expenditure within the EGI organisation is distributed across
the

member
states in terms of their investment in
their national computing infrastructure (i.e.
hardware
,

energy, maintenance etc.
)

EGI
supports

an association of
NGIs
and provides them wi
th a
network
,

management and coordination function across Europe. There is a fee for
participation in EGI payable by each member state. The delivery of our infrastructure is
depend
ent

on the technology we have to use.

Q12.

What expenses

are or should be covered

by government/industry/users
(in the form of funding/projects/payment
)?


The cost of running EG
I

is depend
ent

on the investment in each member state in terms of
hardware

since
,

as mentioned,
there is no
CAPEX

within EGI
.
eu
. There is some activity in
OPEX which includes personnel, energy, maintenance, consumables, R&D, where they are
working to drive down costs.


The European dimension is and
should be covered by EC funding
.
EGI is

also looking at
payment for usage and they

are looking at how to monitor us
age

to c
harge back to users but
the key

is that there are very few people who will pay directly for such a service and it is not
clear from what level of a user organisation

or country
the payment should be sought.
The
only

source of funds at present is through the NGIs which have a tiered membership fee
,

depending upon GDP of the country involved. If the debate about who pays for this continues
(and it will) then any other funding model becomes artificial or charge per use
will be
implemented and this will have a consequence on the experiments that are performed because
at present usage is not monitored
. This mean
s

that quite large experiments are performed
because there is no project size related charge for performing the e
xperiment

as

is done at
amazon.com

If there was a charge for this
,

then the size
(and possibly value)
of experiments
would diminish.


Can you explain the number of investments you mentioned in one of your presentations?

The
EGI
In
SPIRE

project is paid 25

M
E
uro by the
EC
. H
owever
,

since the hosting
,

buildings,
power, staff, hardware investment
and replacement
etc is all done at a local level and the
countries are paying for this. T
he overall cost of the project is of the order of 330 MEuro (for
4 years). T
his means that although the EC are providing 25Meuro through
EGI
-
InSPIRE

they
are only supporting the infrastructure at a level of about 10%. This has been pointed out to
them on several occasions and is the reason that the NGIs have a considerable say in the
directions taken. If the
EC

paid a larger percentage of the overall
cost they would get a larger
say in the directions that are pursued.


FP7
-
ICT
-
248295/EGI interview



Page
16

of
25





RI Usage and External Users

Q13.

How important is the
usage

(e.g. number and/or size of users) to make
your facility sustainable? Usage at the national level, Usage by international
user g
roups


Q14.

What are the mechanisms to stimulate / facilitate access to
external
users
?

Q15.

Do you plan on
growing

your external user community?

These three questions can be answered together.
Most of the CPU usage is still High Energy
Physics
(~
90%
).
There are po
ssibilities for expansion. Other communities have been able to
use it but their needs have not
always
been
taken on board in the past. Going forward it
would
be
interesting to expand the service to other users and in terms of cycles there are a number of
u
ser communities which would have similar requirements
to

the High
Energy
Physics
communities. And it is hoped that it would be possible to reduce the percentage of HEP to a
level of 40
-
50 % of a much larger user base.
Thi
s is not achievable in a 2 year

tim
eframe but
can be done in a longer term.


Why do people use Grid instead of HPC?

HPC is very expensive and there is a requirement to have
computing requirements
that benefit
from the technical architecture that the HPC equipment exhibits. HPC tends to solv
e single
complex problems. However, a lot of data analysis problems are throughput oriented because
there are enormous amounts of data coming out of one (e.g. x
-
ray) machine so being able to
do a large number of individual
analyses

on the

data on multiple machines speeds up the
solution. There is a requirement to set up problems for the EGI model and many communities
have problems that need solutions like this (lots of parallel calculations)
.

It is becoming more
clear to people that distri
buted computing is a paradigm that can be used to solve user
problems and run
large numbers

of simulations simultaneously which could reveal insights
that were hidden because the user was performing one simulation at a time.


What is the major reason why
new user communities would not use EGI

The lack of availability of user specific software
and/
or the needed database
on the computers
is really the inhibiting factor in expanding the service provided to larger communities.
Bioinformatics for example ha
s

sp
ecific software and databases that need to be in place to run
their simulations. This is normally catered for and supported at local level through the NGIs
but moving to different user group
s

will have the same problem. So porting user software
onto a

grid

architecture is the big challenge in expanding the user base.
The difficulty
l
ies

in
having

many different packages
.

The advent of cloud computing
/virtualisation

will diminish
this problem and possibly make it easier to configure new users onto the system
.

The l
ocal
system administrator
workload may reduce as they
only
have
to provide access to
virtual

package installers.


FP7
-
ICT
-
248295/EGI interview



Page
17

of
25



Do the new
users need

to

invest in hardware?

Many of these new users would
need to invest in hardware to run their simulations. However
,

the NGIs have a

multi
-
dis
ciplinary remit and there can be investments though the NGIs
and
their resource centres
that
can
facilitate the purchase of specific hardware

for specific
purposes
.

Many NGIs are now using ‘multi
-
disciplinar
it
y
’ as a metric for s
uccess as an
infrastructural support.


In future it is possible that EGI will act as a broker to commercial cloud computing services to
satisfy some user requirements.





Opinion on EU policies

Q16.

What is your opinion on the
EU policies

in your field of acti
vity? Are
you
satisfied
? Do you have
suggestions for improvement

(e.g. with respect to
Horizon 2020

vision)?

Q17.

What are you three most important messages to the commission
regarding Horizon 2020.

Ensure that in projects there is a mechanism that people can use to pay for computing services
to be delivered.
T
he
EC

has to give the research
ers

the ability to pay the infrastructure
providers for their services if it wants a market model or it has to pa
y the e
-
infrastructure from
central funds to deliver those services. The money has to be in the system for this to work but
how it is delivered is open for discussion
.

The
EC

should provide some consistency. EGI ha
s

a multi
-
decade commitment to their user
communities.
The EC should recognize
NGI
s

are still in their infancy and will still need
support over the coming years to deliver an efficient pan European infrastructure.


The role of private activities.

The
EC

seem
s

to be

worried that there is not a European Amazon and they seem to want to
create one. It would
appear to
be better
integrate resources from multiple providers by having

an entity to
buy
services from existing suppliers. There is a clear business
opportunity

i
n
terms of brokering and handling of resources and the
EC

seem
s

to want to encourage
industrial participation in this but still want to retain a hand in how it works.

This approach
needs to be carefully considered
.



FP7
-
ICT
-
248295/EGI interview



Page
18

of
25




Usefulness of an European Stakeholder G
roup

Q18.

Is there a European Stakeholder Group (e.g. including national
government representatives, EC representatives and the RI
-
owners)? What are
the topics covered by this SG? Should the focus be broader then sustainability
(e.g. operational principles, leg
al aspects, …)? Any suggestions for
improvements?

T
here certainly

is

a need for a stakeholders group, however, at the moment, we have to
o

many: As we try and seek closer integration with the different e
-
infrastructure partners needed
to fulfil the missio
n, to deliver integrated infrastructure, it’s getting more challenging to do
that with all those different
b
oards, to try to get a coherent strategic procedure from

them.
There’s been discussion about an ERIC for EGI and Dante and also one for PRACE and
ot
hers.
The feasibility of

an ERIC for e
-
infrastructure

needs to be considered.
That ERIC
-
Council would be the definitive European level policy area / forum for distributed computing.
Underneath that umbrella you would have the PRACE
,
EGI and networking.

They may still
have their own operational Council
s
. But the focus would be much more on delivering that
information

within the financial constraints coming through from that level. This would bring
us
all under the same umbrella

and provide a central strat
egic point for the EC to contribute to
policy across all European distributed computing infrastructures.
However, t
here is a
requi
rement to gather together all o
f

the e
-
infrastructure groups and get them all to talk to the
EC

at t
h
e same time to get a cons
istent message

and

can work together
coherently
to deliver
it
.




FP7
-
ICT
-
248295/EGI interview



Page
19

of
25




Questions
on

Policy




RI access policy

Q19.

What is your policy on providing
access to your facilities
? How do you
grant access and on what
basis?

Many of the NGIs and the resource centres have

a
multi
-
disciplinary mission, delivering
resources. They are tapping into resources locally or nationally. We can make it easier for
these individual resource centres to gain access because of our global coordination and for
communities that do not have a
ccess to as many resources as they require, we can connect to
the commercial cloud providers to scale up to, provide a mix, to manage their work load
capacity. They can access locally or if they have to have a peak demand they can
try to get
resources else
where.




Access granting mechanisms


Q20.

Can you explain the motivation of your current
access granting
mechanisms
? Do you think the arrangements should be modified in one way or
another?

Technical communities are formed within the NGIs and
are established across
EGI
.

T
hese
communities provide the basis and foundation for access. Once a community has been
established in one or more NGIs and the software they use works appropriately then they

can

access the system.




Responsibility and care of
output quality

Q21.

What is your primary Output?



The
primary output
of EGI
is delivery of the infrastructure 24/7.


Q22.

How do you ensure the quality of your output and who is responsible for
it? Is this a big issue? How do you assure the quality of your data?
Do you
foresee changes in the way you assure the quality of the data?


Delivery of
the infrastructure is dependent on the technology that can be used. Quality control
is performed on
input from
technology prov
iders and that helps to catch
a

l
ot of issues.
This

FP7
-
ICT
-
248295/EGI interview



Page
20

of
25


ensures that, at least to a first approximation, the reliability of the software that users can get
access to. This is
realised
through the
operations

arm of the organisation. There is also a
software provisioning activity and EGI work
s

with
external
software providers to bring i
n

suitable software. Acceptance criteria are provided to set t
argets for technology providers
.
Software is deployed and assessed against these criteria followed by a final testing (
staged
rollout
) w
here guinea
-
pig (evaluation)
sites with different environments, volunteer to evaluate
the software. This is performed in the knowledge that it is new software and the evaluation
proceeds as the load on it is increased. If it is still working without problems after a few days,
the soft
ware is deemed to be ok and can be
deployed

across the network.


How many cycles have been done?

In the first year there were incremental releases

-

EMI

(their

main software provider) provided
a release in May
. T
his was released
(by EGI)
in July after checking. A further release was
made in August and one in September (from
The Initiative for Globus in Europe

(IGE)) and
this represented a milestone because it meant that they now
release
software from two
different
technology
providers

out
into the infrastructure

and
that
this
means

that
EGI.
eu

is
open
to do this
for other technologies
and
other communities
.




Quality and security of data
?

It is the users responsibility to e
nsure the quality of their data.
We can only facilitate their
requi
rements.


How do you guarantee that data is safe?

The user software has access control lists. We have technology that allows replicas to be built.

Technology has been developed so that m
edical community data can be encrypted wit
h

split

encryption key
that enables a key that is split into N parts be restored from N
-
1 parts
(in the
software
H
ydra)
.



FP7
-
ICT
-
248295/EGI interview



Page
21

of
25




Questions on Operating Principles




Links with other RIs

Q23.

Do you work
together

with other RI’s? Both outside Europe and with the
EU countries? How important i
s this
collaboration
?

Within EGI
-
In
SPIRE

there are
partnerships with the Asian
-
Pacific Area.
They

also work with
Open Science
Grid
in North
-
Americ
a

(the

North American part of WLCG) and n
ot only
d
o
the end
-
users scientists collaborate across with UU
-
OSG bu
t have an operational relationship
with OSG

as well. Across this collaboration there is an
integrated accounting, monitorin
g, and
security response. This is necessary because it is very difficult to trace incidents between
countries and across different co
ntinents
and different time zones and they have to be able to
have a response to these issues. This is in place and it is effective.

Within Europe,
EGI

is
working closely with PRACE
to see how they can
integrate the
ir

workflows together.
For example PRACE

could generate petab
y
tes of data but the analysis of
this data would not be a good use of HPC capabilities. Probably the best solution would be to
analyse the data using a distributed computing based approach. So interacting with EGI would
be an efficient collaboration which u
ses the best of each technological approach. This kind of
workflow merging is being investigated at present.

Best practice for interacting
between these networks and how to integrate the solutions
will be
discussed at
the

Community For
um in Munich next Ma
rch (2012).




RI confidentiality and data protection

Q24.

How

do you maintain confidentiality across the RI (scientists, industrial
users,)
? How are results/data
stored

and
protected
? Is this a serious
issue
?

The software services
allow users to apply
access co
ntrol

lists on the software.

The

technology
can

allow replicas to be built
so that

you can have a logical view
of
where to find
a file.

You can
also
encrypt data and distribute the key to different locations

such as is done
with medical image data on the n
etwork.




Obsolete equipment replacement policies

Q25.

Have you a plan in place to
replace equipment

as it becomes obsolete?
How does this relate to the
sustainability

of your RI?

Hardware replacement is the role of the NGIs


FP7
-
ICT
-
248295/EGI interview



Page
22

of
25




RI policies in case of missing goals

Q26.

If something goes wrong or is delayed, do you compensate the customer?
If so, what instrument would you use? Is this a
serious issue
?


Nothing ever goes wrong

!

EGI has

sufficient redundancy by design

and

we

believe that
everything is covered
. Since we
do not take money, there is no compensation

to be provided
.
If something did go wrong,
and it was due to a failure of the organisation, a
n

explanation

of
what had happened
and the solutions put in place to stop it happening again would be
provided
to the a
ppropriate Management Board of that community
.




Most important RI KPIs

Q27.

In your opinion, what are the
most important KPIs

that reflect the
performance of your RI?

Performance
defined by

operational activity.
This, as stated above, means that it is crucial t
o
have the infrastructure working 24/7. However
,

at the review of the EGI
-
In
SPIRE

project
,

the
EC

said that they did not consider this to be sufficient and that they felt that the infrastructure
sustainability and the growth of new user communities should
also be among the key
indicators. KPI for the i
nfrastructure is still 24/7 ope
r
a
tion

but for the project they have been
advised

to

focus on

increasing

levels of adoption and usage.




FP7
-
ICT
-
248295/EGI interview



Page
23

of
25




Other items that were discussed.



RI’s major challenges, successes and fa
ilures

Q28.

W
hat do you see as
the major challenges

of the
future

for your RI?

T
he major challenge is that of growth and sustainability.
Growth is the most important.

If
EGI

can grow,
they

can sustain
themselves, if not they will have to start redistributing their
resources between users within a tight financial envelope.


HEP
are

traditionally the most important users, but is there also interest from industry?

Naturally, since its inception High
Energy Physics has played a major role in the usage of the
infrastructure. However, t
he growth in
other

(i.e
. not HEP)
usage over the last year is about
20 or 30%, which on its own is very healthy.
This increase has been masked because,
it was
very easy fo
r the
high energy

physics community
to double their usage,
(
even from an already
very high level
) and they currently
utilise 90
% of the capacity of the infrastructure.
This
factor of two increase
and their 90% capacity utilisation
easily masks the growth i
n
other

community usage

because it is all occurring within the 10% that CERN d
oes not

use.

Q29.

Is it possible to get industrial interest in the services you provide?

The challenge
EGI has

is

that

regulations in many countries prohibit
publicly funded
organisa
tions from engaging in
commercial activities
. So
providing
services to
Pfizer
or GSK
would be seen as illegal in some countries. Furthermore
,

using the GEANT
/Dante

network
and running commercial services over that could also be illegal.

Pre
-
competitive
activities can and have taken place with EDG and EG
EE
;

but if EGI accept
s

money for their work
,
does it become a commercial service
?

There is also the problem that
some

NGIs would allow
this
and others not and this make
s

it difficult to go to the European
level.

If it does and it takes EGI into competition with Amazon for example, litigation
from
the US
would happen quite quickly

to stop publicly funded services competing with a
commercial venture
.
Also if you’re providing a commercial service you will need

guarantees,
and have the appropriate agreements in place within the organization, etc.


Is the EC pushing to have industry on board?

It’s not to say we cannot do it, or we can
as part of the plan for sustainability. However, it is
unlikely
in the short te
rm
,

unless

is

it pushed by the EC.

There may be some countries and
participants to some networks who are interested in a limited commercial venture and this
could be explored as part of a two tier infrastructure that to date has not been done.



FP7
-
ICT
-
248295/EGI interview



Page
24

of
25


Q30.

What do you

see as the
greatest successes/failures

of your RI?

As
with any

international organization, there are a lot of politics involved, and it is naïve to
assume that this sort of thing disappear
s

over time
. T
here have been many mistakes in the past
about how th
ings have been dealt with, both politically and technically and these legacies
can
really run on for ten years. It’s very hard to deal with th
is kind of legacy
. Also it is clear that
discussions you make today could cause disruption for decades to come …



I
nteraction with other organizations

Q31.

What is your role with
respect to other organizations

(e.g. EEF versus
EGI, NICTD versus eIPF) ?

EGI

participate
s

and contribute
s

in all of these organizations



Horizon 2020 workshop

Q32.

Would you be interested in the
Horizon 2020 workshop
in
Brussels (
Q2,
2012)

Dr Newhouse w
ould definitely
be interested in participation
if given enough notice.





FP7
-
ICT
-
248295/EGI interview



Page
25

of
25




Key Statements from Interview


We have extracted the following statements from the interview which we felt were important.

Please let us know if you are happy with us transmitting this information publicly and to the
commission as part of a synthesis of comments from the various interviews we have
undertaken.


S1.

Part of what the EC asks us to do is to become sustainable. If the

EC is
asking

EGI to
be sustainable,
then there will be a different relationship if activities are supported by
community funding than if they are supported solely by EC funding.
The EC
’s role in
EGI’s

governance
will change as the funding changes
.


The o
verall cost of the project is of the order of 330 MEuro (for 4 years). This means
that although the EC
are providing 25Meuro through the
EGI
-
InSPIRE

project,

they
are only supporting the infrastructure at a level of about 10%. This
is why
the NGIs
have a c
onsiderable say in the directions taken

by EGI
. If the
EC

paid a larger
percentage of the overall cost they would get a larger say in the directions that are
pursued.

S2.

The lack of availability of user specific software and/or the required database on
computers is really the inhibiting factor in expanding the service provided to larger
communities.

Mechanisms (such as vitualisation) need to be developed to help remove
these barriers.

S3.

The
EC

has to give researchers the ability to pay the infrastructure p
roviders for their
services if it wants a market model or it has to pay the e
-
infrastructure from central
funds to deliver those services. The money has to be in the system for this to work, but
how it is delivered is open for discussion?

S4.

The
EC

seems to b
e worried that there is not a European Amazon and they seem to
want to create one. It would
appear to
be better
integrate resources from multiple
providers by having

an entity to buy
/broker

services from existing suppliers. There is a
clear business opport
unity in terms of brokering and handling of resources and the
EC

seems to want to encourage industrial participation in this but still want to retain a
hand in how it works. This approach
needs to be carefully considered
.

S5.

Access supplied by EGI requires a
much longer and more sustainable existence than
that provided by short duration projects.

S6.

The European dimension is and should be covered by EC funding.

S7.

There’s been discussion about an ERIC for EGI and Dante and also one for PRACE
and others.
The feasibil
ity of

an ERIC for e
-
infrastructure

needs to be considered
.
That ERIC
-
Council would be the definitive European level policy area / forum for
distributed computing.