Measuring the Contribution of Sustainable Development Research to Society and Policy-Making

loutclankedΤεχνίτη Νοημοσύνη και Ρομποτική

13 Νοε 2013 (πριν από 3 χρόνια και 9 μήνες)

62 εμφανίσεις

Measuring the Contribution of
Sustainable
Development Research to Society and
Policy-Making
Luke Georghiou
Manchester Institute of Innovation Research
Manchester Business School
Some initial trade-offs

Research as consumption good vs research
as an investment

In first case value is cultural, in second there are
users with the expectation of a return at some point
in the future

Outputs vs Impacts

Output may be scientific publication or even new
product or service but impact only realised when
output interacts with economy or society eg sales
or green technology or application of new
regulation
More trade-offs

Short term vs. long term

many different time-profiles over which impacts are
manifested including

short-term effects which terminate abruptly as market conditions
change

outputs which are not used for some years and then become
very important, perhaps because complementary technologies
have been developed

extrapolation is dangerous.

Intended vs. unintended

excessive focus on project or programme goals as the basis
for evaluation could lead to important unintended effects
being missed

eg in medical research when a drug developed for one condition
turns out to be important in the treatment of another

Unintended effects also include most of those in the negative
category.
INNOVATION
Problems in calculating returns to R&D
Timing
Attribution
Appropriability
RESEARCH
EFFECTS
ICT Technology Pillars
Nano-electronics, photonics and
integrated micro/nano-system
Ubiquitous and unlimited capacity
communication networks
Embedded systems, computing and
control
Software, Grids, security and
dependability
Knowledge, cognitive and learning
systems
Simulation, visualisation, interaction and
mixed realities
New perspectives in ICT drawing on other
science and technology disciplines
Integration of Technologies
Personal environment
Home environments
Robotic systems
Intelligent infrastructures
Applications Research
ICT meeting societal challenges

health

inclusion

mobility

environment

government
ICT for content, creativity and personal development

media

learning

cultural
ICT supporting businesses and industry

business
processes

Manufacturing
ICT for trust and confidence
Future and Emerging
Technologies
Stronger, lasting growth
More & better jobs
Sustainability
FP7
A more attractive place to invest and work

Internal market

Improved regulation

Open competitive markets

Expand/improve infrastructure
Knowledge and innovation for growth

Increase R&D

Facilitate innovation, uptake of ICT and
sustainable use of resources

Contribute to strong industrial base
Creating more & better jobs

More employment & modernise social
protection

Adaptable workers & flexible labour markets

Better education & skills
Revised Lisbon
Information space
Open stable markets for electronic
communications & digital services economy
Innovation & investment in ICT
Deploy services

e government
Research leadership
Investment & improvement
Effective
adoption of ICT
Inclusion & better QoL

Knowledge society

Social development
i2010
ICT Technology Pillars
Nano-electronics, photonics and
integrated micro/nano-system
Ubiquitous and unlimited capacity
communication networks
Embedded systems, computing and
control
Software, Grids, security and
dependability
Knowledge, cognitive and learning
systems
Simulation, visualisation, interaction and
mixed realities
New perspectives in ICT drawing on other
science and technology disciplines
Integration of Technologies
Personal environment
Home environments
Robotic systems
Intelligent infrastructures
Applications Research
ICT meeting societal challenges

health

inclusion

mobility

environment

government
ICT for content, creativity and personal development

media

learning

cultural
ICT supporting businesses and industry

business
processes

Manufacturing
ICT for trust and confidence
Future and Emerging
Technologies
Stronger, lasting growth
More & better jobs
Sustainability
FP7
A more attractive place to invest and work

Internal market

Improved regulation

Open competitive markets

Expand/improve infrastructure
Knowledge and innovation for growth

Increase R&D

Facilitate innovation, uptake of ICT and
sustainable use of resources

Contribute to strong industrial base
Creating more & better jobs

More employment & modernise social
protection

Adaptable workers & flexible labour markets

Better education & skills
Revised Lisbon
Information space
Open stable markets for electronic
communications & digital services economy
Innovation & investment in ICT
Deploy services

e government
Research leadership
Investment & improvement
Effective
adoption of ICT
Inclusion & better QoL

Knowledge society

Social development
i2010
Overall linkage mapping can become very dense

Previous slide shows only a selection of
the more obvious linkages

While important not to lose the argument
of connected rationale must also be able
to examine elements of it in isolation to
allow more detailed arguments to be
developed
Different logics

Horizontal logic

Exploring interdependency between high level objectives or
between intermediate level actions (ie the proposals for FP7)
as per previous example

Vertical logic

Exploring interdependency between an objective and the
relevant parts of FP7

Can isolate as binary link, or

Consider combined effect of all aspects of the Programme on
that objective, or

Consider multiple effects on objectives of a single Programme
activity

Systemic logic

Considering implications of change across the whole system
Vertical logic – binary chain example

Pair of binary relationships

Overall Lisbon strategy sees increased
R&D as necessary condition for growth
by making business more innovative,
productive etc

Activity here is sponsorship of pre-
competitive R&D

Stated rationale for spending on
research rests on:


European industry lags in investment cf
major competitors”


More intensive cooperation makes most of
current capabilities”
Growth
Increase
R&D
ICT
Technology
Pillars
Expected impacts and outcomes en route to
increased R&D
ICT Technology
Pillars
Immediate
impacts
Intermediate
impacts
Ultimate
impacts
Activities
Collaborative
projects
Networks of
excellence
Joint
technology
initiatives
etc
New consortia
formed
Companies invest
own resources
Technology area
new to participants
More ambitious
projects
undertaken
papers IPR products services
New markets
entered
Increased
sales and cost
reductions
Increased R&D
Trained
researchers
Higher
return
on R&D
Resear
chers
employ
ed
Problems continued

Timing

Effects may happen long after research

Issues of discounting

Attribution

See previous graph – what complementary inputs
needed? Linearmodel issues

Appropriability

Who gets the benefit from a research project?

Spillovers
Inadequacy of linear model of
innovation

On the one hand we have research which is
addressed specifically to solving problems of
sustainability

Eg sustainable construction

May even be the defining characteristic of a whole
field

Eg environmental monitoring

On the other hand sustainable solutions may depend
upon advances in knowledge in areas of research
such as nano, bio, cognitive science and complexity
where initial aim was

general advancement of knowledge; or

pursuit of a different objective

What kind of translational model is needed?
Spillovers

Benefits accrue to agents other than the party
undertaking the research

Rationale for policy intervention social rate of return
exceeds inadequate private rate

Eg We may need to subsidise for research leading to green
technologies because market alone will not produce them

Jaffe describes 3 types

Knowledge spillovers

Knowledge created by one agent can be used by another
without compensation, or with compensation less than the value
of the knowledge

Eg reverse engineering, imitation

May be deliberate disclosure through publication or patent
Spillovers continued

Market spillovers

Market forces cause buyers of new product or
product made with new process to get some of the
benefit because not all superiority of price
reduction captured in price (consumer surplus)

Network spillovers

Arise when commercial or economic value of a
new technology is dependent upon developments
in related technologies

Eg communication systems

Firms may
fail to coordinate
their activities
without intervention
Nature of socio-economic effects
Intermediate

outputs

Products

Processes

Servic
es

Standards

Knowledge and
skills

Dissemination
Impacts/effects

Competitiveness

Employment

Organisation

Quality of life

Control & care of

the
environment

Regional development 

Development of

infrastructure

Production &

rational
use of

energy

Industrial

development

Regulation &

policy
Nature of Social Impact

Often public goods where government is the actual or
proxy customer

benefits not captured through market mechanism

paid through taxation

enforced through legislation/regulation

Not easily reduced to monetary equivalent

may not be social consensus on valuation of
quality of life

May be trade-off or complementary relation with
wealth creation

co-production of economic and social effects

Introduces new stakeholders to evaluation

social groups
Science and Quality of Life*
Development of
techniques to detect
problems

Environmental sensors,
health monitoring
equipment, security
systems


Development of
preventative measures

Cleaner technology,
healthier food

Development of methods
for remediation &
amelioration of problems


L
and reclamation,
medical treatment

Knowledge to inform
better policymaking

Climate change
forecasts, understanding
of science behind
regulations & standards


* with thanks to Ian Miles
Specific Environment Issues

Usually link to policy or regulation

May involve inter-temporal pricing

Some environmental changes are irreversible
with implications for future generations

May not be obvious who are the “end-users”

Problem if proxy-customers not fulfilling
duties
Impact gap
R&D
Effect
POLICY
MEASURE

(regulation,
law,
standard,
policy,
practice)
Evaluation

A
Evaluation

B
Practical solutions?

UK Government allocates block funding
for University research by means of an
ex post evaluation every 5-7 years

Previously called Research Assessment
Exercise

Allocated funds on basis of quality of
outputs (peer review); research
environment and esteem)

Next exercise renamed Research
Excellence Framework
REF and IMpact

Major innovation to include impact and government
insisting that it will count for 25% of score


Impact
: An assessment of demonstrable economic
and social impacts that have been achieved through
activity within the submitted unit that builds on
excellent research. This is to assess the extent to
which a submitted unit has built upon its strong record
of excellent research to make a positive impact on the
economy and society within the assessment period.
Throughout this document, where we refer to ‘impact’
or ‘social and economic impact’, we include economic,
social, public policy, cultural and quality of life
impacts. “
Method for assessment

One case study for every 5-10 people entered

Impact must be evident in the 2008-12 assessment
period but could come from research up to 15 years
before

Case study includes

explanation of the nature of the impact; how far-reaching it
is/who the beneficiaries are; and how significant the benefits
are

appropriate indicators of the impact

an outline of what the underpinning research was, when this
was undertaken and by whom

what efforts were made by staff in the unit to exploit or apply
the findings or secure the impact through its research
expertise

key research outputs that underpin the impact

external reports or documents, or contact details of a user,
that could corroborate the impact or the unit’s contribution
Indicators for impact on policy

Better informed public policy-making or improved public services

Research income from government organisations

Changes to legislation/regulations/government policy (including
references in relevant documents)

Changes to public service practices/guidelines (including
references in guidelines)

Measures of improved public services (for example, increased
literary and numeracy rates)

Staff exchanges with government organisations

Participation on public policy/advisory committees

Influence on public policy debate (for example, as indicated by
citations by non-government organisations or the media)

A framework for impact and
internalising spillovers

Understanding the relation between research and
innovation

How to transfer to more sustainable modes of
development

overcome lock-in to dominant sociotechnical regimes

Put in place infrastructures and policy/regulatory frameworks

Critical role for users and link to demand-side
innovation policies

Building constituencies including researchers

Coordination, engagement and building the capacities
and incentives for users to play a major role in the
process


Embedding in a Grand Challenge initiative more likely
to succeed?
Research and policy

Similar feedback loops to
innovation

Role of policy users

Risk of purchaser lock-in

Is there a need for
coordinated Grand
Challenge Initiatives to carry
forward SD research?
Conclusions

Linkages to innovation and policy play key role in
constructing agenda for SD research

Before measuring impact we must have
understanding of systemic model which links

Research and innovation

Research and policy

The pathways and complementary or intervening
factors may not be the same from one sector to
another

Without evaluation and measurement we work only on
theory and belief without the assurance that an
evidence base can give us

Evidence is needed not only for managers but to
reassure the stakeholders who fund and use research