Measuring the Contribution of Sustainable Development Research to Society and Policy-Making

Measuring the Contribution of
Sustainable
Development Research to Society and
Policy-Making
Luke Georghiou
Manchester Institute of Innovation Research
Manchester Business School
Some initial trade-offs
•
Research as consumption good vs research
as an investment
–
In first case value is cultural, in second there are
users with the expectation of a return at some point
in the future
•
Outputs vs Impacts
–
Output may be scientific publication or even new
product or service but impact only realised when
output interacts with economy or society eg sales
or green technology or application of new
regulation
More trade-offs
•
Short term vs. long term
–
many different time-profiles over which impacts are
manifested including
•
short-term effects which terminate abruptly as market conditions
change
•
outputs which are not used for some years and then become
very important, perhaps because complementary technologies
have been developed
•
extrapolation is dangerous.
•
Intended vs. unintended
–
excessive focus on project or programme goals as the basis
for evaluation could lead to important unintended effects
being missed
•
eg in medical research when a drug developed for one condition
turns out to be important in the treatment of another
–
Unintended effects also include most of those in the negative
category.
INNOVATION
Problems in calculating returns to R&D
Timing
Attribution
Appropriability
RESEARCH
EFFECTS
ICT Technology Pillars
Nano-electronics, photonics and
integrated micro/nano-system
Ubiquitous and unlimited capacity
communication networks
Embedded systems, computing and
control
Software, Grids, security and
dependability
Knowledge, cognitive and learning
systems
Simulation, visualisation, interaction and
mixed realities
New perspectives in ICT drawing on other
science and technology disciplines
Integration of Technologies
Personal environment
Home environments
Robotic systems
Intelligent infrastructures
Applications Research
ICT meeting societal challenges
•
health
•
inclusion
•
mobility
•
environment
•
government
ICT for content, creativity and personal development
•
media
•
learning
•
cultural
ICT supporting businesses and industry
•
business
processes
•
Manufacturing
ICT for trust and confidence
Future and Emerging
Technologies
Stronger, lasting growth
More & better jobs
Sustainability
FP7
A more attractive place to invest and work
•
Internal market
•
Improved regulation
•
Open competitive markets
•
Expand/improve infrastructure
Knowledge and innovation for growth
•
Increase R&D
•
Facilitate innovation, uptake of ICT and
sustainable use of resources
•
Contribute to strong industrial base
Creating more & better jobs
•
More employment & modernise social
protection
•
Adaptable workers & flexible labour markets
•
Better education & skills
Revised Lisbon
Information space
Open stable markets for electronic
communications & digital services economy
Innovation & investment in ICT
Deploy services
•
e government
Research leadership
Investment & improvement
Effective
adoption of ICT
Inclusion & better QoL
•
Knowledge society
•
Social development
i2010
ICT Technology Pillars
Nano-electronics, photonics and
integrated micro/nano-system
Ubiquitous and unlimited capacity
communication networks
Embedded systems, computing and
control
Software, Grids, security and
dependability
Knowledge, cognitive and learning
systems
Simulation, visualisation, interaction and
mixed realities
New perspectives in ICT drawing on other
science and technology disciplines
Integration of Technologies
Personal environment
Home environments
Robotic systems
Intelligent infrastructures
Applications Research
ICT meeting societal challenges
•
health
•
inclusion
•
mobility
•
environment
•
government
ICT for content, creativity and personal development
•
media
•
learning
•
cultural
ICT supporting businesses and industry
•
business
processes
•
Manufacturing
ICT for trust and confidence
Future and Emerging
Technologies
Stronger, lasting growth
More & better jobs
Sustainability
FP7
A more attractive place to invest and work
•
Internal market
•
Improved regulation
•
Open competitive markets
•
Expand/improve infrastructure
Knowledge and innovation for growth
•
Increase R&D
•
Facilitate innovation, uptake of ICT and
sustainable use of resources
•
Contribute to strong industrial base
Creating more & better jobs
•
More employment & modernise social
protection
•
Adaptable workers & flexible labour markets
•
Better education & skills
Revised Lisbon
Information space
Open stable markets for electronic
communications & digital services economy
Innovation & investment in ICT
Deploy services
•
e government
Research leadership
Investment & improvement
Effective
adoption of ICT
Inclusion & better QoL
•
Knowledge society
•
Social development
i2010
Overall linkage mapping can become very dense
•
Previous slide shows only a selection of
the more obvious linkages
•
While important not to lose the argument
of connected rationale must also be able
to examine elements of it in isolation to
allow more detailed arguments to be
developed
Different logics
•
Horizontal logic
–
Exploring interdependency between high level objectives or
between intermediate level actions (ie the proposals for FP7)
as per previous example
•
Vertical logic
–
Exploring interdependency between an objective and the
relevant parts of FP7
•
Can isolate as binary link, or
•
Consider combined effect of all aspects of the Programme on
that objective, or
•
Consider multiple effects on objectives of a single Programme
activity
•
Systemic logic
–
Considering implications of change across the whole system
Vertical logic – binary chain example
•
Pair of binary relationships
•
Overall Lisbon strategy sees increased
R&D as necessary condition for growth
by making business more innovative,
productive etc
•
Activity here is sponsorship of pre-
competitive R&D
•
Stated rationale for spending on
research rests on:
–
“
European industry lags in investment cf
major competitors”
–
“
More intensive cooperation makes most of
current capabilities”
Growth
Increase
R&D
ICT
Technology
Pillars
Expected impacts and outcomes en route to
increased R&D
ICT Technology
Pillars
Immediate
impacts
Intermediate
impacts
Ultimate
impacts
Activities
Collaborative
projects
Networks of
excellence
Joint
technology
initiatives
etc
New consortia
formed
Companies invest
own resources
Technology area
new to participants
More ambitious
projects
undertaken
papers IPR products services
New markets
entered
Increased
sales and cost
reductions
Increased R&D
Trained
researchers
Higher
return
on R&D
Resear
chers
employ
ed
Problems continued
•
Timing
–
Effects may happen long after research
–
Issues of discounting
•
Attribution
–
See previous graph – what complementary inputs
needed? Linearmodel issues
•
Appropriability
–
Who gets the benefit from a research project?
–
Spillovers
Inadequacy of linear model of
innovation
•
On the one hand we have research which is
addressed specifically to solving problems of
sustainability
–
Eg sustainable construction
•
May even be the defining characteristic of a whole
field
–
Eg environmental monitoring
•
On the other hand sustainable solutions may depend
upon advances in knowledge in areas of research
such as nano, bio, cognitive science and complexity
where initial aim was
–
general advancement of knowledge; or
–
pursuit of a different objective
•
What kind of translational model is needed?
Spillovers
•
Benefits accrue to agents other than the party
undertaking the research
•
Rationale for policy intervention social rate of return
exceeds inadequate private rate
–
Eg We may need to subsidise for research leading to green
technologies because market alone will not produce them
•
Jaffe describes 3 types
–
Knowledge spillovers
•
Knowledge created by one agent can be used by another
without compensation, or with compensation less than the value
of the knowledge
•
Eg reverse engineering, imitation
•
May be deliberate disclosure through publication or patent
Spillovers continued
•
Market spillovers
–
Market forces cause buyers of new product or
product made with new process to get some of the
benefit because not all superiority of price
reduction captured in price (consumer surplus)
•
Network spillovers
–
Arise when commercial or economic value of a
new technology is dependent upon developments
in related technologies
–
Eg communication systems
–
Firms may
fail to coordinate
their activities
without intervention
Nature of socio-economic effects
Intermediate

outputs
•
Products
•
Processes
•
Servic
es
•
Standards
•
Knowledge and
skills
•
Dissemination
Impacts/effects
•
Competitiveness
•
Employment
•
Organisation
•
Quality of life
•
Control & care of

the
environment
•
Regional development 
•
Development of

infrastructure
•
Production &

rational
use of

energy
•
Industrial

development
•
Regulation &

policy
Nature of Social Impact
•
Often public goods where government is the actual or
proxy customer
–
benefits not captured through market mechanism
–
paid through taxation
–
enforced through legislation/regulation
•
Not easily reduced to monetary equivalent
–
may not be social consensus on valuation of
quality of life
•
May be trade-off or complementary relation with
wealth creation
–
co-production of economic and social effects
•
Introduces new stakeholders to evaluation
–
social groups
Science and Quality of Life*
Development of
techniques to detect
problems

Environmental sensors,
health monitoring
equipment, security
systems


Development of
preventative measures

Cleaner technology,
healthier food

Development of methods
for remediation &
amelioration of problems


L
and reclamation,
medical treatment

Knowledge to inform
better policymaking

Climate change
forecasts, understanding
of science behind
regulations & standards


* with thanks to Ian Miles
Specific Environment Issues
•
Usually link to policy or regulation
•
May involve inter-temporal pricing
•
Some environmental changes are irreversible
with implications for future generations
•
May not be obvious who are the “end-users”
•
Problem if proxy-customers not fulfilling
duties
Impact gap
R&D
Effect
POLICY
MEASURE

(regulation,
law,
standard,
policy,
practice)
Evaluation

A
Evaluation

B
Practical solutions?
•
UK Government allocates block funding
for University research by means of an
ex post evaluation every 5-7 years
–
Previously called Research Assessment
Exercise
–
Allocated funds on basis of quality of
outputs (peer review); research
environment and esteem)
•
Next exercise renamed Research
Excellence Framework
REF and IMpact
•
Major innovation to include impact and government
insisting that it will count for 25% of score
•
“
Impact
: An assessment of demonstrable economic
and social impacts that have been achieved through
activity within the submitted unit that builds on
excellent research. This is to assess the extent to
which a submitted unit has built upon its strong record
of excellent research to make a positive impact on the
economy and society within the assessment period.
Throughout this document, where we refer to ‘impact’
or ‘social and economic impact’, we include economic,
social, public policy, cultural and quality of life
impacts. “
Method for assessment
•
One case study for every 5-10 people entered
•
Impact must be evident in the 2008-12 assessment
period but could come from research up to 15 years
before
•
Case study includes
–
explanation of the nature of the impact; how far-reaching it
is/who the beneficiaries are; and how significant the benefits
are
–
appropriate indicators of the impact
–
an outline of what the underpinning research was, when this
was undertaken and by whom
–
what efforts were made by staff in the unit to exploit or apply
the findings or secure the impact through its research
expertise
–
key research outputs that underpin the impact
–
external reports or documents, or contact details of a user,
that could corroborate the impact or the unit’s contribution
Indicators for impact on policy
•
Better informed public policy-making or improved public services
•
Research income from government organisations
•
Changes to legislation/regulations/government policy (including
references in relevant documents)
•
Changes to public service practices/guidelines (including
references in guidelines)
•
Measures of improved public services (for example, increased
literary and numeracy rates)
•
Staff exchanges with government organisations
•
Participation on public policy/advisory committees
•
Influence on public policy debate (for example, as indicated by
citations by non-government organisations or the media)

A framework for impact and
internalising spillovers
•
Understanding the relation between research and
innovation
•
How to transfer to more sustainable modes of
development
–
overcome lock-in to dominant sociotechnical regimes
–
Put in place infrastructures and policy/regulatory frameworks
•
Critical role for users and link to demand-side
innovation policies
•
Building constituencies including researchers
•
Coordination, engagement and building the capacities
and incentives for users to play a major role in the
process

•
Embedding in a Grand Challenge initiative more likely
to succeed?
Research and policy
•
Similar feedback loops to
innovation
•
Role of policy users
–
Risk of purchaser lock-in
•
Is there a need for
coordinated Grand
Challenge Initiatives to carry
forward SD research?
Conclusions
•
Linkages to innovation and policy play key role in
constructing agenda for SD research
•
Before measuring impact we must have
understanding of systemic model which links
–
Research and innovation
–
Research and policy
•
The pathways and complementary or intervening
factors may not be the same from one sector to
another
•
Without evaluation and measurement we work only on
theory and belief without the assurance that an
evidence base can give us
•
Evidence is needed not only for managers but to
reassure the stakeholders who fund and use research