Simon French, University of Warwick, UK

brewerobstructionAI and Robotics

Nov 7, 2013 (3 years and 7 months ago)


Simon French, University of Warwick, UK

Expert Judgment, Meta
Analysis and Participatory Risk Analysis

There are three contexts in which one might wish to combine expert judgments of uncertainty: the
expert problem, the group decision problem and the textbook problem. Much has been written on
the first two, which have the focus of a single decision context,

but little on the third. The textbook
problem arises when one needs to draw together expert judgments into a decision analysis when
their judgments were made originally in a context free manner or perhaps for other decision
contexts. In many ways the tex
tbook problem parallels that of performing a meta
analysis of
empirical studies. However, there are differences. In this paper, we discuss those difficulties and
then focus on two closely related issues: how should expert judgment studies be published so a
s to
facilitate subsequent meta
analyses and how should such meta
analyses be performed?

Yee Mey Goh, University of Loughborough, UK

Uncertainty in Competitive Bidding

Supporting Pricing Decision with Probabilistic Models

The suppliers of long
life produ
cts such as submarines and airplanes no longer simply sell these
products but provide their capability or availability. This means that companies that traditionally
design and manufacture long
life products now compete through the provision of a service. T
companies face a high level of uncertainty due to the novelty of the process of designing and
managing the contract and the long
term nature of services. As a consequence, it is difficult for
companies to determine an appropriate price bid for the ser
vice, which will enable them to win the
contract as well as make a profit. The research aims at supporting these companies in their pricing
decision process by providing a tool that models the uncertainties at the competitive bidding stage
and depicts thei
r influence on the probability of winning the contract and the probability of making a
profit. An industrial case study will be described to demonstrate the approach.

Oswaldo Morales Napoles & Anca Hanea, TU Delft, Netherlands

Conditional rank correlation
elicitation exercise

Dependence measures such as rank correlations are commonly used in different types of models.
Whenever field data is available these may be estimated directly from data. However, field data is
often not available and structured expert
judgment is sought instead.

Methods for eliciting rank correlations from experts have been proposed in the past (Cooke &
Goossens 1999), (Clemen et al. 1999). One option is to ask experts directly for an estimate of the
rank correlation between pairs of v
ariables. Another option is to acquire estimates of other related
quantities, e.g. conditional probabilities of exceedance, probabilities of concordance or discordance.
The rank correlations of interest could be then estimated using these quantities. Prev
ious results
indicate that an
accurate way to obtain a subjective measure of bivariate dependence is by simply
asking the experts to estimate the correlation between the two variables in question (
Clemen and et
al 2000

Recently non
parametric Bayesian
networks have been introduced as flexible tools for high
dimensional dependence modelling (Hanea et al. 2006). The input for these models consists of
univariate marginal distributions and (conditional) rank correlations. In absence of field data
al) rank correlations have been assessed from experts through the elicitation of
conditional probabilities of exceedance or ratios of rank correlations (Morales Napoles et al. 2008).
To our knowledge, there is no experimental study available that would, at

least, give some indication
about the accuracy of estimating conditional rank correlations from experts. Even less is known
about the accuracy of one method with respect to the other. In this study we describe data collected
from a controlled exercise. Th
e question of interest is whether experts can estimate more accurately
conditional rank correlations through estimates of conditional probabilities of exceedence or
through ratios of rank correlation coefficients.

Tim Bedford, University of Strathclyde, U

The Moment Model Alternative to Cooke’s Classical Model

Quantitative expert judgement is used in many areas of risk analysis to provide assessments

uncertainty. A leading method is Cooke’s classical model, which provides a way of
eighting experts
depending on their performance in answering so
called calibration questions. The moment method
for expert judgement combination is an alternative to Cooke’s method which provides a different
way of calibrating experts. It has better theoretical properties
in that it uses a strictly proper scoring
rule, that is, an expert who wishes to maximise his score can only do so by stating what he actually
believes. The method also requires the specification of weights for the calibration process. In this
paper we con
sider a special case of the moment model that scores location and variability
assessments. We provide an approach to setting the calibration weights and repeat the comparison
of this moment model with Cooke’s classical model we conducted earlier for this
special case.

Lesley Walls, University of Strathclyde, UK

Importance of the Process for Expert Judgement Elicitation

Drawing upon experience of eliciting probabilistic engineering judgement to support reliability
modelling of industrial problems, this tal
k will examine key challenges and suggest ways of designing
a robust elicitation process. An overview of our elicitation experience space will be given to provide
a frame of reference. Examples of how common biases manifest themselves within technical ris
k and
reliability applications will be described. Perceptions of engineers and managers about the role of
probabilistic expert judgement in reliability modelling will be shared. The important role that
theoretical frameworks play as the basis for an elic
itation process design will be discussed.

Simon Blake, Durham University, UK

Calculating a Health Index to make decisions on asset replacement

A Distribution Network Operator in the electricity industry manages a large portfolio of ageing
assets, with an annual budget for replacing those in the worst state. This is determined by allocating
a health index (HI) to each asset, on a scale of 0 to 10,

which takes into account a diverse range of
factors including asset age, location, utilisation, inspection data and laboratory analysis. Weighting
and balancing these factors is done using a complex set of formulae, derived by incorporating expert
t as far as possible. How this is done, and the results and consequences, constitutes an
informative insight into the industrial decision
making process.

Taposhri Ganguly, University of Strathclyde, UK

Mathematical Aggregation of Probabilistic expert jud