Operators Versus Automation

fanaticalpumaMechanics

Nov 5, 2013 (3 years and 10 months ago)

61 views

Operators Versus Automation

Andrew Rae

A couple of warm
-
up questions


Would you voluntarily accept a device added to your
car that detected the local speed limit and prevented
you exceeding that limit in all circumstances?


Do you believe that trains should be fitted with
vigilance devices which apply the brakes if the
driver isn’t aware/awake/conscious?

In different circumstances, people are willing to place
different levels of trust in automation.

Chernobyl 4 versus USS Thresher


Chernobyl: Automatic Shut
-
Down System was
disabled to allow special testing


RESULT:


Explosion and fire


CONCLUSION:

Operators
should not

be allowed to turn off



vital safety systems


USS Thresher: Automatic Shut
-
Down System
removed power at a time when propulsion power
was the only thing that could save the submarine


RESULT:


Loss of submarine with all crew


CONCLUSION:

Operators
should

be allowed to override vital



safety systems

So who are we going to trust?


For road vehicles: should speed limits be
automatically enforced, or should drivers have
ultimate control?


For military engines: should thermal limits be
permitted to be overridden using “battle shorts”?


For aircraft: should “alpha” (flight envelope)
protection be strict, or soft?


For railways: should a signal interlocking be flexible
in order to move trains out of a dangerous
situation?

The question of ultimate trust


In a given situation, should automation be designed
to prevent system states which the designers judge
to be dangerous, or should the interface provide
facility for the operator to execute any control at any
time?

What conventional hazard analysis has to say


Hazard: Operator error in performing manual task

Mitigation: Automation


Hazard: Automation Error

Mitigation: Operator in the loop


Hazard: Inappropriate operator intervention

Mitigation: Interlock


Hazard: Unforeseen circumstances

Mitigation: Interlock override


Hazard: Inappropriate use of override

Mitigation: ???

Conventional Hazard Analysis (continued)


Conventional hazard analysis does not cope well
with situations where:


Mitigations / Safety Requirements are in conflict


Hazards cannot be quantified and thus directly compared


The risk of human error can be quantified (but not
very accurately without application
-
specific
measurements)


The risk of automation error is very difficult to
quantify


The risk of unforeseen circumstances (unknown
-
unknowns) cannot be quantified


The Fitts’ List Approach

Humans are good at:


Ability to perceive patterns


Ability to improvise and use flexible procedure


Ability to reason inductively


Ability to exercise judgment

Fitts’ List Continued

Machines are good at:


Ability to respond quickly to control signals


Ability to perform repetitive tasks


Ability to do many things at once


Ability to perform complex calculations quickly and
accurately


The “Fitts’ List” approach isn’t perfect


In reality, humans and machines co
-
operate to
perform tasks


The performance of humans and machines is
shaped by environmental factors, and their
interaction with each other

A “performance shaping” approach


In a given situation, the performance of humans and
machines will be moderated by:


Whether the agent is innately suited to the task


The internal state of the agent


The ability of the agent to process inputs


The ability of the agent to decide upon outputs


The ability of the agent to execute outputs


We call each of these considerations “performance
shaping factors”

Performance shaping (continued)


Performance shaping is a common concept in
human
-
centered design


The idea that performance shaping factors can be
applied to machines is less frequently addressed

Performance Shaping Factors for Humans


Task demands and characteristics


Instructions and procedures


Displays and controls


Stresses


Individual skill and capacity


Individual condition


Socio
-
technical factors

Performance shaping factors for machines

INTERNAL

MODEL

RULE

SET

Assumptions

Inputs

Outputs

Performance shaping factors for machines


Incorrect Internal Model


Incorrect assumptions


Incorrect or insufficient inputs


Incorrectly Applied Rule Set


Incorrectly Specified Rule Set


Error in Implementing Rule Set


Error in Transmitting Outputs

Application


Automatic Train Operation


Train driving is a well
-
understood task


Most often performed by humans


Sometimes (e.g. Docklands Light Rail, Copenhagen
Metro) performed by computer


Sometimes performed by computer with a driver in
the cab “just in case”


A good candidate for testing our methodology


Application


Automatic Train Operation

Performance Shaping Factors
for Train Drivers


Task demands are moderate


Instructions and procedures
are generally clear and well
understood


Displays and controls can be
made simple and intuitive


Stresses


job has some
pressures, and also some
boredom


Driver condition can be a
factor


remember Waterfall

Performance Shaping Factors
for Train Computers


Task is well understood


Inputs are generally sufficient


note important exception
that drivers can see out of the
windscreen and computers
can’t


Safety
-
conscious industry /
development environment


Mechanical reliability is high



Automatic Train Operation
-

Conclusion


So should humans or computers be used to drive
trains?


Some supplementary points of interest


Adequate levels of safety and performance must be delivered in
a cost effective way


sometimes performance and cost will tip
the balance towards a particular solution


As automation increases, the drivers’ performance may
decrease


Lower situation awareness


Greater boredom


Greater stress when things do go wrong


There comes a point when things are so automated that the
driver lacks capacity to be an effective safeguard


Conclusion


It is important to make a conscious decision who (or
what) is responsible for making safety decisions


Ultimately, humans will be responsible


Either as operators, or as designers/maintainers of the system


The nature of the task, and the task environment,
dictate whether the operator or the automation is
best suited to making safety decisions


A sound argument can be formulated for assigning
the safety decision, based on performance shaping
factors