Planning for Human-Robot Teaming - Subbarao Kambhampati

Cognitive

Science

1

Kartik

Talamadupula

Subbarao

Kambhampati

J. Benton


Dept. of Computer Science

Arizona State University

Paul
Schermerhorn

Matthias
Scheutz



Cognitive Science Program

Indiana University

Planning for

Human
-
Robot Teaming

Cognitive

Science

Motivation

2

•
Early motivation of AI

–
Autonomous control for


robotic agents

•
Plenty of applications

–
Household Assistance

–
Search and Rescue

–
Military Drones and Mules

•
All scenarios involve humans giving orders

•
Planning must co
-
opt this area

Cognitive

Science

Human
-
Robot Teaming

•
Teaming

–
Share the same goal(s)

–
Autonomous behavior

–
Communication

•
Role of Planning

–
Plan

generation

–
Feedback

acceptance

–
Model

resolution


HUMAN

ROBOT

PLANNER

Planning and Execution
Monitoring

Human Robot Interaction
(HRI)

Mixed Initiative Planning
(MIP)

What are the
factors

that planners must take into account?

Cognitive

Science

Dimensions

Scenario / Environment

•
Inspired by the real world

•
Large amounts of domain knowledge from

–
Humans with experience

–
Technical documents and manuals

•
New knowledge may arrive during execution

–
Planner must handle such contingencies

•
Planner and Robot Features

–
Determined by the needs of the scenario

–
E.g.: NASA needs temporal planning

Cognitive

Science

Dimensions

Robotic Agent

•
Central Actor

–
Execute actions

–
Gather sensory feedback

•
Different types of robots

–
Various capabilities

Gripper

Humanoid

Mobile

Combined

Cognitive

Science

Dimensions

Human User

•
Specifies and updates:

–
Scenario goals

–
Model (in some cases)

•
Must be in communication with robot/system

Novice

Uses the robot merely as
an assistant

Domain Expert

Authority on the
execution environment

System Expert

Authority on the
integrated AI system

Cognitive

Science

Planning

Goal Management

•
Human
-
Robot Teaming

–
Utility stems from delegation of goals

•
Support different types of goals

–
Temporal Goals: Deadlines

–
Priorities: Rewards and Penalties

•
Bonus Goals: Partial Satisfaction

–
Trajectory Goals

–
Conditional Goals

•
Changes to goals on the fly

–
Open World Quantified Goals

[Talamadupula et al., AAAI 2010]

Cognitive

Science

•
One true model of the world

–
Robot

•
High + Low Level models

–
Human User

•
Symbolic model + Add’l knowledge

–
Planner must take this gap into account

•
Model Maintenance v. Model Revision

–
Usability v. Consistency issues

–
Use the human user’s deep knowledge

•
Distinct Models

–
Using two (or more) models

•
Higher level: Task
-
oriented model

•
Lower level: Robot’s capabilities

Planning

Model Management

MODEL

Robot

Human

Cognitive

Science

HRT Tasks: Examples

SEARCH AND
REPORT

RECONNAISSANCE

KITCHEN ROBOT

ROBOT

Mobile

Mobile

Mobile and
Manipulator

HUMAN (USER)

Domain Expert

System Expert

Novice

MODEL

Less Dynamic

Dynamic

Highly Dynamic

GOALS

Evolving

Static

Evolving

COMMUNICATION

Natural Language

APIs

Natural Language

Feature

Task

Cognitive

Science

Case Study

Urban Search and Rescue

•
Human
-
Robot Team in Urban Setting

–
Find and report location of critical assets

–
Human: Domain expert; removed from the scene

SEARCH AND REPORT

•
Deliver medical supplies

•
Bonus Goal: Find and
report injured humans

•
Requirements

–
Updates to knowledge base

–
Goal changes

[Talamadupula et. al., AAAI 2010]


RECONNAISSANCE

•
Gather information

•
High risk to humans

–
E.g. Bomb defusal

•
Requirements

–
Support model changes

–
New capabilities

•
E.g.: Zoom camera

Cognitive

Science

Goal
Manager

Monitor

Planner

Plan

Plan

Problem

Updates

Updated State

Information

Initial

Model

Information

Sensory
Information

Actions

System Integration

Additional

Capabilities

Model

Update

Cognitive

Science

Model Update: Demo Run

12

•
Initial Goal


End of hallway

•
During Execution


Injured humans
(boxes) in rooms
behind doors


•
New action / effect during execution


Push doors to get inside rooms

Cognitive

Science

Conclusions

•
Human
-
Robot Teaming from a planning perspective

•
Planning Challenges

–
Framework for Human
-
Robot Teaming Problems

–
Model and Goal Management

•
Need to define the scope of planning


for these tasks

–
What are the main technical problems

•
Huge potential for novel P&S

applications

•
Companion Robots

•
Military and Service Drones

•
Household Assistants

Cognitive

Science

Future Work

•
Multiple Models

–
Use two (or more) models to direct the planning

•
Task v. Motion Level (BTAMP Workshops)

•
Classical v. More Expressive

•
Robotic Proactiveness

–
“Ask” for help

–
Many sources of knowledge in the real world

–
Putting the “teaming” in HRT

•
More Application Scenarios

–
Design planners sensitive to HRT issues