Integrated long-range UAV/UGV

bouncerarcheryΤεχνίτη Νοημοσύνη και Ρομποτική

14 Νοε 2013 (πριν από 3 χρόνια και 10 μήνες)

132 εμφανίσεις

Integrated
long
-
range UAV/UGV
collaborative target
tracking


Written by
:

Mark B. Moseley and Carol Cheung

iRobot Corp. (USA)

Benjamin P.
Grocholsky

and
Sanjiv

Singh

Carnegie Mellon Univ. (USA)

Referred by
:

Dr. Daisy Tang

Presented by
:

Subhobroto

Sinha

Discussion on


Them:

-
Tuesday 14 April 2009



Us:

-
Monday 28 November 2011

http://www.youtube.com/watch?v=2ozRJr10T1k


Discussion on


Them:

-
Tuesday 14 April 2009



Us:

-
Monday 28 November 2011

http://www.youtube.com/watch?v=2ozRJr10T1k


What’s changed since then?


What’s changed since then?

What’s changed since then?

What’s changed since then?

What’s changed since then?


Them:

-
Tuesday 14 April 2009



Us:

-
Monday 28 November 2011


http://www.youtube.com/watch?v=2ozRJr10T1k



Older

http://www.cis.upenn.edu/~
cjtaylor/PUBLICATIONS/pdfs/HsiehJFR07.pdf


http
://www.mendeley.com/research/cooperative
-
uavugv
-
platform
-
wildfire
-
detection
-
fighting
/



Newer

http://www.cc.gatech.edu/~cepippin/CUSTD_GTRI_TECHREPORT_112010.pdf

D
iagram
of the overall system architecture
.


iRobot and CMU under the U.S. Army ARDEC small
network lethality initiative are developing collaborative
capabilities for surveillance, targeting, and improved
communications based on the fielded
PackBot

UGV
and Raven UAV platforms.


As a demonstration of this capability we have
established a goal mission scenario of performing a
multi
-
agent, mixed
-
initiative search and pursuit of an
elusive target


In this paper, we present the current progress towards
this goal


iRobot and CMU under the U.S. Army ARDEC small
network lethality initiative are developing

collaborative
capabilities for surveillance, targeting, and improved
communications
based on the fielded
PackBot

UGV
and Raven UAV platforms.



As a demonstration of this capability we have
established a goal mission scenario of performing a
multi
-
agent, mixed
-
initiative search and pursuit of an
elusive target


In this paper, we present the current progress towards
this goal



iRobot and CMU under the U.S. Army ARDEC small
network lethality initiative are developing

collaborative
capabilities for surveillance, targeting, and improved
communications
based on the fielded
PackBot

UGV
and Raven UAV platforms.



As a demonstration of this capability we have
established a

goal mission scenario of performing a
multi
-
agent, mixed
-
initiative search and pursuit of an
elusive target


In this paper, we present the current progress towards
this goal


iRobot and CMU under the U.S. Army ARDEC small
network lethality initiative are developing
collaborative
capabilities for surveillance, targeting, and improved
communications

based on the fielded
PackBot

UGV
and Raven UAV platforms.


As a demonstration of this capability we have
established a goal mission scenario of
performing a
multi
-
agent, mixed
-
initiative search and pursuit of an
elusive target


In this paper,
we present the current progress towards
this goal



iRobot and CMU under the U.S. Army ARDEC small
network lethality initiative are developing
collaborative
capabilities for surveillance, targeting, and improved
communications

based on the fielded
PackBot

UGV
and Raven UAV platforms.


As a demonstration of this capability we have
established a goal mission scenario of
performing a
multi
-
agent, mixed
-
initiative search and pursuit of an
elusive target



In this paper,
we present the current progress towards
this goal


Decentralized Data Fusion



A
novel technique for fusing track estimates
from
PackBot

and Raven platforms for a
moving target in an open
environment

Decentralized Data Fusion



A
novel technique for fusing track estimates from
PackBot

and Raven platforms for a moving target
in an open
environment


In
addition, system integration with
AeroVironment's

Digital Data Link onto both air
and ground platforms has extended our
capabilities in communications range to operate
the
PackBot

as well as in increased video and
data throughput.

AeroVironment’s

new Digital Data Link
(DDL)



The DDL provides a low
-
latency, bi
-
directional, high
throughput (>4 Mbps) digital
link


In
addition to video, command, and telemetry channels
the DDL
offers an Ethernet bridge capability that allows
it to be used as a relay for the UGVs in the
system


Use
of the DDL to relay a UGV command link through a
UAV allows the UGV to operate beyond line of sight
and at distances that significantly exceed the current
performance of the
PackBot’s

integrated 802.11
communications system.


AeroVironment’s

new Digital Data Link
(DDL)



The DDL provides a low
-
latency, bi
-
directional, high
throughput (>4 Mbps) digital
link


In
addition to video, command, and telemetry channels
the DDL
offers an Ethernet bridge capability that allows
it to be used as a relay for the UGVs in the
system


Use
of the DDL to
relay a UGV command link through a
UAV allows the UGV to operate beyond line of sight
and at distances that significantly exceed the current
performance

of the
PackBot’s

integrated 802.11
communications system.


AeroVironment’s

new Digital Data Link
(DDL)



The DDL provides a low
-
latency, bi
-
directional, high
throughput (>4 Mbps) digital
link


In
addition to video, command, and telemetry channels
the DDL offers an Ethernet bridge capability that allows
it to be used as a relay for the UGVs in the
system


Use
of the DDL to relay a UGV command link through a
UAV allows the UGV to operate beyond line of sight
and at distances that significantly
exceed

the current
performance of the
PackBot’s

integrated 802.11
communications system
.



Operator Control Unit



In our mixed
-
iniative

approach autonomy
versus user control toggles dynamically based
on the situation and task at hand


Operator Control Unit



In our mixed
-
iniative

approach

autonomy
versus user control toggles dynamically
based
on the situation and task at hand


Operator Control Unit



In our mixed
-
iniative

approach

autonomy
versus user control toggles dynamically
based
on the situation and task at
hand


Our goal is to enable a force multiplying
human/robot interaction by maximizing
situational awareness and ease of control
through the fusion of sensory information
from multiple assets.


Operator Control Unit



In our mixed
-
iniative

approach

autonomy
versus user control toggles dynamically
based
on the situation and task at
hand


Our goal is to enable a

force multiplying
human/robot interaction by maximizing
situational awareness and ease of control
through the fusion of sensory information
from multiple assets
.



Command, Control And
Communications




We integrated disparate communication protocols into a unified
scheme to support collaboration
.


The integration entailed translating messages from the UAV’s
Cursor on Target protocol and from the UGV’s Tactical Mobile Robot
(TMR) protocol to a common iRobot Aware 2.0 message protocol
on the
OCU


Cursor on Target is an extensible XML
-
based messaging protocol
commonly used on UAV and other military systems including
AeroVironment’s

Raven UAV and
CDAS


We also integrated a third protocol, the University of West Virginia
Protocol used by CDAS, which is a simple low
-
overhead messaging
format for integrating general semi
-
autonomous platforms with the
CDAS mission command and control node



Command, Control And
Communications




We
integrated disparate communication protocols into a unified
scheme

to support collaboration
.


The integration entailed
translating messages from the UAV’s
Cursor on Target protocol and from the UGV’s Tactical Mobile Robot
(TMR) protocol to a common iRobot Aware 2.0 message protocol

on the
OCU


Cursor on Target is an extensible XML
-
based messaging protocol
commonly used on UAV and other military systems including
AeroVironment’s

Raven UAV and
CDAS


We also integrated a third protocol, the University of West Virginia
Protocol used by CDAS, which is a simple low
-
overhead messaging
format for integrating general semi
-
autonomous platforms with the
CDAS mission command and control node



Command, Control And
Communications




We
integrated disparate communication protocols into a unified
scheme

to support collaboration
.


The integration entailed
translating messages from the UAV’s
Cursor on Target protocol and from the UGV’s Tactical Mobile Robot
(TMR) protocol to a common iRobot Aware 2.0 message protocol

on the
OCU


Cursor on Target is an
extensible XML
-
based messaging protocol
commonly used on UAV and other military systems

including
AeroVironment’s

Raven UAV and
CDAS


We also integrated a third protocol, the University of West Virginia
Protocol used by CDAS, which is a simple low
-
overhead messaging
format for integrating general semi
-
autonomous platforms with the
CDAS mission command and control node



Command, Control And
Communications




We
integrated disparate communication protocols into a unified
scheme

to support collaboration
.


The integration entailed
translating messages from the UAV’s
Cursor on Target protocol and from the UGV’s Tactical Mobile Robot
(TMR) protocol to a common iRobot Aware 2.0 message protocol

on the
OCU


Cursor on Target is an
extensible XML
-
based messaging protocol
commonly used on UAV and other military systems

including
AeroVironment’s

Raven UAV and
CDAS


We also integrated
a third protocol, the University of West Virginia
Protocol used by CDAS, which is a simple low
-
overhead messaging
format for integrating general semi
-
autonomous platforms

with the
CDAS mission command and control node



Command, Control And
Communications




We integrated disparate communication protocols into a unified
scheme to support collaboration
.


The integration entailed translating messages from the UAV’s
Cursor on Target protocol and from the UGV’s Tactical Mobile Robot
(TMR) protocol to a common iRobot Aware 2.0 message protocol
on the
OCU


Cursor on Target is an extensible XML
-
based messaging protocol
commonly used on UAV and other military systems including
AeroVironment’s

Raven UAV and
CDAS


We also integrated a third protocol, the University of West Virginia
Protocol used by CDAS, which is a simple low
-
overhead messaging
format for integrating general semi
-
autonomous platforms with the
CDAS mission command and control node

Search, Target Tracking, Geo
-
Location,
And Pursuit.



In order to guide the development of a
collaborative UAV
-
UGV framework, we
established the goal of creating a system to
perform search and pursuit of an evasive
target.

Search, Target Tracking, Geo
-
Location,
And Pursuit.



In order to guide the development of a
collaborative UAV
-
UGV framework, we
established the goal of creating a system to
perform search and pursuit of an evasive
target.

Search, Target Tracking, Geo
-
Location,
And Pursuit.



In order to guide the development of a
collaborative UAV
-
UGV framework, we
established the goal of creating a system to
perform search and pursuit of an evasive
target.

Search, Target Tracking, Geo
-
Location,
And Pursuit.



While the UAV is searching, any search
conducted by the UGV will be through
teleoperation


If
a target is selected from the UAV search, the
UGV will autonomously navigate to the target
area and notify the operator when it has
arrived at the
target


It
is the operator’s responsibility to then
locate the target with the UGV

Search, Target Tracking, Geo
-
Location,
And Pursuit.



While the UAV is searching,
any search
conducted by the UGV will be through
teleoperation


If
a target is selected from the UAV search, the
UGV will autonomously navigate to the target
area and notify the operator when it has
arrived at the
target


It
is the operator’s responsibility to then
locate the target with the UGV

Search, Target Tracking, Geo
-
Location,
And Pursuit.



While the UAV is searching,
any search
conducted by the UGV will be through
teleoperation


If
a target is selected from the UAV search,
the
UGV will autonomously navigate to the target
area and notify the operator

when it has
arrived at the
target


It
is the operator’s responsibility to then
locate the target with the UGV

Search, Target Tracking, Geo
-
Location,
And Pursuit.



While the UAV is searching,
any search
conducted by the UGV will be through
teleoperation


If
a target is selected from the UAV search,
the
UGV will autonomously navigate to the target
area and notify the operator

when it has
arrived at the
target


It
is the
operator’s responsibility to then
locate the target with the UGV

Search, Target Tracking, Geo
-
Location,
And Pursuit.



While the UAV is searching, any search
conducted by the UGV will be through
teleoperation


If
a target is selected from the UAV search, the
UGV will autonomously navigate to the target
area and notify the operator when it has
arrived at the
target


It
is the operator’s responsibility to then
locate the target with the UGV


Visual Target Tracking



The Raven UAV provides an MPEG
-
2 video
stream to the OCU
.


One difficulty is tracking a target in
transmitted video streams that are
compressed,
lossy
, and
corrupted


Another significant problem is the large
unexpected camera movements encountered
by a lightweight UAV flying in windy
conditions


Visual Target Tracking



The Raven UAV
provides an MPEG
-
2 video
stream

to the OCU
.


One difficulty is tracking a target in
transmitted video streams that are
compressed,
lossy
, and
corrupted


Another significant problem is the large
unexpected camera movements encountered
by a lightweight UAV flying in windy
conditions


Visual Target Tracking



The Raven UAV
provides an MPEG
-
2 video
stream

to the OCU
.


One
difficulty is tracking a target in
transmitted video streams

that are
compressed,
lossy
, and
corrupted


Another significant problem is the large
unexpected camera movements encountered
by a lightweight UAV flying in windy
conditions


Visual Target Tracking



The Raven UAV
provides an MPEG
-
2 video
stream

to the OCU
.


One
difficulty is tracking a target in
transmitted video streams

that are
compressed,
lossy
, and
corrupted


Another significant problem is the

large
unexpected camera movements

encountered
by a lightweight UAV flying in windy
conditions


Visual Target Tracking



The Raven UAV provides an MPEG
-
2 video
stream to the OCU
.


One difficulty is tracking a target in
transmitted video streams that are
compressed,
lossy
, and
corrupted


Another significant problem is the large
unexpected camera movements encountered
by a lightweight UAV flying in windy
conditions


Geo
-
location



Achieving accurate location estimates is
challenging on small UAV platforms
.


The UGV has a localization system that takes as
input track
odometry
, 3D accelerometer values,
3D angular rate gyro information, and GPS
signals
.


Bias corrected gyro information and
accelerometer data is input into an Unscented
Kalman

Filter which estimates the direction of
gravity.


Geo
-
location



Achieving accurate location estimates is
challenging on small UAV platforms
.


The UGV has a localization system that takes as
input track
odometry
, 3D accelerometer values,
3D angular rate gyro information, and GPS
signals
.


Bias corrected gyro information and
accelerometer data is input into an Unscented
Kalman

Filter which estimates the direction of
gravity.


Geo
-
location



Achieving accurate location estimates is
challenging on small UAV platforms
.


The UGV has a localization system that takes as
input track
odometry
, 3D accelerometer values,
3D angular rate gyro information, and GPS
signals
.


Bias corrected gyro information and
accelerometer data is
input into an Unscented
Kalman

Filter

which estimates the direction of
gravity.


Geo
-
location



Achieving accurate location estimates is
challenging on small UAV platforms
.


The UGV has a localization system that takes as
input track
odometry
, 3D accelerometer values,
3D angular rate gyro information, and GPS
signals
.


Bias corrected gyro information and
accelerometer data is
input into an
Unscented
Kalman

Filter

which estimates the direction of
gravity.


Geo
-
location



Achieving accurate location estimates is
challenging on small UAV platforms
.


The UGV has a localization system that takes as
input track
odometry
, 3D accelerometer values,
3D angular rate gyro information, and GPS
signals
.


Bias corrected gyro information and
accelerometer data is
input into an
Unscented
Kalman

Filter

which estimates the direction of
gravity.

Unscented
Kalman

Filter


Unscented
Kalman

Filter


http://
en.wikipedia.org/wiki/Kalman_filter#Unscented_Kalman_filter



http://
stomach.v2.nl/docs/TechPubs/Tracking_and_AR/wan01unscented.pdf



http://
signal.hut.fi/kurssit/s884221/ukf.pdf



http://www.cs.brown.edu/~jjl/pubs/laviola_acc2003.pdf


Decentralized Data Fusion (DDF)


A fully decentralized architecture is necessary for a large
number of agents on the order of tens to hundreds of
agents, but the drawback is that the solutions are
constrained to local optima rather than reaching the global
optimum.


A robust, versatile approach which supports modularity and
fast reconfiguration is a mixed centralized
-
decentralized
mission planning and task allocation
system


The
OCU within the system acts as the high
-
level
centralized mission planner with vehicle agents taking on
low
-
level tasks in a decentralized, and hence scalable,
fashion.


Decentralized Data Fusion (DDF)


A fully decentralized architecture is necessary for a large
number of agents on the order of tens to hundreds of
agents, but the
drawback is that the solutions are
constrained to local optima

rather than reaching the global
optimum.


A robust, versatile approach which supports modularity and
fast reconfiguration is a mixed centralized
-
decentralized
mission planning and task allocation
system


The
OCU within the system acts as the high
-
level
centralized mission planner with vehicle agents taking on
low
-
level tasks in a decentralized, and hence scalable,
fashion.


Decentralized Data Fusion (DDF)


A fully decentralized architecture is necessary for a large
number of agents on the order of tens to hundreds of
agents, but the
drawback is that the solutions are
constrained to local optima

rather than reaching the global
optimum.


A robust, versatile approach which supports modularity and
fast reconfiguration is

a mixed centralized
-
decentralized
mission planning and task allocation
system


The
OCU within the system acts as the high
-
level
centralized mission planner with vehicle agents taking on
low
-
level tasks in a decentralized, and hence scalable,
fashion.


Decentralized Data Fusion (DDF)


A fully decentralized architecture is necessary for a large
number of agents on the order of tens to hundreds of
agents, but the
drawback is that the solutions are
constrained to local optima

rather than reaching the global
optimum.


A robust, versatile approach which supports modularity and
fast reconfiguration is

a mixed centralized
-
decentralized
mission planning and task allocation
system


The
OCU within the system acts as the high
-
level
centralized mission planner

with vehicle agents taking on
low
-
level tasks in a decentralized, and hence scalable,
fashion.


Decentralized Data Fusion (DDF)


A fully decentralized architecture is necessary for a large
number of agents on the order of tens to hundreds of
agents, but the
drawback is that the solutions are
constrained to local optima

rather than reaching the global
optimum.


A robust, versatile approach which supports modularity and
fast reconfiguration is

a mixed centralized
-
decentralized
mission planning and task allocation
system


The
OCU within the system acts as the high
-
level
centralized mission planne
r with
vehicle agents taking on
low
-
level tasks in a decentralized, and hence scalable,
fashion
.


Decentralized Data Fusion (DDF)


A
network of sensing nodes, each with its own
processing facility, which together do not require
any central fusion or central communication
facility
.


The decentralized estimation network maintains
representations of states relevant to surveillance
tasks. Including, target detection probability
distribution over the search area for search
applications and Gaussian target position and
velocity for geo
-
location and pursuit.


Decentralized Data Fusion (DDF)


A
network of sensing nodes, each with its own
processing facility, which together do not require
any central fusion or central communication
facility
.


The decentralized estimation network maintains
representations of states relevant to surveillance
tasks. Including, target detection probability
distribution over the search area for search
applications and Gaussian target position and
velocity for geo
-
location and pursuit.


Decentralized Data Fusion (DDF)


As shown in Figure 10, each UAV, UGV and
OCU entity acts as a decentralized estimation
and control node within the system network
.


In
the case of the Raven UAV, vision
processing and task control are run at the OCU
due to limited UAV payload capacity.

Decentralized Data Fusion (DDF)


As shown in Figure 10,
each UAV, UGV and
OCU entity acts as a decentralized estimation
and control node within the system network
.


In
the case of the Raven UAV,
vision
processing and task control are run at the OCU
due to limited UAV payload capacity
.

Decentralized Data Fusion (DDF)


Ground based processing of transmitted video
data using existing offline algorithms will be
the first step in establishing the capability
achievable using the stock Raven platform
.


Parallax
, seasonal and lighting differences,
image quality and corruption due to the video
data downlink all pose significant challenges
to this goal.

Decentralized Data Fusion (DDF)


Ground based processing of transmitted video
data using existing offline algorithms will be
the first step
in establishing the capability
achievable using the stock Raven platform
.


Parallax
, seasonal and lighting differences,
image quality and corruption due to the video
data downlink

all pose significant challenges
to this goal.

Decentralized Data Fusion (DDF)


Ground based processing of transmitted video
data using existing offline algorithms will be
the first step
in establishing the capability
achievable using the stock Raven platform
.


Parallax
, seasonal and lighting differences,
image quality and corruption due to the video
data downlink

all pose significant challenges
to this goal.