Introduction to OpenRTM

pillowfistsAI and Robotics

Nov 13, 2013 (3 years and 10 months ago)

73 views

Introduction to OpenRTM

National Institute of Advanced Industrial Science and Technology

Intelligence Systems Research Institute

Noriaki Ando

Outline


Introduction


History


Features


Applications

2


RT = Robot Technology
cf. IT


Not only standalone robots, but also robotic elements
(sensors, actuators, etc….)







RT
-
Middleware (RTM)


Middleware and platform for RT
-
element integration


RT
-
Component (RTC)


Basic software unit in RT
-
Middleware

RT
-
Middleware

+

+

+

+

+

What is RT?

OpenRTM
-
aist

Open
(source)

RT
-
M
iddlware implemented by

AIST

4

The aim of RT
-
Middleware

Problem Solving by Modularization


仕様の明確化


最新技術を容易に利用可能


誰でもロボットが作れる

Realize

low
-
cost robots

Satisfy

various needs

Cost

Technical Issue

Needs









The state of
the art

Mobile base

Manipulator

Sensors
・・・

Various users

System developers

Easy to customize

RT
-
Component

Utilize

the state of the art

Realizing innovation in service robot market

Reusable modules

Spec.

RT
-
Middleware & RT
-
Components

5

RT

Component

Framework

RT

Component

Logic

Logic/algorithm with common interfaces = RT
-
Component

RTC


RT
-
Middleware

RTC

RTC

RTC

RTC

RTC

RTC

RTC

RTC

Execution environment for
RTC

RT
-
Middleware

RTM



RTCs can be distributed across a network


device control


algorithms


applications

etc…


History of OpenRTM
-
aist

6

Takumi (

), Ume (

)

from 1996


in ETL (Electrotechnical Laboratory, former AIST)



(Takumi = meister (de))


Transputer based multi
-
agent
system for manipulation


Language: C, Occam and Euslisp


Network: Transputer (real
-
time)




(Ume = plum (en))


Ethernet based Takumi


Language: C, C++, Euslisp


Interface definition: original definition
language similar to Occam


Network: TCP/IP


OS: Solaris, Windows, VxWorks

7

Another Takumi?

8

Ramen (n
oo
dle) shop Takumi

Heßstr.71 (ecke Schleisheimerstr.)

80798 München

Takumi

Here

TUM

2008

2009

2010

2005

2006

2007

2002

2003

2004

2011

RT
-
Middleware Project

Distributed component based simulator

Key technology for next gen. robot PJ

AIST UCROA PJ

Kaken PJ

AIST standardization PJ

Robot element development PJ

Universal design for environment and manipulation PJ


Intelligent RT Software Project

RTM0.1

release

OpenRTM
-
aist

0.2.0 release

OpenRTM
-
aist

0.4.0 release

OMG RTC

spec. approved

FY

OMG RTC

spec. draft

OMG RTC
spec. release

OpenRTM
-
aist

1.0

release

OpenRTM
-
aist is used in many

national projects for robotics

research as a standard platform

Robot award 2007

History of RT
-
Middleware

from 2002


RT
-
Middleware features


Component
-
based software development


Execution contexts, ports, configuration, etc…


Inter
-
OS/language communication


Standards


Open specification


Specification continuity


Many vendors with RTM implementations


Various PSM implementations


Implementation continuity



10

Basic functions of an RTC

11

Inactive

Active

Error

Activity/Execution contexts

Manage RTC’s lifecycle and execution

Common state machine

Sensor RTC

Composite execution

Contoller RTC

Actuator RTC

Encorder

component

Actuator

Component

Controller

Component


1


T
I
s

T
D
s

Kp

+

-

Ref

Position

Position

Voltage

Data ports


Data
-
centric port


Continuous data I/O


Dynamic connection

Data centric communication

ex. servo


Service
-
oriented port


Provides detailed function


Getting/setting parameters


Changing modes


etc…

Service ports

ex. Stereo vision

Service oriented communication

key

value

Set name

key

value

Set name

An RTC can have several

configuration sets.

Runtime reconfiguration

and dynamic switching

are supported

Configuration


Parameter management


Configuration sets


Dynamic configuration

Stereo Vision

Interface


Mode setting


Coordinate setting


Calibration


etc…

Image

Data

3D depth

data

Service Port

Stereo vision

Component

Data Port

12

Standardization in OMG


OMG

Object Management Group



Software standardization organization


Ex. UML, CORBA, DDS specifications




MDA (Model Driven Architecture)


Improves system life
-
time and executable variable (quality x cost x life
-
time).


The model is described in a PIM, and systems are implemented based
on PSMs that are mapped automatically (or semi
-
automatically) from
the PIM.


PIM (Platform Independent Model)


A model independent of any target platform (CORBA, EJB, SOAP, etc.).
The model is described in a modelling language such as UML.


PSM (Platform Specific Model)


A model converted from a PSM for each target platform


CORBA PSM, SOAP PSM, etc.

OMG RTC Standard

formal/08
-
04
-
04


OMG RTC specification defines


Basic component interface


Execution semantics


Introspection


OMG RTC specification inherits SDO (Super distributed Object)


Specification available online:
http://www.omg.org/spec/RTC/


Version 1.1 due soon

13

pd RTC Packages
«profi l e»
RTC
(from Roboti c Technol ogy Components)
Execution Semantics
Introspection
Lightweight RTC
SDOPackage
(from External Model s)
14

Features of RTC


Provides
rich component lifecycle
to enforce
state coherency among components


Defines data structures for describing
components and other elements


Supports fundamental design patterns


Collaboration of fine
-
grained components tightly
coupled in time (
e.g
. Simulink)


Stimulus response with finite state machines


Dynamic composition of components

collaborating
synchronously or asynchronously


OMG RTC families

15

Name

Vendor

Feature

OpenRTM
-
aist

AIST

C++, Python, Java

OpenRTM.NET

SEC

.NET(C#,VB,C++/CLI, F#, etc..)

miniRTC, microRTC

SEC

RTC for embedded systems using CAN

Dependable RTM

SEC/AIST

Certified RTM for safety
-
critical systems

RTC CANOpen

SIT, CIA

Based on CANOpen standard and RTC
standard

PALRO

Fujisoft

C++ PSM impl. for small humanoid robot

OPRoS

ETRI

Korean national project

GostaiRTC

GOSTAI,
THALES

Based on C++ PSM running on URBI


Implementation diversity of the same specification


Guarantees continuity of products


Makes reuse between implementations easier

Applications


Applications


Robots


DAQ system


Project



Intelligent RT Software Project

16

Success stories

17

HRP
-
4: Kawada/AIST

DAQ
-
Middleware: KEK/J
-
PARC

KEK: High Energy Accelerator Research Organization

J
-
PARC: Japan Proton Accelerator Research Complex

HRP
-
4C: Kawada/AIST

TAIZOU: General Robotics Inc.

HIRO: Kawada/GRX

物質・生命科学実験施設
Materials and Life Science

Experimental Facility

ハドロン実験施設
䡡摲潮 䉥慭B䙡捩汩Fy

ニュートリノ
実験施設
乥畴物湯r瑯
䭡浩K歡湤e

核変換施設
乵捬敡爠
T牡湳浵r慴楯n

Japan Proton Accelerator Research Complex


J
-
PARC,
大強度陽子加速器施設)

J
-
PARC = Japan Proton Accelerator Research Complex

Joint Project between KEK and JAEA (former JAERI)

50 GeV Synchrotron

(0.75 MW)

3 GeV Synchrotron

(25 Hz, 1MW)

Linac

(350m)


500 m

© KEK

19

IROHA

DAQ System

DAQ Middleware

Gatherer

Dispatcher

Logger

Monitor

DAQ
Operator

SiTCP

SiTCP

SiTCP

SiTCP

SiTCP

SiTCP

SiTCP

Web

Server

デバイス
管理

XML/HTTP

Begin

Configure

End

DAQ

Status

Working

Desktop

End

Status

Begin

User

module

Common

module

© KEK

Communication Perfomance

Source

Sink

Ethernet

Sink

Source

Repeater

Ethernet

Model : Dell PowerEdge SC1430

CPU :Intel Xeon 5120 @ 1.86GHz 2 Cores
×
2

Memory: 2GB

NIC: Intel Pro 1000 PCI/e (1GbE)

OS: Scientific Linux 5.4 (i386)

88MB/s = 704 Mbps

© KEK

MLF
実験装置群

SOFIA

(KEK)

大観


(JAEA)

HRC
(KEK)

SuperHRPD
(KEK)

Muon D2

(KEK)

ANNRI

(Tokyo Tech. U.,

JAEA,

Hokkaido U.)

千手

⡊䅅䄩

写楽

⡊䅅䄩

SPICA

(NEDO, KEK)

VIN_ROSE

(Kyoto U.,
KEK)

PLANET


(U. of Tokyo,

JAEA)

NOBORU

(JAEA)

iBIX
(Ibaraki Pref.)


四季



JAEA, KEK, Tohoku U.


NOVA


(NEDO, KEK


䵵潮⁕


(KEK)

アマテラス


⡊䅅䄩

DNA


(JAEA)

Muon D1


(KEK, JAEA)

iMATERIA

(Ibaraki
Pref.)




⡊䅅䄩

NO倠⡋䕋(

中性子実験装置:
19


ミュオン実験装置:
3


HRC

NOP

NOBORU

ANNRI

四季

iBIX

PLANET

SuperHRPD

SPICA

DNA

NOVA

iMATERIA

アマテラス

SOFIA



千手

大観

写楽

© KEK

Intelligent RT Software Project

(FY 2007
-
2011)

22



Platform development



-
1 Software/hardware platform development



-
2 RT
-
Component reusability studies




RTCs for Manipulation




Manipulation intelligence for manufacturing





Manipulation intelligence for daily life



RTCs for Mobile Robot




Mobile robot for service robot




High
-
speed mobile robot




Mobile robot for field service



RTCs for Human Robot Interaction




Communication intelligence for daily life robots


SIT, NES software, fuRo,

Fujitsu, NAIST, Fujisoft,

Toyohashi Univ. of Tech.,

SEC, Univ. of Tokyo, Osaka Univ.,

Tsukuba Univ., Segway Japan,

Kyoto Univ., Tohoku Univ.,

International Rescue system

AIST, SEC, Maekawa Industry,

General Robotics Inc., NEC,

Univ. of Tokyo, Fujisoft

AIST, Toyohashi Univ. of Tech.,

Univ of Tokyo, Tohoku Univ.,

NAIST, Osaka Univ., Tsukuba Univ.

Tokyo Univ. of Science, IDEC,

Mitsubishi Elec., Kyoto Univ.,

Kobe Univ., Yasukawa, Toshiba,

Kyushu Univ., Kyutech Univ.,

Metroporitan Univ. of Tokyo


ATR, eager Inc., Omron, NEC

OpenRTP development process

and tool chain

23

RTCBuilder

RTC

Debugger

RTSystemEditor

Real
-
Time design

Motion Pattern

Design

Scenario
Editor

Mobile robot

Navigation

Dynamics
Simulator

Hardware

design

OpenRT Platform

OpenRTM

+

Eclipse

+

tool chain

System Development

RTC development

Application Development

Complete

Design

Impl

Test

Design

Impl

Test

Design

Impl

Test

Developed Components

Category

Number of Projects

Tools

7

Manipulation RTCs

106

Mobility RTC


93

HRI RTCs

16

Commercial License RTCs

5

Total

229


Each consortium made their
technologies into RTCs


Products are registered as
projects on the openrtm.org
website


229 registered projects


Developed RTCs are used
by other consortiums


Based on this experience,
standard interfaces are
considered


24

ROBOSSA

(Robot Software Suite, AIST)


Maintain middleware, tools and RTCs
developed in the RT Project


ROBOSSA


OpenRTM
-
aist (Middleware)


OpenRTP
-
aist (Tools)


OpenRTC
-
aist (RTC packages for vision,
manipulation, mobile robot and HRI)


AIST continues maintenance of them and
will produce start
-
up companies

25

Summary


RT
-
Middleware: OpenRTM
-
aist


History (from 2002
-
)


Features


Component
-
based development


Standard


Multiple vendor RTM implementation


Applications & projects


Humanoid robots, DAQ


RT software platform project: 229 projects
registered on the website

26


27

Reference Architecture

(for Manipulation)

28

Camera

Image

Recoginition

Manipulation

planning

Servo

Manipulator

frontend

Command

Servo

control

Motor &

driver

Motor &

driver

Motor &

driver

Motor &

driver

Camera & image recognition

Planning

Manipulator

Reference Architecture

(for Mobile Robot Navigation)

29

UI

Map
management

/navigation

Sub goal
tracking

Mobile robot

hardware

Motor &

driver

Motor &

driver

Motor &

driver

Motor &

driver

map

sensor

map

sensor

destination

status

Sub goal points

position


#include "BasicDataType.idl"

#include “ExtendedDataTypes.idl” // defined in RTM1.0

module IIS

{


struct TimedVelocity2D


{


RTC::Time tm;


sequence<long> id;


RTC::Velocity2D data; // defined in RTM1.0


sequence<double> error;


};

struct TimedPose2D


{


RTC::Time tm;


sequence<long> id;


RTC::Pose2D data; // defined in RTM 1.0



sequence<double> error;


};


struct TimedPose2DSeq


{


RTC::Time tm;


sequence<long> id;


sequence<RTC::Pose2D> data;


sequence<double> error;


}; };

Reference Architecture

(for Human Robot Interaction)

30

Voice

input

Voice

filter

Voice

Recoginition

Image

input

Speech

interval

detection

Person

detection

Conversation

control

Conversation

contents

management

Conversation

history

management

Motion

creation

Speech

synthesize

Sound

output

HRI

GUI

Object

recognition

Motion

control