Plasma: A Scripting Language for Processing Media Streams

strawberriesfarctateΛογισμικό & κατασκευή λογ/κού

4 Νοε 2013 (πριν από 4 χρόνια και 4 μέρες)

79 εμφανίσεις

Plasma:
A
Scripting
Language
for
P
ro
cessing
Media
Streams
T
a
o
Z
h
u
,
P
a
v
el
Korsh
uno
v,
Bing
Liu
a
nd
W
e
i
T
sang
Ooi
Departmen
t
of
Computer
Science,
National
Univ
ersit
y
of
Singap
ore,
Singap
ore.
ABSTRA
CT
Media
s
treaming
h
as
found
applications
i
n
man
y
d
omains
s
uc
h
a
s
education,
e
n
t
ertainmen
t
,
c
omm
unication
and
v
ideo
surv
eillance.
Man
y
of
these
a
pplications
r
equire
non-trivial
manipulations
o
f
m
edia
s
treams,
b
ey
ond
the
u
sual
capture/pla
y
bac
k
op
erations
s
upp
orted
b
y
t
ypical
m
u
ltimedia
s
oft
w
a
re
and
t
o
ols.
T
o
s
upp
ort
r
apid
dev
elopmen
t
of
suc
h
applications,
w
e
ha
v
e
designed
and
implemen
t
ed
a
s
cripting
language
called
P
lasma.
Plasma
treats
media
s
treams
as
first-class
o
b
j
ects,
and
c
aters
to
the
c
haracteristic
di!erences
b
e
t
w
een
stored
media

les
and
liv
e
m
edia
s
treams.
I
n
t
his
p
ap
er,
w
e
illustrate
the
design
and
f
eatures
o
f
P
lasma
through
s
ev
eral
small
examples,
and
describ
e
t
w
o
example
applications
t
hat
w
e
dev
elop
ed
on
top
of
Plasma.
These
t
w
o
applications
demonstrate
that
using
P
lasma,
complex
a
pplications
t
hat
c
omp
ose,
mix,
a
nd
filter
m
ultimedia
s
treams
can
b
e
written
with
relativ
ely
little
e!ort.
1.
INTR
ODUCTION
Man
y
y
e
ars
o
f
r
esearc
h
i
n
t
he
area
of
media
c
ompression,
middlew
are
t
ec
hnology
,
a
nd
in
ter-net
w
o
rking
h
as
enabled
daily
use
o
f
s
treaming
v
ideo
and
a
udio
o
v
e
r
t
he
In
ternet.
M
an
y
a
cademic
and
r
esearc
h
i
nstitutions
are
r
outinely
c
apturing
and
w
eb-casting
l
ectures.
A
cademic
conferences
and
w
orkshops,
s
uc
h
a
s
S
IGCOMM
and
I
PTPS
ha
v
e
b
e
gun
t
o
b
roadcast
their
t
ec
hnical
s
essions
liv
e
o
v
e
r
t
he
In
ternet.
A
t
t
he
same
time,
v
ideo
conferencing
is
b
ecoming
c
ommon
in
homes
(
e.g.
iChat,
MSN
M
essenger)
a
nd
institutions
(
e.g.
Access
G
rid
1
).
These
dev
elopmen
t
s
o
v
e
r
t
he
last
f
ew
y
e
ars
i
ndicate
that
streaming
m
edia
i
s
s
lo
wly
b
ecoming
a
n
imp
ortan
t
p
art
of
ev
ery
d
a
y
’s
computing
e
xp
erience.
The
i
ncreasing
usage
of
streaming
v
ideo
and
a
udio
c
auses
p
roliferation
of
serv
ers
a
nd
pla
y
ers
t
o
c
apture,
transmit,
and
p
la
ybac
k
s
treaming
m
edia.
These
s
oft
w
a
re
to
ols,
h
o
w
e
v
e
r,
seldom
go
b
e
y
o
nd
the
s
imple
function-
alities
o
f
c
apturing,
t
ransmitting,
receiving
and
p
la
ying
bac
k
.
N
on-trivial
manipulations
o
f
m
edia
s
treams
suc
h
as
comp
osition,
mixing,
or
filtering
are
s
eldom
s
upp
orted.
These
o
p
e
rations
o
n
m
edia
s
treams
are
imp
ortan
t
i
n
man
y
s
cenarios.
W
e
describ
e
s
ome
of
these
s
cenarios
next.

In
a
t
ypical
liv
e
w
eb-cast
pro
duction
en
vironmen
t
with
m
u
ltiple
cameras,
a
d
irector
is
resp
onsible
f
or
pro-
ducing
the
s
tream
s
en
t
t
o
public
audience.
2
The
d
irector
con
t
rols
what
the
a
udience
sees
b
y
switc
hing
b
e
t
w
een
in
teresting
c
amera
views,
adding
titles,
inserting
logos,
i
nserting
r
olling
c
redits
and
o
p
ening
se-
quence,
or
comp
osing
t
he
output
stream
using
v
ideo
streams
from
di!eren
t
cameras.
All
t
hese
o
p
e
rations
require
manipulations
o
f
liv
e
v
ideo
streams.

In
Access
G
rid,
di!eren
t
conference
v
e
n
ues
migh
t
h
a
v
e
d
i!eren
t
c
onference
setup
a
nd
hence
cannot
c
om-
m
unicate
with
eac
h
other.
A
ccess
G
rid
allo
ws
creation
of
network
servic
es
,
3
whic
h
t
ransco
de
video
and
audio
among
s
ites
t
o
enable
i
n
t
erop
erabilit
y
.
Complex
net
w
o
rk
services
could
allo
w

ltering
of
inactiv
e
(“blue
s
creen”)
v
ideo
streams,
o
r
c
omp
osition
of
m
u
ltiple
streams
from
a
s
ingle
site
to
enhance
viewing
exp
e
rience.

Some
existing
I
P-based,
distributed,
video
monitoring
s
ystems
capture
v
ideo
using
net
w
o
rk
cameras
o
r
video
sensors,
and
t
ransmit
the
v
ideo
o
v
er
the
net
w
o
rk
for
a
nalysis
a
nd
monitoring.
4
One
approac
h
t
o
scale
up
suc
h
systems
t
o
large
n
u
m
b
er
of
video
sources
is
to
filter
o
ut
unin
t
eresting
v
ideo
streams.
W
e
5
prop
osed
an
approac
h
w
here
l
o
w
qualit
y
v
ideo
streams
are
s
en
t
t
o
p
ro
xy
for
a
nalysis
a
nd
only
when
in
teresting
e
v
e
n
t
s
a
re
detected,
high
qualit
y
s
treams
are
s
en
t
t
o
u
sers
for
v
iewing.
Pro
cessing
of
liv
e
v
ideo
streams
is
needed
in
suc
h
system.
Comp
osition
of
video
streams
using
liv
e
v
ideo
feeds
from
cameras
a
nd
recorded
video
clips
f
rom
a
n
a
rc
hiv
e
is
useful
in
video
monitoring
s
ystem
a
s
w
ell.
F
o
r
e
xample,
a
user
migh
t
w
an
t
t
o
c
ompare
t
w
o
in
teresting
v
ideos
s
ide-b
y-side.

In
video
jo
c
k
eying,
a
V
J
c
an
select
di!eren
t
videos
e
ither
from
disk
or
from
cameras,
and
a
pply
mixing
and

ltering
to
create
sp
ecial
e!ects,
reacting
t
o
t
he
b
e
ats
of
a
s
ong.
Despite
the
i
ncreasing
needs
to
pro
cess
v
ideo
streams,
t
here
i
s
a
lac
k
o
f
t
o
ols
to
supp
ort
s
uc
h
needs.
M
ost
a
v
ailable
to
ols
and
libraries
supp
ort
o
nly
limited
p
ro
cessing
op
erations
o
n
v
ideo
streams
and
a
re
hard
to
extend
(e.g.
Op
enMash
6
).
On
the
other
hand,
man
y
t
o
ols
for
p
ro
cessing
stored
video
exist.
These
i
nclude
man
y
scripting
languages,
designed
for
r
apid
protot
yping
o
f
s
tored
media
p
ro
cessing
application
(e.g.
VideoScript,
7
Rivl
8
).
These
languages,
ho
w
e
v
e
r,
do
not
s
upp
ort
liv
e
m
edia
s
treams.
Extending
existing
s
cripting
languages
designed
for
p
ro
cessing
stored
media

les
t
o
p
ro
cess
liv
e
m
edia
s
treams
is
not
s
traigh
tforw
a
rd.
L
iv
e
m
edia
h
as
di!eren
t
prop
erties
f
rom
s
tored
media.
Firstly
,
a
n
a
bstraction
for
a
liv
e
media
s
tream
s
hould
e
xp
ose
t
he
net
w
o
rk
conditions
a
sso
ciated
with
the
m
edia
s
tream,
suc
h
as
pac
k
et
loss
r
ate.
One
useful
application
for
s
uc
h
e
xp
osure
i
s
t
hat
i
t
allo
ws
scriptable
adaptation
strategies
b
ased
on
net
w
o
rk
conditions.
Secondly
,
pro
cessing
a
s
tored
media

le
in
v
olv
es
only
storage
devices
and
d
ispla
y
devices.
O
n
t
he
other
hand,
a
liv
e
m
edia
s
tream
h
as
a
net
w
o
rk
source
and
destination,
p
ossibly
c
onsisting
o
f
a
group
o
f
r
eceiv
e
rs.
The
s
ource
o
f
t
he
media
s
tream
i
s
n
o
longer
limited
t
o
s
torage
devices,
but
could
b
e
c
amera,
t
elevision
or
other
video
sources.
F
inally
,
t
emp
o
ral
manipulations,
suc
h
as
concatenating
t
w
o
video
files
and
deleting
certain
scene
from
a
v
ideo,
a
re
not
m
eaningful
f
or
liv
e
m
edia
s
treams.
O
n
t
he
other
hand,
spatial
manipulations,
suc
h
as
creating
a
p
icture-in-picture
e
!ect
or
adding
logos
or
sub-titles
t
o
liv
e
m
edia
s
treams
are
c
ommon
op
erations.
T
o
address
t
he
need
for
a
n
e
asy-to-use
t
o
o
l
f
or
manipulating
liv
e
m
edia
s
treams,
w
e
h
a
v
e
designed
and
implemen
t
ed
a
new
scripting
language
called
Plasma
(Programming
L
Anguage
for
S
treaming
M
ediA).
W
ith
the
di!erences
b
e
t
w
een
stored
media
a
nd
liv
e
m
edia
i
n
mind,
our
language
is
designed
with
the
f
ollo
wing
f
eatures:

Exp
o
sure
of
underlying
net
w
o
rk
conditions
t
o
p
rogrammers;

In
tegration
with
audio/video
devices
as
sources
and
s
inks;
a
nd

F
o
cus
on
spatial
comp
osition
op
erations.
These
f
eatures
d
i!eren
tiate
Plasma
from
existing
languages
i
n
t
he
literature.
Although
designed
with
sp
ecific
features
t
o
s
upp
ort
liv
e
m
edia
s
treams,
P
lasma
supp
orts
manipulation
of
stored
media
lik
e
man
y
e
xisting
languages.
Supp
orting
b
oth
liv
e
a
nd
stored
media
i
n
P
lasma
enables
mixture
of
b
oth
media
t
yp
es
easily
.
Suc
h
mixing
o
p
e
rations
a
re
useful
in
man
y
c
on
texts.
F
o
r
i
nstance,
a
w
eb-cast
op
erator
migh
t
w
an
t
t
o
i
nsert
a
pre-recorded
op
ening
sequence
prior
to
broadcasting
liv
e
s
treams.
O
thers
migh
t
w
an
t
t
o
insert
pre-recorded
adv
e
rtisemen
t
s
egmen
t
s
i
n
b
et
w
een
sessions
w
hile
w
eb-casting
s
eminars.
Another
trivial
and
commonly
used
op
eration
is
to
receiv
e
liv
e
m
edia
s
treams
and
r
ecord
it
to
disk
as
stored
media.
In
this
pap
e
r,
w
e
highligh
t
t
he
design
and
f
eatures
o
f
P
lasma,
and
s
ho
w
h
o
w
useful,
c
omplex
manipulations
of
media
s
treams
can
b
e
d
one
e
asily
in
our
language.
W
e

rst
describ
e
t
he
basic
a
bstractions
a
nd
op
erations
supp
orted
b
y
P
lasma
in
Section
2.
Section
3
f
urther
elab
orates
o
n
t
he
features
a
nd
usefulness
o
f
P
lasma
through
sev
e
ral
e
xamples.
W
e
describ
e
t
he
implemen
tation
of
Plasma
briefly
in
Section
4,
and
t
w
o
applications
w
ritten
using
P
lasma
in
Section
5.
Section
6
c
ompares
Plasma
with
existing
t
o
ols
and
languages
i
n
t
he
literature,
a
nd
Section
7
c
oncludes.
2.
DESIGN
OF
PLASMA
2.1.
Plasma
as
a
S
cripting
Language
W
e
faced
a
design
decision
at
the
b
eginning
of
the
p
ro
ject,
i
n
deciding
whether
to
implemen
t
Plasma
as
a
soft
w
a
re
library
,
a
system-lev
el
programming
language
or
a
s
cripting
language.
W
e
c
hose
to
implemen
t
Plasma
as
a
s
cripting
language,
p
artly
b
ecause
o
f
t
he
man
y
a
dv
an
tages
o
f
s
cripting.
A
s
cripting
language
allo
ws
rapid
protot
yping
a
nd
normally
has
s
uccinct
syn
tax
(and
t
h
u
s
s
horter
co
de).
F
urthermore,
scripts
w
ritten
c
an
b
e
executed
on
di!eren
t
platforms
without
r
e-compilation,
a
s
long
a
s
t
he
language
in
terpreter
i
s
p
orted.
Another
main
reason
that
driv
es
us
to
w
a
rds
u
sing
a
s
cripting
language
is
that
w
e
w
o
uld
lik
e
t
o
e
xploit
the
strengths
o
f
t
he
Tcl/Tk
language,
i
n
p
articular
the
s
trength
of
Tk
as
a
G
UI
building
t
o
ol.
As
a
s
cripting
language
for
building
G
UI
programs,
T
k
e
xcels
a
t
t
w
o
things:
(i)
sp
ecifying
la
y
o
uts
o
f
widgets
and
(ii)
binding
of
GUI
ev
en
ts
to
callbac
ks.
B
y
b
orro
wing
t
he
syn
tax
of
Tk,
P
lasma
allo
ws
spatial
arrangemen
t
a
nd
comp
osition
of
di!eren
t
media
s
treams
easily
.
B
y
s
upp
orting
e
v
e
n
t
s
a
nd
callbac
ks,
P
lasma
allo
ws
callbac
ks
that
manipulate
media
s
treams
to
b
e
written
w
hen
certain
ev
en
ts
(e.g.,
net
w
o
rk
congestion)
a
re
triggered.
Due
t
o
t
he
ubiquit
y
o
f
Tk
as
the
G
UI
to
olkit
for
man
y
s
cripting
languages
(including
P
e
rl,
R
ub
y
,
Python
and
o
f
c
ourse,
Tcl),
w
e
b
e
liev
e
that
the
p
rogramming
m
o
del
and
s
yn
tax
used
in
Tk
are
f
amiliar
to
man
y
p
rogrammers.
F
or
these
r
easons,
w
e
designed
the
s
yn
tax
of
Plasma
to
closely
r
esem
ble
t
hose
of
Tk,
a
nd
implemen
t
ed
Plasma
as
an
extension
to
the
Tcl
s
cripting
language,
t
he
same
language
Tk
is
built
up
on.
2.2.
Abstractions
in
Plasma
Plasma
supp
orts
t
w
o
main
t
y
p
e
s
o
f
a
bstract
o
b
j
ects,
me
dia
o
bje
c
ts
and
events
.
A
media
o
b
j
ect
is
a
unified
abstraction
for
liv
e
m
edia
s
treams,
s
tored
media

les
a
nd
analog
media
s
ources.
A
media
o
b
j
ect
can
b
e
e
ither
a
video
or
an
audio
o
b
j
ect,
a
nd
it
can
r
ecursiv
e
ly
con
tain
other
media
o
b
j
ects.
As
an
example,
c
onsider
a
c
omp
osite
media
o
b
j
ect
represen
ting
a
w
eb-cast
seminar.
T
his
o
b
j
ect
consists
of
a
v
ideo
ob
ject
and
a
n
a
udio
o
b
j
ect.
T
he
video
ob
ject
can,
in
turn,
c
on
tain
t
w
o
other
video
ob
jects,
comp
osed
as
a
p
icture-in-picture
a
rrangemen
t,
one
sho
wing
t
he
sp
eak
e
r,
the
other
sho
wing
t
he
slides.
An
ev
en
t
o
b
j
ect,
w
hic
h
can
b
e
c
reated
a
nd
b
o
und
to
a
c
allbac
k
b
y
t
he
programmer,
represen
ts
an
in
teresting
phenomenon
t
hat
o
ccurs
a
t
s
ome
time
instance.
Common
GUI
e
v
e
n
t
s
s
uc
h
a
s
mouse
and
k
eyb
oard
i
nputs
a
re
supp
orted,
as
w
e
ll
as
timer
e
v
e
n
t
s.
Most
in
terestingly
,
p
rogrammers
can
define
ev
en
ts
that
are
b
ased
on
net
w
o
rk
conditions
(
e.g.,
when
net
w
o
rk
is
congested)
or
con
t
en
t
o
f
t
he
video
streams
(e.g.,
when
motion
is
detected).
Net
w
o
rk-based
e
v
e
n
t
s
allo
w
customized
adaptations
b
ased
on
net
w
o
rk
conditions,
while
con
t
en
t-based
e
v
e
n
t
s
allo
w
c
o
ntinuous
queries
on
media
s
treams
to
b
e
written.
Suc
h
c
on
ten
t
-based
e
v
e
n
t
s
c
ould
b
e
useful
for
building
video
surv
eillance
applications.
2.3.
Plasma
and
I
ndiv
a
The
m
edia
o
b
j
ect
abstraction
in
Plasma
can
r
epresen
t
n
ot
only
pre-recorded
and
liv
e
s
treams,
but
analog
media
devices
suc
h
as
cameras,
cable
b
o
x
es
and
V
CR
as
w
e
ll.
T
o
in
teract
with
suc
h
devices,
P
lasma
uses
a
middlew
are
called
I
ndiv
a
.
9
W
e
briefly
presen
t
I
ndiv
a
b
e
lo
w.
Indiv
a
(INfrastructure
f
or
DIstributed
Video
and
A
udio)
is
a
middlew
are
o
v
e
r
a
distribute
d
a
udio/vide
o
en-
vir
o
nment
.
Suc
h
a
n
e
n
v
ironmen
t
t
y
pically
consists
of
audio/video
equipmen
ts
and
c
omputers
u
sed
t
o
c
apture
media
s
ignals,
c
on
trol
equipmen
ts
and
p
ro
cess
m
edia
s
treams.
T
he
media
s
ignals
and
m
edia
s
treams
are
t
rans-
mitted
o
v
e
r
a
udio/video
routing
net
w
o
rks
a
nd
IP
net
w
o
rks.
Indiv
a
pro
v
ides
a
l
a
y
er
of
abstractions
a
nd
a
s
imple
set
o
f
A
PIs
o
v
e
r
s
uc
h
a
n
e
n
v
ironmen
t
,
h
iding
t
he
implemen
tation
details
of
the
e
n
v
ironmen
t
from
applications.
Indiv
a
uses
a
UNIX

le
system
metaphor
f
or
managing
s
oft
w
a
re
pro
cesses,
hardw
a
re
devices
and
m
edia
d
ata
in
a
d
istributed
audio/video
en
vironmen
t.
These
r
esources
are
n
amed
a
nd
organized
in
to
a
h
ierarc
hical
n
ame
space,
m
u
c
h
lik
e
a
UNIX

le
system.
F
ile
name
extensions
i
ndicate
the
t
yp
e
o
f
t
he
resource
(e.g.,
.cam
indicates
that
a
r
esource
i
s
a
camera).
Indiv
a
relies
o
n
a
cen
tralized
serv
er,
c
alled
Indiva
manager
,
t
o
s
tore
t
he
name
space
and
m
eta-data
for
t
he
managed
r
esources.
A
pplications
c
onnect
to
the
I
ndiv
a
manager,
and
i
ssue
c
ommand
t
o
the
manager
t
o
manipulate
the
underlying
e
n
v
ironmen
t
(e.g.
displa
y
a
particular
video
on
to
a
p
ro
jector
screen).
Indiv
a
pro
v
ides
what
Plasma
needs
to
in
teract
with
analog
media
devices.
F
rom
P
lasma
p
oin
t
o
f
v
iews,
a
n
analog
media
device
i
s
j
ust
a
nother
media
o
b
j
ect
that
can
b
e
manipulated
i
n
a
unified
w
a
y
as,
s
a
y
,
a
n
M
PEG
video
file.
In
ternally
,
P
lasma
relies
o
n
I
ndiv
a
to
capture
m
edia
s
ignals
from
these
devices,
a
nd
transmit
them
as
media
s
treams.
I
n
t
he
next
s
ection,
w
e
will
see
e
xamples
o
n
h
o
w
Plasma
uses
Indiv
a
to
access
r
emote
cameras.
3.
EXAMPLES
In
this
section,
w
e
p
resen
t
sev
e
ral
e
xamples
t
o
illustrate
the
f
eatures
a
nd
p
o
w
e
r
o
f
t
he
Plasma
scripting
language.
1
media
.m
indiva://imgr.nus.edu/cs/room420/speaker.cam
2
.
m
output
capture.mpg
Figure
1.
A
P
lasma
s
cript
for
capturing
a
video
from
a
r
emote
c
amera
a
nd
sa
ving
i
t
t
o
l
o
c
al
disk.
3.1.
Example
1:
Capture
from
Camera
to
Disk
Our
first
example,
s
ho
wn
in
Figure
1,
captures
a
v
ideo
from
a
r
emote
camera
and
s
a
v
es
it
to
disk
as
an
MPEG
file.
Line
1
o
f
F
igure
1
c
reates
a
media
o
b
j
ect
represen
ting
a
camera,
s
p
ecified
b
y
a
giv
en
URL.
The
line
c
onsists
of
three
tok
ens.
T
he
first
tok
en,
media
is
a
P
lasma
command
f
or
creating
a
media
o
b
j
ect.
T
he
second
tok
en,
.m
,
i
s
t
he
name
giv
en
to
the
m
edia
o
b
j
ect
created.
The
w
a
y
Plasma
creates
a
media
o
b
j
ect
is
similar
to
ho
w
Tk
creates
a
widget.
W
e
retain
the
n
aming
c
on
v
e
n
tion
of
widgets
in
Tk,
b
y
p
receding
all
ob
ject
names
with
a
pe
r
i
od
‘.’
.
T
he
rest
of
the
line
s
p
ecifies
parameters
for
c
reation
of
the
m
edia
o
b
j
ect.
I
n
t
his
e
xample,
w
e
c
reate
a
m
edia
o
b
j
ect
from
a
c
amera.
W
e
s
p
ecify
the
c
amera
using
its
corresp
onding
name
as
managed
b
y
t
he
Indiv
a
middlew
are.
The
U
RL,
with
sc
heme
indiva://
,
also
s
p
ecifies
the
n
ame
of
the
I
ndiv
a
manager
(
imgr.nus.edu
)
and
t
he
path
of
the
c
amera
in
the
I
ndiv
a
name
space
(
cs/room420/speaker.cam
).
After
Line
1
c
reates
t
he
media
o
b
j
ect,
L
ine
2
sa
v
e
s
t
he
ob
ject
to
lo
cal
d
isk
a
s
a
file
n
amed
capture.mpg
.
Note
that
the
s
yn
tax
here
again
follo
ws
the
c
on
v
e
n
tion
of
Tk.
I
n
o
b
j
ect-orien
ted
s
t
y
le,
w
e

rst
w
rite
the
n
ame
of
the
o
b
j
ect
w
e
w
a
n
t
to
op
erate
on
(
.m
),
follo
w
e
d
b
y
t
he
op
eration
(
output
),
follo
w
e
d
b
y
t
he
parameters
to
the
op
eration.
I
n
t
his
c
ase,
the
p
arameter
i
s
t
he
name
of
the
o
utput
file
capture.mpg
.
Ev
en
though
this
simple
Plasma
script
has
o
nly
t
w
o
lines
of
co
de,
the
e
xecution
of
the
s
cript
i
s
r
ather
complex.
Firstly
,
P
lasma
recognizes
that
the
U
RL
is
an
Indiv
a
URL,
and
c
on
tacts
the
I
ndiv
a
manager
t
o
r
equest
f
or
a
media
s
tream
f
rom
t
he
camera.
I
ndiv
a
launc
hes
the
necessary
soft
w
a
re
pro
cesses
t
o
c
apture,
enco
de
and
t
ransmit
the
s
tream
t
o
t
he
host
where
t
he
script
is
executed.
The
P
lasma
in
terpreter
o
p
ens
a
s
o
c
k
e
t
t
o
r
eceiv
e
the
m
edia
stream
for
p
ro
cessing.
I
n
L
ine
2
,
P
lasma
recognizes
that
the
s
tream
i
s
t
o
b
e
s
a
v
ed
as
an
MPEG
video
based
on
the

lename
extension.
P
lasma
then
reads
t
he
video
stream
from
the
s
o
c
k
e
t,
trans-co
des
it
to
MPEG
(if
necessary)
a
nd
writes
t
he
resulting
v
ideo
to
a

le
called
capture.mpg
.
Eac
h
of
Plasma
commands
supp
orts
a
n
um
b
e
r
o
f
c
onfiguration
options
t
hat
c
an
b
e
used
to
o
v
erwrite
the
default
b
eha
vior
of
the
c
ommands.
F
or
instance,
the
o
ptions
-fps
and
-bps
can
b
e
u
sed
t
o
c
onfigure
t
he
frame-rate
and
b
it-rate
of
the
m
edia
o
b
j
ect.
T
o
stop
capturing,
w
e
can
s
imply
delete
t
he
ob
ject
.m
.
3.2.
Example
2:
Spatial
Comp
osition
T
e
mp
oral
manipulations
f
or
liv
e
m
edia
s
treams
are
n
ot
as
imp
ortan
t
a
s
s
patial
manipulations.
F
o
r
t
his
r
eason,
one
o
f
t
he
guiding
p
rinciples
in
designing
P
lasma
is
that
it
should
supp
ort
e
asy
s
patial
comp
osition
op
erations.
Figure
2
s
ho
ws
one
s
uc
h
e
xample.
I
n
t
his
s
cenario,
the
u
ser
w
ould
lik
e
t
o
v
iew
t
w
o
video
streams
side-b
y-side,
the

rst
i
s
a
curren
t
liv
e
v
iew
f
rom
a
surv
eillance
camera
inside
a
c
omputer
lab.
T
he
second
is
a
s
urv
e
illance
video
from
an
arc
h
iv
e.
The
u
ser
p
ossibly
w
an
ts
to
compare
w
hat
h
as
b
een
stolen
o
r
m
o
v
ed
in
a
lab.
W
hile
it
is
p
o
ssible
t
o
p
la
y
t
hese
t
w
o
video
streams
at
the
s
ame
time
as
t
w
o
separate
video
streams,
c
omp
osing
i
t
i
n
t
o
a
s
ingle
video
has
its
adv
a
n
tage.
F
or
instance,
the
u
ser
c
an
sa
v
e
b
oth
views
i
n
t
o
a
single
video
file
u
sing
the
output
op
eration
in
tro
duced
in
the
p
revious
e
xample.
In
Figure
2,
the

rst
t
hree
lines
uses
the
media
command
t
o
c
reate
three
d
i!eren
t
m
edia
o
b
j
ects.
Unlik
e
t
he
others,
the

rst
m
edia
o
b
j
ect,
c
alled
.out
,
i
s
c
reated
without
a
n
y
parameters.
T
his
c
ommand
c
reates
a
media
ob
ject
that
acts
as
empt
y
c
on
tainer.
T
his
c
on
tainer
is
analogous
t
o
a
frame
widget
i
n
T
k.
Line
2
a
nd
3
o
f
F
igure
2
c
reate
t
w
o
media
o
b
j
ects,
one
f
rom
a
c
amera,
s
p
ecified
b
y
an
Indiv
a
URL,
and
a
nother
from
an
arc
h
iv
ed
stream
from
a
R
TSP
s
erv
e
r.
The
s
yn
tax
of
Line
2
a
nd
3
i
s
s
imilar
to
that
of
Example
1,
except
that,
in
this
example,
1
media
.out
2
media
.out.door
indiva://imgr.nus.edu/cs/lab0432/door.cam
3
media
.out.suspect
rtsp://archive.nus.edu/2005/6/20/door.mpg
4
mpack
.out.door
.out.suspect
-side
left
5
.out
play
Figure
2.
A
P
lasma
s
cript
for
arranging
t
w
o
media
s
treams
side-b
y-side.
the
n
ame
of
the
o
b
j
ect
is
prefixed
b
y
.out
,
t
he
con
tainer
ob
ject.
T
his
n
aming
c
on
v
e
n
tion
means
t
hat
t
he
ob
jects
.out.door
and
.out.suspect
are
c
reated
a
s
c
hildren
o
f
.out
.
T
his
c
on
v
e
n
tion
is
again
b
o
rro
w
e
d
f
rom
T
k.
After
w
e
created
t
w
o
video
streams,
c
on
tained
within
a
new
media
o
b
j
ect,
w
e
need
to
sp
ecify
ho
w
t
hese
t
w
o
streams
should
b
e
spatially
arranged.
Plasma
uses
a
c
ommand
c
alled
mpack
,
w
hic
h
b
eha
v
e
s
s
imilarly
to
the
pack
command
i
n
T
k.
Line
4
o
f
F
igure
2
illustrates
h
o
w
this
command
i
s
u
sed.
The
mpack
command
tak
es
in
a
list
o
f
media
o
b
j
ects
a
s
p
arameters,
a
nd
arranges
the
o
b
j
ects
a
ccording
t
o
t
he
options
s
p
ecified.
In
this
example,
t
he
t
w
o
video
streams
are
a
rranged
side-b
y-side,
from
left
to
righ
t.
Finally
,
L
ine
5
of
this
example
calls
the
o
p
e
ration
play
of
ob
ject
.out
.
T
his
o
p
e
ration
has
t
he
follo
wing
e!ects.
The
P
lasma
in
terpreter

rst
c
reates
t
he
t
w
o
video
streams
b
y
con
tacting
t
he
Indiv
a
manager
a
nd
R
T
SP
serv
er,
a
nd
op
ens
t
w
o
so
c
k
ets
t
o
r
eceiv
e
the
t
w
o
streams.
T
he
t
w
o
video
streams
are
t
hen
deco
ded,
spatially
comp
osed
in
to
a
new
video
stream
(called
.out
).
The
new
video
stream
is
pla
y
ed
out
o
n
t
o
t
he
screen.
Figure
3
sho
w
s
a
sample
output
video
from
the
s
cript.
Figure
3.
Example
o
utput
from
Plasma
where
t
w
o
videos
a
re
arranged
side-b
y-side.
This
example
illustrates
t
hat
P
lasma
b
o
rro
ws
hea
vily
from
the
s
yn
tax
of
Tk
for
s
patial
arrangemen
t
a
nd
for
sp
ecifying
paren
t
-c
hild
relationship
among
o
b
j
ects.
Plasma,
ho
w
e
v
e
r,
extends
the
s
eman
tic
of
these
c
ommand
t
o
meet
its
o
w
n
r
equiremen
t
s.
F
o
r
i
nstance,
one
c
ommon
op
eration
in
spatial
comp
osition
is
to
o
v
erla
y
o
ne
video
on
top
of
another,
but
the
pack
command
f
rom
T
k
d
o
e
s
n
ot
allo
w
t
w
o
widgets
to
o
v
erlap.
T
he
mpack
command
in
Plasma
allo
ws
this
arrangemen
t,
b
y
c
hec
king
if
the
c
on
tainer
ob
ject
is
empt
y
.
If
the
c
on
tainer
is
non-empt
y
,
the
c
hild
media
o
b
j
ect
is
o
v
erla
y
e
d
o
n
top
of
the
p
aren
t
a
s
a
new
la
y
e
r.
Figure
4
illustrates
t
his
o
p
e
ration
b
y
sho
wing
h
o
w
picture-in-picture
e
!ects
c
an
b
e
ac
hiev
ed
in
Plasma.
1
media
.out
indiva://imgr.nus.edu/cs/lab0432/door.cam
2
media
.out.suspect
rtsp://archive.nus.edu/2005/6/20/door.mpg
3
.out.suspect
scale
0.3
4
mpack
.out
.out.suspect
-anchor
sw
Figure
4.
A
P
lasma
s
cript
for
arranging
t
w
o
media
s
treams
as
a
p
icture-in-picture.
The
c
o
d
e
s
ho
wn
in
Figure
4
i
s
s
imilar
to
that
sho
w
n
i
n
F
igure
2,
except
that
the
c
on
tainer
media
o
b
j
ect
.out
con
tains
a
liv
e
c
amera
view,
a
nd
the
c
hild
media
o
b
j
ect
.out.suspect
con
tains
a
view
of
an
arc
h
iv
ed
video.
When
w
e
call
mpack
in
Line
4,
Plasma
o
v
erla
ys
the
c
hild
video
o
v
er
the
c
on
tainer
video.
T
he
option
-anchor
sw
tells
Plasma
to
put
the
c
hild
video
on
the
s
outh-w
est
c
orner
of
the
c
on
tainer.
T
he
command
scale
on
Line
3
simply
reduces
the
width
and
heigh
t
o
f
t
he
video
to
30%
of
its
original
size
so
that
the
c
on
tainer
video
is
not
obstructed.
Figure
5
s
ho
ws
a
s
ample
output
video
from
this
script.
Figure
5.
Example
o
utput
from
Plasma
w
here
t
w
o
videos
a
re
arranged
as
a
p
icture-in-picture.
3.3.
Example
3:
Ev
en
ts
The
next
e
xample
sho
w
s
t
he
ev
en
t-driv
en
asp
ect
of
Plasma.
W
e
sho
w
ho
w
e
v
e
n
t
s
a
re
created
a
nd
b
o
und
to
callbac
ks
in
this
example.
1
mevent
<HasMotion>
{obj}
{
2
[$obj
has_motion]
3 }
4
media
.camera
indiva://imgr.nus.edu/cs/lab0432/door.cam
5
mbind
<HasMotion>
.camera
{
6
i
f
{![mexists
.out]}
{
7
media
.out
[.camera
url]
8
.out
output
rtp://224.4.4.4:44444/
9 }
10
}
11
mevent
<NoMotion>
{obj}
{
12
![$obj
has_motion]
13
}
14
mbind
<NoMotion>
.camera
{
15
if
{[mexists
.out]}
{
16
mremove
.out
17
}
18
}
Figure
6.
A
P
lasma
s
cript
for
transmitting
data
from
camera
as
R
T
P
s
tream
u
p
o
n
d
etecting
motion.
Figure
6
s
ho
ws
a
P
lasma
script
that
transmits
video
stream
captured
from
a
s
urv
e
illance
camera
in
a
c
omputer
lab,
w
henev
er
motions
a
re
detected
in
the
r
o
om.
The

rst
t
hree
lines
in
this
script
create
an
ev
en
t
with
command
mevent
.
T
he
command
mevent
has
t
hree
argumen
t
s

the

rst
a
rgumen
t
i
s
t
he
name
of
the
e
v
e
n
t
,
t
he
second
is
a
list
o
f
p
arameters
needed
to
define
the
e
v
e
n
t
,
a
nd
the
t
hird
argumen
t
is
an
expression
whic
h
i
s
e
v
aluated
p
e
rio
d
ically
to
c
hec
k
i
f
t
he
ev
en
t
s
hould
b
e
t
riggered.
The
n
ame
of
the
e
v
e
n
t
is
surrounded
b
y
a
p
air
of
triangular
brac
k
e
ts,
f
ollo
wing
t
he
naming
c
on
v
e
n
tion
of
ev
en
ts
in
Tk.
T
he
ev
en
t
c
reated
here
i
s
t
riggered
b
y
detection
of
motion
in
the
i
nput
video
(parameterized
b
y
v
a
riable
obj
).
The
p
arameterization
of
the
v
ideo
ob
ject
allo
ws
the
same
ev
en
t
t
o
b
e
a
pplied
t
o
d
i!eren
t
v
ideo
ob
jects
i
n
P
lasma.
Line
4
o
f
t
he
script
ab
o
v
e
c
reates
a
video
ob
ject,
c
aptured
f
rom
a
s
urv
e
illance
camera.
L
ine
5
to
10
then
bind
this
video
ob
ject
to
the
motion
detection
ev
en
t
a
nd
sp
ecify
the
c
allbac
k
f
unction
of
the
e
v
e
n
t
.
I
n
t
he
callbac
k,
a
new
video
stream
named
.out
with
the
s
ame
URL
a
s
t
he
camera
is
created
(
Line
7)
and
i
s
t
ransmitted
t
o
m
u
lticast
s
ession
224.4.4.4
at
p
o
rt
44444
(Line
8).
The
e
v
e
n
t
expression
(Line
2
)
i
s
e
v
aluated
p
erio
dically
.
B
y
default,
it
is
ev
aluated
e
v
e
ry
frame.
T
h
u
s,
in
the
e
xample
ab
o
v
e,
the
e
v
e
n
t
is
triggered
f
or
ev
ery
f
rame
and
t
he
callbac
k
i
s
e
xecuted
for
e
v
e
ry
frame,
a
s
long
as
there
i
s
motion.
T
he
conditional
statemen
t
in
Line
6
o
f
F
igure
6
ensures
that
a
new
video
stream
is
created
when
motion
is
detected
for
t
he
first
time
only
,
n
ot
ev
ery
f
rame!
Finally
,
L
ine
1
1
t
o
1
8
o
f
t
he
example
stop
transmission
of
the
m
ulticast
s
tream
w
hen
motion
no
longer
e
xists
in
the
s
ource
v
ideo.
3.4.
Example
4:
Exp
o
sing
Underlying
Net
w
ork
Conditions
One
of
the
d
i!eren
tiating
f
eature
of
Plasma
is
that
it
exp
o
ses
t
he
net
w
o
rk
statistics
a
sso
ciated
with
a
m
edia
stream
to
the
p
rogrammer.
This
exp
o
sure
allo
ws
programmer
t
o
mak
e
a
pplication-lev
el
decisions
b
ased
on
the
net
w
o
rk
conditions.
While
finer
grain
adaptations
s
uc
h
a
s
r
ate
con
t
rol
o
r
a
daptiv
e
F
EC
are
b
est
l
eft
t
o
t
he
media
streaming
engine
i
nstead
of
con
t
rolled
t
hrough
scripts,
suc
h
fine
grain
adaptations
a
re
not
p
ossible
i
n
certain
scenarios.
A
n
e
xample
is
video
streams
that
are
t
ransmitted
f
rom
net
w
o
rk
cameras.
In
the
next
e
xample,
s
ho
wn
in
Figure
7,
w
e
illustrate
ho
w
c
oarse
grain
adaptation
can
b
e
a
c
h
iev
e
d
u
sing
Plasma.
1
mevent
<HighLoss>
{obj}
{
2
[$obj
lossrate]
>
.
1
3 }
4
mevent
<LowLoss>
{obj}
{
5
[$obj
lossrate]
<=
.1
6 }
7
media
.camera
http://192.168.0.4/mpeg4/1/media.amp
8
.camera
configure
-fps
30
9
mbind
<HighLoss>
.camera
{
10
.camera
configure
-fps
20
11
}
12
mbind
<LowLoss>
.camera
{
13
.camera
configure
-fps
30
14
}
Figure
7.
A
P
lasma
s
cript
t
hat
c
hanges
frame
r
ate
f
rom
a
net
w
ork
c
amera
d
ep
ending
on
loss
rate.
Figure
7
illustrates
t
he
use
o
f
lossrate
command
o
f
a
media
s
tream
o
b
j
ect.
T
he
command
r
eturns
a
n
um
b
e
r
b
e
t
w
een
0
a
nd
1
i
ndicating
t
he
w
e
igh
ted
a
v
e
rage
pac
k
et
loss
r
ate
of
the
s
tream.
Using
t
his
c
ommand,
the
s
cript
defines
t
w
o
ev
en
ts
HighLoss
and
LowLoss
,
w
hic
h
corresp
ond
t
o
loss
r
ate
higher
than
10%
and
n
o
h
igher
than
10%
resp
ectiv
ely
(
Line
1
t
o
6).
The
c
allbac
ks
of
these
t
w
o
ev
en
ts
configure
t
he
net
w
o
rk
camera
(at
IP
address
192.168.0.4)
to
send
its
video
at
lo
w
e
r
o
r
h
igher
frame
rate,
dep
ending
on
the
s
tate
the
net
w
o
rk
is
in.
4.
IMPLEMENT
A
TION
The
c
ore
f
unctionalities
o
f
P
lasma
are
implemen
t
ed
in
C
a
nd
C++,
with
Tcl
b
indings
t
o
p
ro
vide
the
s
cripting
in
terface.
A
library
libplasma
is
pro
v
ided
as
a
T
cl
dynamically
loadable
library
.
T
his
library
allo
ws
users
t
o
launc
h
a
Tk
shell,
load
the
library
,
a
nd
use
P
lasma
commands
in
an
in
teractiv
e
manner.
O
f
c
ourse,
a
u
ser
c
an
also
r
un
a
P
lasma
script
as
a
b
atc
h

le
b
y
passing
the
n
ame
of
the
s
cript

le
as
argumen
t
to
the
i
n
t
erpreter.
Plasma
mak
es
hea
vy
use
o
f
e
xisting
o
p
e
n
s
ource
libraries,
i
ncluding
FFMPEG
for
v
ideo
deco
ding
and
en-
co
ding,
L
AME
f
or
audio
deco
ding
and
enco
ding,
F
reeT
yp
e
a
nd
Glyph
Keep
er
for
t
ext
r
endering,
Dali
for
image
pro
cessing,
S
DL
for
a
udio
a
nd
video
displa
y
a
nd
Op
enMash
f
or
in
terfacing
with
Indiv
a
.
5.
APPLICA
TIONS
T
o
demonstrate
and
v
erify
t
he
ease-of-use
o
f
P
lasma
as
a
s
cripting
language,
w
e
h
a
v
e
dev
elop
ed
t
w
o
applications
on
top
of
Plasma.
The

rst
a
pplication
is
called
V
JOnline,
whic
h
i
s
a
n
a
pplication
for
v
ideo
jo
c
k
ey
to
add
e
!ects
and
c
omp
osite
video
displa
ys
on
the

y
.
The
s
econd
application
is
called
S
LIME.
S
LIME
is
a
v
ideo
editor
for
la
y
o
ut
and
c
omp
osition
of
videos
f
rom
m
ultiple
media
s
ources.
I
n
t
he
rest
of
this
section,
w
e
e
lab
orate
on
the
functionalities
o
f
t
hese
t
w
o
applications
a
nd
sho
w
ho
w
P
lasma
allo
ws
these
a
pplications
t
o
b
e
built
quic
kly
.
5.1.
VJOnline
VJOnline
i
s
a
to
ol
for
a
VJ
(video
jo
c
k
ey)
o
r
w
eb-cast
director.
U
sing
VJOnline,
users
c
an
op
en
di!eren
t
video
and
a
udio
s
ources,
manipulate
them
with
preset
e!ects,
and
b
roadcast
them
to
the
a
udience
o
v
er
the
I
n
t
ernet.
VJOnline
s
upp
orts
(i)
transition
e!ects
s
uc
h
a
s
f
ade
i
n,
p
o
p
i
n,
slide
i
n,
and
curtain
in,
(ii)
picture-in-picture
e!ects,
and
(iii)
text
marquee
e!ects.
A
s
creenshot
o
f
V
JOnline
i
s
s
ho
wn
in
Figure
8.
Since
the
v
ideo
e!ects
a
nd
pro
cessing
op
erations
a
re
implemen
t
ed
in
Plasma,
it
tak
es
only
ab
out
300
lines
of
co
de
to
implemen
t
VJOnline.
Out
of
these,
ab
out
240
lines
of
co
de
are
f
or
media
p
ro
cessing,
a
nd
the
r
est
a
re
Tk
co
de
for
c
onstructing
t
he
GUI.
Figure
8
s
ho
ws
a
s
creenshot
o
f
t
he
user
in
terface
o
f
V
JOnline.
Figure
8.
Screenshot
of
VJOnline
5.2.
SLIME
The
n
ame
SLIME
s
tands
for
S
tored/LIv
e
Media
E
ditor.
I
t
i
s
a
n
a
pplication
that
is
designed
to
pro
v
ide
i
n
t
uitiv
e,
drag-and-drop
in
terface
f
or
comp
osing
v
ideo.
S
LIME
is
designed
with
one
o
f
t
he
three
g
rand
m
u
ltimedia
c
hal-
lenges
in
mind

“making
v
ide
o
e
d
iting
a
s
e
asy
a
s
w
or
d
p
r
o
c
essing”
.
10
SLIME
h
as
not
f
ully
ac
hiev
ed
this
goal
as
it
fo
cuses
on
spatial
editing
o
nly
.
As
far
a
s
s
patial
comp
osition
is
concerned,
ho
w
e
v
e
r,
w
e
design
SLIME’s
u
ser
in
terface
t
o
b
e
s
imilar
to
common
dra
wing
i
n
t
erface
found
in
desktop
soft
w
a
re
suc
h
as
Microsoft
P
o
w
e
rP
oin
t
a
nd
Adob
e
P
hotoshop.
A
t
ypical
user
of
SLIME
o
p
ens
a
s
et
of
video
clips,
and
d
rag-and-drops
t
he
clips
o
n
t
o
a
scratc
h
c
an
v
a
s.
Eac
h
clip
is
represen
ted
a
s
a
n
image
sho
wing
a
snapshot
of
a
f
rame
in
the
v
ideo.
O
n
t
he
scratc
h
c
an
v
a
s,
the
u
ser
comp
oses
the
o
utput
video
b
y
mo
ving,
r
esizing,
and
c
ropping
the
v
ideo
clips
t
hrough
clic
king
and
d
ragging
t
heir
corresp
onding
images.
Similar
to
common
dra
wing
s
oft
w
a
re,
t
he
user
can
c
lic
k
a
n
y
where
o
n
t
he
can
v
as,
a
nd
t
y
p
e
in
text
strings,
whic
h
will
b
e
o
v
erla
y
e
d
o
n
top
of
the
r
esulting
v
ideo.
S
imilarly
,
a
user
can
o
v
e
rla
y
another
image
(suc
h
a
s
logo)
on
top
of
the
c
omp
osed
video.
T
he
user
can
s
elect
individual
o
b
j
ect
or
group
o
f
o
b
j
ects
on
can
v
as,
a
nd
c
h
ange
their
p
rop
e
rties
(
ho
w
long
t
he
o
v
erla
y
t
ext
will
app
e
ar,
a
pply
a
dditional
sp
ecial
e!ects,
etc.).
After
the
u
ser
i
s
s
atisfied
with
the
c
omp
osed
video,
s
he
can
t
hen
pla
y
bac
k
the
c
omp
osed
video,
o
r
s
a
v
e
the
v
ideo
to
a

le.
A
screenshot
o
f
S
LIME
is
sho
w
n
i
n
F
igure
9.
Figure
9.
Screenshot
of
SLIME
Despite
the
c
omplexit
y
of
SLIME,
the
a
pplication
is
written
i
n
o
nly
a
b
o
ut
8000
lines
of
Tcl/Tk
co
de.
SLIME
uses
libraries
suc
h
as
vTcl
for
G
UI
constructions,
TkZinc
f
or
dra
wing
a
nd
manipulating
o
b
j
ects
o
n
c
an
v
a
s,
and
external
program
suc
h
as
Mpla
y
e
r
f
or
pla
y
bac
k
.
M
ost
i
n
t
erestingly
,
S
LIME
relies
o
n
P
lasma
for
p
ro
cessing
the
video.
W
hen
a
u
ser
w
an
ts
to
sa
v
e
or
pla
y
bac
k
a
v
ideo,
S
LIME
analyzes
the
l
a
y
out
a
nd
prop
erties
o
f
o
b
j
ects
o
n
its
scratc
h
c
an
v
a
s,
and
g
enerates
t
he
equiv
alen
t
Plasma
script,
w
hic
h
is
then
passed
t
o
P
lasma
in
terpreter
t
o
generate
the
o
utput
video.
T
h
u
s,
no
actual
video
pro
cessing
co
de
is
written
a
s
p
art
o
f
S
LIME
(except
to
extract
th
um
bnails
and
s
napshots
for
s
cratc
h
c
an
v
a
s).
During
the
i
nitial
dev
elopmen
t
phase
of
SLIME,
a
p
reliminary
v
ersion
of
Plasma
whic
h
s
upp
orts
only
stored
media
i
s
u
sed.
A
t
a
later
s
tage
of
the
p
ro
ject,
w
hen
supp
ort
f
or
liv
e
m
edia
s
treams
is
stable
enough
i
n
P
lasma,
SLIME
s
witc
hes
to
using
t
his
new
er
v
e
rsion.
I
n
t
erestingly
,
S
LIME
suddenly
s
upp
orts
streaming
v
ideo
as
one
o
f
the
v
ideo
sources,
allo
wing
c
omp
osition
of
video
consisting
o
f
b
oth
liv
e
a
nd
stored
media,
with
minimal
c
h
anges
to
its
co
de.
This
exp
e
rience
demonstrates
t
he
generalit
y
o
f
a
bstractions
o
v
e
r
m
edia
s
treams
pro
v
ided
b
y
Plasma.
6.
RELA
TED
W
ORK
The
c
omplex
n
ature
of
m
u
ltimedia
a
pplications
h
as
spurred
man
y
t
o
olkits
and
libraries
to
simplify
t
he
soft
w
a
re
dev
elopmen
t
pro
cess
s
ince
mid-90’s.
M
uc
h
r
esearc
h
e
!ort
has
b
een
put
in
to
dev
elopmen
t
of
to
olkits,
middlew
are,
and
p
rogramming
f
ramew
o
rks
f
or
distributed
m
u
ltimedia
a
pplications.
These
e
!orts
i
nclude
Berk
eley
CMT,
11
MASH
12
!
,
I
nfoPip
es,
13
and
N
MM.
14
These
t
o
olkits
t
y
pically
pro
v
ide
a
bstractions
i
n
t
he
form
of
sources,
s
inks,
filters,
a
nd
connections,
allo
wing
p
rogrammers
to
connect
the
d
i!eren
t
c
omp
onen
ts
to
build
a
d
istributed
m
u
lti-
media
a
pplication.
O
ther
e!orts
suc
h
as
QCompiler
15
and
P
LASMA
16

fo
cus
on
building
a
daptiv
e
d
istributed
m
u
ltimedia
a
pplications.
In
con
t
rast,
a
pplications
built
with
our
s
cripting
language
are
n
ot
distributed

t
hey
are
mean
t
t
o
r
un
on
a
s
ingle
host
and
c
omm
unicate
with
others
s
erv
e
rs
through
s
tandard
or
op
en
proto
cols
suc
h
as
HTTP
and
R
TSP
.
F
u
rther,
P
lasma
fo
cuses
on
pro
cessing
of
stored
media
a
nd
liv
e
m
edia
s
treams,
with
emphasis
on
spatial
comp
osition.
T
herefore
P
lasma
complimen
t
s
t
he
design
goals
of
these
d
istributed
m
u
ltimedia
t
o
olkits.
Man
y
scripting
languages
f
or
pro
cessing
media
e
xist
as
w
e
ll.
Some
examples
i
nclude
V
uSystem,
17
Rivl,
8
VideoScript
7
and
AviSyn
t
h.
18
These
languages
f
o
cus
on
pro
cessing
stored
video
only
,
a
nd
do
not
allo
w
e
asy
manipulation
of
video
streams.
SMIL
19
pro
v
ides
a
c
omprehensiv
e
s
et
of
functionalities
f
or
spatial-temp
oral
la
y
o
ut
and
t
ransitional
e!ects
on
m
u
ltimedia
d
ata.
Unlik
e
P
lasma,
whic
h
i
s
imp
erativ
e,
SMIL
is
declarativ
e
i
n
n
ature.
F
urther,
e
v
e
n
t
hough
SMIL
supp
orts
adaptiv
e
c
on
ten
t
,
t
he
adaptivit
y
i
s
limited
t
o
s
ystem
p
arameters
(e.g.,
CPU
s
p
eed)
while
Plasma
allo
ws
m
u
c
h
more
flexible
adaptations,
for
i
nstance,
based
o
n
d
ynamic
net
w
o
rk
conditions
o
r
u
ser-defined
ev
en
ts.
MedSMan
20
is
a
s
ystem
f
or
querying
liv
e
m
edia
s
treams
and
p
ro
vides
description
languages
a
nd
an
SQL-lik
e
querying
language
for
managing
liv
e
m
edia
s
treams.
I
t
allo
ws
generation
of
fe
atur
e
str
e
am

s
equence
of
features
(and
a
sso
ciated
m
eta-data)
resulting
f
rom
a
nalysis
o
f
a
media
s
tream.
Similar
to
Plasma,
MedSMan
allo
ws
user-
defined
ev
en
ts
that
are
t
riggered
b
ased
on
results
of
con
tin
uous
queries.
In
the
c
ase
o
f
M
edSMan,
t
he
queries
are
e
xecuted
on
the
f
eature
streams.
P
lasma,
ho
w
e
v
e
r,
do
es
not
s
upp
ort
f
eature
streams.
O
n
t
he
other
hand,
MedSMan
fo
cuses
on
querying
only
and
d
o
e
s
n
ot
supp
ort
s
patial
comp
osition
and
i
s
n
ot
net
w
o
rk-a
w
a
re.
7.
CONCLUSION
This
pap
e
r
p
resen
t
s
a
new
scripting
language
(built
up
on
Tcl/Tk)
c
alled
P
lasma.
The
r
ational
b
ehind
designing
and
dev
eloping
t
his
new
language
is
that,
existing
libraries
and
t
o
olkits
lac
k
s
u"cien
t
s
upp
ort
f
or
non-trivial
manipulations
o
f
v
ideo
and
a
udio
s
treams,
a
feature
that
w
e
b
e
liev
e
is
increasingly
critical.
With
target
a
pplica-
tions
s
uc
h
a
s
v
ideo
conferencing,
w
eb-casting,
and
v
ideo
surv
eillance
in
mind,
w
e
designed
Plasma
to
treat
m
edia
streams
as
first
class
c
itizens
in
the
language

P
lasma
exp
o
ses
t
he
underlying
net
w
o
rk
statistics
o
f
liv
e
s
treams,
allo
ws
con
t
rol
o
f
v
ideo
and
a
udio
devices
(whic
h
c
an
pro
duce
streams)
a
nd
enables
easy
spatial
comp
osition
of
streams.
Plasma
is
still
under
dev
elopmen
t
.
B
esides
con
tin
uing
dev
elopmen
t
w
o
rk
to
optimize
its
p
e
rformance
and
extending
Plasma
with
useful
op
erations,
w
e
are
i
n
t
erested
i
n
implemen
ting
g
eneric
streams
in
Plasma.
Generic
streams
w
o
uld
b
e
u
seful
i
n
a
pplications
s
uc
h
a
s
v
ideo
surv
eillance
or
sensor
net
w
o
rk,
w
here
d
ata
streams
(e.g.
strings,
n
u
m
b
ers)
are
r
eceiv
e
d
a
nd
transmitted.
Besides
allo
wing
c
on
tin
uous
queries
o
v
e
r
t
hese
d
ata,
one
i
n-
teresting
new
abilit
y
o
f
P
lasma
if
suc
h
generic
stream
is
supp
orted,
is
comp
osition
of
data
streams
with
media
streams.
T
his
a
bilit
y
i
s
u
seful,
for
e
xample,
i
f
w
e
w
an
t
t
o
d
ispla
y
scrolling
s
to
c
k
tic
k
e
rs
on
a
v
ideo.
!
and
i
ts
later
i
ncarnation,
O
p
e
nMash
(h
ttp://www.op
enmash.org)

not
t
o
b
e
c
onfused
w
ith
o
ur
language,
P
lasma
REFERENCES
1.
L.
Childers,
T.
Disz,
R
.
O
lson,
M
.
P
apk
a
,
R
.
S
tev
ens,
a
nd
T.
Udeshi,

Access
G
rid:
immersiv
e
g
roup-to-
group
c
ollab
orativ
e
v
isualization,”
in
P
roceed
i
n
g
s
o
f
t
h
e
4
t
h
I
n
t
e
r
n
a
t
i
o
n
a
l
I
mmersive
Pr
oje
c
tion
T
e
chnolo
gy
Workshop
,
(
Ames,
I
A),
J
une
2000.
2.
T.
Y
u
,
D
.
W
u,
K.
Ma
y
e
r-P
atel,
and
L
.
A
.
R
o
w
e
,

dc:
a
liv
e
w
eb
cast
con
t
rol
s
ystem,”
in
P
roceed
i
n
g
s
o
f
t
h
e
SPIE
Multime
d
ia
Computing
a
nd
Networking
(MMCN),
V
ol.
4312
,
(
San
J
ose,
CA),
Jan.
2001.
3.
H.
Gao,
R.
L.
Stev
ens,
a
nd
M.
E.
P
a
pk
a,
“The
design
of
net
w
o
rk
services
for
a
dv
anced
collab
orativ
e
en
viornmen
ts,”
in
P
roceed
i
n
g
s
o
f
t
h
e
3
r
d
W
o
r
k
s
h
o
p
o
n
Ad
v
a
n
ced
C
o
l
l
a
b
o
r
a
t
i
v
e
E
n
v
i
o
r
n
m
e
n
t
s
(
W
A
C
E
)
,
(Seattle,
W
A),
J
une
2003.
4.
M.
V
alera
and
S
.
V
elastin,

In
telligen
t
distributed
surv
eillance
systems:
a
r
eview,”
I
E
E
P
roceed
i
n
g
s
o
n
Vision,
I
mage
and
S
ignal
P
r
o
c
essing
152
,
pp.
192

204,
Apr.
2005.
5.
P
.
Korsh
uno
v
a
nd
W.
T.
Ooi,
“Critical
video
qualit
y
f
or
distributed
automated
v
ideo
surv
eillance,”
i
n
P
roceed
i
n
g
s
o
f
t
h
e
13th
A
C
M
I
nternational
Confer
enc
e
on
Multime
d
ia
,
pp.
151–160,
(Hilton,
S
ingap
ore),
No
v.
2005.
6.
Op
enMash
Consortium.
h
ttp://www.op
enmash.org.
7.
VideoScript.
h
ttp://www.videoscript.com.
8.
J.
Sw
artz
and
B
.
C
.
S
mith,

A
r
esolution
indep
enden
t
v
ideo
language,”
in
P
roceed
i
n
g
s
o
f
t
h
e
3
r
d
A
C
M
International
Confer
enc
e
on
Multime
d
ia
,
pp.
179–188,
(San
F
r
ancisco,
CA),
No
v.
1995.
9.
W.
T.
Ooi,
P
.
Pletc
her,
a
nd
L.
A.
Ro
w
e
,

Indiv
a
:
a
middlew
are
f
or
managing
d
istributed
media
e
n
v
iron-
men
t,”
in
P
roceed
i
n
g
s
o
f
A
C
M
/
S
P
I
E
M
u
l
t
i
m
ed
i
a
C
o
m
p
u
t
i
n
g
a
n
d
N
e
t
w
o
r
k
i
n
g
(
M
M
C
N
)
,
V
o
l
.
5305
,
pp.
211–
224,
(San
Jose,
C
A),
J
an.
2004.
10.
L.
A.
Ro
w
e
and
R
.
J
ain,

A
C
M
S
IGMM
retreat
r
ep
ort
o
n
f
uture
d
irections
i
n
m
ultimedia
r
esearc
h,”
AC
M
T
r
ansactions
on
Multime
d
ia
Computing,
Communic
ations
a
nd
Applic
ations
1
(1),
pp.
3–13,
2005.
11.
K.
Ma
y
e
r-P
atel
a
nd
L.
A.
Ro
w
e
,

Design
and
p
erformance
of
the
B
erk
e
ley
Con
tin
uous
Media
T
o
olkit,”
in
P
roceed
i
n
g
s
o
f
A
C
M
/
S
P
I
E
M
u
l
t
i
m
ed
i
a
C
o
m
p
u
t
i
n
g
a
n
d
N
e
t
w
o
r
k
i
n
g
(
M
M
C
N
)
,
V
o
l
.
3020
,
pp.
194–206,
(San
Jose,
C
A),
J
an.
1997.
12.
S.
McCanne,
E.
Brew
er,
R
.
K
atz,
L
.
A
.
R
o
w
e
,
E
.
A
mir,
Y
.
C
ha
w
athe,
A.
Co
op
ersmith,
K
.
M
a
y
er-P
atel,
S.
Raman,
A
.
S
c
h
uett,
D.
Simpson,
A.
Sw
an,
T
.
L
.
T
ung,
D
.
W
u,
and
B
.
C
.
S
mith,

T
o
w
a
rd
a
c
ommon
infrastructure
f
or
m
u
ltimedia-net
w
o
rking
middlew
are,”
i
n
P
roceed
i
n
g
s
o
f
7
t
h
I
n
t
e
r
n
a
t
i
o
n
a
l
W
o
r
k
s
h
o
p
o
n
Network
and
O
p
e
r
ating
Systems
Supp
ort
for
D
igital
A
u
dio
a
nd
Vide
o
(
NOSSD
A
V
)
,
pp.
39–49,
(St.
Louis,
MO),
Ma
y
1997.
13.
A.
P
.
Blac
k,
J.
Huang,
R.
Koster,
J
.
W
alp
ole,
a
nd
C.
Pu,

Infopip
e
s:
an
abstraction
for
m
ultimedia
streaming,”
Multime
d
ia
System
8
(5),
pp.
406–419,
2002.
14.
M.
Lohse,
M.
Repplinger,
and
P
.
S
lusallek,
“An
o
p
e
n
middlew
are
a
rc
hitecture
f
or
net
w
o
rk-in
t
egrated
m
ulti-
media,”
in
P
roceed
i
n
g
s
o
f
J
o
i
n
t
I
n
t
e
r
n
a
t
i
o
n
a
l
W
o
r
k
s
hops
on
Inter
a
ctive
D
istribute
d
M
ultime
dia
S
ystems
and
Pr
oto
c
o
ls
for
M
ultime
dia
S
ystems
(IDMS/PR
OMS)
,
L
e
ctur
e
Notes
i
n
C
omputer
S
cienc
e
2515
,
pp.
327–338,
Springer,
2002.
15.
D.
Wic
h
adakul,
X
.
G
u,
and
K
.
N
ahrstedt,

A
p
rogramming
f
ramew
o
rk
for
q
ualit
y-a
w
a
re
ubiquitous
m
ulti-
media
a
pplications,”
i
n
P
roceed
i
n
g
s
o
f
t
h
e
10th
A
C
M
I
nternational
Confer
enc
e
on
Multime
d
ia
,
pp.
631–640,
(Juan-les-Pins,
F
r
ance),
D
ec.
2002.
16.
O.
La
y
aida
a
nd
D.
Hagimon
t,
“PLASMA:
a
c
omp
onen
t-based
f
ramew
o
rk
for
building
s
elf-adaptiv
e
a
ppli-
cations,”
i
n
P
roceed
i
n
g
s
o
f
E
m
bed
d
e
d
M
u
l
t
i
m
e
d
i
a
P
roce
s
s
i
n
g
a
n
d
C
o
m
m
u
n
i
ca
t
i
o
n
s
,
pp.
185–196,
(San
Jose,
CA),
Jan.
2005.
17.
C.
Lindblad,
D
.
W
etherall,
and
D
.
L
.
T
ennenhouse,
“The
V
uSystem:
a
programming
s
ystem
f
or
visual
pro-
cessing
of
digital
video,”
in
P
roceed
i
n
g
s
o
f
t
h
e
2
n
d
A
C
M
I
n
t
e
r
n
a
t
i
o
n
a
l
C
o
n
f
e
r
e
n
ce
o
n
M
u
l
t
i
m
ed
i
a
,
pp.
307–
314,
(San
F
r
ancisco,
CA),
Oct.
1994.
18.
AviSyn
t
h.
h
ttp://www.a
visyn
t
h.org.
19.
L.
Rutledge,
“SMIL
2.0:
XML
f
or
W
e
b
m
ultimedia,”
IEEE
Internet
Computing
5
(5),
pp.
78–84,
2001.
20.
B.
Liu,
A.
Gupta,
and
R
.
J
ain,

MedSMan:
a
streaming
d
ata
managemen
t
system
o
v
er
liv
e
m
ultimedia,”
in
P
roceed
i
n
g
s
o
f
t
h
e
13th
A
C
M
I
nternational
Confer
enc
e
on
Multime
d
ia
,
pp.
171–180,
(Hilton,
S
ingap
ore),
No
v.
2005.