with fine-grained state tracking

makeshiftklipInternet και Εφαρμογές Web

31 Οκτ 2013 (πριν από 4 χρόνια και 8 μέρες)

71 εμφανίσεις

Optimizing dynamic dispatch
with fine
-
grained state tracking

Salikh Zakirov, Shigeru Chiba and
Etsuya Shibayama

Tokyo Institute of Technology

Dept. of Mathematical and Computing
Sciences

2010
-
10
-
18


code composition technique

Mixin

2

Server

BaseServer

Server

BaseServer

Additional

Security

Additional

Security

Mixin use declaration

Mixin semantics


Temporary change in class hierarchy


Available in Ruby, Python,

JavaScript

Dynamic mixin

3

Server

BaseServer

Server

BaseServer

Additional

Security

Dynamic mixin (2)


Powerful

technique of dynamic languages


Enables


dynamic patching


dynamic monitoring


Can be used to implement


Aspect
-
oriented programming


Context
-
oriented programming


Widely used in Ruby, Python


e.g. Object
-
Relational Mapping

4

Dynamic mixin in Ruby


Ruby has dynamic mixin


but only “install”, no “remove” operation


“remove” can

be implemented easily


23 lines

5

Target

application


Mixin is installed and removed frequently


Application server with dynamic features








6

class

BaseServer


def

process() …
end

end

class

Server < BaseServer


def
process()


if

request.isSensitive
()


Server.class_eval
{



include

AdditionalSecurity


} end


super

# delegate to superclass



# remove mixin


end

end

module

AdditionalSecurity


def

process()





# security check


super
# delegate to superclass


end

end

Overhead is high

Reasons


Invalidation granularity


clearing whole method cache


invalidating
all

inline caches


next calls require full method lookup


Inline caching saves just 1 target


which changes with mixin operations


even though mixin operations are mostly
repeated

7

Our

research problem


Improve performance of application which
frequently uses dynamic mixin


Make invalidation

granularity smaller


Make dynamic dispatch target cacheable in
presence of dynamic mixin operations

8

Proposal


Reduce

granularity of inline cache
invalidation


Fine
-
grained state tracking


Cache multiple dispatch targets


Polymorphic inline caching


Enable cache reuse

on repeated mixin
installation and removal


Alternate caching

9

Basics: Inline caching

10

ic

method

cat.speak
()

class

consider a call site

cat.speak
()

(executable code)

method = lookup(cat, ”speak”)

method(cat)

Dynamic dispatch implementation

if (cat has type ic.class) {


ic.method(cat)

} else {


ic.method = lookup(cat, ”speak”)


ic.class = cat.class


ic.method(cat)

}

Inline caching

Expensive!

But the result
is mostly the
same

Cat

Animal

subclass

cat

instance

speak() { … }

method

implementation

speak

Cat

Inline caching: problem

11

ic

method

cat.speak
()

class

if (cat has type ic.class) {


ic.method(cat)

} else {


ic.method = lookup(cat, ”speak”)


ic.class = cat.class


ic.method(cat)

}

Inline caching

Cat

Animal

cat

instance

Training

speak() { … }

speak(){ … }

speak

Cat


What if the

method has been overridden?

Inline caching: invalidation

12

ic

method

cat.speak
()

class

Cat

Animal

cat

instance

Training

speak() { … }

speak(){ … }

speak

Cat

if (cat has type ic.class
&&

state ==
ic.state
) {


ic.method(cat)

} else {


ic.method =
lookup
(cat, ”speak”)


ic.class = cat.class;
ic.state = state


ic.method(cat)

}

1

Global state

state

1

speak

2

2

Single global state object



too coarse invalidation granularity

Fine
-
grained state

tracking


Many state objects


small invalidation extent


share as much as possible


One state object for each family of methods
called from the same call site


State objects associated with lookup path


links updated during method lookups


Invariant


Any change that may affect method dispatch
must also trigger change of associated state
object

13

method

class

pstate

speak
*1*

State object allocation

14

speak() { *1* }

Animal

Cat

1

speak

ic

No

implemmentation


here

if (cat has type ic.class
&&

ic.pstate.
state

==
ic.state

) {


ic.method(cat)

} else {


ic.method
, ic.pstate
=
lookup
(cat, ”speak”,
ic.pstate
)


ic.class =
cat.class
; ic.state = state


method(cat)

}

inline caching code

1

cat.speak
()

state

1

Cat

speak() { *1* }

Animal

Cat

speak

ic

method

class

pstate

cat.speak
()

state

speak
*1*

speak
*2*

1

1

2

Mixin installation

15

1

Training

speak() { *2* }

2

2

Cat

if (cat has type ic.class
&&

ic.pstate.
state

==
ic.state

) {


ic.method(cat)

} else {


ic.method
, ic.pstate
=
lookup
(cat, ”speak”,
ic.pstate
)


ic.class =
cat.class
; ic.state = state


method(cat)

}

inline caching code

Training

speak() { *2* }

Cat

speak

speak() { *1* }

Animal

pstate

if (cat has type ic.class
&&

ic.pstate.
state

==
ic.state

) {


ic.method(cat)

} else {


ic.method
, ic.pstate
=
lookup
(cat, ”speak”,
ic.pstate
)


ic.class =
cat.class
; ic.state = state


method(cat)

}

inline caching code

method

class

cat.speak
()

state

2

speak
*2*

2

3

speak
*1*

3

Mixin removal

16

3

2

ic

Cat

speak() { *1* }

Animal

Cat

speak

Training

speak() { *2* }

method

pstate

state


Detect

repetition


Conflicts detected by
state check


speak
*1*

speak
*2*

3

4

Alternate caching

17

A

3

4

super

Animal

alternate cache

speak



3

4

Training

ic

class

cat.speak
()

Cat

Inline cache contents oscillates

speak() { *1* }

Animal

Cat

speak

Training

speak() { *2* }

method

class

pstate

state


Use multiple entries in
inline cache


Polymorphic caching

18

4

ic

3

super

Animal

alternate cache

speak



3

4

Training

cat.speak
()

Cat

Cat

*1*

*2*

3

4

Q

Q

Cat

speak

Training

speak() { *2* }

speak() { *1* }

Animal

State object merge

19

executable

code

cat.speak
()

S

Overridden by

One
-
time invalidation

animal.speak
()

cat

instance

animal

instance

while(true) {


remove mixin

}

Overheads of proposed scheme


Increased memory use


1 state

object per polymorphic method family


additional method

entries


alternate cache


polymorphic inline cache entries


Some operations become slower


Lookup

needs to track and update state objects


Explicit state object checks on method dispatch

20

Generalizations (beyond Ruby)


Delegation object model


track arbitrary delegation pointer change


Thread
-
local delegation


allow for thread
-
local modification of
delegation pointer


by having thread
-
local state object values



Details in the article…

21

Evaluation


Implementation

based on Ruby 1.9.2


Hardware


Intel Core i7 860 2.8 GHz

22

Evaluation: microbenchmarks


Single method call overhead


Inline cache hit


state checks 1%


polymorphic inline caching 49% overhead


Full lookup


2x slowdown



23

Dynamic mixin
-
heavy microbenchmark

100%

23%

17%

15%

base
method cache
state checks
fgst
fgst+PIC+altern
Normalized execution time
24

(smaller is better)

Evaluation: application


Application server with dynamic mixin on
each request


25

100%

70%

58%

60%

52%

baseline
method
cache state
checks
fgst
fgst + PIC
fgst + PIC +
altern
Normalized execution time
(smaller is better)

Evaluation


Fine
-
grained state tracking considerably
reduces overhead


Alternate caching brings only small
improvement


Number of call sites affected by mixin is low


Lookup cost / inline cache hit cost is low


about 1.6x on Ruby

26

Related work


Dependency tracking in Self


focused on reducing recompilation, rather
than reducing method lookups


Inline caching for Objective
-
C


state object associated with method, no
dynamic mixin support


27

Conclusion


We proposed combination of techniques


Fine
-
grained state tracking


Alternate caching


Polymorphic inline caching


To increase efficiency of inline caching


with frequent dynamic mixin installation
and removal

28

Thank you for your attention

29

Method

caching in Ruby


Global hashtable


indexed by method name and class


On method lookup


gives answer in 1 hash lookup


On miss


answer obtained by recursive lookup


result stored in method cache


On method redefinition or mixin operation


method cache cleared completely


30