FluentScript_Building_DSLs_Meetupx - CodePlex

peruvianwageslaveInternet and Web Development

Feb 5, 2013 (4 years and 10 months ago)

162 views

Building DSLs for .NET using
FluentScript

Author:

Kishore Reddy

Company:

CodeHelix Solutions Inc

Site:

www.codehelixsolutions.com


Source/docs:

http://fluentscript.codeplex.com

Fluentscript:

www.fluentscript.com




Goals of Presentation

DSLs


What are DSLs


Approaches to developing DSLs


FluentScript


Summary of
FluentScript

and goals


Features ( that facilitate DSLs )


Plugins

/ Security / Using
FluentScript


Interpreter Process


Upcoming features


Sample DSL

order to buy
300

shares
of

IBM
at

$
150.50
on

9/20/2012

What are DSLs ?

DSL: Domain Specific Language



A
mini
-
language

for a particular
specialty
( domain )


(e.g.
Databases, Finance, Inventory


examples in this presentation
)


Goals and key features of DSLs:


Intuitive ( easy to read and easy to write )


Geared toward
domain

users and representation of the domain


Improves productivity


Minimal
syntactic noise
( e.g. no extraneous characters/syntax “;”)


Examples:

SQL :
select

*
from

users
where

role = ‘moderator’

Finance :
order to buy
300

shares of IBM at
150.50

on
10/24/2012

at
9:30
am





Internal DSLs

Basically using features of an
existing language
to create “Fluent” APIs

Leverages all the power of the existing language but has fixed syntax.


Ruby / Groovy :

order_to_buy

300
.shares, ‘IBM’,
150.50
, date ( '
8/20/2012
’ ), time ( ‘
9:30 am





C# / Java :

public

class
Order {



public

void

ToBuy(
Shares

shares,
string

sym,
double
price,
DateTime

dt

) {



// code here ….


}



public static void
CreateOrders() {




Order.ToBuy

(
300
.Shares(), "IBM",
150.50
,







8/20/2012
”.Date(), “
9:30 am
”.Time() );


}

External DSLs

Essentially creating
new

mini
-
language
for a specialty (domain).

Freedom to create your own syntax


Some tools make this process easier:

1.
ANTLR

2.
YACC

3.
Other tools ( Language Work Benches )


order to buy
300
, IBM,
150.50
,
9/20/2012

at
9:30

am


Some potential drawbacks:

1.
Learn to write grammar

2.
Can become complex

3.
Reinvent language features ( if, while, for, functions, etc )








What is
FluentScript









Internal DSLs





External DSLs




Hybrid approach


less noise
and

extensible


Open
-
Source
dynamic
*

scripting language


Language features (
var
, if, for, while, functions, etc )


C# / .NET 4.0
-

( available on
Codeplex

)


Host Language

New Language

FluentScript

FluentScript

Goals


Intuitive,
extensible
, safe



Intuitive


-

Minimal syntactic noise


Features


-

Datatypes, functions, plugins


Plugins


-

Every feature is a plugin (if, while)


Safety


-

Limits ( loops, recursion ), setup


In
FluentScript
, every feature is a
plugin


Every
plugin

can be turned
on/off

Using
FluentScript


http://fluentscript.codeplex.com/documentation


Embedded



FluentScript.dll

Used to integrate
FluentScript

into your

C# application



Execute
FluentScript

code from C#


Use C# objects in
FluentScript


Call
FluentScript

functions from C#


Access
FluentScript

variables in C#


using

ComLib.Lang

var

i

=
new

Interpreter();

i.Execute
( ‘
hello world in fluentscript
’ );


Standalone



FS.exe


Use
FluentScript

as a standalone

program, useful for general scripting



Execute scripts in
FluentScript


Dev scripts ( setup scripts )


QA automation scripts


Admin commands


Install: C:
\
tools
\
fluentscript
\
0.9.8.8

Cmd

>
cd
: c:
\
tools
\
fluentscript
\
0.9.8.8


fs.exe

file
:
scripts
\
helloworld.js


Resources for
FluentScript

Project site:

http://fluentscript.codeplex.com

Fluentscript:

www.fluentscript.com


Company:
www.codehelixsolutions.com






DSL Example1

# Example 1: Sample command in a fictional finance

#




related stock/asset ordering DSL.


order to buy
300

shares





of IBM





at $
150.50





on
10/24/2012

at
9:30

am

DSL techniques in
FluentScript

1. Data types are common to most DSLs


Such as dates, times,
bool

etc.
They need easier representation


Why does date have to be
new
DateTime
( 2012, 10, 24)

?


Why does time have to be
new TimeSpan( 9, 30, 45 )

?


2. Functions are very powerful but under
-
utilized


Function calls can be made much more flexible for internal DSLs

-
Enable non
-
conventional features that facilitate DSLs

-
Allow easier functions calls and with minimal syntactic noise

-
Allow easier method calls and with minimal syntactic noise


3. Need to ensure various levels of security


Simple mistakes can crash your system ( e.g.
infinite loops
, I/O )


DataTypes

-

Plugins

Data Type

Friendly Formats


Bool

on

off yes no true false


Date

September 10th


2012

Sep 10
th

2012 8/10/2012


Time

9:30 am 9am midnight


Day

Monday

Tuesday


Number

40.52

$40.52


String

‘name is john
’, “name is
#{user}



Tables

[


name

| pages | author


‘C# ’ , 120 , ‘Microsoft’


‘Erlang’ , 110 , ‘Joe’

]


URI

http://www.microsoft.com

home
\
app
\
log.txt
C:
\
file.txt

Data Types

note
:
< <= > >=
can be aliased via
ComparePlugin



# Dates

birthday
=

January

1
st

1979

if

birthday
is before

8/1/1991


print

you can vote


# time

time

=
3:30

pm

if

time
<

5
pm
then



print

still have to work!


# Uri

site =
www.microsoft.com


# Day

if

today
is not
Monday
then



print

its not start of week



# Money

salary = $
250

if

salary
more than

$
200

then



print

I worked overtime


# Email

email =
bat.man@gotham.com

DataTypes/Expressions



# Paths

home =
c:
\
apps
\
stockapp
\

file =
@home
\
logs
\
log.txt


# Environment

env.path

@user


# Version

version =
0.9.8.8


# Units

result =
3

feet +
5

inches


# Tables

books = [


name | pages | artist


‘C#’ ,
130

, ‘John’


‘Ruby’,
110

, ‘Dave’


]

favs = books
where

book.pages <
130


# Holiday

holiday = new years 2012


# multi
-
word string literals

@words
( fluent script, code camp)



Functions
-

Basics

Basics features:
( Ruby ,
Scala
, Groovy )

-
Optional semicolons

-
Optional parenthesis

-
Named parameters


refill_inventory

‘AB
-
1234’,
200


refill_inventory

product
: ‘AB
-
1234’,
amount
:
200


Functions


Advanced Features
1

Multi
-
word function names: using
FluentFuncPlugin


( Will discuss ambiguity and performance later on )

1.
Underscore matches






function

refill_inventory
( id, amount )






refill inventory

‘AB
-
1234’,
200

2. Camel case matches





function

refillInventory
( id, amount )






refill inventory

‘AB
-
1234’,
200

3. Multi
-
word names






function

refill inventory

( id, amount )






refill inventory

‘AB
-
1234’,
200



Functions


Advanced Features
2

4. Name aliases






function

hours, hrs, hour, hr
( num )






return new
Time( num, 0, 0 )






hours
(
4

)
hrs
(
3

)
hour
(
2

)
hr
(
1

)

5.
Suffixes






4

hours

3
hrs

2
hour
1
hr


6. Keyword aliases






def
hours, hrs, hour, hr
( num )






step
hours, hrs, hour, hr
( num )



Functions


Advanced Features
3

7. Function Wildcards:
by using
FuncWildCardPlugin

( Think of a wildcard in a game of cards, the wildcard can be anything )


# @wildcard: ‘name email’

# @parts: [ ‘name’, ‘email’ ]

# @args: [ ‘john’, ‘john@microsoft.com’ ]

def
create_user_by
*
( wildcard, parts, args)


{ .. code here .. }


create_user_by name email:
‘john’,
john@ms.com

create user by name email:
‘john’,
john@ms.com

create user by name email
(

‘john’,
john@ms.com

)

create user by name email birthday:
‘john’,
jd@ms.com
, 10/24/1980




Functions


Advanced Features
4

8. Fluent method calls :
FluentMemberPlugin



-

Optional dot notation


-

Reverse order of class name and instance name



-

Not case sensitive


delete file
c:
\
temp.txt

//
File.Delete
( “c:
\
\
temp.txt” )

user activate
‘kreddy’
//
user.activate
( “kreddy” )

activate user
‘kreddy’
//
user.activate
( “kreddy” )


<instance> <method> <params>

<method> <instance> <params>





Functions


Advanced Features
5

9. Fluent parameters: by using
FluentFuncPlugin


// @
arg
: name: shares, alias:

, type: shares,
eg
: 300 shares

// @
arg
: name: symbol, alias:
of
, type: text,
eg
: ‘IBM’

// @
arg
: name: price, alias:
at
, type: number,
eg
. 150.40

// @
arg
: name: date, alias:
on
, type: date, e.g. 8/15/2012

function

orderToBuy
( shares, symbol, price, date ) {



// create the order.

}

function

shares
( num ) {
return

new

Shares( num ); }



Functions


Advanced Features
5

9. Fluent parameters: by using
FluentFuncPlugin



@words( IBM, CITI BANK )



// Without parameter names

order to buy
300
,

IBM, $
150.45
,

10/24/2012

at
9:30

am


// With parameter names

order to buy
300
shares
of

IBM
at

$
150.45
,





on

10/24/2012

at

9:30

am


Functions


Advanced Features
5


Can still use colon, comma, parenthesis, for explicitly specifying parameters

Useful at times for making code less ambiguous to both end user / interpreter

// using colon
:

order to buy
300

shares of
:

IBM at
:

$
150.45

on
:

10/24/2012

at
9:30

am



// using
:
and
,


order to buy
300

shares
,

of
:
IBM
,

at
:

$
150.45
,

on
:

10/24/2012

at
9:30

am


// using
: ) ,

order to buy
(
300

shares
,

of
:
IBM
,

at
:

$
150.45
,

on
:

10/24/2012

at
9:30

am
)

Plugins

Almost every feature in
FluentScript

is a plugin

e.g.
var
,

if, for, while, function etc.


Plugins are used to:


Hook into various phases of the interpreter


Extend types, create new expressions/features


Extend syntax/semantics of interpreter


e.g.
9:30
pm,
10/24/2012
,
www.yahoo.com



Security

Overview

Optional security features can be enabled depending on
context of DSL and/or domain users.



Runtime limits


(Loops, recursion, string length, array size etc )




Plugins enabled/disabled


Examples:

1.
Maximum number of loops

2.
Prevent recursion

3.
Plugin specific configuration ( disable
File

I/O,
Exec
)



Miscellaneous

Ambiguity

1. Improved error checking will catch some ambiguity

2. Fallbacks: You can fallback to using explicit calls


delete file



=

File
.
Delete


refill inventory


=

refill_inventory


fluent script


=

‘fluent script’


create user by email

=

create_user_by email

Performance

1.
System level
plugins

(
var
, if, while
):
1

token of look ahead

2.
Fluent
plugins

(
fluentfunc
,
fluentmember
, words)


These are more expensive
plugins

since they require more
tokens to look at. E.g.
3

tokens in
create user by




Interpreter process

Lexer

( 2 plugin hooks)

Tokens

( token buffer
-
7 )

Parser

( 4 plugin hooks)

Interpreter

( no hooks yet )

Token

Kind

Pos

Value

0

Identifier

0

num

1

Symbol

4

=

2

Literal

6

10

3

Symbol

8

;

Assign: =

num

10

num = 10;

Executes top
level nodes in
the AST tree.
E.g. Assign

Plugins

Compone
nt

Role

Supports
Hooks?

Plugins

examples

Lexer

Converts characters to tokens

Yes
-

2

8/15/2012

http://yahoo.com

Parser

Converts tokens into AST

nodes

Yes

-

4

Expr
:
9:30

am

Stmt
: delete

file

Suffix
:
3

hours

Tokens
: is not

Plugins in
FluentScript

are similar to parser combinators with subtle differences

Plugins in
FluentScript

typically implement 2 methods and 1 property:

1.
StartTokens

= string[] { “
while
”}

2.
CanHandle( Token
token

) :
true

for system level
plugins

(
if, while, for
)

3.
Parse()


The CanHandle method indicates whether or not the plugin can take over parsing

The Parse method actually performs the parsing

FluentScript

Summary

Intuitive + Extensible + Safe



Hybrid approach to DSLs

:
Internal + External




Minimal noise



: easy to
read

&

write



Friendly types



:
9:30

pm,
10/24/2012
,

c:
\
file.txt



Flexible function features

:
order to buy
300

shares
of

IBM



Extensible




: plugins (
if, while for, dates, times

)



Safe





: ( limit loops, no recursion, setup )


Use cases


DSLs for end
-
users
( Finance, Accounting, etc )


Automation scripts for QA / Dev


Admin scripts for Support teams


Unit
-
testing / BDD testing framework


Introduce programming to younger students


Possible
general purpose language ???


Upcoming features


Modules


Multi
-
file support


Improved error handling and notification


More flexible function syntax


Lambda / Functions as first class types


Optionally use do/end instead of { } for block scope


More interpreter phases and hooks


Translators to convert
fluentscript

to
javascript


Hybrid static / dynamic typing


New release 1
st

of month! (4 releases in last 4 months)

http://fluentscript.codeplex.com


Conclusion

order to buy
300

shares





of

IBM





at

$
150.50





on

10/24/2012

at
9:30

am


Live demo :

www.fluentscript.com

Source/Docs:
http://fluentscript.codeplex.com

Company

:
www.codehelixsolutions.com