SSCA_for_Ruby_rev7 - UCCS | College of Engineering and ...

arrogantpreviousInternet and Web Development

Feb 2, 2013 (4 years and 2 months ago)

200 views


1



Abstract

The amount of software being developed and new
programming languages being created is on the rise. As new
languages are born, security is often overlooked until the
language has reached a certain maturity and acceptance level.
Manual code audits

become unpractical and error prone when
the number of source lines of code become
s

excessive
,

and
leveraging code generation and frameworks only
compound

this
problem.
Ruby is one of the program
ming

languages that
are

gaining popularity

due to web develo
pment

and
the related R
uby
on Rail framework

which

integrates

database
with the
Model
View Controller design pattern to

facilitate the construction of

the web applications.

Currently the programming language Ruby
has very limited number
of
static
source co
de analysis tools and
even smaller number of these

tools

locate potentially vulnerable
code
.

The programming languages C/C++, Python, Perl, Java
and C# all have active open source and commercial static source
code analysis products to help with automated
code analysis.
This paper attempt
s

to remediate that for the programming
language Ruby.
O
pen source static source code analysis tools

are
compared

and evaluate
d

in order to extend one in support of the
Ruby language.
S
ecurity problems

that

exist in modern
programming languages

are e
xamined and categorized

to build a
database of functions that the
enhanced
static source code
analyzer should report.
The results of applying
this new tool to
scan a variety of open source and contrived Ruby app
lications
have
demonstrated

the validity and practicality

of static source
code analysis on the Ruby programming language.


Index Terms

Ruby, Static source code analysis, security,
programming


I.

I
NTRODUCTION

Ruby is a dynamic, open source programming langu
age
with

a focus on simplicity and
productivity [
1]
.

The
popularity and use of the Ruby programming language has
started to rapidly grow since mid 2006[2].

Gartner reports that
they expect 4 million Ruby developer by 2013 and that “Ruby
will enjoy a high

concentration

among
corporate

IT
developers than typical dynamic ‘scripting’
languages
, such as
PHP”

[3].

They also attribute much of the success of Ruby to
the advent of the Ruby on Rails framework.

It is clear that
more and more developers are using t
he Ruby language and
also leveraging it to program web applications.

A recent study has found 97% of web applications
vulnerable to one form of attack or another [4].

According to
the TIOBE Programming Community Index, Ruby currently
rates in the top 10 pr
ogramming languages

[2]
.

With such
increase of

vulnerabilities

in general

and wide acceptance of
the language, further
investigation

into building secure and
trustworthy Ruby applications is needed. There are currently
best practices and security guidelines for Ruby and Ruby on
Rails applications [5], however, there are very limited tools in
the domain of automated auditing to help ma
ke sure those best
practices and given security
criteria

are met.

To create such static source code analysis tool, first
the
Ruby language security vulnerabilities

need to be categorized
and rated
.
We use existing vulnerable functions from the Perl
langua
ge to map the same functions to Ruby to create a Ruby
vulnerability database.
This
database

is

then
integrated

in
to an
existin
g source code analysis tool, which is used to
conduct
source code level security audits

on a set of open source Ruby
software pac
kages
to understand the effectiveness of the tool
.

A.

Ruby Language Syntax

This section attempt
s

to outline some of the Ruby syntax
and
related software packages
that will need to be considered
when
perform
ing source code analysis on the language.

We
have ch
osen examples that show interesting features of the
language but demonstrate concerns for analyzing it.


1.

Ruby supports multiple ways to call a function:


add value1, value2

add(value1, value2)


2.

Ruby does support curly braces {} but does not use
them for
surrounding classes and methods:


class Math

def add (val1, val2)





return val1 + val2






end

end


3.

Ruby reuses method names

on different objects, so
when scanning for a given function, one will need to
keep the name of the object in mind. This is different
from C syntax where all function calls are unique (i.e.
Static Source Code Analysis for Ruby

Michael S. Gerschefske


Department of Computer Science

University of Colorado at Colorado Springs


in Partial Fulfillment of the Requirements for

the Degree of Master of Engineering

in Information Assuranc
e, 2009


2


fopen):


File.new(“a.txt”,”r”)

Dir.new(“dir”)


4.

Ruby supports the
concept of Duck typing. Duck
typing is a style of dynamic typing in which an object’s
current set of methods and properities determines the
valid semantics, rather than its inheritance from a
particular class or implementation of a specific
interface
[19]
.

In essence, if an object has both quack
and waddle methods, it can be treated as a duck,
regardless of what it really is.


5.

Ruby has a concept of sandboxing called the Safe.
This allows a program to be ‘locked’ in the Safe at
different security levels
[
6]
. At safe level
0

nothing is
done, however at safe level
1

the main restriction is
that
strings must be untainted before
system commands are
called. At level two, many file and direct
ory operations
become revoked and as the safe level further rises Rub
y
does things like preventing the loading of lib files from
world writeable directories
[15]
.

II.

C
ODE
A
NALYSIS FOR
R
UBY

Before we can search for bugs or security vulnerabilities in
Ruby code we first need to understand what those could be.
Not all programmin
g languages are susceptible to the same
programming errors

or vulnerabilities
, so we first need a good
understanding of those that exist in Ruby. Furthermore we
want to automate the tasks of searching for common bugs.

A.

Understanding Ruby Vulnerabilities

Interpreted programming languages like Ruby
must

be
handled differently than traditional programming languages
like C/C++.
Ruby lacks pointers, has garbage collection and
array index checking
and so forth,

inherently

make
s

it easier
and safer
to write

w
ithout introducing security problems such
as buffer overflow and stack execution in

C
1
.

Ruby is a
lso a

sophisticated scripting language

in that

it has
the ability to understand when objects have
interacted with
external

user input

and marks them as tainte
d
.

The developer
can then check to see if an object is tainted by calling the
“tainted”

method.
A
fter the developer has checked the value,
they have the ability to mark the object as untainted by calling
the
untaint

method. Calling untaint does not force

the
developer to check the
user input

correctly, nor does it require
the developer to check the value at all. The Ruby language
guide recommends against rolling your own checks/regex
expressions before untainting a value and recommends using
some built i
n libraries. However, even this can be dangerous
because while one library may prevent against XSS, it could

1

Note: These assumptions are based off the Ruby programming language
as being run by an interpreter. While there may be buffer overflows in the
interpreter that is not what is being analyzed, rather the Ruby language syntax.
The interpreter errors can be

dealt with by maintaining good security patch
management practices. Furthermore, this does not cover Ruby being
embedded into C code.

still contain SQL or command injection in it. Ruby seems to
lack the native ability to mark how the object has been
untainted, which would preven
t this problem. Alternatively,
some have suggested never untainting an object, however, this
is not realistic if running at a safe level greater than 0
[10]
.


The Ruby safe feature, at level 1, automatically prevents
the following functions from being cal
led with tainted input:


File.chmod

File.chown

File.lstat

File.truncate

File.umask

File.flock

IO.ioctl

IO.stat

Kernel.fork

Kernel.syscall

Kernel.trap

Process.setpgid

Process.edgid

Process.setsid

autoload

Process.setpriority

File.chroot





D
ata that has been marked untainted by the Ruby
run
time
does not necessarily mean the data is safe. The untaint

function only checks if the value contains special characters
that could potentially allow for escaping out of a command
and doing a potential SQL or command injection. The untaint
function does not protect against bad

design

logic.

A
ssume a
Rails web ap
plication

that allows users to post
reminders to themselves on a website, and then in turn, they
can come back at a later time to check them and view other
users’ reminders. One might write these reminders to the file
system (or could just as easily be a
database) that contain all
the reminders. These reminders could be stored in /local/data/
to prevent other users from directly accessing them. Now, if
by mis
s
-
configuration, the login passwords file is stored in this
same directory, a rouge user may be a
ble to recover this.
Given:


/local/data/user1

/local/data/user2

/local/data/passwdfile



and the code:

fname

= params[:user]

fname.untaint

if fname

=~ /([a
-
z0
-
9])/

f =
File.open(File.join(“/local/data
”,
fname
)


it
would be possible to allow access to the
miss
-
configured
password file.

Problems like this exist in large projects as
communication breakdown occurs as not everyone know
s

how
their

piece fits in with everyone else’s and the entire
architecture
[7]
.

In this case if we assume two different
developers working on different parts of the system, this could
realistically happen. However, if our static source code
analysis tools marks File IO as dangerous, this may be
something that can be marked for a fu
rther code review
automatically.


Furthermore, Ruby is also susceptible to TOCTOU (Time
of Check to Time of Use) vulnerabilities. Given certain
instances of Ruby method calls and how they are used, these
race conditions can exist. Furthermore, combined w
ith
symlink attacks
[12]

this can be a very dangerous problem.
Symlink attacks
usually

involve a world writeable directory,
such as
/tmp

and the given application making checks and
then writing to that directory. If the attacker can create a file,

3


or lin
k after the application checked if the file exist, but before
the application wrote to the file, the attacker could potentially
direct where the contents of the file was written.


The following example shows this problem in Ruby:


01
if !File.exists(“/tmp/
user”)

02

f = File.open(“/tmp/user”, “w”)

03

f.puts “my secrets here”

04
end

Code Example 2


now if in between the check for if the file exists and the
opening of the file, some user creates a symbolic link to
/
tmp/user

and directs it to their home dir
ectory, eventually,
if ran enough time, will allow the user to be successful in
having access to

the contents of the file.

Furthermore, this
kind of attack could be directed to cause denial of service to
the system, if the attacker chose to overwrite file
s that the
application had access to.


We have successfully carried out this attack with Ruby,
however for our testing purposes, we added a
sleep 10

in
between the lines one and two off code example 2. This gave
us ample time to manually create the symbol
ic link. Without
the sleep method, precise timing would be needed to carry out
this attack, however, through the use of automated scripting, it
should not take too long before it is successful.

B.

Code Analysis

Code analysis is a needed phase of the software

development lifecy
cle and can help reduce bugs
.

Because
manual code analysis is an extremely time consuming and
mundane process it often gets done either incorrectly or not at
all. This is often referred to as the “go home, get done”
princip
le

w
here
towards the end of the date

people are more
focused on leaving work then trying to sift through thousands
of lines of code looking for security vulnerabilities.
Automated code analysis tools help

this by

limiting the scope
of an application that needs to
be audited. Rather than audit
the entire application, we can instead look at the parts that are
more likely to have bugs.

Automated code analysis is not guaranteed to find all the
bugs in a given application and at best can only discover 50%
of the bugs

[9]
. The other 50% of the bugs lie in architecture,
configuration and other aspects outside the source code.

While not all bugs are vulnerabilities, all vulnerabilities are
bugs
.
F
ixing bugs can help reduce the chance of the bug
manifesting into a vulner
ability. Therefore pointing out any
part of code that may just be a bug will help increase the
overall security of the application. Qmail, a mailer program
like sendmail, had a major release in 1998 and since have not
had one exploitable vulnerability

[11
]
. Their success has been
partially through making the assumption that all bugs are
vulnerabilities.

A good stra
tegy for auditing source code is to identify all
points in the source code where the program may take input,
whether from the network, file sys
tem, or user
and perform
manual code analysis on
those

locations
[8]
.


A
ny IO coming
into the application must be checked. Using this model, we
could assume that no output needs to be looked at, however,
outbound IO, if not properly checked, could have ad
verse
effects on the application, host or even clients, so it must be
checked as well.

C.

Static Source Code Analysis Tools

Static source code analysis tools attempt to reduce the
burden of manual code analysis.
T
hey attempt to find areas in
code which may b
e more
susceptible

to errors as a starting
point to conduct intelligent manual code analysis.

They are not
a fool proof product and do not eliminate the need for manual
code analysis nor do they stake claim to find all the
vulnerabilities in source code.

Rather than create a new code analysis tool we decided to
extend an existing tool. We looked into RATS, ITS4
[20]

and
flawfinder

[21]
. While the source code to ITS4 is available, it
is not an open source product and is maintained by
Cigit
al.
RATS and fl
awfinder were both created to
be

a completely
open source static source code analysis tool. Currently
flawfinder only works for C code, however, RATS supports C,
PHP, Python and Perl. Since RATS already supports multiple
languages we found that it was th
e best choice to extend
the
support
for Ruby
.

Many of the source code analysis tools that exist today
are
ran outside of the development environment. This requires
that the developer perform security checks in a separate
application and then bring the re
sults back into the
programming environment in order to make the changes. The
tool SSVChecker allows running both RATS and ITS4 inside
the eclipse environment and allows the user to click
vulnerabilities and go directly to the line in code being
affected

[22]
.

D.

RATS

RATS, Rough Analysis Tool for Security, is an open source
static source code analysis tool. It supports C, Python, Perl
and PHP languages. It
has XML files that contain the
vulnerabilities and can be

modified to

support new
vulnerabilities
. It is also capable of generating output in either
console, HTML or XML documents. It supports multiple
warning levels that allow different verbosity of error levels,
high, medium and low.

RATS was released as an open source
alternative to ITS4.

RATS w
orks by passing a file to scan, or giving it a
directory which it will recursively scan for file extensions it
understands and process and report them. The first thing
RATS does is read in all the available XML files which allow
it to store in memory all
the vulnerabilities for all the different
languages. This allows it to report on multiple language
vulnerabilities on multiple files without having to run it
multiple times.

RATS then does a token scan on the files, which interacts
with the lex file that
models each of the languages. The lex
allows for different regular expressions and if needed, calling
extra C code to return the tokens. Some of the tokens, such as
the comments, require this processing to determine where the
comment ends. As each token

is processed, it is vetted
through the system. Simple tokens such as constant strings are

4


analyzed and for now simply disregarded. The identifier
tokens are further analyzed

and processed to determine if they
are function calls or not. This is done by
analyzing if the next
token is a parenthesis ‘(‘, something which is generally
optional in the Ruby language.

It then checks if that token is
in one of its already loaded in vulnerability databases and if it
is, it logs it for output. Additionally, it ch
ecks to see if the
function call is involved in a TOCTOU race condition. If it is,
it attempts to log it and the other functions involved with the
race condition. Finally, at the end, RATS outputs all of its
logged vulnerabilities to the screen.


RATS is

meant to provide a starting point to perform a
manual source code analysis.
Its

goal is to minimize false
negatives
, but still may generate many false
positives
.

III.

M
ODIFYING
R
ATS

First, the programming/build environment must be setup
correctly at which p
oint we can start to extend RATS via C
code. Additionally, a lex file needs to be created that properly
characterizes the Ruby language syntax to return tokens. At
that point RATS is extended to support a new language.
Following that, an appropriate dat
abase of vulnerabilities must
be created so that RATS will search for those vulnerabilities in
the new programming language.

A.

Setting Up the RATS Development Environment

RATS supports both Linux and Windows with the same
code baseline. We have chosen to do

the following
development on the windows platform. The following
components are required to be installed before development
could begin on the Windows platform:


1.

Microsoft
Visual Studio 2008

2.

Cygwin with flex and make

3.

expat (
http://expat.sourceforge.net/
)


With these tools installed, all source code editing, debugging
and compiling was done in Visual Studio.
RATS already
included a Visual Studio 6 project and workspace file which
was compatible with Visual Stu
dio 2008.
Cygwin was
used

to
run the accompanying make file which compiled all the lexical
programming language files into .c files.


On Ubuntu, to setup the source to compile we need to issue
the following commands and then we can run the make
command:


aptitude install build
-
essential

aptitude install libexpat1 libexpat1
-
dev

B.

Modifying RATS


RATS provides

a framework for static source code analysis.
Since it already provides language support for C, Python, Perl
and PHP it was a logical choice to extend to Ruby. This is
mainly because of the existing support for python, which
python’s syntax is more close
ly related to Ruby then either
Python or Ruby is to C.

In
the
RATS engine.h file

all the language are initially
defined and Ruby was added accordingly:


#define LANG_PYTHON


1


#define LANG_C






2


#define LANG_PERL



3

#define LANG_PHP




4

#define LANG
_RUBY



5


We also modified RATS to include scanning for .rb files so
the parser would automatically use the Ruby language when it
encounters those files.


Much effort went into the creation of the Ruby lexical
document to allow RATS to properly parse Ruby
. All of
Ruby’s native language constructs
(keywords) had to be
modeled to return appropriate tokens to the parser. This also
included modeling for constant values and strings.

The lexical
file also includes definitions to deal with Ruby style
comments,

both =begin/=end and # style. By default RATS
does not support tokenizing comments for Perl, Python or
PHP. This prevents those languages from using comments to
tell the RATS parser to ignore vulnerabilities. This can be
especially useful if a given vu
lnerability in code has been
reviewed and the developer wished to mark it as not
vulnerable anymore. After being marked, RATS will suppress
any warnings that function may give. The RATS parser will
ignore vulnerable functions that have a comment appended

to
that line that either contains rats:ignore or its4:ignore, for
backward compatibility. We have extended this functionality
to the Ruby l
anguage
, but have left the other languages RATS
supports unmodified.


Additionally, RATS needed to be modified to a
llow
identification of both styles of method invocation in Ruby.
The modification, which only affects the Ruby language in
RATS, attempts to process all identifiers, whether they are
methods of variables. However, the lexical piece is set to
prevent
anal
ysis of
identifiers starting with @, @@ or $,
which is the conventions for
instance, class
and global
variable. Within the
Rails

community there is a large push
towards convention over configuration, which assumes the
developer is following the given conv
ention, and if so, is
rewarded in the framework being easier to use
[14]
. Much in
the same way we are assuming convention with this extension

as Ruby is an extremely dynamic language
.
That being said,
following Ruby standard convention, the RATS Ruby code

analysis tool will present the developer with potentially less
false
positives
.


One possible example of this is
,

consider the code below:


01 file = open(“test.txt”,
“w”)

02

03 open = “text.txt”

04 open = open open, gets.chomp

05

0
6
send_file(
'/uploads/' +
params[:fname])


5



Line one shows simple syntax as to how a file is normally
opened in Ruby. Line four attempts to demonstrate how
dynamic the Ruby language in fact is.

Ruby understands that
the first token
on line four after the equal sign i
s a method call
,
which we can deduce through syntax as well. However, the
second open operand after the method call, being open, we (as
RATS) are not able to determine if this is
supposed

to be
another method call to the open method, or whether it is
refe
rencing the open variable. Ruby is able to make this
distinction because it understands that the open method does
not have any empty method signatures. In fact, Ruby will
allow you to do syntax like this
, even including using keyword
names as variables,

as wanted as long as
it can still make since
of it. Since RATS does not model all the methods and their
method signatures, it is not possible to make this distinction.
We
know it’s a parameter, however,
we
cannot tell if it’
s a
method call or a variable
so we must treat it as a method call
and occur any false positives this may generate.


We may also want to check the user of variable names in
our source code analysis. Again,
Rails

has the concept of
convention over configuration. That being said, some
convention
s dictate

the use of certain variable names when
working with different frameworks. A popular convention
in
rails
is passing the parameters from a web page request via the
params variable. If we configure RATS to allow for warnings
when using t
he params variable, we should be allowed to warn
at most attempts when accessing un
-
trusted data. Again, since
we had to modify RATS to do this for line four, we also take
advantage of this with line 6. Line 6 shows taking parameters
in from the params v
ariable (given by the Rails framework)
and then returning back a file. Line 6 is an example of a
directory traversal attack which an attacker using
../../
,
could return any file on the system that the application has
access to. Having RATS warn about the

user of the params
variable
being used solves this problem.

C.

Reviewing
Perl Vulnerability Database

To get a better understanding of what vulnerabilities should
exist in the Ruby database we should first look at the other
language databases to understand
how to categorize some of
the vulnerabilities. After we understand what the
vulnerabilities are, we can start to look for those in the Ruby
language and create our own database.


W
e have extracted the
Perl function
vulnerabilities and their
ratings from RATS
and have listed it in the table below
. The
vulnerabilities are rated from high to low, and the ones marked
TOCTOU are rated low as well.

srand

medium

syscall

high

symlink

medium

rand

medium

connect

high

truncate

medium

getc

toctou

system

medium

chroot

medium

readdir

toctou

open

medium

umask

medium

read

toctou

unlink

medium

kill

medium

sysread

toctou

mkdir

medium

ioctl

medium

exec

medium

chdir

medium

eval

high

fcntl

medium

rmdir

medium

glob

high

bind

medium

chown

medium

fork

low

setpgrp

medium

chmod

medium

gethostbyname

high

setpriority

medium

link

medium

gethostbyaddr

high







At a high level we notice these vulnerabilities can be broken
into a few main groups. These groups are:

1.

System
execution

2.

File system access

3.

Kernel interactions

4.

Security and encryption

5.

Remote
communication

We could probably assume it would be a good idea to at a
minimum to map all these functions to Ruby and then copy
them into the Ruby vulnerability database and th
e assumption
would be mostly right. However, first we need to understand
what makes all these functions
vulnerable
. While we wont
look at each
individual

function, we will look at the four high
level categories we have identified and map the functions in
to
those categories.

Some functions we will specifically call out
by name to better appreciate their vulnerabilities.

1)

System Execution

This would be any function that allows commands to be
executed. These commands are either passed to the system to
be ex
ecuted

(i.e, echo, ls, cat)

or allowed to be evaluated and
executed by the
interpreter

(i.e., inline scripting)
.

All system
execution needs to make sure that if any user input is passed to
it, it is properly checked to prevent command injection.

In Perl,
we can map these functions to include exec,
syscall,
system and eval.
In Ruby this will be a very similar case,
however, it should include as many commands that allow
execution as we can discover via the Ruby documentation. In
addition, Ruby also support
s execution by using the back
-
tick
operators, however, RATS already has that check so we don’t
need that in the vulnerability database.

2)

File System Access

These are all the functions that allow with interacting with
the file system.

Many of the file system commands, if done
incorrectly, could create a TOCTOU race condition. Also
when interacting with the file system, all user values need to
be properly checked to prevent them from being able to do
rogue

activities

on the file syste
m. Our previous examples in
this paper show how if unchecked,
what is possible for a user
to do.

Referencing the Perl vulnerabilities table, the functions
involved her are chown, chmod, chroot, open, setpgrp and all
the other functions that deal with file

system access. Of note,
rmdir has been problematic in the past for TOCTOU race
violations

[12]
.

3)

Kernel Interaction

These functions are the set that interact with kernel or OS to
accomplish specific tasks. These functions usually don’t take
user input, b
ut should be checked in case. Proper analysis
should be done as if these functions are used incorrectly they
can change how the program runs and also affect performance
and even stability on the host machine.

In our Perl tab
le these functions include fork
,
ioctl

and kill
.
Fork can not only affect system performance, but fork makes a
replica

of the

original

thread when forking

[18]
. So if there
are any security specifics in the original thread, they will get
copied as well, this is especially problematic
for random
seed
s.


6


4)

Security and Encryption

No one should ever roll their own encryption

(unless they
are creating encryption products)
. However, if you do you
should not use the stock random functions included with any
language. While people may not use t
his for encryption it is
sometimes used to generate unique hashes for session keys,
which are not secure. Therefore, with such easy abuse of the
random number generators that is something we need to check
for. Any security or encryption algorithms that a
re difficult to
use or are often used incorrectly, this would be a prime
category to add these too.

In our example above, the Perl
language includes both srand and rand.

5)

Remote
communication

These are the set of functions responsible for talking with
othe
r system via some communication channel. This could
include network communication, but could also include inter
process communication. Communication channels can be
interrupted, denied and even forged, so any time we create or
talk over such a channel, w
e need to be aware of that and plan
accordingly.

In our Perl example, the methods here are socket,
gethostbyname and gethostbyaddr. The gethostby* functions
were added because of the ability to forge there results
through network manipulation
and as suc
h cannot be trusted.


Things such as arp poisoning could allow an attacker to man
in the middle these requests and replace the host with
whatever they wanted.


D.

Derived/Created
Ruby Vulnerability Database

With RATS extended to support Ruby and now

that we
h
ave a better understanding
of the types of vulnerabilities that
can exist in a language, we need to start to build the list of
vulnerabilities for the Ruby language.

We will use the
vulnerabilities defined in the Perl database as a start and also
leverage

the functions that Ruby disables at safe level 1.
Additionally we will try to create an initial list of any other
functions that may cause vulnerabilities by analyzing the
functions in Ruby’s core libraries and researching what
functions have been report
ed as dangerous.

The list of Perl vulnerabilities and the Ruby safe level 1
functions have a large intersection
.

Looking closely at the
safe level 1 intersection we can correlate some initial
vulnerability levels based on the matching Perl function
vulner
ability level. The third column of function on the right
did not intersect the two lists.


chmod

medium

chown

medium

lstat

umask

medium

ioctl

medium

stat

fork

low

syscall

high

trap

edgid=

medium

setpgid

medium

autoload

chroot

medium

truncate

medium

setsid



setpriority

medium

flock


The lstat and stat functions allow statusing meta
information about files from the
file system
.

The trap function
allows trapping of signals, autoload loads ruby files during
runtime, flock locks files and setsid

establish the process as a
new session a process group leader
[16]
. This remaining
function, while not marked as vulnerable in the Perl
vulnerabilities, are marked as potentially vulnerable by the
Ruby programmers guide. Therefore, we will treat these
f
unctions as a medium level in our XML database.

The remaining Perl functions can be mapped to the
following Ruby functions:

Perl

Ruby

Severity

Perl

Ruby

Severity

srand

srand

medium

connect

connect

high

rand

rand

medium

system

system

medium

getc

getc

toctou

open

open

medium

readdir

*Dir*

toctou

unlink

unlink

medium

read

read

toctou

mkdir

mkdir

medium

sysread

sysread

toctou

chdir

chdir

medium

exec

exec

medium

rmdir

rmdir

medium

fcntl

fcntl

medium

link

link

medium

bind

bind

medium

symlink

symlink

medium

g
ethost

byname

“same”

high

kill

kill

medium

g
ethost

byaddr

getaddrinfo

high

eval

eval

high




glob

glob

high


This allows us to reuse the existing RATS Perl database and
extend that to our new Ruby database. All the functions map
with the exception of the readdir function. The Dir class in
Ruby
can be accessed like a variable array to return the files in
a direc
tory as such: Dir[“/tmp”]. Therefore in order find this,
we must mark the Dir variable as vulnerable and search for
access to that throughout the application. Unfortunately, this
will generate false positives for developers using Dir as either
a variable

name, or as a reference, however, we rather
generate extra false positives than to have false negatives.


While one might be inclined to stop there, we decided to
evaluate the methods in the following classes: IO, File, Kernel
to try and find any methods
that could be used in potentially
vulnerable ways.

After evaluating the File classes we have found the
following methods below, which we believe may allow for
TOCTOU race conditions. Additionally, the rm_rf class
specifically warns in the documentation ab
out the possibility
of TOCTOU
.


1.

File.file?

2.

File.directory?

3.

File.exist?

4.

File.executable?

5.

File.identical?

6.

File.zero?

7.

FileUtils.rm_r
/safe_unlink (alias)

8.

FileUtils.rm_rf
/rmtree (alias)

9.

FileUtils.remove_entry_secure


Additionally, we find in this class the foll
owing functions of
significance

because they interact with the file system
:

1.

chmod_R

2.

chown_R

3.

ln_s

4.

mkdir_p


7


5.

mkpath

6.

touch

These functions, if user input is passed to them and not
properly checked, could cause adverse effects to the
application and host.

We wi
ll mark these a
t

a vulnerability
level of medium

because there similar method calls in the Perl
vulnerability database are also marked as medium
.


After analysis of the IO class by inspection, we find the
only additional method is popen. popen

allows execution of a
command and passing its standard input/output back via
objects. Because any system command can be executed it
needs to be checked to make sure no unchecked user input is
passed to it.

Both popen and system are shown to be difficult

to use correctly and are commonly used in security critical
applications [23].
Additionally, we have found a popen3
command that is outside of the IO class, but should be
checked for nonetheless.


After
reviewing

the Kernel class we find that only one
additional method should be added. The method load allows
the application during runtime to load external files. If a path
to the file to load is taken by a user input, the input must be
checked to make sure it’s s
afe. Of note, most of the other
kernel functions, syscall, etc., have already been added by
either the Perl list or the safe level 1 list.



Other potentially dangerous commands are send_files
which was covered earlier in this paper. That method will be
marked as a high on the vulnerability scale because if
improperly checked, can allow for an attacker to access all the
files the current application user has access to.

Additionally,
any time the untaint method is used it should be checked to
make sure th
e object should really be untainted. If improper
checks are done, marking an object as untainted may give it
more privileges with having malicious data still in it. We will
be marking the untaint method as medium because of this.



With the ability to al
so check variable names we can also
create a list of potentially vulnerable variables. The first
would be the params variable, which included unchecked user
data. If the application incorrectly validates the data in this
variable, it can cause adverse ef
fects on the application. We
will mark
this

medium because it can potentially contain
malformed data.

IV.

RATS

R
UBY
T
ESTS

In order to validate the tool we have chosen to run the tool
on a variety of Ruby code and analyze the output.

We have
performed the tes
t with
both
code known to contain
vulnerabilities to validate the tool can find vulnerabilities and
we have also analyzed various open source Ruby projects to
determine if they have vulnerabilities in them.

We have ran the tool on the projects to generate
audit reports
and then have picked various audit points and preformed
manual audits to determine the validity of the warnings. We
expect the tool to generate false positives; however, we also
expect the tool to provide dramatic reductions in lines of code

that require manual auditing.

We have chosen three Ruby projects to test the tool on. The
first test IPAL was chosen because it was created to contain
vulnerabilities. The second test is a Ruby web app that was
chosen because it was small in size and de
veloper base and is
still in early stages of development and could potentially
contain errors. The third test is a early release of a popular
Ruby software package and was tested because the early
versions may contain
vulnerabilities that have been patche
d by
later versions.

RATS maintains extremely high scanning speed with the
Ruby
language
. The speed results on many of the tests are
large negative numbers because the current RATS
implementation is not able to handle that small of times. We
have found w
hen running RATS, to scan 503,847 lines of
code, took approximately 3.953 seconds.

A.

IPAL Test

The first test of RATS was on a Ruby application from the
UCSB iCTF (
international

Capture The Flag) 2006
competition. iCTF is a hacking competition where
geograp
hically dispersed teams download a vulnerable
VMware image which contains many services, and have to try
to patch all the services while trying to keep other teams out
and yet at the same time gain access to other teams systems.

The 2006 competition had a
Ruby on Rails service called
ipal which if a factious service that allows users to deposit and
withdraw funds from their bank account and also provided
currency exchange.

The service is known to contain
vulnerabilities, however, where they are and what th
ey are is
unknown to the user.

Running the newly modified RATS with the new
vulnerability

database provides
us with the

following results:


RATS Results.

Severity: High

Issue: popen

Unchecked user input could all
execution

of system
commands.

File:
ipal/a
pp/controllers/deposit_controller.rb

Lines: 38


Severity: Medium

Issue: params

Use of params, verify all user values are checked before
using. Never pass params directly to a new object i.e.
Object.new(params[:user])

File:
ipal/app/controllers/
admin_controller.rb

Lines: 52 43 42 38 28 20

File:
ipal/app/controllers/currency_controller.rb

Lines: 48 39 38 34 24 16

File:
ipal/app/controllers/deposit_controller.rb

Lines: 91 87 83 74 73 69 54 33 33 21

File:
ipal/app/controllers/
session_controller.rb

Lines: 31 20 18 18

File:
ipal/app/controllers/user_controller.rb

Lines: 20 72 64 55 13 10





File:
ipal/vendor/plugins/







acts_as_authenticated/generators/


8








authenticated/templates/controller.rb





Lines: 26 16 14 14


Severity: Low

Issue: directory?

A potential TOCTOU (Time Of Check, Time Of Use)
vulnerability exists. This is the first line where a check has
occurred
. No matching uses were detected.

File:
ipal/config/boot.rb

Lines: 15


Severity: Low

Issue: directory?

A potential TOCTOU (Time Of Check, Time Of Use)
vulnerability exists. This is the first line where a check has
occurred
. No matching uses were detected.

File:
ipal/public/dispatch.rb

Lines: 9


Total lines analyzed:
3108

Total time
0.000000

seconds

-
2147
483648

lines per second


RATS has analyzed 3108 lines of code and has marked 39
lines of code that need to be manually audited. This will
reduce the number of lines needing to be audited during a
manual code review by nearly 98.7%.


1)

IPAL Manual Audit
Test Point
1

Continuing to perform that manual audit we

notice
that
there is
one high
v
ulnerability issue and multiple medium
ones.

The high vulnerability is flagged in the
deposit_controller.rb on line 38. The code has been marked as
a high vulnerabilit
y because it is a system exec command.
The code is as follows:


29
def create

30

31
@currencies = currencies_for_select

32

33
if params[:check_id].nil? or



params[:check_id].length != 172

34

flash[:error] = 'Invalid check ID'

35

render

:action => 'new' and return

36
end

37

38
ccheck = IO.popen("/usr/bin/ccheck






#{params[:check_id]}", "r")

39

r
esult = ccheck.gets

40

ccheck.close


In this case some
limited
error checking
is done

on
params[:check_id
], however, it is still passing
users provided
untrusted data to popen.
If the user were to pass any string of
length 172, it would be passed to the popen command.
If
the

user were to

create the input string 1
72 characters long and

format
the
string as s
uch:


1111111111111111111111111111111111111111111111
1111111111111111111111111111111111111111111111
1111111111111111111111111111111111111111111111
111111111111111111111 | ls < ~/out


t
hey could create the file

out


in the applications home
directory. This c
an be further extended to any malicious
command the user want
ed

to execute.

This is a positive catch
for the RATS application.


2)

IPAL Manual Audit
Test Point
2

Continuing to the

next

medium

level param warnings, the
first file, admin_controller.rb

shows a potential problem at line
20 which reads:


19 def show

20 @deposit =



Deposit.find(params[:id])

21 end


Here the params are being passed to the find method without
any parameter checking. In this case it’s okay because the
find method d
oes not
create database rows. The user may be
able to inject multiple ids to find, but in this case, it does not
pose a security risk.
As it turns out our application can and
does generate false positives and this is one of them. Luckily,
this is an easy

one to spot and no further code review is
necessary.



3)

IPAL Manual Audit
Test Point
3

Part of this
factious

bank is the ability
to let

anyone sign up
for banking service. There is both a users section and an
admin section. There is an online form that allows us to do
this and that is controlled by session_controller.rb. We see
that the session_controller has a warning on line 31:


31

def
signup

28

@user = User.new(params[:user])

29 …


This is a huge security issues, known as the mass assignment
problem. Ruby will take all the inputs received from the user
and assign them to the user object with no regard. Here is an
example post:


user[login]=name&user[email]=a@b.c&user[p
assword]=password ...


Now someone with
fewer scruples

might modify that post to
something like this:


user[login]=name&user[email]=a@b.c&user[p
assword]=password&user[admin]=true ...


Doing so, Ruby would automatica
lly assign admin=true to the
object and in this case, they would successfully gain admin
access. There are many way
s to protect against this, such as
being
explicitly passi
ng the individual attributes
, or using
attr_accessible

to set which parameters in an object can be
mass assigned
.



9


4)

IPAL Manual Audit Conclusions

We find that after analysis with the RATS tools that we are
able to find security vulnerabilities in the code. While RATS
does generate many false
positives
, having

a manual audit
reduction of 98.7% has greatly reduced the time needed to
perform the audit.

B.

rubysocialforum test

Rubysocialforum is an open source project on the Ruby
Forge website. It currently
has

an evaluation release that can
be downloaded
. It can b
e thought of as a Web 2.0 style BBS
(Bulletin Board System). It is complete with user logins,
account creating and user posting abilities. Additionally, users
can use themes and also delete posts. There is currently only
one developer on the project and

it has been chosen to be
analyzed because of its relatively small developer and code
base.


After running RATS on the source code we find that it
scanned over 7200 lines of code and over 150 files. The
output of RATS is listed below:


RATS results.

Severity: Medium

Issue: params

Use of params, verify all user values are checked before using.
Never pass params directly to a new object i.e.
Object.new(params[:user])

File:
rubysocialforum/app/controllers/

application.rb

Lines: 56 26 25 23 22 20 19 19 1
5 14 10 9

File:
rubysocialforum/app/controllers/

base_controller.rb

Lines: 45 37 37 36 36

File:
rubysocialforum/app/controllers/

blog_controller.rb

Lines: 29 27 22 21 19 9 8

File:
rubysocialforum/app/controllers/

category_controller.rb

Lines: 31 25 24 2
4 23 23 17 15

File:
rubysocialforum/app/controllers/

message_controller.rb

Lines: 131 130 124 115 107 105 92 90 88 87 68 49
35 34 28 27

File:
rubysocialforum/app/controllers/

moderations_controller.rb

Lines: 48 39 38 34 24 16

File:
rubysocialforum/app/c
ontrollers/

populate_controller.rb

Lines: 223 215 214 213 212 211 210 150 150 102 75
75 73 61

File:
rubysocialforum/app/controllers/

posts_controller.rb

Lines: 357 337 463 461 452 448 439 438 427 328
319 317 300 295 294 272 192 179 137 136 134 132
132

130 126 122 118 116 110 106 105 100 93 85 84
84 83 83 76 52 51 46 45 30 29 25

File:
rubysocialforum/app/controllers/

static_controller.rb

Lines: 8

File:
rubysocialforum/app/controllers/

theme_controller.rb

Lines: 355 261 255 247 246 245

File:
rubysocia
lforum/app/controllers/

topic_controller.rb

Lines: 43 30 29 20 18

File:
rubysocialforum/app/controllers/

users_controller.rb

Lines: 328 314 313 304 285 276 256 239 238 237
200 196 195 194 193 192 187 187 97 96 95 94 90 90
41 33 27 15

Severity: Medium

Issue: rand

Make sure this function is not being used for any security
related tasks.

File:
rubysocialforum/app/controllers/

cache_test.rb

Lines: 2

File:
rubysocialforum/app/controllers/

populate_controller.rb

Lines: 56 57 58 60 60 99 99 132 144 211 212
213

File:
rubysocialforum/app/controllers/

posts_controller.rb

Lines: 213 399 401

File:
rubysocialforum/app/controllers/

theme_controller.rb

Lines: 355

File:
rubysocialforum/app/controllers/

users_controller.rb

Lines: 42 147

File:
rubysocialforum/app/m
odels/category.rb

Lines: 30

File:
rubysocialforum/app/models/post.rb

Lines: 161 202 354

File:
rubysocialforum/app/views/posts/

tagScreen.rhtml

Lines: 14


Total lines analyzed:
7290

Total time
0.000000

seconds


RATS has

analyzed 7290 lines of code and has marked 182
lines of code that need to be manually
audited
. This will
reduce

the number of lines needing to be audited during

a
manual code
review

by nearly 97.5%.


1)

rubysocialforum Manual Audit

Test Point

1

RATS has not
ified us that the user_controller.rb file has
medium security vulnerabilities with the params variable. It
warns us that on line 256 there is a specific instance of this
that needs to be audited; line 256 is below:


250
def create

251


if !@user.isAdmin

2
52



render :action=> "/posts/error",

253

:message=>"permission
denied


for creating

10


user"

254
return

255
end

256

user = User.new(params[:user])

257

user.isAdmin = false;

258 u
ser.permissionGroupsId =


PermissionGroups::RegularUser


After manual audit analysis we find this line is susceptible
to the mass assignment vulnerability. However, lines 257 and
258 mediate this problem by overwriting the important
security related parameters set via the mass ass
ignment. We
will mark this as a false positive for the tool, but a great catch
nonetheless because this example could turn into a security
problem down the road. If the user object were ever extended
to support more security features and those features w
ere not
explicitly set to false like 257 and 258 it could open up a
security hole.


2)

rubysocialforum Manual Audit Test Point 2

We continue to audit the code and look at some of the rand
vulnerabilities listed in populate_control.rb:


52
#insertion routines

53

def dummyPost post = nil

54


serial = Post.maximum(:id) + 1

55

post = Post.new() if !post

56

t = ["Create ","Destroy ","Dance ","fight





,"Humph "][rand(5)]

57

t += ["Green ","Huge ","Smiling ","stinking



,"Singing "][rand(5)]

58

t += ["Capitalist ","Worker ","phones ","Gods




,"Children "][rand(5)]

59

post.title = t + serial.to_s

60
post.body = (["great ","good ","fair ","poor



,"lousy "][rand(5)] + serial.to_s +


" ") * rand(200)


In this
example we see rand being used on lines 56, 57, 58 and
60 as the tool has warned us. In this case, rand is not being
used for security purposes but rather building text for the
website. We can therefore conclude that the following lines
are safe from any

rand vulnerabilities.

C.

rubygems
-
0.2.0 test

RubyGems is a project that bring
s

package management and
distribution to Ruby. It is similar to other package
management systems like aptitude and dpkg.

It allows
developers to package up all their code and prov
ides users an
easy way to download and install it via using

gem install
abc
.

We decided to use an older version to keep the code
base both smaller and simpler. Of note, RubyGems is not
meant to have a huge emphasis on security
[17]
. The fact that
the ap
plications purpose is to download, build and install
applications means there is not a need to secure RubyGems
because the application being download could be just as
malicious.


We have ran RATS against it and found the following:



Analyzing
rubygems
-
0.2
.0/bin/generate_yaml_index.rb

Analyzing
rubygems
-
0.2.0/doc/makedoc.rb

Analyzing
rubygems
-
0.2.0/example/lib/test/wow.rb

Analyzing
rubygems
-
0.2.0/example/lib/test.rb

Analyzing
rubygems
-
0.2.0/install.rb

Analyzing
rubygems
-
0.2.0/lib/rubygems/builder.rb

Analyzing
rubygems
-
0.2.0/lib/rubygems/cache.rb

Analyzing
rubygems
-
0.2.0/lib/rubygems/doc_manager.rb

Analyzing
rubygems
-
0.2.0/lib/rubygems/format.rb

Analyzing
rubygems
-
0.2.0/lib/rubygems/installer.rb

Analyzing
rubygems
-
0.2.0/lib/rubygems/remote_installer.rb

Analyzing
rubygems
-
0.2.0/lib/rubygems/specification.rb

Analyzing
rubygems
-
0.2.0/lib/rubygems/validator.rb

Analyzing
rubygems
-
0.2.0/lib/rubygems/version.rb

Analyzing
rubygems
-
0.2.0/lib/rubygems.rb

Analyzing
rubygems
-
0.2.0/packages/sources/lib/sources.rb

An
alyzing
rubygems
-
0.2.0/test/test_remote_installer.rb

Analyzing
rubygems
-
0.2.0/test/test_version_comparison.rb

RATS results.


Severity: Medium

Issue: open

This method allows I/O access outside of the application. All
I/O should be validated.

File:
rubygem
s
-
0.2.0/bin/generate_yaml_index.rb

Lines: 22

File:
rubygems
-
0.2.0/install.rb

Lines: 61

File:
rubygems
-
0.2.0/lib/rubygems/builder.rb

Lines: 96

File:
rubygems
-
0.2.0/lib/rubygems/format.rb

Lines: 28

File:
rubygems
-
0.2.0/lib/rubygems/installer.rb

Lines: 74

96

File:
rubygems
-
0.2.0/lib/rubygems/remote_installer.rb

Lines: 142

File:
rubygems
-
0.2.0/lib/rubygems/validator.rb

Lines: 86

Severity: Medium

Issue: backtick

The backtick will act just like an call to exec(
), so care should
be exercised that the string being backtick evaluated does not
come from an untrusted source.

File:
rubygems
-
0.2.0/doc/makedoc.rb

Lines: 3

Severity: Medium

Issue: mkdir_p

File:
rubygems
-
0.2.0/lib/rubygems/doc_manager.rb

Lines: 17 22

Fi
le:
rubygems
-
0.2.0/lib/rubygems/installer.rb

Lines: 43 48 51 54 95

File:
rubygems
-
0.2.0/lib/rubygems.rb

Lines: 82 86

Inputs detected at the following points


Total lines analyzed:
1667

Total time
0.000000

seconds



11


RATS has

analyzed 1667 lines of code and has marked 18
lines of code that need to be manually audited. This will
reduce the number of lines needing to be audited during a
manual code review by nearly 98.9%.


1)

rubygems
-
0.2.0 Manual Audit Test Point 1

Upon further i
nspection of the medium vulnerability of the
backtick issue,
RATS
points us to analyze line three in the
makedoc.rb
file which is as follows:



1
Dir.glob("*.txt").each do |file|

2

htmlfile = File.basename(file)


[0...
-
(File.extname(file).size)]+".html"

3

`wiki2html
-
b .
-
s doc.css
-
t "RubyGems"


-
o #{htmlfile} #{file}`

4
end


We

see here that both the file and htmlfile variables are
passed to the exec call without any input checking. Normally,
this would be extremely dangerous and allow
parameter
injection. However, because of the nature of this application,
this would not be an attack target. Consequently, the
makedoc.rb has been removed from later versions of
RubyGems through deprecation.


D.

Precision and Recall


In order to determine t
he
accuracy

of this tool we must
evaluate
its

effectiveness. We have chosen to carry out the
tests on the ipal example described above. Our tests will
validate the tool against the code base and against manual
code auditors. The manual code auditors wil
l help discover
any extra errors in the ipal code base that the tool may not
discover.


Doing a detailed code analysis of ipal, we find 12 unique
vulnerabilities. These vulnerabilities are discovered by
determining the unique and sha
red vulnerabilities th
at both

the
manual code auditors
and the tool

discovered
. The following
table describes the breakout of shared and unique
vulnerabilities to Auditors A1, A2 and the Automated RATS:



Uniq to 1

Uniq to 2

Shared

Auto
U

A1

8

2

1

Auto
U

A2

1

1

8

A1
U

A2

2

8

1

Auto
U

(A1 & A2)

1

3

8


This c
an be further displayed in the V
enn diagram below. We
see that RATS and code auditor 2 share 8 vulnerabilities and
each have a single unique one. Further, we see Auditor one
shares a single vulnerability with both
Auditor 2 and RATS,
yet has two un
ique vulnerabilities that neither

discovered.



The following table describes the amount of correctly
identified tokens in the ipal application. In the ipal
application
,

there are 509 lines of code and 12,068 tokens.
In

order to calculate Precision and Recall the following metrics
are needed [24]:



S++: Number of
tokens correctly identified as a
vulnerability



S+
-
: Number of tokens
incorrectly
identi
fied as a
vulnerability



S
--
: Number of tokens correctly identified as not
a
vulnerability



S
-
+: Number of tokens
incorrectly identified as not a
vulnerability


Precision can be calculated by:

|



|
(
|



|





)


Recall can be calculated by:

|



|
(
|



|





)


Similarly, the
negative precision and
negative
recall

can be
calculated by
:

|



|
(
|



|





)

and
|



|
(
|



|





)


The following table displays the calculated precision and
recall for both sets of
metrics for the manual code auditors and
the tool. In the case of RATS, we would not expect to see a
high Recall+ rate. This is because RATS tries to error on the
side of a 100% precision rate while trading off on the recall
rate.

Again, this is because

RATS is built to try and flag any
Audit

S++

S+
-

Total
True+

S
--

S
-
+

Total
True
-

Auto

9

23

12

12030

3

12056

Auditor
1

3

19

12

12028

9

12056

Auditor
2

9

0

12

12053

3

12056


12


potential vulnerability and is only meant as a starting point for
a manual code audit. The trade off is because there is such a
high precision, it is expected to also have a large false positive
rate as it will flag thin
gs that may look dangerous, but in
actuality are not.

Audit

Precision
+

Recall+

Precision
-

Recall
-

Auto

75.00%

28.13%

99.98%

99.81%

Auditor
1

25.00%

13.64%

99.93%

99.84%

Auditor
2

75.00%

100.00%

99.98%

100.00%

Further analysis shows us Auditor 2 achieved the same
precision as the tool and a 100% recall rate. This may lead
one to believe that a manual code audit would be more
efficient then the tool, because it delivers a better recall rate.
Of note though, the

manual could audit for both auditors took
60 minutes to complete, while the code analysis for the tool
took fractions of a second.

Furthermore, we believe the tool can
be modified to
increase the prec
ision to 100%. There were two main issues
that cause

our tool to not be 100%. The first was we did not
parse the contents of strings to determine if a variable was
being called, this was an oversight and can be added to the
lexical file to scan for these extra tokens. The second was our
initial vulnerabil
ity database was not (and is not) 100%
complete. We can add some of the vulnerable functions
suggested by the manual code auditors and achieve a higher
prevision rate with no code change.

Furthermore, we can consider Recall+ as our True Positive
Rate and
1


(
Precision
-
)

as our False Positive Rate.

Audit

TPR

FPR

Auto

28.13%

0.0249%

Auditor 1

13.64%

0.0748%

Auditor 2

100.00%

0.0249%

As noted above, we should expect our FPR to
go to 0%,

which we are n
ot too far off from. While 0.0249
% is a good
FPR, when scanning code bases with large number of lines,
this allows for a lot of potential vulnerabilities. Give a code
base of 100,000 lines of code,
0.0249
% could allow 20 false
negatives to occur. Our trade of is that as we near 0% for t
he
FPR, our TPR rate is also expected to drop increasing the size
of a suggested manual code audit.

Lastly, given the time savings and the low false positive rate
we find the tool to be an effective tool to assist with manual
code auditing of the Ruby prog
ramming language.

V.

C
ONCLUSION

We have enhanced

RATS
to audit

Ruby

applications

and
demonstrate

success
fully on a vulnerable web application in
an iCTF competition and several open source software
packages
. RATS is

able to now identify areas in Ruby code
that either have a vulnerability, or have the potential for
vulnerabilities. We have been able to successfully reduce the
lines of code needed to perform an initial security audit
substantially, allowing the audits

to be more detailed and
meaningful.


Some of the difficulties in extending RATS have primarily
been the creation of the vulnerability database. It has been a
difficult tradeoff of which to include as an extras would create
more false positives. There is

no one place that lists all the
vulnerabilities and potential vulnerabilities that exist in Ruby,
so
extensive
searching and correlating between other
languages is required to build a vulnerability database for a
new language.

VI.

F
UTURE
W
ORK

We have created
a starting point of which Ruby source code
analysis can begin. Some of the current limitations with
RATS is how tokenizes and scans the tokens against the
vulnerability database. Future work could include modifying
t
his so rather tha
n scanning a single t
oken for a vulnerability, a
group of tokens could be scanned as a vulnerability. This
would help eliminate some of the false positives seen in the
scan. Additionally, because Ruby is such a dynamic language,
and the fact th
at

it is ultimately the interpr
eter that decides
how to run the Ruby script, it may be best to perform any
deeper code analysis using the Ruby language and creating a
more dynamic source code analysis tool.

A
PPENDIX

A



I
NSTALLING
R
ATS

On Windows, uncompress the rats rar file and run ra
ts.exe
from the command line.

On Ubuntu issue the following commands:

aptitude install build
-
essential

aptitude install libexpat1 libexpat1
-
dev

wget http://www.uccs.edu/~mgersch2/rats
-
2.2.rar

unrar e rats
-
2.2.rar

chmod +x mkinstalldirs

make

sudo make insta
l
l

A
PPENDIX
B



U
SING
R
UBY ON
RATS

The same syntax to running RATS has been maintained.
Consult the man page or run RATS with

help for more
information. Ruby on RATS automatically identifies and
Ruby file passed to it by the file extension, so no
special
commands are needed for the Ruby extension.

To run RATS on a ruby file it can be called as so:

rats myrubyfile.rb

To have RATS scan a directory, it can be called as so:

rats ./directory_to_scan/

RATS will then output to the console screen all the
vulnerabilities it has found and will mark them medium low
and high. This output will contain the source code file with
the line number it found the vulnerability at. At this point a
manual source code
analysis can begin and be greatly
minimized by the aid of this tools result
s
.


This example shows running it on the webdemon.rb file:



13


C:
\
Users
\
Mike
\
Desktop
\
rats
-
2.2>rats
webdemon.rb

Entries in perl database: 33

Entries in ruby database: 46

Entries in pyth
on database: 62

Entries in c database: 334

Entries in php database: 55

Analyzing webdemon.rb

webdemon.rb:221: High: system

Make sure user data is not pass to
system.


webdemon.rb:56: Medium: trap

Ruby safe level 2 disables this function
as it could be pote
ntially dangerous.
Verify this function is being used in a
safe manner.


Total lines analyzed: 320

Total time 0.000000 seconds

-
2147483648 lines per second


C:
\
Users
\
Mike
\
Desktop
\
rats
-
2.2>


The results here show that a system call and a signal trap are
be
ing used. These two lines then need to be manually audited
to determine if any vulnerability may exist.

A
CKNOWLEDGMENT

We acknowledge UCCS for providing us a learning
environment

which students
can
grow and thrive and
ultimately made this and many more things possible. A
special thanks
as well
to Dr. Vigna and
UCSB for
hosting all
the
iCTF
’s and providing lot of examples of source code full
of security vulnerabilities
.

R
EFERENCES

[1]

Ruby Programming La
nguage,

www.ruby
-
lang.com


[2]

TIOBE Software. (2008, June). TIOBE Index. Retrieved
June 22, 2008, from TIOBE:
http://www.tiobe.com/index.php/content/paperinfo/tpci/in
dex.html


[3]

Gartner via Durkee, R. (2007, Septermber). 2007
OWASP Top 10 Most Critical Web Application Security
Vulnerabilities. Retrieved June 22, 2008, from OWASP:
http://www.owasp.org/images/f/fb/OWASP_Top_10_200
7_v6.ppt

[4]

Sanctum via Durkee, R. (2007, September). 2007
OWASP Top 10 Most Critical Web Application Security
Vulnerabilities. Retrieved June 22, 2008, from OWASP:
http://www.owasp.org/images/f/fb/OWASP_Top_10_200
7_v6.ppt

[5]

OWASP Ruby on Rails Security Guide
https://www.owasp.org/ima
ges/8/89/Rails_Security_2.pdf


[6]

Programming Ruby: The Pragmatic Programmer’s Guide

http://www.ruby
-
doc.org/docs/ProgrammingRuby/

[7]

Lorin J. May,
Major Causes of Software Failure
,
Crosstalk, Hill AF
B, UT

[8]

John Viega, Gray McGraw, (2002).
Building Secure
Software

How to avoid Problems the Right Way. Boston:
Addison
-
Wesley. p
127

[9]

McGraw, G. (2006). Software Security Building Security
In. Boston: Addison
-
Wesley. p56

[10]

Stuart Halloway, (2008). Never untain
t
http://blog.thinkrelevance.com/2008/1/17/never
-
untaint


[11]

Matasano Chargen


Blog Archive: Thoughts on Ten
Years of qmail security, Thomas Ptacek

http://www.matasano.com/log/989/thoughts
-
on
-
ten
-
years
-
of
-
qmail
-
security/

[12]

Leveraging Race Conditions via Symbolic Links
http://capec.mitre.org/data/definitions/27.html


[13]

Wikipedia: Symlink Race

http://en.wikipedia.org/wiki/Symlink_race

[14]

Wikipedia: Ruby on Rails

http://en.wikipedia.org/wiki/Ruby_on_Rails


[15]

Andrew Hurst (2004) Analysis of Perl’s Taint Mode

http://hurstdog.org/papers/hurst04taint.pdf

[16]

Documenting the Ruby Language

http://www.ruby
-
doc.org/


[17]

RubyGems Manual

http://rubygems.org/read/chapter/14

[18]

Cypher Punks: Random Number Generation

http://www.cypherpunks.to/~peter/06_random.pdf

p21

[19]

Wikipedia: Duck typing

http://en.wikipedia.org/wiki/Duck_typing


[20]

ITS4: Software Securit
y Tool,
http://www.cigital.com/its4/

[21]

FlawFinder,
http://www.dwheeler.com/flawfinder/

[22]

Dehlinger, J., Feng, Q., and Hu, L. 2006. SSVChecker
:
unifying static security vulnerability detection tools in an
Eclipse plug
-
in. In
Proceedings of the 2006 OOPSLA
Workshop on Eclipse Technology Exchange

(Portland,
Oregon, October 22
-

23, 2006). eclipse '06. ACM, New
York, NY, 30
-
34

[23]

Viega, J.; Bloch, J.T
.; Kohno, Y.; McGraw, G., "ITS4: a
static vulnerability scanner for C and C++ code,"
Computer Security Applications, 2000. ACSAC '00. 16th
Annual Conference

, vol., no., pp.257
-
267, Dec 2000

[24]

J. Kalita,
Measuring Performance of an Unsupervised
Learning Algo
rithm
. Draft Unpublished, 2007.




Michael S. Gerschefske

received a Bachelors of Science in
Computer Engineering at the University of Colorado,
Colorado Springs 2005. Additionally this paper has been
written in pursuit of a Masters in Engineering in Information
Assurance at the University of Colorado, Colorado

Springs.







14





Appendix
C


ipal Code Audits


Results from automated source code analysis:


File Name

Line

Vulnerability

Pos

False
Pos

False
Neg


admin_controller.rb

20

params


x




28

params

x



Mass Assignment


38

params


x




42

params


x




43

params

x



Mass Assignment


52

params


x



currency_controller.rb

16

params


x




24

params

x



Mass Assignment


34

params


x




38

params


x




39

params

x



Mass Assignment


48

params


x



deposit_controller.rb

21

params


x




33

params


x


Possible Design Flaw


38

popen

x



Improperly checked input
passed to popen


54

params


x




69

params


x




73

params


x




74

params

x



Mass Assignment


83

params


x




87

params


x




91

params


x


Possible Design Flaw


98

rand

x



Used to generate
transaction file names


102

params



x


session_controller.rb

18

params


x




20

params


x




31

params

x



Mass
assignment

user_controller.rb

10

params


x




13

params


x




20

params


x




55

params

x



Mass assignment


64

params


x




72

params


x










Totals



9

23

1




Results from Manual code auditor 1:


File Name

Line

Vulnerability

Pos

False
Pos


user.rb

37

Digest::SHA1.hexdigest

x


Weak hash function


55

encrypt

x


Predictable session tokens


15


admin_controller.rb

20

params


x



38

params


x



42

params


x



52

params


x


currency_controller.rb

16

params


x



34

params


x



38

params


x



48

params


x


deposit_controller.rb

21

params


x



54

params


x



69

params


x



73

params


x



83

params


x



87

params


x



112

Sql Injection


x


session_controller.rb

32

params

x


Mass Assignment

user_controller.rb

13

params


x



20

params


x



64

params


x



72

params


x








Totals



3

19



Results from Manual code auditor 2:


File Name

Line

Vulnerability

Pos

False Pos


admin_controller.rb

28

params

x


Mass Assignment


43

params

x


Mass Assignment

currency_controller.rb

24

params

x


Mass Assignment


39

params

x


Mass Assignment

deposit_controller.rb

38

popen

x


Unchecked process call


74

params

x


Mass Assignment


102

params

x


Unchecked write to file

session_controller.rb

31

params

x


Mass Assignment

user_controller.rb

55

params

x


Mass Assignment







Totals



9

0