The OWASP Foundation

fanaticalpumaMechanics

Nov 5, 2013 (3 years and 9 months ago)

78 views

Copyright © The OWASP Foundation

Permission is granted to copy, distribute and/or modify this document
under the terms of the OWASP License.

The OWASP Foundation

OWASP

AppSec

DC

2009

http://www.owasp.org


Business Logic Bots: Friend or
Foe?

Amichai Shulman, CTO, Imperva

Ofer Shezaf, Leader, OWASP Israel

OWASP

Amichai Shulman


Background


CTO and Co
-
founder at Imperva


Application security pen
-
testing and consulting for
commercial and government organizations


Security research for IDF


Leads the Application Defense Center (ADC)


Discovered over 20 commercial application
vulnerabilities



Credited by Oracle, MS
-
SQL, IBM and others


Speaker at Industry Events


RSA, OWASP, Black Hat, Info Security UK, and
many others


Lecturer on Info Security


Technion

-

Israel Institute of Technology


2

OWASP

Ofer Shezaf


Background:


Design of Web Application Firewalls, at
Breach Security.


Security research for the Israeli
Government.


Open Source and Community projects:


Officer, The Web Application Security
Consortium.


Leader, OWASP Israeli chapter


Project Lead WAFEC, The Web Application
Firewall Evaluation Criteria.


Project Lead, WHID, The Web Hacking
Incident Database.


Based out of Tel
-
Aviv, Israel.


3

OWASP

Agenda


The challenge of business logic bots


Business logic attacks


Business process automation:


The friendly side of web automation


Business logic bots:


Malicious web automaton


Solutions


Detection


Mitigation

OWASP

Business Logic Bots (BLBs)

A subset of Business Logic Threats (BLTs).

Compared to syntactic attacks:

5

Technical Attacks

Business Logic Attacks

Malformed requests

Normal requests

Invalid input values

Legitimate input values

Change

functionality

Abuse

functionality

Attack the
application and
only indirectly the business

Attack directly the business

Usually a single request

Often multiple requests

OWASP

Business Logic Bots

What a pain in the neck…

6

Business Logic Threats

Normal requests

Legitimate input values

Abuse

functionality

Attack directly the business

Often multiple requests

As

a result

Hard to detect

Not

always illegal

Key for mitigation!

OWASP

Web Automation

7

OWASP

Web Automation


The fact is that web automation is in wide use


Online form automation


Tracking competition


Personal and institutional stock trading


Indexing services


Comparative shopping


Web Services and other web APIs


Bottom line is that business level automation may or
may not be define as an attack based on the context
of things


Who is the source


Which part of the business logic is being invoked



8

OWASP

BUSINESS LOGIC BOTS

Born to be bad:

OWASP

Brute Force


Determine an unknown value by using an
automated process to try a large number
of possible values.


Can be used for:


Cracking login credentials


Guessing session identifiers


Guessing file and directory names (often
called


“Forceful Browsing”)


Credit card information such as CVV and

expiration date.

10

OWASP

Logic Bots Denial of Service


Locking resources:


Lock resources by doing nothing


Invoke and use slowly transaction based processes.


Abusing resource sensitive function:


Search


SQL Like injection


Regular expression injection.


Performed in a distributed fashion.

OWASP

Web Spam


Abuse public web pages to post
links in order to elevate site’s
ranking in search results (Abusive
SEO)


Advertise through Comment Spam


Abuse forms that send email for
spam

OWASP

Click Fraud


Abuse pay per click advertising by generating automated
clicks.


Performed by owners of web sites displaying ads,
competitors and vandals.


Variants include:


Referrer click fraud.


CSRF click fraud.

13

OWASP

Hardcore Robotics


Queue Jumping


Ticketmaster confessed to “fighting like
the dickens“ queue jumping.


Travel agents known to automate air line
ticketing systems.


Auctions Sniping


Watching a timed online auction and
placing a winning bid at the last possible
moment giving the other bidders no time
to outbid the sniper.


Poll Skewing



OWASP

Gaming Bots


MUD, Virtual Worlds & Second Life
bots:


Gain Wealth, and turn it into money in
Second Life.


Scripted Clients


GUI Bots


Poker Bots:


Share information between several
bots at one table.


Monitor tables to choose the weak
ones.


Play well.


OWASP

Information Harvesting


Harvests:


E
-
mail and personal information


Competitive information


Record oriented information such as CVs


Entire Web sites for creating a mirror


Executed from:


Local computer


Distributed, potentially using
bot

net


Trojans, exploiting the victims credentials at the site


16

OWASP

ENOUGH WITH THE FUD!

OWASP

Solutions


The solution is comprised of two separate
problems


Detection


Mitigation


Detection


Detect automation (absolute)


Flag unauthorized use of automation (subjective)


Mitigation


Effective


Does not break application

18

OWASP

Detection


Basic Tools


Black listing:


IP Addresses (IP Reputation)


Anonymous Proxies,
TOR exit nodes, highly active bots


User Agents


Ad
-
hoc attack vector patterns


Ad
-
hoc comment spam patterns


Request structure


Missing / mismatch Host header


Irregular header combinations


Naïve, but eliminates the masses

OWASP

Detection


Proactive Techniques


Introduce extra content into the response


The extra content is interpreted in a different manner
by a human driven browser and by an automated tool


Must not affect visuals


Must not break application


Positive detection


Extra content affects a robot but not human


Examples:


An invisible link outside of the view


An invisible link generated by a script


Can be used to detect scraping

20

OWASP

Detection


Proactive Techniques


Negative Detection


Extra content affects a browser but not a robot


Examples


Zero size
imgae


Reference created through script


onError

script of a missing image


Can be used to detect repetition attacks driven by
scripts like busy waiting, comment spam, etc.


21

OWASP

Detection


Frequency Measurement


Count the frequency of
“events”

within some
scope

in a given time frame


Challenges


What’s an event?


What’s the best scope?


What’s the right threshold?


Allow detection of script related attacks and
brute force attacks

22

OWASP

Detection


Frequency Measurement


Flexible event definition


A combination of URL, parameter names and their values


Variance (all matches, different URLs only, same
parameter value only, etc.)


Session state (started, authenticated)


Examples


Access to
/app/prod/



Access to
/app/doAction.aspx
with parameter
cmd
=commit


New session


Access to
/app/prodDetails.aspx

with different values of the
prodID

parameter


Access to /app/doPurchase.aspx with the same value in the
ccnumber

parameter


23

OWASP

Detection


Frequency Measurement


Flexible Scope Definition


Some events must be performed in the context of a
session so it makes sense to measure the frequency
within a session


Some events need to be counted in the context of an
IP address (e.g. new sessions…)


Some attacks must occur in the context of a specific
application user


Detecting distributed attacks may require a wider
“system”
conext


24

OWASP

Detection


Frequency Measurement


Threshold Definition


Some attacks are brutal enough and can be defined
in terms of
absolute

numbers


Login brute force


Exhaustive traversing of resources with “semi” predictable
identifiers


Repetition attacks within a session context


Other attacks must be defined in terms of deviation
from either a short term (last minutes) or long term
(few days) observed frequency


Distributed attacks


25

OWASP

Detection


Flow


Some attacks, either inherently or for
performance reasons bypass normal application
flow


Traversing a product catalog


Skipping transaction validation


Not easy to implement


Referer

header can be forged


Flows are hard to define and track in modern
applications that use frames and AJAX


Can detect some types of forceful browsing and
man in the browser attacks

26

OWASP

Detection


Flow


Possible implementations


Track
Referer

(weak)


Introduce a
nonce

(may end up being very intrusive)


How to define / detect flows


Guided configuration


Learning algorithms

27

OWASP

Detection


Click Rate


Humans take time to respond (even the fast
ones)


Can be thought of as an extension to flow


Some observations:


Clickable events, within a session, need to be at some
minimal distance from one another


Within a session, over time, clickable events should
be relatively slow


Can detect general script attacks as well as man
in the browser attacks

28

OWASP

Detection
-

Summary


Will a single method do the trick?


I don’t think so.


Will there be false positives?


Yes!


Do I care?


No! Let me tell you why…

29

OWASP

Mitigation


Attacks are automated


I can’t prevent the attack from going on


I can however try to “defuse” its effects


Examples:


Slow down a brute force attack


Reduce the rate of a
DDoS

attack


Make the victim aware of a man
-
in
-
the
-
browser attack


Enforce flow on transactions


Disinformation


Preventative measures may increase the cost of
automation to the level that makes it much less
attractive for anything but high end targets


30

OWASP

Mitigation
-

Blocking


Dropping requests can only occur in very specific
cases


IP blacklists


User
-
agent blacklists


Strongly enforced flow (e.g. through nonce in a form)


Dropping requests that fail to answer the
challenges described in the following slides

31

OWASP

Mitigation


That Which Makes Us Human


Provide a Turing test that only a human can
solve.


Usually called CAPTCHA. Traditionally character
recognition


Other methods exists


Choose the correct description of an image


Solve a simple riddle


32

John had one thousand apples and five oranges. He ate
as many of his apples as there is letters in word
"apple". Also he ate two bananas :
-
). How many apples
John have?

OWASP

Mitigation


That Which Makes Us Human


There are automated tools and algorithms today
that solve CAPCHA’s of various types


I don’t care


If a brute force login program solves one CAPTCH per
second then it is ineffective


If a client solves a CAPTCHA faster than a human
being (no less than one second) then it can easily be
identified as a robot and further challenged (see next
slide)


33

OWASP

Mitigation


Throttling


Slowing down an attack is most often the best
way to make it ineffective.


A second of delay can make the difference for an
automated attack but will not be noticed by most
humans


Server side throttling may have sever impact on
server (quickly consume connection resources)


Client side computational challenges


Client is required to solve a computational challenge
that can be easily verified by server


Code for solving the challenge is introduced into the
response in the form of a script


34

OWASP

Mitigation


Throttling


Examples


Weak encryption of a random string
together with a public message


Reverse hash, partial hash collisions


Throttling can be per IP for those
requests that don’t have a session
yet and then applied by session to
those sessions that display
suspicious behavior.

35

OWASP

Mitigation


Adaptive Authentication


When automation is detected in the context of a
user (man in the browser)


Ask for additional authentication


Repeat password


Previously recorded questions


Makes the attack apparent to a victim

36

OWASP

Mitigation
-

Disinformation


Feed the client with bogus
information


A client follows a hidden link


Respond to the request with a page that
includes a large number of server
distinguishable random links


Whenever one of the random link is
request generate yet another random
page


A client that follows a hidden link
that was generated by a script


Respond with a page that include a
script that runs for a long time before
generating a new random link

37

OWASP

Mitigation
-

Summary


Mitigation methods should take into
consideration the possibility of false positives


Most often the system’s reaction to a suspected
automation attempt should not be blocking but
rather challenging the client


Legitimate clients are not materially affected


Automated clients become ineffective

38

OWASP

Summary


Automated business layer attacks are proliferating
today and expected to grow in number and
sophistication in the near term


Detecting and mitigating these attacks require a set
of sophisticated tools that are different than the
standard web application security tools


Some of the issues have nothing to do with the way
the application code is written


It’s bound to be a cat and mouse game as robots
become more sophisticated


As a consequence of the above, solutions should be
external to the application code


39

OWASP

Amichai Shulman, shulman@imperva.com

Ofer
Shezaf,
shezaf@xiom.com


OWASP

Web Spam example

A
bot

generated
membership
request on
XIOM.COM

Googling

reveals many forums where
same
bot

left a message (as no
registration was required)

Links points to a semi
legitimate web site

OWASP

Poll Skewing

Google
Trends

Time Most Influential People
Poll


Attack?

Damage?