Detecting and Defending against Web-Server Fingerprinting

guideflannelΔιακομιστές

4 Δεκ 2013 (πριν από 3 χρόνια και 4 μήνες)

67 εμφανίσεις

Detecting and Defending against
Web
-
Server Fingerprinting


Annual Computer Security Applications Conference 2002


http://www.acsac.org/2002/abstracts/96.html


Presented by: Lee Hui Huang

Outline


Introduction


How to perform HTTP server
fingerprinting


Tools for fingerprinting


How to detect fingerprinting
attempts


How to protect against HTTP server
fingerprinting

Definition of Fingerprinting



Fingerprinting is a heuristic method
of observing the behavior of a
software component for the purpose
of
determining its identity
.




In general, it involves sending
specific requests to the component
and observing the response.

For HTTP servers:



Send specifically chosen HTTP
Request messages and observe the
Response messages.

Information gathering Goals


Some of the information an attacker
hopes to obtain:



open ports


services running on these open ports


Operating System

Why fingerprinting is necessary


Without the relevant information, an
attacker:


Won’t know what exploits to use


Have to try all possible attacks.


Trying all possible attacks :


Time consuming


will generate a lot of suspicious
traffic

Why fingerprinting is necessary


With the necessary information, an attack
can be more efficient.

For example:


I know a server is running
IIS 5.0 on Windows
2000
. So try
IPP buffer overflow

(CA
-
2001
-
10
) attack to gain a command prompt on the
server.



If an
Apache web server 1.2.2

is running, try
to exploit
Apache Web Server Chunk
Handling Vulnerability

(CA
-
2002
-
17)
to
execute arbitrary code on it.

Why fingerprinting is necessary


Malicious programs like worms also
need to do some probing (like port
scans) in order to select potential
victims to attack.

Why web servers are attractive
targets


Very common. Easy to find and
connect to one.



Much vulnerability exists in existing
web services.



Not all web servers have all the
necessary patches

How Fingerprinting is possible


To generate a web server’s
fingerprint a set of characteristics
that differentiates a specific server’s
use of HTTP from that of other
servers must be identified.

How Fingerprinting is possible


Specifications defined in the HTTP RFCs
are agreed upon, not enforced.


Words like MUST, SHOULD and MAY used
to described features. E.g.



All responses to the HEAD request method
MUST

NOT include a
message
-
body, …”



“The field value
MAY

be preceded by any amount of LWS, though
a single SP is preferred.”




The difference in degree of compliance
allows HTTP servers to be fingerprinted

Fingerprinting Methodology


A list of characteristics to be use for
fingerprinting can be assembled.



For each of these characteristics,
design a HTTP Request that will
provoke a Response exhibiting the
characteristic.

Fingerprinting Methodology


Characteristics can be divided into 3
categories:


Lexical
: specific words, phrases and
punctuation used.


Syntactic
: the ordering and context
of elements in the response


Semantic
: a server’s specific
interpretation of a Request from
among the possible interpretations.

Fingerprinting Methodology


Lexical category: variations in the
actual words used, capitalizations
and punctuation





Fingerprinting Methodology

Methods:

1.
Check for difference in response code
message



Each HTTP response message contains a
number indicating whether the attempt to
satisfy the request has succeeded or not as well
as the corresponding text message



E.g. for the error code 404,



Apache uses “Not Found”




Microsoft IIS/5.0 uses “Object Not Found”.

Fingerprinting Methodology

2.
Difference in the header words




variation occurs in the capitalization



For example :


some servers use “
C
ontent
-
L
ength


Others use “
C
ontent
-
l
ength”


Fingerprinting Methodology

3.
Difference in
use Line

Terminators



RFC specified behavior is to use

\
r
\
n”

to separate elements of the
header


But some older servers use only “
\
n”


More recent servers
use “
\
r
\
n”


Fingerprinting Methodology


The HTTP response message also
contains a field called Server.

Example:

HTTP/1.1 200 OK

Connection: close

Date: Thu, 06 Aug 1998 12:00:15 GMT

Server: Apache/1.3.0 (Unix)


Last
-
Modified: Mon, 22 Jun 1998 09:23:24 GMT

Content
-
Length: 6821

Content
-
Type: text/html


Fingerprinting Methodology


Syntactic category: the difference in
the ordering and format of Request
elements like the headers and the
contents

Fingerprinting Methodology

Methods:

1.
Difference in ordering of
Headers

For example:


For Apache servers, the “Date”
header is before “Server” header.


For Netscape
-
FastTrack/4.1, it is
the opposite.

Fingerprinting Methodology

2.
Difference in
ordering of list

items


Sometimes a list of items is returned
as the contents of a header



E.g. If the OPTIONS method is sent
in an HTTP Request, a list of allowed
methods for the given URI are
returned in an “Allow” header.


The order of these elements tends to
vary between servers.



But not all headers with lists can be
used. E.g. the Content
-
Language
header which identifies the language
types

Fingerprinting Methodology

3.
Difference in formatting of some

elements
.



Some elements have formats that are
variable or unspecified by the RFCs.


E.g. “ETag” header which provides a
unique identifier for a given document



Apache/1.3.11 uses “0
-
574 38379154;3a5b7811”


Jigsaw/2.1.2

uses “mvanct:s0jndthg”

Fingerprinting Methodology


Semantic category
: When a
request message is received, the
server has to decide on a
interpretation for it before it can
respond to it.




Many variation on how servers
interpret both well
-
formed and mal
-
formed Requests

Fingerprinting Methodology


Methods:

1.
Check for the Existence of Response
Line and Headers



E.g.
A specially crafted request message will
cause some servers to believe the
requester is an HTTP/0.9 based client



These server will respond with a message
without any headers.

Fingerprinting Methodology

2.
Look for specific headers



A server chooses what headers to include
in a Response.


Many of them (e.g. ETag) are optional.



E.g. When there is a “501 Method Not Implemented” error,



Apache servers send an “Allow” header with
a list of the
allowed methods for the designated URI


Jigsaw/2.1.2 does not.


Fingerprinting Methodology

3.
Compare the response code
from different servers for ad Hoc
requests



Given a malformed request, different
servers may assign it a different type
of error.




E.g.
When a random text string (e.g. “hi”)
is sent,


Apache interprets it as a bad Request
from a HTTP/0.9 client
.


Will respond with a header less message
warning that the method “hi” is not
implemented.



Microsoft IIS interprets it as a
malformed Request from an HTTP/1.X
client.




Will respond with a “400 Bad Request”.


Many ways of developing such
ad
hoc
test requests.


Can try changing the method line,
the headers and the body size


For example:


"GET”


"GET / HTTP/999.99”


"GET / HTTP/9.Q"


"HEAD /////////// HTTP/1.0"


"HEAD /.
\

HTTP/1.0"


"HEAD /asdfasdf/../ HTTP/1.0"


"HEAD /./././././././././ HTTP/1.0”

Fingerprinting Methodology

3.
Make use of the different error
ranges of the different servers



Server



URL Length

Response

-----------------------------------------------------------------------------------
Apache/1.3.12 (Win)

1
-
216


404 Not Found






217
-
8176

403 Forbidden






8177
-
up

414 Request
-
URI Too








Large

-----------------------------------------------------------------------------------


Netscape
-
FastTrack/4.1

1
-
4089


404 Not found






4090
-
8123

500 Server Error






8124
-
8176

413 Request Entity







Too Large






8177
-
up

400 Bad request

Tools for Fingerprinting


HMap

was written by the authors of
this paper
http://wwwcsif.cs.ucdavis.edu/~leed/hmap/



HMap uses the test described above.




Sample HMap output:






matches : mismatches : unknowns


Apache/1.3.22 (Win32)


116 : 0 : 7

Apache/1.3.12 (Win32)


113 : 3 : 7

Apache/1.3.14 (Win32)


113 : 3 : 7

Apache/1.3.17 (Win32)


113 : 3 : 7



Interpreting the result:


116 of the tested characteristics were a direct match for
Apache/1.3.22;


None of them were different;



7 of them couldn’t be determined.








Tools for Fingerprinting


NMap

(
http://www.insecure.org/nmap/
)



NMap is a versatile port scanner that can
perform ping sweeps, OS fingerprinting,
among others.



Command to use:


nmap

O

sV www.website.com



-
O

option performs OS fingerprinting


-
sV option performs version detection



Sample output:


Starting nmap 3.50 ( http://www.insecure.org/nmap ) at 2004
-
03
-
29 20:02
Malay Peninsula Standard Time

Interesting ports on www.website.com (192.168.123.144):

(The 1650 ports scanned but not shown below are in state: closed)


PORT STATE SERVICE VERSION

21/tcp open ftp


Microsoft ftpd

80/tcp open http Microsoft IIS webserver 5.1

135/tcp open msrpc Microsoft Windows msrpc

139/tcp open netbios
-
ssn

443/tcp open https?

445/tcp open microsoft
-
ds Microsoft Windows XP microsoft
-
ds


Device type: general purpose

Running: Microsoft Windows 95/98/ME|NT/2K/XP

OS details: Microsoft Windows Millennium Edition (Me), Windows 2000
Professional or Advanced Server, or Windows XP


Nmap run completed
--

1 IP address (1 host up) scanned in 42.972 seconds


Tools for Fingerprinting


Telnet

can be used to connect to
port 80 of a web server to get its
name.


Example:


Use:
telnet www.website.com 80 to
connect to the server.


Then issue a GET request:
GET / HTTP/1.1

Sample response message received:


HTTP/1.1 200 OK

Date: Mon, 29 Mar 2004 13:52:24 GMT

Server: Apache/1.3.29 (Unix) mod_ssl/2.8.16 OpenSSL/0.9.7d

Transfer
-
Encoding: chunked

Content
-
Type: text/html

Via: 1.1 Application and Content Networking System Software 5.0.7

Connection: Close








If source code of the web server is
available, it can be modified to return a
false value in the Server field.

Tools for Fingerprinting


Netcraft (
www.netcraft.com
) conducts
periodic surveys to determine the market
share of various web servers.



One of their feature allows a user to find
out information about a web server. E.g.


Web service running


Operating System


Uptime of the server

Stealth Techniques


Running the full suite of HTTP server
fingerprinting tests is very effective
in identifying a server.



But it is easy to be detected by an
IDS.



An attacker can try a few techniques
to minimize detection

Stealth Techniques

1. Have several computers and run a
subset of tests from each of them
and then correlate the data.



IDSes might not detect a pattern of
behavior if it analyzes records on a
host by host basis.

Stealth Techniques

2. Run the tests over a long period of
time.



If the IDS notice one unusual
Request then it might classify it as a
random error and flush the
observation after some time.

Stealth Techniques

3.
Do only the short forms of large Request tests




Server



URL Length

Response

-----------------------------------------------------------------------------------
Apache/1.3.12 (Win)

1
-
216


404 Not Found






217
-
8176

403 Forbidden






8177
-
up

414 Request
-
URI Too








Large

-----------------------------------------------------------------------------------


Netscape
-
FastTrack/4.1

1
-
4089


404 Not found






4090
-
8123

500 Server Error






8124
-
8176

413 Request Entity







Too Large






8177
-
up

400 Bad request



A URL of length 217 is enough to differentiate
the servers.

Stealth Techniques

4. Change the contents of these large
test requests so they vary in more
than just the length.



Currently the long requests are
composed of all the same characters
and do not look like legal Requests.

Stealth Techniques

5. Make a search tree of characteristics.



Do the minimal subset of tests that
can identify a server, instead of doing
every test

Stealth Techniques

6. Use URL encoding to mask the
contents of Requests.



URL encoding rewrites the request
URL in hexadecimal.


Some IDSes do not perform URL
decoding.


Pattern matching involving URL
encoding is more cpu intensive

Detecting Fingerprinting Activities


Currently it is difficult to find an IDS
that looks for web server
fingerprinting activity.


IDSes that looks for CGI attack
probes do exist and presumably
could be extended to include
detection of HTTP
-
based identity
probing.

Detecting Fingerprinting Activities


Probing does not necessary mean an
attack will follow.



Hence IDS shouldn’t issue an alert
every time probing is detected.



Ideally, there should be a more
comprehensive system to determine
what to do with this information.

Detecting Fingerprinting Activities


Some suspicious HTTP request
messages an IDS can look for:


1.
Unusual Request Element Size


Some tests use very large elements.


E.g. large URIs and large numbers of
headers

Detecting Fingerprinting Activities

2. Use of Unknown or Unusual methods



Watch out for use of Unknown methods
(e.g. “QWERTY”) or Unusual methods that
normal browsers rarely or never send (e.g.
“TRACE”).



The same applies to unknown or unusual
header fields.

Detecting Fingerprinting Activities

3. Look for unusual constructions



Most Requests have a fairly simple format



E.g. method line of METHOD URI HTTPVERSION
followed by common headers



Unusual request e.g. those with an inappropriate
body or those that use incorrect line terminators
should be examined

Detecting Fingerprinting Activities

4. Look for Unusual method line
syntax



Most browsers are fairly well
behaved when they issue a Request


Watch for unusual spacing or
corrupted version information


E.g.
"GET / HTTP/9.Q"

Detecting Fingerprinting Activities

5. Verify the web client’s identity



If a request’s User
-
Agent field indicates
one type of browser but the client behaves
like another, suspicion should be raised



May result in many false alarms since
some privacy applications purposely hide
the name of the browser

Detecting Fingerprinting Activities


Some Suspicious HTTP Reponses an IDS
can look for:


1. Headerless Responses


HTTP/0.9 type clients are rare these days.



If a server sends back a headerless response,
it is possible that someone is pretending to
be such a client or a request that has
confused a server was received

Detecting Fingerprinting Activities

2. Unusual and Repeated Errors


Many of the fingerprinting tests are
attempts to provoke error responses.
(non
-
200 OK).


Rare error Reponses e.g. “413 Request
Entity Too Large” should raise suspicion.


Errors that are common in other servers
but rare in a particular server should also
be noted.

Defending against Web Server
Fingerprinting

1. Decide on a set of Common Interface and
Behavior for web servers



Fingerprinting succeeds because each vendor
has a slightly different interpretation of how an
HTTP server should behave


If all the developers agree on a common
interface, then fingerprinting would be more
difficult


As web server attacks become more frequent
and severe this sort of strategy may become
more attractive


Defending against Web Server
Fingerprinting

2. The web server software should have
configuration options to allow the
administrator to have more control over
the lexical, syntactic and semantic
elements of Responses.



E.g. by modifying a configuration file, the
server can be made to behave like
another server or act like no known server.


Currently, it is uncommon to find any
server configuration options that allow
identity hiding.



Apache’s “ServerTokens” directive allows
an administrator to vary the Server
header’s level of detail, but doesn’t hide it
entirely.



E.g. the server can either return


“Server: Apache” or


”Server: Apache/2.0.41 (Unix)”

Defending against Web Server
Fingerprinting

3. Use Variable Responses



E.g. instead of returning the same
“Not Found” message all the time, a
number of different messages could
all be used randomly. E.g. “File Not
Found” or “Not found”


Fingerprinting becomes more difficult
since a static picture of the target
can not be developed



Ability of a client and server to
converse will not be affected since
each of the variations would be
semantically equivalent.

Conclusion


Hiding the identity of an HTTP server
is no easy task.



The response to the carefully created
HTTP requests contains enough
information to identify a web server
with high certainty.



However by using the same fingerprinting
methodology as reference, IDS that can
detect fingerprinting attempts can be
developed



Web server developers can increase the
security of their web servers by modifying
the servers so that fingerprinting attempts
will be unreliable


References



Detecting and Defending against Web
-
Server Fingerprinting


ACSAC 2002


http://www.acsac.org/2002/abstracts/96.html


Hmap web page


http://wwwcsif.cs.ucdavis.edu/~leed/hmap/



Nmap web page


http://www.insecure.org/nmap/



NetCraft


http://www.netcraft.com



CERT Advisories


http://www.cert.org/advisories/