State of Software Security Report

beckonhissingInternet και Εφαρμογές Web

10 Νοε 2013 (πριν από 3 χρόνια και 7 μήνες)

140 εμφανίσεις

September 22,2010
State of Software
Security Report
The Intractable Problemof Insecure Software
Software Security Simplified
VOLUME 2
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
AS EVERY CIO AND CISO IS AWARE,
the flood of news generated by attacks against
insecure software continues unabated across all industry verticals and market segments.
Since the publication of Volume 1 at the beginning of the year,there have been multiple
new zero-day vulnerabilities reported in Microsoft Windows,at least six material data
breaches,10K filings from Intel disclosing a breach similar to the Chinese attack on
Google,and widely covered security concerns about mobile apps,cloud service providers
and SCADA systems.Yet despite this evidence that software security efforts are not
keeping pace with attacks there is good news to report.
It is heartening to see that CXO software security concerns are beginning to translate
into concerted efforts to move from ad-hoc security testing to a new paradigm of
application risk governance characterized by standardized processes and operating con-
trols extended uniformly across the enterprise.Given the state of the application threat
environment,it is not surprising that over 60%of all of Veracode enterprise customers
are launching a formal,comprehensive security programfor the very first time.It is this
action that has driven the submission of nearly 1,400 new applications representing
nearly 200%increase in the use of Veracode’s cloud-based assessment service over the
past reporting period.
This report represents the code-level analysis of 2,922 applications (as compared to
1,591 applications in Volume 1),a sure sign that more development and security
teams are taking the security of internally developed and third-party components and
applications seriously.The data also shows that once vulnerabilities are detected and
remediation advice is provided,developers are quick to achieve an acceptable level of
security.And,when a class of software—such as financial services applications—
makes security a priority it does appear that security quality improves,particularly with
respect to common vulnerabilities such as Cross-site Scripting.When this evidence
of progress is juxtaposed with my conversations with CIOs and CISOs who are awak-
ening to the importance of security accountability across the software supply chain,
I see a climate that is conducive to more secure software in the future.
For you who are ready to act now,this report comprises security intelligence gleaned
frombillions of lines of code analyzed by the world’s first and only cloud-based applica-
tion risk management services platform.It is our hope that we can assist you to make
and buy more secure software.
Best Regards,
MatthewMoynahan
Chief Executive Officer,Veracode
veracode.com/ceo-blog
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
1
Table of Contents
Introduction..................................................................................2
E
xecutive Summary............................................................................3
Software Supply Chain..........................................................................6
Security of Applications........................................................................16
Application Threat Space Trends.................................................................29
Addendum..................................................................................31
Assurance Level Definitions.....................................................................32
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
2
Introduction
The State of Software Security is a semi-annual report that draws on continuously updated information in Veracode’s
cloud-based application risk management services platform.Unlike a survey,the data comes fromactual code-level
analysis of billions of lines of code and thousands of applications.
The resulting security intelligence cannot be found anywhere else.It represents multiple testing methodologies
(static binary,dynamic,and manual) on the full spectrumof application types (components,shared libraries,web
and non-web applications) and programming languages (including Java,C/C++,.NET,ColdFusion,and PHP) from
every part of the software supply chain (Internally Developed,Open Source,Outsourced,Commercial).For those
executives,security and development professionals who want to better understand the vulnerabilities that threaten
the integrity and performance of software in the software supply chain,this series of reports is essential reading.
In Volume 2 of the State of Software Security there are nearly 1,400 more applications than in the inaugural report,
reflecting the growing use of independent,cloud-based application risk management services.As before,the report
first examines the security quality of applications by type of supplier in the software supply chain and then explores
application security by language,industry,and by application type across both web and non-web applications.
Newin Volume 2 are data fromthird-party assessments,the first inclusion of PHP and ColdFusion applications,
a comparison of static binary,dynamic,and manual testing effectiveness,and additional analytics on Financial
industry applications.
Veracode welcomes any questions or comments fromreaders and will continually strive to improve and enrich the
quality and detail of our analysis.Additionally,we invite all members of the software supply chain to participate in
constructive dialogue on the topic of software security at veracode.com/ceo-blog.
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
3
Executive Summary
The following are some of the most significant findings in the State of Software Security Volume 2,representing
2,922 applications assessed in the last 18 months by Veracode SecurityReview
®
,a cloud-based application risk
management services platform.
1.More than half of all software failed to meet an acceptable level of security and 8 out of 10 web
applications failed to comply with the OWASP Top 10
2.Cross-site Scripting remains the most prevalent of all vulnerabilities
3.Third-party applications were found to have the lowest security quality
4.Developers repaired security vulnerabilities quickly
5.Suppliers of Cloud/Web applications were the most requested third-party assessments
6.No single method of application security testing is adequate by itself
7.The security quality of applications fromBanks,Insurance,and Financial Services industries was not
commensurate with their business criticality
Key Findings
1.More than half of all software failed to meet an acceptable level of security and 8 out of 10 web
applications failed to comply with the OWASP Top 10
57%of all applications were found to have unacceptable application security quality on first submission,even
when standards were adjusted for applications considered less business critical (Figure 3).Even more troublesome,
more than 80%of internally developed and commercial web applications failed to comply with the OWASP Top 10
(Figure 5),an industry standard list of critical web application errors.
The level of risk in terms of repair costs,business continuity,and brand fromso many business critical applications
failing to meet an acceptable level of security on first submission is staggering.The potential exposure to brand
reputation and loss of revenue frominterruptions to business operations is significant.
Recommendation:Utilize industry standards such as OWASP Top 10 and CWE/SANS Top 25 list of most danger-
ous software errors as minimumthresholds and compliance policies to which applications need to adhere.
2.Cross-site Scripting remains the most prevalent of all vulnerabilities
Cross-site Scripting (XSS) remains the most prevalent vulnerability category,accounting for 51%of all vulnerabilities
uncovered by Veracode’s combined static binary,dynamic,and manual security testing methods (Figure 13)..NET
applications,in particular,exhibited an abnormally high rate of Cross-site Scripting vulnerabilities,resulting from
the use of.NET controls that do not automatically encode output (Table 4).While not as numerous,Cryptographic
Issues—a category that includes unencrypted or inadequate encryption of data—appeared in the most applications,
with 41%of all applications containing one or more vulnerabilities in this category (Figure 14).These statistics un-
derscore the need for developers to become better educated and better equipped to avoid common vulnerabilities.
Recommendation:These flaws are easy to fix once found (Figure 4).Focusing on developer education and
awareness is a cost-effective way to avoid introducing them.
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
4
3.Third-party applications were found to have the lowest security quality
Third-party code is getting more attention since Veracode first highlighted in Volume 1 of this report,that between
30%and 70%of software submitted as internally developed contained identifiable third-party components.Both
Safecode.org
1
and a report fromthe research firmSecunia
2
have recently reinforced the elevated risks associated
w
ith third-party software in the supply chain.In Figure 3,Veracode shows that applications fromall types of
third-party suppliers were less secure than Internally Developed applications on first submission.Third-party
suppliers failed to achieve acceptable levels of security 81%of the time.However,in Figure 2 it is also evident
that third-party code is an essential part of the every organization’s portfolio,comprising 29%of all applications
submitted to Veracode.Furthermore,between 20%and 37%of very high or high criticality applications are
sourced fromthird-parties.
Recommendation:Both internal and third-party components and applications must be subjected to the same level
of security verification to ensure consistent security quality across the application portfolio.Procurement contracts
for outsourced or commercial software vendors should insist upon the authority to performindependent security
testing and specify minimumsecurity acceptance criteria.
4.Developers repaired security vulnerabilities quickly
A common misperception is that it is easy to find defects and difficult to fix them.While this may often be true of
functional defects in software it is less true for security defects.Observing a variance fromfunctional specifications
is relatively easy but determining the root cause can be hard.Conversely,determining that an application allows
someone to do something it was never intended to do is actually quite difficult but relatively easy to fix once known
(Figure 4).Among the most encouraging data in this report,the evidence that development teams using Veracode
can fully remediate unacceptable levels of security quality in only 16 days and 1.1 resubmissions on average is
among the best reasons to equip development teams with effective security testing and training—they can and
did improve the state of software security quickly when properly informed.
Recommendation:Equip development teams with the appropriate application security resources and knowledge
and plan for security verification and remediation in the project timeline fromthe outset.
5.Cloud/Web applications were the most requested third-party assessments
Assessments of third-party applications at the request of a purchasing organization have grown linearly over the past
6 quarters,reflecting the increased concern over the security of software in the supply chain and the availability of
effective,newtechnologies such as cloud-based,static binary analysis that make third-party assessments possible
without requiring source code or tools.In a newsection of the report,Veracode explored the types of applications
most often reviewed by request.As Figure 8 shows,suppliers of cloud and web applications made up nearly 60%
of all third-party assessments requested,while integrators and commercial software providers made up most of the
rest in equal parts.Since cloud-based applications are relatively new,their significant presence indicates the reason-
able security concerns they raise and the criticality of the work they perform.Like other third-party software,these
assessments resulted in lowlevels of acceptable security and rapid remediation.
Recommendation:Require Third-party Cloud/Web application and service providers to demonstrate verification
of application security quality.
1
www.safecode.org
2
www.theregister.co.uk/2010/07/12/secunia_threat_report
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
5
6.No single method of application security testing is adequate by itself
Others have reported this year on the inadequacy of web application scanning.
3
Veracode’s code-level analysis of
vulnerabilities using multiple testing techniques on the same applications confirms that dynamic web application
scanning tools are not sufficient as the sole testing method.Similarly,manual penetration testing,while necessary
t
o fully comply with policies such as the OWASP Top 10 and the CWE/SANS Top 25,lacks consistency of cover-
age and will rarely detect all instances of commonly occurring vulnerabilities.However,while the evidence shows
that static binary analysis provides the most consistent breadth and depth of coverage,it is also true that not all
design and business logic vulnerabilities are discoverable with static methods alone.
Recommendation:CISOs and CIOs should viewdifferent testing techniques as operating controls that each play
an important role in a comprehensive policy driven program.Multiple testing techniques should be adopted based
on application business criticality and type of application.The use of multiple techniques is the only way to comply
with industry standard security polices such as the OWASP Top 10 and the CWE/SANS Top 25 Most Dangerous
Software Errors.
7.The security quality of applications fromBanks,Insurance,and Financial Services industries was not
commensurate with their business criticality
In a very interesting dichotomy,Financial Industry applications were found to have the best rawcode-level security
scores of any industry but only average levels of acceptability when the business criticality of an application was
considered.This speaks to the high degree of awareness such firms have about code-level threats but also to
the inadequate application risk management practices employed relative to the importance of these applications.
Financial Services applications in particular demonstrated an exceptionally lowprevalence of the most common
vulnerabilities—less than half the rate of Cross-site Scripting errors as compared to Banks and Insurance (Table 7).
The implication is that training,testing,and a high degree of focus on specific types of errors can make a signifi-
cant difference.The net result is both encouraging because improvement is possible;and sobering because the
most critical of applications remain too insecure.
Recommendation:Inventory and classify the application inventory based on business criticality.In the absence
of this business context,an understanding of the code-level security quality is insufficient.What seems to be
good code-level security quality may still not render the application fit for purpose when business criticality is
taken into account.
3
www.darkreading.com/vulnerability_management/security/app-security/showArticle.jhtml?articleID=222601207
Software Supply Chain
While people tend to think that software is written fromscratch,modern economics and productivity imperatives
have long since changed the reality.Today software is truly a composition of code originating frommultiple sources
across the world and most organizations rely on third party software suppliers for critical applications.
In this section we examine the security quality of software
produced by the software supply chain most often found
in organizations:Internally Developed,Commercial,Open
Source,and Outsourced.Only by understanding the
various degrees of software security quality produced by
supply chain participants can we begin to understand the
requirements to change policies and processes,properly
manage application risk in organizations,and protect critical
software infrastructure.
For CIOs and CISOs,the evidence continues to point to an increasing percentage of software infrastructure and
associated liability coming fromunknown and unmanaged third-parties.While nearly a third of all applications submit-
ted to Veracode were identified as third-party,code-level analysis reveals that third-party code in the supply chain is
significantly understated by most organizations.
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
6
For CIOs and CISOs,the evidence
continues to point to an increasing
percentage of software infrastructure
and associated liability coming from
unknown and unmanaged third-parties.
Veracode sampling found as much as 76%of code
submitted as Internally Developed was identifiably
fromthird-parties,most often in the formof Open
Source components and Commercial shared libraries
and components.Furthermore,there was a “nesting
effect” as third-party components themselves often
contained other third-party components.
Distribution of Application Development by Supplier Type
Figure 1 reveals that close to a third of the applications analyzed during the reporting period were identified as third-
party (Commercial,Open Source and Outsourced vendors).The percentage of outsourced applications represented in
t
he dataset was lowat 1%.Part of this is a data labeling issue.Organizations sometimes consider code developed by
outsourcers as “internally developed.” Veracode encountered many instances where flaws in “internally developed”
code were traced back to software supplied by outsourcing partners.Another factor is that outsourcing contracts
have been silent on the topic of security testing and remediation.As these contracts renew,Veracode expects to see
independent security verification requirements inserted and an increase in the percentage of identifiably outsourced
code submitted.
Distribution of Application Business Criticality by Supplier Type
We knowthat not all applications have the same level of criticality to the business.However,it is instructive to examine
the sources fromwhich the most business critical applications are derived.Veracode explored the relationship between
application supplier type and business criticality.As Figure 2 illustrates,20%of Very High and 37%of High criticality ap-
plications are developed by third-parties.Domain expertise,
proven functionality,and time-to-market are all factors in the
decision to develop applications internally or procure them
fromthird-parties.The significant presence of third-party
applications identified as critical increases the importance
of applying uniformapplication security verification policies
across internally developed applications and those procured
fromthird-party suppliers.
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
7
The significant presence of third-party
applications identified as “critical”
increases the importance of applying
uniformapplication security verification
policies across all supplier types.
Internally Developed
Commercial
Open Source
Outsourced
22%
6%
1%
71%
Applications by Supplier
Figure 1:Application by Supplier
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
8
Distribution of Application Type and Programming Language by Supplier Type
Table 1 illustrates that nearly one third of commercially developed applications and over half of open source applications
are written in C/C++ indicating a significant reliance on this language platformby these types of software suppliers.
It further indicates that over 65%of the software developed by these same suppliers are non-web applications,while
Internally Developed and Outsourced suppliers are relied on for web applications to about the same degree.The
language and type of application differences among suppliers allows for
policies and acceptance criteria to be tailored to the most prevalent risks
and,among other things,clearly indicates the requirement for C/C++
language and non-web application support when choosing security testing
approaches to third-party software.
V
ery High
High
M
edium
Low
Very Low
1
0%0% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Commercial
Internally Developed
Open Source
Outsourced
*
10% 1%
63%
26%
4%
1%
2
%
1
%
74%
8
0%
21%
2
%
7
1%
2
7%
100%
1
7%
Application Business Criticality by Supplier
Figure 2:Application Business Criticality by Supplier
(* small sample size)
Internally Developed
Commercial
Open Source
Outsourced
11%
29%
51%
0%
56%
45%
45%
81%
33%
24%
4%
14%
2%
3%
0%
5%
61%
36%
29%
71%
39%
65%
71%
29%
C/C++ Java.NET Other Web Non-Web
Supplier Application Profiles
Table 1:Supplier Application Profiles
Support for C/C++ and
non-web applications is
required when choosing
security testing approaches
to third-party software.
Distribution of Security Quality and Remediation Efforts by Supplier Type
The illustration below(Figure 3) depicts Supplier Performance on First Submission as measured by the Veracode
risk adjusted verification methodology.When calculated as a percentage of total applications submitted 57%of all
a
pplications were deemed to have “unacceptable” security quality upon first submission.Outsourced vendors
achieved the lowest scores followed by Commercial suppliers,Open
Source and Internally Developed applications.These poor results were
consistent with the Veracode’s first State of Software Security report.
It remains clear that most organizations do not have developers trained
in secure coding principles or have not implemented a secure software
development lifecycle.
Applications that do not achieve an acceptable level of security on first submission are returned to the supplier with
potential vulnerabilities identified by location in the code and with remediation instructions.Of those applications that
were remediated and resubmitted,Figure 4 shows that most achieved acceptable levels of security within 16 days
and in 1.1 builds (i.e.resubmissions following the initial analysis of the application).These encouraging results point
to the effects of independent,cloud-based security testing.With a similar approach across supply chain participants,
CIOs and CISOs can use this information to quantify application security risk versus the cost to mitigate,better
estimate software development project costs and schedules,and control “rework charges” associated with security
vulnerabilities in third-party agreements.
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
9
Overall
Outsourced
Open Source
Internally Developed
Commercial
S
10%0% 20% 30% 40% 50% 60% 70% 80% 90% 100%
46% 54%
42% 58%
7% 93%
43% 57%
35% 65%
A
cceptable
N
ot Acceptable
*
Supplier Performance on First Submission
(Adjusted for Business Criticality)
Figure 3:Supplier Performance on First Submission (Adjusted for Business Criticality)
No real change in percentage
of applications deemed to have
“unacceptable” security quality
upon first submission—58%in
Volume 1,57%in Volume 2.
Distribution by Supplier’s Ability to Meet Security Compliance Policy by Supplier
CIOs,CISOs,customers and internal auditors are increasingly enforcing compliance with application security policies.
Two independent policy standards,one specifically for web applications fromOWASP (OWASP Top 10) and one
for applications of any type fromthe US Government,MITRE and the SANS Institute (CWE/SANS Top 25 Most
Dangerous Software Errors) have been adopted by many organi-
zations.An analysis of a supplier’s ability to meet these industry
standards is useful when determining software acceptance
criteria.For software providers,evidence of compliance with
these policies,such as the VERAFIED HIGH ASSURANCE
4
marks for OWASP Top 10 and CWE/SANS Top 25,anticipates
customer security concerns and can differentiate their products.
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
10
2
0
1
5
1
0
5
0
1
.6
1
.2
0
.8
0.4
0
1
2
I
nternally Developed
C
ommercial
O
pen Source
15
1
6
1.07
1
.16
1
.08
1
.1
O
verall
19
DAYSTOREMEDIATE
REMEDIATIONSUBMISSION
TOPASS
Remediation Performance by Supplier
Figure 4:Remediation Performance by Supplier
10%0% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Not Acceptable
12% 88%
40% 60%
7% 93%
Open Source
Internally Developed
Commercial
Acceptable
Figure 5:OWASP Top 10 Compliance by Supplier on First Submission
OWASP Top 10 Compliance by Supplier on First Submission
Adopting OWASP Top 10 or
CWE/SANS Top 25 policies promotes
uniformverification standards and
performance measurement across
application inventory.
4
www.veracode.com/directory/VERAFIED-logo-program.html
Figure 5 shows the percentage of web applications that met the OWASP Top 10 (2010) policy by supplier.An
application was labeled Not Acceptable if it contained any vulnerabilities defined in the standard lists.The number of
Commercial and Internally developed web applications that were not acceptable is staggering at more than 80%.The
difference between this extraordinary indicator of insecurity when compared to the bad but much higher acceptable
levels of security identified earlier is largely explained by the high number of web applications that were submitted
as lower business criticality.Another contributing factor may be due to
the increasing number of microsites that are generally developed on be-
half of large enterprises to support time-based marketing or commercial
initiatives where time-to-market is the most important driver.Given the
level of interconnectedness of software in most organizations Veracode
observes that lowbusiness criticality values for web applications or the
temporal nature of their existence probably understates the risk and
encourages customer to adopt more stringent policies such as the
OWASP Top 10 for all web applications.
Figure 6 examines suppliers ability to deliver applications as measured by compliance against the CWE/SANS Top 25
Most Dangerous Software Errors.All applications both web and non-web were included in this analysis.Commercial
and Internally developed applications performed the best with about 50%and 52%of applications meeting accept-
ance respectively.The difference in the ranking of open source applications as worse in the ranking when compared
to their performance against OWASP may be due to the fact that most open source applications analyzed in the
dataset are non-web applications.
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
11
More than 8 out of 10
commercial and interally
developed web applications
failed against OWASP Top
10 upon first submission.
10%0% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Not Acceptable
52% 48%
38% 62%
20% 80%
50% 50%
Outsourced
Open Source
Internally Developed
Commercial
Acceptable
*
Figure 6:CWE/SANS Top 25 Compliance by Supplier on First Submission
CWE/SANS Top 25 Compliance by Supplier on First Submission
Distribution of Most Common Security Vulnerabilities by Supplier
The distribution of security vulnerabilities by type of supplier may point to more or less effective practices and help
in choosing future suppliers.Table 2 reveals relatively similar results by suppliers in terms of both prevalence and
t
ype of vulnerabilities detected.Cross-site scripting and cryptographic issues appear in the top five vulnerabilities
across all supplier types.
Third-Party Risk Assessments
Newin this volume is an analysis of third-party risk assessments performed against vendors at the request of a buyer
of software or software development services.These buyers may be purchasing already developed applications for
internal use (e.g.Commercial-off-the-shelf or COTS applications),applications to be developed by someone else,or
applications and components to be re-distributed under a re-licensing arrangement.Mergers and acquisitions may also
trigger a third-party assessment.Third-party risk assessments are among the fastest growing types of assessments
requested of Veracode,with linear growth rates over the last 6 quarters.
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
12
Vulnerability Distribution by Supplier
Cross-site Scripting 49%
(
XSS)
CRLF Injection 14%
Information Leakage 10%
Cryptographic Issues 6%
SQL Injection 5%
Directory Traversal 3%
Buffer Overflow 3%
Potential Backdoor 2%
Untrusted Search 2%
Path
Time and State 2%
Error Handling 1%
Encapsulation 1%
Credentials Mgmt
<1%
API Abuse
<1%
Insufficient Input
<1%
Validation
Cross-site Scripting 56%
(
XSS)
Information Leakage 16%
CRLF Injection 6%
Cryptographic Issues 5%
Directory Traversal 4%
SQL Injection 4%
Buffer Overflow 2%
Potential Backdoor 2%
Numeric Errors 2%
Error Handling 2%
Time and State 1%
Credentials Mgmt
<1%
Buffer Mgmt Errors
<1%
API Abuse
<1%
OS Command
<1%
Injection
CRLF Injection 15%
Numeric Errors 14%
Buffer Mgmt Errors 14%
Cross-site Scripting 13%
(XSS)
Cryptographic Issues 12%
Error Handling 9%
Buffer Overflow 9%
Time and State 4%
Directory Traversal 4%
Potential Backdoor 1%
SQL Injection 1%
Information Leakage 1%
API Abuse
<1%
Credentials Mgmt
<1%
Session Fixation
<1%
Cross-site Scripting 31%
(
XSS)
Directory Traversal 16%
Cryptographic Issues 14%
Time and State 12%
Information Leakage 9%
Credentials Mgmt 8%
API Abuse 6%
CRLF Injection 3%
SQL Injection 2%
Insufficient Input
<1%
Validation
Error Handling
<1%
Session Fixation
<1%
OS Command
<1%
Injection
Other
<1%
Race Conditions
<1%
Internally Developed Commercial Open Source Outsourced
Table 2:Vulnerability Distribution by Supplier
Figure 7 shows the types of enterprises that are requesting third-party assessments.They are predominantly in the
Financial (including Banks,Insurance,and Financial Services) or Software/IT Services market categories where this
category represents enterprises that are both software producers and providers of IT services and equipment.
One of the most striking themes fromthese assessments is the implication for cloud-based services.Figure 8
shows that Vendors that provide cloud based services,either in Cloud only or Cloud as an option (Cloud+Deployment)
accounted for almost 60%of all reviewed third-party applications.
The other Vendor Types for which reviews were requested were
general ISV’s or companies that specialize in integrating disparate
components fromseveral sources—all of which are likely
participants in cloud-based solutions.
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
13
S
oftware/IT Services
F
inancial
Other
28%
17%
55%
F
igure 7:Requester Distribution by Industry
Requester Distribution by Industry
Cloud only or Cloud as an option
(Cloud + Deployment) accounted
for almost 60%of all reviewed
third-party applications.
Cloud + Deployed
Integration
ISV
Cloud
Consulting
Deployed
18%
14%
21%
45%
1%
1%
Figure 8:Reviewed Application Count by Vendor Type
Reviewed Application Count by Vendor Type
The relative proportion of third-party reviews broken down by application functional area is provided in Figure 9.
In this diagram,the categories used for functional area are derived fromthe Balanced Scorecard model (BSC),a
widely-used strategic planning and management system.
5
BSC identifies four functional perspectives by which to view
and measure an organization:Financial,Customer,Operations,and Learning and Growth.Any application that deals
with day-to-day business activity is included in the “Operations” category shown in Figure 9.This includes business
process management applications,product development,information management utilities,IT management tools,
and applications to support all non-financial governance functions such as legal and operational risk management.The
“Customer” category includes all content management,customer relationship management and web-facing services
provided to customers.The “Learning and Growth” category includes applications to support HR,training,and human
capital management.Financial applications include traditional accounting and finance functions as well as an important
and growing class of application that provides mobile access for banking and other finance related tasks.
It is interesting to note that Operations is the leading func-
tional area for third-party assessments which comprises about
the same portion of requests as the combination of Finance
and Customer applications.This indicates that companies are
proactively requiring assessments of applications across a
wide variety of internal applications (Operations and Finance)
as well as external customer-facing web sites.
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
14
Operations
Financial
Customer
Learning Growth
15%
11%
29%
45%
Figure 9:Requested Third-party Assessments by Application Purpose
Application Type Definitions:Operations category includes applications supporting day-to-day
non-financial business activity such as product development,information management utilities,
IT management tools etc.;Financial category traditional accounting and finance applications and
newer mobile banking applications;Customer category includes customer relationship manage-
ment and content management applications and web customer support applications;Learning
and Growth includes applications to support HR,training and human capital management.
Requested Third-party Assessments by Application Purpose
Companies are proactively requiring
assessments of applications across a
wide variety of internal applications
(Operations and Finance) as well as
external customer-facing web sites.
5
The Balanced Scorecard (BSC) was originated in the 1990s by Drs.Robert Kaplan (Harvard Business School) and David Norton as a performance
measurement framework to enrich traditional financial performance measures with strategic non-financial performance measures,thereby giving
a more ‘balanced’ viewof organizational performance.See www.balancedscorecard.org for additional information
Figure 10 reveals that,like third-party supplier code in general,third-party risk assessments result in high rates of unac-
ceptable security on first submission.4 out of 5 assessments failed to achieve acceptable levels of security on first
review.Most third-party assessed suppliers also remediated faster than applications on average,with three-quarters
of all applications requiring only 11 days to achieve acceptable levels of security quality.It should be noted that many
customers implementing a third-party risk management program
employ a customer success programmanger or an internal resource
that is tasked with policy creation,coordination of third-parties and
programexecution.This focus may be contributing to a relatively
short amount of time for achieving compliance.The fast turnaround
further implies that requiring a third-party assessment does not result
in delayed deployment of more than a couple of weeks,making it
worth the trade-off.
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
15
High ROI with minimal impact
to timeline fromthird-party risk
assessments:Three-quarters
required less than 11 days to
achieve security quality level
required by requesting enterprise.
Third-party assessments is one of the fastest growing types of security programs as CIOs and CISOs become
aware of the unbounded risk inherent in the software supply chain.At one company,a facilitated engagement
with third-parties improved the state of software security for all parties.
» ProgramTime
6 months
» Third-Parties Assessed
Close to 40 applications fromdistinct vendors
(in excess of 50 million lines of code)
» Vulnerabilities Remediated
Over 500 Severity 5 and 4 vulnerabilities
(over 7000 vulnerabilities in total)
» Lessons Learned
The impossible is possible.Facilitated independent
verification improved security for a large number of
third-party applications in a short timeframe.
» Next Steps
Additional third-parties are proactively pursuing
verification and the company is using the intelligence
gained so far to revise third-party acceptance policies.
Not Acceptable
Acceptable
19%
81%
Figure 10:Third-party Assessments:Performance Upon Initial Submission
Third-party Assessments:Performance Upon Initial Submission
A PROFILE IN VERIFICATION
Security of Applications
The previous section presented information fromthe Software Supplier and Purchaser perspectives in an attempt
to help enterprises properly manage application risk in the software supply chain.In this section of the report we
explore security risks related to web and non web applications,programming languages,types of vulnerabilities,and
industry alignment.Newin this report,we further consider the effectiveness of multiple security testing techniques
and provide a deeper investigation of application security in Banking,Insurance,and Financial Services companies.
As background,software vulnerabilities are the attack points in applications used by hackers to compromise a system.
Different types of applications have different attack points.For example,web applications have different attack sur-
faces than desktop software or databases.Additionally,vulnerabilities can vary significantly by programming language
and platforms such as the Windows versus BlackBerry operating systems.It is also possible for applications in differ-
ent industries to have different vulnerabilities based on the secure coding skills of the engineering population serving
those industries (e.g.Financial Services versus Retail) and the sophistication of their software development practices
or central security teams.
While no software will ever be perfectly secure,understanding what makes applications more or less vulnerable
provides the basis for CIOs,CISOs,and software professionals to manage application portfolio risk rather than
remain blindly susceptible to catastrophic loss of information,business continuity,and reputation.
Distribution of Application by Type
All applications analyzed by Veracode are inventoried and classified according to a profile which includes key
characteristics such as whether the application is web-facing,its language and platform,and the industry of the
organization submitting it.In this reporting period we observed a
slight shift in favor of non-web applications.They grewto 44%
(from40%as reported in Volume 1) and web applications were
down to 56%(from60%as reported in Volume 1).This reflects
a heightened security awareness for legacy and back-end appli-
cations and not just those applications exposed to the web.
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
16
Web Applications
Non-Web Applications
44%
56%
Web versus Non-Web Applications
Figure 11:Web versus Non-Web Applications
Non-web applications analyzed
grewfrom40%in the prior report
to 44%,reflecting the expansion of
application security efforts beyond
web applications to legacy and
back-end applications.
Distribution of Applications by Language
An analysis of the Distribution of Applications by Language is a useful indicator and reasonable proxy for the
ever-changing attack surface of the world’s software infrastructure.
In our last report we showed the relative distributions of three development platforms—Java,C/C++,and.NET.Java
still leads at 50%,up slightly from47%in our last report.However,C/C++ and.NET have swapped positions,and we
are nowseeing.NET applications leading C/C++ by a factor of 3 to 2.
Newin this report are two newplatforms,ColdFusion and PHP,which
account for 1.4%and 0.7%of all applications,respectively.These
numbers should not be used as a representation of the market share
of these two platforms because Veracode only recently developed the
capabilities required to analyze them.We expect that over time,these
percentages will increase to better approximate the real-world
distribution of these platforms in the enterprise.
To better understand the impact of programming language on application security,Table 3 shows the median flaw
density for each.The median flaws per thousand lines of code (KLOC) for Java,C/C++,and.NET are similar.Many
people ask whether switching languages will improve application security.Our data shows that all applications,no
matter what language is used,require secure development practices to be secure.
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
17
Java
.NET
C
/C++
ColdFusion
PHP
50%
19%
1%
1%
29%
<
Applications by Language Family
Figure 12:Applications by Language Family
Our data shows that all
applications,no matter
what language is used,
require secure development
practices to be secure.
ColdFusion was very different with a median of 5.2 security flaws per KLOC.While ColdFusion applications were
a small percentage of the total number of applications assessed in this period,more than 35 applications were
included.The ColdFusion Markup Language is very compact relative to Java and.NET,which may explain most of
this difference.The simplicity of CFML also allows less accomplished programmers—and even non-programmers—
to develop web applications,so it stands to reason that developer inexperience is another contributing factor to
ColdFusion’s flawdensity.
Distribution of Applications by Vulnerability Type
The charts on the following page depict top vulnerability categories by prevalence,presented fromtwo different
perspectives.The first is by vulnerability frequency,which illustrates the percentage of the total vulnerabilities
discovered.The second is by affected applications,which shows the percentage of applications containing one or
more of the vulnerabilities in each category.Rows highlighted in red are vulnerability categories that also appear in
the CWE/SANS Top 25 (2010) or OWASP Top 10 (2010) standards.There is considerable overlap between Veracode
findings and these industry standards,further confirming the relevance of these vulnerability categories as top
areas of security weakness to focus on for enterprises.
Cross-site Scripting (XSS) remains the most prevalent vulnerability category by frequency,accounting for 51%of
all vulnerabilities in the data set.The next three categories behind XSS—Information Leakage,CRLF Injection,and
Cryptographic Issues—maintain the same rankings as we observed in our last report.SQL Injection was somewhat
more prevalent this time and Potential Backdoors broke into the top ten categories.The rise in Potential Backdoors
isn’t necessarily cause for alarm.Automated scanning cannot reliably judge intent.In order to distinguish the
malicious cases fromthe legitimate ones,Potential Backdoors should be inspected carefully by someone with
an understanding of the application’s intended design and function.
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
18
C/C++
ColdFusion
Java
.Net
0.01
1.83
0.01
0.01
0.03
5.28
0.03
0.04
1.07
8.90
0.56
0.72
0.13
11.98
0.16
0.16
First Quartile Median Mean Third Quartile
FlawDensity by Language
Table 3:FlawDensity by Language
Even though a category may account for a small percentage of the total vulnerabilities,the frequency with which
it appears across different applications may be a more illuminating statistic.Viewing the vulnerabilities by affected
applications,Cryptographic Issues remains atop the list,with 41%of all applications containing one or more vulnera-
bilities in this category.This category once again comprised insufficient entropy,plain text storage of sensitive data,
use of hardcoded cryptographic keys,and use of algorithms with inadequate encryption strength.The unnerving part
of this statistic is that while static analysis can uncover a variety of cryptographic flaws,there are far more complex
mistakes that can only be detected by a skilled practitioner,either through design reviewor a focused code review.
If automated scanning is uncovering simple cryptographic mistakes at a rate that affects over 40%of all applications,
one has to wonder howmany other cryptographic issues are lurking beneath the surface.Once again,this statistic
underscores the need for developers to become better educated with this less publicized but still prevalent issue in
order to safely incorporate cryptographic mechanisms into their applications.
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
19
Top Vulnerability Categories
(Overall Prevalence)
Figure 13:Top Vulnerability Categories (Overall Prevalence)
C
ross-site Scripting (XSS)
I
nformation Leakage
CRLF Injection
C
ryptographic Issues
SQL Injection
D
irectory Traversal
Buffer Overflow
P
otential Backdoor
T
ime and State
E
rror Handling
Credentials Management
Numeric Errors
Untrusted Search Path
API Abuse
Encapsulation
5%0% 10% 15% 20% 25% 30% 35% 40% 45% 50%
55%
51%
12%
1
1%
6
%
4%
4%
2%
2%
2%
1%
1
%
1%
1%
1%
1%
I
ndicate categories that are in the OWASP Top 10 or CWE/SANS Top 25
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
20
Vulnerabilities by Language Distribution
The table on the following page presents the most prevalent categories
(by share of total vulnerabilities discovered) based on language family..NET
applications continue to exhibit an abnormally high frequency of Cross-site
Scripting vulnerabilities relative to the other enterprise languages.This can
be attributed to developer use of.NET controls that do not automatically
encode output.While we reported flawdensity in aggregate for ColdFusion
above,vulnerability rankings by type of vulnerability for Cold Fusion and
PHP are not displayed due to the small sample size for this reporting period.
Cryptographic Issues
Information Leakage
C
ross-site Scripting (XSS)
Directory Traversal
C
RLF Injection
SQL Injection
T
ime and State
Credentials Management
API Abuse
Encapsulation
I
nsufficient Input Validation
Error Handling
Potential Backdoor
Buffer Overflow
OS Command Injection
5%0% 10% 15% 20% 25% 30% 35% 40% 45%
4
1%
4
0%
34%
27%
2
6%
24%
2
0%
1
8%
1
2%
9%
8%
8%
8%
8%
7%
I
ndicate categories that are in the OWASP Top 10 or CWE/SANS Top 25
Top Vulnerability Categories
(Percent of Application Affected)
Figure 14:Top Vulnerability Categories (Percent of Application Affected)
.NET applications
continue to exhibit an
abnormally high frequency
of Cross-site Scripting.
Vulnerability Distribution by Analysis Type
As the market for application security testing changed frommanual penetration testing alone to automated dynamic
and static analysis,providers of security services and products tended to gravitate to one of the methods,reflecting
their particular expertise.With different providers promoting different methods,the idea that one method was most
appropriate for certain types of applications and certain people and another for other types and other people began to
take hold in organizations.Web applications were tested by security professionals or QA teams using dynamic web
application scanners.Developers were asked to use static code analyzers.Manual penetration testing services were
used by external consultants on the most critical applications.Although they were not actually mutually exclusive,
these distinctions and the total cost of implementing more than one led many organizations to choose between
methods rather than choose the right mix of methods for an application based on its business criticality.
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
21
Java C/C++.NET
Vulnerability Distribution by Language
Cross-site Scripting (XSS) 46%
CRLF Injection 17%
Information Leakage 16%
Cryptographic Issues 7%
Directory Traversal 4%
SQL Injection 3%
Time and State 2%
Untrusted Search Path 2%
Credentials Mgmt 1%
Encapsulation 1%
API Abuse 1%
Insufficient Input Validation <1%
Race Conditions <1%
OS Command Injection <1%
Dangerous Functions <1%
Buffer Overflow 32%
Potential Backdoor 21%
Error Handling 18%
Numeric Errors 13%
Buffer Mgmt Errors 7%
Cryptographic Issues 3%
Directory Traversal 2%
Dangerous Functions 1%
Time and State <1%
Race Conditions <1%
API Abuse <1%
Format String <1%
OS Command Injections <1%
Credentials Mgmt <1%
Untrusted Search Path <1%
Cross-site Scripting (XSS) 66%
Cryptographic Issues 13%
Directory Traversal 8%
CRLF Injection 4%
Information Leakage 4%
Insufficient Input Validation 2%
SQL Injection 1%
Credentials Mgmt 1%
Potential Backdoor <1%
Time and State <1%
Error Handling <1%
OS Command Injection <1%
Buffer Overflow <1%
Untrusted Search Path <1%
Dangerous Functions <1%
Table 4:Vulnerability Distribution by Language
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
22
The problemwith over-relying on one method is that there is no single testing methodology that can detect all of
the important vulnerability types with sufficient depth and breadth—there is no silver bullet.For example,conducting
a manual penetration test while ignoring automated testing is not ideal;penetration testing can identify complex
security issues but it generally provides spotty application coverage,particularly within a single vulnerability category.
Similarly,relying solely on automated static analysis provides excellent coverage but does not account for business
logic,design flaws or environment and configuration issues.Meanwhile,dynamic analysis only tests code paths that
it can discover externally which,as benchmarking comparisons of dynamic web application scanners have shown,
can lead to embarrassing results.
6
While most security experts would agree with these points anecdotally,they
generally lacked hard data until now.
The following table depicts the top 10 vulnerability categories by prevalence,for each analysis type.This allows us
to see what vulnerabilities automated static binary analysis,automated dynamic analysis,and manual penetration
testing find most frequently.The first thing to notice is that many of the most common vulnerability types are pres-
ent in all three tables—Cross-site Scripting,SQL Injection,and Information Leakage,to name a few.However,there
are vulnerability types that are most frequently detected using one particular method,for example,Buffer Overflows
(static),Server Configurations (dynamic) and Authorization Issues (manual).
Static Dynamic Manual
Top 10 Vulnerability by Analysis Type
Cross-site Scripting (XSS) 52%
CRLF Injection 11%
Information Leakage 11%
Cryptographic Issues 6%
Directory Traversal 4%
SQL Injection 3%
Buffer Overflow 3%
Potential Backdoor 2%
Time and State 2%
Error Handling 1%
Information Leakage 44%
SQL Injection 27%
Cross-site Scripting (XSS) 26%
Server Configuration 2%
OS Command Injection <1%
Other <1%
Session Fixation <1%
Cryptographic Issues <1%
Insufficient Input Validation <1%
Authentication Issues <1%
Cross-site Scripting (XSS) 26%
Information Leakage 21%
Other 12%
Cryptographic Issues 11%
SQL Injection 11%
Authorization Issues 7%
Authentication Issues 5%
Insufficient Input Validation 2%
Credentials Mgmt 2%
Directory Traversal 1%
Table 5:Top 10 Vulnerability by Analysis Type
6
www.darkreading.com/vulnerability_management/security/app-security/showArticle.jhtml?articleID=222601207
The previous table provided a high-level viewinto the types of vulnerabilities that were most frequently detected by
different analysis methods,for all applications in the data set.However,we can learn more by drilling down into the
subset of the web applications in the data set that were subjected to both automated static and automated dynamic
testing at customer request.These applications represented 21%of the total web applications analyzed in the past
18 months.In these cases,dynamic analysis came up empty for CRLF Injection and nearly empty for Cryptographic
Errors.It’s not impossible for dynamic testing to find some varieties of CRLF Injection—HTTP Response Splitting
is one example—but the bulk of that category in this application set,which we knowbecause it was found by static
binary analysis,was Log Injection vulnerabilities.These simply cannot be detected by any dynamic method except in
rare circumstances.Furthermore,it’s a striking statistic that static binary analysis detected over 20 times more XSS
vulnerabilities and nearly twice as many SQL injection vulnerabilities than dynamic analysis across these applications,
on average.
What accounts for the disparity between static and dynamic methods,independent of vendor?One major contributing
factor is that static analysis provides comprehensive coverage of the application whereas dynamic analysis only tests
code paths that it can discover externally.Often,dynamic (and even manual) testing completely overlooks portions
of the application that are only reachable under certain circumstances.For example,application functionality may be
gated behind a series of forms that trigger different behavior depending on howthey are filled out.Also,applications
that support different types of users (e.g.view-only,author,editor,administrator,power user,etc.) often restrict the
functionality that each user level can access,meaning that the application must be scanned multiple times,iterating
over all of the user roles,in order to maximize coverage.However,while coverage may be an issue,dynamic analysis
is performed against a live application instance,so a higher percentage of its reported vulnerabilities may be demon-
strably exploitable.Sacrificing coverage in order to reduce the vulnerability triage effort is a risky tradeoff but it is a
tradeoff that some enterprises may choose to make during the early stages of an application security program.
The lesson for CISOs and CIOs is that a robust application security programmust incorporate multiple testing
methods in order to ensure that applications are assessed with sufficient coverage,measured by both depth and
breadth.Becoming overly dependent on too fewanalysis methodologies guarantees blind spots when assessing
overall application risk.
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
23
Distribution of Vulnerabilities by Industry
Industries experience differing levels of cyber threats,may employ web or non-web applications of differing criticality,
use programming languages to different degrees,and have varying levels of maturation with respect to application risk
m
anagement.As a result,comparisons across very different industries can be challenging,although comparisons
within industry sub-segments as we do in the next section can be enlightening.In Table 6 the exceptionally high degree
of Cross-site Scripting errors in Government applications suggests a combination of these factors require a concerted
training and assessment effort to eliminate an easily rectified vulnerability.Software industry security and development
teams face a disproportionately high rate of Potential Backdoors that are more difficult to find and possibly to repair.
Finance industry vulnerabilities are explored in more depth later.
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
24
Finance Software Government Other
Vulnerability Distribution by Industry
Cross-site Scripting 54%
(XSS)
Information Leakage 17%
Cryptographic Issues 7%
CRLF Injection 6%
SQL Injection 4%
Directory Traversal 3%
Buffer Overflow 2%
Time and State 1%
Encapsulation 1%
Insufficient Input 1%
Validation
Potential Backdoor 1%
Credentials Mgmt 1%
Error Handling
<1%
API Abuse
<1%
Buffer Mgmt Errors
<1%
Cross-site Scripting 30%
(XSS)
Information Leakage 19%
Cryptographic Issues 10%
CRLF Injection 8%
Directory Traversal 7%
SQL Injection 5%
Numeric Errors 4%
Buffer Overflow 4%
Error Handling 3%
Potential Backdoor 3%
Time and State 2%
Buffer Mgmt Errors 2%
Credentials Mgmt 1%
API Abuse 1%
Encapsulation
<1%
Cross-site Scripting 87%
(XSS)
SQL Injection 7%
CRLF Injection 2%
Information Leakage 1%
Cryptographic Issues 1%
OS Command
<1%
Injection
Time and State
<1%
Directory Traversal
<1%
Credentials Mgmt
<1%
Encapsulation
<1%
API Abuse
<1%
Error Handling
<1%
Insufficient Input
<1%
Validation
Race Conditions
<1%
Buffer Mgmt Errors
<1%
Cross-site Scripting 45%
(XSS)
CRLF Injection 20%
Information Leakage 10%
Cryptographic Issues 6%
Directory Traversal 3%
SQL Injection 3%
Untrusted Search 3%
Path
Buffer Overflow 3%
Potential Backdoor 2%
Time and State 2%
Error Handling 2%
Credentials Mgmt 1%
API Abuse 1%
Encapsulation 1%
Insufficient Input
<1%
Validation
Table 6:Vulnerability Distribution by Industry
Industry group definitions
The Finance-related industries group combines applications fromthe Financial Service,Insurance,and Banking
industries (self identified);the Software industries category combines applications fromthe Computer Software,
Computer Services,and Security Products and Services industries (self identified);Other combines applications
f
romall other industries including Energy,Healthcare,Pharmaceuticals,Media and Entertainment,Computer
Hardware,Manufacturing,Education,Aerospace and Defense and Telecommunications (self identified).
Finance Industry Sub-segment Analysis
In the first State of Software Security report earlier this year,the category of Financial Applications submitted by
any organization was found to have more acceptable security on first submission than applications overall.We were
subsequently asked if this finding held true for financial applications submitted by banks,insurance,and financial
services companies.The assumption was that businesses in the financial industry with the greatest potential to be
targeted by hackers would have even better application security.In this report we examined applications frombanks,
insurance,and financial services companies in particular.The news is both encouraging and sobering.
As indicated in Figure 15,banks (81%),insurance (76%),and financial
services (84%) organizations have among the best rawsecurity quality
scores on first submission of all applications (100%is the best score
that an application can achieve).This indicates that security and devel-
opment teams in these organizations have taken proactive steps to
improve security quality.
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
25
Banks,insurance,and financial
services companies have
among the best rawsecurity
quality scores.
VERACODESECURITYQUALITYSCORE
100
90
80
70
60
50
40
30
Bank
Financial Services
Insurance
Veracode Mean RawScore by Financial Sub-segment
Figure 15:Veracode Mean RawScore by Financial Sub-segment
However,when business criticality of the applications is taken into consideration these organizations fare no better
than all applications,with 56%found unacceptable on first submission as shown in Figure 16.Not surprisingly,
this lower level of acceptability when compared to the high rawscores reflects the high level of business criticality
assigned to these sensitive applications.Worse,more than 60%of banking and insurance applications were
unacceptable on first submission,while Financial Services companies were modestly but significantly better than
applications overall at 54%unacceptable.
Among the most encouraging of all findings related to Financial Industry applications is the striking reduction by
Financial Services companies in the most persistent and numerous type of vulnerability,Cross-site Scripting.In Table 7
the type of vulnerabilities found in Banking and Insurance are generally the same as in all applications,with Cross-site
Scripting making up over 70%of all vulnerabilities.Its pervasiveness continues in these companies and applications in
general despite its widely publicized role in breached applications.By comparison,Cross-site Scripting vulnerabilities in
Financial Services companies (credit cards,payment processors,brokerages,etc.) were only 33%of all vulnerabilities.
This significant difference is evidence of both the concern Financial Services companies have following sensational
headlines on past breaches and the rapid improvement submitting companies made as they educated development
teams and expanded the number of applications tested.The lesson for all companies is that rapid improvement in
secure development is possible with training and by applying the lessons learned frominitial independent security
verification projects to the larger application portfolio.
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
26
10%0% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Not Acceptable
46% 54%
35% 65%
4
4% 56%
37% 63%
O
verall
I
nsurance
Financial Services
Bank
Acceptable
Financial Sub-segment Performance on First Submission
(Adjusted for Business Criticality)
Figure 16:Financial Sub-segment Performance on First Submission
(Adjusted for Business Criticality)
Distribution of Application Security Performance by Business Criticality
Veracode’s risk adjusted benchmark applies a sliding scale,requiring applications of higher business criticality
(assurance levels) to have a higher degree of security quality (Refer to addendumfor detailed methodology).This
pragmatic approach allows organizations to optimize their remediation effort and expense intelligently across their
portfolio rather than spend excessively to bring less business critical applications up to the same standard as highly
critical applications.
We investigated the performance of applications upon initial submissions relative to their business criticality.While
only 32%of applications designated as “Very High” business criticality were deemed to have acceptable perform-
ance on initial submission,this still marks a modest improvement since our last report.However,results from
applications of “High” business criticality were somewhat worse with only 38%acceptable upon initial submission.
Performance on initial submission of the most mission-critical applications in an organization remains poor.
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
27
Insurance Bank Financial Services
Vulnerability Distribution by Financial Sub-segment
Cross-site Scripting (XSS) 71%
Information Leakage 10%
CRLF Leakage 6%
SQL Injection 6%
Cryptographic Issues 3%
Encapsulation 1%
Directory Traversal 1%
Time and State 1%
Credentials Mgmt <1%
Insufficient Input Validation <1%
API Abuse <1%
Race Conditions 0%
Other 0%
OS Command Injection 0%
Authentication Issues 0%
Cross-site Scripting (XSS) 72%
Information Leakage 8%
Cryptographic Issues 4%
Directory Traversal 3%
Buffer Overflow 3%
SQL Injection 2%
Error Handling 1%
CRLF Leakage 1%
Time and State 1%
Encapsulation 1%
Potential Backdoor <1%
Credentials Mgmt <1%
Server Configuration <1%
Insufficient Input Validation <1%
Buffer Mgmt Erros <1%
Cross-site Scripting (XSS) 33%
Information Leakage 26%
Cryptographic Issues 11%
CRLF Injection 8%
Directory Traversal 6%
Buffer Overflow 4%
SQL Injection 4%
Time and State 2%
Potential Backdoor 1%
Insufficient Input Validation 1%
Credentials Mgmt 1%
Error Handling <1%
Encapsulation <1%
Buffer Mgmt Errors <1%
API Abuse <1%
Table 7:Vulnerability Distribution by Financial Sub-segment
We explored the impact of industry on the application performance above,given the potential that some industry
verticals could have more security-aware development teams and more formal development practices.The results
of our analysis in Figure 18 showSoftware-related Industries continue to fare the worst with only 40%found to be
acceptable.Government and Financial continue to maintain the top two spots however,as discussed above,the
criticality of applications factors significantly into Financial application acceptance levels.The data continues to
underscore that all industries have work to do to improve the security performance of their applications.
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
28
All Industries
Finance
Government
Software
Other
10%0% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Acceptable
Not Acceptable
40% 60%
57% 43%
44% 56%
43% 57%
43% 57%
Application Performance by Industry on First Submission
(Adjusted for Business Criticality)
Figure 18:Application Performance by Industry on First Submission
(Adjusted for Business Criticality)
1
0%0% 20% 30% 40% 50% 60% 70% 80% 90% 100%
5
8% 42%
38% 62%
3
2% 68%
V
ery High
H
igh
M
edium
Application Performance by Business Criticality on First Submission
F
igure 17:Application Performance by Business Criticality on First Submission
Application Threat Space Trends
One of the trends we have seen within organizations that have more mature application security processes is an
understanding of the top vulnerability categories that are being actively exploited and are well known even outside
of application security specialists.The first category where we sawthis happen was SQL Injection and nowwe are
seeing this trend with Cross-site Scripting (XSS).Lower prevalence of SQL Injection and XSS is a good indicator that
an organization has supplied application security training to their developers and/or application security processes
are integrated into the software development lifecycle.
Another trend we are seeing is organizations are starting to performstatic analysis on web applications in addition
to dynamic analysis.Dynamic analysis was the first automated application security testing technology available and
within the web application category it has achieved significant adoption.Noworganizations that have traditionally
performed only dynamic analysis on web apps are starting to add static analysis and are seeing the benefits of a
complementary testing technique.
2010 was the year that smartphones that enabled easy mobile app installation reached critical mass.There were
over 70 Million Blackberry,iPhone,and Android devices sold in 2009 and likely much greater than that sold in 2010.
Attackers are starting to take notice that there are vulnerabilities in the software running on these devices such as
the PDF vulnerability on iOS 4.0 that allowed jailbreaking right over the web.There are also ample opportunities to
sneak malicious functionality into mobile apps.We sawan iPhone flashlight app that had Apple forbidden tethering
functionality,an Android game that sent the phone’s GPS location to an attacker,and the BBC showed that even one
of their technology reporters could write his own Trojan spyware game.
The mobile platforms of 2010 feel like the Windows platformdid in 1999.
Vulnerable software and malicious spyware started a steep rise around
that time on the Windows platform.Major changes in the way software
is developed,tested,and distributed will be needed to prevent the 2010s
frombeing the decade of mobile insecurity.
Within the past fewmonths,backdoors have been in the headlines yet
again,with the discovery of a wormtargeting a SCADA product written
by Siemens.The backdoor exploited by the wormwas a hard-coded
default password that had been known publicly for over two years but was never patched.At a time when critical
infrastructure systems are being widely acknowledged as a weak link in our national defense,SCADA software and
similar products lacking robust security design should be scrutinized more carefully than ever for common coding
errors as well as malicious backdoors.
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
29
Major changes in the way
software is developed,
tested,and distributed will
be needed to prevent the
2010s frombeing the decade
of mobile insecurity.
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
30
We have also seen developments in howsoftware companies interact with the vulnerability research community.
Google and Mozilla increased bounty payments to over $3,000 per serious bug for researchers who report vulnerabili-
ties without releasing details to the public.It’s likely that over time,others will introduce similar programs as one facet
of a proactive product security strategy.Another noteworthy development was a subtle change to the TippingPoint
Zero Day Initiative (ZDI),a programthat compensates researchers for security vulnerabilities and then engages with
the software vendors on the reporter’s behalf.In response to a growing backlog of high-risk vulnerabilities being
ignored by vendors,sometimes for years,ZDI updated its disclosure policy to give vendors six months to produce
a fix before technical details are released.This puts mild pressure on the vendor to take action and ultimately helps
enterprises and consumers better understand and quantify the risks introduced by vulnerable third-party software.
Another platformtrend that is impacting application security is the
move to cloud based applications.We are seeing an uptick on the
percentage of applications we review,especially for third-parties,that
are applications deployed in the cloud.With nearly 60%of all third-
party assessment requests targeting applications identified as cloud
or as having a cloud option (cloud+deployed) it appears that many
customers are more concerned about the security of software they
are using that is deployed on a cloud platformrather than purchased
and deployed on premise.
Overall,it has been a good year for application security awareness.More organizations are getting up to speed on
static analysis that had relied previously only on dynamic analysis and the awareness and remediation of common
vulnerability categories such as SQL injection and XSS is on the rise in mature organizations.On the other hand,
while software on mature platforms,such as on-premise Windows and Unix,get more security testing,software
on newplatforms such as mobile and cloud are barely getting started.
Many customers are more
concerned about the security
of software they are using that
is deployed on a cloud platform
rather than purchased and
deployed on premise.
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
31
Addendum
Methodology
A
bout Veracode’s Risk Adjusted Verification Methodology
The Veracode SecurityReviewuses static and dynamic analysis (for web applications) to inspect executables and
identify security vulnerabilities in applications.Using both static and dynamic analysis helps reduce false negatives
and detect a broader range of security vulnerabilities.The static binary analysis engine creates a model of the data
and control flowof the binary executable;the model is then verified for security vulnerabilities using a set of auto-
mated security scans.Dynamic analysis uses an automated web scanning technique to detect security vulnerabilities
in a web application at runtime.Once the automated process is complete,a security analyst verifies the output to
ensure the lowest false positive rates in the industry.The end result is an accurate list of security vulnerabilities for
the classes of automated scans applied to the application.
About Software Assurance Levels
The foundation of the Veracode rating systemis the concept that higher assurance applications require higher
security quality scores to be acceptable risks.Lower assurance applications can tolerate lower security quality.The
assurance level is dictated by the typical deployed environment and the value of data used by the application.Factors
that determine assurance level include reputation damage,financial loss,operational risk,sensitive information
disclosure,personal safety,and legal violations.
About the Data Set
The data represents 2,922 applications submitted for analysis by large and small companies,commercial software
providers,open source projects,and software outsourcers.An application was counted only once even if it was
submitted multiple times as vulnerabilities were remediated and newversions uploaded.The report contains findings
about applications that were subjected to static,dynamic,or manual analysis through the Veracode SecurityReview
®
Platform.The report considers data that was provided by Veracode’s customers (application portfolio information
such as assurance level,industry,application origin) and information that was calculated or derived in the course
of Veracode’s analysis (application size,application compiler and platform,types of vulnerabilities,Veracode rating).
In any study of this size there is a risk that sampling issues will arise because of the nature of the way the data was
collected.For instance,it should be kept in mind that all the applications in this study came fromorganizations that
were motivated enough about application security to engage Veracode for an independent assessment of software
risk.Care has been taken to only present comparisons where a statistically significant sample size was present
About the Findings
Unless otherwise stated,all comparisons are made on the basis of the count of unique application builds submitted
and rated.
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
32
Assurance Level Definitions
Veracode’s Business Criticality designations are based on the Assurance Level standard developed by NIST,as
detailed below:
Very High (AL5)
This is typically an application where the safety of life or limb is dependent on the system;it is mission critical the appli-
cation maintain 100%availability for the long termviability of the project or business.Examples are control software for
industrial,transportation or medical equipment or critical business systems such as financial trading systems.
High (AL4)
This is typically an important multi-user business application reachable fromthe Internet and is critical that the appli-
cation maintain high availability to accomplish its mission.Exploitation of high assurance applications cause serious
brand damage and business/financial loss and could lead to long termbusiness impact.Exploitation is a result of a
breach in any two impact categories of confidentiality,integrity and availability of the application.
Medium(AL3)
This is typically a multi-user application connected to the Internet or any systemthat processes financial or private cus-
tomer information.Exploitation of mediumassurance applications typically result in material business impact resulting
in some financial loss,brand damage or business liability.Exploitation is a result of a breach in confidentiality,integrity
or availability of the application.An example is a financial services company’s internal 401K management system.
Low(AL2)
This is typically an internal only application that requires lowlevels of application security such as authentication to pro-
tect access to non-critical business information and prevent IT disruptions.Exploitation of lowassurance applications
may lead to minor levels of inconvenience,distress or IT disruption.An example internal systemis a conference room
reservation or business card order system.
Very Low(AL1)
Applications that have no material business impact should its confidentiality,data integrity and availability be affected.
Code security analysis is not required for this assurance level and security spending should be directed to other
higher level assurance applications.
Table 8:Business Criticality Descriptions
Source:U.S.Government.OMB MemorandumM-04-04;NIST FIPS Pub.199
Business Criticality
Very High
High
Medium
Low
Very Low
Description
Mission critical for business/safety of life and limb on the line
Exploitation causes serious brand damage and financial loss with long termbusiness impact
Applications connected to the Internet that process financial or private customer information
Typically internal applications with non-critical business impact
Applications with no material business impact
V
ERACODE STATE OF SOFTWARE SECURITY REPORT:VOLUME 2
ABOUT VERACODE
Veracode is the world’s leader in cloud-based application risk management.Veracode
SecurityReviewis the industry’s first solution to use patented binary code analysis,dynamic
web assessments,and partner or Veracode delivered manual penetration testing,combined
with developer e-learning and access to open source security ratings to independently
assess and manage application risk across internally developed applications and third-party
software without exposing a company’s source code.Delivered as a cloud-based service,
Veracode provides the simplest,most complete,and most accurate way to implement
security best practices,reduce operational cost and comply with internal security policies
or external standards such as OWASP Top 10,CWE/SANS Top 25 and PCI.
Veracode,Inc.
4 Van de Graaff Drive
Burlington,MA 01803
Tel +1.781.425.6040
Fax +1.781.425.6039
www.veracode.com
© 2010 Veracode,Inc.
All rights reserved.
SSSR/0910