Business Case for Test Automation - SiliconIndia

cabbagepatchtapeInternet and Web Development

Feb 5, 2013 (5 years and 4 months ago)



Company Confidential

Business Case for
Test Automation


Chief Technology Officer

ITC Infotech India Limited

Business Case for
Test Automation


Company Confidential


Test Automation

Myths and Reality

Return on Investments

Key take outs and summary


Company Confidential

Test Automation

Myths and Reality


Company Confidential

Test Automation Myths

Test Automation is simple, every tester can do it

This myth is promoted by the tool sales people. They are trying
to promote the following test automation process:

Record the script

Enhance the script by adding functions and data driving

Run the scripts

Report results

Under the influence of this myth a QA manager can proudly
report: All our testers are developing test automation.


Company Confidential


Test automation is a software development task

Automation should be designed, developed and tested

You need to have some kind of a programming background to
implement test automation. Test Automation is not as complex as
C++/C#/Java development.

Test automation standards should be developed

Automated test components are assets that should be treated like
application source code, unit tested, integration tested and
performance aspects considered.


Company Confidential

Myth #2

Commercial test tools are expensive

Under the influence of this myth some companies, especially
the small ones:

Try to develop their own test automation tools

Use scripting languages like Perl and Ruby

Use shareware test tools

Do not consider test automation at all


Company Confidential


Commercial tools are cheap

Per seat license for most expensive automation tool is $8K

This tool will be used for 5 years.

Maintenance/Support fees are 20% of tool cost or $1,800 per year

The cost of this tool is $8K/5+$1,800 = $3,100 per year

The automation developer cost with overhead is $100K per year

The cost of this tool is just 3% of the person who uses it, but
productivity gain can be very significant


Company Confidential

Commercial Tool Benefits:

Customer support. Many of the open source tools come and go with little to
no support

Most commercial tools are constantly being updated as technologies

Most commercial tools usually have more functionality (QTP can test
various GUI applications: Web,
, Java, VB, C/C++, PowerBuilder, etc.

Web only)

Commercial tools usually have a large community of users, which translates
into better availability of qualified resources

Commercial tools require less advanced programming

More test automation frameworks are available for commercial tools

Commercial tools are integrated with Test Management tools which make
reporting and execution much simpler.


Company Confidential

Return On Investment (ROI)


Company Confidential

Classic ROI Calculation


Automation Cost = Price Of HW + Price of SW + Development Cost +
Maintenance Cost + Execution Cost

Manual Testing Cost = Development Cost + Maintenance Cost +
Execution Cost

ROI = (Manual Testing Cost

Automation Cost)/Automation Cost

Looks right, doesn’t it?


Company Confidential

Problems with Classic ROI Calculation

You can’t compare Automated Testing and Manual Testing. They
are not the same and they provide different information about the

You can’t compare cost of multiple execution of automated tests
vs. manual tests. You would never dream of executing that many
test cases manually.


Company Confidential

Automation Real ROI

ROI value IS NOT the value of Automation vs. Cost of executing
these tests manually

Automation ROI value IS the benefit of this type of testing, and it
can be:

Reducing Time to Market / Execution Time

Increased Test Efficiency (Productivity)

Increased Test Effectiveness & test coverage

Decrease in test defects escaping to production

Improved test repeatability


Company Confidential

Reduced Time to Market

Can get a greater market share

Makes people available to work on other projects

Higher margins, if no competitive products are currently available


Company Confidential

Productivity and Effectiveness

More testing gets done faster, increasing the odds of finding defects

Defects found early have better chances to be fixed

Manual Testers can concentrate on clever ways to finding defects,
instead of typing test inputs and verify output.

About 7% of bug fixes create new bugs, sometimes in already tested
parts of the system. With automation you can rerun tests for those
modules. This almost never happened when testing done manually.


Company Confidential

Manual VS Automated Testing Example

Test Cycle

Manual Execution Time (days)

Automation Execution Time (days)
















There is a suite of manual test cases that have been deemed as
candidates for automation. When run manually, the execution can start
immediately (ITC = 0 days). It takes the manual testing team seven
business days to execute all of the test cases (TEC = 7 days). The Total
Testing Time will always be a linear factor of seven days.

If the test cases are to be automated it will take 30 business days to
complete (ITC = 30 days). Once the test scripts have been written it
takes half of a business day to complete the tests with the necessary
hardware resources (TEC = 0.5 days).


Company Confidential

ROI summary

Each project requires different types of automation

there is no easy
formula available to calculate ROI

Performing ROI calculation can help to determine upfront what type
of automation, what level of skills, what tools will be required.


Company Confidential

Thank You