Business Case for Test Automation - SiliconIndia

cabbagepatchtapeInternet and Web Development

Feb 5, 2013 (4 years and 6 months ago)

172 views

©

Company Confidential


www.itcinfotech.com

Business Case for
Test Automation

S.Janardhanan

Chief Technology Officer

ITC Infotech India Limited

Business Case for
Test Automation

©

Company Confidential


www.itcinfotech.com

Agenda


Test Automation


Myths and Reality


Return on Investments


Key take outs and summary


©

Company Confidential


www.itcinfotech.com

Test Automation


Myths and Reality


©

Company Confidential


www.itcinfotech.com

Test Automation Myths


Test Automation is simple, every tester can do it



This myth is promoted by the tool sales people. They are trying
to promote the following test automation process:



Record the script


Enhance the script by adding functions and data driving


Run the scripts


Report results



Under the influence of this myth a QA manager can proudly
report: All our testers are developing test automation.


©

Company Confidential


www.itcinfotech.com

Reality
-

Test automation is a software development task


Automation should be designed, developed and tested



You need to have some kind of a programming background to
implement test automation. Test Automation is not as complex as
C++/C#/Java development.



Test automation standards should be developed



Automated test components are assets that should be treated like
application source code, unit tested, integration tested and
performance aspects considered.


©

Company Confidential


www.itcinfotech.com

Myth #2


Commercial test tools are expensive


Under the influence of this myth some companies, especially
the small ones:



Try to develop their own test automation tools


Use scripting languages like Perl and Ruby


Use shareware test tools


Do not consider test automation at all

©

Company Confidential


www.itcinfotech.com

Reality


Commercial tools are cheap


Per seat license for most expensive automation tool is $8K



This tool will be used for 5 years.



Maintenance/Support fees are 20% of tool cost or $1,800 per year



The cost of this tool is $8K/5+$1,800 = $3,100 per year



The automation developer cost with overhead is $100K per year



The cost of this tool is just 3% of the person who uses it, but
productivity gain can be very significant

©

Company Confidential


www.itcinfotech.com

Commercial Tool Benefits:


Customer support. Many of the open source tools come and go with little to
no support



Most commercial tools are constantly being updated as technologies
change



Most commercial tools usually have more functionality (QTP can test
various GUI applications: Web,
.Net
, Java, VB, C/C++, PowerBuilder, etc.
vs. WATIR


Web only)



Commercial tools usually have a large community of users, which translates
into better availability of qualified resources



Commercial tools require less advanced programming



More test automation frameworks are available for commercial tools



Commercial tools are integrated with Test Management tools which make
reporting and execution much simpler.

©

Company Confidential


www.itcinfotech.com

Return On Investment (ROI)


©

Company Confidential


www.itcinfotech.com

Classic ROI Calculation

ROI = BENEFIT/COST


Automation Cost = Price Of HW + Price of SW + Development Cost +
Maintenance Cost + Execution Cost


Manual Testing Cost = Development Cost + Maintenance Cost +
Execution Cost


ROI = (Manual Testing Cost
-

Automation Cost)/Automation Cost


Looks right, doesn’t it?

©

Company Confidential


www.itcinfotech.com

Problems with Classic ROI Calculation


You can’t compare Automated Testing and Manual Testing. They
are not the same and they provide different information about the
AUT.



You can’t compare cost of multiple execution of automated tests
vs. manual tests. You would never dream of executing that many
test cases manually.

©

Company Confidential


www.itcinfotech.com

Automation Real ROI


ROI value IS NOT the value of Automation vs. Cost of executing
these tests manually



Automation ROI value IS the benefit of this type of testing, and it
can be:



Reducing Time to Market / Execution Time


Increased Test Efficiency (Productivity)


Increased Test Effectiveness & test coverage


Decrease in test defects escaping to production


Improved test repeatability


©

Company Confidential


www.itcinfotech.com

Reduced Time to Market


Can get a greater market share



Makes people available to work on other projects



Higher margins, if no competitive products are currently available



©

Company Confidential


www.itcinfotech.com

Productivity and Effectiveness


More testing gets done faster, increasing the odds of finding defects


Defects found early have better chances to be fixed


Manual Testers can concentrate on clever ways to finding defects,
instead of typing test inputs and verify output.


About 7% of bug fixes create new bugs, sometimes in already tested
parts of the system. With automation you can rerun tests for those
modules. This almost never happened when testing done manually.


©

Company Confidential


www.itcinfotech.com

Manual VS Automated Testing Example

Test Cycle

Manual Execution Time (days)

Automation Execution Time (days)

1

7

30.5

3

21

31.5

5

35

32.5

7

49

33.5

9

63

34.5



There is a suite of manual test cases that have been deemed as
candidates for automation. When run manually, the execution can start
immediately (ITC = 0 days). It takes the manual testing team seven
business days to execute all of the test cases (TEC = 7 days). The Total
Testing Time will always be a linear factor of seven days.


If the test cases are to be automated it will take 30 business days to
complete (ITC = 30 days). Once the test scripts have been written it
takes half of a business day to complete the tests with the necessary
hardware resources (TEC = 0.5 days).


©

Company Confidential


www.itcinfotech.com

ROI summary


Each project requires different types of automation
-

there is no easy
formula available to calculate ROI



Performing ROI calculation can help to determine upfront what type
of automation, what level of skills, what tools will be required.

©

Company Confidential


www.itcinfotech.com

Thank You