Can Automated Testing for ServiceNow® Save You $1,000,000?
By now, most people understand that manual testing is less ideal than automated testing, but have you ever really crunched the numbers?
Have no fear. In this article, we will show an easy method for calculating just how much additional money you are spending on manual testing, and help you determine how much money you can potentially save by getting automated testing up and running in your organization. We will also take a look at the deeper ROI implications and benefits of automating your ServiceNow testing correctly.
Our example organization performs two version upgrades each year. Based on their work load and their employee headcount, they are able to perform 1 build per month consisting of 600 test cases. They typically average 10 cases per hour and pay their QA testers $80,000 per year, which we rounded to $38.50 per hour. (Note – This number only accounts for salary, and does not factor in fully burdened costs, such as insurance, benefits, and so on.) Finally, the percentage of time that can be fully dedicated to testing is a little less than half of their duties; which we average to about 45%.
STEP 1: Calculating Time Investment for Each Test Plan
From here, we can calculate how many hours are required for each regression. For a typical company, it is likely that employees will be able to manually complete 10 test plans each hour.
At 600 Test Cases, that means the company will need 60 hours to execute the test plan from end to end. In other words, a team of 2 testers will take almost 4 FTE (Full-Time Equivalent) days for each full test plan execution.
STEP 2: Calculating Cost for Each Test Plan
Now that we have the time for each test plan, the salary expense, and the percentage needed to dedicate to testing, we can calculate the cost incurred each time that a test plan is run.
Over $5,000 to run a test is obviously a serious consideration, no matter how large your company is. With that in mind, it is no wonder some companies feel the need to only test once a month.
STEP 3: Calculating Savings
Finally, we can easily find the total yearly cost by multiplying the cost per test plan with the sum of the yearly upgrade and the yearly number of builds.
As you can see, a typical company could spend almost $72,000 to complete a process that could be automated instead. That number can easily increase with even minor variations. If they were working with 800 test cases instead of 600, the total cost per year would increase to over $95,000. Or, if the number of builds per year increased to 18 instead of 12, the total would be greater than $100,000 per year.
So, How Much Would The Same Work Cost with Automated Testing?
Scratching the Surface of ROI:
Other than the cost of the solution and the implementation, automated testing would reduce your expenses by the final number calculated by using the formula above. In other words, that is the calculation of how much time and effort you can save with Cerna Solutions’ CapIO – Automated Testing for ServiceNow.
In our first example, the company would save almost $72,000 that year, minus the cost of the solution and implementation. However, the next year, that savings increases even more as the initial costs no longer apply, leading to substantial long term savings for the organization.
Big Picture ROI:
The formula above only calculates what an organization had been able to do while still manual testing. In other words, based on their previously limited resources, this is the amount of quality assurance that the company can reasonably pay for to perform the minimal coverage for the status quo of manual testing. It does not calculate what the company would need to spend in order conduct the amount of testing that would sufficiently cover their development needs.
Let’s take a look at our formula set one more time, but now let’s imagine what it would cost for a company to test as much as they NEED TO, instead of how much they can AFFORD TO.
Based on our formula, if the same company wanted to test weekly instead of monthly, this would cost $277,182 more than it would with an automated solution.
That’s a significant number to consider, but in reality, testing weekly may not be satisfactory in an environment where development is done on a daily basis. If you subtract weekends and holidays, there are about 250 working days in the year. If you increase your total builds to 250 throughout the year, an automated testing solution could save you $1,293,516 every year based on our formula above.
This calculation is only used to identify the basic costs of a company’s application testing operation. However, it does not show the additional savings that come from other key benefits of automated testing such as finding defects earlier in the development process and preventing defects from reaching production.
This is highlighted in the 1:10:100 rule of quality assurance, which states that fixing defects in development is possibly 100 times quicker and less expensive than fixing a defect in production. While the magnitude may vary for each company, it is indisputable that defects which are not fixed in the design or implementation phase are exponentially more of an issue to fix later in production. For companies that use manual testing, those high costs are impossible to avoid without a solution that is capable of testing the areas of the application that were unexpectedly affected by the initial development. In other words, if you don’t know to look for the defect in the first place, you are unlikely to find it until it reaches production.
By utilizing an automated testing system, you are doing more than just saving money on automated testing, you are significantly increasing the value of your testing far beyond anything that could be possible with a manual solution. Our example company would need to spend over 1 million dollars to achieve the same results with a manual solution. Unfortunately for those companies, testing ServiceNow development only once a month would present a significant disadvantage to their operation and dramatically hinder the type of innovation and growth that a powerful platform like ServiceNow should be able to offer.
Getting Started with CapIO
CapIO by Cerna Solutions has allowed companies – much like the one in this example – to turn a week’s worth of testing into a short 45-minute process. As one of the first automated testing solutions available for ServiceNow, it allows users to hit the ground running with over 300 pre-built commands specifically for ServiceNow, as well as additional features such as an action recorder and test creation wizard, to allow for new tests to be built with little-to-no coding required. More importantly, CapIO allows for users to create tests without fear that they will break in future versions and upgrades, saving countless hours of coding and support necessary with other solutions. This not only saves you time and money, but allows you to focus on growing and innovating your ServiceNow practice to meet its full potential.