Antony Edwards - 24 September 2019
Automated Testing is a requirement if your organization is going to do DevOps at scale. Testing is too vast and complex to be done by humans without support.
There is always a fine line between managing the costs of an Automated Testing Team and ensuring that the organization is seeing a return on its investment. Automated Testing, regardless of if you are using a licensed or open-source testing tool, has hidden costs. The license costs are just the most obvious.Understanding the hidden costs of Automated Testing and how to mitigate their impacts is key to a successful Automated Testing Program.
- Tool Infrastructure Costs: Licensed and Open Source Automation Tools have infrastructure costs. Device, Server, Database, and Cloud costs must be a factor when evaluating which type of Automation Tool to use
- The processes used to create, prepare, execute, and maintain Automated Testing Scripts: All Automated Testing Tools need help to be successful. In some cases, it will be as simple as access to the application under test. However, for most, there are costs in building supporting frameworks, libraries, and supporting artifacts to keep the tool moving. Here, the maintenance cost of Automated Testing is the primary factor in driving expenses upwards. For example, each time the application under test changes, the core of the Automated Test (in many cases, the Object Repository), the Automated Script itself, and the Automated Testing Framework sitting on top need to be changed. That is up to three layers of complexity for every application change. It is equivalent to Automation Engineers having to do about three times more work for each function change in the application.
- The people who perform, maintain, and execute the Automated Testing: Automated Testing Tools have become more complicated, rather than less, over the years. Complex Automated Testing Tools require hiring either Automated Testing Engineers or SDETs (Software Development Engineers in Test). Regardless if they are onshore or offshore, these resources are more expensive than Non-Technical Testers or Industry-knowledgeable Testers. The tradeoff is that by spending more budget on costly Automated Testing Developers, you are losing Domain Expertise with having to let go or re-purpose Non-Technical Testers.
Controlling Costs in Automated Testing comes down to tracking the Total Cost of Ownership (in People, Processes, and Tools), and then looking for ways to keep those costs in line with Organizational Priorities. Additionally, look for Automated Testing Tools that:
- Do not require multiple agents to be installed, can connect to Cloud Environments (even being Cloud-hosted is a plus), and can work out of the box with no extra add-ons, frameworks, or harnesses
- Technical and Non-Technical Testers can use it. Automation Engineers are expensive. Having an Automated Testing Tool that doesn’t require in-depth technical knowledge can ultimately lower costs in the end
- If you are using a third party to do your Automated Testing, make sure that there are no proprietary frameworks, harnesses, or programs being used to support the Automated Testing Tool. If you are not getting the Return on Investment promised, then chances are more time is being spent getting the Automation to work instead of the Automated Testing Tool working for you.
Take the first step in controlling your Automation Costs by downloading our Fact Sheet. The Fact Sheet compares how Eggplant handles the hidden costs in Automation vs. the Automated Testing Tool you are using now.