Candice Arnold - 17 April 2019
A recent SD Times article suggested that AI is not delivering tangible testing improvements, cautioning “…customers need to realize there’s probably more hype than reality in most of what test solution vendors are saying.” This sentiment reminds us of the unhelpfully polarized “manual-vs-automated test execution” debate that held back testing for 10 years and we’ve only recently moved on from with the realization that there’s a place for both in testing. It would be catastrophic for the testing industry if we went into another 10 year debate of “AI-vs-non-AI” only to realize that there are strong benefits to AI, but it probably doesn’t solve everything.
These sentiments are unhelpful and dangerous to the testing community. Sure, there are exaggerated claims about the use and impact of AI in some products, but there are also fantastic products that demonstrate that the technology can deliver real improvements today. Below are just a few examples of how AI can help testing now—not several years down the road:
- Auto-generation of test scripts: This is not an all or nothing scenario as there are certainly complex test cases for which AI can’t auto-generate all the code. However, the technology can easily auto-generate the code that clicks the buttons on the screen, fills out a form, logs into the app, and other similar functions. At a macro level, this can ease a significant resource burden on human testers. In our experience, we’ve seen over 80 percent of code be reliably auto-generated. Just think about the efficiency improvements this could drive in your organization.
- Test optimization: If you’ve ever used Amazon, then you know that recommendation engine technologies are not that futuristic. AI can be applied to testing in the same way to help companies determine which test is most likely to find a defect based on the risk information you can gather.
- Release impact: Neural networks, combined with test history and data from current test runs, can predict the impact on users of a forthcoming release. For example, will it make customer satisfaction go up or down? Armed with this information, companies can make any schedule adjustments necessary to ensure their users continue to be delighted by their site experience.
- Customer impact: Similarly, neural networks, machine learning and other AI techniques can be used to understand how technical behaviors impact business performance. For example, are slow load times negatively impacting sales? COOK is an example of just one of the organizations we’ve worked with to deliver AI-driven insights that improve site performance and, in turn, increase online revenue.
As a results of Eggplant’s strength in these and other aspects of AI-driven testing, we were named a Leader in The Forrester Wave™ on test automation tools.
So, with all due respect to the opinions expressed in the SD Times piece, I think it’s an unfair categorization of where the industry is with AI that ultimately sells the technology short. Can AI currently generate 100 percent of the code for every complex test case? No. But the technology can automate numerous other critical functions and drive immediate improvements with a bottom-line impact.
For more on how AI is poised to transform testing, check out our webinar on the topic here.