John Bates - 25 September 2017
You can find 28 million apps on Google Play and 22 million in Apple’s App Store. Yet, nearly one in four people who download an app use it only once. Apps are incredibly slow under certain circumstances, don’t work in key parts of the workflow, and have less-than-optimal usability. The app scrap heap is growing because many organizations are still testing to ensure code quality, not a superior user experience (UX).
I touched on this in my last blog, explaining how consumerization, the digital experience, DevOps, and IoT have changed a lot about software products in a short amount of time—how they’re architected and produced, who uses them, and those users’ increasing expectations. These changes have also shifted the core requirements for testing.
For example, UX went from not being that important to a top priority. Teams that used to release quarterly are now expected to release continuously. We went from having to test one or two devices to accommodating many platforms and IoT devices.
The current state of testing by the numbers
A study by the analyst firm PAC reveals some interesting data about the state of testing today.
While 86 percent of teams say they’re currently meeting their test objectives, if you dig a bit deeper, only 18 percent say they’re actually meeting customer expectations for quality. This user experience gap is resulting in low user adoption, satisfaction, conversion, and retention.
With agile, DevOps, and continuous delivery reducing dev cycles to weeks, having less time to test highlights the criticality of efficiency and productivity, but 56 percent of teams say they can’t use test automation for a significant part of their testing. This isn’t surprising given the evolution of mobile, dynamic web, IoT, and technology fragmentation. But what is surprising is that teams are trying to tackle these testing changes and challenges with the same testing approaches from 15 years ago.
A new, game-changing approach
At Testplant, we’ve developed a new approach to testing that uses AI, machine learning, and analytics to create amazing digital experiences—and that’s sure to make app dev and test teams’ jobs a lot easier. We call it Digital Automation Intelligence and it’s powered by our integrated suite of award-winning solutions, including Eggplant AI, Eggplant Functional, and Eggplant Performance.
Here’s how our Digital Automation Intelligence approach is different.
Test through the eyes of the user and service APIs. Traditional approaches to test automation look at the low-level code and then try to guess what it means from a UX perspective. Eggplant solutions interact with apps like a real user, analyzing the screen—not the code—using intelligent image recognition and text recognition. Acting as an artificial user is the only way to test the true UX, where the UI is the source of truth. Putting the user back at the center of testing allows you to:
- Focus on user satisfaction and close that user expectation gap.
- Address the automation gap by testing any device and technology just as a user would.
- Tackle the productivity gap by giving your product experts and business analysts a more intuitive, natural way to test.
Continuously test all aspects of the UX. Functionality, performance, usability, and security are all critical elements of the user experience. Make them critical parts of your testing with our Digital Automation Intelligence Suite.
Expand automation beyond test execution. Lots of people talk about test automation but if you look at the entire test process, we’ve really only automated one key area: test execution. Eggplant uses AI and machine learning to auto-generate test cases and optimize test execution. So you can find bugs, achieve broad coverage of the UX, and be confident about coverage when you launch a release. Analytics helps you automate results analysis and root-cause analysis, leading to better productivity, faster time to market, and the ability to keep up with the pace and evolution of DevOps and continuous delivery.
Report test status in terms of the UX. Here’s where predictive analytics comes in, enabling better release decisions and helping you focus testing on the areas that will have the greatest impact on the user. Rather than reporting test status purely in metrics terms (like test-case pass rate and defect density), the goal of Digital Automation Intelligence is to report app quality status in user terms, like shipping now will likely result in 20-percent fewer app freezes and a 15-percent increase in customer conversion.
Take a coherent approach to monitoring and testing. It’s critical to continue to monitor UX in production, especially in a modern, DevOps environment where products move between the lab and production and back again so quickly. And where product, dev, test, and operations teams work closely together. Real world experience has shown us that traditional monitoring approaches can report a response time of two seconds when users were really experiencing eight seconds. True user experience can only be derived by monitoring the end-to-end user experience not just the delay at the Web server or some particular part of the technology stack.
Turn testing into a profit center
With Digital Automation Intelligence, you can transform testing from a compliance function to a profit center. Predict the impact that your release will have on your real users. Boost team productivity and efficiency. Deliver a compelling user experience that drives customer satisfaction and adoption. And avoid the app scrap heap.
I invite you to learn more about Digital Automation Intelligence in person at one of our roadshows in in London, Los Angeles, Dallas, and Philadelphia. If you prefer a virtual experience, request a demo with one of our experienced technical consultants. Or, email eggplantAI@testplant.com to become part of the Eggplant AI early access program.