Jaspar Casey - 15 July 2021
DevOps and its security-minded sibling, DevSecOps, have emerged as a set of practices to shorten development lifecycles and improve software quality. As more teams adopt practices like two-week sprints and continuous integration/continuous deployment (CI/CD), software cycles are indeed getting shorter. However, with testing considered the biggest bottleneck by developers, it’s clear that the demands of testing software thoroughly are somewhat at odds with the shortened release cycles demanded by DevOps. Test automation is the answer to this friction.
The aim of this article is to explain how test automation is essential to realizing and operationalizing the potential of these transformative practices.
From the Phoenix Project to the fighter jet
While DevOps practices have been embraced by teams from open-source projects to the world’s biggest companies , what you may not know is that the Department of Defense is also massively pushing for adoption across its vast IT infrastructure and software acquisition efforts.
Given that Eggplant is widely used in the defense and aerospace sectors, we thought it would be helpful to refer to the DoD’s guidance on how test automation is essential to DevSecOps.
Shift Left as defense policy
Shift Left, a term which refers to moving testing earlier in the software development lifecycle, is now policy, with the DoD’s Enterprise DevSecOps 2.0 plan urging developers to shift “both development tests and operational tests left.” This means that development and quality teams need to build tests long before the deployment stage of the cycle. It also suggests, for the sake of efficiency and expediency, that test scripts and models should be adaptable enough to work throughout the process, and do not need to be built from scratch at each discrete stage.
The role of automation in DevSecOps
Automation is an essential enabling technology throughout the DevSecOps cycle. As the DoD suggests in its checklist for adopting a DevSecOps culture, “embrace automation for anything done repeatedly.” When you consider that much of the testing required throughout the software lifecycle is repetitive, it’s clear that test automation is a necessity, an idea reinforced by DoD Instructions 5000.87 and 5000.89. Indeed, it is recommended that “software development testing … and operational test and evaluation will be automated to the maximum extent practicable.” A great example of this in practice is the evaluation phase of the delivery of software development contracts.
Automating the handover process
Automation can help in the evaluation and delivery phases as well. For many software projects in the defense sector, the final step before handover is demonstrating that the software can pass a number of directed tests under the supervision of a government official or other arbiter of project quality. This phase can often last over a week and is as boring as it is necessary. However, with automation, this can be greatly sped up, with the directed test paths programmed as scripts that can drive the application under supervision. This ends up saving days of effort and time, to the great relief of QA managers and inspectors alike.
The value of non-invasive test automation
There are a number of well-established tools in the test automation world, such as Selenium and JUnit. However, because these tools work at the code level, they need access to the underlying code and object properties of any software they are testing.
In contrast, non-invasive testing tools have emerged. Non-invasive implies two aspects:
- No access required to the source code. This means testing at the interface, rather than code, level.
- Two-system testing. This allows tests to be designed and run from a separate machine, which then securely connects to the system under test.
Non-invasive testing allows teams to test software throughout the development lifecycle, from prototyping through to final deployment, without jeopardizing the security or integrity of the system or any of its data. This is particularly valuable in teams practicing DevSecOps, because of the emphasis placed on observability of an application’s performance and behavior.
Eggplant was built from the ground up to achieve precisely this: securely test and automate any piece of software on any device or operating system, regardless of the language it was written in.
To learn more about Eggplant, please get in touch. You can also watch a recent webinar hosted by DevOps.com, where Eggplant’s Jay Epps spoke on a panel about the importance of test automation in continuous delivery.
We also just published an explainer video for test automation in defense and aerospace.