Faster software delivery through approaches like DevOps or Continuous Delivery impacts software testing practices. This article summarizes the steps that software testing teams can undertake in order to adapt to the fast development process in projects with Continuous Integration/Continuous Delivery (CI/CD).
Author: Andrei Mikhailau, ScienceSoft USA Corporation
Organizations turn to DevOps and adopt Continuous Integration/Continuous Delivery (CI/CD) pipelines to speed up time to market and release high-quality software more frequently. The adoption of CI/CD inevitably affects software testing, as the testing process has to adapt to frequent code commits and shorter delivery cycles. However, according to the World Quality Report 2019, 43% of respondents name their software testing process to be too slow and unable to keep up with the pace of CI/CD-driven projects. In this article, we shed light on two approaches that help test teams speed up the testing process and remain efficient in projects with CI/CD pipelines.
CI/CD implications on software testing
The adoption of CI/CD pipelines affects software testing in two key ways. First, testing activities start at the initial stages of the software delivery life cycle, often as early as at the requirements’ analysis stage. Second, testing activities become nimbler. To satisfy these requirements, the testing process can be modified by:
- Building up an automated testing suit.
- Adopting test automation practices.
Before we have an in-depth look at both activities, let’s clarify the notions.
Automated testing means having software to execute test scripts covering functional test cases. Test automation, in its turn, is a broader concept, which comprises the automation of both testing as such and testing-related activities, for instance, preparing test data and provisioning test environments.
Automated testing
In projects with CI/CD, all relevant testing types can be automated, but it is advisable to start with automating a regression test suit. The reason is that the number of times a regression test suit is executed rises proportionally to the number of software releases, which are getting ever more frequent.
Frequently executed functional test cases, for instance, those used for validating the same application functionality with diverse test data, are considered good candidates for automation as well. At the same time, testing in CI/CD does not presuppose eliminating manual testing efforts. Released from executing monotonous activities, test teams can concentrate on non-trivial use cases, e.g., those requiring deep domain expertise.
Also, automated tests start already at the unit level in CI/CD projects. Thus, development teams become more involved in the software validation process while test teams take over at the API and user interface (UI) levels. And for the sake of further speed and agility, a larger share of automated tests should be executed at the API, rather than at the UI level.
Test automation
One of the latest trends in software development, test automation allows setting up continuous testing pipelines, where the sequence of automated tests is executed without manual involvement for each new code commit. Test automation requires collaboration among test, development and operations teams on the following activities:
Automated environments provisioning
A fast pace of projects with CI/CD requires the right test environments to be provisioned quickly, which can be achieved with pre-configured tools (Chef, Puppet, and others). These tools can be applied for automating environment request, plan, design and build provisioning, as well as environment support activities, including configurations maintenance and code migrations. Automated provisioning and management of environments bring such benefits as consistency, accuracy and the elimination of errors arising due to manual configuration.
Automated test data provisioning
In projects with CI/CD, the increasing number of test runs calls for fast on-demand provisioning of test data, which can be achieved by employing test data generation tools like DTM data generator, EMS data generator, and others.
Such tools allow test engineers to request, provision and manipulate their own data sets without the help from development and operations test teams and provide APIs for the integration with automated testing frameworks and CI/CD tools.
Triggering the execution of automated test cases and generating test reports
Test teams can cooperate with the operations teams on setting up continuous integration jobs that automatically trigger the execution of automated test cases whenever a new build is performed. To enable that, operations teams set up the environment comprising a continuous integration server, a software project management tool, a framework for automated testing and a test executor, and make these components communicate in such a way that every new committed build is automatically tested. Operations teams can specify when and on which application versions and browsers the tests will run. Moreover, a CI/CD tool can be set to automatically create test reports as soon as the test run is completed.
Summary
In projects with CI/CD, software testers need to adapt to a higher frequency and a faster speed of software delivery life cycles. To keep up with the development process, test teams should employ automated testing and test automation. Combined, both practices allow building continuous testing pipelines, i.e., automatically executing tests on every new software change, which enables frequent releases of quality software.
About the Author
Andrei Mikhailau is Software Testing Director at ScienceSoft headquartered in McKinney, Texas. Andrei has 15+ years of experience in a range of SDLC processes, from testing and BA to and project management and QA. Throughout his career Andrei has successfully worked under various methodologies: RUP, MSF, XP, Agile and CMMI III/IV levels. He also has a proven expertise in developing a quality testing process under these methodologies.
Contact SoftwareTestingMagazine.com if you want to contribute quality software testing and software quality assurance content on this website.