Test Impact Analysis

November 13, 2017 Software Testing Magazine Articles, Knowledge 0

In the days of DevOps supported by approaches like continuous deployment, the concepts of continuous testing and test automation are essential to support the speed needed for delivering quickly solutions (and hopefully value) to the users. Some of the big questions in the software testing community are “How much should we automate our tests?” and “What tests should we run?”. The technique of Test Impact Analysis helps to answer to this question.

In the article “The Rise of Test Impact Analysis”, Paul Hammant defines Test Impact Analysis (TIA) as “a modern way of speeding up the test automation phase of a build.” In his article “Accelerated Continuous Testing with Test Impact Analysis”, Pratap Lakshman write that “TIA is all about incremental validation by automatic test selection. For a given code commit entering the pipeline TIA will select and run only the relevant tests required to validate that commit.” This technique helps software QA people to solve the following issue: they either verify too many tests (automated or manual), or too few, which results either in lower productivity or lower quality.

The basic principle of Test Impact Analysis is simple: you perform an analysis of the source code to determine which tests you should run after a change in the production code. The tool used for TIA, like Microsoft’s Visual Studio, examines the baseline data to identify which tests call into the modified methods. The software developers and the testers should understand what are the tests that get automatically selected and what are the policies for test selection. It is also recommended selecting previously failing tests. Then, the Test Impact engine updates its baseline as tests are successfully executed.