Pairwise testing is a black box testing techniques that aims to achieve a nice test coverage of an application with a good benefit-cost ratio. This article provides a list of some of the best tools that are available to automate the pairwise testing approach.
Author: Nataliia Syvynska, TestMatick, https://testmatick.com/
The International Software Testing Qualifications Board (ISTQB) defines Pairwise Testing (also named All-Pairs Testing) as « A black-box test design technique in which test cases are designed to execute all possible discrete combinations of each pair of input parameters».
Many software bugs are caused by either a single input parameter or an interaction between pairs of parameters. Bugs involving interactions between more than two parameters are less common and more expensive to discover. Software testing efforts are therefore reaching their limits when you want to explore all possible inputs interactions. Pairwise testing is a combinatorial software testing technique for picking test cases like all-pairs testing is a useful cost-benefit compromise that enables a significant reduction in the number of test cases without drastically compromising functional coverage.
There are two contexts where pairwise testing can be used with great benefits:
- Running the same tests with multiple variations or environments, for instance different browsers or operating systems
- Testing a complex software function with two or more inputs
You can apply test automation to pairwise testing and here is a list of tools and online services that are available to practice the pairwise testing approach:
Allpairs is a command line file based on a Perl script. Using the application requires basic knowledge of console commands:
- work and navigate between directories;
- running the application directly from a directory (including test files).
Procedure:
- provision of a table with the data and the specified parameters;
- the table must be exported to a text file;
- creation of a directory, subdirectory (text files are located);
- possibility to specify a file for processing;
- getting a list of test checks.
Website: https://www.satisfice.com/download/allpairs
PICT (Pairwise Independent Combinatorial Tool) – this application is launched from the command line. The input for the program is a text file with parameters and specified values. When exiting, the program displays the generated test scripts.
Website: https://github.com/microsoft/pict
Pairwiser is an online free service that provides the opportunity to:
- sign in;
- create a test plan;
- download test plans;
- define parameters of the system under test (click on “Define Parameters”);
- set parameters;
- create test scenarios (click on “Generate Tests”);
- get a final list.
Website: https://inductive.no/pairwiser
Pairwise Tool is a service used for all-pairs testing. Action sequences:
- develop a list of input variables;
- possibility to view the number of variations for testing (click on the “Generate Pairwise” button);
- as a result: obtain the list with the enumeration of test checks (click on the “Generate All Combinations” button).
Website: https://pairwise.teremokgames.com/
Pairwise Pict Online is a free service (version of Microsoft Pict) allowing to:
- specify the parameters;
- indicate test values;
- specify the conditions of test scenarios (click on the “Generate” button);
- receiving generated table as a file.
Website: https://pairwise.yuuniworks.com/
Pairwise Generator is a service that can generate a list of test checks. It suggests a sequence of actions for its use:
- specify the name of the test parameters;
- specify test parameters value;
- specify test conditions when generating tests (click on the “Show conditions” and “Generate” buttons);
- automatic loading of the Excel file (with the list of possible scenarios).
Website: https://slothman.dev/pairwise-generator
Pairwise testing is an opportunity to determine the maximum number of bugs with the minimum number of checks. It is also a good way to perform software testing in limited periods.
Pairwise Testing References
Pairwise definition in the ISTQB glossary
Pairwisetesting.org list of tools