Head to any site in which testers and developers are seeking help on their testing, and you’ll see questions on which tools individuals should adopt, learn and use. For example, I recently read a question from one person who wanted to know whether they should learn about Postman or RestAssured for API automation.
Author: Mark Winteringham
On the surface, this seems like a question that can be easily answered, which was demonstrated in some of the responses to the original poster’s question. Each person replying shared their own experiences and what they felt was their preferred tool. This is helpful to understand the preferences of others but it doesn’t answer the question: What tool do you need to learn about?
What are you hoping to achieve?
Instead of focusing on what tools can and cannot achieve, whether it’s for API automation or other situations, we should analyse why we need specific tools for API automation. The challenge we have when it comes to tool selection is avoiding the law of instruments trap:
“If all you have is a hammer then everything looks like a nail”
Meaning that if we ignore why we need tooling in the first place and just opt for the most popular tools. We risk selecting tools that at best aren’t the right fit for us and at worst will actively disrupt our work and waste our time.
To demonstrate what I mean; when someone says to me that they need a tool for “API Testing” my first response would be: what type of testing do you mean? Testing contains a wide range of activities: exploratory testing, automating smoke checks, contract testing, performance, and the list goes on. Different tools are going to help with different activities. Going back to the earlier question asked on Reddit “Should I use Postman or RestAssured?” If we look at what each tool offers in terms of features we’ll observe that they are relatively similar. They allow us to send HTTP requests and assert the responses to determine if something meets our expectations or doesn’t. Postman has a nicely presented UI, RestAssured has a neatly arranged API. Different implementations, but the same functionality at its core. So it’s hard to make a comparison based on their features. Instead, let’s think about what we want to achieve with our tools.
When I am doing exploratory testing, I like to use Postman. It’s easy to import requests, there is a history feature to leverage and I can quickly create API requests to experiment with an API. It’s also really easy to create data-driven API requests in the runner tool that allow me to send hundreds of requests to then analyse to see if any errors appear or trends reveal themselves. I can do all this in RestAssured, when I am exploratory testing, but this is not necessarily the best use of my time. If I am battling CSV imports in Java, my attention isn’t on the testing at hand, which impacts the quality of my testing.
Equally, I find when I am building automation, I prefer tools like RestAssured over Postman. I like that I have control over how I build my automated API checks. I can implement mocks easily, create custom reports and cleanly arrange my code. I also can embed my automation codebase with the production code (Assuming the codebase is in Java). This encourages team ownership of automation and helps everyone to leverage API automation.
To be clear, these are just examples based on my own experiences in testing and not definite answers to what you should and shouldn’t use. I found that these tools worked for me in different ways for different activities by paying attention to what I wanted to achieve and then evaluating tools that suited my needs on a case by case basis.
What is your context
As I mentioned, my examples of selecting tools for specific testing activities are not definite answers on which tools to use and when. That’s because although my examples might share some similarities with what others might be doing in their testing, the context in which those activities are being carried out might be different. To demonstrate what I mean by how context can influence our decision-making, let’s take three different examples:
- Production architecture – The language and libraries you use will influence what tools you might want to use. If your APIs are built in Java, then perhaps RestAssured is a great choice. You can easily combine automation and production code into one project. But if your APIs are built using Python then using RestAssured means multiple languages and projects to support. Which could be an unnecessary overhead.
- Team experience – If we want teams to adopt specific tool sets then appreciating everybody’s skill sets and preferences is a must. It may be that Tool A is the most effective tool for the job, compared to Tool B that everyone uses. But if you have limited time and budget to train people before the project is concluded then it might be that the limited features of Tool B are an acceptable trade-off and enough to get you over the finish line.
- Culture – Finally, imagine you have decided you want to use the new SuperWicked API Testing tool Pro. What good is that if you can’t install it because IT policies have locked down your machines? Management is being advised to use different tools by another team or external sales team. Testing is seen as unnecessary! The culture of your organisation will limit or enhance the choices you want to make. Just because you used that great tool in a previous role doesn’t mean it’s going to work in your new context.
What these examples demonstrate is a need to understand your context and how it impacts your ability test or ‘Testability’. Understanding what is going on in our projects and organisations will help us to appreciate what choices we can and can’t make.
Get experienced
What I’ve hopefully demonstrated is that there is no right or wrong API testing tool to choose (This can of course be applied to all tool selections). So if there is no single tool to get good with then you’re free to learn any tool you like.
My colleague, Richard Bradshaw, and I like to talk about being ‘Tool aware’. This means taking the time to learn about lots of different tools that are available to you. So go try out Postman, RestAssured, Curl or the multitude of other API tools out there. What you’ll find is you will build up a library of knowledge about these tools, what they can do, what they can’t and what they can help with. The result is that if you combine your growing knowledge of tools with your analysis of what you want a tool to help you with and how it will fit best into your context. You’ll be able to quickly identify the right tool for the job.
Conclusion
The choice of API automation tools and tools for testing/automation, in general, is staggering. There has been an explosion of them in the past few years and many people have many opinions on what is right and what is wrong. But in the end, they aren’t you and they aren’t working on the same challenges as you and your team. What is the best tool for API automation? The one that you know will help you achieve what you want to achieve in a way that fits your context.
About the Author
Mark Winteringham is an international speaker and instructor in Software Testing. Drawing on over 10 years of testing expertise working on award-winning projects across a wide range of technology sectors and clients including BBC, Barclays, UK Government and Thomson Reuters. Mark has helped teams across the world to deliver modern risk-based testing strategies through his training in API Testing, Automation in Testing, Behaviour Driven Development and Exploratory testing techniques. He is the co-founder of Ministry of Testing Essentials, an online community raising awareness of careers in testing and improving testing education. He also co-created the Automation in Testing namespace which offers a modern approach to test automation practices.
Mark Winteringham is the author of the book Testing Web APIs published by Manning. Software Testing Magazine readers can buy the book and all other Manning publications with a 35% discount using the code blsofttm23.