Some organizations have published recently reports on the status and the trend of the software testing domain. We have aggregated the main results of these reports in this article to give you an overview of the current software testing issues and practices.
Software testing is facing different evolutions that need a new perspective. The speed of software delivery is getting faster, influenced by approaches like DevOps and Agile software development. The challenge is also to deliver and test solutions that have to work on different channels from the desktop to the mobile. On the technical side, the shift to mobile applications requires to support and validate a large number of mobile platforms and versions distributed on a vast amount of devices from different suppliers. The Internet of Things (IoT) is also a new trend that creates customer-facing software on different gadgets or appliances.
Allocation of software quality budget. Source [1].
The reports referenced in this article are the Capgemini Sogeti World Quality Report 2015-16 [1], the Sauce Labs State of Testing Report [2] and the Practitest State of Testing Survey 2015 [3]. Each of them have their own type of respondents and survey field.
Test Automation
The budget allocated to software testing has increased recently and managers are afraid that it is getting out of control. They cite the reliance on manual testing as their biggest challenge, so the amount of test automation is increasing. According to [1], the average amount of test case automation has increased from 29% in 2014 to 45% in 2015. Among the main benefits of test automation, respondents in [1] mention reducing costs and increasing speed. The same people cite the changes in functionalities and the lack of test automation in their delivery methodologies as the main challenges for test automation. In survey [2], 29% of participants say that their testing efforts are mostly or fully automated. Only 5% rely only on manual testing. In this survey, almost 80% of the participants use continuous integration. In survey [3], 25% of participants automate 50% or more of their tests. Functional testing (75%) and load testing (49%) are the area where most of the test automation occurred.
Top five technical challenges in software testing. Source [1]
Mobile Testing
According to [1], mobile testing is now a standard practice performed by 92% of the organizations. However, many of the participants complain that they don’t have the internal resources and the time to perform this activity. The number of organizations investing in getting internally the mobile devices needed to perform testing has strongly increased these last two years. Managers also complain that their testers sometimes lack the needed skill set for mobile testing. Survey [2] shows that almost 50% of participants use the cloud for mobile testing.
Location of mobile testing. Source [2]
Agile Testing
According to [1], Agile has been adopted by 54% of the organizations. Exploratory testing and Test-Driven Development (TDD) are mentioned as the most used Agile testing practices. The rate of Agile adoption is 88% in the survey [2] where weekly deployment is the rule for 67% of the respondents and 86% report that development and QA teams think of themselves as partners that communicate in real time, that is without using a bug tracking system or emails. In survey [3], 88% of participants’ development teams follow an Agile approach and the adoption of the TDD practice is close to 20%.
Security
When an application is distributed on a mobile or accessed through a web interface, security is the number one concern in software testing. According to [1], security testing is mostly performed dynamically. Organizations often mix internal and external resources to perform security testing. In [3], security testing was considered an important skill for software testers by 90% of the participants.
Most performed activities in security testing. Source [1]
References
1. Capgemini Sogeti World Quality Report 2015-16, 80 pages, Participants: 1560 executives
https://www.capgemini.com/thought-leadership/world-quality-report-2015-16
2. Sauce Labs Testing Trends In 2016, 12 pages, Participants: 520 test profess
https://saucelabs.com/resources/white-papers/
3 Practitest State of Testing Survey 2015, 17 pages, Participants: 860
http://qablog.practitest.com/state-of-testing-2015-2/