The technical presentation touches on topics such as test automation framework design, hermetic servers, Docker containers, architecture for testability, test environments provisioning, DevOps collaboration, testing when depending on internal and external services, the joys and pitfalls of parallel execution.
All automated tests, other than unit, are slow and unreliable for the fast development pace every company craves. When a test fails, it’s hard to pinpoint the exact reason why. There are lots of external dependencies and factors outside of your control. At Komfo, we had automated tests running for more than 3 hours every night in relatively large SaaS application. The execution time just kept growing unrestricted, and the tests were getting more unstable and unusable as feedback loop. At one point the continuous integration build for the tests was red for more than 20 days in a row. Regression bugs started to appear undetected in production. We decided to stop this madness and after considerable effort and dedication, currently the same tests run for 3 minutes. This is the story of how we achieved nearly 60x faster tests.
We believe that in the near future this will be standard practice, much like unit tests or continuous integration now. In order for a company to stay competitive, all existing automated tests — e.g. static code analysis, unit tests, API, UI, should complete in less than 5 minutes after every code change.
Video producer: https://developers.google.com/google-test-automation-conference/