Testing in the release cycle
You need to choose when you will run these different tests within your release cycle. You can select a subset of your automated tests to run against every product change as part of a CI/CD pipeline, separate from new feature testing. These CI/CD tests are vital checks to avoid bugs and issues from running live, especially if they are set to block the release process. However, these tests have strict requirements:
- Speed: They must run fast enough that they don’t overly delay releases
- Reliability: They must work consistently and only fail when there is a real issue
- Coverage: They must cover all critical aspects of your product
There is a natural trade-off between speed and coverage: the more you test, the slower it goes, so you need to carefully judge the tests to include. These tests also need to be highly reliable, as they are run so often and can delay releases. Unit tests are ideal here as they have fewer dependencies, but...