What is automated testing?
We discussed why you might want to automate your tests and when to do this, and more specifically, where to start. But we didn't give any thought to what automated testing is. So, let's do that before we conclude this chapter.
With automated testing, we address the automation of application tests, that is, scripting manual application tests that check the validity of features. In our case, these are the features that reside in Dynamics 365 Business Central. You might have noticed that we have been using somewhat different terms for it:
- Test automation
- Automated tests
- Automated testing
These all mean the same thing!!!
On one hand, automated testing is replacing manual, often exploratory testing. It's replacing those manual tests that are repeatable and automatable, and often no fun (anymore) to execute.
On the other hand, they are complementary. Manual testing will still contribute to raising the quality of a feature, making use of creative and experienced human minds able to find holes in the current test design. Automated testing might also include so-called unit tests. These are tests that verify the working of atomic units that altogether make up a feature. Typically, these units would be single, global AL functions – units that would never be tested manually. Ultimately, both manual and automated tests serve the same goal: to verify that the object under test meets the requirements and doesn't do anything outside of the requirements.
Notes
(1) What is exploratory testing? Check out the following link for more information: https://en.wikipedia.org/wiki/Exploratory_testing.
(2) More on unit and functional tests can be found at the following link: https://www.softwaretestinghelp.com/the-difference-between-unit-integration-and-functional-testing/. We will also elaborate more on unit tests in this book.
Some more notes on automated tests
Before we head off to the next chapter, I would like to add a couple of notes on automated tests that haven't been touched on so far.
Automated tests are code too
There is no sense in denying: automated tests are also code. It takes time to design, implement, and maintain them. Like application code, any line of test code has a probability of containing a bug. The challenge is to keep this to a bare minimum. You can achieve this by:
- Making the test design a part of the requirements and requirements review
- Reviewing your test code like you would want to review your application code
- Making sure that tests always end with an adequate number of verifications
And, please, like any source, put your tests under source code management. They're a part of your product. Make sure to reserve time for this in your planning.
Automated tests need maintenance
Over time, your application changes and therefore the tests that go with it, be it manual or automated tests. Any change in the application, if well covered by test scenarios, will reflect in a number of failing tests as these no longer fit. Even a simple change in the order of the application code can lead to one or more tests falling over.
Testing vs. checking
One of the reviewers of this book, Xavi Ametller Serrat, pointed out the difference between testing and checking. With testing, we investigate and learn, while checking is about confirming and verifying. The first is typically a human exercise, while the second is what we can automate. Given this insight, automated testing should rather be called automated checking. In this book, however, the first term will be used as this links more to a common notice.
Note
An interesting read on testing versus checking can be found here: https://www.developsense.com/blog/2009/08/testing-vs-checking/.