This machine learning process step is also very important because it provides a distinctive measure of the quality of your model, and if wrongly chosen, it could either ruin the accuracy of the model or its efficiency in the speed of convergence.
Expressed in a simple way, the loss function is a function that measures the distance from the model's estimated value to the real expected value.
An important fact that we have to take into account is that the objective of almost all of the models is to minimize the error function, and for this, we need it to be differentiable, and the derivative of the error function should be as simple as possible.
Another fact is that when the model gets increasingly complex, the derivative of the error will also get more complex, so we will need to approximate solutions for the derivatives with iterative methods...