A common mistake is to focus on dashboards as the primary means of noticing when something is wrong. Dashboards have their place in the big scheme of things and are an indispensable part of any monitoring solution. However, they are not as critical as we think.
Monitoring systems are not meant to be a substitute for Netflix. They are not supposed to be watched. Instead, they should collect data and, if certain conditions are met, create alerts. Those alerts should try to communicate with the system and trigger a set of actions that will correct the problem automatically. Notifications should be sent to humans only if the system does not know how to fix the issue. In other words, we should strive to create a self-healing system that consults doctors (us, humans) only when it cannot fix itself.
Dashboards come in handy when we know that there is a problem...