Now that we have a basic status page up and running, let's talk about allowing the user to configure some alert conditions. For now, we will inform the user of any alert condition by showing that node's information in red color on the status page.
First, we need to figure out what kind of alerts we want our users to set. From there, we can figure out the technical details. So, let's think about it. Given that the data types we are recording all have numeric values, it makes sense that the user should be able to set thresholds. They can, for instance, set an alert if the system load of any node goes above 1.0 or if the disk usage of a node goes above 80%.
Furthermore, maybe our users don't want to have the same alert conditions for each node. A database node is expected to handle a lot of system load, so maybe our users want to have a separate alert condition for the database node. Finally, if they are doing maintenance on some of the nodes, they may want to stop some alerts from triggering...