Production-level Hadoop ecosystems are implemented with sufficient authentication, authorization, and data security in place. We have already covered various ways to implement security in Hadoop in previous chapters. But, as discussed, while implementation is one thing, monitoring and alerting are another. What if someone log into your system using some other mechanism or someone tries to enter your system by some other route? What if a user performs operations they are not allowed to?
Security information and event management (SIEM) is an audit that logs entries and from the security system and converts them into an actionable item. This actionable information can be used to detect any potential threat, take action, and add a new investigation into the compliance process. Depending on how the SIEM system is designed, this entire process can be either in batch or real time, based on how critical the security threat is. The following...