Any reasonable log management system needs to be able to achieve the following:
Filter logs that aren't important, and therefore should not be counted or stored. These often include log entries at the
INFOorDEBUGlevels (yes, these exist in production systems).Analyze the log entry further and extract as much meaning and new fields as possible.
Enhance/update the log entry prior to storage.
Send notifications on when certain logs are received.
Correlate log events to derive new meaning.
Deal with changes in the log's structure and formatting.
This recipe integrates the JBoss Library and Drools into a bolt to make these goals easily achievable in a declarative and clear manner. Drools is an open source implementation of a forward-chaining rules engine that is able to infer new values and execute the logic based on matching logic. You can find more details on the Drools project at http://www.jboss.org/drools/.