Anatomy of an alert
There are some very fundamental parts of an alert that are generic to any alerting system. They are translatable to Nagios, SCOM, Icinga, or take your pick. In Splunk, however, there are some unique components of an alert that give us the ability to enhance what the alert itself does, and mostly it has to do with SPL(Splunk Processing Language). Once we have gotten the results we want, there are some fun things we can do with an alert.
Search query results
This is the result set of any search that you determine viable for an alert. It is often easiest to use a stats command to set an alert, as it gives an integer that can easily be filtered by a where statement. The amount of history searched is also very important in the setup of an alert.
Alert naming
The naming convention you choose often doesn't sound very important at all. I can tell you that, usually, it is not, right up until you start collecting hundreds, if not thousands, of alerts, which does happen. Creating the...