Staying on top of alerts with PagerDuty
PagerDuty is a fantastic tool to record, alert, and escalate issues, and is insanely popular with DevOps folks due to its ease of use and surprising amount of power. Like most modern software, as a service, it's clean and simple to use and has an extremely approachable and powerful API.
PagerDuty is a great place to flag Puppet errors as it gives you the ability to track how often you are having Puppet-related issues. More importantly, it ensures that an alert is created when issues are found. These issues can automatically be escalated to the appropriate person, and if they're out of hours, they can be routed to whoever is on call. PagerDuty supports alerts via phone, SMS, e-mail, and push alerts, so it's unlikely that they'll be able to sleep through a problem. If they do manage to sleep through the sound of every communication device they own going nuts, then PagerDuty is able to alert the issue to the next person on rotation, and it will continue...