Creating program KPIs

Just as detection engineers want consistent ways of monitoring their detection development cycle, leadership needs mechanisms to observe progress at the team level. Anecdotally, I have witnessed many teams going straight to total counts of detections, or how many detections are produced per sprint. While these are good metrics to account for, they don’t measure the true security impact and effectiveness.

Let’s say we were capturing that the team created 10 IOC-based detections in a given sprint on time and that passed all levels of testing. Unfortunately, what we’re not answering that helps us determine program health includes the following:

• Did SOC find malicious activity from the deployed detections?
• How aligned are we to the threat intelligence TTP priorities?
• Has the detection helped to support true positive findings of an incident?
• Do the detections specifically support larger executive leadership strategies?
...