What are Security Metrics?
Security metrics are meant to provide insights to executive leadership (e.g., “are we more secure today than we were yesterday?”), but are often useless due to metrics / analytics reporting suffering from a Garbage In, Garbage Out (GIGO) problem. Often GIGO issue is rooted in executives trying to explain their perceived needs for metrics to cybersecurity practitioners in a way that describes the design of a "football bat" (e.g., nonsensical solution).
Interestingly, security metrics are often a misnomer. When executives ask for metrics, they really want analytics (e.g., trending):
- Metrics are discrete, “point in time” measurements that provide no context; and
- Analytics are generated from the analysis of metrics to provide context through trending.
Analytics, not metrics, are designed to facilitate decision-making, evaluate performance and improve accountability through the collection, analysis and reporting of relevant performance related data. Security metrics / analytics can leverage:
- Key Performance Indicators (KPIs); and
- Key Risk Indicators (KRIs).
KPIs:
- Are “rearward facing” and focus on historical trending to evaluate performance.
- Should not be weighted.
- Are indicators that enable an organization to monitor its progress towards achieving its defined performance targets.
- Are used to answer the question, “Are we achieving our desired levels of performance?” for a specific control.
KRIs:
- Are “forward facing” and focus on identifying a future-looking trend that impacts risk.
- Should not be weighted.
- Are indicators that enable an organization to define its risk profile and monitor changes to that profile.
- Are used to answer the question, “Are we within our desired risk tolerance level?” for a specific control.