Cybersecurity Metrics Reporting Model

The ComplianceForge Cybersecurity Metrics Reporting Model (CMRM) takes a practical view towards implementing a sustainable metrics reporting capability. At the end of the day, executive management (e.g., CIO, CEO, Board of Directors (BoD), etc.) want an answer to a relatively-straightforward question: “Are we secure?” In order for a CISO to honestly provide an answer, it requires a way for the CISO to measure and quantify an “apples and oranges” landscape where processes and technologies lack both uniform risk weighting and abilities to capture metrics. The SMRM helps solve this aspect of dissimilarity by utilizing a weighted approach to metrics that generate Key Performance Indexes (KPXs) as a way to logically-organize and report individual metrics. Using KPX enables the SMRM to provide a reasonable and defendable answer.

Metrics & Analytics: Garbage In, Garbage Out (GIGO) Problem

Metrics / analytics reporting is plagued by the Garbage In, Garbage Out (GIGO) problem. Often GIGO issue is rooted in executives trying to explain their perceived needs for metrics to cybersecurity practitioners in a way that describes the design of a "football bat" (e.g., nonsensical solution). How ComplianceForge addressed this GIGO problem is start with answering the most important question (e.g., are we secure") and working backwards in a manner to defend that answer.

The “Are we secure?” question is best answered as a numerical score. This quantifiable score is used to visualize the score against a numerical spectrum to provides context, based on the risk profile of the organization. The numerical score would land between “not secure” and “secure” on the spectrum, according to a baseline score definition that would be specific to the organization. This can provide long-term trending to evaluate the direct impact of certain security initiatives. The SMRM can be automated in a Governance, Risk & Compliance (GRC) or Integrated Risk Management (IRM) platform, but it comes as a Microsoft Excel spreadsheet as part of ComplianceForge’s Digital Security Program (DSP). The “Are we secure?” question can be both tracked to display trending and drilled down into KPXs, or individual metrics, to identify why the score changed.

 

Key Performance Index (KPX) is essentially a term that we use to normalize the various metrics in each category. One area of contention with metrics is defining what a KPI or KRI is since people tend to butcher the terminology. Our approach to defining those terms are shown below:

Key Performance Indexes (KPXs)
KPXs are logical groupings of KPIs that allow an organization to monitor an index of metrics about a specific capability or team.

• KPXs are used to answer the question, “Is the XYZ capability operating effectively?” where that capability is an aggregation of multiple individual metrics.
• KPXs may be weighted to highlight risk-heavy topics of concern.
• KPXs may be nested underneath other KPXs to report the hierarchical nature of metrics that help answer the question of “Are we secure?”

KPIs and KRIs are not hierarchical metrics, but are individual metrics that are deemed important to monitor, based on the specific risk or value associated with that metric:

Key Performance Indicators (KPIs)

• KPIs are “rearward facing” and focus on historical trending to evaluate performance.
• KPIs should not be weighted.
• KPIs are indicators that enable an organization to monitor its progress towards achieving its defined performance targets.
• KPIs are used to answer the question, “Are we achieving our desired levels of performance?” for a specific control.

Key Risk Indicators (KRIs)

• KRIs are “forward facing” and focus on identifying a future-looking trend that impacts risk.
• KRIs should not be weighted.
• KRIs are indicators that enable an organization to define its risk profile and monitor changes to that profile.
• KRIs are used to answer the question, “Are we within our desired risk tolerance level?” for a specific control.

The metrics shown in this model are included in the ComplianceForge Digital Security Program (DSP) product. Being transparent on the subject, the entire point of a "canned solution" for metrics is to provide a starting point where someone else does the heavy lifting for you to get to a 70-80% solution that someone within your organization can then run with to customize for your specific needs. This is where ComplianceForge is a business accelerator - we enable you to hit the ground running with your cybersecurity documentation that can takes months or years to create on your own. The "heavy lifting" of the equation is what we provide, not the finalized metrics product. That is really where the demarcation is between what ComplianceForge offers for metrics and how an organization would customize the remaining since you have the organization-specific knowledge side of the metrics equation that cannot be templatized.

ComplianceForge does not sell the SMRM, KPIs, KRIs on their own, since the metrics are part of the DSP solution. With the 1-1 mapping relationship between the DSP and the Secure Controls Framework (SCF), the DSP can help operationalize the SCF controls in a meaningful and efficient manner, so that is something to consider for organizations that want to fully adopt the SCF as its control structure and maximize its effectiveness.

We are happy to discuss our products. Please feel free to contact us with your questions. 

• Efficient CMMC Scoping

  Determining the scope of controls (e.g., assessment boundary) is different than determining control...

• Are you a cyber criminal?

  As a Chief Information Security Officer (CISO) or cybersecurity director, it is likely that you been...

• New Standard For Third-Party Cybersecurity Assessments

  The release of the Cybersecurity & Data Protection Assessment Standards (CDPAS) is important to the...

• Cybersecurity Materiality & Key Controls

  There is a "materiality ecosystem" that exists within modern cybersecurity risk management discussio...

---