This episode of Coffee Thoughts With Tom addresses CMMC as a conformity assessment, since conformity assessments are intended to use a risk-based approach to determine a confidence point (e.g., materiality threshold) instead of a binary approach that requires 100% compliance without compensating controls or Plan of Action & Milestones (POA&M) items. This article does provide a quantifiable solution that is based on industry practices, so read on!
From other cybersecurity practitioners, I've heard that in respect to CMMC there are so many "elephants in the room" that it feels a bit like a circus. Arguably, the biggest elephant in that circus is the DoD's binary approach to pass/fail. While it may disappoint certain readers that this article is not about circus animals, others might enjoy reading about a proposed solution to define a materiality threshold for a risk-managed approach to CMMC assessments. Instead of merely complaining, I want to see a rational discussion on the subject, since the DoD PMO's approach to a "100% pass or 100% fail" in evaluating CMMC practices and processes is both misguided and goes against industry-recognized practices. CMMC will fail without a POA&M component, so the DoD PMO and CMMC-AB needs to have a way to incorporate that need into CMMC assessment criteria and this article addresses that need.
For transparency, the approach described in this article is based on industry-recognized practices and is the basis of what is under development for the Information Assurance Program (IAP) that is part of the Secure Controls Framework (SCF). Both the IAP and CMMC are conformity assessments.
Conformity Assessments
To appreciate the points of this article, you really need to understand the basics of conformity assessments. NIST published an excellent primer on the subject, if you want to read more about conformity assessments: NIST Special Publication 2000-01 - ABC's of Conformity Assessment. In summary, conformity assessments examine "an object of conformity (such as a product, process, system, person, or body) and determines whether the object meets specified requirements."
For the conformity assessment hierarchy of CMMC assessments:
If you read through NIST SP 2000-01, you will not find any mention of the words "fail" or "pass." Conformity assessments are designed to assure that a particular product, service, or system meets a given level of quality or safety. Instead of a 100% pass criteria, conformity assessments rely on a "confidence point" that determines a risk-based threshold to establish if the intent of the objective(s) has been achieved.
NIST Special Publication 2000-02 - Conformity Assessment Considerations for Federal Agencies is a great follow-on read on the subject, since it goes into good detail on the topic of confidence points. Essentially, a confidence point establishes pre-defined criteria that an objective has been achieved, weighing the risk of non-conformity against its associated consequences. What the DoD PMO fails to appreciate is the confidence point is not meant to be a 100% perfection expectation. This is where the concept of materiality comes into play.
"Perfection is the enemy of progress." - Winston Churchill
Confidence Points & Materiality
To help determine what constitutes the confidence point for a CMMC assessment, it is important to leverage industry practices. In particular, the concept of "materiality" is a relevant approach to address this need. In legal terms, "material" is defined as something that is relevant and significant:
For those in the Governance, Risk Management & Compliance (GRC) space, materiality is often relegated to SOX compliance. However, the concept of materiality is much broader than SOX and can be used in any form of conformity assessment.
"Specific to the CMMC, a material weakness is a deficiency, or a combination of deficiencies, in an organization's security controls where it is probable that reasonable threats to regulated data (FCI/CUI) will not be prevented or detected in a timely manner."
In practical terms, a material weakness a deficiency, or a combination of deficiencies, in the internal security practices of an OSC where it is probable that reasonable threats to CUI will not be prevented or detected in a timely manner. Following this logic, the CMMC assessment methodology could identify non-compliant practices and processes as either a (1) minor deficiency or (2) material weakness. A material weakness would be an immediate failure for the CMMC assessment, but a minor deficiency would be something capable of being addressed through compensating controls and temporarily placed on a POA&M. POA&Ms have a long history going back to DITSCAP and DIACAP, but it is still a valid way to remediate risk in NIST SP 800-171, RMF, FedRAMP, FISMA and even outside of the government with compliance obligations such as PCI DSS.
For those OSC already in scope for DFARS, the OSC would be expected to follow established requirements to submit POA&M items for approval by the DoD CIO. For those not in currently in scope for DFARS, POA&M items could be self-regulated until a contract is awarded and the OSC falls under DFARS. The C3PAO would be required to validate if any POA&M items are legitimate per DFARS applicability. That process would allow for the use of a "temporary deficiency" to be used in the CMMC assessment model that would be tracked per CA.2.159 (POA&M requirement within CMMC).
Similar to DITSCAP, DIACAP, RMF, there could be three straightforward ways to define the confidence point for a CMMC assessment:
*If an OSC had an IATC designation, they could get a 180 or 365 day grace period to remediate the deficiencies that would require re-evaluation of those deficient controls by a C3PAO. That approach allows for flexibility in applying compensating controls without disrupting the DoD's supply chain.
At the end of the day, the CMMC requires a defendable and repeatable approach to calculate materiality. Based on an understanding that any non-compliance in a high-risk practices or process would be considered a material weakness of the OSC's security program where no POA&Ms should be accepted, this could open the door for working with low and moderate risk practices. NIST SP 800-171 DoD Assessment Methodology already tags NIST SP 800-171 controls with a score of 1, 3 or 5, so that work could be leveraged to assign corresponding "low, moderate and high" risk rankings to CMMC practices and processes. For the non-NIST SP 800-171 controls, the CMMC-AB could assign a low, moderate or high risk ranking to each practice.
What is being described is a straightforward approach for the DoD to establish a confidence point. For example, any combination of deficiencies that exceed a 10% threshold become a material weakness of the OSC's cybersecurity program since it indicates a failure to protect regulated data (FCI/CUI). That threshold could also be 20% (or any other reasonable value) - it would be up to the DoD to determine the confidence point that reasonably manages cybersecurity risk to the DoD against the financial impact to the DIB.
Specific to this CMMC example:
Below is a quantitative solution that uses a 90% confidence point as the threshold to determining materiality:
o Level 1 - 17 practices
o Level 2 - 72 practices
o Level 3 - 130 practices
o Low-risk practices: 3 points
o Moderate-risk practices: 5 points
o High-risk practices: 20 points
o Processes: 20 points
o Level 1 = 14
o Level 2 = 134
o Level 3 = 153
o Level 1 - 1 point (this means no low or moderate practices can be deficient)
o Level 2 - 13 points
o Level 3 - 15 points
The charts below show an example of how various combinations of low and moderate-risk practices could be deficient to still "pass" the CMMC assessment on an interim (e.g., IATC) basis.
The end result is this would allow for a small number and combination of low and moderate-risk practices to be deficient (tracked and approved via POA&M through the use of compensating controls) and still enable the OSC to participate in the DoD supply chain, since the risk would be sufficiently mitigated. The cells highlighted in green would meet the conformity point. The cells highlighted in red would mean the conformity point was not met and a material weakness would exist to protect regulated data (FCI/CUI).
Summary
As it currently exists, CMMC is not structured as a "real" conformity assessment - it is merely designed as a checklist-based assessment performed by an independent third-party that does not take into account the OSC's technology or business processes to legitimately manage risk. This would be an easy fix by the DoD PMO, since both POA&Ms and the concept of materiality are well-established and would allow for a risk-managed approach to CMMC assessments.
In the end, compensating controls are a good security practice, when done properly. It sufficiently manages the risk of the client (DoD) and takes into account real-world limitations affecting the DIB. The objective solution that is described in this article would also remove subjectivity from the C3PAO by using a point system that is centrally-managed by the CMMC-AB.
As the DoD's "Pathfinder" proof of concept is executed later this year, it will be interesting to see how the topic of deficiencies plays out and if "critical" prime and sub-contractors are allowed exceptions to this draconian pass/fail concept. My bet is that one elephant will ruin the rest of the circus.
About The Author
If you have any questions about this, please feel free to reach out. Tom Cornelius is the Senior Partner at ComplianceForge, an industry leader in cybersecurity and privacy documentation. He is also the founder of the Secure Controls Framework (SCF), a not-for-profit initiative to help companies identify and manage their cybersecurity and privacy requirements.
