Common SOC-CMM assessment mistakes and how to avoid them

Most flawed SOC-CMM assessments fail in the same handful of ways. The finding is consistent across the assessments we review: the problem is rarely the model. The SOC-CMM, created by Rob van Os in 2016 and now at version 2.4, gives you five domains, 27 aspects and a continuous 0-5 scoring scale. That is enough structure to produce a defensible result. What breaks the result is how the scoring is run, who challenges it, and what happens after the report is delivered.

The cost of these mistakes is concrete. A board approves an improvement budget against a maturity number. A regulator under NIS2 (Directive (EU) 2022/2555) or DORA (Regulation (EU) 2022/2554) reads the assessment as evidence of oversight. If the number is inflated or the evidence is missing, both decisions rest on sand. Below are the mistakes we see most often, grouped by where they enter the process.

The single most damaging pattern is accepting a self-assessment as the baseline without external challenge. The data is unambiguous: when independent assessors validate self-scored results, self-assessments overestimate maturity by roughly 0.6 points on the 0-5 scale. That is not a rounding error. It is the difference between reporting a SOC at 3.0 and discovering it operates at 2.4.

The bias is structural, not dishonest. The people scoring the SOC are the people who built it. They score the design they intended, fill gaps from memory, and read ambiguous questions generously. None of this requires bad faith; it only requires the absence of someone whose job is to ask for the evidence behind every answer. Self-assessment remains useful as preparation. It stops being useful the moment it is presented to a board or a regulator as an independent finding.

The 0.6-point gap. A self-scored 3.0 that is really a 2.4 does not just misinform the board. It deprioritizes exactly the improvement work the SOC needs, because the inflated score suggests the work is already done. The correction always costs more than the challenge would have.

Closely related, and present even in externally run assessments: scoring what the SOC intends to do rather than what it demonstrably does. A use case catalogue that exists as a roadmap item is scored as if deployed. A runbook drafted last month is scored as if exercised. An on-call rota agreed verbally is scored as if documented and tested.

The discipline that prevents this is simple to state: every score needs an artifact behind it. A document, an interview that corroborates it, and a sample (a ticket, a dashboard, a runbook execution) that shows the practice is real. When all three align, the score is defensible in front of any audience. When the assessor cannot produce them, the honest score is the lower one. The SOC-CMM's continuous scale exists precisely so partial, in-progress practices score as partial, not as complete.

The SOC-CMM measures two distinct things and teams routinely merge them. Maturity asks how well-managed a practice is: documented, measured, reviewed, improved. Capability asks how complete the function is: does the SOC actually have the people, processes and technology to perform it. A SOC can run a small set of detections with high maturity. It can also own an expensive technology stack with very low maturity in how it is operated.

When the two are averaged into a single undifferentiated number, the report answers neither question. The board cannot tell whether the gap is missing capability (an investment decision) or unmanaged capability (a governance decision). Keep the two readings separate in the scoring and separate in the report. The remediation paths are different, and so are the budget conversations.

The SOC-CMM has five domains: Business, People, Process, Technology and Services. Assessments led by engineers gravitate to Technology and Process and treat Business and People as soft, optional or N/A. The 2026 sector data shows why that is exactly backwards: average maturity is 2.7 in Technology but 2.5 in Business, 2.3 in People, 2.3 in Process and 2.2 in Services. The weakest domains are the ones most often skipped. Governance, mandate, budget alignment, staffing, training and retention are where SOCs are least mature, and an assessment that omits them certifies the strong half of the SOC while ignoring the half that fails first under pressure.

The related mistake is treating those sector averages as targets. An average is a description of the field, not a goal. A telecom SOC defending critical infrastructure under NIS2 may need 3.5 in incident management while 2.5 is acceptable elsewhere. Target maturity should be derived from your risk profile, your regulatory exposure and your service commitments, domain by domain. 'We are above average' is not a defensible position in front of a supervisor; 'we meet the target we justified for each domain' is.

Two process mistakes determine whether the assessment survives contact with scrutiny. The first is failing to build an evidence dossier: a versioned collection that links every score to the documents, interview notes and samples that justify it. Without it, the assessment cannot be defended six months later when internal audit, a supervisor or a new CISO asks how a number was reached. With it, the same questions take minutes to answer.

The second is treating the assessment as a one-off event. A single measurement proves nothing about direction. The value of a maturity assessment compounds on the second and third iterations, when the board sees a trend line: which domains moved, whether the roadmap delivered, where investment translated into measured improvement. Close every assessment with the next one already on the calendar, typically twelve months out, and with each roadmap item carrying an owner and a date. An assessment without a re-assessment cycle is a snapshot; regulators and boards are asking for a film.

• Fix the scope in writing before scoring starts: entities, geographies, coverage hours, and what the MSSP operates versus what you operate.
• Require the three-part evidence test (document, interview, sample) for every score above 2.
• Report maturity and capability as separate readings per domain.
• Set risk-derived targets per domain; use sector averages only as context.
• Schedule the re-baseline before the closing meeting ends.