Vendor assessment completion on schedule. Finding closure rate at its best level since the program was established. The CISO presented the results as evidence of a strengthening security posture. Six weeks later, the penetration test results arrived. The testers had compromised three production systems, accessed a customer data store, and exfiltrated a sample dataset — in fourteen hours. The metrics were accurate. The risk position they described was not.
When Metrics Improve Without Improving Security
Governance metrics improve when the activities they measure improve. Patching coverage improves when more patches are applied. Training completion improves when more training is completed. Finding closure improves when findings are closed faster. Each of these is a genuine operational improvement. Whether operational improvements translate into security posture improvements depends on whether the improved activities are addressing the risks that matter most in the current threat environment.
Patching coverage at 94 percent is objectively better than patching coverage at 90 percent. Whether it is a meaningful security posture improvement depends on which systems are in the 6 percent not patched. If the unpatched 6 percent are the internet-facing systems most likely to be targeted, the coverage improvement has improved governance metrics while leaving the most consequential exposure unaddressed. The metric improved. The risk did not.
This is not a theoretical observation. Post-incident analyses consistently find that the organizations that experienced the incident had improving governance metrics in the period before the incident. The metrics were improving. The specific conditions that produced the incident — the unpatched system, the misconfigured access control, the detection gap — were present and not captured by the metrics that were improving.
Improving governance metrics are evidence that the measured activities are improving. They are not evidence that the risk position is improving, unless the measured activities are the activities that determine the risk position. Verify which is which before presenting the metrics as security posture evidence.
Why the Penetration Test Found What the Metrics Did Not
The penetration test found three compromised production systems and an accessible customer data store because the testers approached the environment from the adversary's perspective: finding the paths of least resistance, exploiting the gaps that the detection was not watching, using the access that had accumulated beyond what was needed. The governance metrics described the activities the governance program was tracking. The penetration test described the attack surface the adversary would find.
The patching coverage metric did not capture that the three compromised systems had been added to the environment after the patching program's scope was defined and were not in the patching program's population. The finding closure rate did not capture that one of the open findings — a medium-severity misconfiguration — was directly on the path the testers used for initial access. The training completion metric did not capture that the three employees whose credentials the testers compromised through phishing had completed the training and clicked the phishing link anyway.
Connecting Metrics to the Adversary's Perspective
Closing the gap between improving metrics and improving security posture requires connecting governance measurement to the threats and attack paths that are most relevant to the specific organization. Patching metrics connected to the vulnerability categories being actively exploited against the organization's sector produce a different — and more meaningful — signal than patching metrics measuring overall coverage. Access review metrics connected to the specific access patterns that privilege escalation attacks exploit produce different signals than completion rate metrics.
The penetration test is the most direct mechanism for connecting governance metrics to adversarial reality — it tests the environment against the perspective of an attacker rather than against the governance program's own measurement framework. Organizations that integrate penetration test findings into their governance metrics — using test results to identify where metrics are improving while security posture is not — create a feedback loop that corrects the divergence over time.
Regular adversarial testing — red team exercises, purple team collaborations, external penetration testing — produces the ground truth against which governance metrics can be calibrated. When adversarial testing consistently finds pathways that the governance metrics suggest should not exist, the metrics are measuring the wrong things. Adjusting what is measured based on what adversarial testing finds produces metrics that more accurately reflect the security posture they are supposed to describe.
Calibrate the metrics against adversarial reality. The metrics that improve without improving security are measuring governance activity, not security posture.
Run the penetration test. Compare what the testers find to what the metrics predicted they would find. Adjust the metrics to close the gap. Repeat annually.
Enterprise practitioner perspective. Not legal advice. Part of the Deep Trust Governance Series by Verisq. Get the free weekly Breach Digest.