Incident response plan: documented and tested annually. Access reviews: conducted quarterly. The questionnaire score came back at 94 out of 100. The vendor was approved. Eighteen months later, the organization exercised its audit rights as part of a vendor governance review. The auditors found that MFA was implemented for most systems but not the ones processing the organization's data. Encryption at rest was implemented in production but not in the backup environment. The incident response plan had been tested once, three years prior. Access reviews for service accounts had never been conducted. The 94-out-of-100 questionnaire had described an organization that did not exist.
The Structural Limitation of Self-Reported Assessments
Vendor security questionnaires are self-reported assessments. The vendor answers questions about their security practices based on their own knowledge and judgment. There is no external verification of the responses at the time they are provided. The questionnaire score reflects the quality and accuracy of the vendor's self-reporting, which may or may not reflect the actual state of their security controls.
This structural limitation is well understood in theory and consistently underestimated in practice. Organizations that review a high-scoring questionnaire and approve a vendor relationship have made a judgment that the vendor's self-reporting is accurate. That judgment is rarely supported by evidence. The questionnaire provides the self-report. The evidence of accuracy requires verification.
Vendor self-reporting is influenced by factors that are not visible in the questionnaire response: the organizational pressure to complete assessments quickly, the interpretation of questions by the person completing the questionnaire who may not have full visibility into all relevant systems, the gap between what the security policy says and what has actually been implemented, and in some cases the incentive to provide responses that produce approval rather than responses that accurately describe the security posture.
A questionnaire score is a self-reported score. It measures the quality of the vendor's self-assessment. It may or may not measure the quality of the vendor's actual security controls. Treating the score as evidence of control quality requires evidence that the self-report is accurate.
What the Audit Found
The Scope Gap in MFA Implementation
The vendor's MFA response described the organization's MFA policy, which required MFA for all user accounts. The policy was accurate. The implementation was not universal. Systems that had been added to the environment after the MFA policy was established had not all been included in the MFA rollout. The systems processing the organization's data were among the ones added post-policy. The questionnaire response accurately described the policy. It did not accurately describe the implementation coverage.
The Backup Environment That Fell Outside Production Standards
Encryption at rest was implemented in the production environment. The backup environment had been built on an older infrastructure that predated the encryption standard. The backup infrastructure had been noted as a technical debt item but had not been prioritized for remediation. The questionnaire question about encryption at rest referred to data at rest broadly. The vendor answered it as a production environment question. The backup environment's condition was not within the responder's mental scope of the question.
The Incident Response Test That Had Not Recurred
The incident response plan had been tested. Once. Three years before the questionnaire was completed. The questionnaire asked whether the incident response plan was tested annually. The vendor's response said yes. The person completing the questionnaire was the same person who had been involved in the test three years earlier. They answered based on the fact that a test had occurred, not based on the fact that annual testing had not occurred since.
What Changes When You Verify
Organizations that exercise audit rights and conduct vendor evidence reviews consistently find gaps between questionnaire responses and verified reality. The gaps are not always significant — in many vendors, the gap is narrow and the questionnaire provides a reasonably accurate picture. In some vendors, particularly smaller organizations with less mature governance, the gap is material.
The organizations that have made verification a systematic part of vendor governance rather than an occasional exercise have found that the verification process changes vendor behavior. Vendors who know their questionnaire responses will be verified provide more accurate responses than vendors who expect questionnaire-only assessment. The verification requirement creates an incentive for accurate self-reporting that the questionnaire-only process does not.
Verification does not require auditing every vendor. It requires auditing enough vendors, with enough rigor, to establish that questionnaire responses are subject to verification — and to identify the vendors whose questionnaire responses diverge most significantly from their actual security posture. The vendors whose divergence is most significant are the vendors whose assessed risk tier should be revised upward.
Verify the questionnaire responses for your highest-risk vendors. The verification is what converts self-reporting into evidence.
Exercise the audit right. The questionnaire tells you what the vendor believes or wants you to believe. The audit tells you what is actually there.
Enterprise practitioner perspective. Not legal advice. Part of the Deep Trust Governance Series by Verisq. Get the free weekly Breach Digest.