Skip to content
Open the dashboard
Documentation

What evidence does a SOC 2 auditor need for AI-written code?

Engineering leadsSecurity8 min read

SOC 2 and ISO 27001 do not have a control for AI-generated code, and auditors do not ask for one. They ask the questions they always asked about change management and access, and those questions get harder when a large share of commits are written by agents. The evidence that answers them is the same for human and agent code: a written standard, proof it was applied to every change, a record of the exceptions, and a way to tie each of those to a control.

The auditor’s question SOC 2 (2017 TSC) ISO 27001:2022 Annex A What answers it for code
Are changes authorized, tested and reviewed before production? CC8.1 8.25, 8.32 A check on every pull request against written standards, with the result recorded per change
Are credentials and keys kept out of code? CC6.1 5.17, 8.24 A blocking rule for tokens, keys and .env files, with evidence of how often it fired and on what
Are vulnerabilities and misconfigurations detected? CC7.1 8.8, 8.9 Deterministic checks for the known-bad patterns, pinned supply-chain inputs, a report of findings over time
Is data in transit protected? CC6.7 8.20, 8.24 Rules against disabled TLS verification and old protocol versions
Are deviations from policy approved and time-bound? CC8.1 5.15, 8.25 Exceptions with a reason, an approver and an expiry, and a list of which were used

The right-hand column is the part teams struggle with. “We have a coding standard” is a document. An auditor wants to see the standard, the mechanism that applied it, and the records.

Evidence is something produced by the process while it ran, not something written afterwards for the audit. For code standards that means:

  1. The standards themselves, versioned. Which rules were in effect, with their text and severity, at any point in the period. If the rules changed mid-quarter, both versions.
  2. A record per change. For each pull request: which rules were evaluated, which passed, which failed, which were suppressed by an exception, and the outcome. Reruns of the same commit counted once.
  3. The exceptions. Each with its reason, approver and expiry, and how many findings it hid.
  4. A summary per control. The rules that support each control, and the pass rate across their evaluations in the period, so the reviewer does not have to map rules to criteria by hand.
  5. Who or what wrote the change. For agent-written code, the declaration in the commit (a Co-Authored-By trailer or a bot author), reported honestly as a declaration rather than proof.
  • A policy document with no record of enforcement.
  • Screenshots of a dashboard taken the week before the audit.
  • An AI review tool’s comments, unless they are deterministic and recorded per change. A reviewer’s opinion that varies run to run is not a control.
  • “The agent was instructed to follow the standards.” Instructions are context; the agent follows them most of the time. Only the check on the change proves the standard was applied.

Each item above maps to something the product already records; nothing is collected for the audit that was not collected for the work.

  • Standards, versioned: every rule in the rulebook has versions and change notes, and every check run records a digest of the exact rules in effect, so a reviewer can tell when the rules changed.
  • A record per change: every groundrule check --upload from CI is a compliance record: the rules evaluated, each result, the exceptions applied, and the coding agents the commits declare. Runs can be signed (Ed25519 or Sigstore) and verified offline later, so the record cannot be altered after the fact. See What happens on GitHub and Sign and verify.
  • Exceptions: the Exceptions used table, soonest expiry first, with approver and expiry.
  • Per control: the By control panel groups the rules by SOC 2 criterion and ISO 27001 control, with the pass rate across the period. The mappings come from the catalog (117 of its 171 rules map to SOC 2 criteria, 120 to ISO 27001 controls) and from your own standards’ spec.compliance.
  • Who wrote it: the Coding agents panel, from commit trailers and bot authors only, labelled as a declaration.
  • The file to hand over: Download CSV on the Compliance page produces one file with every table above for a repository and a period of up to 366 days.

Compliance report documents each panel and the API behind it.

If an audit is within six months and the team uses coding agents, the order that works:

  1. Put the standards you already have in writing in a form a tool can read, starting with the security rules (tokens, keys, TLS, SQL). The security pack covers the common ones.
  2. Run the check on every pull request in observe mode, so records accumulate without blocking anyone.
  3. Move the security rules to enforce once the records show they are clean.
  4. Write exceptions for the legacy modules, with real expiry dates.
  5. Hand the auditor the CSV for the period, and the link to the standards.