Checks

A check is the part of a standard that a machine can run on your code. Each check names an evaluator and gives it options. This page covers every evaluator that ships with Groundrule, every option and default, where findings point, and how to test a check before you rely on it.

How checks work

A check is one entry in a standard’s spec.checks list:

spec:
  checks:
    - evaluator: regex          # which evaluator
      pattern: 'console\.log\(' # options for that evaluator
      message: console.log in application code

Evaluator	What it checks	Source shown with findings	Confidence
regex	A pattern that must not, or must, appear in file contents.	deterministic	certain
files	Files that must or must not exist.	deterministic	certain
dependencies	Packages that must not be declared in manifests.	deterministic	certain
change-set	Files that must change together.	deterministic	certain
semgrep	Semgrep rules. Needs Semgrep installed.	static analysis	high
llm	An AI review. Not yet available.	AI

Every check also accepts minConfidence (see The rule format). Every other key is an option for the evaluator, and an unknown option is an error.

A standard can have several checks. Its findings are those of all its checks. A standard with no checks is guidance: it reaches agent files and is reported as ◇ guidance by check.

Which files a check reads

1. Groundrule lists the repository’s files: tracked files, plus untracked files that git doesn’t ignore. Outside a git repository, it walks the folder.
2. It never reads its own files: the .groundrule/ folder, generated agent files, the managed block in AGENTS.md, CLAUDE.md and Copilot instructions, and any YAML file that declares apiVersion: groundrule.dev/….
3. It keeps the files that match the standard’s scope.paths and not its scope.exclude.
4. When checking a change (the default), content evaluators read only the changed files among those. With check --all, they read all of them.

regex, dependencies and semgrep read file contents. They skip files over 1,000,000 characters and binary files.

New and legacy findings

When you check a change, every finding is marked new or legacy. Only new findings can fail check, unless the configuration sets legacy: enforce.

enforcement.scope	A finding is new when
changed-lines (default)	Its lines overlap lines the change added or modified.
changed-files	Its file was added, modified or renamed in the change.
all	Always. check audits every file.

A finding with no file location (for example “Required file is missing”) is always legacy in a change. It counts with check --all, or with legacy: enforce. See Configuration.

Errors in a check

The CLI validates each check’s options when it runs. A problem is reported against the file and the check’s position, and check exits with 2:

✕ Config error  .groundrule/standards/ACME-006.yaml  spec.checks[2].evaluator: Unknown evaluator "nope". Available: change-set, dependencies, files, llm, regex, semgrep.
✕ Config error  .groundrule/standards/ACME-006.yaml  spec.checks[2].flags: Allowed flags: i, s, u

A check that can’t run in this environment (Semgrep not installed, no change to compare) is not evaluated. It is listed with the reason, and never fails check:

 – ACME-005 not evaluated: Needs a change to compare. Run in a pull request or with --base.

regex

Forbids, or requires, a regular expression in file contents.

Option	Type	Required	Default	Description
pattern	string	Yes		A JavaScript regular expression, without slashes.
flags	string	No	""	Any of i (ignore case), s (. matches newlines) and u (Unicode). g and m are always on.
mode	forbid or require	No	forbid	forbid reports every match. require reports every file that has no match.
message	string	No	See below	The finding’s message.
include	list of globs	No		Only files matching these, within the standard’s scope.
exclude	list of globs	No		Skip files matching these.
maxMatchesPerFile	integer	No	20	Stop after this many matches in one file. 1 to 1,000.

Because m is always on, ^ and $ match at the start and end of every line.

Findings with mode: forbid: one per match.

• Location: the file and the line where the match starts (leading whitespace in the match is skipped). A match across lines also records the end line.
• Evidence: the line, for example 4: console.log("refunding", id, amount);.
• Default message: Matches forbidden pattern /<pattern>/<flags>.

Findings with mode: require: one per file in scope that doesn’t match.

• Location: line 1 of the file.
• Evidence: <file> does not contain /<pattern>/.
• Default message: Required pattern /<pattern>/ not found.

With the default changed-lines scope, a require finding is new only when line 1 of the file changed, for example in a new file. Existing files show as legacy.

# Forbid: no direct Stripe imports outside the gateway
checks:
  - evaluator: regex
    pattern: '\bfrom\s+["'']stripe["'']'
    message: Stripe imported outside the payments gateway

# Require: every workflow declares top-level permissions
checks:
  - evaluator: regex
    mode: require
    pattern: '^permissions[ \t]*:'
    include: [".github/workflows/*.yml", ".github/workflows/*.yaml"]
    message: Workflow does not declare top-level permissions

Errors: Enter a regular expression, pattern is not a valid regular expression, Allowed flags: i, s, u.

Quoting patterns in YAML

Put patterns in single quotes. Inside single quotes, YAML keeps backslashes as they are, so '\bfoo\w+' reaches the evaluator unchanged. To include a single quote, write it twice: '["'']' is the character class ["'].

In double quotes, YAML reads backslashes as escapes. "\w" fails to load with Invalid escape sequence \w, and "\b" silently becomes a backspace character. If you must use double quotes, double every backslash: "\\bfoo\\w+".

files

Requires or forbids files by path.

Option	Type	Required	Description
require	list of globs	One of the first three	Each glob must match at least one file.
requireAny	list of globs	One of the first three	At least one of the globs must match a file.
forbid	list of globs	One of the first three	No file may match these.
allow	list of globs	No	Files exempt from forbid, such as .env.example.
message	string	No	The finding’s message.

At least one of require, requireAny or forbid is needed (Set at least one of require, requireAny, or forbid).

Option	Files it looks at	Finding	Default message
require	Every file in the standard’s scope	One per glob with no match, with no file location. Evidence: No file matches <glob>.	Required file is missing: <glob>
requireAny	Every file in the standard’s scope	One, when no glob matches, with no file location.	None of the required files exist: <globs>
forbid	Changed files in a change; every file with --all	One per matching file, at line 1.	This file is not allowed in the repository: <file>

Missing-file findings have no location, so they count in check --all and are legacy in a change.

# From SEC-002: no .env files, but examples are fine
checks:
  - evaluator: files
    forbid: ["**/.env", "**/.env.*"]
    allow: ["**/.env.example", "**/.env.sample", "**/.env.template"]

# Commit a lockfile
checks:
  - evaluator: files
    requireAny: ["**/package-lock.json", "**/pnpm-lock.yaml", "**/yarn.lock"]
    message: No dependency lockfile is committed

dependencies

Forbids dependencies in package manifests.

Option	Type	Required	Description
forbid	list of strings	Yes	Dependency names or globs. At least one.
ecosystems	list	No	Limit to some of npm, maven, gradle, pip, go, cargo. Default: all six.
allowInstead	list of strings	No	Approved replacements. The fix reads Use <a> or <b> instead.
message	string	No	The finding’s message. Default: Forbidden dependency "<name>".

How to write names in forbid:

Ecosystem	Manifests read	Write names as
npm	package.json (dependencies, devDependencies, optionalDependencies, peerDependencies)	axios, @aws-sdk/*
maven	pom.xml (<dependency> blocks)	group:artifact, or artifact alone to match any group
gradle	build.gradle, build.gradle.kts (implementation, api, compileOnly, runtimeOnly, annotationProcessor, kapt, testImplementation, testRuntimeOnly, testCompileOnly)	Same as Maven
pip	pyproject.toml (PEP 621 dependencies and optional dependencies, Poetry dependency tables), requirements*.txt	The package name. Case and -, _, . don’t matter.
go	go.mod (require)	The module path, for example github.com/dgrijalva/jwt-go
cargo	Cargo.toml (dependencies, dev-dependencies, build-dependencies, target-specific tables)	The crate name

Each finding points at the line that declares the dependency. Evidence reads, for example, package.json declares request (npm), which matches "request". Lockfiles and node_modules/ aren’t read.

In a change, only manifests that changed are read. With changed-lines, adding a forbidden package is new; one that was already there is legacy.

# From TS-002
checks:
  - evaluator: dependencies
    ecosystems: [npm]
    forbid: [request, request-promise, request-promise-native, node-uuid]
    allowInstead: ["fetch", "crypto.randomUUID()"]

change-set

Rules about the shape of a change: when some files change, others must change too.

Option	Type	Required	Description
when	object	Yes	The rule applies when the change touches files matching this.
require	object	Yes	Then the same change must also touch files matching this.
message	string	No	The finding’s message. Default: This change needs <what>, for example This change needs a new migrations/**.

when and require each take:

Field	Type	Matches
changed	list of globs	Files added, modified, renamed or deleted.
added	list of globs	Files newly added.

Each needs at least one of the two (Set changed or added).

The finding points at the first file that triggered the rule, on its first changed line. The evidence lists Changed: <files> and Missing: <what>.

# From PROC-004: entity changes include a migration
checks:
  - evaluator: change-set
    when:
      changed: ["src/main/**/entity/**/*.java"]
    require:
      added: ["src/main/resources/db/migration/V*__*.sql"]

A change is needed to compare. Without --base, check compares your uncommitted work, including untracked files, with the last commit. With --base origin/main, it compares everything since the branch left origin/main. With check --all, there is no change, so the check is not evaluated.

Caution

The standard’s scope.paths limits which changed files the check sees. If you set paths, make sure they cover both the when files and the require files, or leave paths out.

semgrep

Runs Semgrep rules and reports their matches.

Option	Type	Required	Default	Description
rules	string	Yes		A Semgrep rules file, or a registry config such as p/owasp-top-ten (any p/, r/ or s/ config, or auto).
timeoutSeconds	integer	No	300	Stop Semgrep after this long. 1 to 1,800.

A relative rules path resolves against the folder of the standard’s file. Keep rules files outside the standards folder, for example in .groundrule/rules/. Every YAML file in .groundrule/standards/ is loaded as a standard, and a Semgrep rules file isn’t one.

.groundrule/standards/AUTH-017.yaml

checks:
  - evaluator: semgrep
    rules: ../rules/auth-017.yaml

Semgrep must be installed where check runs. If it isn’t, the check is not evaluated, with the reason Semgrep is not installed. Install it with `brew install semgrep` or `pipx install semgrep`. groundrule doctor reports Evaluator semgrep is unavailable in that case.

Groundrule runs semgrep scan --json --quiet --metrics=off --disable-version-check on the files in scope, 400 files at a time. Each result becomes a finding:

• Location: the result’s file and lines.
• Message: the Semgrep rule’s message, or Matched Semgrep rule <id>.
• Evidence: Semgrep rule <id> and the first line of the matched code.
• Confidence: high. Set minConfidence: certain on the check to report Semgrep results as concerns that never fail.

If the rules file doesn’t exist, the check is not evaluated, with semgrep failed: Semgrep rules not found at <path>. Registry configs need network access.

llm (not yet)

AI checks are not yet available. The llm evaluator exists so that standards which plan for one still load: it accepts a question or prompt option, and the check is always reported as not evaluated. The standard’s other checks still run.

checks:
  - evaluator: llm
    minConfidence: high
    question: >
      Does any new state-changing endpoint reach business logic without a
      server-side authorization check?

Keeping false positives down

A check that flags correct code teaches people to ignore it. In order of preference:

1. Narrow the standard’s scope. scope.paths and scope.exclude apply to every check and to agent files. For example, exclude: ["**/*.test.ts", "scripts/**"].
2. Narrow one check. regex takes its own include and exclude.
3. Tighten the pattern. Anchor it (^, \b), match the call and not the word, and test it on both examples (see below).
4. Record known false positives in quality.knownFalsePositives, so people adopting the rule know what to expect. explain shows them.
5. Grant an exception for code that is right to break the rule, with a reason and an expiry. See Overrides and exceptions.
6. Start the rule early. In a connected workspace, publish it at Observe or Advise, look at the evidence, and move it to Enforce when it is clean. See Rollout stages.

For pack rules, the catalog rates each rule’s noise (low, medium, high) and recommends a starting stage. See the Packs reference.

Test a check against its examples

There is no separate test command. Use check on a scratch file. Untracked files count as part of your change.

1. Write the forbidden example into a file the standard covers.

   printf 'import Stripe from "stripe";\n' > src/http/scratch.ts

2. Run only that standard.

   npx @groundrule/cli check --only ACME-004

   You should see the finding, and an exit code of 1 for a blocker:

    ✕ ACME-004  Route all Stripe calls through the gateway
      BLOCKER · deterministic (regex)
      src/http/scratch.ts:1
        Stripe imported outside the payments gateway
        1: import Stripe from "stripe";

3. Replace it with the approved example and run the check again.

   printf 'import { createPaymentIntent } from "../payments/gateway";\n' > src/http/scratch.ts
   npx @groundrule/cli check --only ACME-004

   You should see Passed.

4. Run it on the whole repository to see what it flags in existing code.

   npx @groundrule/cli check --all --only ACME-004

5. Delete the scratch file.

   rm src/http/scratch.ts

If the forbidden example passes, check the file is inside scope.paths, the language matches scope.languages, and the pattern is in single quotes. npx @groundrule/cli explain ACME-004 shows the scope as the CLI read it.

In the dashboard, Describe a rule tests the check it writes against its examples before showing it to you. See Describe a rule.

Related

• The rule format
• Check your changes
• Output formats