CDAL
Back to All Articles

A Rule Without a Mutation Probe is a Decoration: Why Gates Need Selftest

2026-10-26
Michał Sznurowski

Engineering teams adopting AI coding agents frequently assemble extensive .cursorrules, AGENTS.md, or custom validator scripts containing dozens of assertions:

  • "Never cast with as any"
  • "Every finite state machine transition must define a guard"
  • "Never edit protected schema files outside contract windows"

This looks rigorous in documentation until you ask the fundamental question:

How do you know that this rule actually works and is capable of failing red?

1. The Phenomenon of Decorative Quality Gates

In real-world repositories, validator rules frequently degrade silently:

  1. A regular expression contains a subtle syntax error (e.g. unescaped quantifier or missing newline flag).
  2. The checked AST node was refactored three commits ago and no longer matches.
  3. The check was never executed against actual negative, broken code.

The result: broken code passes through CI with a cheerful green checkmark. Developers operate under the illusion of safety while their quality gates are purely decorative.

2. The CDAL Doctrine: Every Rule Must Have a Positive Mutation

In CDAL, every invariant rule in a ContractKind and every forbidden pattern must declare an explicit mutation probe:

// packages/kinds/src/state-machine.ts
{
  rule: 'STATE-REACHABILITY',
  file: 'state-machine.contract.mjs',
  find: "from: 'REQUESTED'",
  replace: "from: 'UNREACHABLE_ORPHAN'",
}

The cdal selftest command clones contracts and test files into an isolated environment, applies each mutation sequentially, and verifies that the validator fails with specifically that exact rule ID.

If a mutation does not trigger an error, cdal selftest fails the build immediately.

3. Negative and Evasion Probes

The same rigor applies to forbidden patterns. The selftest engine checks:

  • Positive probe: Does @ts-ignore in a production .ts file get caught?
  • Negative probe: Does legitimate text mentioning "ignore" get allowed without false alarms?
  • Gate evasion probe: Does an agent attempting to split strings (['@ts', 'ignore'].join('')) get detected?

Only when your gates pass mutation testing can you trust them to protect your repository.

A Rule Without a Mutation Probe is a Decoration: Why Gates Need Selftest — CDAL Blog