Skip to content
View original post on X: Santiago
Published
22/100

Deterministic tests beat written instructions for enforcing agent rules

AISantiago argues that anything checkable should be enforced by deterministic tooling rather than written instructions. He lists unit and integration tests for behavior, formatters and linters for formatting, static type checkers for types, schema validation for API contracts, and static analysis and security scans for security.
SantiagoVerified on X
@svpino
View on X

Deterministic tests > Written instructions

Anything that should be checked automatically should have a test case:

  • Checking behavior → Unit and integration tests
  • Checking formatting → Formatters and linters
  • Checking types → Static type checkers
  • Checking API contracts → Schema validation
  • Checking edge cases → Targeted tests
  • Checking regressions → Regression tests
  • Checking security → Static analysis and security scans

Don't waste tokens telling an agent to follow rules you could enforce with deterministic tests.

Source details

Santiago · x.comPublished · added here

Original title: Deterministic tests > Written instructions