Skip to content
View original post on X: meng shaoX· Pick72/100AI score72/100

Meng Shao shares @poteto's four-layer framework for trusting agent work

AISummary

Meng Shao reposts and translates a framework from @poteto that ranks ways to trust AI agent output, with codebase improvements at the top and human review at the bottom. The framework argues that encoding correct behavior in architecture, types, and invariants works better than relying on prose rules and manual review.

AIWhy it matters

The post walks through a ranked framework for trusting agent output, which is useful for teams deciding where to spend review effort on AI-written code.

Post on XView on X
meng shaoVerified on X
@shao__meng

How can you trust your Agent's work more?

The Grok Bot team's @poteto proposes an "inverted pyramid" framework of four layers for trusting the output of AI Agents. She argues that rather than spending effort "managing" the Agent, it's better to spend effort improving the codebase itself!

Four-Layer Inverted Pyramid (effort investment decreases from top to bottom)

Layer 1: Codebase Improvement (highest investment) Better architecture, a stronger type system and invariants, clear module boundaries.

Lauren believes this is essentially "just good software engineering." The reasoning lies in the nature of Agents: Agents tend to copy existing code and take the shortest path. So rather than fighting this nature, make the shortest path itself the correct path.

Layer 2: Static analysis, Lint rules, tests, CI, formal verification (such as TLA+). This is a high cost-effectiveness second line of defense, using very low token cost to catch problems early, with error feedback that is deterministic and doesn't depend on the Agent "understanding" anything.

Layer 3: Textual guidance (used least). Code comments, rule files (such as AGENTS.md), Skills descriptions, etc. The common weakness of this layer is that it cannot be enforced; it requires the Agent to judge for itself whether to comply, and the Agent's judgment is not always reliable, so it can only serve as a supplement, used in situations that genuinely require subjective judgment.

Layer 4: Human review (hardest to scale). Human energy is limited and should not be consumed by checking the Agent's output line by line, but rather invested in more upstream matters: designing an environment where the Agent does the right thing by default.

Why is the pyramid "inverted"?
Traditionally, teams put a lot of effort into the bottom of the pyramid (writing specifications, manual review), while Lauren advocates reversing this:
· The higher the layer, the more deterministic the means (code structure directly determines which shortcuts the Agent can take) and the more token-efficient (no need for repeated communication to correct course);
· The lower the layer, the more it relies on unreliable understanding and unscalable human labor.

This is in the same direction as the industry's recent consensus of "shifting from prompt engineering to context engineering"; but Lauren pushes it to a more fundamental level: even prompts themselves are only third-tier means, and the real leverage lies in the codebase.

AITranslated from Chinese
lauren@poteto
how do you trust your agents' work more? copy this post/link and give it to your @bot or agent! 1. codebase improvements: this is just good software engineering - better architecture, stronger types and invariants, clear boundaries. agents love to copy what exists and take the shortest path, so make the easy thing the right thing. categorically eliminate problems through code and you won't need to think about it again. 2. static analysis: lint rules, tests, CI, verification (eg TLA+). very token efficient ways to catch issues ahead of time. 3. prose: comments, rules, AGENTS.md, skills - anything that can't be enforced and requires agent judgment. use it minimally, but do use /create-verification-skill. 4. human review: the hardest to scale. at the rate agents land code, it's pretty much impossible to thoroughly review every PR, so spend that effort on the highest leverage point, designing an environment where agents do the right thing by default. this forms an inverse pyramid: you want most of your effort at the top, least at the bottom. codebase improvements are deterministic, making it more token efficient, and lets you sleep more soundly at night. install https://github.com/cursor/plugins/tree/main/pstack and create a new Project: 1. look through my slack/chat messages and code review comments where i've given feedback on PRs, code, architecture 2. use /recall on my past transcripts to learn how i steer agents 3. /poteto-mode use /correct and /architect on past commits to find the most common pitfalls agents fall into, then propose a plan to rearchitect the code i own so it's more agent friendly. answer open questions with prototyping rather than defer to me, and back the plan with real data. 4. before you begin, use /sharpen + /in-your-own-words and confirm this prompt with me
View quoted post on X

Source: meng shao · x.comPublished · added here