https://x.com/i/article/2109673117538783232
Own System One decisions with DiffusionGemma
Everyone is talking about System 1 models like Jev… “Jev is so hot right now.”
The system-one-style classification is showing up in a ton of places already, from agentic tool orchestration to data pre-processing. Here’s my mental stack of options for how to get any work done:
1. Write deterministic code if you can
1. Use XGBoost or a traditional ML algorithm if you can
1. 🆕 Use a system one style decision model if you can 🆕
1. Use a (system two) LLM or SLM
You may even use all of these systems together, preprocessing data for a feature based multi-arm-bandit or offloading ambiguous decisions that are not yet encoded in deterministic code, exposed as tools for a LM agent. It’s a fun time to be alive.
By Hussain Chinoy (@ghchinoy) and Alan Blount (@zeroasterisk)
My teammate Hussain took that spark and asked: what happens when we build a zero-shot decision engine on Google DeepMind's open-weights DiffusionGemma (26B-A4B-it), and push it until its probabilities are invariant enough to gate real production traffic? 🤯
The result is a compelling personal project dgem (Apache 2.0) and an interactive Decision Studio (dgem serve on your laptop). Along the way, Hussain uncovered a fundamental blindspot in how bidirectional diffusion models score choices. His first answer was Invariant Decision Calibration (IDC), which we've continued to improve, measure at scale, and evolve into what dgem ships today: hesitation-gating.
You can use gdem today, deploy it, and optionally fine-tune it, it’s open source on open weights. Read on to learn how we landed on these examples, and stay tuned because we have even more coming soon.
1. One pass vs. word-by-word: Why left-to-right models drift
Watch what happens when a standard chat LLM (generating text word-by-word, left-to-right) triages a compound enterprise support ticket:
"URGENT: Production API returning 502 Bad Gateway for 40 mins. If not resolved in 15 mins we will dispute our USD 45,000 Q3 enterprise invoice and cancel renewal."
To fill out three structured fields (urgent, urgency_score, and department), a word-by-word chat model has to generate 25+ tokens sequentially (2,480 ms). Worse, as it writes from left to right, its attention drifts from the "502 Bad Gateway" outage at the start of the ticket to the "USD 45,000 invoice" dispute at the end, locking in urgency_score: 4 and department: Billing. Once a word-by-word model commits to an early token, it cannot go back.
DiffusionGemma works like a camera sensor rather than a typewriter. Instead of writing one word at a time, dgem turns your JSON schema into a fill-in-the-blank template ([MASK] slots) and resolves every blank simultaneously in a single pass (~125-450 ms):
- urgent: "yes" (99.7% confidence, clear-cut)
- urgency_score: "5" (94.3% confidence, clear-cut)
- department: "Technical" (75.5%) vs. "Billing" (24.5%), Hesitation 51%, Very unsure, because the ticket mixes a live API outage with an invoice dispute
Because DiffusionGemma looks at the entire ticket and all three blanks at the same time (bidirectional attention), the fields stay consistent, and you get an exact probability split for every choice from that single forward pass.That third field is the interesting one: the model is telling you it's unsure. Hold that thought.
2. Why raw confidence & Shannon entropy are not enough
Most routers assume that if Shannon entropy (a measure of how spread out the model's probabilities are across the choices) is near zero, the model is certain and can fast-exit.
Hussain tested that assumption against ChaosNLI, JevBench, the Jev Decision Index (by multimodalart / apolinario) and a 50-case adversarial calibration suite, and found a silent failure mode: Ballot-Order (Box A) Primacy Bias. Basically, the order of the options shouldn’t be why the decision is made.
When you feed DiffusionGemma a 100% content-free blank prompt ([REDACTED / CONTENT-FREE CALIBRATION PROBE]) with options A and B, it does not output 50% / 50%. Because of instruction-tuning biases toward the first option slot, the raw unmasked logits assign:
- 2 choices (A or B): 88.3% to Slot 'A' vs. 11.7% to Slot 'B' (nearly 9-to-1 bias before reading a word)
- 3 choices (A, B, or C): 78.3% to Slot 'A' (more than 2x a fair 33.3% split)
- 4 choices (A, B, C, or D): 49.3% to Slot 'A' (nearly 2x a fair 25.0% split)
On genuinely ambiguous items in ChaosNLI, that Box A habit can make a toss-up look more confident than it is: in a small probe, just reordering the options changed the answer on 2 of 4 items. A naive entropy gate would wave some of those straight through, triggering a False Early Exit, even though reversing the option order flips the model's answer 50% of the time (2 out of 4).
3. Increasing speed and confidence with Invariant Decision Calibration (IDC)
To make confidence scores trustworthy without giving up single-pass speed, Hussain built Invariant Decision Calibration (IDC) into dgem:
Zeroing the scale (Null-Prior De-Biasing, --null-prior-debias): Just like pressing TARE on a kitchen scale before weighing flour, dgem measures the model's blank-prompt Box A bias once and subtracts it (0.1 ms) before scoring your prompt. In our experimentation it:
- Cuts permutation Multi-Class Brier error by 90.2%
- Lifts 50-case guardrail accuracy from 88.0% to 90.0% (Toxicity 83.3% → 100.0%)
Asking both ways in one pass (Dual-Mirror Canvas, --dual-mirror): In a word-by-word LLM, testing forward (A..D) and reversed (D..A) option orderings costs 2x the forward passes. Because DiffusionGemma unmasks all [MASK] slots in parallel, dgem places both forward and reversed choices on the same canvas at 0 ms extra GPU latency (125.0 ms vs. 129.0 ms). In our experimentation it:
- Eliminates option-reversal answer flipping (from 50% down to 0.0% flip rate).
- Catches hidden 50/50 toss-ups when a single pass looks falsely confident (the forward-vs-reverse gap jumps 66.8x, flagging it for handoff).
On small probe sets, both looked great. Then we did what you should always do: re-ran everything at a larger scale (231 JevBench items, repeated runs, run-to-run noise measured in the same session). The picture got more nuanced. The corrections didn't improve decisions reliably, and in one case simply naming the mirrored question nudged the model's answers on its own. So IDC kept evolving: we kept what held up, retired what didn't, and what came out is hesitation-gating, which we continue to refine with every release.
Hesitation-gating answers directly when the model is clear, with hand off to a larger model or a person when it hesitates, passing dgem's probabilities along as a hint.
- Hesitation %: entropy scaled to 0–100% so a 2-option and a 26-option question share one scale. The Studio labels it Clear (<16%), Somewhat unsure (16–50%) or Very unsure (>50%), like the department field above. On its own, hesitation separates wrong answers from right ones with an AUROC of ~0.85 (231 JevBench items).
- It works live: with a 35% threshold fixed in advance, handing hesitant items to gemini-3.8-flash scored 81/100 on safety and 86/100 on faithfulness while handing off only 8–32% of items. The fast path ran at a median 105 ms vs. 3.1 s for Gemini, at roughly $0.02 vs. $1–2 per 1,000 decisions (13 judge suites, n=100 each).
- Offline on JevBench: handing off the 39% of items above 16% hesitation lifted dgem from 187–189 to 221 of 231 (Gemini answering every item: 225). That threshold was chosen on the same items, and this is adjustable.
IDC's lessons now live in the product:
- Order bias is measured and monitored. Every release shuffles option order across several benchmark suites and reports the extra flips against a measured noise floor; hesitation already catches most of the cases where it matters.
- Calibration is per domain. The best temperature ranges from about 1.2 to 3.6 depending on the domain, so you fit one per policy on held-out decisions by configuration instead of trusting a single global value or having to resort to fine tuning.
- The template is part of the prompt. Use neutral question ids and don't reuse option letters across questions; both had measurable effects and are now template rules.
- Prompt layout matters. Putting the input before the questions (the default since v0.2.0) lifted held-out XNLI from 0.662 to 0.699 (n=4,500) and typed decisions from 0.674 to 0.728 (n=2,000).
- Wide taxonomies. A 2-stage tournament bracket gets past the 26-options-per-question limit, up to 255 choices; on a 22-request Decision Index panel it raised coverage from 16/22 to 22/22.
The null-prior and dual-mirror switches remain in the CLI as opt-in research tools.
4. Use cases
Classification models aren't new, but this current interest and newly improved capabilities is a great time to think about where we should be employing “system one”-style fast decision models.
A few I’m currently considering:
- Agent’s LLM context window pre-compiler: Before waking up a frontier System 2 LLM or stuffing raw user inputs into a prompt, you can run a single-pass System 1 model to score intent, urgency, and whether required context is even present (for example, flagging missing_sender_metadata on a suspicious email before an agent wastes a turn guessing).
- Agent’s tool invocation and result handling orchestration, prior to LLM turns: Two patterns have already taken off on X around Jev-style models: using them as "Semantic if statements" inside deterministic code, and filtering large MCP tool catalogs (narrowing 50+ available tools down to the 2 or 3 relevant schemas before the LLM turn starts so tool definitions do not bloat the context window). You can run the exact same fast check on both sides of a tool call: gating whether a proposed tool invocation is grounded and safe before it executes, and triaging or pruning noisy tool outputs (log dumps, search results, API payloads) before they land back in the agent's conversation history.
- Knowledge Graph or LLM wiki entity extraction, resolving natural language, or any input schemas to ontologies: Turning messy notes, meeting transcripts, or arbitrary third-party JSON/CSV schemas into a clean Knowledge Graph or structured LLM wiki usually stalls on entity resolution and taxonomy mapping. Instead of paying a generative LLM to rewrite every record word-by-word, a bidirectional template can classify candidate entities, map arbitrary source fields to your canonical ontology types, and score relationship edges (depends_on, supersedes, owned_by) in one parallel pass. When your ontology has dozens or hundreds of classes, dgem's 2-stage tournament brackets let you resolve up to 255 target categories without blowing up latency. And when your ontology is missing a class, dgem's taxonomy discovery (--suggest-expansions) proposes a new option instead of forcing a bad fit.
- Policy engines: Building on top of the prior example, once you have an ontology, you can use it for security. Because System 1 models restrict output probabilities strictly to the choices in your policy template (such as "allow", "deny", or "escalate"), untrusted text inside a payload cannot prompt-inject the classifier into generating free-form instructions. And with Invariant Decision Calibration (Null-Prior Tare + Dual-Mirror check), the policy engine knows when it is not confident enough: when forward and reversed option orders disagree, it refuses to guess and automatically asks for more detail from another system, a System 2 model, or a human in the loop.
- Decision decomposition: A few more prototypes I have seen involve decomposing more complex design problems to a series of atomic decisions. There is clearly an opportunity to over-reach here and it might be incorrect to reach for a System 1 models when a System 2 model is better for planning and design, but building on the idea of “Semantic if statements” we can apply a consistent and structured process to ask a bunch of small questions quickly and get good results fast.
5. What's next
dgem is zero-shot today: you change its behavior by editing a policy template, with no retraining. Next we're looking at sharper per-domain calibration, more decision templates, and, for domains where zero-shot accuracy plateaus, lightweight fine-tuning (for example, small adapters focused on the answer readout). Like everything so far, it will be measured against this base accuracy and shipped only if it beats zero-shot.
Wanna give it a spin?
Try Hussain's interactive 4-Scene Concept Walkthrough in the Decision Studio (dgem serve) by cloning this Github repo to run DiffusionGemma (26B-A4B-it) on Cloud Run GPUs (NVIDIA L4 / RTX Pro 6000) or locally on Apple Silicon Metal.
What are you working on?
Share your demos below. 👇