How to use decision models for evals

Hamel Husain tests decision models including Jev 1.13, Perplexity Decider 1.1 27B, Liquid d1 and d1-ultra, Cloudflare Clef Flash, and OpenAI Decisions (GPT-6 Luna) on 100 apartment-leasing agent traces. He argues the default 0.50 decision threshold can leave performance on the table and says thresholds should be tuned for precision, recall, and F1 before comparing cost.