Meta's JiTTests generate bug-catching tests before code lands, reports 74 privacy bugs caught
Overview
Meta says its Catching Just-in-Time Tests (JiTTests) have moved from research prototype into production, generating executable tests that fail on code changes before they land.
Two generation workflows produce the tests, and an LLM-as-Judge ensemble plus a rule-based assessor filter out weak catches before engineers review results in aggregate.
Meta reports that 589 engineers, 64% of those it reached out to, have engaged with the system. Engineers report a 60% true-positive rate, according to Meta. Meta also says the system helped catch 74 serious privacy bugs in the first half of 2026 and lowered privacy SEVs by 20%.
Written by AI from the articles below · updated Oct 29, 12:25 PM ET
Check the sources:
Article timeline
The articles in this story. Times are ET.
- Engineering at MetaOfficialPickMeta details how Catching JiTTests generates bug-finding tests before code lands
AIMeta says its Catching Just-in-Time Tests (JiTTests) have moved from research prototype into production, where they generate executable tests that fail on code changes before they land. The system uses two generation workflows and filters weak catches with an LLM-as-Judge ensemble and a rule-based assessor, then has engineers review results at an aggregate level. Meta reports that 589 engineers engaged with it, with a 60% true-positive rate, and that it helped catch 74 serious privacy bugs in the first half of 2026, lowering privacy SEVs by 20%.
Heat Trend
Not enough continuous observations to show a trend yet.