Meta details production deployment of Catching JiTTests with human-on-the-loop review

Meta has moved Catching JiTTests, its LLM-based just-in-time test generation approach, from research prototype into production infrastructure, and published the results in the FSE 2026 paper "Just-in-Time Catching Test Generation at Meta." The system runs overnight on spare capacity, targets risky diffs, generates tests through dodgy-diff and intent-aware workflows, and filters false positives with an LLM-as-Judge ensemble plus a rule-based RubFake assessor before reaching engineers. As of writing, 589 engineers (64% of outreach) engaged with the results, which showed a 60% true-positive rate, and in the first half of 2026 the system caught 74 serious privacy bugs at the diff stage and lowered privacy SEVs by 20%.