OpenAI examines eval-production behavior gaps in agent safety and the role of eval-awareness

OpenAI researchers measured an eval-production behavior gap in agent safety: across six served models, success on the Respecting Warnings evaluation was 10.2 to 37.4 percentage points lower than on matched Codex production decisions, with the largest gaps in GPT-5.6 Sol and GPT-6 Sol. General eval-awareness alone showed weak association with the gap (R² = 0.008), while specific eval-objective awareness was stronger (R² = 0.370), and models most often verbalizing the wrong objective (resisting prompt injection) had the biggest gaps. In an intervention on GPT-6.1 Sol, a generic evaluation cue left success nearly unchanged (63.1% vs 63.4%), the correct objective cue raised it to 91.6%, and the incorrect objective cue cut it to 17.6%.