OpenAI uses automated red teaming to improve fuzzy robustness in GPT-6 Astra and Sol

OpenAI researchers describe how automated red-teaming, using a GPT-5.6 Sol family reasoning model as attacker against GPT-5.4 and GPT-5.5 defenders, generated synthetic jailbreaks that were used as RL training data for GPT-6 Astra and Sol. On fixed jailbreaks from prior attacks against GPT-5.4 and 5.5, GPT-6 Astra answered 91.5% safely on cyber versus 59% for GPT-5.6 Sol, and it stayed more robust across all attacker budgets in an adaptive multiturn eval. OpenAI reports the robustness training did not raise refusals of benign tasks, citing a Pareto improvement on safety and helpfulness in the GPT-6 Astra system card.