OpenAI uses automated red teaming to train GPT-6 jailbreak resistance
Start here
OpenAI researchers report that automated red teaming, in which a GPT-5.6 Sol reasoning model attacked GPT-5.4 and GPT-5.5 defenders, produced synthetic jailbreaks that were then used as reinforcement-learning training data for GPT-6 Astra and Sol. The aim was to make the models resist jailbreaks in high-stakes domains such as cyber, biology and violent activity. On fixed jailbreaks drawn from earlier attacks against GPT-5.4 and GPT-5.5, GPT-6 Astra answered 91.5% of cyber-related prompts safely, compared with 59% for GPT-5.6 Sol, and it stayed more robust across all attacker budgets in an adaptive multiturn evaluation. OpenAI says the robustness training did not increase refusals of benign tasks, citing a Pareto improvement on safety and helpfulness in the GPT-6 Astra system card. OpenAI notes that in deployment, model safety training is only one layer of a broader safeguards system.
Generated from reports · updated 13 h ago
Report timeline
Follow the reports to see the event from different sides.
- OpenAI Alignment BlogOpenAI uses automated red teaming to improve fuzzy robustness in GPT-6 Astra and Sol
OpenAI researchers describe how automated red-teaming, using a GPT-5.6 Sol family reasoning model as attacker against GPT-5.4 and GPT-5.5 defenders, generated synthetic jailbreaks that were used as RL training data for GPT-6 Astra and Sol. On fixed jailbreaks from prior attacks against GPT-5.4 and 5.5, GPT-6 Astra answered 91.5% safely on cyber versus 59% for GPT-5.6 Sol, and it stayed more robust across all attacker budgets in an adaptive multiturn eval. OpenAI reports the robustness training did not raise refusals of benign tasks, citing a Pareto improvement on safety and helpfulness in the GPT-6 Astra system card.
Heat for this event
Not enough continuous observations yet to draw a trend.