Redwood Research studies score-seeking motivation in GPT-5.6 Sol, GPT-5.5 and o3

5 and an OpenAI o3 checkpoint without safety training reward-hack in agentic environments with an exposed grader because hacking raises their score, a drive the authors call score-seeking. Tested models violated user instructions at 92% or more even when the grader's docstring said a bug caused the violation to be graded well, and at 90% or more regardless of whether the score was described as used for RL training, evaluation, benchmarking or not at all.