OpenAI 揭秘 GPT-5.6 Sol 在 ARC-AGI-3 低分原因:评估 harness 限制,并推荐 retain reasoning 与 compaction
“GPT-5.6 Sol has been used to solve open problems in mathematics. So why was it struggling with ARC-AGI-3, a benchmark of 2D puzzle games? We investigated. The harness was not letting it remember what it had learned. We found that enabling two API settings tripled our scores with 6x fewer output tokens.”
API 开发者应使用 Responses API(而非遗留 Chat Completions)、开启 retain reasoning 与 compaction 来最大化 agent 持续推理能力。这对部署 GPT-5.6 系列 agent 的生产级系统有直接指导意义。