OpenAI revises Astra benchmark scores amid rollout glitches and competitor comparisons

OpenAI’s announcement of the GPT-6 Astra model on Sept. 3 experienced a rollout problem, with the blog post delayed and initially inaccessible due to technical errors. After the page became viewable, the firm posted revised evaluation numbers that made Astra appear stronger on several benchmarks, including a halved hallucination rate and higher math scores, while Anthropic’s competing model saw its scores drop. The company later adjusted the same metrics again, restoring earlier values for hallucination rates and tweaking other figures such as cybersecurity and coding performance.

OpenAI said the changes reflect “best-estimate” conditions and acknowledged that evaluation noise can vary by checkpoint and setup. Critics from Stanford and Snorkel AI warned that such rapid re-running of tests could amount to “benchmaxxing,” complicating the interpretation of leaderboard results that influence market perception and investor confidence.