OpenAI is currently struggling to maintain its pace of innovation, despite its success with a new AI model called Orion.

According to a report from The Information, OpenAI’s latest AI model, Orion, achieved performance levels comparable to GPT-4 after just 20% of its training. However, the quality gap between Orion and GPT-5 is expected to be significantly narrower than previous leaps from GPT-3 to GPT-4.

OpenAI employees shared that while Orion performed better at language tasks, it did not excel at tasks like programming.

Large improvements often occur in the early stages of AI training, while later stages yield smaller performance gains. This makes it difficult for the remaining 80% of training to achieve the expected significant breakthroughs.

Orion’s arrival comes at a crucial time, as OpenAI recently raised $6.6 billion in a funding round. With this funding, OpenAI faces rising investor expectations and significant technical challenges in scaling its technology. If early versions of its new model don’t live up to expectations, OpenAI’s ability to raise future funding could be hampered.

Illustration of the law of diminishing returns in developing successive versions of AI. Source: OpenAI

OpenAI's current challenges reflect a broader problem across the AI ​​industry: a shortage of high-quality training data and the pressure to stay relevant in an increasingly competitive field.

A study published in June predicted that the human-generated text data pool will run out between 2026 and 2032, making it difficult for AI companies to mine more from public data. The Information notes that developers have “almost exhausted” the data to power AI breakthroughs in recent years.

To address these issues, OpenAI is considering a completely new strategy for AI development. The Information notes that the AI ​​industry is shifting its focus to improving models after the initial training phase, potentially creating a different approach to scaling.

As OpenAI faces these challenges, it will be important to balance innovation and practical application to meet investor expectations. However, the departure of many senior staff may make it more difficult for OpenAI to find stability and future growth.