With OpenAI and other tech giants rolling out their own chips, analysts have begun to believe that $NVIDIA (NVDA.US)$'s near-monopoly position in the advanced AI chip arena is facing a “threat.”
On Tuesday (August 25), OpenAI disclosed technical details of the company’s first custom inference chip, Jalapeno, claiming that the chip’s response speed and energy efficiency performance have surpassed NVIDIA’s GB300. At present, $Google-C (GOOG.US)$, $Amazon (AMZN.US)$, $Meta Platforms (META.US)$, and others are also developing their own AI chips.
Yole Group technology analyst Adrien Sanchez said Jalapeno is a chip designed specifically for inference, indicating that “chips designed by ultra-large cloud service providers (hyperscalers) can now match—and even exceed—NVIDIA Blackwell-class GPUs in inference energy efficiency.”
Sanchez said that although NVIDIA currently still holds the “vast majority” of the AI compute market and has used the CUDA software platform to create a strong ecosystem lock-in, OpenAI’s new chip “poses a threat to NVIDIA’s inference business profit margins—especially since inference is precisely the fastest-growing segment right now.”
OpenAI has always been an important NVIDIA GPU customer, buying large volumes of NVIDIA chips to train and run big AI models. Now that it has its own chips, it could change the cooperation relationship between the two.
Omdia senior chief analyst Alexander Harrowell commented that Jalapeno is an “impressive achievement, particularly in terms of energy efficiency.” With large-scale deployments, the chip will save on electricity and cooling costs, reduce demand for power distribution infrastructure, and significantly improve unit economics.
TrendForce analyst Fion Chiu said OpenAI’s custom chips may gradually reduce the company’s reliance on NVIDIA for inference tasks.
However, Chiu still believes that, “given that NVIDIA GPUs offer broad programmability, strong performance, a mature software ecosystem, and the ability to handle a wide range of workloads, NVIDIA GPUs will continue to play an important role.”
Research firm SemiAnalysis said that its team visited the OpenAI lab and ran benchmark tests on Jalapeno. The results showed that in almost all test scenarios, Jalapeno’s performance per watt outperformed NVIDIA’s Blackwell.
However, SemiAnalysis also noted that this comparison “has a certain degree of incompleteness and unfairness,” because Jalapeno uses a newer generation of HBM4 memory, making it a better apples-to-apples comparison with NVIDIA’s Rubin platform.
A SemiAnalysis article also pointed out: “The Vera Rubin system has already started shipping to customers, while OpenAI still needs some time to move Jalapeno from the engineering sample stage to a more mature product.”
In recent years, Google has launched new chips for AI training and inference and calls them Tensor Processing Units (TPUs). Meta has also said it has agreed to adopt custom AI chips based on Broadcom technology.
Anthropic said it will commit more than $100 billion over the next 10 years to investing in AWS technology, including current and future versions of Amazon’s in-house AI chip Trainium.
Omdia analyst Harrowell said it expects that by 2028, the shipments of these custom ASIC chips will exceed those of GPUs. However, because GPU prices are clearly much higher, it will take longer for ASICs to surpass GPUs in revenue scale.
“This is the biggest competitive threat facing NVIDIA, because about half of AI infrastructure capital expenditures come from hyperscale cloud service providers, and these companies either already have in-house chip development efforts or have the complete capability to build such projects from scratch.”
In addition, startups such as Cerebras, SambaNova, D-Matrix, Etched, and Fractile are also developing AI chips.
