NVIDIA turns to Groq to fix the GPU inference gap
Summary: NVIDIA and Groq entered into a licensing arrangement that will see NVIDIA pay Groq to use and integrate its chip design and technology around AI inferencing. This is not an acquisition and Groq will continue to operate as an independent company. Groq will be bringing on a new CEO – Simon Edwards – as members of its senior management team, including founder and CEO Jonathan Ross, will be joining NVIDIA. Jonathan Ross was previously a hardware engineer at Google and helped design its proprietary TPU chips. The team will help NVIDIA develop and scale the technology that is being licensed. The licensing deal is reported to be valued at $20b.
Groq: Groq has been in the midst of a steady rise, but has been in business for about ten years. Just a few months back, it raised $750m in a Series D round with a valuation of $6.9b. Last year, Groq projected revenue to come in at $500m and expects to see that climb to $1.2b in 2026. These growth projections had been revised downwards due to a lack of data center capacity. Groq is a developer of chips – language processing units or LPUs – that are tailored and built from the ground up for AI inferencing workloads. The technology is said to run inferencing faster and more efficiently than standard GPU technology. A key part of Groq’s technology, and what NVIDIA is lacking and wants through this licensing deal, is the software, which is enabled by a unique compiler. Groq is also building out a cloud infrastructure arm for direct service delivery. This business is aiming to consume data center colocation.
Acquisition versus licensing: NVIDIA did not acquire Groq and chose to license the technology given concerns around anti-trust regulations. NVIDIA is able to avoid scrutiny but gain access to strategic technology while hiring away the DNA. While this is a licensing agreement it very much looks like an acquire-hire or acquisition.
Angle: NVIDIA dominates the market, but this partnership/deal recognizes the fact that it does not have a general-purpose GPU that can handle all the different aspects of AI infrastructure development. Inferencing is going to require different technology with specific requirements. NVIDIA is looking to lock down its control of the market by integrating technology and capabilities that will handle inferencing and keep everything within its ecosystem, rather than let things become disaggregated. Once AI takes off, inferencing will grow explosively, and the right technology at the right price point will be necessary. NVIDIA could take the internal development path, but is likely concerned about how long that will take. Partnering with Groq addresses that issue.
Cloud and data center impact: The partnership should also provide more resources to Groq for its cloud and data center infrastructure business. NVIDIA will likely become more closely involved in Groq’s cloud platform and that will translate directly into data center infrastructure demand.
About the author: Phil Shih is Managing Director and Founder of Structure Research, an independent research firm focused on the cloud, edge, AI and data center infrastructure service provider markets on a global basis.
Palo Alto Networks integrates zero trust security into NVIDIA AI factory
Article Topics
AI acceleration | AI Chips | AI/ML | cloud inference | Groq | LPU | Nvidia | semiconductors
Comments