Groq, a startup growing chips to run generative AI fashions sooner than standard processors, mentioned on Monday that it’s raised $640 million in a brand new funding spherical led by Blackrock. Neuberger Berman, Sort One Ventures, Cisco, KDDI and Samsung Catalyst Fund additionally participated.
The tranche, which brings Groq’s whole raised to over $1 billion and values the corporate at $2.8 billion, is a significant win for Groq, which reportedly was initially seeking to elevate $300 million at a barely decrease ($2.5 billion) valuation. It greater than doubles Groq’s earlier valuation (~$1 billion) in April 2021, when the corporate raised $300 million in a spherical led by Tiger International Administration and D1 Capital Companions.
Meta chief AI scientist Yann LeCun will function a technical advisor to Groq and Stuart Pann, the previous head of Intel’s foundry enterprise and ex-CIO at HP, will be part of the startup as chief working officer, Groq additionally introduced right now. LeCun’s appointment is a bit surprising, given Meta’s investments in its personal AI chips — nevertheless it undoubtedly offers Groq a strong ally in a cutthroat area.
Groq, which emerged from stealth in 2016, is creating what it calls an LPU (language processing unit) inference engine. The corporate claims its LPUs can run present generative AI fashions related in structure to OpenAI’s ChatGPT and GPT-4o at 10x the velocity and on-tenth the vitality.
Groq CEO Jonathan Ross’ declare to fame helps to invent the tensor processing unit (TPU), Google’s {custom} AI accelerator chip used to coach and run fashions. Ross teamed up with Douglas Wightman, an entrepreneur and former engineer at Google mother or father firm Alphabet’s X moonshot lab, to co-found Groq near a decade in the past.
Groq supplies an LPU-powered developer platform known as GroqCloud that provides “open” fashions like Meta’s Llama 3.1 household, Google’s Gemma, OpenAI’s Whisper and Mistral’s Mixtral, in addition to an API that permits prospects to make use of its chips in cloud cases. (Groq additionally hosts a playground for AI-powered chatbots, GroqChat, that it launched late final yr.) As of July, GroqCloud had over 356,000 builders; Groq says {that a} portion of the proceeds from the spherical will likely be used to scale capability and add new fashions and options.
“Many of those builders are at giant enterprises,” Stuart Pann, Groq’s COO, instructed TechCrunch. “By our estimates, over 75% of the Fortune 100 are represented.”
Because the generative AI increase continues, Groq faces growing competitors from each rival AI chip upstarts and Nvidia, the formidable incumbent within the AI {hardware} sector.
Nvidia controls an estimated 70% to 95% of the marketplace for AI chips used to coach and deploy generative AI fashions, and the agency’s taking aggressive steps to take care of its dominance.
Nvidia has dedicated to releasing a brand new AI chip structure yearly, somewhat than each different yr as was the case traditionally. And it’s reportedly establishing a brand new enterprise unit centered on designing bespoke chips for cloud computing corporations and others, together with AI {hardware}.
Past Nvidia, Groq competes with Amazon, Google and Microsoft, all of which provide — or will quickly provide — {custom} chips for AI workloads within the cloud. Amazon has its Trainium, Inferentia and Graviton processors, out there by AWS; Google Cloud prospects can use the aforementioned TPUs and, in time, Google’s Axion chip; and Microsoft just lately launched Azure cases in preview for its Cobalt 100 CPU, with Maia 100 AI Accelerator cases to come back within the subsequent a number of months.
Groq may take into account Arm, Intel, AMD and a rising variety of startups rivals, too, in an AI chip market that might be attain $400 billion in annual gross sales within the subsequent 5 years, in response to some analysts. Arm and AMD specifically have blossoming AI chip companies, due to hovering capital spending by cloud distributors to satisfy the capability demand for generative AI.
D-Matrix late final yr raised $110 million to commercialize what it’s characterizing as a first-of-its-kind inference compute platform. In June, Etched emerged from stealth with $120 million for a processor custom-built to hurry up the dominant generative AI mannequin structure right now, the transformer. SoftBank’s Masayoshi Son is reportedly seeking to elevate $100 billion for a chip enterprise to compete Nvidia. And OpenAI is mentioned to be in talks with funding corporations to launch an AI chip-making initiative.
To carve out its area of interest, Groq is investing closely in enterprise and authorities outreach.
In March, Groq acquired Definitive Intelligence, a Palo Alto–primarily based agency providing a spread of business-oriented AI options, to type a brand new enterprise unit known as Groq Techniques. Inside Groq Techniques’ purview is serving organizations, together with U.S. authorities companies and sovereign nations, that want to add Groq’s chips to present knowledge facilities or construct new knowledge facilities utilizing Groq processors.
Extra just lately, Groq partnered with Carahsoft, a authorities IT contractor, to promote its options to public sector shoppers by Carahsoft’s reseller companions, and the startup has a letter of intent to put in tens of hundreds of its LPUs at European agency Earth Wind & Energy’s Norway datacenter.
Groq can be collaborating with Saudi Arabian consulting agency Aramco Digital to put in LPUs in future datacenters within the Center East.
On the similar time it’s establishing buyer relationships, Mountain View, California-based Groq is marching towards the subsequent technology of its chip. Final August, the corporate introduced that it could contract with semiconductor agency International Foundries to fabricate 4nm LPUs, that are anticipated to ship efficiency and effectivity features over Groq’s first-gen 13nm chips.
Groq says it plans to deploy over 108,000 LPUs by the top of Q1 2025.