Groq reportedly plans DRAM stacking in next inference chip to offset SRAM limits

Groq plans to stack DRAM in its next inference chip to overcome the small capacity of SRAM, according to industry sources cited by a post from Rohan Paul on X. The post says the 3DS design adds capacity mainly at ordinary DRAM speeds, while hybrid bonding promises bandwidth closer to SRAM.

The post also states that Nvidia paid $20 billion last December to license Groq's technology and hire its leaders, so it describes this chip as effectively Nvidia's next LPU. This is a single secondhand report, and no Groq or Nvidia statement is quoted.