Hot Chips starts this Sunday. Here are the sessions I’m excited about and will be attending. Starting with the one I have a stake in.
We at d-Matrix have been making the rounds presenting our 3D DRAM work, first at ISCA in June, and now at Hot Chips. On Sunday we get into the weeds, covering our next-generation AI accelerator Raptor, how 3D DRAM integrates with the tensor engines, the memory bank architecture, projected performance and power numbers, and the kinds of challenges we've had to account for. It's part of the memory tutorial, co-presented with Meta. If you're at Hot Chips, you shouldn't miss this one.
I'll also publish an article shortly after, deriving some of these numbers from first principles so the material is accessible to a wider audience. Make sure you're subscribed.
And if you’re at Stanford for the conference and would like to meet, drop me a line.
Alright. Regarding the conference itself, it's action packed, as always. There's an underlying theme to how the sessions are organized, even if the program doesn't call it out.
Day 0: All About Memory
Sunday, Aug 23
Micron’s director of HBM design opens the memory session with where memory architecture goes next. Samsung follows on how the HBM base die evolves once it moves to a leading logic process. SK hynix covers advanced packaging for HBM. d-Matrix presents the 3D-DRAM work I described above. And Oxmiq Labs, the company founded by Raja Koduri, closes with HBF in AI compute.
Day 1: CPU & GPU Architecture
Monday, Aug 24
NVIDIA has two, the Vera CPU and the Rubin GPU. If past conferences are any indication, both should give us a lot of details.
AMD also has two, one on the MI400 GPU architecture and a second on the system architecture around it.
Intel has three, the Crescent Island GPU, Diamond Rapids (Xeon CPU), and Wildcat Lake (mobile and PC CPU).
Day 2: Accelerators & Connectivity
Tuesday, Aug 25
Norman Jouppi will be presenting Google's eighth generation TPU family. Norm, in my opinion, has played as important a role in shaping modern computing as Jeff Dean. So when he talks, we listen.
Then, sessions on Bluefield-4 DPU and Spectrum-X followed by a packed two hours on Meta’s AI chip, Microsoft MAIA 200, the NVIDIA Groq LPU, and Cerebras.
The final presentation of the conference is from OpenAI. The details around their Jalapeño inference chip, announced in June, were quite thin, with a full technical report promised in the months after. I’m hoping this is it.
Jalapeño was co-developed from initial design to manufacturing tape-out in just nine months, and the custom AI accelerator program represents what we believe to be the fastest ASIC development cycle ever achieved in high-performance advanced semiconductors.
See you at Hot Chips!
All the Best!! Subbu
Yeah I'm looking forward to the HBM tutorials at the beginning. I think ultimately the HBM architecture will need to change to be better suited for tighter integration. I wrote a piece already on the signal integrity packaging constraints for HBM5 so it would be good to add more knowledge on top of that.
https://www.siliconcodesign.com/p/a-comprehensive-deep-dive-into-high