TPUs: The custom chips that have been running Google’s AI for over a decade

Behind every Google search, every Gmail smart reply, every YouTube recommendation, there’s a chip that most people have never heard of. It’s called a TPU — Tensor Processing Unit — and Google designed it from scratch over ten years ago for one purpose: running AI models.

AI is basically math. A lot of it. The kind of matrix multiplications that would make a normal CPU cry for mercy. TPUs skip all the general-purpose overhead and go straight to crunching tensors. They’re not trying to run your spreadsheet or play a game. They exist to do one thing fast, and they do it absurdly well.

The numbers on the latest generation are frankly ridiculous. We’re talking 121 exaflops of compute power. That’s 121 quintillion floating-point operations per second. And the memory bandwidth? Double what the previous generation had. I remember when a teraflop was a big deal. Now we’re throwing around exaflops like it’s nothing.

What’s interesting is how long Google has been at this. Most companies jumped on the custom AI chip bandwagon in the last few years, but Google started building TPUs back when deep learning was still a niche academic pursuit. They’ve had a decade to iterate, and it shows. The current generation isn’t just faster — it’s smarter about how it moves data around, which matters more than raw compute numbers in practice.

If you want to see how these things actually work, Google put together a video that walks through the architecture. It’s worth watching if you’re into hardware. The short version: tiny dies, massive throughput, and a design philosophy that prioritizes sustained performance over peak benchmarks.

TPUs aren’t sexy. They don’t have RGB lighting or fancy brand names. But they’re the reason a lot of Google’s AI features work as well as they do. And with each generation, the gap between what’s possible and what’s practical keeps shrinking.