TPU

TPUs are Google's custom-designed ASICs built specifically for accelerating neural network matrix computations. They are optimized for the systolic array operations common in transformer training and inference, offering high throughput for large batch matrix multiplications with high memory bandwidth.

Google uses TPU pods to train Gemini and other frontier models, and TPUs are available via Google Cloud.