CuTe-ly writing an SM86 SGEMM

Thu, 05 Feb 2026 00:00:00 +0000

I know what you’re thinking. A fp32 GEMM in 2026? For Ampere??

Yes, SGEMM is rarely ever used these days in the ML space given the massive compute throughput gap between CUDA and tensor cores. For reference, an A100 PCIe’s (now almost 6 years old) peak theoretical performance for bf16 is 312 TFLOPs/sec. and 156 TFLOPs/sec. for tf32, while fp32 sits at a “measly” 19.5 TFLOPs/sec. With mixed-precision training and quantized inference being the norm, GEMM itself is practically never done in fp32 today.

GPU on Daniel Park

CuTe-ly writing an SM86 SGEMM