Posts on Daniel Park

a learning journey through ML systems

Wed, 24 Jun 2026 00:00:00 +0000

As I approach a little over a year since formally starting my journey of self-learning and stumbling through the world of ML systems, I wanted to look back and recap / document my entire roadmap and the insane rabbit holes I’ve fallen through during the past year, and also outline what my goals are for the next couple months.

The real story started in my sophomore year of college, when I started my project GroundUpNeuralNet, a very minimal implementation of an MLP library I wrote in C++ after taking a DSA class. At this point, I wasn’t very concerned (or knowledgeable) about performance, so I leaned heavily on Eigen for the compute. Nevertheless, I still learned a good deal about multivariate calc, backprop, and just generally how to structure a relatively big C++ project. You can still play around with it today here , although it only supports linear layers and a handful of activation + loss functions.

CuTe-ly writing an SM86 SGEMM

Thu, 05 Feb 2026 00:00:00 +0000

I know what you’re thinking. A fp32 GEMM in 2026? For Ampere??

Yes, SGEMM is rarely ever used these days in the ML space given the massive compute throughput gap between CUDA and tensor cores. For reference, an A100 PCIe’s (now almost 6 years old) peak theoretical performance for bf16 is 312 TFLOPs/sec. and 156 TFLOPs/sec. for tf32, while fp32 sits at a “measly” 19.5 TFLOPs/sec. With mixed-precision training and quantized inference being the norm, GEMM itself is practically never done in fp32 today.

Image Rendering Test

Tue, 16 Jan 2024 22:30:00 +0800

Image Rendering Test

The Demo comes from LightGallery All images are from Unsplash

Single Image

Photo by - Daniel Leone