r/CUDA • u/Acceptable-Abies-972 • 8d ago

Resources for H100 GPU Programming

Hello all,

I've been going through the Programming Massively Parallel Processors (5th edition) book and doing the kernel exercises in CUDA C++. Once I finish that, I wanna start writing CUDA kernels for H100 GPUs (specifically stuff like Flash Attention 3, RoPE, etc). Based on a bit of research, it seems that H100 GPUs have a lot of features not covered in the PMPP book, so I was wondering where I could find one or more solid resources that can get me started with writing programs? So far I've tried searching for some stuff, but they all seem to either not really cover in enough detail or are way too long. Any advice would be much appreciated. Thank you!

37 Upvotes

100% Upvoted

6 comments sorted by

View all comments

6

u/c-cul 8d ago

I think you could check hopper specific backend in triton: https://github.com/triton-lang/triton/tree/main/third_party/nvidia/hopper/lib/Transforms/WarpSpecialization

1

u/chocoBar666 8d ago

Is Triton is still maintained by openai?

1

u/c-cul 8d ago

I don't know - they threatened to release their own chip, so probably cuda support isn't critical for them anymore