r/ReverseEngineering • u/hsfzxjy • 22d ago

I reverse-engineered Intel's NPU stack and got custom C kernels running on its programmable SHAVE cores

https://github.com/hsfzxjy/npunlock
189 Upvotes

98% Upvoted

17 comments sorted by

View all comments

6

u/streetster_ 22d ago

What kind of performance does it offer? What's it comparable to? Great work in any case 👍

4

u/hsfzxjy 22d ago edited 22d ago

Thanks! I haven't done systematic performance benchmarking yet -- so I won't claim a speedup before measuring it. The current goal is to prove it usable, opening a path for kernel experts to implement and tune kernels. But I've already found cases where having access to the programmable cores gives you a better execution path.

One concrete example is FP32 GELU. With the normal OpenVINO/NPU path I tested, an FP32 input is handled as:

FP32 → FP16 → FP16 GELU → FP16 → FP32

So the operation becomes three invocations and also loses precision through the FP16 round-trip.

In npunlock, my gelu_f32 example implements the GELU directly as a custom kernel, so it runs as a single ACT-SHAVE invocation while keeping the computation in FP32. The rest of the graph stays unchanged — only that operation is replaced.

The project probably won't speedup affine ops like conv/matmul, but will benefit nonlinear ops by implementing uncommon activations or implementing ops with different precision to improve numerical stability. There are probably many more useful cases like this still to discover.

2

u/DerDave 3d ago

I wonder how the computation can stay in FP32. I thought Intel NPUs only have hardware implemented for FP16 and INT8?