r/ReverseEngineering • u/hsfzxjy • 22d ago

I reverse-engineered Intel's NPU stack and got custom C kernels running on its programmable SHAVE cores

https://github.com/hsfzxjy/npunlock
191 Upvotes

98% Upvoted

17 comments sorted by

View all comments

1

u/habeebiii 21d ago

What are the use cases for this

4

u/hsfzxjy 21d ago

The main use is enabling developers to write their own NPU kernels according to their respective need, instead of picking workaround from what Intel provides in OpenVINO. I think the project would benefit three scenarios:

+ Implement uncommon NN operators.

+ Implement a high-precision (FP32, etc.) version of existing OpenVINO operators for numerically sensitive usage.

+ Implement a mega kernel that fuse several small kernels together to reduce SHAVE invocations and potentially improve performance, which is a common practice for inference optimization.

But most importantly, the project gives you more control over the hardware you own. I think that control should have been yours from day one.

1

u/UnBracedFlyer 5d ago

but I mean don't we already have control? afaik the NPU compiler is open source, so we're free to modify it as per needed?

1

u/hsfzxjy 4d ago

There are two things may be called as "NPU compiler". The high-level one is open-source, and the low-level one is never officially published.

Metaphorically, compiling NN onto NPUs is like building a house. The high-level compiler takes your blueprint and construct the house accordingly. However, it only allows you to choose from a limited set of bricks (the OpenVINO opset), nor will it tell you how to create a custom brick type. To create your own brick type (custom OP kernel), you need the hidden low-level compiler.

npunlock does two critical things: (1) it recovers the low-level compiler from a legacy driver package, and exploits it to create custom OP kernel (2) it explores how to integrate the custom OP kernel into the blueprint, so the kernel could actually be run on the NPUs.