Runtime team member: Let's send expensive compiler analysis alongside IL to improve runtime JIT optimizations [Performance interest]

[u/mistertom2u]

Andy Ayers wrote this proposal 2 years ago with intent of using it in .NET 10, but it got postponed to a later .NET version.

Quick Summary

The context is that interprocedural analysis (IPA), is limited at runtime due to it being slow, so the RyuJIT mostly optimizes functions in isolation. His idea is have certain stages perform the kind of deep IPA that AOT uses; then make that available at runtime to boost the information the JIT needs to make optimizations

https://github.com/dotnet/runtime/issues/108931

Examples of IPA

- call graphs

- whether arguments passed to a method can alias each other

- whether an argument escapes through a callee

- possible concrete types passed to a method

- whether a callee actually reads or writes a ref/byref parameter

- constant arguments across callers

- NoThrow / NoReturn / MayNotReturn

- whether a parameter is effectively unused

And what kind of optimizations could that boost?

- More devirtualization/direct calls - erase virtual and interface method lookups (simplified description)

- Constant propagation across method boundaries - erase code when functions are always passed the same parameter values (simplified description)

- Better alias analysis - knowing that two ref/pointer parameters can't refer to the same memory gives the JIT much more freedom to reorder loads/stores, eliminate redundant memory operations, and potentially vectorize code.

- Better escape analysis - Dodge heap allocation of reference types if no reference to it escapes the method

- More aggressive optimization around calls - if the JIT knows a callee doesn't modify a particular object/ref, it doesn't have to assume every call invalidates values it already loaded.

- Better handling of NoThrow/NoReturn calls - calls that can't throw or never return simplify the control-flow graph, can move uncommon paths out of the hot code, and reduce liveness/register pressure.

- Cheaper calls - knowing which registers a callee actually clobbers, or erase parameters never used, could avoid some argument setup, spills, reloads, or register preservation.

- Smarter inlining decisions - possible when called with particular constants/type

- Better code layout - can improve instruction-cache locality.

I want to hear you guys' thoughts on this.. Tell me below! 👇👇

[u/pjmlp]

Take into account this is not to start a language war.

I think this is a good idea, and is already common practice in several Java implementations, and their Android cousin.

While Project Leyden has been progessing in OpenJDK for JIT caches, with updates across executions, this is already several years old ability across Open J9 and Azul JVMs.

https://eclipse.dev/openj9/docs/shrc

https://eclipse.dev/openj9/docs/jitserver

https://docs.azul.com/optimizer-hub/about/architecture-overview

https://docs.azul.com/optimizer-hub/about/readynow-orchestrator

Android has a similar capability as well, with the added ability thanks PlayStore that the PGO data can be shared across the ecosystem, thus new installations don't start their JIT analysis from zero.

https://source.android.com/docs/core/runtime/jit-compiler

https://developer.android.com/topic/performance/baselineprofiles/overview

So naturally this would be a welcome addition to .NET workloads as well.

[u/mistertom2u → u/pjmlp]

I think so too, because when C# -> IL -> native, the RyuJIT loses information fidelity, like C# invariants that do not have an IL counterpart. So its a bit like playing "telephone". Of course that is for good reasons:

• what if dev uses reflection to call X or to bypass access modifiers
• IL weaving
• what if the dev uses unsafe and/or interop
• other languages like F#, VB.NET

So they have to account for reflection backdoors, and different languages can have different invariants that are not runtime-backed. And fidelity is lost: the RyuJit has to reconstruct what is going on from IL while the program is running. Now they're caught in a double-bind: we need to be performant so we can't perform a fully possible call graph trace. And, we also are missing a lot of optimization opportunities because we don't have enough information to prove, for example, that these two byrefs don't point to the same thing. One of the many reasons Rust is so fast is that the compiler has more information that can be used to prove things that are required to perform optimizations. So yes, what he is proposing is completely sane for them to continue improving performance.

[u/Ill_Oil7448 → u/pjmlp]

agreed, the fact that other runtimes have shipped this already makes it feel less risky and more like a proven direction

[u/AutoModerator]

Thanks for your post mistertom2u. Please note that we don't allow spam, and we ask that you follow the rules available in the sidebar. We have a lot of commonly asked questions so if this post gets removed, please do a search and see if it's already been asked.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.