25x Performance, Three Optimizations

Thanks for posting, that was a really good read, and I ended up landing on very similar optimization in Jolt, but this gives me some food for thought.

I'm using an IR inliner, but currently it only runs under --opt mode, it definitely sounds like I should expand that to the regular release builds as well.

The article's heuristic approach could also work in a complementary way for Jolt. Right now I just have a size budget, but allowing non-recursive and referenced exactly once to inline regardless of size would make sense since they can't grow code.

The really interesting bit for me was generic dispatch problem with var-deref since clojure.core is full of it. There's already a per-site cell cache in the backend, and keyword literals are hoisted into a deterministic per-form let* prologue which is fixpoint-safe, so hoisting (jolt-var ns name) cells the same way would get cached-cell cost with deterministic names.

Many aeons ago when I was working on a Haskell compiler for my thesis one of the biggest things I had to deal with was the tendency of functional languages to emit code in the push-enter model, which is essentially what is being done here, and somewhat resolved through beta reduction.

My approach was to lower to an eval apply model which had the same functional effect as beta reduction here - it removed the intermediate functions that support partial application. Partial application then gets handled by thunks, and the lowering gives you fewer indirect calls to get to the underlying impl.

Another thing I did was detect recursive lambdas (which for me was every function) and lower to iteration when possible. This helped perf in general, but was critical for performance while retaining correctness - .net had only recently introduced vm level guaranteed tail calls and the perf was catastrophically bad (.net has per frame security contexts, and to do this with tail calls it had a heap allocated side stack, which was maintained completely even if there was no change in security context: e.g each tail calls did a heap alloc). In the end my implementation did the classic “catch the stack overflow and then continue execution in a new thread to get more stack space” \o/