Superinstructions bought 2.8%. The profile explained why.

iOS does not allow a page to be both writable and executable. That is not a policy you engineer around; it is enforced by the kernel, and it means no just-in-time compilation, ever. Whatever performance a Smalltalk gets on that platform, the interpreter has to produce.

AI-built. I directed this work; I did not write the code. How that works.

iospharo is a from-scratch C++ interpreter for Pharo — not a port of the Cog JIT VM — running stock Pharo 13 and 14 images on iPhone, iPad and Mac. It passes 13,040 of 13,053 tests on Mac Catalyst (99.90%), and 99.55% on the iOS Simulator. Cog, the reference VM, is roughly 5× faster than its own interpreter when its JIT is running. On iOS I do not get to have that, so every percent in the interpreter is the whole game.

Which is how I ended up spending real effort on superinstructions and getting 2.8%.

The playbook

There is a well-trodden sequence for making a bytecode interpreter faster.

First, replace the switch dispatch loop with computed goto — GCC and Clang's &&label extension — so each handler jumps directly to the next instead of returning to a central switch. This gives the branch predictor a separate indirect branch site per bytecode rather than one shared site, and it is traditionally worth a lot.

Then, superinstructions: find bytecode pairs that occur together constantly, and fuse each pair into a single handler. One dispatch instead of two, and the fused handler can skip intermediate work that only existed to hand a value from the first instruction to the second.

Both were implemented. The second one was done carefully.

Doing it properly

The fusion candidates were not guessed. A profile of 100 million executed bytecodes produced the pair frequencies, and the fusions were chosen from the top of that list:

• SmallInteger comparison followed by a conditional jump — 5.3M pairs. The fused handler branches on the comparison result directly, never allocating the intermediate boolean object.
• push 1 followed by add or subtract — 2.4M pairs, 65% hit rate. Inlines x + 1 without pushing the constant.
• pushNil followed by identity comparison — 1.73M pairs, 46% hit rate.
• dup, pushNil, ==, jump — 1.45M pairs, 96% hit rate. That is the entire ifNotNil: idiom collapsed into one dispatch.
• push 0 followed by = — 356K pairs.

The estimate going in was a 15–30% improvement. The measurement, three runs on identical fresh images:

Baseline (build 114):   17.71  18.18  18.10   avg = 18.00s
Superinstructions:      17.52  17.46  17.48   avg = 17.49s
                                              ~2.8% CPU

Two point eight percent.

Why the estimate was wrong

The fusions worked exactly as designed — they removed about 11% of all dispatch operations. The premise was what failed:

The 2.8% improvement is below the Phase 3 estimate (15–30%) because Apple M1's branch predictor already handles the dispatch jump table efficiently. Eliminating dispatches saves ~11% of dispatch operations but dispatch overhead is a smaller fraction of total time than expected. The bottleneck is method lookup and stack frame setup, not dispatch.

The interpreter-optimization literature was largely written when indirect branch misprediction dominated interpreter time. On a machine of that era, removing a dispatch removed a likely mispredict, and a 15–30% estimate would have been reasonable. An Apple M1 predicts that jump table well enough that the dispatches being eliminated were mostly cheap ones. Removing 11% of a cost that is no longer the largest cost gets you 2.8%, which is roughly what arithmetic would have predicted had I measured the cost first instead of estimating the saving.

The useful output of the experiment was not the 2.8%. It was the sentence about where the time actually goes.

Following the profile instead

Method lookup and stack frame setup. So the next change attacked frame setup directly.

Pharo code is saturated with trivial accessors — methods whose entire body is ^ instVar or instVar := aValue. Each one, executed conventionally, costs a full Smalltalk activation: push a frame, save the instruction pointer, switch method, run two bytecodes, pop the frame.

Accessor inlining detects those methods at method-cache fill time. On a subsequent cache hit the activation is skipped entirely — no frame push or pop, no instruction-pointer save, no method switch. The getter pattern (pushRecvVar + returnTop) and the setter pattern (popStoreRecvVar + returnReceiver) are recognized and executed inline.

Superinstructions only: 17.52  17.46  17.48   avg = 17.49s
+ accessor inlining:    16.36  16.40  16.35   avg = 16.37s   ~6.4%
Total from baseline:                                  ~9.1%

Six point four percent — more than twice what the superinstructions returned, from a smaller and simpler change, because it was aimed at the thing the profile identified rather than the thing the literature emphasized.

The one that returned nothing

Worth recording, because this is the part that usually goes unpublished. A fusion for returnsSelf — the yourself/identity method pattern — was implemented and measured:

Baseline:    14.08  13.19  10.32   avg = 12.53s
returnsSelf: 13.89  12.85  10.55   avg = 12.43s
Difference:  ~0.8% — within noise

The reason is in the profile too: those methods are simply too rare to matter. The optimization was correct, it worked, and it bought nothing. Notice also the spread on those runs — 14.08 to 10.32 — which is wide enough that a 0.8% difference could not have been claimed even if it were real. A less careful version of this experiment reports a small win.

Why the failed experiment was affordable

A profile-guided superinstruction pass is not a small piece of work. Profiling 100 million bytecodes, ranking the pairs, writing fused handlers for six of them, and measuring carefully enough to trust 2.8% is a serious amount of effort to spend on something you then report as roughly nothing.

It was affordable because I did not write it. The interpreter, the fusion handlers and the benchmark harness were written by AI under my direction. When implementing a hypothesis is cheap, the economics of optimization invert: you stop reasoning about which idea is most likely to pay and start measuring several, including the ones the literature treats as obvious. The 2.8% is not a wasted fortnight. It is a measurement, and it bought the sentence about method lookup and frame setup that led straight to the 6.4%.

What does not get cheaper is deciding what to measure, and being willing to publish a number that makes a well-regarded technique look bad on your hardware.

What I would tell someone starting

Measure where the time is before estimating what a change will save. Those are different activities and only one of them is evidence.

Be especially careful with optimizations that are famous. Computed goto and superinstructions are famous because they were transformative on the hardware where they were first measured, and the papers do not come with an expiry date stamped on the branch predictor they assumed. The technique still works; the premise about which cost dominates has quietly changed underneath it.

And record the negative results. The 2.8% and the 0.8% are the two most useful numbers in this post, because they are what redirected the work to the 6.4%.

On platforms that permit writable-executable memory there is a plan for a tiered copy-and-patch JIT, which should be worth 2–5× where it is allowed to run. iOS will stay interpreted, so the interpreter keeps getting attention — guided, from here on, by the profile rather than by the reading list.