Ruby 2.7.0's Rails Ruby Bench Speed is Unchanged from 2.6.0

As of the 25th of December, 2019 we have a released version of Ruby 2.7.0. As you can read in the title - it’s basically the same as 2.6.0.

The 2.7.0 series is remarkable in how little the speed has changed. Overall it has been very stable with very little change in performance. I’ve seen a tiny bit of drift in Rails Ruby Bench results, sometimes as much as 1%-2%, but no more.

The other significant news is also not news: JIT performance is nearly entirely unchanged for Rails apps from 2.6.0. I don’t recommend using CRuby’s MJIT for Rails, and neither does Takashi Kokubun, MJIT’s primary maintainer.

I have a lot of data files to this effect, but… The short version is that, when I run 150 trials of 10,000 HTTP requests each for 2.6.0 versus 2.7.0, the results are well within the margin of error on the measurement. With JIT the results aren’t quite that close, but it’s the same to within a few percent - which means you still shouldn’t turn on JIT for a large Rails app.

I spent some time trying to see if there was a small speedup anywhere in the 2.7 previews that we might have had and missed - there are speed differences of about that size between the fastest and slowest prerelease 2.7 Rubies, which is still very, very small as a span of speeds. And as far as I can tell, no individual change has made a large speed difference, not even 2%. There’s just a very slow drift over time.

Does that mean that Ruby has gotten as fast as it can? Not at all.

Vladimir Makharov (the original author of CRuby’s MJIT) is still working on Mir, a new style of Ruby JIT. Takashi Kokubun is still tuning the existing JIT. I’ve heard interesting things about work from Koichi Sasada on significant reworks of VM subsystems. There are new features happening, and we now have memory compaction.

But I think that at this point, we can reasonably say that the low-hanging performance fruit has been picked. Most speedups from here are going to be more effort-intensive, or require significant architectural changes.