Ruby 3 and JIT: Where, When and How Fast?

You may have heard about Ruby 3 including JIT. Where is JIT coming from? How soon will it be included? How much faster will it be? If I'm worried about debugging with JIT, can I turn it off?

Wait, What's JIT Again?

In case "JIT" isn't a day-to-day word for you: "JIT" is short for "Just-In-Time," or more specifically "Just-In-Time Compiling." Right now, Ruby interprets your program. With JIT, it will convert parts of the Ruby program into machine code, just like a Unix command or an EXE file. Specifically, JIT converts from the kind of Ruby code you read every day into the code that runs most naturally, and fastest, for your processor, often called "machine code" or "machine language."

JIT is different from a "normal" compiler in a few ways. The biggest is that it doesn't compile your whole program. Instead, it compiles just the parts that run the most often, and it compiles them specially to run fastest exactly how your program uses them. It doesn't have to guess how you're calling those methods - it watches your program for awhile and takes notes, then it compiles them.

Alas, JIT removes this excuse. I recommend that you claim you're debugging the AoT settings. Or claim you're running ETL scripts. That works too.

How Much Faster?

There are lies, damned lies and benchmarks. I can't give you an exact percentage speedup for JIT or because there is no such percentage. But there are lots of cases where JIT can speed up a particular program by 50%, 150% or 250% on perfectly reasonable workloads. There are even a few realistic workloads where it can speed things up by 500% or more. But of course there are also a few cases where interpreted is faster than JIT, because nothing in the real world is always an optimization.

The current conservative, simple JIT implementations for CRuby add around 30%-50% to performance, or up to 150% depending how you measure. 30%-50% is quite modest for JIT, but these branches are still simple. And 30%-50% is nothing to sneeze at. That's the equivalent of between 3 and 10 years of "normal" release speedups, all in around a year or two of effort to get JIT working. And that's in addition to the usual speedups, which are still happening. And the JIT can keep improving over time. It opens up a whole world of optimization that old-style "only interpreted" Ruby couldn't do, which is why Ruby implementations with JIT can be a lot faster already. Something like TruffleRuby adds a lot of memory overhead, but can speed the code up by 900% or more - CRuby won't match that, but such things are definitely possible.

Usually I answer "how fast?" with numbers from Rails Ruby Bench. That's my thing, after all! But right now, MJIT isn't stable enough to run a big high-concurrency Rails app. Don't worry, I'll publish numbers when it is!

These numbers aren't terribly recent. And the MJIT and YARV-MJIT numbers are still changing too fast to mean much. Soon!

Where Did CRuby JIT Come From?

JIT has been in Ruby in some form for awhile: JRuby has had it for many years. Rubinius had it for awhile and got rid of it. But "plain" CRuby has never had it just built in... yet. Instead, JIT has been around in various experimental branches that never got into a Ruby release.

Shyouhei Urabe's "deoptimization" branch was pretty good, but never quite made it in. It was a very plain, very simple form of JIT that only enabled a few optimizations, but also guaranteed only a tiny bit of extra memory usage. And the Ruby core team really cares about memory usage.

Then recently Vladimir Makarov, the same guy who rebuilt Ruby 2.4 hash tables, wrote a powerful, low-memory JIT implementation called "MJIT". It leverages your existing C compiler (GCC or CLang) for Ruby JIT. MJIT looks amazing - good enough that he was invited to give a RubyKaigi keynote to explain how MJIT works. He first converts Ruby to use a register-based VM instead of stack-based, and then builds JIT on top of that. But MJIT is pretty new and not stable enough for general release to the world. Making a crash-free JIT implementation that can handle any possible Ruby program is hard, and MJIT isn't there yet. Based on recent numbers, though, MJIT can get 230% the speed of Ruby 2.0.0 on CPU benchmarks, so it's clearly doing some things right!

At the same time MJIT was happening, Takashi Kokubun was writing a powerful LLVM-based Ruby JIT implementation called LLRB, inspired by Evan Phoenix's earlier work. Like MJIT, it didn't get polished enough to unleash upon the entire Ruby world. But Takashi went on to take most of MJIT and turn it into... YARV-MJIT.

YARV-MJIT takes MJIT and strips out the changes to make it a register-based VM. Those changes make Ruby faster, but at the cost of more testing to get everything stable. By removing them, we can get a less-capable Ruby JIT, but get it sooner. Remember all those people telling you to make your feature as small as possible and release it sooner? YARV-MJIT is that principle in action: what if we *just* added JIT, even if it's not as much faster? And turn off JIT by default, so we only get this new experimental feature if we request it? But it's the same JIT as in MJIT, just with some of the features turned off.

When Is It Coming?

This is a hard question, of course. It'll depend on what problems get found and how easy they are to fix.

The pull request for YARV-MJIT is open now, so we may be in the countdown until it lands in Ruby... Though it is not in the Ruby 2.5.0 Christmas release, which is for the best.

YARV-MJIT and MJIT are both improving constantly. Vlad thinks MJIT will take around a year to really mature. But YARV-MJIT lets JIT be included with a normal Ruby release without having to be perfect - it'll only be turned on when we ask for it.

So in a narrow sense, it could happen any day now. But it will probably take a year or more before it gets turned on by default. As with immutable strings, Ruby is including more new features as opt-in. This can work a lot like Feature Toggles (aka Feature Flags or Feature Flippers) - you can include the new features before they're fully ready, but make sure they don't conflict with each other. I like this approach a lot more than how the Ruby 1.8/1.9 transition was handled.

From this tweet.

How Will We Know? Can I Turn It Off?

If you're curious when YARV-MJIT makes it into Ruby, I'd recommend following the pull request above.

And if you're worried that JIT might cause you problems (fair,) keep in mind that you can turn it on or off. The RUBYOPT environment variable works for any CRuby, not just the ones with MJIT or YARV-MJIT, and it lets you pass command-line arguments in for every time you run Ruby, not just the one you're typing now.

Right now even in YARV-MJIT, JIT is off by default. If you want to turn it on:

export RUBYOPT="-j"

For YARV-MJIT, you can deactivate JIT by just not passing any JIT parameters. So don't pass anything starting with "-j" and you shouldn't see any JIT happening.

You can also see what JIT does by passing other "-j" parameter. For instance, passing "-j:w" should print any JIT warnings, while "-j:s" should save the .c source files created by JIT in the /tmp directory instead of deleting them.

Want to do more with JIT? I recommend running "ruby --help" with an MJIT or YARV-MJIT enabled Ruby. Here's what that currently prints -- though these options might change before YARV-MJIT is accepted into Ruby, so you should check your local version:

MJIT options:
  s, save-temps   Save MJIT temporary files in /tmp
  c, cc           C compiler to generate native code (gcc, clang, cl)
  w, warnings     Enable printing MJIT warnings
  d, debug        Enable MJIT debugging (very slow)
  v=num, verbose=num
                  Print MJIT logs of level num or less to stderr
  n=num, num-cache=num
                  Maximum number of JIT codes in a cache

How Can I Help? What's Next for JIT in Ruby?

Want to use JIT in Ruby? One of the first, easiest things you can do is to try it out and see if it works!

You can clone and build it like this:

cd ~/my_src_dir
git clone git@github.com:k0kubun/yarv-mjit.git
cd yarv-mjit
autoconf
./configure
make check

Once you've done that, you can test it locally or install it. To test it locally, I like the "runruby" script:

cd ~/my_src_dir/yarv-mjit
./tool/runruby.rb ~/my_src_dir/my_ruby_script.rb

You can also build and mount locally-built Ruby interpreters with rvm:

# be sure to compile it first!
rvm mount ~/my_src_dir/yarv-mjit yarv-mjit
rvm use ext-yarv-mjit

Remember that you can turn on JIT with "-j" and turn on warnings with "-j:w". If you run your code with YARV-MJIT, let us know! I like Twitter for this, personally, but use what works for you.

If you find a problem with JIT, try to cut it down to a small test case for reproduction. Then you can report it on the Ruby bug for YARV-MJIT. Thanks in advance!