If you already know lots about JIT in general and Ruby’s MJIT in particular… you may not learn much new in this post. But in case you wonder “what is JIT?” or “what is MJIT?” or “what’s different about Ruby’s JIT?” or perhaps “why in the world did they decide to do THAT?”…

Well then, perhaps I can help explain!

Assisting me in this matter will be Arthur Rackham, famed early-twentieth-century children’s illustrator whose works are now in the public domain. This whole post is adapted from slides to a talk I gave at Southeast Ruby in 2018.

I will frequently refer to TruffleRuby, which is one of the most complex and powerful Ruby implementations. That’s not because you should necessarily use it, but because it’s a great example of Ruby with a powerful and complicated JIT implementation.

What is JIT?

Do you already know about interpreted languages versus compiled languages? In a compiled language, before you run the program you’re writing, you run the compiler on it to turn it into a native application. Then you run that. In an interpreted language, the interpreter reads your source code and runs it more directly without converting it.

A compiled language takes a lot of time to do the conversion… once. But afterward, a native application is usually much faster than an interpreted application. The compiler can perform various optimizations where it recognizes that there is an easier or better way to do some operation than the straightforward one and the native code winds up better than the interpreted code - but it takes time for the compiler to analyze the code and perform the optimization.

A language with JIT (“Just In Time” compilation) is a hybrid of compiled and interpreted languages. It begins by running interpreted, but then notices which pieces of your program are called many times. Then it compiles just those specific parts in order to optimize them.

The idea is that if you have used a particular method many times, you’ll probably use it again many times. So it’s worth the time and trouble to compile that method.

A JITted language avoids the slow compilation step, just like interpreted languages do. But they (eventually) get the faster performance for the parts of your program that are used the most, like a compiled language.

Does JIT Work?

In general, JIT can be a very effective method. How effective depends on what language you’re compiling and what features of that language - you’ll see numbers from 6% to 40% or even more in JavaScript, for instance.

And in fact, there’s an outdated blog post by Benoit Daloze about how TruffleRuby (with JIT) can run a particular CPU-heavy benchmark at 900% the speed of standard CRuby, largely because of its much better JIT (see graph below.) I say “outdated” because TruffleRuby is likely to be even faster now… though so is the latest CRuby.

These numbers are from Benoit Daloze in 2016, see link above

And in fact, the most recent CRuby with JIT enabled runs this same benchmark about 280% the speed of older interpreted CRuby.

JIT Tradeoffs

Nothing is perfect in all situations. Every interesting decision you make as an engineer is a tradeoff of some kind.

Compared to interpreting your language, JIT’s two big disadvantages are memory usage and warmup time.

Memory usage makes sense - if you use JIT, you have to have the interpreted version of your method and the compiled, native version. Two versions, more memory. For complicated reasons, sometimes it’s more than two versions - TruffleRuby often has a lot more than two, which is part of why it’s so fast, but uses lots of memory.

In addition to keeping multiple versions of each method, JIT has to track information about the method. How many times was it called? How much time was spent there? With what arguments? Not every JIT keeps all information, but that means a more complicated JIT with better performance will track more information and use more memory.

In addition to memory usage, there’s warmup time. With JIT, the interpreter has to recognize that a method is called a lot and then take time to compile it. That means there’s a delay between when the program starts and when it gets to full speed.

Some JITs try to compile optimistically - to quickly notice that a method is called a lot and compile it. If it does that, it will often compile methods that don’t get called again much, sometimes, which wastes its time. The Java Virtual Machine (JVM) is (in)famous for this, and tends to run very slowly until JIT has finished.

Other JITs compile pessimistically - they compile methods slowly, and only after they have been called many times. This makes for less waste by compiling the wrong methods, but more warmup time near program start before the program is running quickly. There’s not a “right” answer, but instead various interesting tradeoffs and situations.

JIT is best for programs that run for a long time, like background jobs or network servers. For long-running programs there’s plenty of time to compile the most-used methods and plenty of time to benefit from that speedup. As a result, JIT is often counterproductive for small, short-running programs. Think of “gem list” or small Rake tasks as examples where JIT may not help, and could easily hurt.

Why Didn’t Ruby Get JIT Sooner?

A Ruby core developer tests a JIT implementation for stability

JIT’s two big disadvantages (memory usage, startup/warmup time) are both huge CRuby advantages. That made JIT a tough sell.

Ruby’s current JIT, called MJIT for “Method JIT,” was far from the first attempt. Evan Phoenix built an LLVM Ruby JIT long ago that wound up becoming Rubinius. Early prototypes have been around long before MJIT or its at-the-time competitors. JIT in other Ruby implementations (LLVM libs in Rubinius, OMR) have been tried out and rejected many times. Memory usage has been an especially serious hangup. The Core Team wants CRuby to run well on the smallest Heroku dynos and (historically) in embedded environments.

And while it’s possible to tune a JIT implementation to be okay for warmup time, most JIT is not tuned that way. The Java Virtual Machine (JVM) is an especially serious offender here. Since JRuby (Ruby written in Java) is the most popular alternate Ruby implementation, most Ruby programmers think of “Ruby with JIT” startup time as “Ruby with JVM” startup time, which is dismal.

Also, a JIT implementation can be quite large and complicated. The Ruby core team didn’t really want to adopt something large and complicated that they didn’t have much experience with into the core language.

Shyouhei Urabe, a core team member, created a “deoptimization branch” for Ruby that basically proved you could write a mini-JIT with limited memory use, fast startup time and minimal complexity. This convinced Matz that such a thing was possible and opened the door to JIT in CRuby, which had previously seemed difficult or impossible.

Several JIT implementations were developed… And eventually, Vladimir Makarov created an initial implementation for what would become Ruby’s JIT, one that was reasonably quick, had very good startup time and didn’t use much memory — we’ll talk about how below.

And that was it? No, not quite. MJIT wasn’t clearly the best possibility. Vlad’s MJIT-in-development competed with various other Ruby implementations and with Takashi Kokubun’s LLVM-based Ruby JIT. After Vlad convinced Takashi that MJIT was better, Takashi found a way to take roughly the simplest 80% of MJIT and integrate it nicely into Ruby in a way that was easy to deactivate if necessary and touched very little code outside itself, which he called “YARV-MJIT.”

And after months of integration work, YARV-MJIT was accepted provisionally into prerelease Ruby 2.6 to be worked on by the other Ruby core members, to make sure it could be extended and maintained.

And that was how Ruby 2.6 got MJIT in its current form, though still requiring the Ruby programmer to opt into using it.

Making fun of Ruby for not having JIT yet

MJIT: CRuby’s JIT

The MJIT implementation shows early promise

MJIT is an unusual JIT implementation: it uses a Ruby-to-C language translator and a background thread running a C compiler. It literally writes out C language source files on the disk and compiles them into shared libraries which the Ruby process can load and use. This is not at all how most JIT implementations work.

When a method has been called a certain number of times (10,000 times in current prerelease Ruby 2.7), MJIT will mark it to be compiled into native code and put it on a “to compile” queue. MJIT’s background thread will pull methods from the queue and compile them one at a time into native code.

Remember how we talked about the JVM’s slow startup time? That’s partly because it rapidly begins compiling methods to native code, using a lot of memory and processor time. MJIT compiles only one method at once and expects the result to take time to come back. MJIT sacrifices time-to-full-speed to get good performance early on. This is a great match for CRuby’s use in small command-line applications that often don’t run for long.

“Normal” JIT compiles inside the application’s process. That means if it uses a lot of memory for compiling (which it nearly always does) then it’s very hard to free that memory back to the system. Ruby’s MJIT runs the compiler as a separate background process - when the compiling finishes, the memory is automatically and fully freed back to the operating system. This isn’t as efficient — it sets up a whole external process for compiling. But it’s wonderful for avoiding extra memory usage.

How To Use JIT

This has mostly been a conceptual post. But how do you actually use JIT?

In Ruby 2.6 or higher, use the “—jit” argument to Ruby. This will turn JIT on. You can also add “—jit” to your RUBYOPT environment variable, which will automatically pass it to Ruby every time.

Not sure if your version of Ruby is high enough? Run “ruby —version”. Need to install a later Ruby? Use rvm, ruby-build or your version manager of choice. Ruby 2.6 is already released as I write this, with Ruby 2.7 coming at Christmastime of 2019.

What About Rails?

Unfortunately, there is one huge problem with Ruby’s current MJIT. At the time I write this in mid-to-late 2019, MJIT will slow Rails down instead of speeding it up.

That’s a pretty significant footnote.

Problems, Worries and Escape Hatches

If you want to turn JIT off for any reason in Ruby 2.6 or higher, you can use the “—disable-jit” command-line argument to do that. So if you know you don’t want JIT and you may run the same command with Ruby 3, you can explicitly turn JIT off.

Why might you want to turn JIT off?

Slowdowns: you may know you’re running a tiny program like “gem list —local” that won’t benefit from JIT at all.
No compiler available: you’re running on a production machine without GCC, Clang, etc. MJIT won’t work.
You’re benchmarking: you don’t want JIT because you want predictability, not speed.
Memory usage: MJIT is unusually good for JIT, but it’s not free. You may need every byte you can get.
Read-Only /tmp Dir: If you can’t write the .c files to compile, you can’t compile them.
Weird platform: If you’re running Ruby on your old Amiga or iTanium, there isn’t going to be a supported compiler. You may want to turn JIT off out of general worry and distrust.
Known bug: you know of some specific un-fixed bug and you want to avoid it.

What’s My Takeaway?

Telling the playfully frightened children of a once-JITless Ruby

If you’re running a non-Rails Ruby app and you’d like to speed it up, test it out with “—jit”. It’s likely to do you some good - at least if the CPU is slowing you down.

If you’re running a Rails app or you don’t need better CPU performance, don’t do anything. At some point in the future JIT will become default, and then you’ll use it automatically. It’s already pretty safe, but it will be even safer with a longer time to try it out. And by then, it’s likely to help Rails as well.

If you have a specific reason to turn JIT off (see above,) now you know how.

And if you’ve heard of Ruby JIT and you’re wondering how it’s doing, now you know!

Engineering Blog

JIT and Ruby's MJIT