blog/aot-vs-jit-compilation-in-java

[giscus[bot]]

blog/aot-vs-jit-compilation-in-java

AOT compilation, supported by the GraalVM compiler, seems to be the future for Java and the programming languages that compile to JVM bytecode. But what is the difference between compiling source code using a JIT compiler w.r.t using an AOT compiler? This post explains the difference between these two approaches.

https://www.cesarsotovalero.net/blog/aot-vs-jit-compilation-in-java.html

[barais]

Thanks for this great blog post. I just pin it as a great resource for my students.

[cesarsotovalero]

Thanks, @barais, your positive comment means a lot to me!

[JonathanSarco]

I think it's one of the best explanations about JIT & AOT I've found on the Internet.
Saludos!

[1chz]

Thanks for the great article.
I would like to translate this article for Korean developers.
Is it okay if I leave the source and translate the article and post it?

[cesarsotovalero]

Hi @shirohoo, I'm glad that you like this blog post. Yes, you can translate it and post it! Please just add a link to the source (this webpage 😄).

[1chz]

Translated post: https://shirohoo.github.io/backend/java/2022-07-16-aot-vs-jit-in-java/

[luffy314]

My question:

so I tried to google and read about it for a bit but i still couldn't find an answer, maybe it's cause i'm new to java and just beginning to learn so i might not have been able to fully understand what i was reading, so here's my question.

What's the purpose of the JVM? why was this program created to run java programs?

Feel free to point out everywhere i'm wrong but here i go. So for a lot of languages u can just create a compiler for it and have the source code compiled into machine code (for the system ur working on if u want). I believe that's what C and C++ do, they just compile their source code into machine code (or maybe assembly code) and then it gets executed by the operating system. This is a pretty fast and effiecent way to run a program and u will end up with some executable file that u can run at anytime and never have to recompile since it's pretty much machine code to run the program. It sounds pretty simple and an efficent solution, unless i'm missing lots of details that i'm not knowledgable about. Sure u can say that compiled code is platform dependent but what's wrong with that, the code itself is platform independent i believe and u could just pass the source code around to different computer systems, and have specific compilers for each system that u could download to compile the code for u just once and then ur good...never again...i think.

Now you have java, which goes to the lengths of first compiling the source code to bytecode and then having some virtual machine interpret this code to machine code and execute it with the O/S, while at the same time also doing JIT on the code to increase performance. How is this better than the orginial method that was used that i mentioned above. To me it seems like just extra work. Sure people say bytecode is write once read anywhere, but u still need to create differnt jvm's for each system in order to run and execute the bytecode. First u have to compile it to bytecode, then u have to interpret it and JIT the bytecode as well. In the end you put the program together and pass it around, u usually put it inside some jar file and probably do other things to make this program available to the public who have to download the application then they can run it by double clicking it which will essentially have to start up a jvm instance and interpret and JIT all the code again (since they're just a bunch of class files stored in a jar file). meanwhile with the C or c++ program, it's already compiled into machine code so it can run way faster i believe.

I'm hoping u can tell me what and where i'm wrong, what benefits does the jvm give u that make it worth using bytcode and JIT instead of just normal old compiling. to me it seems like it would be much better to use the original method cause of greater speed. I've read that the JVM does a lot of extra work taking care of all sorts of things when it's executing, but isn't that something a normal compiler can be programmed to handle as well when compiling to machine cod and in return it would do it only once instead of everytime u run the program in the case of java.

Does the jvm even save previous cached JIT compilations to reuse and save them when re-executing a program or does it just start all over for the bytecode?

lastly what is ahead of time compilation? is it the same thing as when a C or C++ program compile to machine code or is it a term specifically for jvm's, sort of like a counter to JIT? does it only apply for intermediate code like bytecode?

[luffy314]

Cesar's Answer:

What's the purpose of the JVM?

The JVM transforms bytecode into machine code that is later executed by the operating system. The JVM is an extraordinary piece of human engineering. It makes sure that the bytecode could be executed efficiently in all sorts of hardware devices (from IoT micro-devices to supercomputers). To do so, the JVM needs to “understand” the target hardware architecture, optimize the bytecode, and communicate with the operating system.

why was this program created to run java programs?

The JVM was created with the purpose of making the Java programming language ubiquitous among computing devices. Back in the 90s, it was difficult to compile a program and execute it on a different computer. Thus, the JVM solved the problem of portability among the existing programming languages. The JVM was the first, but today the principal programming languages in the market follow the same principle (e.g., C#, Python, JavaScript, etc.)

Sure u can say that compiled code is platform dependent but what's wrong with that, the code itself is platform independent i believe and u could just pass the source code around to different computer systems, and have specific compilers for each system that u could download to compile the code for u just once and then ur good...never again...i think.

There is nothing wrong with compiling code that is platform-dependent. Indeed, these types of compilers (e.g., LLVM) produce the most efficient machine code that you can get today because they are optimized for a very specific architecture (e.g., x86).

Now imagine a scenario in which you have a software product and you want to sell it following your approach (i.e., passing the source code around different computer systems). The market is very diverse in terms of hardware. Therefore, you need to make sure that there is a proper compiler to compile your code for each existing platform. This means that you need to go client by client and “install” your software there. By installing I mean going through all the phases of software construction (i.e., compiling, testing, building, executing, etc.). Later, you need to maintain your software for every particular client: release new versions, add features, fix bugs, etc. That’s a huge investment of your time and resources as a software creator!

On the other hand, if you have something like the JVM at hand, then everything is much more simple. With the JVM you can use any programming language that produces bytecode (e.g., Java, Scala, Kotlin, Clojure,
JRuby, etc.). The existence of the JVM simplifies the production and delivery of software products. It ensures that the time and effort invested in writing code generates a software artifact that is portable and stable.

The Oracle team behind the JVM has invested a ton of money and resources to make the JVM efficient and backward compatible. So, you can be sure the program that you write today could be executable for many decades in the future, no matter which kind of computers we will have in the future.

Now you have java, which goes to the lengths of first compiling the source code to bytecode and then having some virtual machine interpret this code to machine code and execute it with the O/S, while at the same time also doing JIT on the code to increase performance. How is this better than the orginial method that was used that i mentioned above.

It is not better, it is just very convenient, as I mentioned before.

As you said, C or C++ programs run faster, yes. That’s why there is still a market fit for these languages. However, writing and maintaining C and C++ programs is a burden because they use legacy programming standards and paradigms. Today, they are used to build systems and low-level programs, which require a deep understanding of the hardware architecture. JVM-based languages, on the other hand, are much simpler to use because they add significant abstraction to the programmer (e.g., through the garbage collector). In addition, you can use syntactically different languages, with all the syntactic sugar as you want. As long as there is valid bytecode produced, the JVM will run it.

To me it seems like it would be much better to use the original method cause of greater speed.

If a few milliseconds of speed gain is worth it for your particular use case, then you can use the “original method”. If not, I would recommend relying on the JVM or similar systems because their benefits significantly outweigh the drawbacks of having a program that runs a few milliseconds slower than their C or C++ version. You should take into account that performance is a feature, not an engineering goal, in most cases.

Does the jvm even save previous cached JIT compilations to reuse and save them when re-executing a program or does it just start all over for the bytecode?

Yes, the JVM is pretty smart. It performs the most advanced forms of optimizations. I’m not an expert, but I’m familiar with some of them. For example, it does something called “loop unrolling,” which analyzes for loops through previous execution (a.k.a., warming out phase) in order to optimize further executions.

Lastly what is ahead of time compilation? is it the same thing as when a C or C++ program compile to machine code or is it a term specifically for jvm's, sort of like a counter to JIT?

I tried to explain AOT in the blog post. Overall, AOT is the same as compiling in C or C++. In a nutshell, the code is statically linked and transformed into machine code directly. No JVM is necessary to run it.

Does it only apply for intermediate code like bytecode?

No, AOT compilation is the standard way of compiling a C or C++ program. It is new for the Java world, but quite old, actually.

[LinzeSu]

Hi, thanks for the article but I got a question. Why does the JIT has higher throughput? The article mentioned:

JIT compilers reduce latency thanks to the ability to use concurrent garbage collectors and increase the resilience under peak throughput conditions.

So there's no such technique in AOT and I guess and that makes the difference? Also, is there any other reference that dive into this topic?

[cesarsotovalero]

Hi @LinzeSu,
JIT compilers indeed provide some advantages over AOT compilers. These advantages are due to the dynamic nature of JIT compilation. For example, by allowing the compiler to make optimizations based on the actual runtime behavior of the application. In contrast, AOT compilers generate machine code based on static analysis, which limits their ability to adapt to runtime variations.
Regarding additional references on this topic, here three resources I recommend resources that dive deeper into the topic of JIT/OUT compilation and its advantages:

• The Java HotSpot Performance Engine Architecture, by Oracle
• Inside the Java Virtual Machine, specially Chapter 5
• Optimizing Java: Practical Techniques for Improving JVM Application Performance, book from O'Reilly

[qingboooo]

Thanks for the great post.
Most clear and detail blog for JIT and AOT, it is so ... great.

[algomaster99]

Hi Cesar!

Thanks for the great post. I think there is mistake in figure 1. Java compiler is also platform dependent. However, the output, Java bytecode, is platform independent. My suggestion is that you should show the outputs, bytecode and machine code as platform independent and dependent respectively.