Onto the second talk of the day. Steve Capper is going to tell us about the good bits of Java They do exist [Audience] Could this have been a lightening talk? [Audience laughter] Believe it or not we've got some good stuff here. I was as skeptical as you guys when I first looked. First many apologies for not attending the mini-conf last year I was unfortunately ill on the day I was due to give this talk. Let me figure out how to use a computer. Sorry about this. There we go; it's because I've not woken up. Last year I worked at Linaro in the Enterprise group and we performed analysis on so called 'Big Data' application sets. As many of you know quite a lot of these big data applications are written in Java. I'm from ARM and we were very interested in 64bit ARM support. So this is mainly AArch64 examples for things like assembler but most of the messages are pertinent for any architecture. These good bits are shared between most if not all the architectures. Whilst trying to optimise a lot of these big data applications I stumbled a across quite a few things in the JVM and I thought 'actually that's really clever; that's really cool' So I thought that would make a good basis for an interesting talk. This talk is essentially some of the clever things I found in the Java Virtual Machine; these optimisations are in OpenJDK. Source is available it's all there, readily available and in play now. I'm going to finish with some of the optimisation work we did with Java. People who know me will know I'm not a Java zealot. I don't particularly believe in programming in a language over another one So to make it clear from the outset I'm not attempting to convert anyone to Java programmers. I'm just going to highlight a few salient things in the Java Virtual Machine which I found to be quite clever and interesting and I'll try and talk through them with my understanding of them. Let's jump straight in and let's start with an example. This is a minimal example for computing a SHA1 sum of a file. I've omitted some of the checking in the beginning of the function see when command line parsing and that sort of thing. I've highlighted the salient points in red. Essentially we instantiate a SHA1 crypto message service digest. And we do the equivalent in Java of an mmap. Get it all in memory. And then we just put this status straight into the crypto engine. And eventually at the end of the program we'll spit out the SHA1 hash. It's a very simple program. It's basically mmap, SHA1, output the hash afterwards. In order to concentrate on the CPU aspect rather than worry about IO I decided to cheat a little bit by setting this up. I decided to use a sparse file. As many of you know a sparse file is a file that not all the contents are stored necessarily on disc. The assumption is that the bits that aren't stored are zero. For instance on Linux you can create a 20TB sparse file on a 10MB file system and use it as normal. Just don't write too much to it otherwise you're going to run out of space. The idea behind using a sparse file is I'm just focusing on the computational aspects of the SHA1 sum. I'm not worried about the file system or anything like that. I don't want to worry about the IO. I just want to focus on the actual compute. In order to set up a sparse file I used the following runes. The important point is that you seek and the other important point is you set a count otherwise you'll fill your disc up. I decided to run this against firstly let's get the native SHA1 sum command that's built into Linux and let's normalise these results and say that's 1.0 I used an older version of the OpenJDK and ran the Java program and that's 1.09 times slower than the reference command. That's quite good. Then I used the new OpenJDK, this is now the current JDK as this is a year on. And 0.21 taken. It's significantly faster. I've stressed that I've done nothing surreptitious in the Java program. It is mmap, compute, spit result out. But the OpenJDK has essentially got some more context information. I'll talk about that as we go through. Before when I started Java I had a very simplistic view of Java. Traditionally Java is taught as a virtual machine that runs bytecode. Now when you compile a Java program it compiles into bytecode. The older versions of the Java Virtual Machine would interpret this bytecode and then run through. Newer versions would employ a just-in-time engine and try and compile this bytecode into native machine code. That is not the only thing that goes on when you run a Java program. There is some extra optimisations as well. So this alone would not account for the newer version of the SHA1 sum being significantly faster than the distro supplied one. Java knows about context. It has a class library and these class libraries have reasonably well defined purposes. We have classes that provide crypto services. We have some misc unsafe that every single project seems to pull in their project when they're not supposed to. These have well defined meanings. These do not necessarily have to be written in Java. They come as Java classes, they come supplied. But most JVMs now have a notion of a virtual machine intrinsic. And the virtual machine intrinsic says ok please do a SHA1 in the best possible way that your implementation allows. This is something done automatically by the JVM. You don't ask for it. If the JVM knows what it's running on and it's reasonably recent this will just happen for you for free. And there's quite a few classes that do this. There's quite a few clever things with atomics, there's crypto, there's mathematical routines as well. Most of these routines in the class library have a well defined notion of a virtual machine intrinsic and they do run reasonably optimally. They are a subject of continuous optimisation as well. We've got some runes that are presented on the slides here. These are quite useful if you are interested in how these intrinsics are made. You can ask the JVM to print out a lot of the just-in-time compiled code. You can ask the JVM to print out the native methods as well as these intrinsics and in this particular case after sifting through about 5MB of text I've come across this particular SHA1 sum implementation. This is AArch64. This is employing the cryptographic extensions in the architecture. So it's essentially using the CPU instructions which would explain why it's faster. But again it's done all this automatically. This did not require any specific runes or anything to activate. We'll see a bit later on how you can more easily find the hot spots rather than sifting through a lot of assembler. I've mentioned that the cryptographic engine is employed and again this routine was generated at run time as well. This is one of the important things about certain execution of amps like Java. You don't have to know everything at compile time. You know a lot more information at run time and you can use that in theory to optimise. You can switch off these clever routines. For instance I've got a deactivate here and we get back to the slower performance we expected. Again, this particular set of routines is present in OpenJDK, I think for all the architectures that support it. We get this optimisation for free on X86 and others as well. It works quite well. That was one surprise I came across as the instrinsics. One thing I thought it would be quite good to do would be to go through a slightly more complicated example. And use this example to explain a lot of other things that happen in the JVM as well. I will spend a bit of time going through this example and explain roughly the notion of what it's supposed to be doing. This is an imaginary method that I've contrived to demonstrate a lot of points in the fewest possible lines of code. I'll start with what it's meant to do. This is meant to be a routine that gets a reference to something and let's you know whether or not it's an image and in a hypothetical cache. I'll start with the important thing here the weak reference. In Java and other garbage collected languages we have the notion of references Most of the time when you are running a Java program you have something like a variable name and that is in the current execution context that is referred to as a strong reference to the object. In other words I can see it. I am using it. Please don't get rid of it. Bad things will happen if you do. So the garbage collector knows not to get rid of it. In Java and other languages you also have the notion of a weak reference. This is essentially the programmer saying to the virtual machine "Look I kinda care about this but just a little bit." "If you want to get rid of it feel free to but please let me know." This is why this is for a CacheClass. For instance the JVM in this particular case could decide that it's running quite low on memory this particular xMB image has not been used for a while it can garbage collect it. The important thing is how we go about expressing this in the language. We can't just have a reference to the object because that's a strong reference and the JVM will know it can't get rid of this because the program can see it actively. So we have a level of indirection which is known as a weak reference. We have this hypothetical CacheClass that I've devised. At this point it is a weak reference. Then we get it. This is calling the weak reference routine. Now it becomes a strong reference so it's not going to be garbage collected. When we get to the return path it becomes a weak reference again because our strong reference has disappeared. The salient points in this example are: We're employing a method to get a reference. We're checking an item to see if it's null. So let's say that the JVM decided to garbage collect this before we executed the method. The weak reference class is still valid because we've got a strong reference to it but the actual object behind this is gone. If we're too late and the garbage collector has killed it it will be null and we return. So it's a level of indirection to see does this still exist if so can I please have it and then operate on it as normal and then return becomes weak reference again. This example program is quite useful when we look at how it's implemented in the JVM and we'll go through a few things now. First off we'll go through the bytecode. The only point of this slide is to show it's roughly the same as this. We get our variable. We use our getter. This bit is extra this checkcast. The reason that bit is extra is because we're using the equivalent of a template in Java. And the way that's implemented in Java is it just basically casts everything to an object so that requires extra compiler information. And this is the extra check. The rest of this we load the reference, we check to see if it is null, If it's not null we invoke a virtual function - is it the image? and we return as normal. Essentially the point I'm trying to make is when we compile this to bytecode this execution happens. This null check happens. This execution happens. And we return. In the actual Java class files we've not lost anything. This is what it looks like when it's been JIT'd. Now we've lost lots of things. The JIT has done quite a few clever things which I'll talk about. First off if we look down here there's a single branch here. And this is only if our check cast failed We've got comments on the right hand side. Our get method has been inlined so we're no longer calling. We seem to have lost our null check, that's just gone. And again we've got a get field as well. That's no longer a method, that's been inlined as well. We've also got some other cute things. Those more familiar with AArch64 will understand that the pointers we're using are 32bit not 64bit. What we're doing is getting a pointer and shifting it left 3 and widening it to a 64bit pointer. We've also got 32bit pointers on a 64bit system as well. So that's saving a reasonable amount of memory and cache. To summarise. We don't have any branches or function calls and we've got a lot of inlining. We did have function calls in the class file so it's the JVM; it's the JIT that has done this. We've got no null checks either and I'm going to talk through this now. The null check elimination is quite a clever feature in Java and other programs. The idea behind null check elimination is most of the time this object is not going to be null. If this object is null the operating system knows this quite quickly. So if you try to dereference a null pointer you'll get either a SIGSEGV or a SIGBUS depending on a few circumstances. That goes straight back to the JVM and the JVM knows where the null exception took place. Because it knows where it took place it can look this up and unwind it as part of an exception. Those null checks just go. Completely gone. Most of the time this works and you are saving a reasonable amount of execution. I'll talk about when it doesn't work in a second. That's reasonably clever. We have similar programming techniques in other places even the Linux kernel for instance when you copy data to and from user space it does pretty much identical the same thing. It has an exception unwind table and it knows if it catches a page fault on this particular program counter it can deal with it because it knows the program counter and it knows conceptually what it was doing. In a similar way the JIT know what its doing to a reasonable degree. It can handle the null check elimination. I mentioned the sneaky one. We've got essentially 32bit pointers on a 64bit system. Most of the time in Java people typically specify heap size smaller than 32GB. Which is perfect if you want to use 32bit pointers and left shift 3. Because that gives you 32GB of addressable memory. That's a significant memory saving because otherwise a lot of things would double up. There's a significant number of pointers in Java. The one that should make people jump out of their seat is the fact that most methods in Java are actually virtual. So what the JVM has actually done is inlined a virtual function. A virtual function is essentially a function were you don't know where you're going until run time. You can have several different classes and they share the same virtual function in the base class and dependent upon which specific class you're running different virtual functions will get executed. In C++ that will be a read from a V table and then you know where to go. The JVM's inlined it. We've saved a memory load. We've saved a branch as well The reason the JVM can inline it is because the JVM knows every single class that has been loaded. So it knows that although this looks polymorphic to the casual programmer It actually is monomorphic. The JVM knows this. Because it knows this it can be clever. And this is really clever. That's a significant cost saving. This is all great. I've already mentioned the null check elimination. We're taking a signal as most of you know if we do that a lot it's going to be slow. Jumping into kernel, into user, bouncing around. The JVM also has a notion of 'OK I've been a bit too clever now; I need to back off a bit' Also there's nothing stopping the user loading more classes and rendering the monomorphic assumption invalid. So the JVM needs to have a notion of backpeddling and go 'Ok I've gone to far and need to deoptimise' The JVM has the ability to deoptimise. In other words it essentially knows that for certain code paths everything's OK. But for certain new objects it can't get away with these tricks. By the time the new objects are executed they are going to be safe. There are ramifications for this. This is the important thing to consider with something like Java and other languages and other virtual machines. If you're trying to profile this it means there is a very significant ramification. You can have the same class and method JIT'd multiple ways and executed at the same time. So if you're trying to find a hot spot the program counter's nodding off. Because you can refer to the same thing in several different ways. This is quite common as well as deoptimisation does take place. That's something to bear in mind with JVM and similar runtime environments. You can get a notion of what the JVM's trying to do. You can ask it nicely and add a print compilation option and it will tell you what it's doing. This is reasonably verbose. Typically what happens is the JVM gets excited JIT'ing everything and optimising everything then it settles down. Until you load something new and it gets excited again. There's a lot of logs. This is mainly useful for debugging but it gives you an appreciation that it's doing a lot of work. You can go even further with a log compilation option. That produces a lot of XML and that is useful for people debugging the JVM as well. It's quite handy to get an idea of what's going on. If that is not enough information you also have the ability to go even further. This is beyond the limit of my understanding. I've gone into this little bit just to show you what can be done. You have release builds of OpenJDK and they have debug builds of OpenJDK. The release builds will by default turn off a lot of the diagnostic options. You can switch them back on again. When you do you can also gain insight into the actual, it's colloquially referred to as the C2 JIT, the compiler there. You can see, for instance, objects in timelines and visualize them as they're being optimised at various stages and various things. So this is based on a masters thesis by Thomas Würthinger. This is something you can play with as well and see how far the optimiser goes. And it's also good for people hacking with the JVM. I'll move onto some stuff we did. Last year we were working on the big data. Relatively new architecture ARM64, it's called AArch64 in OpenJDK land but ARM64 in Debian land. We were a bit concerned because everything's all shiny and new. Has it been optimised correctly? Are there any obvious things we need to optimise? And we're also interested because everything was so shiny and new in the whole system. Not just the JVM but the glibc and the kernel as well. So how do we get a view of all of this? I gave a quick talk before at the Debian mini-conf before last [2014] about perf so decided we could try and do some clever things with Linux perf and see if we could get some actual useful debugging information out. We have the flame graphs that are quite well known. We also have some previous work, Johannes had a special perf map agent that could basically hook into perf and it would give you a nice way of running perf-top for want of a better expression and viewing the top Java function names. This is really good work and it's really good for a particular use case if you just want to do a quick snap shot once and see in that snap shot where the hotspots where. For a prolonged work load with all the functions being JIT'd multiple ways with the optimisation going on and everything moving around it require a little bit more information to be captured. I decided to do a little bit of work on a very similar thing to perf-map-agent but an agent that would capture it over a prolonged period of time. Here's an example Flame graph, these are all over the internet. This is the SHA1 computation example that I gave at the beginning. As expected the VM intrinsic SHA1 is the top one. Not expected by me was this quite significant chunk of CPU execution time. And there was a significant amount of time being spent copying memory from the mmapped memory region into a heap and then that was passed to the crypto engine. So we're doing a ton of memory copies for no good reason. That essentially highlighted an example. That was an assumption I made about Java to begin with which was if you do the equivalent of mmap it should just work like mmap right? You should just be able to address the memory. That is not the case. If you've got a file mapping object and you try to address it it has to be copied into safe heap memory first. And that is what was slowing down the programs. If that was omitted you could make the SHA1 computation even quicker. So that would be the logical target you would want to optimise. I wanted to extend Johannes' work with something called a Java Virtual Machine Tools Interface profiling agent. This is part of the Java Virtual Machine standard as you can make a special library and then hook this into the JVM. And the JVM can expose quite a few things to the library. It exposes a reasonable amount of information as well. Perf as well has the ability to look at map files natively. If you are profiling JavaScript, or something similar, I think the Google V8 JavaScript engine will write out a special map file that says these program counter addresses correspond to these function names. I decided to use that in a similar way to what Johannes did for the extended profiling agent but I also decided to capture some more information as well. I decided to capture the disassembly so when we run perf annotate we can see the actual JVM bytecode in our annotation. We can see how it was JIT'd at the time when it was JIT'd. We can see where the hotspots where. And that's good. But we can go even better. We can run an annotated trace that contains the Java class, the Java method and the bytecode all in one place at the same time. You can see everything from the JVM at the same place. This works reasonably well because the perf interface is extremely extensible. And again we can do entire system optimisation. The bits in red here are the Linux kernel. Then we got into libraries. And then we got into Java and more libraries as well. So we can see everything from top to bottom in one fell swoop. This is just a quick slide showing the mechanisms employed. Essentially we have this agent which is a shared object file. And this will spit out useful files here in a standard way. And the Linux perf basically just records the perf data dump file as normal. We have 2 sets of recording going on. To report it it's very easy to do normal reporting with the PID map. This is just out of the box, works with the Google V8 engine as well. If you want to do very clever annotations perf has the ability to have Python scripts passed to it. So you can craft quite a dodgy Python script and that can interface with the perf annotation output. That's how I was able to get the extra Java information in the same annotation. And this is really easy to do; it's quite easy to knock the script up. And again the only thing we do for this profiling is we hook in the profiling agent which dumps out various things. We preserve the frame pointer because that makes things considerably easier on winding. This will effect performance a little bit. And again when we're reporting we just hook in a Python script. It's really easy to hook everything in and get it working. At the moment we have a JVMTI agent. It's actually on http://git.linaro.org now. Since I gave this talk Google have extended perf anyway so it will do quite a lot of similar things out of the box anyway. It's worth having a look at the latest perf. These techniques in this slide deck can be used obviously in other JITs quite easily. The fact that perf is so easy to extend with scripts can be useful for other things. And OpenJDK has a significant amount of cleverness associated with it that I thought was very surprising and good. So that's what I covered in the talk. These are basically references to things like command line arguments and the Flame graphs and stuff like that. If anyone is interested in playing with OpenJDK on ARM64 I'd suggest going here: http://openjdk.linaro.org Where the most recent builds are. Obviously fixes are going in upstream and they're going into distributions as well. They're included in OpenJDK so it should be good as well. I've run through quite a few fundamental things reasonably quickly. I'd be happy to accept any questions or comments And if you want to talk to me privately about Java afterwards feel free to when no-one's looking. [Audience] Applause [Audience] It's not really a question so much as a comment. Last mini-Deb conf we had a talk about using the JVM with other languages. And it seems to me that all this would apply even if you hate Java programming language and want to write in, I don't know, lisp or something instead if you've got a lisp system that can generate JVM bytecode. [Presenter] Yeah, totally. And the other big data language we looked at was Scala. It uses the JVM back end but a completely different language on the front. Cheers guys.