1 99:59:59,999 --> 99:59:59,999 Onto the second talk 2 99:59:59,999 --> 99:59:59,999 Steve Capper is going to tell us about the good bits of Java 3 99:59:59,999 --> 99:59:59,999 They do exist 4 99:59:59,999 --> 99:59:59,999 [Audience] Could this have been a lightening talk? [Audience laughter] 5 99:59:59,999 --> 99:59:59,999 Believe it or not we've got some good stuff here. 6 99:59:59,999 --> 99:59:59,999 I was as skeptical as you guys when I first looked. 7 99:59:59,999 --> 99:59:59,999 First apologies for not attending this mini-conf last year 8 99:59:59,999 --> 99:59:59,999 I was unfortunately ill on the day I was due to give this talk. 9 99:59:59,999 --> 99:59:59,999 Let me figure out how to use a computer. 10 99:59:59,999 --> 99:59:59,999 Sorry about this. 11 99:59:59,999 --> 99:59:59,999 There we go; it's because I've not woken up. 12 99:59:59,999 --> 99:59:59,999 Last year I worked at Linaro in the Enterprise group and we performed analysis 13 99:59:59,999 --> 99:59:59,999 on 'Big Data' applications sets. 14 99:59:59,999 --> 99:59:59,999 As many of you know quite a lot of these big data applications are written in Java. 15 99:59:59,999 --> 99:59:59,999 I'm from ARM and we were very interested in 64bit ARM support. 16 99:59:59,999 --> 99:59:59,999 So this is mainly AArch64 examples for things like assembler 17 99:59:59,999 --> 99:59:59,999 but most of the messages are pertinent for any architecture. 18 99:59:59,999 --> 99:59:59,999 These good bits are shared between most if not all the architectures. 19 99:59:59,999 --> 99:59:59,999 Whilst trying to optimise a lot of these big data applications 20 99:59:59,999 --> 99:59:59,999 I stumbled a across quite a few things in the JVM and I thought 21 99:59:59,999 --> 99:59:59,999 'actually that's really clever; that's really cool' 22 99:59:59,999 --> 99:59:59,999 So I thought that would make a good basis for a talk. 23 99:59:59,999 --> 99:59:59,999 This talk is essentially some of the clever things I found in the 24 99:59:59,999 --> 99:59:59,999 Java Virtual Machine; these optimisations are in Open JDK. 25 99:59:59,999 --> 99:59:59,999 Source is available it's all there, readily available and in play now. 26 99:59:59,999 --> 99:59:59,999 I'm going to finish with some of the optimisation work we did with Java. 27 99:59:59,999 --> 99:59:59,999 People who know me will know I'm not a Java zealot. 28 99:59:59,999 --> 99:59:59,999 I don't particularly believe in programming in a language over another one 29 99:59:59,999 --> 99:59:59,999 So to make it clear from the outset I'm not attempting to convert 30 99:59:59,999 --> 99:59:59,999 anyone to Java programmers. 31 99:59:59,999 --> 99:59:59,999 I'm just going to highlight a few salient things in the Java Virtual Machine 32 99:59:59,999 --> 99:59:59,999 which I found to be quite clever and interesting 33 99:59:59,999 --> 99:59:59,999 and I'll try and talk through them with my understanding of them. 34 99:59:59,999 --> 99:59:59,999 Let's jump straight in and let's start with an example. 35 99:59:59,999 --> 99:59:59,999 This is a minimal example for computing a SHA1 sum of a file. 36 99:59:59,999 --> 99:59:59,999 I've alluded some of the checking in the beginning of the function see when 37 99:59:59,999 --> 99:59:59,999 command line parsing and that sort of thing. 38 99:59:59,999 --> 99:59:59,999 I've highlighted the salient points in red. 39 99:59:59,999 --> 99:59:59,999 Essentially we instantiate a SHA1 crypto message service digest. 40 99:59:59,999 --> 99:59:59,999 And we do the equivalent in Java of an mmap. 41 99:59:59,999 --> 99:59:59,999 Get it all in memory. 42 99:59:59,999 --> 99:59:59,999 And then we just put this status straight into the crypto engine. 43 99:59:59,999 --> 99:59:59,999 And eventually at the end of the program we'll spit out the SHA1 hash. 44 99:59:59,999 --> 99:59:59,999 It's a very simple programme 45 99:59:59,999 --> 99:59:59,999 It's basically mmap, SHA1 output the hash afterwards. 46 99:59:59,999 --> 99:59:59,999 In order to concentrate on the CPU aspect rather than worry about IO 47 99:59:59,999 --> 99:59:59,999 I decided to cheat a little by setting this up. 48 99:59:59,999 --> 99:59:59,999 I decided to use a sparse file. As many of you know a sparse file is a file that not 49 99:59:59,999 --> 99:59:59,999 all the contents are necessarily stored on disc. The assumption is that the bits 50 99:59:59,999 --> 99:59:59,999 that aren't stored are zero. For instance on Linux you can create a 20TB sparse file 51 99:59:59,999 --> 99:59:59,999 on a 10MB file system and use it as normal. 52 99:59:59,999 --> 99:59:59,999 Just don't write too much to it otherwise you're going to run out of space. 53 99:59:59,999 --> 99:59:59,999 The idea behind using a sparse file is I'm just focusing on the computational aspects 54 99:59:59,999 --> 99:59:59,999 of the SHA1 sum. I'm not worried about the file system or anything like that. 55 99:59:59,999 --> 99:59:59,999 I don't want to worry about the IO. I just want to focus on the actual compute. 56 99:59:59,999 --> 99:59:59,999 In order to set up a sparse file I used the following runes. 57 99:59:59,999 --> 99:59:59,999 The important point is that you seek and the other important point 58 99:59:59,999 --> 99:59:59,999 is you set a count otherwise you'll fill your disc up. 59 99:59:59,999 --> 99:59:59,999 I decided to run this against firstly let's get the native SHA1 sum command 60 99:59:59,999 --> 99:59:59,999 that's built into Linux and let's normalise these results and say that's 1.0. 61 99:59:59,999 --> 99:59:59,999 I used an older version of the Open JDK and ran the Java programme 62 99:59:59,999 --> 99:59:59,999 and that's 1.09 times slower than the reference command. That's quite good. 63 99:59:59,999 --> 99:59:59,999 Then I used the new Open JDK, this is now the current JDK as this is a year on. 64 99:59:59,999 --> 99:59:59,999 And 0.21 taken. It's significantly faster. 65 99:59:59,999 --> 99:59:59,999 I've stressed that I've done nothing surreptitious in the Java program. 66 99:59:59,999 --> 99:59:59,999 It is mmap, compute, spit result out. 67 99:59:59,999 --> 99:59:59,999 But the Open JDK has essentially got some more context information. 68 99:59:59,999 --> 99:59:59,999 I'll talk about that as we go through. 69 99:59:59,999 --> 99:59:59,999 Before when I started Java I had a very simplistic view of Java. 70 99:59:59,999 --> 99:59:59,999 Traditionally Java is taught as a virtual machine that runs byte code. 71 99:59:59,999 --> 99:59:59,999 Now when you compile a Java program it compiles into byte code. 72 99:59:59,999 --> 99:59:59,999 The older versions of the Java Virtual Machine would interpret this byte code 73 99:59:59,999 --> 99:59:59,999 and then run through. Newer versions would employ a just-in-time engine and try and 74 99:59:59,999 --> 99:59:59,999 compile this byte code into native machine code. 75 99:59:59,999 --> 99:59:59,999 That is not the only thing that goes on when you run a Java program. 76 99:59:59,999 --> 99:59:59,999 There is some extra optimisations as well. So this alone would not account for 77 99:59:59,999 --> 99:59:59,999 the newer version of the SHA1 sum beingsignificantly faster 78 99:59:59,999 --> 99:59:59,999 than the distro supply one. 79 99:59:59,999 --> 99:59:59,999 Java knows about context. It has a class library and these class libraries 80 99:59:59,999 --> 99:59:59,999 have reasonably well defined purposes. 81 99:59:59,999 --> 99:59:59,999 We have classes that provide crypto services. 82 99:59:59,999 --> 99:59:59,999 We have some misc unsafe that every single project seems to pull in their 83 99:59:59,999 --> 99:59:59,999 project when they're not supposed to. 84 99:59:59,999 --> 99:59:59,999 These have well defined meanings. 85 99:59:59,999 --> 99:59:59,999 These do not necessarily have to be written in Java. 86 99:59:59,999 --> 99:59:59,999 They come as Java classes, they come supplied. 87 99:59:59,999 --> 99:59:59,999 But most JVMs now have a notion of a virtual machine intrinsic 88 99:59:59,999 --> 99:59:59,999 And the virtual machine intrinsic says ok please do a SHA1 in the best possible way 89 99:59:59,999 --> 99:59:59,999 that your implementation allows. This is something done automatically by the JVM. 90 99:59:59,999 --> 99:59:59,999 You don't ask for it. If the JVM knows what it's running on and it's reasonably 91 99:59:59,999 --> 99:59:59,999 recent this will just happen for you for free. 92 99:59:59,999 --> 99:59:59,999 And there's quite a few classes that do this. 93 99:59:59,999 --> 99:59:59,999 There's quite a few clever things with atomics, there's crypto, 94 99:59:59,999 --> 99:59:59,999 there's mathematical routines as well. Most of these routines in the 95 99:59:59,999 --> 99:59:59,999 class library have a well defined notion of a virtual machine intrinsic 96 99:59:59,999 --> 99:59:59,999 and they do run reasonably optimally. 97 99:59:59,999 --> 99:59:59,999 They are a subject of continuous optimisation as well. 98 99:59:59,999 --> 99:59:59,999 We've got some runes that are presented on the slides here. 99 99:59:59,999 --> 99:59:59,999 These are quite useful if you are interested in 100 99:59:59,999 --> 99:59:59,999 how these intrinsics are made. 101 99:59:59,999 --> 99:59:59,999 You can ask the JVM to print out a lot of the just-in-time compiled code. 102 99:59:59,999 --> 99:59:59,999 You can ask the JVM to print out the native methods as well as these intrinsics 103 99:59:59,999 --> 99:59:59,999 and in this particular case after sifting through about 5MB of text 104 99:59:59,999 --> 99:59:59,999 I've come across this particular SHA1 sum implementation. 105 99:59:59,999 --> 99:59:59,999 This is AArch64. This is employing the cryptographic extensions 106 99:59:59,999 --> 99:59:59,999 in the architecture. So it's essentially using the CPU instructions which 107 99:59:59,999 --> 99:59:59,999 would explain why it's faster. But again it's done all this automatically. 108 99:59:59,999 --> 99:59:59,999 This did not require any specific runes or anything to activate. 109 99:59:59,999 --> 99:59:59,999 We'll see a bit later on how you can more easily find the hot spots 110 99:59:59,999 --> 99:59:59,999 rather than sifting through a lot of assembler. 111 99:59:59,999 --> 99:59:59,999 I've mentioned that the cryptographic engine is employed and again 112 99:59:59,999 --> 99:59:59,999 this routine was generated at run time as well. 113 99:59:59,999 --> 99:59:59,999 This is one of the important things about certain execution of amps like Java. 114 99:59:59,999 --> 99:59:59,999 You don't have to know everything at compile time. 115 99:59:59,999 --> 99:59:59,999 You know a lot more information at run time and you can use that 116 99:59:59,999 --> 99:59:59,999 in theory to optimise. 117 99:59:59,999 --> 99:59:59,999 You can switch off these clever routines. 118 99:59:59,999 --> 99:59:59,999 For instance I've got a deactivate here and we get back to the 119 99:59:59,999 --> 99:59:59,999 slower performance we expected. 120 99:59:59,999 --> 99:59:59,999 Again, this particular set of routines is present in Open JDK, 121 99:59:59,999 --> 99:59:59,999 I think for all the architectures that support it. 122 99:59:59,999 --> 99:59:59,999 We get this optimisation for free on X86 and others as well. 123 99:59:59,999 --> 99:59:59,999 It works quite well. 124 99:59:59,999 --> 99:59:59,999 That was one surprise I came across as the instrinsics. 125 99:59:59,999 --> 99:59:59,999 One thing I thought it would be quite good to do would be to go through 126 99:59:59,999 --> 99:59:59,999 a slightly more complicated example. And use this example to explain 127 99:59:59,999 --> 99:59:59,999 a lot of other things that happen in the JVM as well. 128 99:59:59,999 --> 99:59:59,999 I will spend a bit of time going through this example 129 99:59:59,999 --> 99:59:59,999 and explain roughly the notion of what it's supposed to be doing. 130 99:59:59,999 --> 99:59:59,999 This is an imaginary method that I've contrived to demonstrate lot of points 131 99:59:59,999 --> 99:59:59,999 in the fewest possible lines of code. 132 99:59:59,999 --> 99:59:59,999 I'll start with what it's meant to do. 133 99:59:59,999 --> 99:59:59,999 This is meant to be a routine that gets a reference to something and let's you know 134 99:59:59,999 --> 99:59:59,999 whether or not it's an image and in a hypothetical cache. 135 99:59:59,999 --> 99:59:59,999 I'll start with the important thing here the weak reference. 136 99:59:59,999 --> 99:59:59,999 In Java and other garbage collected languages we have the notion of references. 137 99:59:59,999 --> 99:59:59,999 Most of the time when you are running a Java program you have something like a 138 99:59:59,999 --> 99:59:59,999 variable name and that is in the current execution context that is referred to as a 139 99:59:59,999 --> 99:59:59,999 strong reference to the object. In other words I can see it. I am using it. 140 99:59:59,999 --> 99:59:59,999 Please don't get rid of it. Bad things will happen if you do. 141 99:59:59,999 --> 99:59:59,999 So the garbage collector knows not to get rid of it. 142 99:59:59,999 --> 99:59:59,999 In Java and other languages you also have the notion of a weak reference. 143 99:59:59,999 --> 99:59:59,999 This is essentially the programmer saying to the virtual machine 144 99:59:59,999 --> 99:59:59,999 "Look I kinda care about this but just a little bit." 145 99:59:59,999 --> 99:59:59,999 "If you want to get rid of it feel free to but please let me know." 146 99:59:59,999 --> 99:59:59,999 This is why this is for a cache class. For instance the JVM in this particular 147 99:59:59,999 --> 99:59:59,999 case could decide that it's running quite low on memory this particular xMB image 148 99:59:59,999 --> 99:59:59,999 has not been used for a while it can garbage collect it. 149 99:59:59,999 --> 99:59:59,999 The important thing is how we go about expressing this in the language. 150 99:59:59,999 --> 99:59:59,999 We can't just have a reference to the object because that's a strong reference 151 99:59:59,999 --> 99:59:59,999 and the JVM will know it can't get rid of this because the program 152 99:59:59,999 --> 99:59:59,999 can see it actively. 153 99:59:59,999 --> 99:59:59,999 So we have a level of indirection which is known as a weak reference. 154 99:59:59,999 --> 99:59:59,999 We have this hypothetical CacheClass that I've devised. 155 99:59:59,999 --> 99:59:59,999 At this point it is a weak reference. 156 99:59:59,999 --> 99:59:59,999 Then we get it. This is calling the weak reference routine. 157 99:59:59,999 --> 99:59:59,999 Now it becomes a strong reference so it's not going to be garbage collected. 158 99:59:59,999 --> 99:59:59,999 When we get to the return path it becomes a weak reference again 159 99:59:59,999 --> 99:59:59,999 because our strong reference has disappeared. 160 99:59:59,999 --> 99:59:59,999 The salient points in this example are: 161 99:59:59,999 --> 99:59:59,999 We're employing a method to get a reference. 162 99:59:59,999 --> 99:59:59,999 We're checking an item to see if it's null. 163 99:59:59,999 --> 99:59:59,999 So let's say that the JVM decided to garbage collect this 164 99:59:59,999 --> 99:59:59,999 before we executed the method. 165 99:59:59,999 --> 99:59:59,999 The weak reference class is still valid because we've got a strong reference to it 166 99:59:59,999 --> 99:59:59,999 but the actual object behind this is gone. 167 99:59:59,999 --> 99:59:59,999 If we're too late and the garbage collector has killed it 168 99:59:59,999 --> 99:59:59,999 it will be null and we return. 169 99:59:59,999 --> 99:59:59,999 So it's a level of indirection to see does this still exist 170 99:59:59,999 --> 99:59:59,999 if so can I please have it and then operate on it as normal 171 99:59:59,999 --> 99:59:59,999 and then return becomes weak reference again. 172 99:59:59,999 --> 99:59:59,999 This example program is quite useful when we look at how it's implemented in the JVM 173 99:59:59,999 --> 99:59:59,999 and we'll go through a few things now. 174 99:59:59,999 --> 99:59:59,999 First off we'll go through the byte code. 175 99:59:59,999 --> 99:59:59,999 The only point of this slide is to show it's roughly 176 99:59:59,999 --> 99:59:59,999 the same as this. 177 99:59:59,999 --> 99:59:59,999 We get our variable. 178 99:59:59,999 --> 99:59:59,999 We use our getter. 179 99:59:59,999 --> 99:59:59,999 This bit is extra this checkcast. The reason that bit is extra is 180 99:59:59,999 --> 99:59:59,999 because we're using the equivalent of a template in Java. 181 99:59:59,999 --> 99:59:59,999 And the way that's implemented in Java is it just basically casts everything to an 182 99:59:59,999 --> 99:59:59,999 object so that requires extra compiler information. 183 99:59:59,999 --> 99:59:59,999 And this is the extra check. 184 99:59:59,999 --> 99:59:59,999 The rest of this we load the reference, we check to see if it is null, 185 99:59:59,999 --> 99:59:59,999 If it's not null we invoke a virtual function - is it the image? 186 99:59:59,999 --> 99:59:59,999 and we return as normal. 187 99:59:59,999 --> 99:59:59,999 Essentially the point I'm trying to make is when we compile this to byte code 188 99:59:59,999 --> 99:59:59,999 this execution happens. 189 99:59:59,999 --> 99:59:59,999 This null check happens. 190 99:59:59,999 --> 99:59:59,999 This execution happens. 191 99:59:59,999 --> 99:59:59,999 And we return. 192 99:59:59,999 --> 99:59:59,999 In the actual Java class files we've not lost anything. 193 99:59:59,999 --> 99:59:59,999 This is what it looks like when it's been JIT'd. 194 99:59:59,999 --> 99:59:59,999 Now we've lost lots of things. 195 99:59:59,999 --> 99:59:59,999 The JIT has done quite a few clever things which I'll talk about. 196 99:59:59,999 --> 99:59:59,999 First off if we look down here there's a single branch here. 197 99:59:59,999 --> 99:59:59,999 And this is only if our check cast failed 198 99:59:59,999 --> 99:59:59,999 If we've got comments on the right hand side. 199 99:59:59,999 --> 99:59:59,999 Our get method has been in-lined so we're no longer calling. 200 99:59:59,999 --> 99:59:59,999 We seem to have lost our null check, that's just gone. 201 99:59:59,999 --> 99:59:59,999 And again we've got a get field as well. 202 99:59:59,999 --> 99:59:59,999 That's no longer a method, that's been in-lined as well 203 99:59:59,999 --> 99:59:59,999 We've also got some other cute things. 204 99:59:59,999 --> 99:59:59,999 Those more familiar with AArch64 will understand that the pointers we're using 205 99:59:59,999 --> 99:59:59,999 are 32bit not 64bit. 206 99:59:59,999 --> 99:59:59,999 What we're doing is getting a pointer and shifting it left 3 207 99:59:59,999 --> 99:59:59,999 and widening it to a 64bit pointer. 208 99:59:59,999 --> 99:59:59,999 We've also got 32bit pointers on a 64bit system as well. 209 99:59:59,999 --> 99:59:59,999 So that's saving a reasonable amount of memory and cache. 210 99:59:59,999 --> 99:59:59,999 To summarise. We don't have any branches or function calls 211 99:59:59,999 --> 99:59:59,999 and we've got a lot of in-lining. 212 99:59:59,999 --> 99:59:59,999 We did have function calls in the class file so it's the JVM 213 99:59:59,999 --> 99:59:59,999 it's the JIT that has done this. 214 99:59:59,999 --> 99:59:59,999 We've got no null checks either and I'm going to talk through this now. 215 99:59:59,999 --> 99:59:59,999 The null check elimination is quite a clever feature in Java and other programs. 216 99:59:59,999 --> 99:59:59,999 The idea behind null check elimination is 217 99:59:59,999 --> 99:59:59,999 most of the time this object is not going to be null. 218 99:59:59,999 --> 99:59:59,999 If this object is null the operating system knows this quite quickly. 219 99:59:59,999 --> 99:59:59,999 So if you try to de-reference a null pointer you'll get either a SIGSEGV or 220 99:59:59,999 --> 99:59:59,999 a SIGBUST depending on a few circumstances. 221 99:59:59,999 --> 99:59:59,999 That goes straight back to the JVM 222 99:59:59,999 --> 99:59:59,999 and the JVM knows where the null exception took place. 223 99:59:59,999 --> 99:59:59,999 Because it knows where the exception took place it can look this up 224 99:59:59,999 --> 99:59:59,999 and unwind it as part of an exception. 225 99:59:59,999 --> 99:59:59,999 Those null checks just go. Completely gone. 226 99:59:59,999 --> 99:59:59,999 Most of the time this works and you are saving a reasonable amount of execution. 227 99:59:59,999 --> 99:59:59,999 I'll talk about when it doesn't work in a second. 228 99:59:59,999 --> 99:59:59,999 That's reasonably clever. We have similar programming techniques in other places 229 99:59:59,999 --> 99:59:59,999 even the Linux kernel for instance when you copy data to and from user space 230 99:59:59,999 --> 99:59:59,999 it does pretty much identical the same thing. It has an exception unwind table 231 99:59:59,999 --> 99:59:59,999 and it knows if it catches a page fault on this particular program counter 232 99:59:59,999 --> 99:59:59,999 it can deal with it because it knows the program counter and it knows 233 99:59:59,999 --> 99:59:59,999 conceptually what it was doing. 234 99:59:59,999 --> 99:59:59,999 In a similar way the JIT know what its doing to a reasonable degree. 235 99:59:59,999 --> 99:59:59,999 It can handle the null check elimination. 236 99:59:59,999 --> 99:59:59,999 I mentioned the sneaky one. We've got essentially 32bit pointers 237 99:59:59,999 --> 99:59:59,999 on a 64bit system. 238 99:59:59,999 --> 99:59:59,999 Most of the time in Java people typically specify heap size smaller than 32GB. 239 99:59:59,999 --> 99:59:59,999 Which is perfect if you want to use 32bit pointers and left shift 3. 240 99:59:59,999 --> 99:59:59,999 Because that gives you 32GB of addressable memory. 241 99:59:59,999 --> 99:59:59,999 That's a significant memory saving because otherwise a lot of things would double up. 242 99:59:59,999 --> 99:59:59,999 There's a significant number of pointers in Java. 243 99:59:59,999 --> 99:59:59,999 The one that should make people jump out of their seat is 244 99:59:59,999 --> 99:59:59,999 the fact that most methods in Java are actually virtual. 245 99:59:59,999 --> 99:59:59,999 So what the JVM has actually done is in-lined a virtual function. 246 99:59:59,999 --> 99:59:59,999 A virtual function is essentially a function were you don't know where 247 99:59:59,999 --> 99:59:59,999 you're going until run time. 248 99:59:59,999 --> 99:59:59,999 You can have several different classes and they share the same virtual function 249 99:59:59,999 --> 99:59:59,999 in the base class and dependent upon which specific class you're running 250 99:59:59,999 --> 99:59:59,999 different virtual functions will get executed. 251 99:59:59,999 --> 99:59:59,999 In C++ that will be a read from a V table and then you know where to go. 252 99:59:59,999 --> 99:59:59,999 The JVM's in-lined it. 253 99:59:59,999 --> 99:59:59,999 We've saved a memory load. 254 99:59:59,999 --> 99:59:59,999 We've saved a branch as well 255 99:59:59,999 --> 99:59:59,999 The reason the JVM can in-line it is because the JVM knows 256 99:59:59,999 --> 99:59:59,999 every single class that has been loaded. 257 99:59:59,999 --> 99:59:59,999 So it knows that although this looks polymorphic to the casual programmer 258 99:59:59,999 --> 99:59:59,999 It is actually monomorphic. The JVM knows this.