0:00:00.000,0:00:08.511 (Theme Music) 0:00:13.426,0:00:16.543 This talk is about how Bundler works 0:00:17.986,0:00:21.129 How does Bundler work? 0:00:21.804,0:00:24.780 This is an interesting question. 0:00:25.335,0:00:27.576 We'll talk about it for a while. 0:00:28.516,0:00:29.691 This talk is a brief history of 0:00:31.845,0:00:35.845 dependency management in Ruby, 9:59:59.000,9:59:59.000 a discussion of how libraries and 9:59:59.000,9:59:59.000 shared code works now and in the past 9:59:59.000,9:59:59.000 because how it works now is directly a result 9:59:59.000,9:59:59.000 of how it used to work in the past 9:59:59.000,9:59:59.000 and trying to fix problems that happened then. 9:59:59.000,9:59:59.000 Before we get started, let me introduce myself: 9:59:59.000,9:59:59.000 My name is Andre Arko, I'm @indirect on all social media 9:59:59.000,9:59:59.000 that's my avatar, maybe you've seen me on 9:59:59.000,9:59:59.000 a webpage somewhere. As my day job 9:59:59.000,9:59:59.000 I work at Cloud City Development doing 9:59:59.000,9:59:59.000 Ruby, Rails, Ember, and web consulting. 9:59:59.000,9:59:59.000 We do web and mobile development 9:59:59.000,9:59:59.000 and I mostly do architectural consulting 9:59:59.000,9:59:59.000 and Senior Developer pairing and training. 9:59:59.000,9:59:59.000 Talk to me if you're company is interested. 9:59:59.000,9:59:59.000 I also founded Ruby Together, a non-profit, 9:59:59.000,9:59:59.000 it's like npm incorporate without the venture capital. 9:59:59.000,9:59:59.000 Ruby Together is a trade association that 9:59:59.000,9:59:59.000 takes money from companies and people who 9:59:59.000,9:59:59.000 use Ruby and Bundler and RubyGems and all 9:59:59.000,9:59:59.000 of the public infrastructure that Rubyists use 9:59:59.000,9:59:59.000 and pays for developers to work on that 9:59:59.000,9:59:59.000 so that RubyGems.org stays up, and so 9:59:59.000,9:59:59.000 people can have gems, which is pretty cool. 9:59:59.000,9:59:59.000 As part of my work for Ruby Together I work as 9:59:59.000,9:59:59.000 lead of the Bundler team. I've been working on 9:59:59.000,9:59:59.000 Bundler since before 1.0 came out, and I've 9:59:59.000,9:59:59.000 been team lead for the last four years. 9:59:59.000,9:59:59.000 Using Ruby code written by other developers, 9:59:59.000,9:59:59.000 nowadays this is actually really easy, 9:59:59.000,9:59:59.000 you add a line to your Gemfile, 9:59:59.000,9:59:59.000 you go to your terminal and run 9:59:59.000,9:59:59.000 bundle install, and you start using it. 9:59:59.000,9:59:59.000 Pretty cool, that's really easy. 9:59:59.000,9:59:59.000 The thing that I've noticed, talking to people 9:59:59.000,9:59:59.000 who use Bundler and think it's awesome 9:59:59.000,9:59:59.000 is that, it's not actually clear what just happened. 9:59:59.000,9:59:59.000 Based on the text printed out by bundle install 9:59:59.000,9:59:59.000 it seems like something got downloaded 9:59:59.000,9:59:59.000 and something got installed, but it's not clear. 9:59:59.000,9:59:59.000 It's not clear what got downloaded or 9:59:59.000,9:59:59.000 installed, or where it happened. 9:59:59.000,9:59:59.000 What exactly happened there? 9:59:59.000,9:59:59.000 Nobody is really sure. 9:59:59.000,9:59:59.000 How does just putting a line in your Gemfile 9:59:59.000,9:59:59.000 mean you can just start using somebody else's code? 9:59:59.000,9:59:59.000 To explain that, we'll need a little bit of history. 9:59:59.000,9:59:59.000 We're going to back in a time. 9:59:59.000,9:59:59.000 I'm going to give you a tour from the beginning of sharing 9:59:59.000,9:59:59.000 code in Ruby up until now. 9:59:59.000,9:59:59.000 And hopefully by the end of it you'll understand 9:59:59.000,9:59:59.000 why things work the way they do now. 9:59:59.000,9:59:59.000 I'm going to start talking about require, 9:59:59.000,9:59:59.000 which came with the very first version of Ruby ever, in 1994. 9:59:59.000,9:59:59.000 And then talk about setup.rb from 2000, 9:59:59.000,9:59:59.000 and then RubyGems from 2003, and Bundler from 2009. 9:59:59.000,9:59:59.000 And that's what we're still using today. 9:59:59.000,9:59:59.000 The require method has been around since 9:59:59.000,9:59:59.000 1994, with the very first version of Ruby. 9:59:59.000,9:59:59.000 What I should say is that it's been there 9:59:59.000,9:59:59.000 since at least 1997, since that's 9:59:59.000,9:59:59.000 the oldest version controlled Ruby we have. 9:59:59.000,9:59:59.000 It was probably there before that though. 9:59:59.000,9:59:59.000 Require can be broken down into even 9:59:59.000,9:59:59.000 smaller concepts. Using code from 9:59:59.000,9:59:59.000 a file is basically the same as inserting 9:59:59.000,9:59:59.000 that code and having Ruby run it 9:59:59.000,9:59:59.000 as if you'd just written it in the file. 9:59:59.000,9:59:59.000 It's actually possible to implement it yourself, 9:59:59.000,9:59:59.000 with a one-line function. 9:59:59.000,9:59:59.000 This function says; I have a file 9:59:59.000,9:59:59.000 name and I want to require it, 9:59:59.000,9:59:59.000 and you read the file in the memory 9:59:59.000,9:59:59.000 into a string and you pass the 9:59:59.000,9:59:59.000 string to eval, and Ruby runs it 9:59:59.000,9:59:59.000 and it's just like you typed that code yourself. 9:59:59.000,9:59:59.000 There are problems with this. 9:59:59.000,9:59:59.000 Require doesn't work this way in real life. 9:59:59.000,9:59:59.000 I'm sure it's totally fine that this will 9:59:59.000,9:59:59.000 run that same piece of code over and over 9:59:59.000,9:59:59.000 if you require it over and over, you like 9:59:59.000,9:59:59.000 having lots and lots of constants that keep 9:59:59.000,9:59:59.000 getting redefined, I'm sure it's totally fine. 9:59:59.000,9:59:59.000 Working around that, is pretty straightforward. 9:59:59.000,9:59:59.000 Just keep track of what you've required in an Array 9:59:59.000,9:59:59.000 and not require something again if it' been required. 9:59:59.000,9:59:59.000 As you can see here, 9:59:59.000,9:59:59.000 you set up an Array, you check 9:59:59.000,9:59:59.000 to see if the Array already contains 9:59:59.000,9:59:59.000 the filename that just got passed in, 9:59:59.000,9:59:59.000 and if hasn't been required, 9:59:59.000,9:59:59.000 do the same thing we did before, 9:59:59.000,9:59:59.000 read the file in, pass it to eval, 9:59:59.000,9:59:59.000 and then add it to the array, 9:59:59.000,9:59:59.000 so it's not required again later. 9:59:59.000,9:59:59.000 In fact, this exactly what Ruby does, 9:59:59.000,9:59:59.000 but written in C not in Ruby. 9:59:59.000,9:59:59.000 There is a LOADED_FEATURES global variable, 9:59:59.000,9:59:59.000 and it's an Array, and it contains a list 9:59:59.000,9:59:59.000 of all the required files. 9:59:59.000,9:59:59.000 If you want to know if you've required something yet, 9:59:59.000,9:59:59.000 check the LOADED_FEATURES array. 9:59:59.000,9:59:59.000 There is one more problem with this, 9:59:59.000,9:59:59.000 it only works when you pass in absolute paths. 9:59:59.000,9:59:59.000 I'm sure you don't mind you typing the 9:59:59.000,9:59:59.000 full path from wherever you are to 9:59:59.000,9:59:59.000 exactly wherever the file you want to require is. 9:59:59.000,9:59:59.000 I'm sure that's fine too. 9:59:59.000,9:59:59.000 The easiest way to allow requires that 9:59:59.000,9:59:59.000 aren't absolute is to just treat 9:59:59.000,9:59:59.000 all requires as if they're relative 9:59:59.000,9:59:59.000 to the path where you started 9:59:59.000,9:59:59.000 the Ruby program. And that's easy, 9:59:59.000,9:59:59.000 but that doesn't help a lot if you 9:59:59.000,9:59:59.000 want to require Ruby files from different places. 9:59:59.000,9:59:59.000 Say you have a folder full of a library 9:59:59.000,9:59:59.000 you wrote and folder full of an application you wrote 9:59:59.000,9:59:59.000 and you want to use a library from the app, you can't, 9:59:59.000,9:59:59.000 because writing relative paths from wherever 9:59:59.000,9:59:59.000 you started the Ruby program would be terrible. 9:59:59.000,9:59:59.000 Instead we create an Array that holds the 9:59:59.000,9:59:59.000 list of paths we want to load we want to load 9:59:59.000,9:59:59.000 Ruby files from, in a burst of creativity 9:59:59.000,9:59:59.000 I'm just going to call that variable the 9:59:59.000,9:59:59.000 LOAD_PATH, and here's an implementation. 9:59:59.000,9:59:59.000 If you put something in the LOAD_PATH Array, 9:59:59.000,9:59:59.000 you can then pass a relative path to any directory 9:59:59.000,9:59:59.000 that's in the LOAD_PATH Array, and 9:59:59.000,9:59:59.000 it will look for the file. 9:59:59.000,9:59:59.000 If you require "foo", it will look for a file 9:59:59.000,9:59:59.000 named "foo" inside any of the LOAD_PATH directories, 9:59:59.000,9:59:59.000 and if the first one we find searching 9:59:59.000,9:59:59.000 the LOAD_PATH in order from first to last, 9:59:59.000,9:59:59.000 we will require that one. 9:59:59.000,9:59:59.000 Coincidentally, this is exactly what 9:59:59.000,9:59:59.000 Ruby does, there is a global variable 9:59:59.000,9:59:59.000 named LOAD_PATH, and if you put 9:59:59.000,9:59:59.000 a string that contains a path to a directory 9:59:59.000,9:59:59.000 in it, Ruby will look in that directory 9:59:59.000,9:59:59.000 whenever you require something for a file 9:59:59.000,9:59:59.000 with that name. 9:59:59.000,9:59:59.000 You can totally use the LOAD_PATH to require 9:59:59.000,9:59:59.000 files from somewhere else while you're working with them. 9:59:59.000,9:59:59.000 Of course, the LOADPATH, and LOADED_FEATURES 9:59:59.000,9:59:59.000 can both be combined, but that didn't 9:59:59.000,9:59:59.000 fit on a single slide, so I'll leave that 9:59:59.000,9:59:59.000 as an exercise to the listener. 9:59:59.000,9:59:59.000 It's pretty straightforward to be honest. 9:59:59.000,9:59:59.000 Load paths are pretty cool. 9:59:59.000,9:59:59.000 They allow us to load Ruby directories 9:59:59.000,9:59:59.000 even if they're spread across multiple places. 9:59:59.000,9:59:59.000 At this point, we could even 9:59:59.000,9:59:59.000 have automatically, at the start of every script, 9:59:59.000,9:59:59.000 the directory that holds the standard library, 9:59:59.000,9:59:59.000 to the load path, and then all of the 9:59:59.000,9:59:59.000 files that are pretty of the Ruby standard library, 9:59:59.000,9:59:59.000 like Net::HTTP, Set, the cool thing that 9:59:59.000,9:59:59.000 come with Ruby, could just be available for 9:59:59.000,9:59:59.000 require automatically and you wouldn't have 9:59:59.000,9:59:59.000 to worry about putting them in the 9:59:59.000,9:59:59.000 load path yourself. That's exactly 9:59:59.000,9:59:59.000 what Ruby does, the standard library 9:59:59.000,9:59:59.000 starts on the load path when Ruby starts. 9:59:59.000,9:59:59.000 It's pretty great. This was cool, and 9:59:59.000,9:59:59.000 for several years, this was enough. 9:59:59.000,9:59:59.000 People just added things to the load path. 9:59:59.000,9:59:59.000 Or wrote scripts that added things to the 9:59:59.000,9:59:59.000 load path before requiring things before their 9:59:59.000,9:59:59.000 actual script happened. 9:59:59.000,9:59:59.000 The thing that got tedius out just having 9:59:59.000,9:59:59.000 load paths, is that if you want to get code from 9:59:59.000,9:59:59.000 someone else, you have to find that code, 9:59:59.000,9:59:59.000 download it, put it somewhere, remember where, 9:59:59.000,9:59:59.000 put it in the load path, and then require it. 9:59:59.000,9:59:59.000 This was tedious. 9:59:59.000,9:59:59.000 Setup.rb happened next. 9:59:59.000,9:59:59.000 Around the year 2000 everyone is still 9:59:59.000,9:59:59.000 installing share Ruby code by hand. 9:59:59.000,9:59:59.000 That wasn't so much fun. 9:59:59.000,9:59:59.000 A Japanese Ruby developer, Minero Aoki, 9:59:59.000,9:59:59.000 wrote setup.rb, and amazingly, 9:59:59.000,9:59:59.000 even though this was created in 2000, 9:59:59.000,9:59:59.000 setup.rb is still around on the Internet. 9:59:59.000,9:59:59.000 The website for this developer is, 9:59:59.000,9:59:59.000 i.loveruby.net, which is pretty cool, 9:59:59.000,9:59:59.000 and you can even download setup.rb, but 9:59:59.000,9:59:59.000 to be honest, it hasn't been updated since 2005, 9:59:59.000,9:59:59.000 so I'm not sure it's super helpful to you. 9:59:59.000,9:59:59.000 How did setup.rb work? 9:59:59.000,9:59:59.000 At it's core it mimicked the classic 9:59:59.000,9:59:59.000 UNIX installation pattern, 9:59:59.000,9:59:59.000 downloading a piece of software, 9:59:59.000,9:59:59.000 decompressing it, and then running 9:59:59.000,9:59:59.000 configure make, and make install, 9:59:59.000,9:59:59.000 so setup.rb kind of copied that for Ruby. 9:59:59.000,9:59:59.000 You would run ruby setup.rb setup, 9:59:59.000,9:59:59.000 ruby setup.rb config, ruby setup.rb install 9:59:59.000,9:59:59.000 setup.rb would copy all the Ruby files, 9:59:59.000,9:59:59.000 there was a specific directory structure, 9:59:59.000,9:59:59.000 kind of like a Gem today, with 9:59:59.000,9:59:59.000 library files, and bin files you could run as programs, 9:59:59.000,9:59:59.000 and support files, and setup.rb would 9:59:59.000,9:59:59.000 copy all of those files into a directory 9:59:59.000,9:59:59.000 that was already in the load path called, 9:59:59.000,9:59:59.000 site ruby, and that was the ruby files 9:59:59.000,9:59:59.000 you had installed that were specific 9:59:59.000,9:59:59.000 to your computer. 9:59:59.000,9:59:59.000 After setup.rb, using Ruby libraries 9:59:59.000,9:59:59.000 was much easier than it had been. 9:59:59.000,9:59:59.000 You could find a library online, 9:59:59.000,9:59:59.000 download it, you had to untar it by hand, 9:59:59.000,9:59:59.000 and run ruby setup.rb all by hand, 9:59:59.000,9:59:59.000 but then it was all installed, and no more 9:59:59.000,9:59:59.000 manual copying, no more having to 9:59:59.000,9:59:59.000 manage all these files. 9:59:59.000,9:59:59.000 Everything was in the load path, 9:59:59.000,9:59:59.000 you could just require it after setup.rb ran. 9:59:59.000,9:59:59.000 After a little while, some of the 9:59:59.000,9:59:59.000 shortcomings of this scheme became apparent, too. 9:59:59.000,9:59:59.000 There were no versions for any libraries, 9:59:59.000,9:59:59.000 and after you run setup.rb there's not even 9:59:59.000,9:59:59.000 a way to tell what version you have, unless 9:59:59.000,9:59:59.000 you write it down, or the library author 9:59:59.000,9:59:59.000 was really nice, and put the version into 9:59:59.000,9:59:59.000 the code somehow. There was no way 9:59:59.000,9:59:59.000 to uninstall, everything thrown into 9:59:59.000,9:59:59.000 the same directory. You'd run setup.rb 9:59:59.000,9:59:59.000 for 5 different Ruby libraries and 9:59:59.000,9:59:59.000 now all of their files are in one directory. 9:59:59.000,9:59:59.000 Good luck figuring out which ones belongs to which. 9:59:59.000,9:59:59.000 If you delete the wrong one, too bad. 9:59:59.000,9:59:59.000 Upgrading was super fun, if there was 9:59:59.000,9:59:59.000 a new version of the library, which 9:59:59.000,9:59:59.000 good luck finding that out, you 9:59:59.000,9:59:59.000 have to remember the website you got 9:59:59.000,9:59:59.000 it from in the first place. 9:59:59.000,9:59:59.000 I hope you write all these down. 9:59:59.000,9:59:59.000 I hope you've written down every 9:59:59.000,9:59:59.000 website you've ever downloaded Ruby from. 9:59:59.000,9:59:59.000 You have to go back to that website, 9:59:59.000,9:59:59.000 remember which version you have, which 9:59:59.000,9:59:59.000 as I said before, there's nothing there unless 9:59:59.000,9:59:59.000 you wrote it down. 9:59:59.000,9:59:59.000 And then you have to download the 9:59:59.000,9:59:59.000 tarball with the new version, and 9:59:59.000,9:59:59.000 decompress it, and CD into it and 9:59:59.000,9:59:59.000 run ruby setup.rb on it all, and 9:59:59.000,9:59:59.000 hope that the new version didn't delete 9:59:59.000,9:59:59.000 any files because the old files are still there. 9:59:59.000,9:59:59.000 This was tedious, it was really tedious. 9:59:59.000,9:59:59.000 People frequently had no idea what 9:59:59.000,9:59:59.000 was actually happening with their libraries. 9:59:59.000,9:59:59.000 It was not uncommon for people to be like 9:59:59.000,9:59:59.000 "Oh this doesn't work, I'll just fix it 9:59:59.000,9:59:59.000 in my site ruby directory, ok everything 9:59:59.000,9:59:59.000 is great now" 9:59:59.000,9:59:59.000 Super awesome. 9:59:59.000,9:59:59.000 At some point, some people were like 9:59:59.000,9:59:59.000 this is not great. What if you could just 9:59:59.000,9:59:59.000 gem install. That would be cool. 9:59:59.000,9:59:59.000 And so in 2003, RubyGems came to the rescue. 9:59:59.000,9:59:59.000 And fixed all of the problems with setup.rb 9:59:59.000,9:59:59.000 that were known. You could check 9:59:59.000,9:59:59.000 to see if a library existed by running gem list, 9:59:59.000,9:59:59.000 install a gem by gem install, uninstall gems. 9:59:59.000,9:59:59.000 RubyGems kept each of these libraries in different directories. 9:59:59.000,9:59:59.000 You knew which libraries you had, and how to uninstall 9:59:59.000,9:59:59.000 and install new versions, all with one command. 9:59:59.000,9:59:59.000 No having to find it on the internet somewhere, 9:59:59.000,9:59:59.000 download, and unpack it, setup.rb it. 9:59:59.000,9:59:59.000 And RubyGems had another super cool trick 9:59:59.000,9:59:59.000 up it's sleeves -- versions. 9:59:59.000,9:59:59.000 RubyGems actually kept each version of each 9:59:59.000,9:59:59.000 gem in a different place. You could install 9:59:59.000,9:59:59.000 multiple versions of the same library. 9:59:59.000,9:59:59.000 And they could all be in your Ruby 9:59:59.000,9:59:59.000 because they didn't all go into one giant folder, 9:59:59.000,9:59:59.000 they went into their own separate folders. 9:59:59.000,9:59:59.000 Folders for rails 4.1, 4.2, and 5.0. 9:59:59.000,9:59:59.000 To make this work, because require doesn't 9:59:59.000,9:59:59.000 support versioning, inherently, 9:59:59.000,9:59:59.000 RubyGems added a gem method that 9:59:59.000,9:59:59.000 let's you say, I need version 1.0 of rack, 9:59:59.000,9:59:59.000 and RubyGems will check to make sure it's 9:59:59.000,9:59:59.000 installed, put that directory, just the one 9:59:59.000,9:59:59.000 with rack 1.0 into your load path. 9:59:59.000,9:59:59.000 So when you run require "rack" you'll 9:59:59.000,9:59:59.000 get rack 1.0, it's pretty cool. 9:59:59.000,9:59:59.000 Calling the gem method, told RubyGems 9:59:59.000,9:59:59.000 you wanted to manipulate the load path to 9:59:59.000,9:59:59.000 load exactly the version you knew your 9:59:59.000,9:59:59.000 code wanted to talk to. 9:59:59.000,9:59:59.000 It was pretty useful. 9:59:59.000,9:59:59.000 RubyGems also has a way to support 9:59:59.000,9:59:59.000 versioning even in commands that 9:59:59.000,9:59:59.000 come with gems. The rack gem 9:59:59.000,9:59:59.000 comes with the rackup command, and 9:59:59.000,9:59:59.000 if you have multiple versions of rack installed, 9:59:59.000,9:59:59.000 the rack command could run any of those versions. 9:59:59.000,9:59:59.000 RubyGems defaults to the newest version you have 9:59:59.000,9:59:59.000 installed, hoping the newest is the right one. 9:59:59.000,9:59:59.000 But if that's not, RubyGems checks the first 9:59:59.000,9:59:59.000 argument to the command for something 9:59:59.000,9:59:59.000 with underscores on either sides, 9:59:59.000,9:59:59.000 it takes that as the version number 9:59:59.000,9:59:59.000 that you want to use. 9:59:59.000,9:59:59.000 In the above example, we're running 9:59:59.000,9:59:59.000 rackup from rack version 1.2.2, and only 1.2.2. 9:59:59.000,9:59:59.000 If you don't have that version installed, RubyGems will 9:59:59.000,9:59:59.000 make you install that version first. 9:59:59.000,9:59:59.000 RubyGems was really, really successful. 9:59:59.000,9:59:59.000 Ruby grew in popularity a lot, but RubyGems 9:59:59.000,9:59:59.000 made sharing Ruby code grow a lot. 9:59:59.000,9:59:59.000 Present day we have 100,000 gems, with 1,000,000 versions. 9:59:59.000,9:59:59.000 That's a lot of shared Ruby code. 9:59:59.000,9:59:59.000 You probably knew this was coming, 9:59:59.000,9:59:59.000 but as cool as RubyGems is, it still had 9:59:59.000,9:59:59.000 some problems. If you have multiple 9:59:59.000,9:59:59.000 applications that all use RubyGems to load 9:59:59.000,9:59:59.000 their dependencies, this can be problematic. 9:59:59.000,9:59:59.000 It's hard to coordinate across multiple applications 9:59:59.000,9:59:59.000 because, each installation of Ruby itself just has 9:59:59.000,9:59:59.000 a set of gems. If you ran gem install, now 9:59:59.000,9:59:59.000 there are all these gems. 9:59:59.000,9:59:59.000 If one developer runs gem install "foo" and 9:59:59.000,9:59:59.000 starts using "foo" in their application, 9:59:59.000,9:59:59.000 commits that code and checks it in, 9:59:59.000,9:59:59.000 and the next person checks it out 9:59:59.000,9:59:59.000 and tries to run the application, 9:59:59.000,9:59:59.000 it's going to explode, because it doesn't 9:59:59.000,9:59:59.000 know what foo is, you need to fix that. 9:59:59.000,9:59:59.000 It led to an area of pure manual dependency management. 9:59:59.000,9:59:59.000 Start a new job, hooray! 9:59:59.000,9:59:59.000 This literally happened to me in 2008. 9:59:59.000,9:59:59.000 New job, welcome to the team, here's 9:59:59.000,9:59:59.000 your cool new laptop, we 9:59:59.000,9:59:59.000 except you to have the application 9:59:59.000,9:59:59.000 running by next week. 9:59:59.000,9:59:59.000 It actually took me only 3 and a half days, 9:59:59.000,9:59:59.000 working overtime on this. It was amazing. 9:59:59.000,9:59:59.000 [Audience Laughs] 9:59:59.000,9:59:59.000 To figure out which gems to run gem install, 9:59:59.000,9:59:59.000 I looked in the README and there 9:59:59.000,9:59:59.000 was a list. 9:59:59.000,9:59:59.000 And I installed all of them. 9:59:59.000,9:59:59.000 But clearly there was some that people 9:59:59.000,9:59:59.000 forgot to put in the README, and 9:59:59.000,9:59:59.000 then it kind of worked, but I wasn't 9:59:59.000,9:59:59.000 able to get images working. And then 9:59:59.000,9:59:59.000 some other developer was like, 9:59:59.000,9:59:59.000 you need to install imagemagick, 9:59:59.000,9:59:59.000 this was before homebrew. It was terrifying. 9:59:59.000,9:59:59.000 To try and fix this problem, 9:59:59.000,9:59:59.000 of do we just put the gems in the README? 9:59:59.000,9:59:59.000 How do we know if we have 9:59:59.000,9:59:59.000 written everything in the README? 9:59:59.000,9:59:59.000 "I don't know? Try it?" 9:59:59.000,9:59:59.000 Of course, you'd need a new machine 9:59:59.000,9:59:59.000 to try it on, because after 3 years 9:59:59.000,9:59:59.000 of using Ruby you generally have 9:59:59.000,9:59:59.000 installed every gem, and you have 9:59:59.000,9:59:59.000 no idea what's important and what's not. 9:59:59.000,9:59:59.000 It's terrible. 9:59:59.000,9:59:59.000 People started to work on tools to help this problem. 9:59:59.000,9:59:59.000 Rails added config.gem, this is Rails 2.3, 2.4 era. 9:59:59.000,9:59:59.000 You would put all the gems you need in application.rb 9:59:59.000,9:59:59.000 This was super helpful if you needed 9:59:59.000,9:59:59.000 to know for sure this was the 9:59:59.000,9:59:59.000 master list of all the gems 9:59:59.000,9:59:59.000 you needed in your application, but 9:59:59.000,9:59:59.000 you could only access that list when 9:59:59.000,9:59:59.000 Rails was already loaded. 9:59:59.000,9:59:59.000 It was pretty bad. 9:59:59.000,9:59:59.000 Because RubyGems automatically uses 9:59:59.000,9:59:59.000 the newest version of each gem, just having 9:59:59.000,9:59:59.000 an older version installed, didn't mean it 9:59:59.000,9:59:59.000 would be used. And if you install 9:59:59.000,9:59:59.000 some gem a month after the other person did, 9:59:59.000,9:59:59.000 maybe there's a new version? You would 9:59:59.000,9:59:59.000 just get the new version automatically. 9:59:59.000,9:59:59.000 This is also totally a real-life experience that happened to me in 2009. 9:59:59.000,9:59:59.000 Debug a production server that just randomly throws exceptions. 9:59:59.000,9:59:59.000 For three days. 9:59:59.000,9:59:59.000 The other production servers are fine. 9:59:59.000,9:59:59.000 We can't reproduce this problem on a single developer laptop. 9:59:59.000,9:59:59.000 What is going on? This is so weird. 9:59:59.000,9:59:59.000 After 3 days I finally thought to look at 9:59:59.000,9:59:59.000 the output from the gemlist for the entire production 9:59:59.000,9:59:59.000 machine and I was like, oh this production 9:59:59.000,9:59:59.000 server has gem version 1.1.3 and every 9:59:59.000,9:59:59.000 other production server and developer laptop has 1.1.4. 9:59:59.000,9:59:59.000 That was the problem. 9:59:59.000,9:59:59.000 There was a bug and only that server 9:59:59.000,9:59:59.000 had this problem. 9:59:59.000,9:59:59.000 And then, like I was saying, about Rails versions, 9:59:59.000,9:59:59.000 you could gem install rails, be happy, 9:59:59.000,9:59:59.000 make a new app, run your server, 9:59:59.000,9:59:59.000 everything is great. And then 9:59:59.000,9:59:59.000 you switch to another application 9:59:59.000,9:59:59.000 that already existed, didn't get 9:59:59.000,9:59:59.000 written to use that version of rails, 9:59:59.000,9:59:59.000 got writen to use some older version of rails. 9:59:59.000,9:59:59.000 You're like, "Okay, let's go!" 9:59:59.000,9:59:59.000 "Boom", because you didn't have 9:59:59.000,9:59:59.000 the right version of Rails. 9:59:59.000,9:59:59.000 If you put your rails version in the rails config 9:59:59.000,9:59:59.000 rails would complain you had the wrong version, 9:59:59.000,9:59:59.000 but rails had to be successfully started up to 9:59:59.000,9:59:59.000 tell you that you had the wrong version, so 9:59:59.000,9:59:59.000 it didn't actually help. 9:59:59.000,9:59:59.000 Ultimately, it was a significant part of my job 9:59:59.000,9:59:59.000 to figure this shit out by hand, and it sucked. 9:59:59.000,9:59:59.000 Depending on what you did on your team, 9:59:59.000,9:59:59.000 some people on my team at the time spent 9:59:59.000,9:59:59.000 a quarter or a third of their time 9:59:59.000,9:59:59.000 doing nothing but figuring out and fixing 9:59:59.000,9:59:59.000 dependency management issues. 9:59:59.000,9:59:59.000 And I felt really, really bad for them. 9:59:59.000,9:59:59.000 Sometimes it was me and I felt really bad for me. 9:59:59.000,9:59:59.000 Then there's one more, even after, 9:59:59.000,9:59:59.000 you've done all of this by hand management, 9:59:59.000,9:59:59.000 there's one more problem that RubyGems has 9:59:59.000,9:59:59.000 that is another reason why bundler was created. 9:59:59.000,9:59:59.000 Activation Errors, they happen in ruby gems 9:59:59.000,9:59:59.000 when you load an application and start by 9:59:59.000,9:59:59.000 requiring gems, ruby gems will load the newest 9:59:59.000,9:59:59.000 versions of those gems that it can. 9:59:59.000,9:59:59.000 Sometimes a gem's dependents need 9:59:59.000,9:59:59.000 other gems, that need other gems, 9:59:59.000,9:59:59.000 and you'll get the newest version of the 9:59:59.000,9:59:59.000 child gem. And later you'll say, I also 9:59:59.000,9:59:59.000 need this gem, but that gem won't work with the other. 9:59:59.000,9:59:59.000 So how common can this be really? 9:59:59.000,9:59:59.000 Unfortunately, it was super common. 9:59:59.000,9:59:59.000 Not like happens to you every day common, 9:59:59.000,9:59:59.000 but like happens you two or three times a year 9:59:59.000,9:59:59.000 and when it does you basically tear 9:59:59.000,9:59:59.000 all your hair out, delete your entire 9:59:59.000,9:59:59.000 ruby install, uninstall and reinstall all your gems, 9:59:59.000,9:59:59.000 because figuring out exactly which combo 9:59:59.000,9:59:59.000 of installed gems was causing this 9:59:59.000,9:59:59.000 problem was a total nightmare. 9:59:59.000,9:59:59.000 This is a real-life activation error. 9:59:59.000,9:59:59.000 I salvaged this from a presentation I gave in 2010 9:59:59.000,9:59:59.000 about why Bundler exists. 9:59:59.000,9:59:59.000 This is a rails app, it's loading, and 9:59:59.000,9:59:59.000 rails of course depends on ActionPack, this 9:59:59.000,9:59:59.000 was the Rails 2.3 era, ActionPack depends on Rack, 9:59:59.000,9:59:59.000 Rack is a gem that helps Rails talk to web servers. 9:59:59.000,9:59:59.000 And thin, which is a web server, also depends on rack. 9:59:59.000,9:59:59.000 So, rack is how rails talks to thin, how thin 9:59:59.000,9:59:59.000 talks to rails, but there's a problem. 9:59:59.000,9:59:59.000 thin is perfectly happy to use rack 1.1, which makes some 9:59:59.000,9:59:59.000 changes to how rack works. 9:59:59.000,9:59:59.000 ActionPack is not happy to use rack 1.1, and 9:59:59.000,9:59:59.000 can only use rack 1.0. And so 9:59:59.000,9:59:59.000 when you run your server, it loads thin 9:59:59.000,9:59:59.000 first because thin is the server. 9:59:59.000,9:59:59.000 And thin gets to work trying to load the rails app 9:59:59.000,9:59:59.000 and your rails app says "I can't use that rack, sorry" 9:59:59.000,9:59:59.000 The reason this happens is runtime resolution. 9:59:59.000,9:59:59.000 RubyGems figures out which versions 9:59:59.000,9:59:59.000 of which gems of which gems it should load. 9:59:59.000,9:59:59.000 After RubyGems is already running. 9:59:59.000,9:59:59.000 You say, "Hey I need a thing", and 9:59:59.000,9:59:59.000 it's like "Okay, this version might work". 9:59:59.000,9:59:59.000 And if later on you say, [br] 9:59:59.000,9:59:59.000 "I need a thing that doesn't work with things you've already done" 9:59:59.000,9:59:59.000 RubyGems just has to be like, can't fix that. 9:59:59.000,9:59:59.000 The fix for this problem is to figure out all the versions 9:59:59.000,9:59:59.000 before you run your application. 9:59:59.000,9:59:59.000 You have to know the versions you're going 9:59:59.000,9:59:59.000 to use are all versions that can work together. 9:59:59.000,9:59:59.000 Resolving things at install time, 9:59:59.000,9:59:59.000 knowing you're installing versions that work together. 9:59:59.000,9:59:59.000 How do we make sure all the versions we're 9:59:59.000,9:59:59.000 installing work together? 9:59:59.000,9:59:59.000 That's actually where Bundler comes in. 9:59:59.000,9:59:59.000 Before Bundler, the process of figuring out 9:59:59.000,9:59:59.000 which gems would work together 9:59:59.000,9:59:59.000 was done entirely by hand and it 9:59:59.000,9:59:59.000 consisted of gem uninstall, 9:59:59.000,9:59:59.000 gem install a slighty older version, does rails start up yet? 9:59:59.000,9:59:59.000 Repeat the process. 9:59:59.000,9:59:59.000 When the exception stopped you knew you'd won. 9:59:59.000,9:59:59.000 Unsurprisingly, computers are faster at this than people. 9:59:59.000,9:59:59.000 Computers are also good and accurate at trying 9:59:59.000,9:59:59.000 many, many, many options until one works. 9:59:59.000,9:59:59.000 This is what Bundler does. 9:59:59.000,9:59:59.000 Bundler figures out the entire list of every gem 9:59:59.000,9:59:59.000 and every version of every gem that 9:59:59.000,9:59:59.000 you need, but that also all 9:59:59.000,9:59:59.000 work together with one another. 9:59:59.000,9:59:59.000 This is called Dependency Graph Resolution, 9:59:59.000,9:59:59.000 and there's an entire academic literature about this. 9:59:59.000,9:59:59.000 It's kind of well-known hard problem, it's 9:59:59.000,9:59:59.000 part of the set of problems called NP complete, 9:59:59.000,9:59:59.000 and the totally fantastic thing, and I say 9:59:59.000,9:59:59.000 this as a person who has to fix Bundler 9:59:59.000,9:59:59.000 when it doesn't work, in theory, you can construct 9:59:59.000,9:59:59.000 a set of gems in a gemfile such that 9:59:59.000,9:59:59.000 it is not possible to find a set of gems that 9:59:59.000,9:59:59.000 work together until after the heat death of the universe. 9:59:59.000,9:59:59.000 [Audience Laughs] 9:59:59.000,9:59:59.000 Most of the time we don't have that long to wait. 9:59:59.000,9:59:59.000 We use a lot of tricks, shortcuts, and heuristics 9:59:59.000,9:59:59.000 to figure out which gems to try first and 9:59:59.000,9:59:59.000 hopefully finish before you've drunk 9:59:59.000,9:59:59.000 that cup of coffee or whatever. 9:59:59.000,9:59:59.000 We have a large built-up set of tricks over the years 9:59:59.000,9:59:59.000 and most Gemfiles resolve in less than 10 seconds. 9:59:59.000,9:59:59.000 Which is pretty cool, considering the upper bound 9:59:59.000,9:59:59.000 on that is practically infinity. 9:59:59.000,9:59:59.000 After finding versions that work together 9:59:59.000,9:59:59.000 because this problem was really hard, 9:59:59.000,9:59:59.000 and we don't want to do this over and over. 9:59:59.000,9:59:59.000 Bundler writes down the exact versions of every gem 9:59:59.000,9:59:59.000 that did all work together, so they can be reused 9:59:59.000,9:59:59.000 by other people who are also interested in running 9:59:59.000,9:59:59.000 your application. That file is called Gemfile.lock. 9:59:59.000,9:59:59.000 Shows which gems to be installed, 9:59:59.000,9:59:59.000 the versions to install, and as a bonus 9:59:59.000,9:59:59.000 the lock file is what makes it possible 9:59:59.000,9:59:59.000 to install the exact same version of every 9:59:59.000,9:59:59.000 gem on every machine that's running this application. 9:59:59.000,9:59:59.000 That means when you develop on your laptop 9:59:59.000,9:59:59.000 you get whatever version of the gem that was 9:59:59.000,9:59:59.000 newest when you were developing because run 9:59:59.000,9:59:59.000 bundle install and got newest version by default. 9:59:59.000,9:59:59.000 Because of the lock file, when you put 9:59:59.000,9:59:59.000 that on your production server, you're guaranteed 9:59:59.000,9:59:59.000 to have the same versions. And you won't 9:59:59.000,9:59:59.000 have to spend 3 days figuring out why 9:59:59.000,9:59:59.000 that production server doesn't quite 9:59:59.000,9:59:59.000 work all of the time. 9:59:59.000,9:59:59.000 It's pretty great. 9:59:59.000,9:59:59.000 Fundamentally, the core of bundler consist of two steps. 9:59:59.000,9:59:59.000 bundle install, and bundle exec. 9:59:59.000,9:59:59.000 The steps for bundle install are simple. 9:59:59.000,9:59:59.000 They're totally understandable in plain english 9:59:59.000,9:59:59.000 It fits on a single slide, which is great. 9:59:59.000,9:59:59.000 I edited this slide for ten minutes deleting words. 9:59:59.000,9:59:59.000 So the steps are: 9:59:59.000,9:59:59.000 1. Read the Gemfile 9:59:59.000,9:59:59.000 2.Ask RubyGems.org for a list of all the gems we need 9:59:59.000,9:59:59.000 3. Find versions of those gems both allowed by Gemfile 9:59:59.000,9:59:59.000 4. Once found, write all those down in lock and install them all. 9:59:59.000,9:59:59.000 And that's how bundle install works. 9:59:59.000,9:59:59.000 BundleInstall uses RubyGems under the covers 9:59:59.000,9:59:59.000 to the installation, and so every 9:59:59.000,9:59:59.000 bundle is it's own little rubygems isolated install. 9:59:59.000,9:59:59.000 Every application has it's own rubygems thanks to bundler. 9:59:59.000,9:59:59.000 The next step is bundle exec. 9:59:59.000,9:59:59.000 This is how we use that applications dedicated ruby gems 9:59:59.000,9:59:59.000 instead of the one with whatever in it 9:59:59.000,9:59:59.000 because you ran gem install last year. 9:59:59.000,9:59:59.000 The way bundle exec works is: 9:59:59.000,9:59:59.000 1. Reads the Gemfile, and lock if it's there. 9:59:59.000,9:59:59.000 2a. Use locked gems if possible OR 9:59:59.000,9:59:59.000 2b. Find versions that work together like install would. 9:59:59.000,9:59:59.000 except bundle exec doesn't do any installing. 9:59:59.000,9:59:59.000 3. Deletes any existing gems in the LOAD_PATH 9:59:59.000,9:59:59.000 4. Adds the exact gem at the exact version at the load path. 9:59:59.000,9:59:59.000 That's it. That's all bundle exec does. 9:59:59.000,9:59:59.000 Once all the gems work together, and 9:59:59.000,9:59:59.000 there exact versions are in the load path 9:59:59.000,9:59:59.000 your application is happy. There is no 9:59:59.000,9:59:59.000 activation errors, all your requires succeed, I hope. 9:59:59.000,9:59:59.000 Everything is pretty great. 9:59:59.000,9:59:59.000 As I think I promised in the abstract for this talk, 9:59:59.000,9:59:59.000 here's a bundle exec removing pro tip. 9:59:59.000,9:59:59.000 I don't really like typing bundle exec, I find it 9:59:59.000,9:59:59.000 really annoying, but bundler provides a way 9:59:59.000,9:59:59.000 to not have to type it all the time. 9:59:59.000,9:59:59.000 And it's to create programs that map to 9:59:59.000,9:59:59.000 ruby gems installation that 9:59:59.000,9:59:59.000 belongs to that application. 9:59:59.000,9:59:59.000 You can use the binstubs command, 9:59:59.000,9:59:59.000 bundle binstubs [some gem] 9:59:59.000,9:59:59.000 and it will create, in the bin directory, 9:59:59.000,9:59:59.000 a program for that gem, that only 9:59:59.000,9:59:59.000 runs the exact version that belongs to 9:59:59.000,9:59:59.000 that application. So if you have 9:59:59.000,9:59:59.000 rspec in your rails app, you can have 9:59:59.000,9:59:59.000 bin/rspec that will only load the rspec 9:59:59.000,9:59:59.000 for your app. This way you can have 9:59:59.000,9:59:59.000 bin/rspec refer to rspec 3, and this application 9:59:59.000,9:59:59.000 can have rspec 2. Rails has started to do this. 9:59:59.000,9:59:59.000 Rails 4 ships with bin/rails bin/rake that are scoped 9:59:59.000,9:59:59.000 so when you run bin/rails, you get the exact 9:59:59.000,9:59:59.000 rails version for this application and not another one. 9:59:59.000,9:59:59.000 When you run bin/rake you get the exact version of rake. 9:59:59.000,9:59:59.000 Pretty cool, no more bundle exec. 9:59:59.000,9:59:59.000 If everyone did this, you can check in these binstubs 9:59:59.000,9:59:59.000 so you can take bin/rspec, but it in git, 9:59:59.000,9:59:59.000 and it'll be mapped to that application forever, 9:59:59.000,9:59:59.000 so no one would have bundle exec 9:59:59.000,9:59:59.000 ever again if everyone did this. 9:59:59.000,9:59:59.000 Now we bundle install, all our gems 9:59:59.000,9:59:59.000 show up. We have versions 9:59:59.000,9:59:59.000 dedicated for individual applications. 9:59:59.000,9:59:59.000 But, as you probably sensed a problem 9:59:59.000,9:59:59.000 going through history, that wasn't actually 9:59:59.000,9:59:59.000 the end. There are still problems 9:59:59.000,9:59:59.000 that show up after bundler came out. 9:59:59.000,9:59:59.000 The biggest problem that was left was 9:59:59.000,9:59:59.000 running bundle install, took forever. 9:59:59.000,9:59:59.000 If you lived a long time from the United States 9:59:59.000,9:59:59.000 it took a really long time. 9:59:59.000,9:59:59.000 I talked to some developers in South Africa 9:59:59.000,9:59:59.000 when I went there to give a talk 9:59:59.000,9:59:59.000 and they told me about how running 9:59:59.000,9:59:59.000 bundle install means they literally get 9:59:59.000,9:59:59.000 up to start making a cup of coffee 9:59:59.000,9:59:59.000 that they can finish before bundle install does. 9:59:59.000,9:59:59.000 To try and speed things up, bundler 1.1 9:59:59.000,9:59:59.000 created a completely different 9:59:59.000,9:59:59.000 way to get information from rubygems about gems. 9:59:59.000,9:59:59.000 And that sped things up by 50%, a big win. 9:59:59.000,9:59:59.000 We keep working on this, bundler 1.9 just 9:59:59.000,9:59:59.000 came out this month. There's a bunch more 9:59:59.000,9:59:59.000 improvements we're working on. 9:59:59.000,9:59:59.000 If you're interested in following along with that, 9:59:59.000,9:59:59.000 the bundler websites has news annoucements 9:59:59.000,9:59:59.000 at bundler.io, and twitter we're also @bundlerio. 9:59:59.000,9:59:59.000 Having said all of this, if you use Bundler, 9:59:59.000,9:59:59.000 I would totally love to have your help working on it. 9:59:59.000,9:59:59.000 It's an open source project. 9:59:59.000,9:59:59.000 We've dedicated a lot of time to making it easy 9:59:59.000,9:59:59.000 for people who don't know how to do open source 9:59:59.000,9:59:59.000 to help with Bundler, and to start working on Bundler, 9:59:59.000,9:59:59.000 and to get into open source that way. 9:59:59.000,9:59:59.000 It's a project at Github.com/bundler/bundler. 9:59:59.000,9:59:59.000 If you're interested but don't know where to start 9:59:59.000,9:59:59.000 email the bundler team at team@bundler.io 9:59:59.000,9:59:59.000 and we'll get you set up. 9:59:59.000,9:59:59.000 On the other hand, if you 9:59:59.000,9:59:59.000 have a job that means you have money, 9:59:59.000,9:59:59.000 but not time, join Ruby Together, and give 9:59:59.000,9:59:59.000 us money, and we'll work on Bundler, and it'll be 9:59:59.000,9:59:59.000 better. As RubyTogether grows, we will also be 9:59:59.000,9:59:59.000 tackling bigger community issues. 9:59:59.000,9:59:59.000 We want to add easy to use gem mirrors so you 9:59:59.000,9:59:59.000 don't have to go all the way to rubygems.org 9:59:59.000,9:59:59.000 for your office or data center, we want to 9:59:59.000,9:59:59.000 add better public benchmarks. There's a project 9:59:59.000,9:59:59.000 calling ruby-bench that's starting to do that, 9:59:59.000,9:59:59.000 and we'd really like to expand it. 9:59:59.000,9:59:59.000 There's a bunch of other things 9:59:59.000,9:59:59.000 that RubyTogether is working on that are cool 9:59:59.000,9:59:59.000 If you want Bundler or RubyTogether stickers 9:59:59.000,9:59:59.000 I have a giant pile, so find me later. 9:59:59.000,9:59:59.000 That's it. 9:59:59.000,9:59:59.000 [Audience Applause]