[Script Info]
Title: 
[Events]
Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
Dialogue: 0,0:00:00.00,0:00:06.54,Default,,0000,0000,0000,,I guess we should do an intro to to this as well,
Dialogue: 0,0:00:06.54,0:00:09.58,Default,,0000,0000,0000,,so this is a just sort of a
Dialogue: 0,0:00:09.58,0:00:14.74,Default,,0000,0000,0000,,free-form Q&A lecture where you, as in\Nthe two people sitting here, but also
Dialogue: 0,0:00:14.74,0:00:19.84,Default,,0000,0000,0000,,everyone at home who did not come here\Nin person get to ask questions and we
Dialogue: 0,0:00:19.84,0:00:22.96,Default,,0000,0000,0000,,have a bunch of questions people asked\Nin advance but you can also ask
Dialogue: 0,0:00:22.96,0:00:27.37,Default,,0000,0000,0000,,additional questions during, for the two\Nof you who are here, you can do it either
Dialogue: 0,0:00:27.37,0:00:33.61,Default,,0000,0000,0000,,by raising your hand or you can submit it on\Nthe forum and be anonymous, it's up to you
Dialogue: 0,0:00:33.61,0:00:35.67,Default,,0000,0000,0000,,regardless though, what we're gonna\Ndo is just go through some of the
Dialogue: 0,0:00:35.68,0:00:40.24,Default,,0000,0000,0000,,questions have been asked and try to\Ngive as helpful answers as we can
Dialogue: 0,0:00:40.24,0:00:43.69,Default,,0000,0000,0000,,although they are unprepared on our side and
Dialogue: 0,0:00:43.79,0:00:45.61,Default,,0000,0000,0000,,yeah that's the plan I guess we go
Dialogue: 0,0:00:45.61,0:00:48.91,Default,,0000,0000,0000,,from popular to least popular
Dialogue: 0,0:00:48.91,0:00:49.99,Default,,0000,0000,0000,,fire away
Dialogue: 0,0:00:49.99,0:00:52.09,Default,,0000,0000,0000,,all right so for our first question any
Dialogue: 0,0:00:52.09,0:00:55.96,Default,,0000,0000,0000,,recommendations on learning operating\Nsystem related topics like processes,
Dialogue: 0,0:00:55.96,0:00:59.86,Default,,0000,0000,0000,,virtual memory, interrupts,\Nmemory management, etc
Dialogue: 0,0:00:59.86,0:01:01.81,Default,,0000,0000,0000,,so I think this is a
Dialogue: 0,0:01:01.81,0:01:07.18,Default,,0000,0000,0000,,is an interesting question because these\Nare really low level concepts that often
Dialogue: 0,0:01:07.18,0:01:11.39,Default,,0000,0000,0000,,do not matter, unless you have to\Ndeal with this in some capacity,
Dialogue: 0,0:01:11.39,0:01:12.77,Default,,0000,0000,0000,,right so
Dialogue: 0,0:01:12.89,0:01:17.67,Default,,0000,0000,0000,,one instance where this matters is you're\Nwriting really low level code like
Dialogue: 0,0:01:17.68,0:01:20.50,Default,,0000,0000,0000,,you're implementing a kernel or something\Nlike that, or you want to
Dialogue: 0,0:01:20.50,0:01:22.81,Default,,0000,0000,0000,,just hack on the Linux kernel.
Dialogue: 0,0:01:22.81,0:01:24.75,Default,,0000,0000,0000,,It's rare otherwise that you need to work with
Dialogue: 0,0:01:24.75,0:01:27.71,Default,,0000,0000,0000,,especially like virtual memory and\Ninterrupts and stuff yourself
Dialogue: 0,0:01:27.85,0:01:32.07,Default,,0000,0000,0000,,processes, I think are a more general concept\Nthat we've talked a little bit about in
Dialogue: 0,0:01:32.07,0:01:36.61,Default,,0000,0000,0000,,this class as well and tools like\Nhtop, pgrep, kill, and signals and
Dialogue: 0,0:01:36.76,0:01:37.71,Default,,0000,0000,0000,,that sort of stuff
Dialogue: 0,0:01:37.71,0:01:39.31,Default,,0000,0000,0000,,in terms of learning it
Dialogue: 0,0:01:39.31,0:01:45.37,Default,,0000,0000,0000,,maybe one of the best ways, is to try to\Ntake either an introductory class on the
Dialogue: 0,0:01:45.37,0:01:51.40,Default,,0000,0000,0000,,topic, so for example MIT has a class\Ncalled 6.828, which is where
Dialogue: 0,0:01:51.40,0:01:55.09,Default,,0000,0000,0000,,you essentially build and develop your\Nown operating system based on some code
Dialogue: 0,0:01:55.09,0:01:58.63,Default,,0000,0000,0000,,that you're given, and all of those labs\Nare publicly available and all the
Dialogue: 0,0:01:58.63,0:02:01.60,Default,,0000,0000,0000,,resources for the class are publicly available,\Nand so that is a good way to
Dialogue: 0,0:02:01.60,0:02:04.00,Default,,0000,0000,0000,,really learn them is by doing them yourself.
Dialogue: 0,0:02:04.00,0:02:05.20,Default,,0000,0000,0000,,There are also various
Dialogue: 0,0:02:05.20,0:02:11.20,Default,,0000,0000,0000,,tutorials online that basically guide\Nyou through how do you write a kernel
Dialogue: 0,0:02:11.20,0:02:15.43,Default,,0000,0000,0000,,from scratch. Not necessarily a very\Nelaborate one, not one you would want
Dialogue: 0,0:02:15.43,0:02:20.56,Default,,0000,0000,0000,,to run any real software on, but just to\Nteach you the basics and so that would
Dialogue: 0,0:02:20.56,0:02:21.93,Default,,0000,0000,0000,,be another thing to look up.
Dialogue: 0,0:02:21.93,0:02:24.13,Default,,0000,0000,0000,,Like how do I write a kernel in and then your
Dialogue: 0,0:02:24.13,0:02:27.61,Default,,0000,0000,0000,,language of choice. You will probably not\Nfind one that lets you do it in Python
Dialogue: 0,0:02:27.61,0:02:33.61,Default,,0000,0000,0000,,but in like C, C++, Rust, there\Nare a bunch of topics like this
Dialogue: 0,0:02:33.61,0:02:36.95,Default,,0000,0000,0000,,one other note on operating systems
Dialogue: 0,0:02:36.95,0:02:39.93,Default,,0000,0000,0000,,so like Jon mentioned MIT has a 6.828 class but
Dialogue: 0,0:02:39.94,0:02:43.39,Default,,0000,0000,0000,,if you're looking for a more high-level\Noverview, not necessarily programming or
Dialogue: 0,0:02:43.39,0:02:46.00,Default,,0000,0000,0000,,an operating system, but just learning about\Nthe concepts another good resource
Dialogue: 0,0:02:46.00,0:02:51.33,Default,,0000,0000,0000,,is a book called "Modern Operating\NSystems" by Andy Tannenbaum
Dialogue: 0,0:02:51.33,0:02:58.37,Default,,0000,0000,0000,,there's also actually a book called the "The FreeBSD\NOperating System" which is really good,
Dialogue: 0,0:02:58.37,0:03:03.03,Default,,0000,0000,0000,,It doesn't go through Linux, but it goes\Nthrough FreeBSD and the BSD kernel is
Dialogue: 0,0:03:03.03,0:03:07.18,Default,,0000,0000,0000,,arguably better organized than the Linux\None and better documented and so it
Dialogue: 0,0:03:07.18,0:03:11.59,Default,,0000,0000,0000,,might be a gentler introduction to some of those\Ntopics than trying to understand Linux
Dialogue: 0,0:03:11.59,0:03:14.95,Default,,0000,0000,0000,,You want to check it as answered?
Dialogue: 0,0:03:14.95,0:03:16.51,Default,,0000,0000,0000,,- Yes + Nice
Dialogue: 0,0:03:16.51,0:03:17.45,Default,,0000,0000,0000,,Answered
Dialogue: 0,0:03:17.45,0:03:19.37,Default,,0000,0000,0000,,For our next question,
Dialogue: 0,0:03:19.37,0:03:23.95,Default,,0000,0000,0000,,What are some of the tools you'd\Nprioritize learning first?
Dialogue: 0,0:03:23.95,0:03:29.55,Default,,0000,0000,0000,,- Maybe we can all go through and\Ngive our opinion on this? + Yeah
Dialogue: 0,0:03:29.55,0:03:31.71,Default,,0000,0000,0000,,Tools to prioritize learning first?
Dialogue: 0,0:03:31.71,0:03:36.45,Default,,0000,0000,0000,,I think learning your editor well,\Njust serves you in all capacities
Dialogue: 0,0:03:36.51,0:03:40.51,Default,,0000,0000,0000,,like being efficient at editing files,\Nis just like a majority of
Dialogue: 0,0:03:40.51,0:03:45.04,Default,,0000,0000,0000,,what you're going to spend your time doing.\NAnd in general, just using your
Dialogue: 0,0:03:45.04,0:03:49.21,Default,,0000,0000,0000,,keyboard more and your mouse less. It means\Nthat you get to spend more of your
Dialogue: 0,0:03:49.31,0:03:53.75,Default,,0000,0000,0000,,time doing useful things and\Nless of your time moving
Dialogue: 0,0:03:53.75,0:03:56.25,Default,,0000,0000,0000,,I think that would be my top priority,
Dialogue: 0,0:04:04.51,0:04:06.75,Default,,0000,0000,0000,,so I would say that for what
Dialogue: 0,0:04:06.76,0:04:09.67,Default,,0000,0000,0000,,tool to prioritize will depend\Non what exactly you're doing
Dialogue: 0,0:04:09.67,0:04:16.15,Default,,0000,0000,0000,,I think the core idea is you should try\Nto find the types of tasks that you are
Dialogue: 0,0:04:16.15,0:04:18.37,Default,,0000,0000,0000,,doing repetitively and so
Dialogue: 0,0:04:18.37,0:04:23.79,Default,,0000,0000,0000,,if you are doing some sort of like\Nmachine learning workload and
Dialogue: 0,0:04:24.01,0:04:27.13,Default,,0000,0000,0000,,you find yourself using Jupyter notebooks,\Nlike the one we presented
Dialogue: 0,0:04:27.13,0:04:32.56,Default,,0000,0000,0000,,yesterday, a lot. Then again, using\Na mouse for that might not be
Dialogue: 0,0:04:32.56,0:04:35.83,Default,,0000,0000,0000,,the best idea and you want to familiarize\Nwith the keyboard shortcuts
Dialogue: 0,0:04:35.83,0:04:40.75,Default,,0000,0000,0000,,and pretty much with anything you will\Nend up figuring out that there are some
Dialogue: 0,0:04:40.75,0:04:45.61,Default,,0000,0000,0000,,repetitive tasks, and you're running a\Ncomputer, and just trying to figure out
Dialogue: 0,0:04:45.61,0:04:48.31,Default,,0000,0000,0000,,oh there's probably a better way to do this
Dialogue: 0,0:04:48.43,0:04:50.87,Default,,0000,0000,0000,,be it a terminal, be it an editor
Dialogue: 0,0:04:51.11,0:04:55.89,Default,,0000,0000,0000,,And it might be really interesting to\Nlearn to use some of the topics that
Dialogue: 0,0:04:55.90,0:05:01.12,Default,,0000,0000,0000,,we have covered, but if they're not\Nextremely useful in a everyday
Dialogue: 0,0:05:01.12,0:05:05.43,Default,,0000,0000,0000,,basis then it might not be worth prioritizing them
Dialogue: 0,0:05:06.59,0:05:07.45,Default,,0000,0000,0000,,Out of the topics
Dialogue: 0,0:05:07.53,0:05:11.61,Default,,0000,0000,0000,,covered in this class, in my opinion, two\Nof the most useful things are version
Dialogue: 0,0:05:11.62,0:05:15.22,Default,,0000,0000,0000,,control and text editors, and I think they're\Na little bit different from each
Dialogue: 0,0:05:15.22,0:05:18.88,Default,,0000,0000,0000,,other, in the sense that text editors I\Nthink are really useful to learn well,
Dialogue: 0,0:05:18.88,0:05:21.97,Default,,0000,0000,0000,,but it was probably the case that before\Nwe started using Vim and all its fancy
Dialogue: 0,0:05:21.97,0:05:25.39,Default,,0000,0000,0000,,keyboard shortcuts you had some other\Ntext editor you were using before and
Dialogue: 0,0:05:25.39,0:05:29.89,Default,,0000,0000,0000,,you could edit text just fine maybe a little\Nbit inefficiently, whereas I think
Dialogue: 0,0:05:29.89,0:05:33.10,Default,,0000,0000,0000,,version control is another really useful\Nskill and that's one where if you don't
Dialogue: 0,0:05:33.10,0:05:36.58,Default,,0000,0000,0000,,really know the tool properly, it can actually\Nlead to some problems like loss
Dialogue: 0,0:05:36.58,0:05:39.49,Default,,0000,0000,0000,,of data or just inability to collaborate\Nproperly with people. So I
Dialogue: 0,0:05:39.49,0:05:42.73,Default,,0000,0000,0000,,think version control is one of the first\Nthings that's worth learning well.
Dialogue: 0,0:05:42.73,0:05:46.87,Default,,0000,0000,0000,,Yeah, I agree with that. I think\Nlearning a tool like Git is just
Dialogue: 0,0:05:46.87,0:05:49.69,Default,,0000,0000,0000,,gonna save you so much heartache down the line.
Dialogue: 0,0:05:49.69,0:05:51.43,Default,,0000,0000,0000,,It, also, to add on to that,
Dialogue: 0,0:05:51.57,0:05:57.31,Default,,0000,0000,0000,,it really helps you collaborate with others,\Nand Anish touched a little bit on GitHub
Dialogue: 0,0:05:57.31,0:06:01.30,Default,,0000,0000,0000,,in the last lecture, and just learning\Nto use that tool well in order
Dialogue: 0,0:06:01.30,0:06:05.32,Default,,0000,0000,0000,,to work on larger software projects\Nthat other people are working on is
Dialogue: 0,0:06:05.32,0:06:06.43,Default,,0000,0000,0000,,an invaluable skill.
Dialogue: 0,0:06:10.07,0:06:11.39,Default,,0000,0000,0000,,For our next question,
Dialogue: 0,0:06:11.39,0:06:12.87,Default,,0000,0000,0000,,"When do I use Python versus a
Dialogue: 0,0:06:12.88,0:06:16.05,Default,,0000,0000,0000,,Bash script versus some other language?"
Dialogue: 0,0:06:16.05,0:06:19.66,Default,,0000,0000,0000,,This is tough, because I think this comes
Dialogue: 0,0:06:19.66,0:06:21.63,Default,,0000,0000,0000,,down to what Jose was saying earlier too,
Dialogue: 0,0:06:21.77,0:06:23.73,Default,,0000,0000,0000,,that it really depends on\Nwhat you're trying to do.
Dialogue: 0,0:06:23.73,0:06:27.16,Default,,0000,0000,0000,,For me, I think for Bash scripts in particular,
Dialogue: 0,0:06:27.16,0:06:28.79,Default,,0000,0000,0000,,Bash scripts are for
Dialogue: 0,0:06:28.89,0:06:33.43,Default,,0000,0000,0000,,automating running a bunch of commands.\NYou don't want to write any
Dialogue: 0,0:06:33.43,0:06:35.41,Default,,0000,0000,0000,,other, like, business logic in Bash.
Dialogue: 0,0:06:35.41,0:06:39.01,Default,,0000,0000,0000,,Like, it is just for, 'I want to run these
Dialogue: 0,0:06:39.01,0:06:44.11,Default,,0000,0000,0000,,commands, in this order... maybe with\Narguments?' But - but, like, even that,
Dialogue: 0,0:06:44.11,0:06:47.58,Default,,0000,0000,0000,,it's unclear that you want a Bash script\Nonce you start taking arguments.
Dialogue: 0,0:06:47.58,0:06:52.69,Default,,0000,0000,0000,,Similarly, like, once you start doing any\Nkind of, like, text processing, or
Dialogue: 0,0:06:52.69,0:06:55.13,Default,,0000,0000,0000,,configuration, all that,
Dialogue: 0,0:06:55.13,0:06:59.11,Default,,0000,0000,0000,,reach for a language that is... a more, a more serious
Dialogue: 0,0:06:59.11,0:07:01.03,Default,,0000,0000,0000,,programming language than Bash is.
Dialogue: 0,0:07:01.09,0:07:03.45,Default,,0000,0000,0000,,Bash is really for short, one-off
Dialogue: 0,0:07:03.46,0:07:10.21,Default,,0000,0000,0000,,scripts, or ones that have a very well-defined\Nuse case, on the terminal, in
Dialogue: 0,0:07:10.21,0:07:12.85,Default,,0000,0000,0000,,the shell, probably.
Dialogue: 0,0:07:12.85,0:07:15.94,Default,,0000,0000,0000,,For a slightly more concrete guideline,\Nyou might say, 'Write a
Dialogue: 0,0:07:15.94,0:07:19.21,Default,,0000,0000,0000,,Bash script if it's less than a hundred\Nlines of code or so', but once it gets
Dialogue: 0,0:07:19.21,0:07:21.61,Default,,0000,0000,0000,,beyond that point, Bash is kind of\Nunwieldy, and it's probably worth
Dialogue: 0,0:07:21.61,0:07:25.09,Default,,0000,0000,0000,,switching to a more serious programming\Nlanguage, like Python.
Dialogue: 0,0:07:25.09,0:07:26.51,Default,,0000,0000,0000,,And, to add to that,
Dialogue: 0,0:07:26.51,0:07:32.21,Default,,0000,0000,0000,,I would say, like, I found myself writing,\Nsometimes, scripts in Python, because
Dialogue: 0,0:07:32.21,0:07:36.91,Default,,0000,0000,0000,,if I have already solved some subproblem\Nthat covers part of the problem in Python,
Dialogue: 0,0:07:36.91,0:07:40.63,Default,,0000,0000,0000,,I find it much easier to compose the\Nprevious solution that I found out in
Dialogue: 0,0:07:40.63,0:07:45.73,Default,,0000,0000,0000,,Python than just try to reuse Bash code,\Nthat I don't find as reusable as Python.
Dialogue: 0,0:07:45.73,0:07:49.60,Default,,0000,0000,0000,,And in the same way it's kind of nice that\Na lot of people have written something
Dialogue: 0,0:07:49.60,0:07:52.63,Default,,0000,0000,0000,,like Python libraries or like Ruby libraries\Nto do a lot of these things,
Dialogue: 0,0:07:52.63,0:07:58.45,Default,,0000,0000,0000,,whereas, in Bash, it's kind of hard\Nto have, like, code reuse.
Dialogue: 0,0:07:58.45,0:08:01.72,Default,,0000,0000,0000,,And, in fact,
Dialogue: 0,0:08:01.72,0:08:07.63,Default,,0000,0000,0000,,I think to add to that, usually, if you\Nfind a library, in some language that
Dialogue: 0,0:08:07.63,0:08:12.09,Default,,0000,0000,0000,,helps with the task you're trying to\Ndo, use that language for the job.
Dialogue: 0,0:08:12.09,0:08:15.67,Default,,0000,0000,0000,,And in Bash, there are no libraries. There\Nare only the programs on your computer.
Dialogue: 0,0:08:15.77,0:08:18.93,Default,,0000,0000,0000,,So you probably don't want to use\Nit, unless like there's a program
Dialogue: 0,0:08:18.94,0:08:23.74,Default,,0000,0000,0000,,you can just invoke. I do think another\Nthing worth remembering about Bash is:
Dialogue: 0,0:08:23.74,0:08:26.45,Default,,0000,0000,0000,,Bash is really hard to get right.
Dialogue: 0,0:08:26.45,0:08:30.53,Default,,0000,0000,0000,,It's very easy to get it right for the particular\Nuse case you're trying to solve right now,
Dialogue: 0,0:08:30.53,0:08:32.47,Default,,0000,0000,0000,,but, like, things like,
Dialogue: 0,0:08:32.47,0:08:35.89,Default,,0000,0000,0000,,"What if one of the filenames has a space in it?"
Dialogue: 0,0:08:35.89,0:08:38.89,Default,,0000,0000,0000,,It has caused so many bugs, and so
Dialogue: 0,0:08:38.89,0:08:43.15,Default,,0000,0000,0000,,many problems in Bash scripts. And, if you\Nuse a - a real programming language, then
Dialogue: 0,0:08:43.15,0:08:46.64,Default,,0000,0000,0000,,those problems just go away.
Dialogue: 0,0:08:46.65,0:08:50.49,Default,,0000,0000,0000,,Yes! Checked it.
Dialogue: 0,0:08:50.57,0:08:51.57,Default,,0000,0000,0000,,For our next question,
Dialogue: 0,0:08:51.57,0:08:56.21,Default,,0000,0000,0000,,what is the difference between sourcing\Na script, and executing that script?
Dialogue: 0,0:08:57.07,0:09:02.71,Default,,0000,0000,0000,,Ooh. So, this, actually, we got in office\Nhours a - a while back, as well, which is,
Dialogue: 0,0:09:02.87,0:09:06.99,Default,,0000,0000,0000,,'Aren't they the same? Like, aren't they\Nboth just running the Bash script?'
Dialogue: 0,0:09:06.99,0:09:08.05,Default,,0000,0000,0000,,And, it is true
Dialogue: 0,0:09:08.05,0:09:12.19,Default,,0000,0000,0000,,both of these will end up executing the\Nlines of code that are in the script.
Dialogue: 0,0:09:12.19,0:09:16.57,Default,,0000,0000,0000,,The ways in which they differ is that\Nsourcing a script is telling your
Dialogue: 0,0:09:16.57,0:09:22.99,Default,,0000,0000,0000,,current Bash script, your current Bash\Nsession, to execute that program,
Dialogue: 0,0:09:23.13,0:09:28.91,Default,,0000,0000,0000,,whereas the other one is, 'Start up a new instance\Nof Bash, and run the program there, instead.'
Dialogue: 0,0:09:29.29,0:09:34.93,Default,,0000,0000,0000,,And, this matters for things like... Imagine that\N"script.sh" tries to change directories.
Dialogue: 0,0:09:34.93,0:09:37.84,Default,,0000,0000,0000,,If you are running the script,\Nas in the second invocation,
Dialogue: 0,0:09:37.84,0:09:42.76,Default,,0000,0000,0000,,"./script.sh", then the new\Nprocess is going to change
Dialogue: 0,0:09:42.76,0:09:46.89,Default,,0000,0000,0000,,directories. But, by the time that script\Nexits, and returns to your shell,
Dialogue: 0,0:09:46.89,0:09:51.83,Default,,0000,0000,0000,,your shell still remains in the same place. However,\Nif you do "cd" in a script, and you "source" it,
Dialogue: 0,0:09:51.83,0:09:55.24,Default,,0000,0000,0000,,your current instance of Bash is the\None that ends up running it, and
Dialogue: 0,0:09:55.24,0:09:57.95,Default,,0000,0000,0000,,so, it ends up "cd"-ing where you are.
Dialogue: 0,0:09:57.95,0:10:01.17,Default,,0000,0000,0000,,This is also why, if you define functions,
Dialogue: 0,0:10:01.17,0:10:04.75,Default,,0000,0000,0000,,for example, that you may want to\Nexecute in your shell session,
Dialogue: 0,0:10:04.75,0:10:07.01,Default,,0000,0000,0000,,you need to source the script, not run it,
Dialogue: 0,0:10:07.01,0:10:10.26,Default,,0000,0000,0000,,because if you run it, that function\Nwill be defined in the
Dialogue: 0,0:10:10.26,0:10:11.93,Default,,0000,0000,0000,,instance of Bash,
Dialogue: 0,0:10:11.93,0:10:16.83,Default,,0000,0000,0000,,in the Bash process that gets launched, but it\Nwill not be defined in your current shell.
Dialogue: 0,0:10:16.83,0:10:22.87,Default,,0000,0000,0000,,I think those are two of the biggest\Ndifferences between the two.
Dialogue: 0,0:10:29.21,0:10:29.71,Default,,0000,0000,0000,,Next question...
Dialogue: 0,0:10:29.87,0:10:35.13,Default,,0000,0000,0000,,"What are the places where various packages and tools\Nare stored and how does referencing them work?
Dialogue: 0,0:10:35.13,0:10:39.17,Default,,0000,0000,0000,,What even is /bin or /lib?"
Dialogue: 0,0:10:39.17,0:10:45.09,Default,,0000,0000,0000,,So, as we covered in the first lecture,\Nthere is this PATH environment variable,
Dialogue: 0,0:10:45.09,0:10:49.55,Default,,0000,0000,0000,,which is like a semicolon-separated-\Nstring of all the places
Dialogue: 0,0:10:49.55,0:10:55.11,Default,,0000,0000,0000,,where your shell is gonna look for binaries.\NAnd, if you just do something like
Dialogue: 0,0:10:55.11,0:10:58.17,Default,,0000,0000,0000,,"echo $PATH", you're gonna get this list;
Dialogue: 0,0:10:58.17,0:11:02.25,Default,,0000,0000,0000,,all these places are gonna\Nbe consulted, in order.
Dialogue: 0,0:11:02.25,0:11:03.60,Default,,0000,0000,0000,,It's gonna go through all of them, and, in fact,
Dialogue: 0,0:11:03.60,0:11:07.01,Default,,0000,0000,0000,,- There is already... Did we cover which? + Yeah
Dialogue: 0,0:11:07.21,0:11:10.01,Default,,0000,0000,0000,,So, if you run "which", and a specific command,
Dialogue: 0,0:11:10.02,0:11:14.07,Default,,0000,0000,0000,,the shell is actually gonna tell\Nyou where it's finding this (command).
Dialogue: 0,0:11:14.07,0:11:15.39,Default,,0000,0000,0000,,Beyond that,
Dialogue: 0,0:11:15.39,0:11:20.43,Default,,0000,0000,0000,,there is like some conventions where a lot\Nof programs will install their binaries
Dialogue: 0,0:11:20.43,0:11:24.07,Default,,0000,0000,0000,,and they're like /usr/bin (or at\Nleast they will include symlinks)
Dialogue: 0,0:11:24.07,0:11:26.05,Default,,0000,0000,0000,,in /usr/bin so you can find them
Dialogue: 0,0:11:26.19,0:11:28.21,Default,,0000,0000,0000,,There's also a /usr/local/bin
Dialogue: 0,0:11:28.21,0:11:33.95,Default,,0000,0000,0000,,There are special directories. For example,\N/usr/sbin it's only for sudo user and
Dialogue: 0,0:11:33.95,0:11:38.49,Default,,0000,0000,0000,,some of these conventions are slightly\Ndifferent between different distros so
Dialogue: 0,0:11:38.49,0:11:47.57,Default,,0000,0000,0000,,I know like some distros for example install\Nthe user libraries under /opt for example
Dialogue: 0,0:11:51.19,0:11:55.49,Default,,0000,0000,0000,,Yeah I think one thing just\Nto talk a little bit of more
Dialogue: 0,0:11:55.65,0:12:00.63,Default,,0000,0000,0000,,about /bin and then Anish maybe you can\Ndo the other folders so when it comes to
Dialogue: 0,0:12:00.63,0:12:02.79,Default,,0000,0000,0000,,/bin the convention
Dialogue: 0,0:12:02.79,0:12:10.05,Default,,0000,0000,0000,,There are conventions, and the conventions are\Nusually /bin are for essential system utilities
Dialogue: 0,0:12:10.05,0:12:12.53,Default,,0000,0000,0000,,/usr/bin are for user programs and
Dialogue: 0,0:12:12.53,0:12:17.43,Default,,0000,0000,0000,,/usr/local/bin are for user\Ncompiled programs, sort of
Dialogue: 0,0:12:17.43,0:12:21.69,Default,,0000,0000,0000,,so things that you installed that you intend\Nthe user to run, are in /usr/bin
Dialogue: 0,0:12:21.69,0:12:26.71,Default,,0000,0000,0000,,things that a user has compiled themselves and stuck\Non your system, probably goes in /usr/local/bin
Dialogue: 0,0:12:26.71,0:12:29.99,Default,,0000,0000,0000,,but again, this varies a lot from machine\Nto machine, and distro to distro
Dialogue: 0,0:12:29.99,0:12:33.97,Default,,0000,0000,0000,,On Arch Linux, for example, /bin\Nis a symlink to /usr/bin
Dialogue: 0,0:12:33.97,0:12:40.26,Default,,0000,0000,0000,,They're the same and as Jose mentioned, there's\Nalso /sbin which is for programs that are
Dialogue: 0,0:12:40.26,0:12:43.80,Default,,0000,0000,0000,,intended to only be run as root, that\Nalso varies from distro to distro
Dialogue: 0,0:12:43.80,0:12:47.25,Default,,0000,0000,0000,,whether you even have that directory, and\Non many systems like /usr/local/bin
Dialogue: 0,0:12:47.25,0:12:51.15,Default,,0000,0000,0000,,might not even be in your PATH, or\Nmight not even exist on your system
Dialogue: 0,0:12:51.15,0:12:55.83,Default,,0000,0000,0000,,On BSD on the other hand /usr/local/bin\Nis often used a lot more heavily
Dialogue: 0,0:12:56.73,0:12:57.23,Default,,0000,0000,0000,,yeah so
Dialogue: 0,0:12:57.23,0:13:01.11,Default,,0000,0000,0000,,What we were talking about so far, these\Nare all ways that files and folders are
Dialogue: 0,0:13:01.11,0:13:05.07,Default,,0000,0000,0000,,organized on Linux things or Linux or\NBSD things vary a little bit between
Dialogue: 0,0:13:05.07,0:13:07.15,Default,,0000,0000,0000,,that and macOS or other platforms
Dialogue: 0,0:13:07.15,0:13:09.30,Default,,0000,0000,0000,,I think for the specific locations,
Dialogue: 0,0:13:09.30,0:13:11.47,Default,,0000,0000,0000,,if you to know exactly what it's\Nused for, you can look it up
Dialogue: 0,0:13:11.47,0:13:17.29,Default,,0000,0000,0000,,But some general patterns to keep in mind or anything\Nwith /bin in it has binary executable programs in it,
Dialogue: 0,0:13:17.29,0:13:19.89,Default,,0000,0000,0000,,anything with \lib in it, has\Nlibraries in it so things that
Dialogue: 0,0:13:19.89,0:13:25.08,Default,,0000,0000,0000,,programs can link against, and then some\Nother things that are useful to know are
Dialogue: 0,0:13:25.08,0:13:29.43,Default,,0000,0000,0000,,there's a /etc on many systems, which\Nhas configuration files in it and
Dialogue: 0,0:13:29.43,0:13:34.31,Default,,0000,0000,0000,,then there's /home, which underneath that directory\Ncontains each user's home directory
Dialogue: 0,0:13:34.31,0:13:38.52,Default,,0000,0000,0000,,so like on a linux box my username\Nor if it's Anish will
Dialogue: 0,0:13:38.65,0:13:41.35,Default,,0000,0000,0000,,correspond to a home directory /home/anish
Dialogue: 0,0:13:42.07,0:13:43.35,Default,,0000,0000,0000,,Yeah I guess there are
Dialogue: 0,0:13:43.35,0:13:47.67,Default,,0000,0000,0000,,a couple of others like /tmp is usually\Na temporary directory that gets
Dialogue: 0,0:13:47.67,0:13:51.35,Default,,0000,0000,0000,,erased when you reboot not always but sometimes,\Nyou should check on your system
Dialogue: 0,0:13:51.73,0:13:59.21,Default,,0000,0000,0000,,There's a /var which often holds like\Nfiles the change over time so
Dialogue: 0,0:13:59.21,0:14:06.15,Default,,0000,0000,0000,,these these are usually going to be things\Nlike lock files for package managers
Dialogue: 0,0:14:06.15,0:14:12.43,Default,,0000,0000,0000,,they're gonna be things like log files\Nfiles to keep track of process IDs
Dialogue: 0,0:14:12.43,0:14:16.47,Default,,0000,0000,0000,,then there's /dev which shows devices so
Dialogue: 0,0:14:16.47,0:14:20.55,Default,,0000,0000,0000,,usually so these are special files that\Ncorrespond to devices on your system we
Dialogue: 0,0:14:20.55,0:14:27.39,Default,,0000,0000,0000,,talked about /sys, Anish mentioned /etc
Dialogue: 0,0:14:29.05,0:14:36.03,Default,,0000,0000,0000,,/opt is a common one for just like third-party\Nsoftware that basically it's usually for
Dialogue: 0,0:14:36.03,0:14:40.95,Default,,0000,0000,0000,,companies ported their software to Linux\Nbut they don't actually understand what
Dialogue: 0,0:14:40.95,0:14:45.39,Default,,0000,0000,0000,,running software on Linux is like, and\Nso they just have a directory with all
Dialogue: 0,0:14:45.39,0:14:51.41,Default,,0000,0000,0000,,their stuff in it and when those get installed\Nthey usually get installed into /opt
Dialogue: 0,0:14:51.41,0:14:55.65,Default,,0000,0000,0000,,I think those are the ones off the top of my head
Dialogue: 0,0:14:55.65,0:14:57.77,Default,,0000,0000,0000,,yeah
Dialogue: 0,0:14:57.77,0:15:02.27,Default,,0000,0000,0000,,And we will list these in our lecture notes\Nwhich will produce after this lecture
Dialogue: 0,0:15:02.27,0:15:04.43,Default,,0000,0000,0000,,Next question
Dialogue: 0,0:15:04.43,0:15:07.08,Default,,0000,0000,0000,,Should I apt-get install a Python whatever
Dialogue: 0,0:15:07.08,0:15:10.69,Default,,0000,0000,0000,,package or pip install that package
Dialogue: 0,0:15:10.69,0:15:13.89,Default,,0000,0000,0000,,so this is a good question that I think at
Dialogue: 0,0:15:13.89,0:15:17.31,Default,,0000,0000,0000,,a higher level this question is asking\Nshould I use my systems package manager
Dialogue: 0,0:15:17.31,0:15:20.85,Default,,0000,0000,0000,,to install things or should I use some other\Npackage manager. Like in this case
Dialogue: 0,0:15:20.85,0:15:25.02,Default,,0000,0000,0000,,one that's more specific to a particular\Nlanguage. And the answer here is also
Dialogue: 0,0:15:25.02,0:15:28.59,Default,,0000,0000,0000,,kind of it depends, sometimes it's nice\Nto manage things using a system package
Dialogue: 0,0:15:28.59,0:15:31.95,Default,,0000,0000,0000,,manager so everything can be installed\Nand upgraded in a single place but
Dialogue: 0,0:15:31.95,0:15:35.16,Default,,0000,0000,0000,,I think oftentimes whatever is available\Nin the system repositories the things
Dialogue: 0,0:15:35.16,0:15:37.80,Default,,0000,0000,0000,,you can get via a tool like\Napt-get or something similar
Dialogue: 0,0:15:37.80,0:15:41.04,Default,,0000,0000,0000,,might be slightly out of date compared to\Nthe more language specific repository
Dialogue: 0,0:15:41.04,0:15:45.06,Default,,0000,0000,0000,,so for example a lot of the Python packages\NI use I really want the most
Dialogue: 0,0:15:45.06,0:15:47.77,Default,,0000,0000,0000,,up-to-date version and so\NI use pip to install them
Dialogue: 0,0:15:48.55,0:15:51.09,Default,,0000,0000,0000,,Then, to extend on that is
Dialogue: 0,0:15:51.09,0:15:57.75,Default,,0000,0000,0000,,sometimes the case the system packages\Nmight require some other
Dialogue: 0,0:15:57.75,0:16:02.46,Default,,0000,0000,0000,,dependencies that you might not have realized\Nabout, and it's also might be
Dialogue: 0,0:16:02.46,0:16:07.20,Default,,0000,0000,0000,,the case or like for some systems,\Nat least for like alpine Linux they
Dialogue: 0,0:16:07.20,0:16:11.22,Default,,0000,0000,0000,,don't have wheels for like a lot of the\NPython packages so it will just take
Dialogue: 0,0:16:11.22,0:16:15.33,Default,,0000,0000,0000,,longer to compile them, it will take more\Nspace because they have to compile them
Dialogue: 0,0:16:15.33,0:16:20.76,Default,,0000,0000,0000,,from scratch. Whereas if you just go\Nto pip, pip has binaries for a lot of
Dialogue: 0,0:16:20.76,0:16:23.47,Default,,0000,0000,0000,,different platforms and that will probably work
Dialogue: 0,0:16:23.47,0:16:29.19,Default,,0000,0000,0000,,You also should be aware that pip might not do\Nthe exact same thing in different computers
Dialogue: 0,0:16:29.19,0:16:33.60,Default,,0000,0000,0000,,So, for example, if you are in a kind of laptop\Nor like a desktop that is running like
Dialogue: 0,0:16:33.60,0:16:38.97,Default,,0000,0000,0000,,a x86 or x86_64 you probably have binaries,\Nbut if you're running something
Dialogue: 0,0:16:38.97,0:16:43.47,Default,,0000,0000,0000,,like Raspberry Pi or some other kind of\Nembedded device. These are running on a
Dialogue: 0,0:16:43.47,0:16:47.61,Default,,0000,0000,0000,,different kind of hardware architecture\Nand you might not have binaries
Dialogue: 0,0:16:47.61,0:16:51.84,Default,,0000,0000,0000,,I think that's also good to take into account,\Nin that case in might be worthwhile to
Dialogue: 0,0:16:51.84,0:16:58.55,Default,,0000,0000,0000,,use the system packages just because they\Nwill take much shorter to get them
Dialogue: 0,0:16:58.55,0:17:01.69,Default,,0000,0000,0000,,than to just to compile from scratch\Nthe entire Python installation
Dialogue: 0,0:17:01.69,0:17:06.74,Default,,0000,0000,0000,,Apart from that, I don't think I can think of any exceptions\Nwhere I would actually use the system packages
Dialogue: 0,0:17:06.74,0:17:09.25,Default,,0000,0000,0000,,instead of the Python provided ones
Dialogue: 0,0:17:19.01,0:17:20.85,Default,,0000,0000,0000,,So, one other thing to keep in mind is that
Dialogue: 0,0:17:20.86,0:17:26.18,Default,,0000,0000,0000,,sometimes you will have more than one\Nprogram on your computer and you might
Dialogue: 0,0:17:26.18,0:17:29.96,Default,,0000,0000,0000,,be developing more than one program on\Nyour computer and for some reason not
Dialogue: 0,0:17:29.96,0:17:33.86,Default,,0000,0000,0000,,all programs are always built with the latest\Nversion of things, sometimes they
Dialogue: 0,0:17:33.86,0:17:39.35,Default,,0000,0000,0000,,are a little bit behind, and when you\Ninstall something system-wide you can
Dialogue: 0,0:17:39.35,0:17:44.69,Default,,0000,0000,0000,,only... depends on your exact system,\Nbut often you just have one version
Dialogue: 0,0:17:44.69,0:17:49.71,Default,,0000,0000,0000,,what pip lets you do, especially combined\Nwith something like python's virtualenv,
Dialogue: 0,0:17:49.71,0:17:54.53,Default,,0000,0000,0000,,and similar concepts exist for other\Nlanguages, where you can sort of say
Dialogue: 0,0:17:54.53,0:17:59.66,Default,,0000,0000,0000,,I want to (NPM does the same thing as well\Nwith its node modules, for example) where
Dialogue: 0,0:17:59.66,0:18:05.99,Default,,0000,0000,0000,,I'm gonna compile the dependencies of\Nthis package in sort of a subdirectory
Dialogue: 0,0:18:05.99,0:18:10.43,Default,,0000,0000,0000,,of its own, and all of the versions that it\Nrequires are going to be built in there
Dialogue: 0,0:18:10.43,0:18:13.91,Default,,0000,0000,0000,,and you can do this separately for separate\Nprojects so there they have
Dialogue: 0,0:18:13.91,0:18:16.91,Default,,0000,0000,0000,,different dependencies or the same dependencies\Nwith different versions
Dialogue: 0,0:18:16.91,0:18:20.93,Default,,0000,0000,0000,,they still sort of kept separate. And that\Nis one thing that's hard to achieve
Dialogue: 0,0:18:20.93,0:18:22.65,Default,,0000,0000,0000,,with system packages
Dialogue: 0,0:18:27.13,0:18:27.85,Default,,0000,0000,0000,,Next question
Dialogue: 0,0:18:27.91,0:18:32.77,Default,,0000,0000,0000,,What's the easiest and best profiling tools\Nto use to improve performance of my code?
Dialogue: 0,0:18:34.35,0:18:39.23,Default,,0000,0000,0000,,This is a topic we could talk\Nabout for a very long time
Dialogue: 0,0:18:39.23,0:18:42.88,Default,,0000,0000,0000,,The easiest and best is to print stuff using time
Dialogue: 0,0:18:42.88,0:18:48.43,Default,,0000,0000,0000,,Like, I'm not joking, very often\Nthe easiest thing is in your code
Dialogue: 0,0:18:48.97,0:18:53.75,Default,,0000,0000,0000,,At the top you figure out what the current\Ntime is, and then you do sort of
Dialogue: 0,0:18:53.75,0:18:57.92,Default,,0000,0000,0000,,a binary search over your program of add\Na print statement that prints how much
Dialogue: 0,0:18:57.92,0:19:02.51,Default,,0000,0000,0000,,time has elapsed since the start of your\Nprogram and then you do that until you
Dialogue: 0,0:19:02.51,0:19:06.32,Default,,0000,0000,0000,,find the segment of code that took the\Nlongest. And then you go into that
Dialogue: 0,0:19:06.32,0:19:09.53,Default,,0000,0000,0000,,function and then you do the same thing\Nagain and you keep doing this until you
Dialogue: 0,0:19:09.53,0:19:14.03,Default,,0000,0000,0000,,find roughly where the time was spent. It's\Nnot foolproof, but it is really easy
Dialogue: 0,0:19:14.03,0:19:16.72,Default,,0000,0000,0000,,and it gives you good information quickly
Dialogue: 0,0:19:16.72,0:19:25.36,Default,,0000,0000,0000,,if you do need more advanced information\NValgrind has a tool called cache-grind?
Dialogue: 0,0:19:25.36,0:19:29.43,Default,,0000,0000,0000,,call grind? Cache grind? One of the two.
Dialogue: 0,0:19:29.43,0:19:33.31,Default,,0000,0000,0000,,and this tool lets you run your program and
Dialogue: 0,0:19:33.31,0:19:38.74,Default,,0000,0000,0000,,measure how long everything takes and\Nall of the call stacks, like which
Dialogue: 0,0:19:38.74,0:19:42.52,Default,,0000,0000,0000,,function called which function, and what\Nyou end up with is a really neat
Dialogue: 0,0:19:42.52,0:19:47.08,Default,,0000,0000,0000,,annotation of your entire program source\Nwith the heat of every line basically
Dialogue: 0,0:19:47.08,0:19:51.76,Default,,0000,0000,0000,,how much time was spent there. It does\Nslow down your program by like an order
Dialogue: 0,0:19:51.76,0:19:56.02,Default,,0000,0000,0000,,of magnitude or more, and it doesn't really\Nsupport threads but it is really
Dialogue: 0,0:19:56.02,0:20:01.12,Default,,0000,0000,0000,,useful if you can use it. If you can't,\Nthen tools like perf or similar tools
Dialogue: 0,0:20:01.12,0:20:05.20,Default,,0000,0000,0000,,for other languages that do usually some\Nkind of sampling profiling like we
Dialogue: 0,0:20:05.20,0:20:09.81,Default,,0000,0000,0000,,talked about in the profiler lecture, can\Ngive you pretty useful data quickly,
Dialogue: 0,0:20:09.81,0:20:15.16,Default,,0000,0000,0000,,but it's a lot of data around\Nthis, but they're a little bit
Dialogue: 0,0:20:15.16,0:20:18.97,Default,,0000,0000,0000,,biased and what kind of things they usually\Nhighlight as a problem and it
Dialogue: 0,0:20:18.97,0:20:22.96,Default,,0000,0000,0000,,can sometimes be hard to extract meaningful\Ninformation about what should
Dialogue: 0,0:20:22.96,0:20:27.70,Default,,0000,0000,0000,,I change in response to them. Whereas the\Nsort of print approach very quickly
Dialogue: 0,0:20:27.70,0:20:32.17,Default,,0000,0000,0000,,gives you like this section\Nof code is bad or slow
Dialogue: 0,0:20:32.17,0:20:34.87,Default,,0000,0000,0000,,I think would be my answer
Dialogue: 0,0:20:34.87,0:20:40.43,Default,,0000,0000,0000,,Flamegraphs are great, they're a good way\Nto visualize some of this information
Dialogue: 0,0:20:41.49,0:20:45.55,Default,,0000,0000,0000,,Yeah I just have one thing to add,\Noftentimes programming languages
Dialogue: 0,0:20:45.55,0:20:48.91,Default,,0000,0000,0000,,have language specific tools for profiling\Nso to figure out what's the
Dialogue: 0,0:20:48.91,0:20:52.19,Default,,0000,0000,0000,,right tool to use for your language like if\Nyou're doing JavaScript in the web browser
Dialogue: 0,0:20:52.19,0:20:55.41,Default,,0000,0000,0000,,the web browser has a really nice tool for\Ndoing profiling you should just use that
Dialogue: 0,0:20:55.41,0:21:00.47,Default,,0000,0000,0000,,or if you are using go, for example, go has a built-in\Nprofiler is really good you should just use that
Dialogue: 0,0:21:01.71,0:21:04.25,Default,,0000,0000,0000,,A last thing to add to that
Dialogue: 0,0:21:04.25,0:21:09.95,Default,,0000,0000,0000,,Sometimes you might find that doing this binary\Nsearch over time that you're kind of
Dialogue: 0,0:21:09.96,0:21:14.35,Default,,0000,0000,0000,,finding where the time is going, but this\Ntime is sometimes happening because
Dialogue: 0,0:21:14.35,0:21:18.46,Default,,0000,0000,0000,,you're waiting on the network, or you're\Nwaiting for some file, and in that case
Dialogue: 0,0:21:18.46,0:21:23.44,Default,,0000,0000,0000,,you want to make sure that the time\Nthat is, if I want to write
Dialogue: 0,0:21:23.44,0:21:27.31,Default,,0000,0000,0000,,like 1 gigabyte file or like read 1\Ngigabyte file and put it into memory
Dialogue: 0,0:21:27.31,0:21:32.26,Default,,0000,0000,0000,,you want to check that the actual time\Nthere, is the minimum amount of time
Dialogue: 0,0:21:32.26,0:21:36.22,Default,,0000,0000,0000,,you actually have to wait. If it's ten times\Nlonger, you should try to use some
Dialogue: 0,0:21:36.22,0:21:39.37,Default,,0000,0000,0000,,other tools that we covered in the debugging\Nand profiling section to see
Dialogue: 0,0:21:39.37,0:21:45.67,Default,,0000,0000,0000,,why you're not utilizing all your\Nresources because that might...
Dialogue: 0,0:21:50.51,0:21:56.07,Default,,0000,0000,0000,,Because that might be a lot of what's happening\Nthing, like for example, in my research
Dialogue: 0,0:21:56.08,0:21:59.41,Default,,0000,0000,0000,,in machine learning workloads, a lot of\Ntime is loading data and you have to
Dialogue: 0,0:21:59.41,0:22:02.98,Default,,0000,0000,0000,,make sure well like the time it takes to\Nload data is actually the minimum amount
Dialogue: 0,0:22:02.98,0:22:07.50,Default,,0000,0000,0000,,of time you want to have that happening
Dialogue: 0,0:22:08.04,0:22:13.48,Default,,0000,0000,0000,,And to build on that, there are actually\Nspecialized tools for doing things like
Dialogue: 0,0:22:13.48,0:22:17.35,Default,,0000,0000,0000,,analyzing wait times. Very often when\Nyou're waiting for something what's
Dialogue: 0,0:22:17.35,0:22:20.59,Default,,0000,0000,0000,,really happening is you're issuing your\Nsystem call, and that system call takes
Dialogue: 0,0:22:20.59,0:22:24.19,Default,,0000,0000,0000,,some amount of time to respond. Like you do\Na really large write, or a really large read
Dialogue: 0,0:22:24.19,0:22:28.36,Default,,0000,0000,0000,,or you do many of them, and one thing\Nthat can be really handy here is
Dialogue: 0,0:22:28.36,0:22:31.84,Default,,0000,0000,0000,,to try to get information out of the\Nkernel about where your program is
Dialogue: 0,0:22:31.84,0:22:37.00,Default,,0000,0000,0000,,spending its time. And so there's (it's\Nnot new), but there's a relatively
Dialogue: 0,0:22:37.00,0:22:42.82,Default,,0000,0000,0000,,newly available thing called BPF or eBPF.\NWhich is essentially kernel tracing
Dialogue: 0,0:22:42.82,0:22:48.53,Default,,0000,0000,0000,,and you can do some really cool things with\Nit, and that includes tracing user programs.
Dialogue: 0,0:22:48.53,0:22:51.76,Default,,0000,0000,0000,,It can be a little bit awkward to\Nget started with, there's a tool
Dialogue: 0,0:22:51.76,0:22:56.20,Default,,0000,0000,0000,,called BPF trace that i would recommend\Nyou looking to, if you need to do like
Dialogue: 0,0:22:56.20,0:23:00.04,Default,,0000,0000,0000,,this kind of low-level performance debugging.\NBut it is really good for this
Dialogue: 0,0:23:00.04,0:23:04.60,Default,,0000,0000,0000,,kind of stuff. You can get things like\Nhistograms over how much time was spent
Dialogue: 0,0:23:04.60,0:23:06.67,Default,,0000,0000,0000,,in particular system calls
Dialogue: 0,0:23:06.67,0:23:09.72,Default,,0000,0000,0000,,It's a great tool
Dialogue: 0,0:23:12.25,0:23:15.35,Default,,0000,0000,0000,,What browser plugins do you use?
Dialogue: 0,0:23:16.73,0:23:19.73,Default,,0000,0000,0000,,I try to use as few as I can get away with using
Dialogue: 0,0:23:19.73,0:23:25.99,Default,,0000,0000,0000,,because I don't like things being in\Nmy browser, but there are a couple of
Dialogue: 0,0:23:25.99,0:23:30.31,Default,,0000,0000,0000,,ones that are sort of staples.\NThe first one is uBlock Origin.
Dialogue: 0,0:23:30.31,0:23:36.61,Default,,0000,0000,0000,,So uBlock Origin is one of many ad blockers but\Nit's a little bit more than an ad blocker.
Dialogue: 0,0:23:36.61,0:23:42.53,Default,,0000,0000,0000,,It is (a what do they call it?) a\Nnetwork filtering tool so it lets
Dialogue: 0,0:23:42.53,0:23:47.33,Default,,0000,0000,0000,,you do more things than just block ads.\NIt also lets you like block connections
Dialogue: 0,0:23:47.33,0:23:51.35,Default,,0000,0000,0000,,to certain domains, block connections\Nfor certain types of resources
Dialogue: 0,0:23:51.35,0:23:56.03,Default,,0000,0000,0000,,So I have mine set up in what they call\Nthe Advanced Mode, where basically
Dialogue: 0,0:23:56.03,0:24:02.45,Default,,0000,0000,0000,,you can disable basically all network requests.\NBut it's not just Network requests,
Dialogue: 0,0:24:02.45,0:24:07.43,Default,,0000,0000,0000,,It's also like I have disabled all inline\Nscripts on every page and all
Dialogue: 0,0:24:07.43,0:24:11.54,Default,,0000,0000,0000,,third-party images and resources, and then\Nyou can sort of create a whitelist
Dialogue: 0,0:24:11.54,0:24:16.35,Default,,0000,0000,0000,,for every page so it gives you really\Nlow-level tools around how to
Dialogue: 0,0:24:16.35,0:24:20.33,Default,,0000,0000,0000,,how to improve the security of your browsing.\NBut you can also set it in not the
Dialogue: 0,0:24:20.33,0:24:23.99,Default,,0000,0000,0000,,advanced mode, and then it does much of\Nthe same as a regular ad blocker would
Dialogue: 0,0:24:23.99,0:24:28.10,Default,,0000,0000,0000,,do, although in a fairly efficient way\Nif you're looking at an ad blocker it's
Dialogue: 0,0:24:28.10,0:24:31.51,Default,,0000,0000,0000,,probably the one to use and it\Nworks on like every browser
Dialogue: 0,0:24:31.51,0:24:34.45,Default,,0000,0000,0000,,That would be my top pick I think,
Dialogue: 0,0:24:39.11,0:24:44.39,Default,,0000,0000,0000,,I think probably the one I\Nuse like the most actively
Dialogue: 0,0:24:44.39,0:24:50.39,Default,,0000,0000,0000,,is one called Stylus. It lets you modify\Nthe CSS or like the stylesheets
Dialogue: 0,0:24:50.39,0:24:54.56,Default,,0000,0000,0000,,that webpages have. And it's pretty\Nneat, because sometimes you're
Dialogue: 0,0:24:54.56,0:24:58.55,Default,,0000,0000,0000,,looking at a website and you want\Nto hide some part of the website
Dialogue: 0,0:24:58.55,0:25:04.21,Default,,0000,0000,0000,,you don't care about. Like maybe a ad, maybe\Nsome sidebar you're not finding useful
Dialogue: 0,0:25:04.21,0:25:06.29,Default,,0000,0000,0000,,The thing is, at the end of\Nthe day these things are
Dialogue: 0,0:25:06.29,0:25:09.59,Default,,0000,0000,0000,,displaying in your browser, and you\Nhave control of what code is
Dialogue: 0,0:25:09.59,0:25:13.13,Default,,0000,0000,0000,,executing and similar to what Jon was\Nsaying, like you can customize this
Dialogue: 0,0:25:13.13,0:25:18.49,Default,,0000,0000,0000,,to no end, and what I have for a lot of\Nweb pages like hide this this part, or
Dialogue: 0,0:25:18.49,0:25:23.39,Default,,0000,0000,0000,,also trying to make like dark modes for\Nthem like you can change pretty much the
Dialogue: 0,0:25:23.39,0:25:26.81,Default,,0000,0000,0000,,color for every single website. And what\Nis actually pretty neat is that there's
Dialogue: 0,0:25:26.81,0:25:31.46,Default,,0000,0000,0000,,like a repository online of people that\Nhave contributed this is stylesheets
Dialogue: 0,0:25:31.46,0:25:35.03,Default,,0000,0000,0000,,for the websites. So someone probably\Nhas (done) one for GitHub
Dialogue: 0,0:25:35.03,0:25:38.78,Default,,0000,0000,0000,,Like I want dark GitHub and someone has\Nalready contributed one that makes
Dialogue: 0,0:25:38.78,0:25:44.63,Default,,0000,0000,0000,,that much more pleasing to browse. Apart\Nfrom that, one that it's not really
Dialogue: 0,0:25:44.63,0:25:49.49,Default,,0000,0000,0000,,fancy, but I have found incredibly helpful\Nis one that just takes a screenshot an
Dialogue: 0,0:25:49.49,0:25:53.12,Default,,0000,0000,0000,,entire website. And It will\Nscroll for you and make
Dialogue: 0,0:25:53.12,0:25:57.71,Default,,0000,0000,0000,,compound image of the entire website and that's\Nreally great for when you're trying to
Dialogue: 0,0:25:57.71,0:26:00.11,Default,,0000,0000,0000,,print a website and is just terrible.
Dialogue: 0,0:26:00.11,0:26:00.61,Default,,0000,0000,0000,,(It's built into Firefox)
Dialogue: 0,0:26:00.61,0:26:02.67,Default,,0000,0000,0000,,oh interesting
Dialogue: 0,0:26:02.67,0:26:05.75,Default,,0000,0000,0000,,oh now that you mention builtin to Firefox,\Nanother one that I really like about
Dialogue: 0,0:26:05.75,0:26:09.07,Default,,0000,0000,0000,,Firefox is the multi account containers
Dialogue: 0,0:26:09.07,0:26:10.83,Default,,0000,0000,0000,,(Oh yeah, it's fantastic)
Dialogue: 0,0:26:10.83,0:26:12.29,Default,,0000,0000,0000,,Which kind of lets you
Dialogue: 0,0:26:12.29,0:26:16.67,Default,,0000,0000,0000,,By default a lot of web browsers, like\Nfor example Chrome, have this
Dialogue: 0,0:26:16.67,0:26:20.60,Default,,0000,0000,0000,,notion of like there's session that you\Nhave, where you have all your cookies
Dialogue: 0,0:26:20.60,0:26:24.56,Default,,0000,0000,0000,,and they are kind of all shared from the\Ndifferent websites in the sense of
Dialogue: 0,0:26:24.56,0:26:30.81,Default,,0000,0000,0000,,you keep opening new tabs and unless you go into\Nincognito you kind of have the same profile
Dialogue: 0,0:26:30.81,0:26:34.19,Default,,0000,0000,0000,,And that profile is the same for\Nall websites, there is this
Dialogue: 0,0:26:34.19,0:26:35.85,Default,,0000,0000,0000,,Is it an extension or is it built in?
Dialogue: 0,0:26:35.85,0:26:40.57,Default,,0000,0000,0000,,(it's a mix, it's complicated)
Dialogue: 0,0:26:41.09,0:26:46.21,Default,,0000,0000,0000,,So I think you actually have to say you want\Nto install it or enable it and again
Dialogue: 0,0:26:46.22,0:26:49.88,Default,,0000,0000,0000,,the name is Multi Account Containers and\Nthese let you tell Firefox to have
Dialogue: 0,0:26:49.88,0:26:53.96,Default,,0000,0000,0000,,separate isolated sessions. So\Nfor example, you want to say
Dialogue: 0,0:26:53.96,0:26:58.85,Default,,0000,0000,0000,,I have a separate sessions for whenever I\Nvisit to Google or whenever I visit Amazon
Dialogue: 0,0:26:58.85,0:27:01.79,Default,,0000,0000,0000,,and that can be pretty neat, because then you can
Dialogue: 0,0:27:01.79,0:27:08.17,Default,,0000,0000,0000,,At a browser level it's ensuring that no information\Nsharing is happening between the two of them
Dialogue: 0,0:27:08.17,0:27:11.96,Default,,0000,0000,0000,,And it's much more convenient than\Nhaving to open a incognito window
Dialogue: 0,0:27:11.96,0:27:14.47,Default,,0000,0000,0000,,where it's gonna clean all the time the stuff
Dialogue: 0,0:27:14.47,0:27:17.31,Default,,0000,0000,0000,,(One thing to mention is Stylus vs Stylish)
Dialogue: 0,0:27:17.53,0:27:19.65,Default,,0000,0000,0000,,Oh yeah, I forgot about that
Dialogue: 0,0:27:19.65,0:27:24.93,Default,,0000,0000,0000,,One important thing is the browser extension\Nfor side loading CSS Stylesheets
Dialogue: 0,0:27:24.93,0:27:31.85,Default,,0000,0000,0000,,it's called a Stylus and that's different\Nfrom the older one that was
Dialogue: 0,0:27:31.85,0:27:37.40,Default,,0000,0000,0000,,called Stylish, because that one got\Nbought at some point by some shady
Dialogue: 0,0:27:37.40,0:27:40.71,Default,,0000,0000,0000,,company, that started abusing it not only to have
Dialogue: 0,0:27:40.71,0:27:45.78,Default,,0000,0000,0000,,that functionality, but also to read your\Nentire browser history and send that
Dialogue: 0,0:27:45.78,0:27:48.49,Default,,0000,0000,0000,,back to their servers so they could data mine it.
Dialogue: 0,0:27:48.49,0:27:53.73,Default,,0000,0000,0000,,So, then people just built this open-source alternative\Nthat is called Stylus, and that's the one
Dialogue: 0,0:27:53.73,0:27:58.95,Default,,0000,0000,0000,,we recommend. Said that, I think the repository\Nfor styles is the same for the
Dialogue: 0,0:27:58.95,0:28:03.61,Default,,0000,0000,0000,,two of them, but I would have\Nto double check that.
Dialogue: 0,0:28:03.61,0:28:05.95,Default,,0000,0000,0000,,Do you have any browser plugins Anish?
Dialogue: 0,0:28:06.07,0:28:09.31,Default,,0000,0000,0000,,Yes, so I also have some recommendations\Nfor browser plugins
Dialogue: 0,0:28:09.31,0:28:13.99,Default,,0000,0000,0000,,I also use uBlock Origin and I also use Stylus,
Dialogue: 0,0:28:13.99,0:28:18.51,Default,,0000,0000,0000,,but one other one that I'd recommend is\Nintegration with a password manager
Dialogue: 0,0:28:18.51,0:28:21.63,Default,,0000,0000,0000,,So this is a topic that we have in\Nthe lecture notes for the security
Dialogue: 0,0:28:21.63,0:28:24.84,Default,,0000,0000,0000,,lecture, but we didn't really get to talk\Nabout in detail. But basically password
Dialogue: 0,0:28:24.84,0:28:27.81,Default,,0000,0000,0000,,managers do a really good job of increasing\Nyour security when working
Dialogue: 0,0:28:27.81,0:28:31.83,Default,,0000,0000,0000,,with online accounts, and having browser\Nintegration with your password manager
Dialogue: 0,0:28:31.83,0:28:34.41,Default,,0000,0000,0000,,can save you a lot of time like you\Ncan open up a website then it can
Dialogue: 0,0:28:34.41,0:28:37.38,Default,,0000,0000,0000,,autofill your login information for you\Nsir and you go and copy and paste it
Dialogue: 0,0:28:37.38,0:28:40.32,Default,,0000,0000,0000,,back and forth between a separate program\Nif it's not integrated with your
Dialogue: 0,0:28:40.32,0:28:43.41,Default,,0000,0000,0000,,web browser, and it can also, this integration,\Ncan save you from certain
Dialogue: 0,0:28:43.41,0:28:47.65,Default,,0000,0000,0000,,attacks that would otherwise be possible if\Nyou were doing this manual copy pasting.
Dialogue: 0,0:28:47.65,0:28:50.79,Default,,0000,0000,0000,,For example, phishing attacks. So\Nyou find a website that looks very
Dialogue: 0,0:28:50.79,0:28:54.21,Default,,0000,0000,0000,,similar to Facebook and you go to log in\Nwith your facebook login credentials and
Dialogue: 0,0:28:54.21,0:28:56.85,Default,,0000,0000,0000,,you go to your password manager and copy\Npaste the correct credentials into this
Dialogue: 0,0:28:56.85,0:29:00.06,Default,,0000,0000,0000,,funny web site and now all of a sudden\Nit has your password but if you have
Dialogue: 0,0:29:00.06,0:29:03.09,Default,,0000,0000,0000,,browser integration then the extension\Ncan automatically check
Dialogue: 0,0:29:03.09,0:29:06.95,Default,,0000,0000,0000,,like. Am I on F A C E B O O K.com,or\Nis it some other domain
Dialogue: 0,0:29:06.95,0:29:10.67,Default,,0000,0000,0000,,that maybe look similar and it will not enter\Nthe login information if it's the wrong domain
Dialogue: 0,0:29:10.67,0:29:15.79,Default,,0000,0000,0000,,so browser extension for\Npassword managing is good
Dialogue: 0,0:29:15.79,0:29:17.93,Default,,0000,0000,0000,,Yeah I agree
Dialogue: 0,0:29:19.49,0:29:20.71,Default,,0000,0000,0000,,Next question
Dialogue: 0,0:29:20.71,0:29:23.99,Default,,0000,0000,0000,,What are other useful data wrangling tools?
Dialogue: 0,0:29:23.99,0:29:32.42,Default,,0000,0000,0000,,So in yesterday's lecture, I mentioned curl, so\Ncurl is a fantastic tool for just making web
Dialogue: 0,0:29:32.42,0:29:35.81,Default,,0000,0000,0000,,requests and dumping them to your terminal.\NYou can also use it for things
Dialogue: 0,0:29:35.81,0:29:41.19,Default,,0000,0000,0000,,like uploading files which is really handy.
Dialogue: 0,0:29:41.19,0:29:48.43,Default,,0000,0000,0000,,In the exercises of that lecture we also talked about\NJQ and pup which are command line tools that let you
Dialogue: 0,0:29:48.43,0:29:52.99,Default,,0000,0000,0000,,basically write queries over JSON\Nand HTML documents respectively
Dialogue: 0,0:29:52.99,0:30:00.39,Default,,0000,0000,0000,,that can be really handy. Other\Ndata wrangling tools?
Dialogue: 0,0:30:00.39,0:30:03.82,Default,,0000,0000,0000,,Ah Perl, the Perl programming language is
Dialogue: 0,0:30:03.82,0:30:08.06,Default,,0000,0000,0000,,often referred to as a write only\Nprogramming language because it's
Dialogue: 0,0:30:08.06,0:30:13.43,Default,,0000,0000,0000,,impossible to read even if you wrote it.\NBut it is fantastic at doing just like
Dialogue: 0,0:30:13.43,0:30:21.56,Default,,0000,0000,0000,,straight up text processing, like nothing\Nbeats it there, so maybe worth learning
Dialogue: 0,0:30:21.56,0:30:24.33,Default,,0000,0000,0000,,some very rudimentary Perl just\Nto write some of those scripts
Dialogue: 0,0:30:24.33,0:30:29.37,Default,,0000,0000,0000,,It's easier often than writing some like hacked-up\Ncombination of grep and awk and sed,
Dialogue: 0,0:30:29.37,0:30:36.31,Default,,0000,0000,0000,,and it will be much faster to just tack something\Nup than writing it up in Python, for example
Dialogue: 0,0:30:36.31,0:30:44.03,Default,,0000,0000,0000,,but apart from that, other data wrangling
Dialogue: 0,0:30:44.03,0:30:47.07,Default,,0000,0000,0000,,No, not off the top of my head really
Dialogue: 0,0:30:47.07,0:30:53.66,Default,,0000,0000,0000,,column -t, if you pipe any white space separated
Dialogue: 0,0:30:53.66,0:30:58.82,Default,,0000,0000,0000,,input into column -t it will align all\Nthe white space of the columns so that
Dialogue: 0,0:30:58.82,0:31:05.77,Default,,0000,0000,0000,,you get nicely aligned columns that's, and\Nhead and tail but we talked about those
Dialogue: 0,0:31:09.01,0:31:13.79,Default,,0000,0000,0000,,I think a couple of additions to that,\Nthat I find myself using commonly
Dialogue: 0,0:31:13.79,0:31:19.88,Default,,0000,0000,0000,,one is vim. Vim can be pretty useful\Nfor like data wrangling on itself
Dialogue: 0,0:31:19.88,0:31:22.46,Default,,0000,0000,0000,,Sometimes you might find that the operation\Nthat you're trying to do is
Dialogue: 0,0:31:22.46,0:31:27.71,Default,,0000,0000,0000,,hard to put down in terms of piping\Ndifferent operators but if you
Dialogue: 0,0:31:27.71,0:31:32.53,Default,,0000,0000,0000,,can just open the file and just record
Dialogue: 0,0:31:32.53,0:31:37.30,Default,,0000,0000,0000,,a couple of quick vim macros to do what you\Nwant it to do, it might be like much,
Dialogue: 0,0:31:37.30,0:31:42.31,Default,,0000,0000,0000,,much easier. That's one, and then the other\None, if you're dealing with tabular
Dialogue: 0,0:31:42.31,0:31:46.09,Default,,0000,0000,0000,,data and you want to do more complex operations\Nlike sorting by one column,
Dialogue: 0,0:31:46.09,0:31:51.16,Default,,0000,0000,0000,,then grouping and then computing some sort\Nof statistic, I think a lot of that
Dialogue: 0,0:31:51.16,0:31:55.95,Default,,0000,0000,0000,,workload I ended up just using Python\Nand pandas because it's built for that
Dialogue: 0,0:31:55.95,0:32:00.19,Default,,0000,0000,0000,,And one of the pretty neat features that\NI find myself also using is that it
Dialogue: 0,0:32:00.19,0:32:03.93,Default,,0000,0000,0000,,will export to many different formats.\NSo this intermediate state
Dialogue: 0,0:32:03.93,0:32:09.22,Default,,0000,0000,0000,,has its own kind of pandas dataframe\Nobject but it can
Dialogue: 0,0:32:09.22,0:32:14.17,Default,,0000,0000,0000,,export to HTM, LaTeX, a lot of different\Nlike table formats so if your end
Dialogue: 0,0:32:14.17,0:32:19.53,Default,,0000,0000,0000,,product is some sort of summary table, then pandas\NI think it's a fantastic choice for that
Dialogue: 0,0:32:21.11,0:32:24.79,Default,,0000,0000,0000,,I would second the vim and also\NPython I think those are
Dialogue: 0,0:32:24.79,0:32:29.05,Default,,0000,0000,0000,,two of my most used data wrangling tools.\NFor the vim one, last year we had a demo
Dialogue: 0,0:32:29.05,0:32:31.84,Default,,0000,0000,0000,,in the series in the lecture notes, but\Nwe didn't cover it in class we had a
Dialogue: 0,0:32:31.84,0:32:38.05,Default,,0000,0000,0000,,demo of turning an XML file into a JSON version\Nof that same data using only vim macros
Dialogue: 0,0:32:38.05,0:32:40.33,Default,,0000,0000,0000,,And I think that's actually the\Nway I would do it in practice
Dialogue: 0,0:32:40.33,0:32:43.24,Default,,0000,0000,0000,,I don't want to go find a tool that does\Nthis conversion it is actually simple
Dialogue: 0,0:32:43.24,0:32:45.43,Default,,0000,0000,0000,,to encode as a vim macro,\Nthen I just do it that way
Dialogue: 0,0:32:45.43,0:32:48.99,Default,,0000,0000,0000,,And then also Python especially in an interactive\Ntool like a Jupyter notebook
Dialogue: 0,0:32:48.99,0:32:51.17,Default,,0000,0000,0000,,is a really great way of doing data wrangling
Dialogue: 0,0:32:51.17,0:32:52.95,Default,,0000,0000,0000,,A third tool I'd mention which\NI don't remember if we
Dialogue: 0,0:32:52.96,0:32:55.36,Default,,0000,0000,0000,,covered in the data wrangling\Nlecture or elsewhere
Dialogue: 0,0:32:55.36,0:32:58.75,Default,,0000,0000,0000,,is a tool called pandoc which can do transformations\Nbetween different text
Dialogue: 0,0:32:58.75,0:33:02.98,Default,,0000,0000,0000,,document formats so you can convert from\Nplaintext to HTML or HTML to markdown
Dialogue: 0,0:33:02.98,0:33:07.36,Default,,0000,0000,0000,,or LaTeX to HTML or many other formats\Nit actually it supports a large
Dialogue: 0,0:33:07.36,0:33:10.47,Default,,0000,0000,0000,,list of input formats and a\Nlarge list of output formats
Dialogue: 0,0:33:10.47,0:33:16.36,Default,,0000,0000,0000,,I think there's one last one which I mentioned briefly\Nin the lecture on data wrangling which is
Dialogue: 0,0:33:16.36,0:33:20.44,Default,,0000,0000,0000,,the R programming language, it's\Nan awful (I think it's an awful)
Dialogue: 0,0:33:20.44,0:33:25.12,Default,,0000,0000,0000,,language to program in. And i would never\Nuse it in the middle of a data wrangling
Dialogue: 0,0:33:25.12,0:33:30.95,Default,,0000,0000,0000,,pipeline, but at the end, in order to like produce\Npretty plots and statistics R is great
Dialogue: 0,0:33:30.95,0:33:35.58,Default,,0000,0000,0000,,Because R is built for doing\Nstatistics and plotting
Dialogue: 0,0:33:35.58,0:33:40.59,Default,,0000,0000,0000,,there's a library for are called\Nggplot which is just amazing
Dialogue: 0,0:33:40.59,0:33:46.55,Default,,0000,0000,0000,,ggplot2 i guess technically It's\Ngreat, it produces very
Dialogue: 0,0:33:46.55,0:33:51.43,Default,,0000,0000,0000,,nice visualizations and it lets you do,\Nit does very easily do things like
Dialogue: 0,0:33:51.43,0:33:57.56,Default,,0000,0000,0000,,If you have a data set that has like multiple\Nfacets like it's not just X and Y
Dialogue: 0,0:33:57.56,0:34:03.11,Default,,0000,0000,0000,,it's like X Y Z and some other variable,\Nand then you want to plot like the
Dialogue: 0,0:34:03.11,0:34:07.58,Default,,0000,0000,0000,,throughput grouped by all of those parameters\Nat the same time and produce
Dialogue: 0,0:34:07.58,0:34:11.99,Default,,0000,0000,0000,,a visualization. R very easily let's you\Ndo this and I haven't seen anywhere
Dialogue: 0,0:34:11.99,0:34:14.89,Default,,0000,0000,0000,,that lets you do that as easily
Dialogue: 0,0:34:16.97,0:34:17.95,Default,,0000,0000,0000,,Next question,
Dialogue: 0,0:34:17.95,0:34:20.51,Default,,0000,0000,0000,,What's the difference between\NDocker and a virtual machine
Dialogue: 0,0:34:23.27,0:34:27.73,Default,,0000,0000,0000,,What's the easiest way to explain this? So docker
Dialogue: 0,0:34:27.74,0:34:31.22,Default,,0000,0000,0000,,starts something called containers and\Ndocker is not the only program that
Dialogue: 0,0:34:31.22,0:34:36.56,Default,,0000,0000,0000,,starts containers. There are many others\Nand usually they rely on some feature of
Dialogue: 0,0:34:36.56,0:34:40.40,Default,,0000,0000,0000,,the underlying kernel in the case of\Ndocker they use something called LXC
Dialogue: 0,0:34:40.40,0:34:47.57,Default,,0000,0000,0000,,which are Linux containers and the basic\Npremise there is if you want to start
Dialogue: 0,0:34:47.57,0:34:53.18,Default,,0000,0000,0000,,what looks like a virtual machine that\Nis running roughly the same operating
Dialogue: 0,0:34:53.18,0:34:57.41,Default,,0000,0000,0000,,system as you are already running on your\Ncomputer then you don't really need
Dialogue: 0,0:34:57.41,0:35:04.70,Default,,0000,0000,0000,,to run another instance of the kernel\Nreally that other virtual machine can
Dialogue: 0,0:35:04.70,0:35:09.95,Default,,0000,0000,0000,,share a kernel. And you can just use the\Nkernels built in isolation mechanisms to
Dialogue: 0,0:35:09.95,0:35:13.79,Default,,0000,0000,0000,,spin up a program that thinks it's\Nrunning on its own hardware but in
Dialogue: 0,0:35:13.79,0:35:18.50,Default,,0000,0000,0000,,reality it's sharing the kernel and so this\Nmeans that containers can often run
Dialogue: 0,0:35:18.50,0:35:22.61,Default,,0000,0000,0000,,with much lower overhead than a full virtual\Nmachine will do but you should
Dialogue: 0,0:35:22.61,0:35:26.39,Default,,0000,0000,0000,,keep in mind that it also has somewhat weaker\Nisolation because you are sharing
Dialogue: 0,0:35:26.39,0:35:30.83,Default,,0000,0000,0000,,a kernel between the two if you spin up\Na virtual machine the only thing that's
Dialogue: 0,0:35:30.83,0:35:35.93,Default,,0000,0000,0000,,shared is sort of the hardware and to\Nsome extent the hypervisor, whereas
Dialogue: 0,0:35:35.93,0:35:40.79,Default,,0000,0000,0000,,with a docker container you're sharing\Nthe full kernel and the that is a
Dialogue: 0,0:35:40.79,0:35:44.92,Default,,0000,0000,0000,,different threat model that you\Nmight have to keep in mind
Dialogue: 0,0:35:47.34,0:35:52.36,Default,,0000,0000,0000,,One another small note there as Jon pointed\Nout, to use containers something
Dialogue: 0,0:35:52.36,0:35:55.63,Default,,0000,0000,0000,,like Docker you need the underlying operating\Nsystem to be roughly the same
Dialogue: 0,0:35:55.63,0:36:00.07,Default,,0000,0000,0000,,as whatever the program that's running\Non top of the container expects and so
Dialogue: 0,0:36:00.07,0:36:03.79,Default,,0000,0000,0000,,if you're using macOS for example, the\Nway you use docker is you run Linux
Dialogue: 0,0:36:03.79,0:36:08.26,Default,,0000,0000,0000,,inside a virtual machine and then you can\Nrun Docker on top of Linux so maybe
Dialogue: 0,0:36:08.26,0:36:11.74,Default,,0000,0000,0000,,if you're going for containers in order\Nto get better performance your trading
Dialogue: 0,0:36:11.74,0:36:15.13,Default,,0000,0000,0000,,isolation for performance if you're running\Non Mac OS that may not work out
Dialogue: 0,0:36:15.13,0:36:17.45,Default,,0000,0000,0000,,exactly as expected
Dialogue: 0,0:36:17.45,0:36:21.22,Default,,0000,0000,0000,,And one last note is that there\Nis a slight difference, so
Dialogue: 0,0:36:21.22,0:36:25.72,Default,,0000,0000,0000,,with Docker and containers,\None of the gotchas you have
Dialogue: 0,0:36:25.72,0:36:29.41,Default,,0000,0000,0000,,to be familiar with is that containers\Nare more similar to virtual
Dialogue: 0,0:36:29.41,0:36:33.07,Default,,0000,0000,0000,,machines in the sense of that they will\Npersist all the storage that you
Dialogue: 0,0:36:33.07,0:36:35.97,Default,,0000,0000,0000,,have where Docker by default won't have that.
Dialogue: 0,0:36:35.97,0:36:37.79,Default,,0000,0000,0000,,Like Docker is supposed to be running
Dialogue: 0,0:36:37.79,0:36:41.77,Default,,0000,0000,0000,,So the main idea is like I want\Nto run some software and
Dialogue: 0,0:36:41.77,0:36:45.67,Default,,0000,0000,0000,,I get the image and it runs and if you\Nwant to have any kind of persistent
Dialogue: 0,0:36:45.67,0:36:50.08,Default,,0000,0000,0000,,storage that links to the host system\Nyou have to kind of manually specify
Dialogue: 0,0:36:50.08,0:36:56.05,Default,,0000,0000,0000,,that, whereas a virtual machine is using\Nsome virtual disk that is being provided
Dialogue: 0,0:36:56.05,0:37:02.67,Default,,0000,0000,0000,,Next question
Dialogue: 0,0:37:02.67,0:37:05.11,Default,,0000,0000,0000,,What are the advantages of each operating system
Dialogue: 0,0:37:05.11,0:37:08.53,Default,,0000,0000,0000,,and how can we choose between them?\NFor example, choosing the best Linux
Dialogue: 0,0:37:08.53,0:37:10.55,Default,,0000,0000,0000,,distribution for our purposes
Dialogue: 0,0:37:14.25,0:37:16.81,Default,,0000,0000,0000,,I will say that for many, many tasks the
Dialogue: 0,0:37:16.81,0:37:20.17,Default,,0000,0000,0000,,specific Linux distribution that you're\Nrunning is not that important
Dialogue: 0,0:37:20.17,0:37:23.73,Default,,0000,0000,0000,,the thing is, it's just what kind of
Dialogue: 0,0:37:23.73,0:37:27.65,Default,,0000,0000,0000,,knowing that there are different types\Nor like groups of distributions,
Dialogue: 0,0:37:27.65,0:37:32.25,Default,,0000,0000,0000,,So for example, there are some distributions\Nthat have really frequent updates
Dialogue: 0,0:37:32.25,0:37:38.97,Default,,0000,0000,0000,,but they kind of break more easily. So for\Nexample Arch Linux has a rolling update
Dialogue: 0,0:37:38.97,0:37:43.51,Default,,0000,0000,0000,,way of pushing updates, where things might\Nbreak but they're fine with the things
Dialogue: 0,0:37:43.51,0:37:47.89,Default,,0000,0000,0000,,being that way. Where maybe where you\Nhave some really important web server
Dialogue: 0,0:37:47.89,0:37:51.40,Default,,0000,0000,0000,,that is hosting all your business\Nanalytics you want that thing
Dialogue: 0,0:37:51.40,0:37:55.96,Default,,0000,0000,0000,,to have like a much more steady way of\Nupdates. So that's for example why you
Dialogue: 0,0:37:55.96,0:37:58.12,Default,,0000,0000,0000,,will see distributions like Debian being
Dialogue: 0,0:37:58.12,0:38:02.95,Default,,0000,0000,0000,,much more conservative about what they push, or\Neven for example Ubuntu makes a difference
Dialogue: 0,0:38:02.95,0:38:07.00,Default,,0000,0000,0000,,between the Long Term Releases\Nthat they are only update every
Dialogue: 0,0:38:07.00,0:38:12.28,Default,,0000,0000,0000,,two years and the more periodic\Nreleases of one there is a
Dialogue: 0,0:38:12.28,0:38:16.66,Default,,0000,0000,0000,,it's like two a year that they make.\NSo, kind of knowing that there's the
Dialogue: 0,0:38:16.66,0:38:21.34,Default,,0000,0000,0000,,difference apart from that some distributions\Nhave different ways
Dialogue: 0,0:38:21.34,0:38:27.19,Default,,0000,0000,0000,,of providing the binaries\Nto you and the way they
Dialogue: 0,0:38:27.19,0:38:33.79,Default,,0000,0000,0000,,have the repositories so I think a lot of Red\NHat Linux don't want non free drivers in
Dialogue: 0,0:38:33.79,0:38:37.36,Default,,0000,0000,0000,,their official repositories where I\Nthink Ubuntu is fine with some of
Dialogue: 0,0:38:37.36,0:38:42.49,Default,,0000,0000,0000,,them, apart from that I think like just\Na lot of what is core to most Linux
Dialogue: 0,0:38:42.49,0:38:47.41,Default,,0000,0000,0000,,distros is kind of shared between them\Nand there's a lot of learning in the
Dialogue: 0,0:38:47.41,0:38:51.43,Default,,0000,0000,0000,,common ground. So you don't have\Nto worry about the specifics
Dialogue: 0,0:38:52.39,0:38:56.35,Default,,0000,0000,0000,,Keeping with the theme of this class being somewhat\Nopinionated, I'm gonna go ahead and say
Dialogue: 0,0:38:56.35,0:39:00.04,Default,,0000,0000,0000,,that if you're using Linux especially for\Nthe first time choose something like
Dialogue: 0,0:39:00.04,0:39:03.85,Default,,0000,0000,0000,,Ubuntu or Debian. So you Ubuntu to is a\NDebian based distribution but maybe is a
Dialogue: 0,0:39:03.85,0:39:07.42,Default,,0000,0000,0000,,little bit more friendly, Debian is a little\Nbit more minimalist. I use Debian
Dialogue: 0,0:39:07.42,0:39:10.45,Default,,0000,0000,0000,,and all my servers, for example. And I use\NDebian desktop on my desktop computers
Dialogue: 0,0:39:10.45,0:39:15.43,Default,,0000,0000,0000,,that run Linux if you're going for maybe\Ntrying to learn more things and you want
Dialogue: 0,0:39:15.43,0:39:19.39,Default,,0000,0000,0000,,a distribution that trades stability for\Nhaving more up-to-date software maybe
Dialogue: 0,0:39:19.39,0:39:21.91,Default,,0000,0000,0000,,at the expense of you having to fix a\Nbroken distribution every once in a
Dialogue: 0,0:39:21.91,0:39:26.91,Default,,0000,0000,0000,,while then maybe you can consider something\Nlike Arch Linux or Gentoo
Dialogue: 0,0:39:26.91,0:39:32.68,Default,,0000,0000,0000,,or Slackware. Oh man, I'd say that like\Nif you're installing Linux and just like
Dialogue: 0,0:39:32.68,0:39:34.89,Default,,0000,0000,0000,,want to get work done Debian is a great choice
Dialogue: 0,0:39:35.91,0:39:38.27,Default,,0000,0000,0000,,Yeah I think I agree with that.
Dialogue: 0,0:39:38.27,0:39:40.97,Default,,0000,0000,0000,,The other observation is like\Nyou couldn't install BSD
Dialogue: 0,0:39:40.97,0:39:46.69,Default,,0000,0000,0000,,BSD has gotten, has come a long way from\Nwhere it was. There's still a bunch of
Dialogue: 0,0:39:46.69,0:39:50.92,Default,,0000,0000,0000,,software you can't really get for BSD but\Nit gives you a very well-documented
Dialogue: 0,0:39:50.92,0:39:55.84,Default,,0000,0000,0000,,experience and and one thing that's different\Nabout BSD compared to Linux is
Dialogue: 0,0:39:55.84,0:40:02.53,Default,,0000,0000,0000,,that in an BSD when you install BSD you\Nget a full operating system, mostly
Dialogue: 0,0:40:02.65,0:40:07.53,Default,,0000,0000,0000,,So many of the programs are maintained by\Nthe same team that maintains the kernel
Dialogue: 0,0:40:07.54,0:40:11.35,Default,,0000,0000,0000,,and everything is sort of upgraded together,\Nwhich is a little different
Dialogue: 0,0:40:11.35,0:40:13.27,Default,,0000,0000,0000,,than how thanks work in the Linux world it does
Dialogue: 0,0:40:13.27,0:40:16.75,Default,,0000,0000,0000,,mean that things often move a little bit\Nslower. I would not use it for things
Dialogue: 0,0:40:16.75,0:40:21.79,Default,,0000,0000,0000,,like gaming either, because drivers support\Nis meh. But it is an interesting
Dialogue: 0,0:40:21.79,0:40:30.66,Default,,0000,0000,0000,,environment to look at. And then for things\Nlike Mac OS and Windows I think
Dialogue: 0,0:40:30.66,0:40:36.04,Default,,0000,0000,0000,,If you are a programmer, I don't know why\Nyou are using Windows unless you are
Dialogue: 0,0:40:36.04,0:40:42.40,Default,,0000,0000,0000,,building things for Windows; or you want\Nto be able to do gaming and stuff
Dialogue: 0,0:40:42.40,0:40:46.89,Default,,0000,0000,0000,,but in that case, maybe try dual booting,\Neven though that's a pain too
Dialogue: 0,0:40:46.89,0:40:52.03,Default,,0000,0000,0000,,Mac OS is a is a good sort of middle point\Nbetween the two where you get a system
Dialogue: 0,0:40:52.03,0:40:57.85,Default,,0000,0000,0000,,that is like relatively nicely polished\Nfor you. But you still have access to
Dialogue: 0,0:40:57.85,0:41:01.19,Default,,0000,0000,0000,,some of the lower-level bits\Nat least to a certain extent.
Dialogue: 0,0:41:01.19,0:41:07.45,Default,,0000,0000,0000,,it's also really easy to dual boot Mac OS and Windows\Nit is not quite the case with like Mac OS and
Dialogue: 0,0:41:07.45,0:41:09.65,Default,,0000,0000,0000,,Linux or Linux and Windows
Dialogue: 0,0:41:13.91,0:41:15.75,Default,,0000,0000,0000,,Alright, for the rest of the\Nquestions so these are
Dialogue: 0,0:41:15.76,0:41:18.76,Default,,0000,0000,0000,,all 0 upvote questions so maybe we can go\Nthrough them quickly in the last five
Dialogue: 0,0:41:18.76,0:41:23.47,Default,,0000,0000,0000,,or so minutes of class. So the next\None is Vim versus Emacs? Vim!
Dialogue: 0,0:41:23.47,0:41:30.91,Default,,0000,0000,0000,,Easy answer, but a more serious answer is like I think\Nall three of us use vim as our primary editor
Dialogue: 0,0:41:30.91,0:41:34.93,Default,,0000,0000,0000,,I use Emacs for some research specific\Nstuff which requires Emacs but
Dialogue: 0,0:41:34.93,0:41:38.68,Default,,0000,0000,0000,,at a higher level both editors have interesting\Nideas behind them and if you
Dialogue: 0,0:41:38.68,0:41:43.06,Default,,0000,0000,0000,,have the time is worth exploring both\Nto see which fits you better and also
Dialogue: 0,0:41:43.06,0:41:46.81,Default,,0000,0000,0000,,you can use Emacs and run it in a vim\Nemulation mode. I actually know a
Dialogue: 0,0:41:46.81,0:41:49.09,Default,,0000,0000,0000,,good number of people who do that so\Nthey get access to some of the cool
Dialogue: 0,0:41:49.09,0:41:52.63,Default,,0000,0000,0000,,Emacs functionality and some of the cool\Nphilosophy behind that like Emacs is
Dialogue: 0,0:41:52.63,0:41:55.39,Default,,0000,0000,0000,,programmable through Lisp which is kind of cool.
Dialogue: 0,0:41:55.39,0:41:59.41,Default,,0000,0000,0000,,Much better than vimscript, but people like\Nvim's modal editing, so there's an
Dialogue: 0,0:41:59.41,0:42:04.48,Default,,0000,0000,0000,,emacs plugin called evil mode which gives\Nyou vim modal editing within Emacs so
Dialogue: 0,0:42:04.48,0:42:08.08,Default,,0000,0000,0000,,it's not necessarily a binary choice you\Ncan kind of combine both tools if you
Dialogue: 0,0:42:08.08,0:42:11.15,Default,,0000,0000,0000,,want to. And it's worth exploring\Nboth if you have the time.
Dialogue: 0,0:42:11.15,0:42:12.73,Default,,0000,0000,0000,,Next question
Dialogue: 0,0:42:12.73,0:42:15.67,Default,,0000,0000,0000,,Any tips or tricks for machine\Nlearning applications?
Dialogue: 0,0:42:19.27,0:42:22.35,Default,,0000,0000,0000,,I think, like knowing how
Dialogue: 0,0:42:22.36,0:42:24.79,Default,,0000,0000,0000,,a lot of these tools, mainly the data wrangling
Dialogue: 0,0:42:24.79,0:42:30.04,Default,,0000,0000,0000,,a lot of the shell tools, it's really\Nimportant because it seems a lot
Dialogue: 0,0:42:30.04,0:42:33.85,Default,,0000,0000,0000,,of what you're doing as machine learning\Nresearcher is trying different things
Dialogue: 0,0:42:33.85,0:42:39.49,Default,,0000,0000,0000,,but I think one core aspect of doing that,\Nand like a lot of scientific work is being
Dialogue: 0,0:42:39.49,0:42:44.50,Default,,0000,0000,0000,,able to have reproducible results\Nand logging them in a sensible way
Dialogue: 0,0:42:44.50,0:42:47.71,Default,,0000,0000,0000,,So for example, instead of trying to come\Nup with really hacky solutions of how
Dialogue: 0,0:42:47.71,0:42:51.15,Default,,0000,0000,0000,,you name your folders to make\Nsense of the experiments
Dialogue: 0,0:42:51.15,0:42:53.25,Default,,0000,0000,0000,,Maybe it's just worth having for example
Dialogue: 0,0:42:53.25,0:42:55.93,Default,,0000,0000,0000,,what I do is have like a JSON\Nfile that describes the
Dialogue: 0,0:42:55.93,0:43:00.37,Default,,0000,0000,0000,,entire experiment I know like all the parameters\Nthat are within and then I can
Dialogue: 0,0:43:00.37,0:43:05.11,Default,,0000,0000,0000,,really quickly, using the tools that\Nwe have covered, query for all the
Dialogue: 0,0:43:05.11,0:43:09.70,Default,,0000,0000,0000,,experiments that have some specific\Npurpose or use some data set
Dialogue: 0,0:43:09.70,0:43:15.07,Default,,0000,0000,0000,,Things like that. Apart from that, the other\Nside of this is, if you are running
Dialogue: 0,0:43:15.07,0:43:19.87,Default,,0000,0000,0000,,kind of things for training machine\Nlearning applications and you
Dialogue: 0,0:43:19.87,0:43:23.98,Default,,0000,0000,0000,,are not already using some sort of\Ncluster, like university or your
Dialogue: 0,0:43:23.98,0:43:28.30,Default,,0000,0000,0000,,company is providing and you're just kind\Nof manually sshing, like a lot of
Dialogue: 0,0:43:28.30,0:43:31.23,Default,,0000,0000,0000,,labs do, because that's kind of the easy way
Dialogue: 0,0:43:31.23,0:43:36.67,Default,,0000,0000,0000,,It's worth automating a lot of that job\Nbecause it might not seem like it but
Dialogue: 0,0:43:36.67,0:43:40.60,Default,,0000,0000,0000,,manually doing a lot of these operations\Ntakes away a lot of your time and also
Dialogue: 0,0:43:40.60,0:43:45.03,Default,,0000,0000,0000,,kind of your mental energy\Nfor running these things
Dialogue: 0,0:43:48.55,0:43:51.69,Default,,0000,0000,0000,,Anymore vim tips?
Dialogue: 0,0:43:51.69,0:43:56.77,Default,,0000,0000,0000,,I have one. So in the vim lecture we tried\Nnot to link you to too many different
Dialogue: 0,0:43:56.77,0:44:00.13,Default,,0000,0000,0000,,vim plugins because we didn't want that\Nlecture to be overwhelming but I think
Dialogue: 0,0:44:00.13,0:44:02.92,Default,,0000,0000,0000,,it's actually worth exploring vim plugins\Nbecause there are lots and lots
Dialogue: 0,0:44:02.92,0:44:07.09,Default,,0000,0000,0000,,of really cool ones out there.\NOne resource you can use is the
Dialogue: 0,0:44:07.09,0:44:10.57,Default,,0000,0000,0000,,different instructors dotfiles like a lot\Nof us, I think I use like two dozen
Dialogue: 0,0:44:10.57,0:44:14.32,Default,,0000,0000,0000,,vim plugins and I find a lot of them quite\Nhelpful and I use them every day
Dialogue: 0,0:44:14.32,0:44:18.31,Default,,0000,0000,0000,,we all use slightly different subsets of\Nthem. So go look at what we use or look
Dialogue: 0,0:44:18.31,0:44:22.13,Default,,0000,0000,0000,,at some of the other resources we've linked\Nto and you might find some stuff useful
Dialogue: 0,0:44:22.79,0:44:26.95,Default,,0000,0000,0000,,A thing to add to that is, I don't think\Nwe went into a lot detail in the
Dialogue: 0,0:44:27.04,0:44:31.57,Default,,0000,0000,0000,,lecture, correct me if I'm wrong. It's\Ngetting familiar with the leader key
Dialogue: 0,0:44:31.57,0:44:35.02,Default,,0000,0000,0000,,Which is kind of a special key\Nthat a lot of programs will
Dialogue: 0,0:44:35.02,0:44:39.08,Default,,0000,0000,0000,,especially plugins, that will link to\Nand for a lot of the common operations
Dialogue: 0,0:44:39.08,0:44:44.66,Default,,0000,0000,0000,,vim has short ways of doing it, but you\Ncan just figure out like quicker
Dialogue: 0,0:44:44.66,0:44:50.03,Default,,0000,0000,0000,,versions for doing them. So for example, like\NI know that you can do like semicolon WQ
Dialogue: 0,0:44:50.03,0:44:55.52,Default,,0000,0000,0000,,to save and exit or that you\Ncan do like capital ZZ but I
Dialogue: 0,0:44:55.52,0:44:59.24,Default,,0000,0000,0000,,just actually just do leader (which for\Nme is the space) and then W. And I have
Dialogue: 0,0:44:59.24,0:45:04.13,Default,,0000,0000,0000,,done that for a lot of a lot of kind of\Ncommon operations that I keep doing all
Dialogue: 0,0:45:04.13,0:45:08.09,Default,,0000,0000,0000,,the time. Because just saving one keystroke\Nfor an extremely common operation
Dialogue: 0,0:45:08.09,0:45:11.37,Default,,0000,0000,0000,,is just saving thousands a month
Dialogue: 0,0:45:11.37,0:45:12.95,Default,,0000,0000,0000,,Yeah just to expand a little bit
Dialogue: 0,0:45:12.95,0:45:17.03,Default,,0000,0000,0000,,on what the leader key is so in vim you\Ncan bind some keys I can do like ctrl J
Dialogue: 0,0:45:17.03,0:45:20.48,Default,,0000,0000,0000,,does something like holding one key and\Nthen pressing another I can bind that to
Dialogue: 0,0:45:20.48,0:45:23.78,Default,,0000,0000,0000,,something or I can bind a single keystroke\Nto something. What the leader
Dialogue: 0,0:45:23.78,0:45:26.03,Default,,0000,0000,0000,,key lets you do, is bind
Dialogue: 0,0:45:26.03,0:45:28.31,Default,,0000,0000,0000,,So you can assign any key\Nto be the leader key and
Dialogue: 0,0:45:28.31,0:45:32.84,Default,,0000,0000,0000,,then you can assign leader followed by\Nsome other key to some action so for
Dialogue: 0,0:45:32.84,0:45:36.83,Default,,0000,0000,0000,,example like Jose's leader key is space\Nand they can combine space and then
Dialogue: 0,0:45:36.83,0:45:41.60,Default,,0000,0000,0000,,releasing space followed by some other\Nkey to an arbitrary vim command so it
Dialogue: 0,0:45:41.60,0:45:45.63,Default,,0000,0000,0000,,just gives you yet another way of binding\Nlike a whole set of key combinations.
Dialogue: 0,0:45:45.63,0:45:49.75,Default,,0000,0000,0000,,Leader key plus kind of any key on\Nthe keyboard to some functionality
Dialogue: 0,0:45:49.75,0:45:53.75,Default,,0000,0000,0000,,I think I've I forget whether\Nwe covered macros in the vim
Dialogue: 0,0:45:53.75,0:45:58.58,Default,,0000,0000,0000,,uh sure but like vim macros are worth\Nlearning they're not that complicated
Dialogue: 0,0:45:58.58,0:46:03.14,Default,,0000,0000,0000,,but knowing that they're there and knowing\Nhow to use them is going to save
Dialogue: 0,0:46:03.14,0:46:09.50,Default,,0000,0000,0000,,you so much time. The other one is something\Ncalled marks. So in vim you can
Dialogue: 0,0:46:09.50,0:46:13.49,Default,,0000,0000,0000,,press m and then any letter on your keyboard\Nto make a mark in that file and
Dialogue: 0,0:46:13.49,0:46:18.02,Default,,0000,0000,0000,,then you can press apostrophe on the\Nsame letter to jump back to the same
Dialogue: 0,0:46:18.02,0:46:21.80,Default,,0000,0000,0000,,place. This is really useful if you're\Nlike moving back and forth
Dialogue: 0,0:46:21.80,0:46:25.49,Default,,0000,0000,0000,,between two different parts of your code\Nfor example. You can mark one as A and
Dialogue: 0,0:46:25.49,0:46:29.61,Default,,0000,0000,0000,,one as B and you can then jump between\Nthem with tick A and tick B.
Dialogue: 0,0:46:29.61,0:46:34.85,Default,,0000,0000,0000,,There's also Ctrl+O which jumps to the previous\Nplace you were in the file no matter
Dialogue: 0,0:46:34.85,0:46:40.61,Default,,0000,0000,0000,,what caused you to move. So for example\Nif I am in a some line and then I jump
Dialogue: 0,0:46:40.61,0:46:45.20,Default,,0000,0000,0000,,to B and then I jump to A, Ctrl+O will\Ntake me back to B and then back to the
Dialogue: 0,0:46:45.20,0:46:48.83,Default,,0000,0000,0000,,place I originally was. This can also be\Nhandy for things like if you're doing a
Dialogue: 0,0:46:48.83,0:46:52.67,Default,,0000,0000,0000,,search then the place that you\Nstarted the search is a part of
Dialogue: 0,0:46:52.67,0:46:56.21,Default,,0000,0000,0000,,that stack. So I can do a search I can\Nthen like step through the results
Dialogue: 0,0:46:56.21,0:47:00.80,Default,,0000,0000,0000,,and like change them and then Ctrl+O\Nall the way back up to the search
Dialogue: 0,0:47:00.80,0:47:06.20,Default,,0000,0000,0000,,Ctrl+O also lets you move across files so\Nif I go from one file to somewhere else in
Dialogue: 0,0:47:06.20,0:47:09.68,Default,,0000,0000,0000,,different file and somewhere else in the\Nfirst file Ctrl+O will move me back
Dialogue: 0,0:47:09.68,0:47:15.26,Default,,0000,0000,0000,,through that stack and then there's\NCtrl+I to move forward in that
Dialogue: 0,0:47:15.26,0:47:20.84,Default,,0000,0000,0000,,stack and so it's not as though you\Npop it and it goes away forever
Dialogue: 0,0:47:20.84,0:47:26.54,Default,,0000,0000,0000,,The command colon earlier is really handy.\NSo, colon earlier gives you an earlier
Dialogue: 0,0:47:26.54,0:47:32.87,Default,,0000,0000,0000,,version of the same file and it it does\Nthis based on time not based on actions
Dialogue: 0,0:47:32.87,0:47:36.65,Default,,0000,0000,0000,,so for example if you press a bunch of like\Nundo and redo and make some changes
Dialogue: 0,0:47:36.65,0:47:42.56,Default,,0000,0000,0000,,and stuff, earlier will take a literally\Nearlier as in time version of your file
Dialogue: 0,0:47:42.56,0:47:46.97,Default,,0000,0000,0000,,and restore it to your buffer. This can\Nsometimes be good if you like undid and
Dialogue: 0,0:47:46.97,0:47:50.84,Default,,0000,0000,0000,,then rewrote something and then realize\Nyou actually wanted the version that was
Dialogue: 0,0:47:50.84,0:47:55.10,Default,,0000,0000,0000,,there before you started undoing earlier\Nlet's you do this. And there's a plug-in
Dialogue: 0,0:47:55.10,0:48:01.97,Default,,0000,0000,0000,,called undo tree or something like\Nthat There are several of these,
Dialogue: 0,0:48:01.97,0:48:05.78,Default,,0000,0000,0000,,that let you actually explore the full\Ntree of undo history the vim keeps
Dialogue: 0,0:48:05.78,0:48:09.20,Default,,0000,0000,0000,,because it doesn't just keep a linear history\Nit actually keeps the full tree
Dialogue: 0,0:48:09.20,0:48:12.77,Default,,0000,0000,0000,,and letting you explore that might in\Nsome cases save you from having to
Dialogue: 0,0:48:12.77,0:48:16.46,Default,,0000,0000,0000,,re-type stuff you typed in the past or\Nstuff you just forgot exactly what you
Dialogue: 0,0:48:16.46,0:48:21.08,Default,,0000,0000,0000,,had there that used to work and no longer\Nworks. And this is one final one I
Dialogue: 0,0:48:21.08,0:48:26.75,Default,,0000,0000,0000,,want to mention which is, we mentioned\Nhow in vim you have verbs and nouns
Dialogue: 0,0:48:26.75,0:48:33.20,Default,,0000,0000,0000,,right to your verbs like delete or yank\Nand then you have nouns like next of
Dialogue: 0,0:48:33.20,0:48:37.40,Default,,0000,0000,0000,,this character or percent to swap brackets\Nand that sort of stuff the
Dialogue: 0,0:48:37.40,0:48:44.57,Default,,0000,0000,0000,,search command is a noun so you can do\Nthings like D slash and then a string
Dialogue: 0,0:48:44.57,0:48:50.26,Default,,0000,0000,0000,,and it will delete up to the next match\Nof that pattern this is extremely useful
Dialogue: 0,0:48:50.26,0:48:54.25,Default,,0000,0000,0000,,and I use it all the time
Dialogue: 0,0:48:58.50,0:49:03.52,Default,,0000,0000,0000,,One another neat addition on the undo stuff\Nthat I find incredibly valuable in
Dialogue: 0,0:49:03.52,0:49:08.20,Default,,0000,0000,0000,,an everyday basis is that like one of\Nthe built-in functionalities of vim
Dialogue: 0,0:49:08.20,0:49:13.51,Default,,0000,0000,0000,,is that you can specify an undo directory\Nand if you have a specified an
Dialogue: 0,0:49:13.51,0:49:17.62,Default,,0000,0000,0000,,undo directory by default vim, if you\Ndon't have this enabled, whenever you
Dialogue: 0,0:49:17.62,0:49:23.09,Default,,0000,0000,0000,,enter a file your undo history is\Nclean, there's nothing in there
Dialogue: 0,0:49:23.09,0:49:26.37,Default,,0000,0000,0000,,and as you make changes and then\Nundo them you kind of create this
Dialogue: 0,0:49:26.38,0:49:32.80,Default,,0000,0000,0000,,history but as soon as you exit the\Nfile that's lost. Sorry, as soon
Dialogue: 0,0:49:32.80,0:49:37.18,Default,,0000,0000,0000,,as you exit vim, that's lost. However\Nif you have an undodir, vim is
Dialogue: 0,0:49:37.18,0:49:41.65,Default,,0000,0000,0000,,gonna persist all those changes into\Nthis directory so no matter how many
Dialogue: 0,0:49:41.65,0:49:45.58,Default,,0000,0000,0000,,times you enter and leave that history\Nis persisted and it's incredibly
Dialogue: 0,0:49:45.58,0:49:48.19,Default,,0000,0000,0000,,helpful because even like
Dialogue: 0,0:49:48.19,0:49:50.29,Default,,0000,0000,0000,,it can be very helpful for\Nsome files that you modify
Dialogue: 0,0:49:50.29,0:49:54.76,Default,,0000,0000,0000,,often because then you can kind of keep\Nthe flow. But it's also sometimes really
Dialogue: 0,0:49:54.76,0:50:00.01,Default,,0000,0000,0000,,helpful if you modify your bashrc see and\Nsomething broke like five days later and
Dialogue: 0,0:50:00.01,0:50:03.07,Default,,0000,0000,0000,,then you've vim again. Like what actually\Ndid I change ,if you don't
Dialogue: 0,0:50:03.07,0:50:06.76,Default,,0000,0000,0000,,have say like version control, then\Nyou can just check the undos and
Dialogue: 0,0:50:06.76,0:50:10.66,Default,,0000,0000,0000,,that's actually what happened. And\Nthe last one, it's also really
Dialogue: 0,0:50:10.66,0:50:14.89,Default,,0000,0000,0000,,worth familiarizing yourself with registers\Nand what different special
Dialogue: 0,0:50:14.89,0:50:20.38,Default,,0000,0000,0000,,registers vim uses. So for example if\Nyou want to copy/paste really that's
Dialogue: 0,0:50:20.38,0:50:26.20,Default,,0000,0000,0000,,gone into in a specific register and if you\Nwant to for example use the a OS a copy
Dialogue: 0,0:50:26.20,0:50:30.04,Default,,0000,0000,0000,,like the OS clipboard, you should\Nbe copying or yanking
Dialogue: 0,0:50:30.04,0:50:36.25,Default,,0000,0000,0000,,copying and pasting from a different register\Nand there's a lot of them and yeah
Dialogue: 0,0:50:36.25,0:50:41.31,Default,,0000,0000,0000,,I think that you should explore, there's\Na lot of things to know about registers
Dialogue: 0,0:50:42.27,0:50:45.07,Default,,0000,0000,0000,,The next question is asking about two-factor\Nauthentication and I'll just give
Dialogue: 0,0:50:45.07,0:50:48.49,Default,,0000,0000,0000,,a very quick answer to this one in the interest\Nof time. So it's worth using two
Dialogue: 0,0:50:48.49,0:50:52.48,Default,,0000,0000,0000,,factor auth for anything security sensitive\Nso I use it for my GitHub
Dialogue: 0,0:50:52.48,0:50:56.71,Default,,0000,0000,0000,,account and for my email and stuff like\Nthat. And there's a bunch of different
Dialogue: 0,0:50:56.71,0:51:01.36,Default,,0000,0000,0000,,types of two-factor auth. From SMS based\Nto factor auth where you get special
Dialogue: 0,0:51:01.36,0:51:04.63,Default,,0000,0000,0000,,like a number texted to you when you try\Nto log in you have to type that number
Dialogue: 0,0:51:04.63,0:51:08.71,Default,,0000,0000,0000,,and to other tools like universal to\Nfactor this is like those Yubikeys
Dialogue: 0,0:51:08.71,0:51:11.35,Default,,0000,0000,0000,,that you plug into your you have\Nto tap it every time you login
Dialogue: 0,0:51:11.35,0:51:18.13,Default,,0000,0000,0000,,so not all, (yeah Jon is holding a\NYubikey), not all two-factor auth is
Dialogue: 0,0:51:18.13,0:51:22.24,Default,,0000,0000,0000,,created equal and you really want to be\Nusing something like U2F rather than SMS
Dialogue: 0,0:51:22.24,0:51:25.30,Default,,0000,0000,0000,,based to factor auth. There something\Nbased on one-time pass codes that you
Dialogue: 0,0:51:25.30,0:51:28.81,Default,,0000,0000,0000,,have to type in we don't have time to get\Ninto the details of why some methods
Dialogue: 0,0:51:28.81,0:51:32.02,Default,,0000,0000,0000,,are better than others but at a high\Nlevel use U2F and the Internet has
Dialogue: 0,0:51:32.02,0:51:37.56,Default,,0000,0000,0000,,plenty of explanations for why other\Nmethods are not a great idea
Dialogue: 0,0:51:37.71,0:51:41.85,Default,,0000,0000,0000,,Last question, any comments on differences\Nbetween web browsers?
Dialogue: 0,0:51:48.17,0:51:50.17,Default,,0000,0000,0000,,Yes
Dialogue: 0,0:51:54.71,0:52:00.45,Default,,0000,0000,0000,,Differences between web browsers, there\Nare fewer and fewer differences between
Dialogue: 0,0:52:00.46,0:52:06.00,Default,,0000,0000,0000,,web browsers these day. At this point\Nalmost all web browsers are chrome
Dialogue: 0,0:52:06.00,0:52:09.58,Default,,0000,0000,0000,,Either because you're using Chrome or\Nbecause you're using a browser that's
Dialogue: 0,0:52:09.58,0:52:15.55,Default,,0000,0000,0000,,using the same browser engine as Chrome.\NIt's a little bit sad, one might say, but
Dialogue: 0,0:52:15.55,0:52:20.51,Default,,0000,0000,0000,,I think these days whether you choose
Dialogue: 0,0:52:20.51,0:52:24.45,Default,,0000,0000,0000,,Chrome is a great browser for security reasons
Dialogue: 0,0:52:24.45,0:52:28.47,Default,,0000,0000,0000,,if you want to have something\Nthat's more customizable or
Dialogue: 0,0:52:28.47,0:52:39.49,Default,,0000,0000,0000,,you don't want to be tied to Google then\Nuse Firefox, don't use Safari it's a
Dialogue: 0,0:52:39.49,0:52:45.70,Default,,0000,0000,0000,,worse version of Chrome. The new Internet\NExplorer edge is pretty decent and also
Dialogue: 0,0:52:45.70,0:52:50.82,Default,,0000,0000,0000,,uses the same browser engine as\NChrome and that's probably fine
Dialogue: 0,0:52:50.82,0:52:54.64,Default,,0000,0000,0000,,although avoid it if you can because it\Nhas some like legacy modes you don't
Dialogue: 0,0:52:54.64,0:52:58.06,Default,,0000,0000,0000,,want to deal with. I think that's
Dialogue: 0,0:52:58.06,0:53:03.09,Default,,0000,0000,0000,,Oh, there's a cool new browser called flow
Dialogue: 0,0:53:03.09,0:53:05.50,Default,,0000,0000,0000,,that you can't use for anything useful\Nyet but they're actually writing
Dialogue: 0,0:53:05.50,0:53:08.69,Default,,0000,0000,0000,,their own browser engine and that's really neat
Dialogue: 0,0:53:08.69,0:53:14.95,Default,,0000,0000,0000,,Firefox also has this project called servo which is\Nthey're really implementing their browser engine
Dialogue: 0,0:53:14.95,0:53:19.57,Default,,0000,0000,0000,,in Rust in order to write it to be like\Nsuper concurrent and what they've done
Dialogue: 0,0:53:19.57,0:53:24.96,Default,,0000,0000,0000,,is they've started to take modules\Nfrom that version and port them
Dialogue: 0,0:53:24.96,0:53:29.04,Default,,0000,0000,0000,,over to gecko or integrate them with gecko\Nwhich is the main browser engine
Dialogue: 0,0:53:29.04,0:53:32.22,Default,,0000,0000,0000,,for Firefox just to get those\Nspeed ups there as well
Dialogue: 0,0:53:32.22,0:53:37.03,Default,,0000,0000,0000,,and that's a neat neat thing\Nyou can be watching out for
Dialogue: 0,0:53:39.23,0:53:41.85,Default,,0000,0000,0000,,That is all the questions, hey we did it. Nice
Dialogue: 0,0:53:41.85,0:53:50.75,Default,,0000,0000,0000,,I guess thanks for taking the missing semester\Nclass and let's do it again next year