1
00:00:01,023 --> 00:00:03,411
Hi, can everyone hear me okay?

2
00:00:03,411 --> 00:00:06,577
Okay, so welcome back to

3
00:00:06,577 --> 00:00:09,350
the missing semester of your CS education,

4
00:00:09,350 --> 00:00:12,410
today we're having as a 
lecture topic "potpourri"

5
00:00:12,410 --> 00:00:15,170
It's gonna be some miscellaneous
combination of topics

6
00:00:15,170 --> 00:00:18,820
that we the instructors find
that are interesting

7
00:00:19,000 --> 00:00:21,201
But none of them were on their own lecture

8
00:00:21,201 --> 00:00:24,900
because they're certain topics that 
we just want you to know about

9
00:00:24,900 --> 00:00:27,510
because they can be really helpful

10
00:00:28,639 --> 00:00:33,058
And since we're not gonna delve into
a lot of detail in the topics

11
00:00:33,059 --> 00:00:36,972
If you're more interested about them just
feel free to come and ask us questions

12
00:00:36,972 --> 00:00:40,870
at the end or as we go over 
them in the lecture

13
00:00:40,870 --> 00:00:45,660
So the first thing I want to talk about 
is keyboard re-mappings

14
00:00:46,180 --> 00:00:48,456
So by now you probably realized that

15
00:00:48,456 --> 00:00:52,469
we have encouraged you to use the
keyboard as your main input method

16
00:00:52,469 --> 00:00:55,149
so for example, when we went
into the editors lecture,

17
00:00:55,149 --> 00:00:59,639
one of the main ideas of Vim was using
your keyboard as much as possible

18
00:00:59,640 --> 00:01:05,109
so you don't have to rely on going to the
mouse because going to the mouse is slow

19
00:01:05,209 --> 00:01:08,460
And the thing is your keyboard, as with
many things in your computer

20
00:01:08,460 --> 00:01:11,600
is nothing kind of magical,
it can be configured

21
00:01:11,600 --> 00:01:16,739
And it's worth configuring because a lot
of the defaults might not be optimal

22
00:01:16,990 --> 00:01:22,140
The most simple modification that you
can do is just to remap keys

23
00:01:22,450 --> 00:01:25,594
So one of the things we alerted
in the editor lecture

24
00:01:25,594 --> 00:01:28,430
is the Caps Lock key is a really good key

25
00:01:28,430 --> 00:01:31,049
because it's kind of right
in the home row

26
00:01:31,049 --> 00:01:35,250
and it's kind of large and there,
but it's not useful

27
00:01:35,250 --> 00:01:38,500
You'll probably realize that you don't
use your Caps Lock as often

28
00:01:38,500 --> 00:01:40,200
as when you'd want to use that

29
00:01:40,200 --> 00:01:42,550
So you can just remap your 
Caps Lock key

30
00:01:42,550 --> 00:01:44,450
to something more useful as we mentioned

31
00:01:44,450 --> 00:01:46,322
like Escape if you're a Vim user

32
00:01:46,322 --> 00:01:49,050
or like Control if you're an Emacs user

33
00:01:49,050 --> 00:01:52,180
or useful re-mappings, like a lot of the

34
00:01:52,180 --> 00:01:56,006
upper row 'F' function keys,
or like Print Screen

35
00:01:56,006 --> 00:01:58,649
You can remap them for example
to your media key

36
00:01:58,649 --> 00:02:01,079
so like when you type Print Screen,

37
00:02:01,079 --> 00:02:03,613
you probably don't have to 
do Print Screen that often

38
00:02:03,613 --> 00:02:06,959
but you probably want to play
or pause your music

39
00:02:07,940 --> 00:02:11,272
And pretty much every operating system has

40
00:02:11,272 --> 00:02:14,269
some tools that you can use
to configure this

41
00:02:14,269 --> 00:02:16,343
I'm not gonna go into the details

42
00:02:16,343 --> 00:02:19,460
but there's some of them
listed in the notes

43
00:02:20,080 --> 00:02:23,140
Erm... Let me check

44
00:02:23,140 --> 00:02:26,597
Oh yeah, another thing that you can 
do with keyboard re-mappings is

45
00:02:26,597 --> 00:02:29,160
that you can do more complex combinations

46
00:02:29,160 --> 00:02:33,360
You can have a combination of keys 
mapped to some action

47
00:02:33,360 --> 00:02:36,305
So for example, I have keyboard 
re-mappings that

48
00:02:36,305 --> 00:02:40,236
whenever I do Ctrl+Enter, I open
a new terminal window

49
00:02:40,236 --> 00:02:42,692
because that's a thing I do
fairly often and

50
00:02:42,692 --> 00:02:45,600
by default on Mac there is no 
key binding to do that

51
00:02:45,820 --> 00:02:49,090
or Ctrl+Shift+Enter will open a 
new browser window,

52
00:02:49,090 --> 00:02:51,480
another operation that I do
on a daily basis

53
00:02:51,480 --> 00:02:56,160
So I don't have to grab my mouse
and go to Chrome to do that

54
00:02:56,499 --> 00:02:59,699
You can also do remapping to 
perform actions

55
00:02:59,699 --> 00:03:02,550
If you don't want to be typing your
password all the...

56
00:03:02,550 --> 00:03:03,807
Sorry, not your password

57
00:03:03,807 --> 00:03:06,785
Your Email or your password,
or for example, your MIT ID

58
00:03:06,785 --> 00:03:08,920
like, you may not remember it by heart

59
00:03:08,920 --> 00:03:12,330
then you can just have a
keyboard combination that

60
00:03:12,330 --> 00:03:16,800
will just perform the action of
pasting that text

61
00:03:19,580 --> 00:03:21,377
Lastly, there are more...

62
00:03:21,377 --> 00:03:24,727
Right now it looks like you just have
to do some function of

63
00:03:24,727 --> 00:03:28,360
"this is the keys that you press and
this is the action that happens"

64
00:03:28,360 --> 00:03:32,233
But actually there are more complex
keyboard combinations, and

65
00:03:32,233 --> 00:03:36,970
as you go through you'll learn
so you can do keyboard sequences

66
00:03:36,970 --> 00:03:39,110
So for example when we were
dealing with tmux,

67
00:03:39,110 --> 00:03:41,168
in tmux there was this notion of

68
00:03:41,168 --> 00:03:46,097
first you press Ctrl+A or Ctrl+B, like
you press some prefix

69
00:03:46,097 --> 00:03:49,660
and then some other key and
that means something

70
00:03:49,660 --> 00:03:51,406
A lot of these softwares allow that

71
00:03:51,406 --> 00:03:55,889
So for example, in my keyboard, since
I'm not using Caps Lock at all, but

72
00:03:55,889 --> 00:04:00,219
every so often I have to use when my 
undergrad has some software that

73
00:04:00,219 --> 00:04:03,200
relied on Caps Lock for changing modes

74
00:04:03,200 --> 00:04:08,764
Then I can press Shift five times
in a row quickly, and then

75
00:04:08,764 --> 00:04:12,469
this software that is in the middle
interpreting these commands and

76
00:04:12,469 --> 00:04:17,049
remapping to some other, will send a
single Caps Lock command for that

77
00:04:17,840 --> 00:04:20,238
Some more examples of that is that

78
00:04:20,238 --> 00:04:27,770
I mentioned that you can use your
Caps Lock key to map to Escape or Control

79
00:04:27,770 --> 00:04:29,660
but you actually can remap it to both

80
00:04:29,660 --> 00:04:31,370
So in my computer when

81
00:04:31,370 --> 00:04:35,100
I just tap the Caps Lock key, that's
interpreted as an Escape

82
00:04:35,100 --> 00:04:40,060
However if I press it and hold it,
this software can understand the

83
00:04:40,060 --> 00:04:42,128
difference between quickly pressing it and

84
00:04:42,128 --> 00:04:46,155
just holding it for using in 
combination with some other key

85
00:04:46,155 --> 00:04:48,789
and then in that case
it's mapped to Control

86
00:04:48,860 --> 00:04:51,833
So a lot of these more
advanced configurations are

87
00:04:51,833 --> 00:04:55,190
supported on a lot of these tools

88
00:04:55,580 --> 00:04:59,859
As I mentioned we have a 
short list of good defaults

89
00:05:00,190 --> 00:05:05,440
for these programs for
Windows, Mac OS and Linux

90
00:05:05,680 --> 00:05:08,360
Any questions on this topic?

91
00:05:13,120 --> 00:05:16,820
Okay, now I'm going to cover an
unrelated topic to keyboard mappings

92
00:05:16,820 --> 00:05:17,530
[chuckles]

93
00:05:17,530 --> 00:05:20,912
We're going to see a lot of these
unrelated transitions in this lecture

94
00:05:20,912 --> 00:05:23,600
And it's the concept of "daemons"

95
00:05:23,600 --> 00:05:25,209
So probably you have...

96
00:05:25,209 --> 00:05:28,502
Maybe if you are not familiar with 
the world, it might seem alien,

97
00:05:28,502 --> 00:05:31,023
but the concept of daemon
you're pretty familiar with

98
00:05:31,023 --> 00:05:33,247
Most computers when you are running them

99
00:05:33,247 --> 00:05:36,903
there's software that you start and run,

100
00:05:36,903 --> 00:05:38,905
like the commands that we have been seeing

101
00:05:38,905 --> 00:05:42,720
You'd type "ls" and then you're
calling the List command

102
00:05:42,720 --> 00:05:46,776
The "ls" command executes because you
asked it to execute and then it finishes

103
00:05:47,296 --> 00:05:51,534
But a lot of other programs are just
running as background processes

104
00:05:51,534 --> 00:05:54,820
and they're just executing
in the background

105
00:05:54,820 --> 00:05:57,385
and waiting for events to happen

106
00:05:57,385 --> 00:06:00,502
or enabling some sort of functionality
in your computer

107
00:06:01,669 --> 00:06:05,309
Examples of these processes may be
like your Network Man[ager],

108
00:06:05,309 --> 00:06:08,427
like the part of your computer
that is managing the network

109
00:06:08,427 --> 00:06:12,289
or the part of your computer
that is managing the display

110
00:06:12,289 --> 00:06:14,119
Things like that

111
00:06:14,570 --> 00:06:18,930
You will see that a lot of what 
is enabled by daemons is

112
00:06:18,930 --> 00:06:21,410
usually programs that end with a 'd'

113
00:06:21,410 --> 00:06:25,869
So for instance, when you are 
SSHing into a computer,

114
00:06:25,870 --> 00:06:30,399
the receiving computer has to
have a SSH daemon

115
00:06:30,399 --> 00:06:32,639
and the program is called "sshd"

116
00:06:32,639 --> 00:06:34,860
and if this program is not running then

117
00:06:34,860 --> 00:06:37,790
there's no way for me
to SSH into the computer

118
00:06:37,790 --> 00:06:41,350
If the program is running, then
that program will be listening

119
00:06:41,350 --> 00:06:44,430
and when you do SSH to that server,

120
00:06:44,430 --> 00:06:48,202
then some incoming request
is gonna enter the computer

121
00:06:48,202 --> 00:06:52,519
the computer is gonna send it to this
daemon that is running in the background

122
00:06:52,519 --> 00:06:56,280
and then the daemon is gonna check
whether you have authorization,

123
00:06:56,280 --> 00:07:01,113
and if so, it's gonna start some login
shell that you can start executing from

124
00:07:01,403 --> 00:07:06,592
And different OSes handle this
somewhat differently

125
00:07:06,592 --> 00:07:11,409
The main idea is they all have
some sort of system daemon

126
00:07:11,409 --> 00:07:14,816
that responds a lot of
these smaller daemons

127
00:07:14,816 --> 00:07:19,106
In Linux, which is one of the OSes that
we are choosing for lot of the examples,

128
00:07:19,106 --> 00:07:24,290
the tool that you're using
is the "systemd"

129
00:07:24,290 --> 00:07:29,160
Again, first the system daemon is gonna
start a lot of these processes

130
00:07:29,160 --> 00:07:33,380
and if you use this "systemctl" command,

131
00:07:33,380 --> 00:07:37,460
you can check for the status
of different daemons

132
00:07:37,460 --> 00:07:40,586
you can check for which ones are running,

133
00:07:43,380 --> 00:07:47,160
you can ask it to start processes,
to stop them

134
00:07:47,160 --> 00:07:49,680
This is kind of an once-off operation

135
00:07:49,680 --> 00:07:53,387
You can also enable it
and then disable them,

136
00:07:53,387 --> 00:07:57,312
which will tell the system
to run them at boot

137
00:07:57,312 --> 00:08:00,380
or stop running the boot
if they were enabled

138
00:08:01,340 --> 00:08:07,465
And perhaps more interestingly, you can
configure your own "systemd" units

139
00:08:07,465 --> 00:08:13,190
So, so far all the examples are a lot
of what the computer has to do,

140
00:08:13,190 --> 00:08:17,890
but say you want to run a web server.
One solution, you could just like,

141
00:08:17,890 --> 00:08:23,030
every time you start your computer
you could open a tmux session

142
00:08:23,030 --> 00:08:24,880
and then execute the command,

143
00:08:24,880 --> 00:08:30,050
but that's not really the way that your 
computer expects daemons to be run

144
00:08:30,050 --> 00:08:32,666
The way your computer expects
daemons to be run is

145
00:08:32,666 --> 00:08:35,446
by using some sort of "systemd" unit

146
00:08:35,446 --> 00:08:40,533
It's like a configuration that tells
"systemd" how to execute this process

147
00:08:40,533 --> 00:08:43,960
So an example of this is...

148
00:08:45,428 --> 00:08:47,912
Here's a very simple example

149
00:08:47,912 --> 00:08:51,109
So what is happening here is

150
00:08:51,109 --> 00:08:56,694
we're describing to "systemd" what needs
to be done for this program to execute.

151
00:08:56,694 --> 00:09:00,911
This example is just running
a simple Python app

152
00:09:00,911 --> 00:09:04,854
You can think of it as a web server that
can be implemented using

153
00:09:04,854 --> 00:09:08,349
some Python web server library.
And here we're saying

154
00:09:08,349 --> 00:09:12,546
this is the description, we're
saying after, like this is important

155
00:09:12,546 --> 00:09:16,800
"Systemd" has a list of
services that have to start

156
00:09:16,800 --> 00:09:19,019
Like, all these daemons have to be started

157
00:09:19,019 --> 00:09:22,815
but maybe there are dependencies 
between these daemons, so here

158
00:09:22,815 --> 00:09:27,923
we're saying "no, you should only start
this after the network has been set up"

159
00:09:27,923 --> 00:09:30,669
because otherwise how will you even try to

160
00:09:30,669 --> 00:09:35,409
configure our web server if I cannot
listen to a network port?

161
00:09:35,409 --> 00:09:38,700
And then we are defining what
users should run this, because

162
00:09:38,700 --> 00:09:42,120
you may want to run this as
your user, or maybe other user,

163
00:09:42,120 --> 00:09:45,360
or maybe the root user
should be running this,

164
00:09:45,360 --> 00:09:48,780
and then what command to run
and under what directory

165
00:09:49,060 --> 00:09:50,761
And whenever you have this,

166
00:09:50,761 --> 00:09:55,060
there can be small corner cases that
you might have to debug, but

167
00:09:55,060 --> 00:09:58,029
this is kind of the core idea and
it can be really useful to

168
00:09:58,029 --> 00:10:01,339
automate the process of
running processes in the background

169
00:10:01,740 --> 00:10:06,219
A small side note to this is
the fact that if you just want

170
00:10:06,219 --> 00:10:10,936
to run a command every so often, 
like in some periodicity,

171
00:10:10,936 --> 00:10:14,639
say every morning I want to
do something in my computer,

172
00:10:14,639 --> 00:10:19,980
you could write a daemon that just does
something and then sleeps for a day, but

173
00:10:19,980 --> 00:10:27,040
actually Linux and Mac OS have already a
daemon that does this called "crond"

174
00:10:27,040 --> 00:10:31,190
and "crond" will take another type of
configuration file where you can say

175
00:10:31,190 --> 00:10:35,660
oh, I want to run a command
every day at 8 AM,

176
00:10:35,660 --> 00:10:38,120
or I want to run a command
every 5 minutes,

177
00:10:38,120 --> 00:10:42,170
and it will just check for
this event and execute it

178
00:10:42,500 --> 00:10:44,640
And with a lot of things
you will find that

179
00:10:44,640 --> 00:10:47,780
there are already daemons that
have been configured for that

180
00:10:48,370 --> 00:10:51,970
Any questions regarding daemons?

181
00:10:53,920 --> 00:10:57,180
[student talking indistinctly]

182
00:10:57,760 --> 00:10:59,930
So the question is whether there is

183
00:10:59,930 --> 00:11:02,700
a folder in the computer
where all of these are

184
00:11:02,700 --> 00:11:04,620
So yeah, like yes and no

185
00:11:04,620 --> 00:11:09,333
Some of these configuration files are
in a couple of different folders

186
00:11:09,333 --> 00:11:13,260
depending on whether they are
system daemons or they are user daemons

187
00:11:13,260 --> 00:11:18,260
Here you can see at the very first line
is where you will place this

188
00:11:18,260 --> 00:11:22,968
for the system daemon to recognize
that it has been installed

189
00:11:22,968 --> 00:11:25,986
but if you just want to list 
all the daemons that are running,

190
00:11:25,986 --> 00:11:30,015
in Linux for example, you can just do 
"systemctl status"

191
00:11:30,015 --> 00:11:33,725
and that's gonna print
a tree of all the systems

192
00:11:33,725 --> 00:11:37,882
and which daemon was spawned
by which other daemon

193
00:11:37,882 --> 00:11:42,128
and a lot of them will be
spawned directly by "systemd"

194
00:11:44,150 --> 00:11:49,080
The next topic is going to be
file systems in user space

195
00:11:49,080 --> 00:11:55,798
So, kind of a quick intro to this is the
fact that whenever you're using a modern..

196
00:11:55,798 --> 00:11:58,260
Oh, yeah, sorry

197
00:11:58,260 --> 00:12:02,254
And whenever you're using a
modern operating system,

198
00:12:02,254 --> 00:12:07,134
you are not tied to a specific file system

199
00:12:07,134 --> 00:12:11,480
So modern systems are fairly modular
and you can for example,

200
00:12:11,480 --> 00:12:15,881
in Linux there are different file systems
that you can use, and

201
00:12:15,881 --> 00:12:22,760
the way this works is because the kernel,
which is what is running most of the

202
00:12:22,760 --> 00:12:29,540
operating system, has some modules that
know how to interact with a file system

203
00:12:29,540 --> 00:12:37,493
So usually when you do something
like "touch foobar",

204
00:12:38,890 --> 00:12:44,524
this is happening at a user level

205
00:12:44,524 --> 00:12:49,655
and then this is going through
to the kernel level

206
00:12:49,655 --> 00:12:54,959
and there is some kind of
layer here that is checking

207
00:12:54,959 --> 00:12:59,940
where this action is happening to 
figure out what file system it is under

208
00:12:59,940 --> 00:13:02,628
So for example, you will
have multiple disks

209
00:13:02,628 --> 00:13:05,302
and all the different disks
have different file systems,

210
00:13:05,302 --> 00:13:10,101
so the kernel has to figure out which
file system operations to use,

211
00:13:10,101 --> 00:13:18,499
and say this file might be in an "Ext4",
which is the most common Linux one,

212
00:13:18,499 --> 00:13:23,552
then whenever you do "touch foobar",
the kernel will hear that

213
00:13:23,552 --> 00:13:28,389
and then it will try to figure out like,
oh, this lives in an Ext4 file system and

214
00:13:28,389 --> 00:13:35,939
it will perform the associated instruction
for creating a file in an Ext4 file system

215
00:13:35,939 --> 00:13:39,109
However, the caveat of having
a system like this is

216
00:13:39,109 --> 00:13:44,903
right now I cannot have user code
that defines how to create a file

217
00:13:44,903 --> 00:13:47,919
and that might be kind of
useful in some cases

218
00:13:47,919 --> 00:13:52,736
Say I want to have a file system that
every time someone creates a file,

219
00:13:52,736 --> 00:13:57,559
it sends me an email, so I can know that
people are creating these files

220
00:13:57,559 --> 00:14:01,694
Here I cannot modify
the kernel to add this

221
00:14:01,694 --> 00:14:06,019
So the solution to this is
something called "FUSE"

222
00:14:06,019 --> 00:14:13,520
And FUSE is a way of having
file systems in user space

223
00:14:13,520 --> 00:14:15,299
So what FUSE will do is,

224
00:14:15,299 --> 00:14:24,859
if this file instead of being in Ext4,
if this file is in a FUSE file system,

225
00:14:24,859 --> 00:14:31,609
FUSE will forward this operation to
some other part of user call

226
00:14:31,609 --> 00:14:35,342
that will say "oh, create this file"

227
00:14:35,342 --> 00:14:40,617
And here I can have the part of the code
that sends an email to me,

228
00:14:40,617 --> 00:14:43,485
saying "oh, this file has been created"

229
00:14:43,485 --> 00:14:46,839
And in case you want to
still create the file,

230
00:14:46,839 --> 00:14:53,649
it can forward back the request
to do some more kernel operations

231
00:14:54,199 --> 00:14:58,279
It might not seem really practical,
but this is just the theory

232
00:14:58,279 --> 00:15:01,794
In practice, why this is useful is because

233
00:15:01,794 --> 00:15:06,240
now you can have user level code
that executes arbitrary actions

234
00:15:06,240 --> 00:15:09,220
when you try to perform
file system operations

235
00:15:09,640 --> 00:15:15,057
A really Interesting example of this
is called "SSHFS"

236
00:15:19,647 --> 00:15:22,526
On an SSHFS FUSE file system,

237
00:15:22,526 --> 00:15:27,984
whenever you try to create,
open, read, write to a file,

238
00:15:27,984 --> 00:15:30,669
instead of trying to do that
to a local file,

239
00:15:30,669 --> 00:15:35,158
it has an SSH connection
to a remote server

240
00:15:35,158 --> 00:15:39,309
So if I try to create a file here,
It will use that SSH connection to

241
00:15:39,309 --> 00:15:42,793
forward that operation to
the remote system

242
00:15:42,793 --> 00:15:44,796
and then it will perform it there

243
00:15:44,796 --> 00:15:50,402
So to all my local computer, to the rest
of the programs running in my computer,

244
00:15:50,402 --> 00:15:54,300
there is this path that looks
like it is here, but

245
00:15:54,300 --> 00:15:58,218
all the operations that are performed
to the path are forwarded

246
00:15:58,218 --> 00:16:00,040
to the remote file system

247
00:16:00,040 --> 00:16:04,222
And with this idea, you will get
some examples in the notes,

248
00:16:04,222 --> 00:16:07,830
and you will find more online,
of ways people have

249
00:16:07,830 --> 00:16:11,065
leveraged this capability to do
fairly interesting file systems

250
00:16:11,065 --> 00:16:13,981
So for example, if instead of having SSH,

251
00:16:13,981 --> 00:16:18,650
you don't care about SSH because
you use Dropbox or Google Drive,

252
00:16:18,650 --> 00:16:24,210
it's fine, people have implemented
FUSE file systems that will mount locally

253
00:16:24,210 --> 00:16:27,300
and every time you try to do
an operation locally,

254
00:16:27,300 --> 00:16:30,790
actually it goes to one of these
cloud storage providers

255
00:16:30,790 --> 00:16:35,110
so you can also use something like
Amazon S3, or Google Cloud Storage,

256
00:16:35,110 --> 00:16:42,046
that don't have the same
kind of UI system

257
00:16:42,046 --> 00:16:45,400
that we synchronize, as
Dropbox or Google Drive

258
00:16:45,980 --> 00:16:50,906
Another application of this that is not
related to doing something remotely,

259
00:16:50,906 --> 00:16:53,148
is something like
an encrypted file system

260
00:16:53,148 --> 00:16:58,140
You may have a file system that
every time you try to write to a file,

261
00:16:58,140 --> 00:17:02,362
you will try to write it in plain text,
but it will capture that operation

262
00:17:02,362 --> 00:17:04,607
it will encrypt on the go,

263
00:17:04,607 --> 00:17:07,699
and then it will save it as a
regular file in your file system,

264
00:17:07,699 --> 00:17:09,559
but that's actually encrypted

265
00:17:09,559 --> 00:17:15,058
And once you dismount the file system,
once you remove the FUSE connection,

266
00:17:15,058 --> 00:17:19,311
all that is left in your computer are just
regular files that are encrypted

267
00:17:19,620 --> 00:17:27,401
The last topic I wanna cover is backups
and some good practices about them

268
00:17:28,630 --> 00:17:34,129
The main idea is that for every
file that you care about,

269
00:17:34,129 --> 00:17:36,300
if you don't have a backup of that file,

270
00:17:36,300 --> 00:17:38,500
if you don't have a backup
stored of that file,

271
00:17:38,500 --> 00:17:40,470
you can pretty much lose it at any moment

272
00:17:40,470 --> 00:17:42,803
There are many different failure scenarios

273
00:17:42,803 --> 00:17:49,170
One of them is just hardware failure.
So your hard drive can fail at any moment

274
00:17:49,170 --> 00:17:54,510
So if you are just making a copy of your
files in the same drive, that's not useful

275
00:17:54,510 --> 00:17:57,850
If your hard drive fails,
the files are gone

276
00:17:57,850 --> 00:18:02,112
The same goes if you have an external
drive where you are making a copy,

277
00:18:02,112 --> 00:18:06,813
but if you are storing everything in your
home, and your home burns down...

278
00:18:07,024 --> 00:18:11,639
Which yes, it's unlikely, but
if it happens you just lost all your data

279
00:18:11,639 --> 00:18:15,779
So you'd have some sort of
off-site backup for having this solution

280
00:18:17,170 --> 00:18:19,679
Another thing to take into account is that

281
00:18:19,679 --> 00:18:22,949
synchronization or mirroring options
are not backups

282
00:18:22,949 --> 00:18:25,943
So Google Drive, Dropbox
that I was mentioning,

283
00:18:25,943 --> 00:18:30,250
they will just propagate whatever is
happening in your computer

284
00:18:30,250 --> 00:18:33,080
This goes also for
hardware mirroring, like RAID

285
00:18:33,080 --> 00:18:34,765
They are just making a copy

286
00:18:34,765 --> 00:18:36,840
If you accidentally delete a file,

287
00:18:36,840 --> 00:18:42,060
or someone maliciously deletes your files,
or encrypts them using some ransomware,

288
00:18:42,060 --> 00:18:46,600
then you might have a copy, but
you have a copy of the same useless data

289
00:18:46,600 --> 00:18:50,857
You actually have to have a solution of
how you're running your backups

290
00:18:50,857 --> 00:18:53,048
And you should be asking yourself

291
00:18:53,048 --> 00:19:01,280
what actually someone needs to know/have
about you in order to delete all your data

292
00:19:01,280 --> 00:19:09,358
And we have linked different softwares
in the notes about how to do this

293
00:19:09,358 --> 00:19:13,476
The last thing I want to mention about
backups is that a lot of the time

294
00:19:13,476 --> 00:19:16,670
when you think about backups, you just
think about the local files

295
00:19:16,670 --> 00:19:21,070
and like, all my photos and my tax return,
and how can I make a backup of that?

296
00:19:21,070 --> 00:19:25,249
But increasingly in the modern age,
there are more and more web applications

297
00:19:25,249 --> 00:19:28,729
and a lot of data might only live
in some cloud provider,

298
00:19:28,729 --> 00:19:30,570
like for example if you have Webmail,

299
00:19:30,570 --> 00:19:32,868
and you're not synchronizing it
to your computer,

300
00:19:32,868 --> 00:19:36,740
It's only living in
that provider's servers

301
00:19:36,746 --> 00:19:38,994
And if you don't have a copy for that

302
00:19:38,994 --> 00:19:41,740
and for some reason you
lose access to that account

303
00:19:41,740 --> 00:19:44,460
because you forgot your password,
you got hacked,

304
00:19:44,460 --> 00:19:49,120
they think you have violated the
terms of service... All that data is gone

305
00:19:49,120 --> 00:19:53,210
So you should look into some tools
that people have developed

306
00:19:53,210 --> 00:19:57,640
for making offline copies of all that data

307
00:19:57,640 --> 00:20:00,520
so you can make regular backups of that

308
00:20:00,780 --> 00:20:04,900
And that kind of ends this
short section on backups

309
00:20:04,900 --> 00:20:07,100
Any questions so far?

310
00:20:09,620 --> 00:20:12,560
(student) When you said that
a hard drive can fail at any time

311
00:20:12,560 --> 00:20:15,460
Is there a reason for it to fail?

312
00:20:15,460 --> 00:20:16,960
[unintelligible]

313
00:20:16,960 --> 00:20:21,220
Like if I have my external hard drive
sitting at my parents house or something

314
00:20:21,220 --> 00:20:26,280
And my computer here, is that enough?
Or any drive can just fail?

315
00:20:26,420 --> 00:20:30,370
(Jose) Any drive can fail at any moment.
Like we don't...

316
00:20:30,370 --> 00:20:34,006
Different media have
different rates of failure

317
00:20:34,006 --> 00:20:36,680
and there are really good
statistics online

318
00:20:36,680 --> 00:20:37,755
So for example,

319
00:20:37,755 --> 00:20:42,370
spinning hard drives have a higher
rate of failure than SSDs for example,

320
00:20:42,370 --> 00:20:43,920
like Solid-State Drives

321
00:20:43,920 --> 00:20:47,400
And what's another case?
Or like CD drives for example

322
00:20:48,280 --> 00:20:54,360
But if you drop a hard drive it's a
higher rate of that failing, of course

323
00:20:54,360 --> 00:20:59,784
But in general we don't really have
an end-all solution

324
00:20:59,784 --> 00:21:02,140
for saying "this media is not gonna fail"

325
00:21:02,140 --> 00:21:08,352
Like pretty much, like SD cards, SSDs,
hard drives, CDs, degrade with time

326
00:21:08,360 --> 00:21:12,483
Pretty much every data is kind of
bound to this degradation

327
00:21:12,483 --> 00:21:16,149
or like this fact that it could
be lost at any moment

328
00:21:16,149 --> 00:21:19,156
And you should also know that
data can become corrupted,

329
00:21:19,156 --> 00:21:23,869
your disk might look like it's okay,
but maybe some files were corrupted

330
00:21:23,869 --> 00:21:26,230
and something like
synchronization techniques,

331
00:21:26,230 --> 00:21:30,450
like Google Drive or Dropbox,
will propagate that corruption

332
00:21:30,450 --> 00:21:35,000
And by the time that you realize that
things have gone wrong it's maybe too late

333
00:21:39,580 --> 00:21:46,480
(Jon) Alright, we're gonna continue this trend of jumping between random topics and talk about APIs

334
00:21:46,660 --> 00:21:52,580
So so far we've really been talking about how do you do things more efficiently locally on your computer?

335
00:21:52,640 --> 00:21:58,860
Like I want to accomplish this task more efficiently. How do I like configure my editor? How do I use my shell?

336
00:21:58,900 --> 00:22:03,520
But one thing you should realize is that very often you can integrate with the outside world as well

337
00:22:03,660 --> 00:22:08,040
Most services that you interact with in your day-to-day provide some kind of API

338
00:22:08,160 --> 00:22:12,320
For you to interact with the data that they store or the services that they provide

339
00:22:12,400 --> 00:22:15,400
And usually those APIs are pretty well documented

340
00:22:15,500 --> 00:22:22,080
If you looked at the APIs for things like Facebook, or Twitter, or Google Drive, or Gmail

341
00:22:22,140 --> 00:22:28,920
Many of these have interfaces that you can interact with in order to use those services from your local machine

342
00:22:29,140 --> 00:22:34,200
What's really neat is that you can often combine this with some of the stuff that we've talked about in lecture so far

343
00:22:34,280 --> 00:22:38,160
Like for example in the Data Wrangling lecture, we looked at how you can create

344
00:22:38,220 --> 00:22:43,080
These pipelines to extract data from some source that has a different format than you expected

345
00:22:43,600 --> 00:22:47,720
So for instance, the US government has a free service

346
00:22:47,840 --> 00:22:51,880
Where you can request the weather forecast for any given location in the US

347
00:22:52,200 --> 00:22:55,620
And what you do is there is a URL that you request

348
00:22:55,820 --> 00:23:01,240
And if you set the right parameters in that URL and then just fetch it, what you get back is JSON

349
00:23:01,240 --> 00:23:05,720
Which is sort of a well-defined data format that you can then parse

350
00:23:05,760 --> 00:23:10,840
And you can extract things like your 14-day weather forecast

351
00:23:10,960 --> 00:23:14,940
And maybe you then pipe that into your shell and produce some kind of like

352
00:23:15,020 --> 00:23:17,940
Handy alias in your terminal that's just gonna print...

353
00:23:18,020 --> 00:23:23,460
Of some handy reference for the next 14 days of weather in whatever location you were in

354
00:23:23,600 --> 00:23:27,500
These are things you can pretty easily construct and there's some notes...

355
00:23:27,580 --> 00:23:31,960
There are some notes in the notes about how you might go about this

356
00:23:32,140 --> 00:23:37,840
In general when you interact with these APIs, you're going to be using URLs of one form or another

357
00:23:37,880 --> 00:23:41,200
And the exact format varies from service to service

358
00:23:41,260 --> 00:23:44,540
But in general the URL is going to contain some set of parameters

359
00:23:44,600 --> 00:23:49,900
But ultimately you're just gonna issue a web request to them and you're gonna get data back in some format

360
00:23:50,000 --> 00:23:54,820
One command you should be aware of for interacting with these types of things is one called "curl"

361
00:23:54,920 --> 00:23:58,560
So curl is a program that you invoke, you give it a URL

362
00:23:58,620 --> 00:24:01,260
And it just fetches that URL and gives you back the response

363
00:24:01,260 --> 00:24:03,980
What you do with that response is entirely up to you

364
00:24:04,080 --> 00:24:08,660
Maybe you pipe it through a program like "gq". Sorry, "jq"

365
00:24:08,760 --> 00:24:15,000
So jq is a JSON query tool that lets you take in data that's formatted as JSON

366
00:24:15,040 --> 00:24:19,660
And then write a query over it to extract data that you're interested in

367
00:24:19,840 --> 00:24:24,640
And this is one of the ways in which you can layer these tools to extract the data that you're interested in

368
00:24:25,360 --> 00:24:30,360
Some of these services also require that you authenticate in one way or another

369
00:24:30,460 --> 00:24:33,420
Like for example if you want to interact with a Facebook API

370
00:24:33,470 --> 00:24:38,860
You need to have some authenticated token that proves who you are as far as Facebook is concerned

371
00:24:39,040 --> 00:24:43,740
Otherwise, they can't say whether you're allowed to say, create a post as a given user

372
00:24:43,860 --> 00:24:47,580
Very often these things are gonna use something called "OAuth", although not always

373
00:24:47,780 --> 00:24:51,700
And you should look at the documentation for every service you care about

374
00:24:51,740 --> 00:24:55,740
In general though you will get some kind of secret token back from the service

375
00:24:55,840 --> 00:24:58,620
That you have to include in the requests that you make to them

376
00:24:58,720 --> 00:25:04,100
Either in the URL or in additional sort of web headers, which you can also send with curl

377
00:25:04,220 --> 00:25:10,580
Keep in mind though that these tokens are secret. They are another representation of your user

378
00:25:10,680 --> 00:25:14,120
And anyone who gets their hand on them can basically pretend to be you

379
00:25:14,180 --> 00:25:17,420
They can do whatever you can do with that token

380
00:25:17,420 --> 00:25:21,840
So keep this in mind, don't stick them in your "dotfiles" and then push them onto GitHub

381
00:25:21,940 --> 00:25:26,360
That will land you in trouble. You should think of them as a password

382
00:25:26,480 --> 00:25:29,380
There are also really neat tools online for integrating services

383
00:25:29,380 --> 00:25:32,540
So there's a service called "If This Then That"

384
00:25:32,660 --> 00:25:38,580
Which basically provides integrations with a bunch of different services and lets you chain them together

385
00:25:38,660 --> 00:25:41,960
And then also access them partially locally if you wish

386
00:25:42,000 --> 00:25:44,020
This is something that's worth looking into

387
00:25:44,040 --> 00:25:49,220
If there's a particular service you would like to interact with in a more efficient manner

388
00:25:49,520 --> 00:25:52,680
Any questions about APIs?

389
00:25:55,160 --> 00:25:56,450
Alright

390
00:25:56,450 --> 00:25:59,940
Switching gears entirely, let's talk about command-line arguments

391
00:26:00,000 --> 00:26:04,080
So command-line tools. There are a lot of them

392
00:26:04,080 --> 00:26:08,100
And most of them take different arguments because they do different things

393
00:26:08,200 --> 00:26:13,400
We've talked about looking at man pages for commands and that will tell you how this particular command works

394
00:26:13,400 --> 00:26:16,800
What kind of flags and options you might give to it

395
00:26:16,880 --> 00:26:19,420
And what it actually does when you invoke it

396
00:26:19,580 --> 00:26:23,400
But there are some common themes that are useful to know about

397
00:26:23,400 --> 00:26:29,400
Either in arguments that many programs take, or just concepts the many of them apply

398
00:26:29,620 --> 00:26:35,100
The first of these we already mentioned a little bit in the lecture on command-line environments

399
00:26:35,120 --> 00:26:40,700
Which is the "--help" flag. Very often you can pass this to a program and instead of running,

400
00:26:40,700 --> 00:26:46,840
It will just print out information about how you can run this program, often in a very short, condensed way

401
00:26:47,900 --> 00:26:55,040
A similar one is the "--version" flag, which just prints the version of the software you're using

402
00:26:55,080 --> 00:26:58,000
This can be really handy if you're doing something like filing a bug report,

403
00:26:58,010 --> 00:27:00,010
Which Anish is going to talk a little bit about later

404
00:27:00,060 --> 00:27:04,460
And you want to report what version you're running on in case the bug has been fixed since

405
00:27:04,720 --> 00:27:10,460
Often you can also do "-V" and that means the same as version. But again, check the man page

406
00:27:11,500 --> 00:27:16,680
There's also "--verbose", or "-v" often

407
00:27:16,760 --> 00:27:20,760
Which is a flag that lets you increase the output of the program

408
00:27:21,000 --> 00:27:26,260
It makes the program print more about what it is doing. And very often you can repeat this flag

409
00:27:26,320 --> 00:27:31,340
So you can do like "-vvvvv" in order to get more information from that tool

410
00:27:31,460 --> 00:27:34,740
And this can be especially useful if you're trying to debug a problem

411
00:27:34,760 --> 00:27:39,140
If you're like running "rsync" and you want to know why did it decide to copy this file?

412
00:27:39,140 --> 00:27:44,120
Or why did it decide not to copy this file? That kind of debug output can be useful

413
00:27:44,200 --> 00:27:47,960
And often there's a sort of an inverse flag called "quiet" or "silent"

414
00:27:48,000 --> 00:27:52,880
Which means that the tool will not print anything unless it was like an error

415
00:27:53,000 --> 00:27:55,140
Anything else, it will stay quiet about

416
00:27:56,120 --> 00:27:59,420
Many tools, especially those that do destructive actions

417
00:27:59,480 --> 00:28:03,440
Or some kind of action that you cannot undo

418
00:28:03,520 --> 00:28:06,320
Provide what's known as a "dry run flag"

419
00:28:06,600 --> 00:28:09,800
Exactly how this is represented in the command line varies from tool to tool

420
00:28:09,860 --> 00:28:16,880
But essentially what this dry run mode will do is it will run the tool, but it will not actually make any changes

421
00:28:17,040 --> 00:28:23,020
Instead it will just inform you of what it would have done if you hadn't run it with dry run

422
00:28:23,920 --> 00:28:30,400
Many of these tools also have an interactive mode. So for example, th "rm" and "mv" tools both do

423
00:28:30,540 --> 00:28:33,200
Often just "-i", although not always

424
00:28:33,320 --> 00:28:35,660
When you run a tool in interactive mode

425
00:28:35,660 --> 00:28:40,340
It will usually prompt you whenever it's about to do an action that you can't undo

426
00:28:40,460 --> 00:28:44,380
And it will sort of prompt you for a confirmation that it should actually go ahead

427
00:28:46,700 --> 00:28:50,100
When we're talking about destructive tools

428
00:28:50,100 --> 00:28:52,700
Many of them are non-recursive by default

429
00:28:52,780 --> 00:28:57,000
If you try to remove a directory or you try to operate on a full directory

430
00:28:57,220 --> 00:29:00,260
They will not continue into the files inside of that directory

431
00:29:00,480 --> 00:29:05,180
The reason being you might accidentally like, remove your entire hard drive and that seems bad

432
00:29:05,560 --> 00:29:08,100
Therefore for many of these tools, they have a "recurse" flag

433
00:29:08,220 --> 00:29:10,680
Often "-r", but again not always

434
00:29:10,780 --> 00:29:17,180
Which lets them traverse down into the tree to go deeper, but you need to opt-in to this behavior

435
00:29:17,180 --> 00:29:21,760
So this is for example, the case for "rm". This is also the case for "cp"

436
00:29:23,660 --> 00:29:28,220
In many tools, when they ask you to give a file name or a path

437
00:29:28,260 --> 00:29:31,440
And we talked about this a little bit in the data wrangling lecture

438
00:29:31,440 --> 00:29:34,720
Instead of giving a file name, you can often just give a dash

439
00:29:34,780 --> 00:29:40,000
Just a single "-", and what that means is standard input or standard output

440
00:29:40,040 --> 00:29:43,280
Depending on whether that argument is an input file or an output file

441
00:29:43,380 --> 00:29:48,860
This is handy to know about if you're trying to construct those kinds of data wrangling pipelines that we've talked about before

442
00:29:49,100 --> 00:29:54,860
Many tools will also default to using standard in or standard out if you don't give any file name at all

443
00:29:57,400 --> 00:30:03,480
Sometimes you want to pass something that looks like a flag or an option to a command

444
00:30:03,620 --> 00:30:07,120
But you don't actually want it to be interpreted as a flag or an option

445
00:30:07,180 --> 00:30:14,700
Consider for example, if you wanted to remove a file called "-i", what would you do?

446
00:30:14,820 --> 00:30:17,520
Right, if you write the following command...

447
00:30:19,460 --> 00:30:22,020
"rm -i"

448
00:30:22,160 --> 00:30:28,860
Well, "-i" is a flag to "rm", so "rm" would, when you run this command say

449
00:30:28,940 --> 00:30:32,620
Tell me what file to remove, you haven't given me a file

450
00:30:32,880 --> 00:30:35,580
And it's because it interprets this as a flag

451
00:30:36,140 --> 00:30:38,500
Similarly, if you do something like...

452
00:30:38,780 --> 00:30:48,760
"ssh some machine, some command, and let's say, dash r"

453
00:30:49,740 --> 00:30:56,400
So this is saying - run command "foo" on this machine over SSH, and I wanna pass "-r" to "foo"

454
00:30:57,180 --> 00:31:02,460
Well, the way that both of these are gonna get interpreted is that these are flags

455
00:31:02,580 --> 00:31:07,180
Or in this case, this is a flag. But to this command

456
00:31:07,960 --> 00:31:10,540
Which is probably not what you expected

457
00:31:10,920 --> 00:31:14,500
Actually in the case of SSH, it has some weird special behavior for some of these

458
00:31:14,649 --> 00:31:18,089
But often if you want something to not be interpreted as a flag

459
00:31:18,090 --> 00:31:22,439
There's a very simple way to opt-out of that, and that is using double dash

460
00:31:22,840 --> 00:31:26,020
If you use double dash, what you tell the command

461
00:31:26,120 --> 00:31:30,500
Is that verything following this, you should not interpret

462
00:31:31,000 --> 00:31:36,479
So it will not be considered a flag or an option. In the case of "rm" you can do this

463
00:31:36,820 --> 00:31:40,480
And now "rm" you will see that the first argument is a "--"

464
00:31:40,940 --> 00:31:44,520
And then it will keep reading arguments, but it will not interpret them as flags

465
00:31:44,820 --> 00:31:50,720
So when it gets the "-i", it will not interpret it as the dash i flag, but just as an argument dash i

466
00:31:51,520 --> 00:31:58,580
Similarly, for SSH you can do this to indicate that these are both positional arguments

467
00:31:58,700 --> 00:32:04,020
They are not flags or options and you should not interpret things that start with a dash

468
00:32:08,720 --> 00:32:12,920
(student) But if you do like "--version", it's not gonna trigger that?

469
00:32:13,180 --> 00:32:17,940
(Jon) No, so this is a " -- ", with a space on both sides

470
00:32:20,220 --> 00:32:25,040
Any questions about any of this sort of command line conventions business?

471
00:32:28,840 --> 00:32:30,920
Alright then let's talk about window managers

472
00:32:31,809 --> 00:32:35,428
So most of you are used to some kind of drag-and-drop window manager

473
00:32:36,039 --> 00:32:42,599
If you're running Windows, or Mac OS, or Ubuntu - what comes with the machine is like, there are windows and

474
00:32:42,600 --> 00:32:47,100
They overlap partially on screen and you can like drag-and-drop and move them around and resize them and stuff

475
00:32:47,820 --> 00:32:51,120
And that works fine

476
00:32:51,160 --> 00:32:56,200
But it is not the only way to manage windows on your computer it turns out

477
00:32:56,740 --> 00:33:02,309
So what you are used to is something called a floating window manager, but not all window managers are floating

478
00:33:03,070 --> 00:33:09,810
Often you can opt-in to other types of window managers that have different behavior for how they arrange your desktop

479
00:33:10,600 --> 00:33:13,540
A common alternative is a tiling window manager

480
00:33:13,640 --> 00:33:17,500
So in a tiling window manager, rather than having floating windows,

481
00:33:17,760 --> 00:33:23,840
Everything is set up into a tiled layout. When you start a program, its window is maximized

482
00:33:24,180 --> 00:33:28,400
If you start another program, the original window shrinks in size

483
00:33:28,400 --> 00:33:32,280
And then the new window takes up some subset of the total desktop space

484
00:33:32,460 --> 00:33:36,660
At no time is your desktop background visible unless you have no programs open

485
00:33:37,150 --> 00:33:42,150
All of the programs you have open on any given desktop are going to share that space

486
00:33:42,700 --> 00:33:48,220
This looks a little bit like tmux panes, like we talked about earlier, where you can sort of split them in various directions

487
00:33:49,140 --> 00:33:54,640
And one of the reasons why this is handy is it means you basically never need to go to your mouse

488
00:33:54,940 --> 00:33:58,940
In order to move between windows, there are keyboard shortcuts to move to different windows

489
00:33:58,940 --> 00:34:02,960
There are keyboard shortcuts for resizing the windows or swapping them around on screen

490
00:34:03,100 --> 00:34:07,380
And this turns out to be a pretty efficient way to manage windows in your computer

491
00:34:10,149 --> 00:34:15,059
Well, I won't go into too much of detail of what kind of window managers you might use

492
00:34:15,060 --> 00:34:19,320
Just know that these exist out there and they're worth giving a look. They can be a lot more efficient to work with

493
00:34:20,460 --> 00:34:23,100
Questions about window managers?

494
00:34:24,420 --> 00:34:28,120
All right, VPNs. Totally related to the previous topic

495
00:34:28,440 --> 00:34:32,500
So VPNs are like all the rage these days and this makes me really sad

496
00:34:32,700 --> 00:34:35,660
It's not clear that VPNs are all the rage for any good reason

497
00:34:36,180 --> 00:34:39,840
Because you should be aware of what a VPN does and does not get you

498
00:34:40,480 --> 00:34:47,700
A VPN, in the best case, is really just a way for you to change your internet service provider

499
00:34:48,040 --> 00:34:54,420
It's a way for you to make traffic on the Internet seem like it's coming from somewhere else than where you actually are

500
00:34:57,480 --> 00:35:00,780
While that might seem attractive for certain purposes,

501
00:35:00,780 --> 00:35:04,040
It's a little unclear what it buys you in terms of security

502
00:35:04,060 --> 00:35:07,480
Because all you're really doing is shifting who you are trusting

503
00:35:07,600 --> 00:35:11,220
Rather than trusting who is providing your current internet service,

504
00:35:11,240 --> 00:35:16,340
You're trusting that whatever business is giving you that VPN service...

505
00:35:16,420 --> 00:35:20,240
You're trusting that they, first of all, have set up this VPN business correctly,

506
00:35:20,240 --> 00:35:23,820
But also that they are not tracking what you are doing

507
00:35:24,040 --> 00:35:28,780
And it's not clear whether that change of trust is actually worth it

508
00:35:28,860 --> 00:35:33,560
If you're sitting at some like dodgy public Wi-Fi network then maybe

509
00:35:33,640 --> 00:35:36,460
But if you're sitting at MIT, it's not clear

510
00:35:36,460 --> 00:35:40,700
Do you trust your VPN provider more than you trust MIT's IS&T?

511
00:35:40,700 --> 00:35:44,100
Or maybe you do, but that is a decision that you need to make

512
00:35:44,120 --> 00:35:47,400
About what you trust, who you trust and why?

513
00:35:47,740 --> 00:35:51,680
You should also know that much of your traffic,

514
00:35:51,720 --> 00:35:57,280
Especially the stuff that's on a sensitive nature on the Internet, is already encrypted

515
00:35:57,300 --> 00:36:03,520
Whether that's sort of HTTPS or other protocols that use something like TLS, a lot of the sensitive data is already

516
00:36:03,620 --> 00:36:09,580
Sent over encrypted channels and it doesn't really matter who your network provider is if you're on a dodgy Wi-Fi network

517
00:36:09,580 --> 00:36:15,220
The stuff that matters is probably encrypted anyway. Might not be, but if it's not

518
00:36:15,280 --> 00:36:20,580
Then your VPN provider can also see it in plain text just as much as whoever's hosting this dodgy Wi-Fi network

519
00:36:21,120 --> 00:36:25,780
And notice that I said "in the best case above"

520
00:36:25,780 --> 00:36:31,820
There are VPN providers who have been shown to be malicious, that do logging of all your traffic,

521
00:36:31,880 --> 00:36:38,380
That sell that traffic to third parties. There are VPN providers that have forgotten to enable encryption on the VPN

522
00:36:38,480 --> 00:36:41,640
All of these are real problems

523
00:36:41,800 --> 00:36:48,800
And so you should think very carefully about whether a VPN actually serves any good purpose for you

524
00:36:48,980 --> 00:36:52,620
Questions about VPNs? Yes?

525
00:36:52,740 --> 00:36:56,100
(student) So I have a question about public Wi-Fi networks, because

526
00:36:56,140 --> 00:37:02,800
The traffic from your computer to the router isn't encrypted between the computer and the router, right?

527
00:37:02,920 --> 00:37:06,880
Except for what normally is via HTTPS and [unintelligible]

528
00:37:06,980 --> 00:37:13,640
So then doesn't that mean that people could sniff out what domains I'm going to via the DNS request?

529
00:37:13,700 --> 00:37:15,700
(Jon) So, it's a very good question

530
00:37:15,720 --> 00:37:22,080
If you're on a public Wi-Fi network, then the traffic between you and the wireless access point is not encrypted

531
00:37:22,280 --> 00:37:27,180
At least it's not encrypted sort of on the outer layer, but it might be encrypted in like HTTPS, for example

532
00:37:27,360 --> 00:37:34,069
And it is totally true the people observing that Wi-Fi network will be able to see anything that is not encrypted

533
00:37:34,100 --> 00:37:39,760
But the solution to that is to encrypt all your traffic, rather than necessarily going through a VPN

534
00:37:39,860 --> 00:37:46,820
So one way to do this for example is to use DNS over TLS or DNS over HTTPS, which gives you

535
00:37:46,920 --> 00:37:51,540
A way to actually encrypt even information that might otherwise leak in plain text

536
00:37:51,960 --> 00:37:55,880
Rather than try to sort of trust some provider to do that for you

537
00:37:55,920 --> 00:38:02,450
Now that said, in some cases you might have a trusted institution that provides a VPN network for you

538
00:38:02,450 --> 00:38:09,080
So for example, MIT provides a VPN network for all MIT students and staff that you can sign up to use

539
00:38:09,120 --> 00:38:13,780
And in that case you probably trust MIT more than the other networks you might be on

540
00:38:13,780 --> 00:38:18,020
So it might be worth it, but it's something for you to think about

541
00:38:18,020 --> 00:38:22,500
(student) When you say you could encrypt it with, what was it, DNS, how would you do that?

542
00:38:24,180 --> 00:38:28,700
(Jon) So, DNS is the way that people turn domain names,

543
00:38:28,700 --> 00:38:32,300
Or your computer turns domain names into IP addresses to know what computer to connect to

544
00:38:32,460 --> 00:38:37,180
And that protocol by default is in plain text. There's nothing encrypted about it

545
00:38:37,200 --> 00:38:40,940
There are various ways to encrypt your DNS traffic

546
00:38:41,060 --> 00:38:43,940
Some of them are standardized and some of them are not

547
00:38:44,000 --> 00:38:49,200
I won't go into the exact mechanics here, but you should google it and look at some of the ways

548
00:38:50,100 --> 00:38:52,680
Okay. The last thing I want to talk about is Markdown

549
00:38:52,820 --> 00:38:59,280
So there is a high chance that some of you are going to write text over the remaining part of your life

550
00:38:59,480 --> 00:39:02,860
And you will want to mark up that text in various simple ways

551
00:39:03,030 --> 00:39:08,929
And one thing you could do is start up Word or use LaTeX or something like that to mark up your documents

552
00:39:08,930 --> 00:39:10,860
But that is a pretty heavy-handed approach

553
00:39:10,860 --> 00:39:13,320
Instead it would be nice if we could just sort of

554
00:39:13,400 --> 00:39:16,380
Write things the way we feel like they should be

555
00:39:16,400 --> 00:39:20,960
I don't know how to describe it in a better way but sort of the natural way where if you want something,

556
00:39:20,980 --> 00:39:25,280
If you want to put emphasis on a word you just put like stars around it or something, and then it just works

557
00:39:25,380 --> 00:39:31,760
Markdown is essentially that. It is a way to try to encode the way that we often write text somewhat naturally

558
00:39:31,860 --> 00:39:37,440
Into a markup language that lets you write things like bold text, links, lists, that sort of stuff

559
00:39:37,560 --> 00:39:41,540
In fact, all of the lecture notes for this class have been written using Markdown

560
00:39:41,720 --> 00:39:46,260
And Markdown is really very straightforward. The basic rules are in the notes

561
00:39:46,300 --> 00:39:53,660
But the basic things you need to know is, in Markdown if you put stars

562
00:39:53,820 --> 00:40:00,480
Around a word, that word is emphasized. Or some sequence of words. If you put double stars

563
00:40:00,640 --> 00:40:04,760
That word is emphasized strongly, also known as bold

564
00:40:04,760 --> 00:40:11,080
There are various other things you could do, like if you put a dash before a line it is now a list

565
00:40:11,080 --> 00:40:14,180
And there's one list item and you can amend list items

566
00:40:14,240 --> 00:40:18,540
If you put "1." in front, or some other number, it becomes a numbered list

567
00:40:18,720 --> 00:40:25,000
If you put a pound sign in front of something it becomes a header, like some kind of title header

568
00:40:25,120 --> 00:40:30,260
If you put multiple of them, they become subheadings and you can keep adding more to these

569
00:40:30,640 --> 00:40:37,820
If you want to write code you can put a single backtick, followed by some code, followed by a backtick

570
00:40:37,880 --> 00:40:41,400
And now that is rendered in Monospaced Font

571
00:40:41,440 --> 00:40:46,920
If you want multiple lines of code, you do a triple backtick, and then code,

572
00:40:46,920 --> 00:40:50,960
And then maybe some more code and then triple backtick

573
00:40:51,240 --> 00:40:53,840
And in many cases like if you're on GitHub for example,

574
00:40:53,860 --> 00:40:58,440
You can even type the name of a language up here after the backticks, without a space

575
00:40:58,480 --> 00:41:01,520
And it will be syntax highlighted in the language of your choice

576
00:41:01,660 --> 00:41:07,900
This is a really handy thing that is supported in so many websites nowadays you might not even realize

577
00:41:07,900 --> 00:41:11,280
Like in Facebook Messenger you can use many of these

578
00:41:11,520 --> 00:41:14,400
They don't actually officially say they support Markdown anywhere

579
00:41:14,440 --> 00:41:17,700
But many of these things just like sort of happen to work and it's worth learning

580
00:41:17,780 --> 00:41:22,980
At least the basics and just start using them. You can do links and stuff as well, but that's already in the notes

581
00:41:23,860 --> 00:41:26,400
Any questions about Markdown?

582
00:41:27,940 --> 00:41:31,800
Right, Anish, you're up

583
00:41:34,700 --> 00:41:37,040
(Anish) Is my microphone working?

584
00:41:38,400 --> 00:41:41,160
Is this working? Can you guys hear me in the back?

585
00:41:42,380 --> 00:41:46,420
The light's green. Oh, I think I can hear it. Okay, great

586
00:41:48,260 --> 00:41:53,500
So, continuing with our theme of random topics that are all unrelated to the previous topics we've been talking about,

587
00:41:53,540 --> 00:41:56,840
The next thing we're going to talk about is a program called "Hammerspoon"

588
00:41:56,850 --> 00:42:00,469
Which is a tool for doing desktop automation on Mac OS

589
00:42:00,560 --> 00:42:05,160
And I think there's similar tools for Windows and Linux, a lot of the ideas can carry over

590
00:42:05,200 --> 00:42:08,640
You can google it if you want to figure out how to do these things on other platforms

591
00:42:08,640 --> 00:42:13,000
But basically Hammerspoon is a program that lets you write Lua scripts, scripts in a programming language

592
00:42:13,290 --> 00:42:15,830
That interact with various operating system functionality

593
00:42:15,960 --> 00:42:20,280
So you can write code that interacts with the keyboard and mouse and connects that to window management,

594
00:42:20,400 --> 00:42:25,980
To display management, the file system, battery and power management, Wi-Fi... All sorts of stuff

595
00:42:25,980 --> 00:42:30,560
Like basically all the things that your operating system manages, this tool lets you hook into those things

596
00:42:30,560 --> 00:42:34,310
And so it can let you do all sorts of neat things by writing just a couple lines of code

597
00:42:34,440 --> 00:42:39,540
Just some examples of cool things you can do with this tool are - you can bind hotkeys to move windows to

598
00:42:39,680 --> 00:42:45,240
Specific locations. So a demonstration of this is here. I have this window open. I press, in my particular setup,

599
00:42:45,280 --> 00:42:49,660
"Option+Command+Right" and this window moves to the right. "Option+Command+Left", this window moves to the left

600
00:42:49,740 --> 00:42:52,360
And I have a couple other shortcuts for moving things to various places

601
00:42:52,370 --> 00:42:57,499
And so I can kind of have an effect similar to tiling window managers that Jon was talking about earlier

602
00:42:57,680 --> 00:43:02,870
I can move windows to different parts of my screen to set things up in a particular way rather than have to use the mouse

603
00:43:02,870 --> 00:43:07,699
To position things where I want them to be and then like click and drag to resize windows to the right shape

604
00:43:07,800 --> 00:43:10,440
Just a keyboard shortcut can do the trick

605
00:43:10,460 --> 00:43:15,100
But this tool is not limited to just moving windows around and binding that to particularly keyboard shortcuts

606
00:43:15,100 --> 00:43:19,000
You can do other things like create a menu bar button with a bunch of different options

607
00:43:19,040 --> 00:43:22,580
And you can bind those different options to do different things. So in my particular case

608
00:43:22,640 --> 00:43:28,420
I've created this little menu and then I have a bunch of different things that I do reasonably frequently and clicking on these things

609
00:43:28,920 --> 00:43:33,440
Invokes a particular Lua function that I've written that interacts with this library. So for example

610
00:43:34,170 --> 00:43:38,780
Here, this "Rescue windows" thing is a particular thing where I often work with multiple displays

611
00:43:38,880 --> 00:43:44,660
And sometimes my operating system gets confused and I have some window that ends up off of my display and

612
00:43:44,740 --> 00:43:47,260
How do I how do I get this thing back?

613
00:43:47,520 --> 00:43:49,780
Well, that's what this... Whoops, not that

614
00:43:52,200 --> 00:43:56,600
That's what this "Rescue windows" thing does. It brings windows that are off the screen back onto the screen

615
00:43:57,620 --> 00:44:01,840
Another neat thing I have setup here is I have particular layouts that I've named

616
00:44:01,920 --> 00:44:03,640
So like a dorm, and a Lab and a Laptop layout

617
00:44:03,650 --> 00:44:04,910
So for example in my Lab

618
00:44:04,910 --> 00:44:08,660
I have this screen and I have another screen and have another screen besides that in a different orientation

619
00:44:08,760 --> 00:44:11,570
And I have this particular setup that I want where I want

620
00:44:11,780 --> 00:44:15,110
Maybe my terminal full screen on here, and my chat program over here and

621
00:44:15,270 --> 00:44:18,409
This screen split up into five segments with different programs in different places

622
00:44:18,980 --> 00:44:26,460
Here I can, when I show up to Lab I can just go here and click "Layout Lab" and it will invoke some code

623
00:44:26,760 --> 00:44:32,200
Which is not all that complicated like 10 lines of code describes a particular layout

624
00:44:32,280 --> 00:44:35,820
And it will instantiate that layout and put all the things where they need to go

625
00:44:36,100 --> 00:44:42,020
I could even in theory automate some of these things where my computer could figure out like I plug in a display and my computer knows

626
00:44:42,020 --> 00:44:46,370
Oh, this is the display that you have in your lab. Let me automatically instantiate this layout for you

627
00:44:46,370 --> 00:44:50,899
That's another thing you can do with Hammerspoon. And also other wacky things you can do like you can do things like

628
00:44:51,660 --> 00:44:55,040
It can detect your Wi-Fi network that you're on so it knows kind of where you are

629
00:44:55,040 --> 00:45:00,110
Maybe I have a different Wi-Fi network name at home versus in lab and I can do things like when I show up to lab

630
00:45:00,420 --> 00:45:04,040
Automatically mute my speakers. So I don't have like embarrassing music play out loud in my lab

631
00:45:05,280 --> 00:45:07,280
Another kind of cool example is

632
00:45:07,890 --> 00:45:09,440
So I have a Mac

633
00:45:09,440 --> 00:45:12,050
It has a fancy power supply and a lot of my friends have

634
00:45:12,240 --> 00:45:15,469
Computers that look the same as this and their power supply bricks look the same as mine and

635
00:45:15,720 --> 00:45:18,860
Sometimes I use their power brick because I forgot mine at home or something

636
00:45:19,110 --> 00:45:21,620
This tool can actually with like three or four lines of code

637
00:45:21,780 --> 00:45:27,469
Do neat things like show you a warning like it'll pop up a warning if you've accidentally taken your friend's power supply and plugged it

638
00:45:27,470 --> 00:45:29,470
Into your computer instead of using your own

639
00:45:29,550 --> 00:45:35,090
So at a high level this tool lets you run arbitrary Lua code and do things like bind it to menu buttons or key presses and

640
00:45:35,340 --> 00:45:39,169
It interacts with a large part of the operating system in order to do all sorts of cool stuff

641
00:45:39,600 --> 00:45:42,500
So that is Hammerspoon, any questions about that?

642
00:45:44,480 --> 00:45:46,520
Cool. Moving on to the next topic

643
00:45:47,490 --> 00:45:51,199
Completely unrelated to the previous one, it's booting and live USBs

644
00:45:52,080 --> 00:45:55,279
So the operating system on your computer, Windows or Mac OS

645
00:45:55,540 --> 00:46:00,790
or whatever you're used to is not exactly the first thing that runs on your machine when it turns on. There's something else that

646
00:46:00,790 --> 00:46:04,060
happens in the boot process before your operating system is loaded and

647
00:46:04,670 --> 00:46:08,470
There's some interesting stuff that you can do there. So you might have seen when you turn on your computer

648
00:46:08,470 --> 00:46:13,330
It says something like press F9 to configure the BIOS or press F12 to enter the boot menu

649
00:46:13,580 --> 00:46:17,860
The particular key sequences may depend on your machine and specific configuration

650
00:46:17,860 --> 00:46:22,809
But this is a general pattern and you can configure all sorts of interesting Hardware related stuff here

651
00:46:22,810 --> 00:46:23,950
So it's worth checking out

652
00:46:23,950 --> 00:46:29,410
And another thing you can do in this boot menu is you can have your computer start off from an alternate boot device

653
00:46:29,420 --> 00:46:34,180
so by default like my laptop here has a solid-state drive and it boots Mac OS when it turns on but I can also

654
00:46:34,250 --> 00:46:40,360
say plug in a USB flash drive that has an operating system installed on the flash drive and then at boot tell my computer to

655
00:46:40,360 --> 00:46:44,799
Boot from that flash drive instead of the built-in solid-state disk, and this is useful

656
00:46:44,800 --> 00:46:45,230
for example

657
00:46:45,230 --> 00:46:47,030
If I've broken my operating system

658
00:46:47,030 --> 00:46:52,239
Install and I want to do something like get the data off my computer or maybe want to fix the operating system

659
00:46:52,240 --> 00:46:55,479
Like maybe there's some critical files somewhere that I've deleted or I forgot my password

660
00:46:55,480 --> 00:46:59,350
I need to go like tweak some files in order to reset it booting from a live USB

661
00:47:00,020 --> 00:47:04,419
Booting from the separate operating system that's installed on a flash drive can let me do that like boot up my operating system

662
00:47:04,760 --> 00:47:07,300
Mount the hard disk that's on my current machine

663
00:47:07,300 --> 00:47:09,939
I'm working on and then go make some tweaks or copy data off of that

664
00:47:10,340 --> 00:47:16,059
And so live USBs are really useful and in the lecture notes, we've linked to a tool that can help you create them really easily

665
00:47:16,970 --> 00:47:19,780
Any questions about the boot process or live USBs?

666
00:47:22,460 --> 00:47:28,760
All right, next topic is virtual machines, Vagrant, Docker, the cloud and OpenStack

667
00:47:28,760 --> 00:47:31,620
I think last year we had an entire lecture on this topic this year

668
00:47:31,630 --> 00:47:33,969
We're going to condense it into like one minute

669
00:47:34,100 --> 00:47:41,229
So at a high level virtual machines and similar tools like containers let you emulate a whole computer system within your current machine

670
00:47:41,230 --> 00:47:44,439
so like I'm running Mac OS here, but within my Mac OS

671
00:47:45,380 --> 00:47:46,460
environment I can

672
00:47:46,460 --> 00:47:49,750
Simulate a machine that's running say Ubuntu or some other operating system

673
00:47:49,750 --> 00:47:55,179
And this is a nice way of creating an isolated environment for testing or for development or for exploration

674
00:47:55,700 --> 00:48:00,640
For example doing things like running potentially malicious code that should be isolated from my current environment. I

675
00:48:01,190 --> 00:48:06,940
Think the most common use case for programmers is to use virtual machines are containers to create development environments

676
00:48:06,940 --> 00:48:11,919
So I'm using Mac OS and I have some of services and stuff and libraries installed in my current machine

677
00:48:11,990 --> 00:48:13,280
But I might want for example

678
00:48:13,280 --> 00:48:18,310
I'm working on some web programming project and I want it to run on an

679
00:48:18,800 --> 00:48:20,090
Debian machine and I need

680
00:48:20,090 --> 00:48:24,489
Postgres, like a database server installed rather than install that all on my Mac OS machine

681
00:48:24,490 --> 00:48:28,869
I can instantiate this new machine just for the development purposes

682
00:48:30,140 --> 00:48:34,600
Now virtual machines, like that's a general concept. There are a bunch of programs that let you

683
00:48:35,270 --> 00:48:37,780
that are called virtual machine hypervisors that

684
00:48:38,330 --> 00:48:44,739
Support this functionality on your machine and then there are tools that let you script these hypervisors in order to specify machine

685
00:48:45,020 --> 00:48:51,040
Configurations like operating system and like what packages you want installed and what services you want installed in plain text

686
00:48:51,040 --> 00:48:56,739
And so this is an example on the screen right here. And this is done using a system called Vagrant.

687
00:48:57,859 --> 00:49:00,789
It's linked in the lecture notes. You can look into this if you're curious.

688
00:49:00,790 --> 00:49:05,439
So basically and the short plain text file I can specify okay. I want a machine that's running Debian

689
00:49:05,440 --> 00:49:09,220
It should have "postgres" and Redis and Python and stuff installed on it.

690
00:49:09,220 --> 00:49:10,880
And then, once I have this configuration,

691
00:49:10,880 --> 00:49:16,840
I can just type in "vagrant up", and what it does is, it reads this file and instantiates a new machine based on this

692
00:49:17,000 --> 00:49:22,659
configuration. And then, after I've done that I can do "vagrant ssh" to SSH into this virtual machine.

693
00:49:22,660 --> 00:49:25,510
So it's not a remote machine running on some other piece of hardware somewhere,

694
00:49:25,540 --> 00:49:30,790
It's just simulated on my own machine, but now here I have an Ubuntu box like I do now, let's be really stashe

695
00:49:30,820 --> 00:49:34,629
It's like or sorry not a bunch of debian here with all the things

696
00:49:34,629 --> 00:49:38,340
I want installed in here and I can do my development inside this isolated environment,

697
00:49:38,440 --> 00:49:42,240
and not, kind of, install all this junk on my MacOS machine.

698
00:49:42,980 --> 00:49:50,080
Now so that's vagrant there's similar tools like docker that are conceptually similar but use containers instead of virtual machines

699
00:49:50,119 --> 00:49:52,719
It's a distinction that we're not going to talk about in too much detail right now

700
00:49:53,690 --> 00:49:55,690
And so you can run VMs on your own computer

701
00:49:55,760 --> 00:49:58,570
but you can also rent virtual machines on the cloud and so

702
00:49:58,730 --> 00:50:02,859
It's a nice way to get instant access to like one example is you might want a computer

703
00:50:02,859 --> 00:50:07,809
That's always on always connected to the internet and has a public IP address. Like maybe you want to run a web server

704
00:50:07,810 --> 00:50:12,249
That's always available or you want to run some other service like say a slack bot or something like that

705
00:50:12,350 --> 00:50:14,000
Well a virtual machine

706
00:50:14,000 --> 00:50:19,239
rented on the cloud is one nice way to get that and these are pretty cheap for like a low capacity machine with a

707
00:50:19,430 --> 00:50:21,430
small CP and small amount of disk space and

708
00:50:22,240 --> 00:50:26,919
You might want to do is get access to a machine that's really powerful like with a lot of CPU cores or with a lot

709
00:50:26,920 --> 00:50:31,480
Of RAM or with a whole bunch of GPUs for some specific purpose like say you're doing deep learning or so

710
00:50:31,480 --> 00:50:33,440
You're doing some other sorts of sensitive computation

711
00:50:33,440 --> 00:50:35,380
Well, that's another thing you can do with VMs on the cloud

712
00:50:35,380 --> 00:50:41,140
And finally, you can get access to many more machines than you have physical access to. Like if I need a thousand machines

713
00:50:41,140 --> 00:50:46,540
But only for two minutes to do some very parallel tasks. That's something I can easily do with virtual machines and

714
00:50:47,840 --> 00:50:51,040
Popular services for doing this are things like Amazon AWS or Google cloud

715
00:50:51,320 --> 00:50:56,600
And if you're a member of MIT CSAIL, you can also get free VMs for research purposes using the CSAIL OpenStack

716
00:50:56,720 --> 00:50:59,000
And so this is also linked in the lecture notes

717
00:50:59,540 --> 00:51:03,459
So any questions about virtual machines, or Vagrant, Docker, or anything like that?

718
00:51:12,530 --> 00:51:17,229
So the question is when I say I'm running Ubuntu, or, actually, in this case, it's Debian

719
00:51:18,140 --> 00:51:22,240
when I'm running Ubuntu here, do I have, like, Ubuntu installed on my machine, or

720
00:51:22,940 --> 00:51:28,060
What exactly is going on here? So, basically, what Vagrant did for me when I type "vagrant up" is,

721
00:51:28,970 --> 00:51:35,169
because I've specified I want Debian here, it downloaded Debian from the internet, like, set up a disk image for this new machine,

722
00:51:35,510 --> 00:51:38,919
installed Debian into that disk image, then went to install these programs

723
00:51:39,080 --> 00:51:44,109
So, like, yes, this is on my computer. But, all of this is just in a particular file that's a disk image.

724
00:51:44,109 --> 00:51:48,489
And then, I'm emulating a machine that is basically completely isolated from my current machine.

725
00:51:48,490 --> 00:51:54,909
This is being run as a process on my current machine. Does that answer the question? Any other questions about VMs?

726
00:51:56,990 --> 00:51:58,100
Great.

727
00:51:58,100 --> 00:52:00,100
Next topic is also going to be a quick mention -

728
00:52:00,440 --> 00:52:01,760
So,

729
00:52:01,760 --> 00:52:05,020
a lot of you are programmers, and you're used to writing programs

730
00:52:05,180 --> 00:52:07,720
in a tool like Vim, or some other editor that you're comfortable with.

731
00:52:07,970 --> 00:52:13,000
Another thing that can be really neat to use for particular tasks, is something called a notebook programming environment.

732
00:52:13,000 --> 00:52:17,469
And this is a more interactive way of writing programs. Here on the screen, I have a demo.

733
00:52:17,470 --> 00:52:23,439
This is something called Jupiter notebook and it can be used for writing Python programs. I think they also support some other languages.

734
00:52:24,140 --> 00:52:26,350
And, basically, this is a nice way of doing interactive programming.

735
00:52:26,390 --> 00:52:29,920
So, normally, you're used to writing a big program in a file, or a collection of files,

736
00:52:29,920 --> 00:52:35,020
and once you're done writing it, you can just run the whole program. But, this lets you be a little bit more flexible and

737
00:52:35,619 --> 00:52:39,818
run little snippets of code at a time. Like, for example, I can break my program into these little pieces.

738
00:52:39,890 --> 00:52:45,969
It's just some random code I wrote. And, I can say, "execute this cell", and I press a particular key combination to execute the cell.

739
00:52:46,069 --> 00:52:50,798
But then, I can go back, and tweak my program a little bit. Like, say I want to have this be lowercase instead.

740
00:52:50,799 --> 00:52:54,609
Then, I can execute this cell, and then go and evaluate this thing, and

741
00:52:55,009 --> 00:52:59,949
this way I can, kind of, run little snippets of code, within a Python environment.

742
00:53:00,799 --> 00:53:05,649
And, it's a nice way of building up programs, piece by piece, rather than having to write everything at once.

743
00:53:05,650 --> 00:53:10,749
This is really useful for particular research purposes, like I think a lot of people use these for doing machine learning work, for example.

744
00:53:13,249 --> 00:53:17,829
Any questions about the idea of notebook programming environments? They're worth checking out.

745
00:53:24,349 --> 00:53:27,998
Oh, so the question is, "This looks like it's online, is there an offline version of Jupiter notebooks?"

746
00:53:27,999 --> 00:53:32,679
So, actually, this is the thing that runs in the browser, but it's running locally...

747
00:53:32,680 --> 00:53:35,499
So, I don't know if you can see it on the screen, because it's kind of small, but up here

748
00:53:35,499 --> 00:53:37,499
it says, "localhost:8888".

749
00:53:37,819 --> 00:53:44,349
Here, I have running on my own local machine, a Jupiter notebook. And, they've just built it

750
00:53:44,349 --> 00:53:46,349
so it runs within the web browser.

751
00:53:47,029 --> 00:53:49,029
That being said, there are also online

752
00:53:49,400 --> 00:53:54,729
Jupiter notebooks that you can use, where the Python kernel is actually running on some remote machine. You might want to do this

753
00:53:54,729 --> 00:53:55,960
for example, like on my laptop,

754
00:53:55,960 --> 00:54:00,159
I don't have a fancy GPU, but in my room, I have a machine with a fancy GPU. And so, when I'm doing machine learning

755
00:54:00,160 --> 00:54:07,119
work, I often SSH into that machine, run a Jupiter notebook on there, and then open up the interface, in my local web browser,

756
00:54:07,119 --> 00:54:09,848
so I have access to that powerful GPU running on my different machine.

757
00:54:11,900 --> 00:54:13,900
Any other questions?

758
00:54:16,279 --> 00:54:17,690
Great.

759
00:54:17,690 --> 00:54:19,690
The final thing we're going to talk about today is

760
00:54:20,299 --> 00:54:24,488
Github. So, we touched on this a little bit during the version control lecture.

761
00:54:24,489 --> 00:54:28,119
But, Github is one of the most popular platforms for open-source software development.

762
00:54:28,400 --> 00:54:35,440
It hosts source code. It hosts git repositories. But, they also have other tools for managing a project.

763
00:54:36,319 --> 00:54:42,129
And, like, a lot of the tools we've talked about in this class are hosted on Github. For example,

764
00:54:42,130 --> 00:54:45,609
like, Hammerspoon, the thing we just talked about, is developed on Github

765
00:54:46,400 --> 00:54:47,930
It's really easy to get started

766
00:54:47,930 --> 00:54:53,950
contributing to open-source projects on Github to help improve the tools that you use every day.

767
00:54:54,260 --> 00:54:57,520
There are two primary ways you can contribute to projects on Github.

768
00:54:59,330 --> 00:55:06,309
Let's open up some repository. We can actually go to the Github repository for the class website.

769
00:55:06,310 --> 00:55:08,310
So, this is an open-source software project.

770
00:55:09,050 --> 00:55:10,610
And...

771
00:55:10,610 --> 00:55:14,799
Let's zoom in a little bit. So the two ways you can contribute to projects on Github -

772
00:55:14,800 --> 00:55:17,410
The two main ways are through issues and pull requests

773
00:55:17,480 --> 00:55:23,949
One thing that's actually really helpful to developers, and also pretty lightweight and easy for users to do, is to report issues with a

774
00:55:23,950 --> 00:55:27,310
software project. Like, say you're using somebody's program and you encounter some bug...

775
00:55:27,560 --> 00:55:31,989
Writing a high quality issue is actually super helpful to developers and hopefully doesn't take you too much time.

776
00:55:31,990 --> 00:55:37,510
And so, you can go to here, like find the project on GitHub, go to the issues page, and click on new issue, and then

777
00:55:37,510 --> 00:55:41,979
write some high quality bug report. And then, hopefully the developer will respond and fix the issue for you.

778
00:55:42,160 --> 00:55:43,370
So, for example, for this class,

779
00:55:43,370 --> 00:55:49,120
like, one of the students in this class pointed out an issue with our lecture notes, and after she pointed it out,

780
00:55:49,120 --> 00:55:54,819
I said, okay, like, that looks like a reasonable thing. Let's fix it. And in this particular case, instead of fixing it myself,

781
00:55:54,820 --> 00:55:59,620
I actually asked this person, "Do they just want to fix it for me?" And so, that leads into the other thing

782
00:55:59,620 --> 00:56:06,009
I want to talk about: issues and pull requests. So, pull requests are the second way to contribute to projects on Github. And,

783
00:56:06,560 --> 00:56:09,880
this involves actually contributing code back to the project. And

784
00:56:10,550 --> 00:56:16,779
so if we look at the pull request for this particular project, you'll see that a bunch of people have submitted code changes.

785
00:56:17,510 --> 00:56:18,800
And,

786
00:56:18,800 --> 00:56:24,160
the process for doing so - so this is showing the difference, the patch, that this person submitted.

787
00:56:25,190 --> 00:56:30,310
Basically, the process for creating pull requests is a little bit more involved than submitting issues. Like you're not just submitting text:

788
00:56:30,310 --> 00:56:32,080
you're actually going to modify their source code.

789
00:56:32,080 --> 00:56:36,489
And so, we've linked to some guides that explain the process in a little bit more detail.

790
00:56:36,490 --> 00:56:41,860
But, at a high level, what you do is, you take the repository on Github, "fork" it, and then download it locally,

791
00:56:41,860 --> 00:56:43,040
so now have your own, local copy.

792
00:56:43,040 --> 00:56:48,130
Then, you can go and work on it, do some development work, and fix a bug or add a feature, and then, eventually, you

793
00:56:48,130 --> 00:56:51,969
send what's called a "pull request" back to the original developer. So, you say, 'Here. I've made some changes.

794
00:56:51,970 --> 00:56:54,640
Can you please incorporate them back into the original project?'

795
00:56:54,640 --> 00:57:00,429
And, after that point, what usually happens with these projects, is that the maintainer will go back and forth with you, giving you feedback on

796
00:57:00,430 --> 00:57:06,110
the changes you've proposed. And, eventually, once everybody's happy, they will "merge" in your changes, and they'll be available to everybody who uses the project.

797
00:57:07,500 --> 00:57:14,179
So, that is how you can contribute to projects on GitHub and make software better for everybody. And, so any questions about GitHub?

798
00:57:19,230 --> 00:57:24,350
Cool, okay. So, that is it for the topics for today. Any questions about the lecture overall?

799
00:57:25,920 --> 00:57:27,840
Great, okay, so before -

800
00:57:27,840 --> 00:57:33,350
before we finish, a quick description about tomorrow's lecture: so, today was all the topics we thought are interesting,

801
00:57:33,350 --> 00:57:34,140
we should talk about,

802
00:57:34,140 --> 00:57:38,900
tomorrow's lecture is gonna be about all the topics you think are interesting and that we should talk about. So, tomorrow is going to be

803
00:57:38,900 --> 00:57:40,940
a Q&A lecture. And, after today,

804
00:57:41,430 --> 00:57:43,170
uh, after the lecture, we'll submit,

805
00:57:43,170 --> 00:57:48,020
we'll email out a link where you can submit questions for us to answer. And so, please go and fill that out,

806
00:57:48,020 --> 00:57:50,020
otherwise, we won't have too much to talk about tomorrow.

807
00:57:51,180 --> 00:57:53,899
Great, so hopefully, see you tomorrow in our Q&A lecture.