1
00:00:00,000 --> 00:00:18,420
36C3 preroll music
2
00:00:18,420 --> 00:00:24,530
Herald-Angel: This Talk will be about… I
have to read this. Mathematical diseases
3
00:00:24,530 --> 00:00:28,600
in climate models and how to cure them.
And I don't have the slightest idea what
4
00:00:28,600 --> 00:00:33,850
these two guys are talking about now. And
when I asked them, they said, just tell
5
00:00:33,850 --> 00:00:40,039
the people it's about next generation
climate models and how to build them.
6
00:00:40,039 --> 00:00:45,789
Which is cool. Throw that on Twitter.
Please welcome Ali Ramadhan and Valentin
7
00:00:45,789 --> 00:00:47,229
Churavy.
8
00:00:47,229 --> 00:00:55,239
applause
9
00:00:55,239 --> 00:00:58,509
Ali Ramadhan: Can you guys hear us? Is
this OK?
10
00:00:58,509 --> 00:01:00,210
Valentin Churavy: I'll stand back...
Ramadhan: I'll stand back a little bit.
11
00:01:00,210 --> 00:01:05,030
OK, cool. Thank you. So if you guys saw
the last talk by karlab... karlabyrinth or
12
00:01:05,030 --> 00:01:08,530
something. So we're kind of expanding on
her talk a little bit. So she talked a lot
13
00:01:08,530 --> 00:01:11,209
about kind of uncertainties…
audio feedback from microphone
14
00:01:11,209 --> 00:01:15,670
uncertainties in climate models. And one
point that she did make was that most of
15
00:01:15,670 --> 00:01:18,410
the uncertainty actually comes from
humans. But there's a really huge
16
00:01:18,410 --> 00:01:22,070
uncertainty that also comes from… comes
from the models. So we're talking more
17
00:01:22,070 --> 00:01:26,170
about the model uncertainties, which is
kind of uncertainties because of unknown
18
00:01:26,170 --> 00:01:31,590
or missing physics, kind of how to cure
them. So it'll be kind of a weird talk. So
19
00:01:31,590 --> 00:01:35,119
I'll talk a little bit more about the
climate modeling part and then kind of how
20
00:01:35,119 --> 00:01:39,840
to cure them involves using new programing
languages. And that's where Valentin will
21
00:01:39,840 --> 00:01:44,810
talk about Julia. So we'll kind of just
start with maybe just giving kind of an
22
00:01:44,810 --> 00:01:50,080
idea of why it's so hard to model the
climate. So if you've… maybe you've seen
23
00:01:50,080 --> 00:01:54,550
images like this a lot where it's like a…
it's a satellite image basically of
24
00:01:54,550 --> 00:01:58,530
clouds. It's used for like weather
forecasting. But you can immediately see
25
00:01:58,530 --> 00:02:02,320
there's lots of, you know, lots of really
small clouds. So basically, if you want to
26
00:02:02,320 --> 00:02:06,250
build a climate model, you've got to be
able to resolve all the physics in these
27
00:02:06,250 --> 00:02:10,770
clouds. So you can actually zoom in a lot.
And see the clouds look pretty big over
28
00:02:10,770 --> 00:02:15,140
here. But if you zoom in on kind of
Central America, then you see even smaller
29
00:02:15,140 --> 00:02:21,509
clouds. And if you zoom in even more so,
so you zoom in on the Yucatan Peninsula,
30
00:02:21,509 --> 00:02:24,780
then you can see the clouds are really,
really small. So you're… there are maybe
31
00:02:24,780 --> 00:02:28,980
five smaller clouds, some some of the
clouds are, you know, a hundred meters or
32
00:02:28,980 --> 00:02:33,709
something. And as the last talk kind of
suggests that most climate models are…
33
00:02:33,709 --> 00:02:38,990
they resolve things of, you know, up to 50
kilometers. So anything smaller than 50
34
00:02:38,990 --> 00:02:43,670
kilometers, the climate model can't really
see. So you have to kind of take that… it
35
00:02:43,670 --> 00:02:46,290
kind of has to account for that because
clouds are important, and if you have more
36
00:02:46,290 --> 00:02:51,779
clouds, then that reflects some of the
heat out. So maybe you cool. But it also
37
00:02:51,779 --> 00:02:55,569
traps more of the heat in so maybe you
warm. And if you have more clouds, maybe
38
00:02:55,569 --> 00:03:00,130
you warm more. But if you have less
clouds, maybe you warm even more. So it's
39
00:03:00,130 --> 00:03:04,549
kind of unsure. We actually don't know if
clouds will make the climate warmer or if
40
00:03:04,549 --> 00:03:07,760
they'll make the climate cooler. So it's
important for your climate models to kind
41
00:03:07,760 --> 00:03:13,200
of resolve or see these little clouds. So
kind of where the mathematical disease
42
00:03:13,200 --> 00:03:16,940
comes in, is that you don't… we don't know
what equation to solve. We don't know
43
00:03:16,940 --> 00:03:21,399
exactly what physics to solve, to see, to
kind of resolve the effect of these little
44
00:03:21,399 --> 00:03:25,120
clouds. So it's kind of the the
mathematical disease. We don't know how to
45
00:03:25,120 --> 00:03:29,939
do it. So you instead use a… well, it's
called a parametrization, which is the
46
00:03:29,939 --> 00:03:33,120
mathematical disease. So in the
atmosphere, the big mathematical disease
47
00:03:33,120 --> 00:03:40,240
is clouds. But if you look at the ocean,
you kind of get a similar… You have also
48
00:03:40,240 --> 00:03:43,719
similar mathematical diseases. So if you
for example, this is model output. We
49
00:03:43,719 --> 00:03:48,830
don't have good satellite imagery of the
oceans. So if you if you look at, for
50
00:03:48,830 --> 00:03:53,400
example, model output from an ocean model,
high resolution ocean model here, it's
51
00:03:53,400 --> 00:03:58,159
kind of centered on the Pacific. So you
can kind of see Japan and China and the
52
00:03:58,159 --> 00:04:03,060
white kind of lines. Those are streamlines
or that the lines tell you where the water
53
00:04:03,060 --> 00:04:07,019
is going. So you could see a lot of kind
of straight lines. You see this curve here
54
00:04:07,019 --> 00:04:13,669
current off of Japan, but you see lots of
circles. So the circles are these eddies
55
00:04:13,669 --> 00:04:17,910
and they're kind of the turbulence of the
ocean. They move, they kind of stir and
56
00:04:17,910 --> 00:04:23,740
mix and transport a lot of salt or heat or
carbon or nutrients or… you know, marine
57
00:04:23,740 --> 00:04:28,270
life or anything. It's the main way the
ocean kind of moves heat from the equator
58
00:04:28,270 --> 00:04:32,970
to the pole. It kind of stirs things
around. So they're really important for
59
00:04:32,970 --> 00:04:38,480
kind of how carbon moves in the ocean, for
how the ocean heats up. And here they look
60
00:04:38,480 --> 00:04:42,259
pretty big. But again, you can zoom in and
you'll see lots of small scale structures.
61
00:04:42,259 --> 00:04:46,500
So we're going to switch to a different
model output and different colors. So here
62
00:04:46,500 --> 00:04:51,260
here's kind of the same area. So you see
Japan in the top left. But what's being
63
00:04:51,260 --> 00:04:55,720
plotted is vorticity. So you have to know
what that is. It's kind of a measure of
64
00:04:55,720 --> 00:05:00,030
how much the fluid or the water is
spinning. But the point is that you have
65
00:05:00,030 --> 00:05:05,340
lots of structure. So there's lots of, you
know, big circles, but there also lots of
66
00:05:05,340 --> 00:05:10,070
really little circles. And again, your
climate model can only see something like
67
00:05:10,070 --> 00:05:14,919
50 kilometers or 100 kilometers. But as
you can see here, there's lots of stuff
68
00:05:14,919 --> 00:05:18,860
that's much smaller than a hundred
kilometers. So if you superimpose kind of
69
00:05:18,860 --> 00:05:23,880
this this grid, maybe that's your climate
model grid. And, you know, basically for
70
00:05:23,880 --> 00:05:27,840
the climate model, every one of these
boxes is like one number. So you can't
71
00:05:27,840 --> 00:05:31,880
really see anything smaller than that. But
there's important dynamics and physics
72
00:05:31,880 --> 00:05:35,380
that happens in like 10 kilometers, which
is a lot smaller than what the climate
73
00:05:35,380 --> 00:05:39,470
model can see. And there's even important
physics that happens that like 100 meters
74
00:05:39,470 --> 00:05:44,290
or 200 meters. So if you want if you want
to, you know, what the climate will look
75
00:05:44,290 --> 00:05:50,909
like, we need to… we need to know about
the physics that happens at 200 meters. So
76
00:05:50,909 --> 00:05:55,020
to give an example of some of the physics
that happens at 10 kilometers, here's kind
77
00:05:55,020 --> 00:06:01,040
of a little animation where this kind of
explains why you get all these eddies or
78
00:06:01,040 --> 00:06:05,530
all the circles in the ocean. So a lot of
times you have, say, hot water, say, in
79
00:06:05,530 --> 00:06:10,520
the north. So the hot water here is all in
orange or yellow and you have a lot of
80
00:06:10,520 --> 00:06:15,340
cold water. So the cold water is in the
south and it's purple. And then once this…
81
00:06:15,340 --> 00:06:20,099
once you add rotation, you end up with
these eddies because what the hot water
82
00:06:20,099 --> 00:06:24,350
wants to do, the hot water is lighter,
it's less dense. So it actually wants to
83
00:06:24,350 --> 00:06:28,810
go on top of the cold water. So usually
have cold at the bottom, hot at the top.
84
00:06:28,810 --> 00:06:34,120
So you have heavy at the bottom and light
at the top. So when you add… without
85
00:06:34,120 --> 00:06:38,050
rotation, the hot water will just go on
top of the cold water. But when you have
86
00:06:38,050 --> 00:06:42,540
rotation, you end up… it kind of wants to
tip over. But it's also rotating. So you
87
00:06:42,540 --> 00:06:47,479
kind of get this beautiful swirling
patterns and these are kind of the same
88
00:06:47,479 --> 00:06:52,380
circular eddies that you see in the real
ocean. But this model here is like two
89
00:06:52,380 --> 00:06:55,139
hundred and fifty kilometers by five
hundred kilometers and it's like one
90
00:06:55,139 --> 00:06:59,710
kilometer deep. So you need a lot of
resolution to be able to resolve this
91
00:06:59,710 --> 00:07:04,750
stuff, but not… your climate model doesn't
have that much resolution. So some of the
92
00:07:04,750 --> 00:07:08,449
features here, like the sharp prints
between the cold and the hot water, your
93
00:07:08,449 --> 00:07:12,879
climate model might not see that. So maybe
if you if you don't resolve this properly,
94
00:07:12,879 --> 00:07:17,270
you get the mixing rate wrong or maybe
that the ocean is the wrong temperature or
95
00:07:17,270 --> 00:07:22,039
something. So it's kind of important to
resolve this stuff. Another one, the color
96
00:07:22,039 --> 00:07:26,170
scheme here is really bad. laughs I'm
sorry, but another one, for example, is
97
00:07:26,170 --> 00:07:32,389
here. Everything's under 100 meter, so
it's a cube of 100 meters on each side and
98
00:07:32,389 --> 00:07:37,340
you're starting with 20 degrees Celsius
water at the top. You have 19 degrees
99
00:07:37,340 --> 00:07:41,370
Celsius water at the bottom initially. So
it's kind of you're… as you go deeper in
100
00:07:41,370 --> 00:07:46,270
the ocean, the water gets colder. And then
if you can imagine, the ocean kind of at
101
00:07:46,270 --> 00:07:50,669
night, it's kind of cold. So the top is
being cooled and you end up with cold
102
00:07:50,669 --> 00:07:54,639
water on the top. The cold water wants to be
at the bottom. So it ends up sinking and you
103
00:07:54,639 --> 00:07:59,030
get all this convection going on. So this
is happening at a lot of places in the
104
00:07:59,030 --> 00:08:03,650
ocean. You get a lot of mixing at the top.
You get this kind of layer at the top of
105
00:08:03,650 --> 00:08:08,090
the ocean. It's kind of constant color,
constant temperature. So this mix layer is
106
00:08:08,090 --> 00:08:12,240
important for the ocean. So knowing how
deep that mix layer is and knowing how
107
00:08:12,240 --> 00:08:16,000
much of the water is being mixed is also
important for for climate. But as you can
108
00:08:16,000 --> 00:08:19,860
imagine, you know, if this happens on very
small scales. So you're climate model has
109
00:08:19,860 --> 00:08:24,870
to know something about what's happening
at this scale. So this isn't I guess the
110
00:08:24,870 --> 00:08:28,921
mathematical diseases in the ocean is, the
climate model cannot see this, so it has
111
00:08:28,921 --> 00:08:32,970
to do something else that's maybe
unphysical to resolve this stuff. And
112
00:08:32,970 --> 00:08:38,420
that's a mathematical disease, I guess.
Aside from the ocean and the atmosphere.
113
00:08:38,420 --> 00:08:41,820
You also have the same problem with sea
ice. So this is kind of just a satellite
114
00:08:41,820 --> 00:08:47,010
picture of where sea ice is forming off
the coast of Antarctica. So you get winds
115
00:08:47,010 --> 00:08:48,680
that kind of come off the
continent and they're
116
00:08:48,680 --> 00:08:51,080
kind of blowing all the
ice that's beeing formed
117
00:08:51,080 --> 00:08:53,900
away. So you get all these
little lines and streaks and they kind of
118
00:08:53,900 --> 00:08:58,390
merge into sea ice. But in this whole
picture is like 20 kilometers. So the
119
00:08:58,390 --> 00:09:01,950
climate model doesn't see this, but
somehow it has to represent all the
120
00:09:01,950 --> 00:09:08,561
physics. And you have kind of similar
things happening with soil moisture, land
121
00:09:08,561 --> 00:09:16,240
and dynamic vegetation, aerosols. So, you
know, these are kind of three places with
122
00:09:16,240 --> 00:09:21,270
pretty pictures. But see, if you look at
the atmosphere, so it's not just clouds.
123
00:09:21,270 --> 00:09:27,350
You also have aerosols, which are like
little particles, or sulfates that are
124
00:09:27,350 --> 00:09:31,680
important for kind of cloud formation and
maybe atmospheric chemistry. But again, we
125
00:09:31,680 --> 00:09:34,700
don't fully understand the physics of
these aerosols. So again, you have to kind
126
00:09:34,700 --> 00:09:40,270
of parametrize them. Same thing with kind
of convictions. You maybe your climate
127
00:09:40,270 --> 00:09:44,160
model doesn't resolve all the very deep
convection in the atmosphere so as to get
128
00:09:44,160 --> 00:09:48,190
all sides to parametrize it. So I guess
you have many kind of mathematical
129
00:09:48,190 --> 00:09:51,291
diseases in the atmosphere. So I'm not
expecting you to understand everything in
130
00:09:51,291 --> 00:09:55,110
this in this picture. But the idea is: The
atmosphere is complicated. There's no way
131
00:09:55,110 --> 00:09:59,740
a climate model is going to kind of, you
know, figure all this out by itself. And
132
00:09:59,740 --> 00:10:04,600
again, you could you could do something
similar for the ocean. So we can just show
133
00:10:04,600 --> 00:10:07,370
an image for like two little parts of
these. But the point is, you know, the
134
00:10:07,370 --> 00:10:11,490
ocean is not kind of just a bucket of
water standing there. So there's lots of
135
00:10:11,490 --> 00:10:15,180
stuff happening deep inside the ocean. And
some of it, we think is important for
136
00:10:15,180 --> 00:10:19,180
climate. Some of it we don't know. Some
might not be important. But again, a lot
137
00:10:19,180 --> 00:10:25,430
of this happens on very small spatial
scales. So we don't know or the climate
138
00:10:25,430 --> 00:10:30,441
model can't always resolve all this stuff.
And again, same thing with kind of sea
139
00:10:30,441 --> 00:10:34,370
ice. Lots of small scale stuff is
important for sea ice. And I think one
140
00:10:34,370 --> 00:10:37,770
person asked about kind of tipping points
and there's kind of two with like sea ice
141
00:10:37,770 --> 00:10:43,880
that are pretty important. One of them is
this CSL biofeedback. So if you have sea
142
00:10:43,880 --> 00:10:48,630
ice that melts. Now you have more ocean
and the ocean can absorb more heat. But
143
00:10:48,630 --> 00:10:51,980
now the earth is warmer, so it melts more
sea ice. So as soon as you kind of start
144
00:10:51,980 --> 00:10:55,670
melting sea ice, maybe you melt even more
sea ice and eventually you reach an earth
145
00:10:55,670 --> 00:11:00,020
with no sea ice. So there's kind of
research into that stuff going on, but
146
00:11:00,020 --> 00:11:04,410
it's a possible tipping point. Another one
is this kind of marine ice sheet,
147
00:11:04,410 --> 00:11:08,970
stability, instability at the bottom of
the ice shelf. So if you start melting
148
00:11:08,970 --> 00:11:13,500
water, if you start melting ice from the
bottom of the ice shelf, then we create
149
00:11:13,500 --> 00:11:17,530
kind of a larger area for more ice to
melt. So maybe once you start melting and
150
00:11:17,530 --> 00:11:20,940
increasing sea level, you just keep
melting more and more and increasing sea
151
00:11:20,940 --> 00:11:27,300
level even more. But again, it's kind of
hard to quantify these things on like 50
152
00:11:27,300 --> 00:11:34,910
or 100 year timescales because it all
happens on very small scales. So yeah, the
153
00:11:34,910 --> 00:11:39,760
point is there's lots of these kind of
parametrizations or mathematical diseases.
154
00:11:39,760 --> 00:11:43,800
And once you start adding them all up, you
end up with lots and lots of kind of
155
00:11:43,800 --> 00:11:48,180
parameters. So this is a really boring
table. But the point is, so this is like
156
00:11:48,180 --> 00:11:53,110
one parametrization for like vertical
mixing in the ocean. It's basically the
157
00:11:53,110 --> 00:11:57,140
process that I showed the rainbow color
movie about to see a climate model for
158
00:11:57,140 --> 00:12:02,060
that. I'm trying to kind of parametrize
that, physics might have like 20
159
00:12:02,060 --> 00:12:05,860
parameters. And, you know, some of them
are crazy like a surface layer fractional
160
00:12:05,860 --> 00:12:10,120
like zero point one or something. And
usually they keep the same constants for
161
00:12:10,120 --> 00:12:14,620
all these values. Usually it's like
someone in like 1994 came up with these 20
162
00:12:14,620 --> 00:12:18,730
numbers and now we all use the same 20
numbers. But you know, maybe they're
163
00:12:18,730 --> 00:12:21,850
different. And like the Pacific or the
Atlantic or like maybe they're different
164
00:12:21,850 --> 00:12:25,740
when it's summer and winter and the
problem is, there's many of these
165
00:12:25,740 --> 00:12:28,920
parametrizations. So you know here's like
20 parameters, but then you have a lot
166
00:12:28,920 --> 00:12:32,430
more for clouds. You have a lot more sea
ice. We add them all up. Suddenly you have
167
00:12:32,430 --> 00:12:38,050
like 100, maybe up to a thousand kind of
tunable parameters. Kind of going back to
168
00:12:38,050 --> 00:12:42,700
this plot that was shown at the last talk.
You can see kind of the all the models
169
00:12:42,700 --> 00:12:47,880
kind of agree really well from like 1850
to 2000, because they're all kind of being
170
00:12:47,880 --> 00:12:50,950
they all have different kind of
parameters, but they all get kind of tuned
171
00:12:50,950 --> 00:12:54,890
or optimized. So they get the 20th
century. Correct. So they get the black
172
00:12:54,890 --> 00:12:59,980
line. Correct. But then when you run them
forward, you run them to like 2300. They
173
00:12:59,980 --> 00:13:02,860
all are slightly different. So they all
start producing different physics and
174
00:13:02,860 --> 00:13:07,550
suddenly you get a huge like red band. So
that's saying you have lots of model
175
00:13:07,550 --> 00:13:12,870
uncertainty. So it's kind of on some
people might say like oh this like tuning
176
00:13:12,870 --> 00:13:17,510
process is like optimization. It's like
not very scientific to be kind of right.
177
00:13:17,510 --> 00:13:22,440
It's kind of like in the past. It's kind
of like the best live we've had. But I
178
00:13:22,440 --> 00:13:26,420
think, you know, we should be able to do a
little bit better. Better than that. So
179
00:13:26,420 --> 00:13:28,700
just to give you the idea, you know, some
people would say, you know, why don't you
180
00:13:28,700 --> 00:13:31,650
just you know, most of the physics, which
you just, you know, resolve all the
181
00:13:31,650 --> 00:13:36,640
physics, you know, but see if you want to
do like a direct numerical simulation, so
182
00:13:36,640 --> 00:13:39,410
it's basically saying you want to resolve
all the motions in the ocean, in the
183
00:13:39,410 --> 00:13:43,310
atmosphere. You basically need to resolve
things down to like one millimeter. So if
184
00:13:43,310 --> 00:13:46,850
you have like a grid spacing of one
millimeter and you consider the volume of
185
00:13:46,850 --> 00:13:50,760
the ocean and the atmosphere, you
basically say you need like 10 to the 28
186
00:13:50,760 --> 00:13:55,470
grid points. You know, that's like imagine
putting cubes of like one millimeter
187
00:13:55,470 --> 00:13:56,850
everywhere in the
ocean and atmosphere.
188
00:13:56,850 --> 00:13:58,630
That's how many great
points you would need.
189
00:13:58,630 --> 00:14:01,830
So unfortunately, you could do that.
But there's not enough computer power or
190
00:14:01,830 --> 00:14:05,800
storage space in the world to do that. So
you're kind of stuck doing something a bit
191
00:14:05,800 --> 00:14:11,120
coarser. Usually most climate models, he's
like 10 to the 8 great points so that you
192
00:14:11,120 --> 00:14:16,390
10 to the 20 to little words. You don't
want to just run a big climate model once
193
00:14:16,390 --> 00:14:19,870
you know you need to run them for very
long times, usually like you run them for
194
00:14:19,870 --> 00:14:23,290
a thousand years or ten thousand years
when you want to run many of them because
195
00:14:23,290 --> 00:14:27,170
you want to collect statistics. So
generally you don't run at the highest
196
00:14:27,170 --> 00:14:31,460
resolution possible. You run kind of at a
lower resolution so you can run many, many
197
00:14:31,460 --> 00:14:36,900
models. So because you can only use so
much resolution, it seems that power
198
00:14:36,900 --> 00:14:39,790
transitions or these kind of mathematical
things, you have to live with them, you've
199
00:14:39,790 --> 00:14:45,310
got to use them. But at least one idea is,
you know, instead of using numbers that
200
00:14:45,310 --> 00:14:49,030
sum. But he came up with in 1994. You
might as well try to figure you know,
201
00:14:49,030 --> 00:14:50,980
better numbers or maybe
you know if the numbers
202
00:14:50,980 --> 00:14:52,210
are kind of different
in different places,
203
00:14:52,210 --> 00:14:55,910
you should find that out. So one
thing you could do, one thing we are
204
00:14:55,910 --> 00:15:01,340
trying to do is get the pressurization is
to kind of agree with like basic physics
205
00:15:01,340 --> 00:15:05,210
or agree with observations. So we have
lots of observations. How many we can run
206
00:15:05,210 --> 00:15:07,300
kind of high resolution
simulations to resolve
207
00:15:07,300 --> 00:15:09,100
a lot of the physics
and then make sure
208
00:15:09,100 --> 00:15:12,100
when you put the prioritization in
the climate model, it actually gives you
209
00:15:12,100 --> 00:15:16,840
the right numbers according to basic
physics or observations. But sometimes
210
00:15:16,840 --> 00:15:20,810
that might mean, you know, different
numbers in the Atlantic, in the Pacific or
211
00:15:20,810 --> 00:15:24,480
different numbers for the winter and the
summer. And you have to run many high
212
00:15:24,480 --> 00:15:28,660
resolution simulations to get enough data
to do this. But indeed, you know, these
213
00:15:28,660 --> 00:15:33,890
days I think we have enough computing
power to do that. So it's kind of do all
214
00:15:33,890 --> 00:15:37,070
these high resolution simulations. We
ended up building a new kind of ocean
215
00:15:37,070 --> 00:15:42,370
model that we run on GPUs because these
are all faster for giving us these
216
00:15:42,370 --> 00:15:46,520
results. So we ended up usually most
climate modeling is done in Fortran. We
217
00:15:46,520 --> 00:15:53,310
decided to go with with Julia for a number
of reasons, which I'll talk about. But the
218
00:15:53,310 --> 00:15:58,220
left figure is kind of that mixed layer or
boundary layer turbulence kind of movie.
219
00:15:58,220 --> 00:16:01,630
But instead of the rainbow color map, now
it's using a more reasonable color maps.
220
00:16:01,630 --> 00:16:07,050
It looks like the ocean, the right is that
old movie. So we're generating tons and
221
00:16:07,050 --> 00:16:11,670
tons of data from using simulations like
this and then hopefully we can get enough
222
00:16:11,670 --> 00:16:15,440
data and like figure out a way to explain
the prior transitions. But it's kind of a
223
00:16:15,440 --> 00:16:19,810
work in progress. So a different idea that
might be more popular here, I don't know.
224
00:16:19,810 --> 00:16:25,770
Is instead of kind of using the existing
permanent positions, you could say, OK,
225
00:16:25,770 --> 00:16:29,960
well, now you have tons and tons of data.
Maybe you just throw in like a neural
226
00:16:29,960 --> 00:16:34,620
network into the differential equations.
Basically, you put in the physics, you
227
00:16:34,620 --> 00:16:37,760
know, and then the neural network is
responsible for the physics you don't
228
00:16:37,760 --> 00:16:43,780
know. So, for example, you know, most
people here might not. I also don't want
229
00:16:43,780 --> 00:16:46,230
to talk about differential equations
because I would take a long time. So just
230
00:16:46,230 --> 00:16:48,280
imagine that the equation
in the middle is kind of
231
00:16:48,280 --> 00:16:49,960
what a climate model
needs to solve.
232
00:16:49,960 --> 00:16:53,240
And the question marks are kind of
physics we don't know. So we don't know
233
00:16:53,240 --> 00:16:58,720
what to put there. But maybe you could put
out a neural network. So number one is
234
00:16:58,720 --> 00:17:03,320
kind of a possible characterisation or a
possible way you could try to franchise
235
00:17:03,320 --> 00:17:05,750
the missing physics where the neural
networks kind of responsible for
236
00:17:05,750 --> 00:17:09,940
everything. We find that doesn't work as
well. So instead, maybe you tell it some
237
00:17:09,940 --> 00:17:13,990
of the physics, maybe tell it about cue,
which is like the heating or cooling at
238
00:17:13,990 --> 00:17:17,540
the surface. And then it's kind of
responsible for resolving the other stuff.
239
00:17:17,540 --> 00:17:22,300
But it's still a work in progress because
the blue is kind explosive your data. The
240
00:17:22,300 --> 00:17:25,690
orange is supposed to be the narwhal and
they don't agree. So it's still a work in
241
00:17:25,690 --> 00:17:28,960
progress, but hopefully we'll be able to
do that better. So this is kind of stuff
242
00:17:28,960 --> 00:17:34,400
that's like a week or two old. But kind of
reach a conclusion, at least from my half
243
00:17:34,400 --> 00:17:39,750
of the talk. So the reason I personally
like Julia as a climate modeler is we were
244
00:17:39,750 --> 00:17:45,000
able to kind of build an ocean model from
scratch basically in less than a year. And
245
00:17:45,000 --> 00:17:47,471
one of the nice things
is that the user interface
246
00:17:47,471 --> 00:17:49,561
or the scripting and the model
backend is all in one language,
247
00:17:49,561 --> 00:17:52,030
whereas in the past
used to usually write the high
248
00:17:52,030 --> 00:17:57,850
level and like Python and maybe the back
end is like Fortran or C. And we find, you
249
00:17:57,850 --> 00:18:01,400
know, when we Julia, it's just as fast as
our legacy model, which was written in
250
00:18:01,400 --> 00:18:07,230
Fortran. And one of the nicest things was
that basically able to write code once and
251
00:18:07,230 --> 00:18:10,510
using there's a need of GPU compiler. So
basically you write your code one single
252
00:18:10,510 --> 00:18:15,180
code base and you go to CPUs and GPUs.
So you'd want to write two different code
253
00:18:15,180 --> 00:18:20,610
bases. And yeah, we find generally because
it's high level language, we're all kind
254
00:18:20,610 --> 00:18:24,860
of more productive. We can give a more
powerful user API and Julia kind of has a
255
00:18:24,860 --> 00:18:30,240
nice multiple dispatch backend so that we
find that makes it easy for the users to
256
00:18:30,240 --> 00:18:35,330
kind of extend the model or hack the
model. And there's. Some people would say
257
00:18:35,330 --> 00:18:38,480
the Julia community is pretty small. But
we find there's a pretty big Julia
258
00:18:38,480 --> 00:18:41,470
community interest in scientific
computing. So we fund kind of all the
259
00:18:41,470 --> 00:18:46,540
packages we need are pretty much
available. So with our client conclude my
260
00:18:46,540 --> 00:18:50,240
half by saying there is most of the
uncertainty in climate modeling basically
261
00:18:50,240 --> 00:18:53,850
comes from humans because they don't know
what humans will do. But there's a huge
262
00:18:53,850 --> 00:18:57,550
model uncertainty basically because of
physics we don't understand or physics,
263
00:18:57,550 --> 00:19:01,610
the kind of model cannot see. You can't
resolve every cloud and you know, every
264
00:19:01,610 --> 00:19:05,390
wave in the oceans you've got you've got
to figure out a way to account for them.
265
00:19:05,390 --> 00:19:09,940
So that's what our prioritization does.
And we're trying to kind of use a lot of
266
00:19:09,940 --> 00:19:14,680
computing power to kind of make sure we
train or come up with good privatizations
267
00:19:14,680 --> 00:19:19,600
instead of kind of tuning the model at the
end. And we're hoping this will lead to
268
00:19:19,600 --> 00:19:23,750
better climate predictions. Maybe you
will. Maybe you won't. But at least, you
269
00:19:23,750 --> 00:19:28,990
know, even if it doesn't. Hopefully we can
say we got rid of the model tuning problem
270
00:19:28,990 --> 00:19:33,110
and hopefully we can make. We find it
that software development for climate
271
00:19:33,110 --> 00:19:37,930
modeling is easier than if we did it in
Fortran. I will say this kind of an
272
00:19:37,930 --> 00:19:41,760
advertisement, but I'm looking to bike her
on Germany for a week and apparently can't
273
00:19:41,760 --> 00:19:46,390
take the next bike out of Leipzig. So if
anyone is looking to sell their bicycle or
274
00:19:46,390 --> 00:19:51,230
wants to make some cash, I'm looking to
rent a bicycle. So yeah, if you have one,
275
00:19:51,230 --> 00:19:55,070
come talk to me, please. Thank you. Danke.
276
00:19:55,070 --> 00:20:02,190
applause
277
00:20:02,190 --> 00:20:09,490
Churavy: So one big question for me always
is how can we ask technologists hub that
278
00:20:09,490 --> 00:20:14,580
think most of us in this room are fairly
decent with computers? The internet is not
279
00:20:14,580 --> 00:20:19,410
necessarily an new island for us. But how
do we use that knowledge to actually
280
00:20:19,410 --> 00:20:25,060
impact real change? And if you haven't
there's some fantastic article:
281
00:20:25,060 --> 00:20:30,030
worrydreams.com/ClimateChange. Which lists
all the possible or not all the possible
282
00:20:30,030 --> 00:20:36,610
but a lot of good ideas to think about and
go like, okay, do my skills apply in that
283
00:20:36,610 --> 00:20:43,490
area? Well, I'm a computer scientist. I do
programing language research. So how do my
284
00:20:43,490 --> 00:20:50,430
skills really apply to climate change? How
can I help? And one of the things that
285
00:20:50,430 --> 00:20:54,880
took me in this article was how, and one
of the realization, and why I do my work
286
00:20:54,880 --> 00:20:59,790
is that the tools that we have built for
scientists and engineers, they are that
287
00:20:59,790 --> 00:21:05,550
poor. Computer scientists like myself have
focused a lot on making programing easier,
288
00:21:05,550 --> 00:21:11,580
more accessible. What we don't necessarily
have kept the scientific community as a
289
00:21:11,580 --> 00:21:18,250
target audience. And then you get into
this position where models are written in
290
00:21:18,250 --> 00:21:23,150
a language. Fortran 74 and isn't that a
nice language, but it's still not one that
291
00:21:23,150 --> 00:21:28,320
is easily picked up and where you find
enthusiasm in younger students for using
292
00:21:28,320 --> 00:21:36,450
it. So I work on Julia and my goal is
basically to make a scientific computing
293
00:21:36,450 --> 00:21:41,410
easier, more accessible and make it easier
to access the huge computing power we have
294
00:21:41,410 --> 00:21:49,960
available to do climate modeling. Ideas,
if you are interesting in this space is,
295
00:21:49,960 --> 00:21:53,590
you don't need to work on Julia
necessarily, but you can think about maybe
296
00:21:53,590 --> 00:21:59,420
I'm to look at modeling for physical
systems, modeling like one of the
297
00:21:59,420 --> 00:22:04,250
questions is can be model air conditioning
units more precisely, get them more
298
00:22:04,250 --> 00:22:09,211
efficient? Or any other technical system.
How do we get that efficiency? But we need
299
00:22:09,211 --> 00:22:15,340
better tools to do that. So the language
down here as an example is modelicar.
300
00:22:15,340 --> 00:22:19,560
There is a project right now, modelicar is
trying to see how we can push the
301
00:22:19,560 --> 00:22:24,770
boundary there. The language up here is Fortran.
You might have seen a little bit of that
302
00:22:24,770 --> 00:22:32,310
in the talk beforehand and it's most often
used to do climate science. So why
303
00:22:32,310 --> 00:22:37,100
programing languages? Why do I think that
my time is best spent to actually work on
304
00:22:37,100 --> 00:22:44,250
programing languages and do that in order
to help people? Well, Wittgenstein says:
305
00:22:44,250 --> 00:22:48,620
"The limits of my language are the limits
of my world." What I can express is what I
306
00:22:48,620 --> 00:22:52,190
think about. And I think people are
multilingual, know that that sometimes
307
00:22:52,190 --> 00:22:56,050
it's easier to think for them. But certain
things in one language say it isn't the
308
00:22:56,050 --> 00:23:00,660
other one. But language is about
communication. It's about communication
309
00:23:00,660 --> 00:23:05,670
with scientists, but it's also about
communication with the computer. And too
310
00:23:05,670 --> 00:23:09,950
often programing language fall into that
trap where it's about, oh, I want to
311
00:23:09,950 --> 00:23:15,720
express my one particular problem or I
wanna express my problem very well for the
312
00:23:15,720 --> 00:23:20,420
compiler, for the computer. I won't talk
to the machine. What if I found that
313
00:23:20,420 --> 00:23:24,640
programming languages are very good to
talk to other scientists, to talk in a
314
00:23:24,640 --> 00:23:29,220
community and to actually collaborate? And
so the project that Ali and I are both
315
00:23:29,220 --> 00:23:35,420
part of has, I think, 30 ish. I don't
know. The numbers are as big as the big
316
00:23:35,420 --> 00:23:40,580
coupe of climate scientists modelers. And
we have a couple of numerical scientists,
317
00:23:40,580 --> 00:23:44,650
computer scientists and engineers and we
all working the same language, being able
318
00:23:44,650 --> 00:23:49,260
to collaborate and actually work on the
same code instead of me working on some
319
00:23:49,260 --> 00:23:56,610
low level implementation and Ali telling
me what to write. That wouldn't be really
320
00:23:56,610 --> 00:24:05,050
efficient. So, yes, my goal is to make
this search easier. Do we really need yet
321
00:24:05,050 --> 00:24:09,110
another high level language? That is a
question I often get. It's like why Julia?
322
00:24:09,110 --> 00:24:14,660
And not why are you not spending your time
and effort doing this for Python? Well, so
323
00:24:14,660 --> 00:24:21,580
this is as a small example, this is Julia
code. It looks rather readable. I find it
324
00:24:21,580 --> 00:24:28,930
doesn't use a semantic whitespace. You may
like that or not. It has all the typical
325
00:24:28,930 --> 00:24:33,960
features that you would expect from a high
level dynamic language. It is using the
326
00:24:33,960 --> 00:24:37,190
M.I.T. license that has a built in package
manager. It's very good for interactive
327
00:24:37,190 --> 00:24:45,040
development, but it has a couple of
unusual wants and those matter. You need
328
00:24:45,040 --> 00:24:49,830
if you want to simulate a climate model,
you need to get top performance on a
329
00:24:49,830 --> 00:24:56,470
supercomputer. Otherwise you won't get an
answer in the time that it matters. Julia
330
00:24:56,470 --> 00:25:02,830
uses just in time ahead of time
compilation, the other great feature is
331
00:25:02,830 --> 00:25:07,670
actually a spitting in Julia. So I can
just look at implementations. I can dive
332
00:25:07,670 --> 00:25:12,610
and dive and dive deeper into somebodies
code and don't have a comprehension
333
00:25:12,610 --> 00:25:20,210
barrier. If I if you ever have spent some
time and tried to figure out how Python
334
00:25:20,210 --> 00:25:26,360
sums numbers under the hood to make it
reasonably fast. Good luck. It's hard.
335
00:25:26,360 --> 00:25:30,060
It's written in C. See, and there is a lot
of barriers in order to understand what's
336
00:25:30,060 --> 00:25:35,380
actually going on. Then reflection and
meta programing. You can do a lot of fun
337
00:25:35,380 --> 00:25:41,140
stuff which we're going to talk about. And
then the big coin for me is that you have
338
00:25:41,140 --> 00:25:46,150
native keep you code generation support so
you can actually take Julia code and run
339
00:25:46,150 --> 00:25:49,920
it on the GPU. You you're not
relying on libraries because libraries
340
00:25:49,920 --> 00:25:59,450
only are can express the things. That was
where writtenin there. So early on last
341
00:25:59,450 --> 00:26:03,550
December, I think we met up for the
climate science project and after deciding
342
00:26:03,550 --> 00:26:08,450
on using Julia for the entire project.
They were like, we we're happy with the
343
00:26:08,450 --> 00:26:12,710
performance, but we have a problem. We
have to duplicate our code for GPUs
344
00:26:12,710 --> 00:26:20,530
and CPUs. What really? It can't be! I mean, I
designed the damn thing, it should be working.
345
00:26:20,530 --> 00:26:25,559
Well, what they had at that point was
basically always a copy of two functions
346
00:26:25,559 --> 00:26:31,170
where one side of it was writing the CPU
code and the other side was implementing a
347
00:26:31,170 --> 00:26:36,710
GPU code. And really, there were only a
couple of GPU specific parts in there. And
348
00:26:36,710 --> 00:26:43,210
if anybody has ever written GPU Code, it's
this pesky which index am I calculation.
349
00:26:43,210 --> 00:26:49,710
Worthy for loop on the CPU to would just
looks quite natural. And I was like, what?
350
00:26:49,710 --> 00:26:55,610
Sit. Come on. What we can do is we can
just wait a kernel so he takes a body of
351
00:26:55,610 --> 00:27:00,490
the for loop, extracts it in a new
function. Add a little bit of sugar and
352
00:27:00,490 --> 00:27:06,460
magic to court GPU kernels and CPU
functions and then we're done. Problem
353
00:27:06,460 --> 00:27:12,730
solved. What the code roughly would look
look like isn't actually this. You can
354
00:27:12,730 --> 00:27:19,670
copy and paste this and it should work.
And so you have two functions. One of them
355
00:27:19,670 --> 00:27:23,679
launches where you extract your kernel.
Then you write a function that takes
356
00:27:23,679 --> 00:27:29,580
another function and runs it function in a
for loop or it launches that function on
357
00:27:29,580 --> 00:27:34,600
the GPU. And then you have this little GPU
snippet is the only bit of us actually
358
00:27:34,600 --> 00:27:39,250
GPU, which calculates the index and
then calls the function F with an index
359
00:27:39,250 --> 00:27:45,780
argument. I'm done here. My, my
contribution to this project was done,
360
00:27:45,780 --> 00:27:49,650
Well, they came back to me and we're like,
now it's not good enough. And I was like,
361
00:27:49,650 --> 00:27:54,550
why? Well, the issue is they needed kernel
fusion. So that's the process of taking
362
00:27:54,550 --> 00:28:00,110
two functions and merging them together.
I'm like, okay, fine. Why do they need
363
00:28:00,110 --> 00:28:04,790
that? Because if you want to be white(?)
average efficient GPO code, you need to be
364
00:28:04,790 --> 00:28:09,980
really concerned about the numbers of
global memory loads and stores. If you
365
00:28:09,980 --> 00:28:13,720
have too many of them or if they are
irregular, you lose a lot of performance
366
00:28:13,720 --> 00:28:20,470
and you need good performance. Otherwise,
we can't simulate the solution once. They
367
00:28:20,470 --> 00:28:24,600
also actually wanted to take use GPU
functionality and low level controlled.
368
00:28:24,600 --> 00:28:29,610
They wanted to look at their kernels and
use shared memory constructs. They wanted
369
00:28:29,610 --> 00:28:36,190
to do precise risk working, minimizing the
number of registers used and they really cared
370
00:28:36,190 --> 00:28:40,751
about low level performance. They were
like, well, we can't do this with the
371
00:28:40,751 --> 00:28:47,790
abstraction you gave us because it builds
up too many barriers. And I could have
372
00:28:47,790 --> 00:28:53,970
given you a few more typical computer
science answer, which would have been OK.
373
00:28:53,970 --> 00:28:59,429
Give me two years and I'll come back to
you and there is a perfect solution which
374
00:28:59,429 --> 00:29:03,350
is like a cloud cover in the sky. And I
write your speech spoke language that does
375
00:29:03,350 --> 00:29:06,850
exactly what you need to do. And at the
end, we have a domain specific language
376
00:29:06,850 --> 00:29:10,590
for climate simulation that will do final
volume and discontinuous cloaking in
377
00:29:10,590 --> 00:29:17,410
everything you want. And I will have a
PhD. Kit. Fantastic. Well, we don't have
378
00:29:17,410 --> 00:29:21,880
the time. The whole climate science
project that we are on has accelerated
379
00:29:21,880 --> 00:29:27,040
timeline because the philanthropist that
the funding that research are. Well, if
380
00:29:27,040 --> 00:29:35,090
you can't give us better answer anytime
soon, it won't matter anymore. So I sat
381
00:29:35,090 --> 00:29:40,580
down and was like, okay, I need a box. I
need something. It has minimal effort.
382
00:29:40,580 --> 00:29:45,150
Quick delivery. I need to be able to fix
it. If I do get it wrong the first time
383
00:29:45,150 --> 00:29:49,600
around and I did, it needs to be hackable.
My collaborator needs to understand it and
384
00:29:49,600 --> 00:29:55,040
actually be able to change it. And it
needs to be happened yesterday. Well,
385
00:29:55,040 --> 00:29:59,370
Julia is good at these kinds of hacks. And
as I've learned, you can actually let them
386
00:29:59,370 --> 00:30:06,259
go into bespoke solutions and have better
abstractions after the fact. So that
387
00:30:06,259 --> 00:30:10,220
you're that you can actually do the fancy
computer science that I really wanted to
388
00:30:10,220 --> 00:30:14,860
do. The product is called GPUify Loops
because I couldn't come up with a worse
389
00:30:14,860 --> 00:30:23,520
name, nobody else could. So we stick with
it. It's a Macro based. And so, Julia, you
390
00:30:23,520 --> 00:30:30,730
can write syntax macros that transform the
transform the written statements into
391
00:30:30,730 --> 00:30:37,330
similar statements so you can insert code
or remove code if you want to. At, right
392
00:30:37,330 --> 00:30:41,330
now target CPUs and GPUs and we are
talking about how do we get multi threaded
393
00:30:41,330 --> 00:30:46,480
into the story, how do we target more on
different GPUs? There are other projects
394
00:30:46,480 --> 00:30:51,470
that are very similar. So there's OCCA,
which is where a lot of these ideas are
395
00:30:51,470 --> 00:30:58,610
coming from and Open ACC in C++ does
something really similar. But basically
396
00:30:58,610 --> 00:31:02,290
you write a for loop, you write an at
loop in front of it, which is the magic
397
00:31:02,290 --> 00:31:09,480
macro that takes a transformation. And you
have two indexed statements and now you
398
00:31:09,480 --> 00:31:15,090
just say I want to launch it on the GPU
and it magically does a job. Get,
399
00:31:15,090 --> 00:31:22,150
fantastic. So let's pick up the entire
implementation of the macro at loop
400
00:31:22,150 --> 00:31:29,270
without the error checking that didn't fit
on the screen a couple of nights. So
401
00:31:29,270 --> 00:31:38,140
everything is here and basically I'm just
manipulating the for loop so that on the
402
00:31:38,140 --> 00:31:46,350
GPU it only iterates one iteration per
index and on CPU it iterates all of the
403
00:31:46,350 --> 00:31:50,890
indices because CPU is single threaded
and a GPU is many, many
404
00:31:50,890 --> 00:31:56,810
multithreaded. Of course there's a little
bit of magic hidden in the device function
405
00:31:56,810 --> 00:32:00,880
because how do I know where I'm running?
And if you're curious how to do that and
406
00:32:00,880 --> 00:32:06,780
then we can talk after afterwards. But
otherwise, it's a very simple,
407
00:32:06,780 --> 00:32:11,440
straightforward transformation. It's
written in Julia. It's a Julia function.
408
00:32:11,440 --> 00:32:17,600
And. Yeah. So you don't need to understand
the code here. I just want to show how quick it
409
00:32:17,600 --> 00:32:24,760
can be to write something like this. If
you know anything about GPU Programming at
410
00:32:24,760 --> 00:32:28,620
all, there should be a little voice in the
head, of the back of your head is like,
411
00:32:28,620 --> 00:32:34,780
wait a second. How can you run a dynamic
programming on a GPU? That shouldn't be
412
00:32:34,780 --> 00:32:43,100
possible. Well, Julia can run on the GPU
because it has a lot of meta programing
413
00:32:43,100 --> 00:32:47,770
facilities for the port for stage
programing. So I can generate code based
414
00:32:47,770 --> 00:32:51,679
on a specific call signature. It has
introspection, reflection mechanisms that
415
00:32:51,679 --> 00:32:56,760
allow me to do some interesting stuff in
the background. It is built upon LVM,
416
00:32:56,760 --> 00:33:02,470
which is a common compiler infrastructure.
And so I can actually write staged
417
00:33:02,470 --> 00:33:08,440
function that would generate an LVM
specific code for my one function and do
418
00:33:08,440 --> 00:33:16,820
so do that during compile time and is a
dynamic language that tries really hard to
419
00:33:16,820 --> 00:33:20,480
avoid runtime uncertainties. And this is
one of the challenges if you're getting
420
00:33:20,480 --> 00:33:26,900
into Julia is to understand that when
you're writing code that has a lot of
421
00:33:26,900 --> 00:33:32,230
runtime uncertainties, you get relative
slow performance, or as fast as Python.
422
00:33:32,230 --> 00:33:36,780
But if you work with the compiler and you
write runtime uncertainties you can get
423
00:33:36,780 --> 00:33:40,490
very fast code and you can run your code
on the GPU, you basically that's the
424
00:33:40,490 --> 00:33:45,340
limites test. If you can run your code on
the GPU, that you did your job well and it
425
00:33:45,340 --> 00:33:50,750
provides tools to understand the behavior
of your code. So a warning runtime
426
00:33:50,750 --> 00:33:55,130
uncertainty. It does that and I don't have
the time to go too deep into the answers.
427
00:33:55,130 --> 00:33:59,240
There is actually a paper about this. It
has a type system that allows you to do
428
00:33:59,240 --> 00:34:03,260
some sophisticated reasoning type
influence to figure out what your code is
429
00:34:03,260 --> 00:34:07,660
doing. Mutable dispatchers actually
helping us quite a lot in making it easier
430
00:34:07,660 --> 00:34:11,409
to do virtualized codes. It was a case of
specialization and just in time
431
00:34:11,409 --> 00:34:18,250
compilation. And so just looking a little
bit closer at some of these topics, if you
432
00:34:18,250 --> 00:34:25,060
want to look at the entire pipeline that
flow when you start while you're
433
00:34:25,060 --> 00:34:30,130
functioning, call it what happens through
the Julia compiler. You have tools to
434
00:34:30,130 --> 00:34:33,830
introspect and all of these on the right
hand side here and then you have tools to
435
00:34:33,830 --> 00:34:43,300
interact on the left hand side. You can
inject code back into the compiler. The
436
00:34:43,300 --> 00:34:48,490
other thing is Julia has dynamic
semantics. So when you difficult, you can
437
00:34:48,490 --> 00:34:54,200
at runtime, redefine your function and
recall it new function and it uses
438
00:34:54,200 --> 00:35:00,930
multiple dispatch. So if you look at the
absolute value call here, which of the 13
439
00:35:00,930 --> 00:35:06,280
possible methods will it call? In C++ or
in other programing languages this called
440
00:35:06,280 --> 00:35:10,930
a virtual function call. So isn't Julia
everything a virtual functional call? No.
441
00:35:10,930 --> 00:35:18,250
This is one of the important points is
when we call a function, let's say because
442
00:35:18,250 --> 00:35:24,230
sign of X, we look at the type of the
input arguments and then we first of all
443
00:35:24,230 --> 00:35:33,590
look at which function is applicable to
our input argument. So in this case, it
444
00:35:33,590 --> 00:35:41,920
would be the real down here because float
64 is a subtype of real. So we choose the
445
00:35:41,920 --> 00:35:50,740
right method using dispatch and then we
specialize that method for the signature.
446
00:35:50,740 --> 00:35:54,320
So the rule in multiople dispact is to
remember is we calling the most specific
447
00:35:54,320 --> 00:35:58,670
method, whatever specific might mean. So
if you have this bit of example, where we
448
00:35:58,670 --> 00:36:06,220
have a function F, which has three
different methods and we have an integer
449
00:36:06,220 --> 00:36:10,960
argument that can be matched on X, or on Y,
and then we have a floating point argument
450
00:36:10,960 --> 00:36:16,690
on Y and we call this with a "1,Hello".
Well, we will select the methods that is
451
00:36:16,690 --> 00:36:24,420
most specific for this argument, which
would be the number 1 here. On the other
452
00:36:24,420 --> 00:36:28,820
hand, if when we have a float 64 and
the second position, then we will call the
453
00:36:28,820 --> 00:36:34,730
second method. Now what happens if I pass
in an integer and the first position and a
454
00:36:34,730 --> 00:36:38,859
floating point in the second position?
Well, you would get a run time error
455
00:36:38,859 --> 00:36:44,180
because we can't make this decision. What
is the most specific method? That's just
456
00:36:44,180 --> 00:36:49,170
something to keep in mind. Method
specialization works really similarly when
457
00:36:49,170 --> 00:36:55,350
you call a method for the first time. This
method sign right now has no
458
00:36:55,350 --> 00:37:01,380
specializations. And then I look back,
call it once and Julia will insert a
459
00:37:01,380 --> 00:37:06,710
speciallisation just for Float64. Before
that it could have been a Float32. The
460
00:37:06,710 --> 00:37:11,430
Float64 is for this method. So
Julia specializes in compilers methods on
461
00:37:11,430 --> 00:37:15,050
concrete called signatures instead of
keeping everything dynamic or everything
462
00:37:15,050 --> 00:37:23,210
ambiguous. You can introspect this process
and there are several macros that are code
463
00:37:23,210 --> 00:37:30,510
lowered or code type that will help you
understand that process. I think I don't
464
00:37:30,510 --> 00:37:35,160
have enough time to go into detail here,
but just as a note, if you have a look at
465
00:37:35,160 --> 00:37:40,600
this, the percentage for means it's an
assignment. So if you reference it later,
466
00:37:40,600 --> 00:37:48,660
so in line 5, we will iterate on the 4
value. And then we can look at the type
467
00:37:48,660 --> 00:37:52,730
information that Julia infers out of that
call. We're calling the function mandel
468
00:37:52,730 --> 00:37:59,490
with the U in 32 and you can see how that
information propagates through the
469
00:37:59,490 --> 00:38:05,109
function itself. And then if you actually
do agressive inlining .., we do aggressive
470
00:38:05,109 --> 00:38:09,750
inlining and optimizations and
devirtualization. And so in the end, we
471
00:38:09,750 --> 00:38:15,960
don't have calls anymore. We only have the
intrinsics that Julia provides on which
472
00:38:15,960 --> 00:38:23,870
programs are actually implemented. So this
is a unsigned less than integer function.
473
00:38:23,870 --> 00:38:27,830
So we are using time and find as an
optimization to find static or near static
474
00:38:27,830 --> 00:38:32,260
site programs. It allows us to do us
agressive virtualization, inlining and
475
00:38:32,260 --> 00:38:36,630
constant propagation. But it raises
problems of cash and validation. So in
476
00:38:36,630 --> 00:38:42,250
bygone days, this used to be the case. I
could define a new function G after
477
00:38:42,250 --> 00:38:48,140
calling G want a function, a new function,
f after calling G once and I would get the
478
00:38:48,140 --> 00:38:53,250
old restore back. That's bad. That's
counter-intuitive. That's not dynamic. So
479
00:38:53,250 --> 00:38:59,560
in Julia 1.0 and I think 0.5 and 0.6
already. We fix that. So we invalidating
480
00:38:59,560 --> 00:39:06,400
the functions that have dependencies on
the function. We just changed. But can we
481
00:39:06,400 --> 00:39:09,710
see latency of your program? If you change
a lot of the functions and you recall them
482
00:39:09,710 --> 00:39:20,781
well hm we need to do a lot of work every
time. We do constant propagation, so it
483
00:39:20,781 --> 00:39:27,420
isn't very simple example. We try to
reduce. We try to exploit as much
484
00:39:27,420 --> 00:39:32,210
information as possible. And so if you
call if you want a function F and you call
485
00:39:32,210 --> 00:39:36,360
the function sign with a constant value,
we actually build just turning you the
486
00:39:36,360 --> 00:39:40,520
constant avoiding the calculation is the
sine entirely. And that can be very
487
00:39:40,520 --> 00:39:49,930
important during hot calls and in a cycle.
This can sometimes go wrong or Julia can
488
00:39:49,930 --> 00:39:53,930
has heuristics in order to decide when or
whether or not these optimizations are
489
00:39:53,930 --> 00:40:00,680
valuable. And so when you introspect your
code, you might see the results that are
490
00:40:00,680 --> 00:40:05,670
not that are not quite, what you want. So
we don't know what the return value here
491
00:40:05,670 --> 00:40:10,150
is. It's just a tuple. We know it's a
tuple, nothing else. Holistic to say, not
492
00:40:10,150 --> 00:40:14,060
specialize. But the nice thing about Julia
and where we get performance voice that we
493
00:40:14,060 --> 00:40:17,970
can actually do for specialisation and
hopefully at some point view makes a
494
00:40:17,970 --> 00:40:26,050
compiler smart enough that these edge
cases disappear. So I can use some secrets
495
00:40:26,050 --> 00:40:33,280
and foresee specialization to happen and
then I can actually infer the precise of
496
00:40:33,280 --> 00:40:40,270
return type of my function. Another thing
to know when you're coming for more
497
00:40:40,270 --> 00:40:45,050
traditional object oriented programing
language is that types are not extensible,
498
00:40:45,050 --> 00:40:50,190
extendable. So you can't inherit from
something like Int64. You can only subtype
499
00:40:50,190 --> 00:40:55,710
abstract types. We do that because
otherwise we couldn't do a lot of
500
00:40:55,710 --> 00:41:01,110
optimizations. When we, when we look at
programms, we can't never assume that you
501
00:41:01,110 --> 00:41:05,310
won't add code. We had a dinamic programming
language at any time in the runtime of your
502
00:41:05,310 --> 00:41:11,760
program you can't add code. And so we don't
have close word semantics, which doesn't
503
00:41:11,760 --> 00:41:16,500
doesn't allow us to say, hey, by the way,
we know all possible subtypes here. You
504
00:41:16,500 --> 00:41:20,800
might add a new type. Later on by
saying a common types are not extendable.
505
00:41:20,800 --> 00:41:27,740
We get a lot of the performance back. So
personally, for me, why do I like Julia?
506
00:41:27,740 --> 00:41:32,510
Or why do I work on Julia? It works like
Pyphon, it talks like Lisp and runs like
507
00:41:32,510 --> 00:41:38,120
Fortran. That's my five sales pitch.
It's very hackable and extendable.
508
00:41:38,120 --> 00:41:46,500
I can poke at the internals
and I can bend them if I need to. It's a
509
00:41:46,500 --> 00:41:52,330
bit of upon LVM. So in reality, for me as
a compiler writer, it's my favorite LVM
510
00:41:52,330 --> 00:41:59,190
front end. I can get the LVM code
that I need to actually run. But for users,
511
00:41:59,190 --> 00:42:04,870
that's hopefully not a concern. If you do
our job right and it has users in
512
00:42:04,870 --> 00:42:09,430
scientific computing and I'm in a prior
life whilst doing a lot of scientific
513
00:42:09,430 --> 00:42:15,040
computing in cognitive science wanting
models. And I care about these users
514
00:42:15,040 --> 00:42:21,190
because I've seen how hard it can be to
actually make progress when the tools you
515
00:42:21,190 --> 00:42:28,780
have are bad. And my personal goal is to
enable scientists and engineers to
516
00:42:28,780 --> 00:42:37,500
collaborate efficiently and actually make
change. Julia is a big project and Climate
517
00:42:37,500 --> 00:42:44,450
is a big project and many people to thank.
And with that, I would like to extend you
518
00:42:44,450 --> 00:42:50,180
an invitation if you're interested. There
is juliacon every year. Where you have a
519
00:42:50,180 --> 00:42:57,830
develop meet up. Last year we were about
60 people are much smaller than CCC. But
520
00:42:57,830 --> 00:43:02,240
next year it will be in Lisbon. So come
join us if you're interested and if you
521
00:43:02,240 --> 00:43:05,970
want to meet scientists who have
interesting problems and are looking for
522
00:43:05,970 --> 00:43:08,570
solutions. Thank you.
523
00:43:08,570 --> 00:43:17,120
Applaus
524
00:43:17,120 --> 00:43:23,860
Herald A: Time for questions and answers,
are there any questions?
525
00:43:23,860 --> 00:43:29,050
Herald H: Yeah, we've got microphones over
there. So just jump to the microphone and
526
00:43:29,050 --> 00:43:33,010
ask your questions so that
everybody could hear.
527
00:43:33,010 --> 00:43:38,510
Question: What do you mean when you say
dead? Julia talks like Lisp and how is
528
00:43:38,510 --> 00:43:43,280
that a good thing Lachen
Churavy: Well, it talks like Lisp, but it
529
00:43:43,280 --> 00:43:48,490
doesn't look like Lisp. I assume that's
what you mean. It doesn't have that many
530
00:43:48,490 --> 00:43:53,960
braces. But no, Lisp has another powerful meta
programming capabilities and macros. And
531
00:43:53,960 --> 00:43:58,320
so we have a lot of that. If you read a
little bit about the history of Lisp. The
532
00:43:58,320 --> 00:44:03,650
original intention was to write NLisp,
which would be Lisp with a nice syntax. And
533
00:44:03,650 --> 00:44:07,320
I think Julia is my personal is NLisp.
It has all these nice features, but it
534
00:44:07,320 --> 00:44:13,390
doesn't have the packet syntax.
Herald A: OK. Thank you.
535
00:44:13,390 --> 00:44:18,580
Question: Thanks for the talk. My question
is regarding the first part of the talk.
536
00:44:18,580 --> 00:44:22,920
You, if I understand correctly, you
simulating a deterministic system there.
537
00:44:22,920 --> 00:44:26,180
So there's no additional noise
term or anything, right?
538
00:44:26,180 --> 00:44:29,990
Ramadhan: Well, if you had infinite
precision, I think it would be
539
00:44:29,990 --> 00:44:34,850
deterministic. But I think by kind design
turbulence itself is not deterministic.
540
00:44:34,850 --> 00:44:38,400
Well, it's a chaotic system,
Question: But the district size version
541
00:44:38,400 --> 00:44:41,400
itself is deterministic. You don't have
the monte carlo part where you have
542
00:44:41,400 --> 00:44:44,470
some noise that you would add to
which might actually be justified
543
00:44:44,470 --> 00:44:49,800
from the physics side. Right?
Ramadhan: Well, I mean, we, if you think if
544
00:44:49,800 --> 00:44:53,650
you ran the same simulation again, you
would not get that. Well, I think if you
545
00:44:53,650 --> 00:44:55,940
ran on the exact same machine,
you would get the
546
00:44:55,940 --> 00:44:58,240
same answer. So in that
sense, it is deterministic.
547
00:44:58,240 --> 00:45:00,910
But if you ran on a slightly
different machine like truncation
548
00:45:00,910 --> 00:45:04,010
error, I'd like the 16th decimal place
could give you a completely different
549
00:45:04,010 --> 00:45:08,180
answer. Question: Sure. So the point I'm
trying. Am I allowed to continue?
550
00:45:08,180 --> 00:45:12,270
Herald H: Yes, of course. There's no one
else. Well, there is one person else. So you
551
00:45:12,270 --> 00:45:17,250
can continue a few minutes if you want to.
Thanks. Laughter
552
00:45:17,250 --> 00:45:20,030
Question: So the point I was
trying to make is,
553
00:45:20,030 --> 00:45:22,420
if you add noise in the
sense that it's a physical
554
00:45:22,420 --> 00:45:24,790
system, you have noise in
there, it might actually allow you to
555
00:45:24,790 --> 00:45:28,890
solve a PDI or discretize a PD, but get a
stochastic simulation itself, which might
556
00:45:28,890 --> 00:45:34,480
be interesting because it often can make
things easier. And also, you mentioned
557
00:45:34,480 --> 00:45:39,010
neural differential equations, right? And
in particular, with physical systems, if
558
00:45:39,010 --> 00:45:43,231
you have an discontinuities, for example,
the DT integral can actually be quite the
559
00:45:43,231 --> 00:45:47,890
problem. And there is work on to just
plug my colleagues work, control neutral
560
00:45:47,890 --> 00:45:50,860
differential equations where you can
actually also built in these
561
00:45:50,860 --> 00:45:53,580
discontinuities, which might also be
interesting for you guys.
562
00:45:53,580 --> 00:45:56,470
Ali: That's why maybe we should talk
because I don't know much about that stuff
563
00:45:56,470 --> 00:45:59,690
where we're kind of just starting up. I
think that so we've been doing this maybe
564
00:45:59,690 --> 00:46:03,478
hopefully continuous, but maybe we'll hit
discontinuities. I don't know. We should
565
00:46:03,478 --> 00:46:07,359
talk, though. Q: And also the math is
beautiful and has no sickness. It's the
566
00:46:07,359 --> 00:46:10,359
physics that mightn't change. I'm a
mathematician. I have to say that. Ali: I know
567
00:46:10,359 --> 00:46:15,480
that the physics is ugly, trust me.
Churavy: Just as quickly, we do have
568
00:46:15,480 --> 00:46:24,570
stickers and I sell cookies, too. They are
in the cookie box and on. I think they for
569
00:46:24,570 --> 00:46:29,530
somebody from our community is giving a
juliaworkshop and we're trying to find a
570
00:46:29,530 --> 00:46:34,290
set up an assembly space and hopefully
that goes out as well.
571
00:46:34,290 --> 00:46:38,490
Herald H: Go on please.
Question: Also, one question for the first
572
00:46:38,490 --> 00:46:44,840
part of the talk I want. I wanted to ask
if it's possible or if you are using
573
00:46:44,840 --> 00:46:49,190
dynamic resolution in your climate models.
Well, you will maybe have a smaller grid
574
00:46:49,190 --> 00:46:54,900
size near the (???) and larger
in the areas that are not that
575
00:46:54,900 --> 00:46:58,140
interesting.
Ramadhan: Like adaptive grids? So I
576
00:46:58,140 --> 00:47:02,520
think we mostly do that in the vertical.
So usually in the ocean, the thinking
577
00:47:02,520 --> 00:47:04,260
things are interesting
in the, you know,
578
00:47:04,260 --> 00:47:06,070
close to the surface.
We have more resolution
579
00:47:06,070 --> 00:47:08,780
there. But as you go deeper,
things get less interesting. So you put
580
00:47:08,780 --> 00:47:14,380
less resolution there. Generally, I think
in general, the idea people have asked
581
00:47:14,380 --> 00:47:17,760
that before, you know, why do you always
use constant grids? Why don't you use
582
00:47:17,760 --> 00:47:21,700
these adaptive grids on your global, you
know, models? And you the answer I've
583
00:47:21,700 --> 00:47:24,670
heard I don't know if it's very
convincing. I think generally there hasn't
584
00:47:24,670 --> 00:47:28,950
been that much research or people who do
research into adaptive grids for kind of
585
00:47:28,950 --> 00:47:34,960
models. Their funding gets cut. But I like
the answer I've heard is a lot of the
586
00:47:34,960 --> 00:47:38,070
time, a lot of the atmosphere and ocean is
turbulent. So if you especially you do
587
00:47:38,070 --> 00:47:42,750
kind of adaptive refinement, then you just
kind of adapt everywhere because there's
588
00:47:42,750 --> 00:47:48,400
kind of turbulence everywhere. But yeah, I
don't I'm not. I guess first for our
589
00:47:48,400 --> 00:47:53,320
simulations we're kind of just some of
the numerical methods are only fast if you
590
00:47:53,320 --> 00:47:57,070
run it on a regular grid. So
that's the reason we don't use adaptive
591
00:47:57,070 --> 00:48:01,310
grids for our simulations. But in general,
adaptive grids for climate models is
592
00:48:01,310 --> 00:48:05,010
interesting beyond like it seems like
there needs to be more research in that
593
00:48:05,010 --> 00:48:07,990
area. So I don't know if I answered your
question, but I kind of just ranted it.
594
00:48:07,990 --> 00:48:10,890
Question: You did, thanks.
Herald H: Go go ahead, please.
595
00:48:10,890 --> 00:48:16,160
Question: Yeah, it's just a few guesses
about us. I think I have. I wept quite a
596
00:48:16,160 --> 00:48:22,170
bit of legacy fortune code in Python. And
my question is, would there be a simple
597
00:48:22,170 --> 00:48:28,740
pass converting Fortran code to Julia,
preferably automatically. Do you have any
598
00:48:28,740 --> 00:48:33,060
ideas about this one?
Churavy: You can do it. Your Julia code
599
00:48:33,060 --> 00:48:39,100
will look like Fortran code. So you
haven't won anything. So, yes. As a good
600
00:48:39,100 --> 00:48:42,170
starting point, you can do that.
Absolutely. But you can also just call
601
00:48:42,170 --> 00:48:46,080
Fortran from Julia and then totally move
over. I generally don't want people to
602
00:48:46,080 --> 00:48:50,060
rework their code, except if there's a
good reason. Like starting from scratch
603
00:48:50,060 --> 00:48:55,470
sometimes helps. It can be a good reason.
Or if you say the solutions, we don't have
604
00:48:55,470 --> 00:49:00,880
the necessary experts to to work with the
old solution anymore. But generally, if
605
00:49:00,880 --> 00:49:04,700
you have Fortran code, I would just say,
well, call Julia from Fortran or from
606
00:49:04,700 --> 00:49:11,040
Julia, get it up to speed and then start
transitioning. Piece by piece. That makes
607
00:49:11,040 --> 00:49:16,770
sense?
Herald H: So any more questions? No more
608
00:49:16,770 --> 00:49:23,530
questions. That's an early read. Ali Ramadhan,
and Valentin Churavy, thank you very much.
609
00:49:23,530 --> 00:49:25,980
Applaus
610
00:49:25,980 --> 00:49:29,370
36C3 postroll music
611
00:49:29,370 --> 00:49:51,000
Subtitles created by c3subtitles.de
in the year 2020. Join, and help us!