HARY KRISHNAN: So, thank you very much
for being here on a Saturday evening, this late.
My talk got pushed to the last, but I
appreciate you being here, first. My name's Hari. I
work at MavenHive. So this is a talk about
Ruby memory model. So before I start, how many
of you have heard about memory model and know
what it is? Show of hands, please. OK. Let's
see where this talk goes. So why I did
I come up with this talk topic. So I
started my career with Java, and I spent a
lot many years with Java, and Java has a
very clearly documented memory model. And it kind of
gets to you because with all that, you don't
feel safe enough doing multi-threaded programming at all. So
with Ruby, we've always been talking about, you know,
doing multi-process for multi-process parallelism,
rather than multi-threaded parallelism,
even though the language actually supports, you know, multi-threading
semantics. Of course we know it's called single-threaded and
all that, but I just got curious, like, what
is the real memory model behind Ruby, and I
just wanted to figure that out. So this talk
is all about my learnings as I went through,
like, various literatures, and figured out, and I tried
to combine, like, get a gist of the whole
thing. And cram it into some twenty minutes so
that I could, like, probably give you a very
useful session, like, from which you can further do
more digging on this, right. So when I talked
to my friends about memory model, the first thing
that comes up to their mind is probably this
- heap, heap, non-heap, stack, whatever. I'm not gonna
talk about that. I'm not gonna talk about this
either. It's not about, you know, optimizing your memory,
or search memory leeks, or garbage collection. This talk
is not about that either. So what the hell
am I gonna talk about? First, a quick exercise.
So let's start with this and see where it
goes. Simple code. Not much to process late in
the day. There's a shared variable called 'n', and
there are thousand threads over that, and each of
those threads want to increment that shared variable hundred
times, right. And what is the expected output? I'm
not gonna question you, I'm just gonna give it
away. It's 100,000. It's fairly straightforward code. I'm sure
all of you have done this, and it's no
big deal. So what's the real output? MRI is
very faithful, it gives you what you expected. 100,000,
right. So what happens next? I'm running it on
Rubinius. This is what you see. And it's always
going to be a different number every time you
run it. And that's JRuby. It gives you a
lower number. Some of you may be guessing already,
and you probably know it, why it gives you
a lower number. So why all this basic stupid
code and some stupid counter over here, right? So
I just wanted to get a really basic example
to explain the concept of increment is not a
single instruction, right. The reason why I'm talking about
this is, I love Ruby because the syntax is
so terse, and it's so simple, it's so readable,
right. But it does not mean every single instruction
on the screen is going to be executed straight
away, right. So at least, to my junior self,
this is the first advice I would give, when
I started, you know, multi-threaded programming. So at least
three steps. Lowered increments store, right. That's, even further,
really simple piece of code like, you know, a
plus equals to, right. So this is what we
really want to happen. You have a count, you
lowered it, you increment it, you stored it. Then
the next thread comes along. It lowers it, increments
it, stores it. You have the next result which
is what you expect, right. But we live in
a world where threads don't want to be our
friend. They do this. One guy comes along, reads
it, increments it. The other guy also reads the
older value, increments it. And both of them go
and save the same value, right. So this is
a classic case of lost update. I'm sure most
of you have seen it in the database world.
But this pretty much happens a lot in the
multi-threading world, right. But why did it not happen
with MRI? And what did you see the right
result?? [00:04:52]? That, I'm sure a lot of you
know, but let's step, let's part that question and
just move a little ahead. So, as you observed
earlier, a lot of reordoring happening in instructions, right.
Like, the threads were context-switching, and they were reordering
statements. So where does this reordering happen? Reordering can
happen at multiple levels. So start from the top.
You have the compiler, which can do simple optimizations
like look closer?? [00:05:20]. Even that can change the
order of your statements in your code, right. Next,
when the code gets translated to, you know, machine-level
language, goes to core, and your CP cores are
at liberty, again, to reorder them for performance. And
next comes the memory system, right. The memory system
is like the combined global memory, which all the
CPUs can read, and also they're individual caches. But
why do CPUs have caches? They want to, memory
is slow, so they want to load, reload all
the values, refactor it, keep it in the cache,
again improve performance. So even the memory system can
conspire against you and reorder the loads and stores
after the memory registers. And that can cause reordering,
right. So this is really, really crazy. Like, I'm
a very stupid programmer, who works at the programming
language level. I don't really understand the structure of
the hardware and things like that. So how do
I keep myself abstracted from all this, you know,
really crazy stuff? So that's essentially a memory model.
So what, what is a memory model? A memory
model describes the interactions of threads through memory and
their shared use of data. So this is straight
out of Wikipedia, right. So if you just read
it first, either you're gonna think it's really simple,
and probably even looks stupid, but otherwise you might
not even understand. So I was the second category.
So what does this all mean? So when there
are so many complications with the reordering, the reads
and writes of memory and things like that, as
a programmer you need certain guarantees from the programming
language, and the virtual machine you're working on top
of, to say this is how multi-threaded shared, I
mean, multi-threaded access to shared memory is going to
work. These are the basic guarantees and these are
the simple rules of how the system works. So
you can reliably work code against that, right. So
in, in effect, a memory model is just a
specification. Any Java programmers here, in the house? Great.
So how many of you know about JSR 133?
The memory model, double check locking - OK. Some
people. Single term issue? OK - some more hands.
So Java was the first programming language which came
up with a concept called memory model, right. Because,
the first thing is, right ones?? [00:07:45] won't run
anywhere. It had to be predictable across platforms, across
reimplementations, and things like that. So the, there had
to be a JSR which specified what is the
memory model that it can code against so that
your multi-threaded code works predictably, and deterministically across platforms
and across virtual machines. Right? So essentially that's where
my, you know, whole thing started. I had gone
through the Java memory model, and was pretty much
really happy that someone had taken the pain to
write it down in clear terms so that you
don't have to worry about multi-threading. Hold on, sorry.
Sorry about that. Cool. So. Memory model gives you
rules at three broad levels. Atomicity, visibility and ordering.
So atomicity is as simple as, you know, variable
assignment. Is a variable assignment an indivisible unit of
work, or not? The rules around that, and it
also talks about rules around, can you assign hashes,
send arrays indivisibly and things like that. These rules
can change based on every alligned version, and things
like that. Next is visibility. So in that example
which you talked about, I mean, we saw two
threads trying to read the same value. Essentially they
are spying on each other. And it was not
clear at what point the data had to become
visible to each of those threads. So essentially visibility
is about that. And that is ensured through memory
barriers and ordering, which is the next thing. So
ordering is about how the loads and stores are
sequenced, or, you know, let's say you want to
write a piece of code, critical section as you
call it. And you don't want the compiler to
do any crazy things to improve performance. So you
say, I make it synchronized, and it has to
behave in a, behave in a nice serial?? [00:09:40]
manner. So that ?? manner is ensured by ordering.
Ordering is a really complex area. It talks about
causality, logical clocks and all that. I won't go
into those details. But I've been worrying you with
all this, you know, computer science basics and all
this. Why the hell am I talking about it
in a Ruby conference? Ruby is single-threaded, anyway. Why
the hell should I care about it, right? OK.
Do you really think languages like Ruby are thread
safe? Show of hands, anyone? So thread safety, I'm
talking only about Ruby - maybe Python. GIL based
languages. Are they thread safe? No? OK. In fact
they're not. Having single-threaded does not mean it's thread-safe,
right. Threads can switch context, and based on how
the language has been implemented and how often the
threads can switch context, and at what point they
can switch, things can go wrong, right. And another
pretty popular myth - I don't think many people
believe it here, in this audience at least. I
don't have concurrency problems because I'm running on single
core. Not true. Again, threads can switch context and
run on the same core and still have dirty
reads and things like that. So concurrency is all
about interleavings, right. Again, goes back to reordering. I
think I've been talking about this too often. And
let's not, again, worry with that. It's about interleavings.
We'll leave it at that. So let's, before we
understand more about, you know, the memory model and
what it has to do with Ruby, let's just
understand a little bit about threading in Ruby. So
all of you know, green threads, as of 1.8,
there was only one worse thread, which was being
multiplexed with multiple Ruby threads, which were being scheduled
on it through global interpreter lock. 1.9 comes along,
there is a one to one mapping between the
Ruby thread and OS thread, but still the Ruby
thread cannot use the OS thread unless it has
the global VM lock as its call now. The
JVL acquire. So does having a Global Interpreter Lock
make you thread safe? It depends. It does make
you thread safe in a way, but let's see.
So how does GIL work? This is a very
simplistic representation of how GIL works. So you have
two threads here. One is already holding the GIL.
So it's, it's working with the OS thread. And
now when there is another thread waiting on it,
waiting on the GIL to do its work, it
sends a, it wakes up the timer thread. Time
thread is, again, another Ruby thread. The timer thread
now goes and interrupts the thread holding the GIL,
and if the GIL, if the thread holding the
GIL is done with whatever it's doing - I'll
get to it in a bit - it just
releases the lock, and now thread two can take
over and do its thing. Well this is the
basic working that at least I understood about GIL.
But there are details to this, right. It's not
as simple as what we saw. So, when you
initialize a thread, or create a thread in Ruby,
you pass it a block of code. So how
does that work? You take a block of code,
you put it inside the thread. What the thread
does is usually it acquires a JVL and a
block?? [00:13:11]. It executes the block of code. It
releases the, returns and releases the lock, right. So
essentially this is how it works. So during that
period of executation of the block, no other thread
is allowed to work. So that makes you almost
thread safe, right? But not really. If that's how
it's going to work, what if that thread is
going to hog the GIL, and not allow any
other thread to work? So there has to be
some kind of lock fairness, right. So that's where
the timer thread comes in and interrupts it. OK.
Does that mean the thread holding the GIL immediately
gives it up, and says here you go, you
can start and work with it? Not really. Again
the thread holding the GIL will only release the
GIL if it is at a context to its
boundary. What that is, is fairly complicated. I don't
want to go into the details. I think people
who here know a lot better C than me,
and are deep C divers really, they can probably
tell you, you know, how, at what the GIL
can get released. If a C thread, a C
code makes a call to Ruby code, can it
or can it not release the GIL? All those
things are there, right. So all these complexities are
really, really hard to deal with. I came across
this blog by Jesse Storimer. It's excellent and I
strongly encourage you to go through the two-part blog
about, you know, nobody understands GIL. It's really, really
important, if you're trying to do any sort of
multi-threaded programming in Ruby. So do you still think
Ruby is thread safe because it's got GIL? I'm
talking about MRI, essentially. So the thing is, we
can't depend on GIL, right. GIL is not documented
anywhere that this is exactly how it works. This
is when the timer thread wakes up. These are
the time slices alotted to the thread acquiring the
JVL. There is no documentation around at what point
the GIL can be released, can it not be
released, and things like that. There's no, it's not
predictable, and if you depend on it, what could
also happen is even within MRI, when you're moving
from version to version, if something changes in GIL,
your code with behave nondeterministically. And what about language
in Ruby implementations that don't even have a GIL?
So obviously that's the big problem, right. If you
write a gem or something which has to be
multi-threaded, and if you're depending on the GIL to
do its thing to keep you safe, then obviously
it cannot work on Rubinius and JRuby. Let that
alone, even, even if you give that up, even
with MRI, it's not entirely correct to say that
you're thread safe, because there is a GIL that
will ensure that only one thread is running. So
what did I find out? Ruby really does not
have a documented memory model. It's pretty much similar
to Python. It doesn't have a clearly documented memory
model. What is the implication of that? So as
I mentioned previously, a memory model is like a
specification. This is exactly how the system has to
provide a certain minimum guarantee to the users of
the language, right, regarding multi threaded access to shared
memory. Now, basically if I don't have a written
down memory model, and I am going to write
a Ruby implementation to model, I have the liberty
to choose whatever memory model I want. So the
code, if you're writing against MRI, may not essentially
work right on my, you know, my implementation of
Ruby. That's the big implication, right. So Ruby right
now depends on underlying virtual machines. Even after ER,
you have bad code compilations, so even MRI is
almost like a VM. So that has no specification
for a memory model, but it does have something,
right, internally. If you have to go through the
C code and understand. It's not guaranteed to remain
the same from version to version, as I understand,
right. And obviously JRuby and Rubinius, they depend on
JVM and LLVM respectively. And they all have a
clearly documented memory model. You could have a read
at it. And the only thing is, if Ruby
had an implementation - sorry, a specification for a
memory model, it could be, you know, implemented using
the constructs available on JVM and LLVM. But this
is what we have. We don't have much to
do. What do we do under the circumstances? We
have to engineer our code for thread safety. We
can't bask under the safety that, there is a
GIL and so it's going to help me keep
my code thread safe. So even I can write
multiple, you know, multi threaded code without actually worrying
about serious synchronization issues and things like that. It's
totally not the right thing to do. I think
any which way, Ruby is a language I love,
and I'm sure all of you love, so. And
it's progressing my leaps and bounds, and eventually we're
going to write more and more complex systems with
Ruby. And who knows, we might have true parallelism
very soon, right. So why, still, stay in the
same mental block that we don't want to write,
you know, thread safe code that's anyway single threaded.
We might as well get into the mindset of
writing proper thread safe code, and try and probably
come up with a memory model, right. But I
think for now we just start engineering code for
thread safety. Simple Mutex, I'm sure all of you
know, but it's really, really important for even a
stupid operation like a plus equals two. So simple
things which are noticed in Ruby code bases and
Rails code bases as well, like generally, is, there
is like a synchronized, you know, a section of
the code has lots of synchronization and everything. It's
really safe. But we leave an innocent accessor lying
around, and that causes a lot of, you know,
pain, like debugging those issues. And general issues like,
you know, state mutations, inside methods is really a
bad idea. So if you're looking for issues around
multi threading, this might be a good place to
start. So I just listed a few of them
here. I didn't want to make a really dense
talk with all the details. You can always catch
me offline and I can tell you some of
my experiences and probably even listen to you and
learn from you about some of the issues that
we can solve by actually writing proper thread safe
code in Ruby. I came across a few gems
which were really, really nice. Both of them happen
to be written by headius. The first one is
atomic. Atomic is almost trying to give you the
similar constructs like the Java utility concurrent package. It
tries to, it's kind of compatible across MRI, JRuby,
and Rubinius, which is also a really nice thing.
So you have atomic integers and atomic floats, which
do increments actually in an atomic way, which is
excellent. And then there is thread_safe library, which also
has a few thread safe data structures. I'm trying
to play around with these libraries right now, but
they may be a good, you know, starting point
if you are trying to do higher level constructs
for concurrency. And that's pretty much it. I'm open
to take questions. Thank you. And before anything I
really would like to thank you all, again for
being here for the talk, and thank the GCRC
organizers, you know, they've done a great job with
this conference. A big shout out to them.
V.O.: Any questions?
H.K.: Yeah?
QUESTION: Hey.
H.K.: Hi.
QUESTION: If, for example, if a Ruby code is running
in the JVM, in JRuby, how does, because none
of the Ruby code is written in a thread
safe way. How do, how does it internally manage
- does it actually, yeah, yesterday Yogi talked about
the point that ActiveRecord is not actually thread safe.
Can you explain it in detail like in a
theoretical way?
H.K.: OK. What is thread safety in
general, right? Thread safety is about how the data
is consistently maintained after multi-threaded access to that shared
data, right. So Ruby essentially has a GIL because
internal implementations are not thread safe, right. That's why
you want to have a GIL to protect you
from those problems. But as far as JRuby is
concerned, or Rubinius is concerned, the implementation itself is
not written in C. JRuby is written in Ruby
again, I mean JRuby itself, and Rubinius is written
in Ruby. And some of these actual internal constructs
are thread safe when compared to MRI. I haven't
actually taken a look in detail into the code
of these code bases, but if they are implemented
properly, you can be thread safe - internally, at
least - so, which means, the base code of
JRuby itself might be thread safe. It's only not
thread safe because the gems on top of it,
which are trying to run. They may have, like,
thread safety issues, right. Does that answer your question,
like, or- ?
QUESTION: About thread safety?? [00:22:09].
H.K.: Sure, sure. So those gems will not work. That's
the point. Like what I want to convey here,
is whatever gems we are offering, and whatever code
we are writing, we might get it - it's
a good idea to get into the habit of
writing thread safe code, so that we can actually
encourage a truly parallel Ruby, right. We don't, we
don't have to stay in the same paradigm of
OK we have to be single threaded.
QUESTION: So Mutex based thread management is one way.
There's also like actors and futures and things like that.
And there's a gem called cellulite-
H.K.: Yup.
QUESTION: That, combined with something called Hamster,
which makes everything immutable-
H.K.: Yup.
QUESTION: Is another way to do it.
H.K.: Yup.
QUESTION: Have you done it or like,
what's your experience with that?
H.K.: Yeah, I have tried out actors, with revactor,
and lockless concurrency is
something I definitely agree is a good idea. But
I'm specifically talking about, you know, lock-based concurrency, like,
Mutex-based concurrency. This area is also important because it's
not like thread mutable state is bad. It is,
it is actually applicable in certain scenarios. When we
are working in this particular paradigm, we still need
the safety of a memory model. Any other questions?
QUESTION: Thanks for the talk Hari. It was really
good.
H.K.: Thanks.
QUESTION: Is there a way that
you would recommend to test if you have done
threading properly or not? I mean, I know, bugs
that come out-
H.K.: Right.
QUESTION: Like I have
written bugs that come out of badly written, you
know, not thread safe code, as.
H.K.: So-
QUESTION: Like, ?? [00:23:46] so, you catch them.
H.K.: At least, my opinion, and a lot of people have
done research in this area, their opinion also is
that it's not possible to write tests against multi
threaded code where there is shared data. Because it's
nondeterministic and nonrepeatable. The kind of results you get,
you can only test it against a heuristic. For
example, if you have a deterministic use case at
the top level, you can probably test it against
that. But exact test cases can never be written
for this.
V.O.: Any more questions?
H.K.: Cool. All right. Thank you so much.