Marijn Haverbeke - The Rust That Could Have Been

This talk will describe a number of language concepts and features that were in the pre-1.0 Rust language at some point but were ultimately abandoned, such as typestate, garbage collection, structural types, and a more or less classical object system. I’ll go over the reasons they were abandoned, and try to convince you that the Rust we have now is the best Rust yet.

---
For more go to https://rustfest.eu or follow us on Twitter: https://twitter.com/rustfest

[ applause ]

Yeah, I'm glad so many people
still showed up

at the very end of the conference.

So yeah, I'm going to be talking about
mostly a number of features

that were part of the Rust language
at some point,

but no longer are.

And, why this is the case
and usually why this is a good thing.

So, as Ryan said, I'm Marijn Haverbeke.

If you are into Javascript,
you might have seen my name before.

Rust has been under development
for a while:

about ten years now, I think.

First - long stretch - was just
Graydon working in isolation and...

Who knows what kind of ideas and
experiments he abandoned at that point.

There's not even a like code repository
from that time public,

so it's like the prehistory.

And then, at some point,

Mozilla adopted the project
and assembled a team.

From a little before that point,
we do have Git history,

and at that point,
everything was discussed in issues

and mailing lists, and there's...

lots of records.

I was part of this pretty much -
the initial team - where we...

moved from the original OCaml-based
compiler to a Rust-based compiler,

and we got a bunch more people
involved in the language.

This was a period of...

like, the first real experience
with the language

and a bunch of people
from different backgrounds...

giving their opinions on the language

and trying to push it
into their favorite direction.

And, we had a lot of like
experiments and dead ends

and overhauls and false starts and...

just lots and lots of churn.

We would sometimes come up with
a breaking change in the morning,

and then have a patch ready
in the afternoon

and then convince someone
to merge it later on and then...

Because there was only one codebase,
we'd just fix everything right away

and people could continue working.

There was some...

some trickiness with actually
getting like a compiler...

that compiles the current code
after you'd make a breaking change,

so you'd first change the compiler,
then upload a snapshot,

then change the code,
and then you could -

everyone could - proceed
with the new snapshot.

And then, of course,
a year and a half ago - I think -

the team cut version 1.0,

and then the process changed entirely,
so now it's like...

everything stays backwards compatible.

It's impressive how seriously they have
been taking backward compatibility.

Experiments move like RFCs
move very slowly

and there has to be a wide consensus

and it has to fit
within the current codebase.

So, that's a whole different stage again.

I'm going to be mostly talking about
the period where I was part of the team.

which was 2011-2012,

and which was probably
the wildest period

in terms of features cut,
features changed - stuff like that.

So, it may seem a bit ridiculous

that we put so much time into
really complicated features

just to end up dropping them again.

But, I think it's kind of an essential
part of getting a complex design

like a programming language right that,

unless you're a super-genius,
you won't really see in advance

what the implications
and the interactions between

the various parts of the system are
and you have to try it and see

how well you can make it work
and how well it fits into the system,

and sometimes you later have to just...

abandon it again.

I think that's part of
a healthy design process for like...

mere mortals who need to
actually see how something works

before they can evaluate it.

I'm structuring this talk about...
around a number of visions

that were part of the language
and then dropped again.

And, I'll try to explain why I think that,
in every case,

it was a real good decision to drop them.

But, it's still interesting to see

what the original visions were
and what we did end up with.

So, these are typestate,
a structural type system and

lightweight processes,
and finally, garbage collection.

Let's start with typestate.

Typestate is... it was actually

an important point in initial
announcements of the language.

and people were very excited about it.

What typestate does is basically
allows you to...

allows the compiler to know more
about value than just its type.

So, an example would be...

This is something of type [sockets],

but we also happen to know that it's open
or this is something of type array,

but we happen to know... or say vector
in the current terminology,

but we happen to know that
it's not empty.

Something like that allowing you to add

more safety to your program -
more static guarantees.

So, when you're programming,
you usually have some mental model

of why the thing you are doing
right now is...

is valid - is not going to crash -

like if you're not just making
random changes

and seeing if the [tests pass],
you will have

some mental model of your... program.

And, to a certain degree,
depending on the language,

you can tell the compiler
about this model

and the compiler can then check

whether you are applying
your model consistently.

So, the simple case is just types -
that you're actually passing the type

that you think you are passing somewhere.

And, if you don't, then,
instead of finding out at runtime,

you find out at compile time -
and this is nice.

There's a kind of...

this computer does not have the fonts
that my computer had but...

Imagine arrowheads on both sides.

There's a kind of spectrum
on which languages fall

in terms of how much

you can actually communicate
to the compiler,

so down on one side,
there's Javascript,

and it's like... syntactically correct.

Okay, let's go ahead - let's run it.

And then, there's like,
on the... way on other side,

there's languages like Coq,
which require you to actually construct

a formal proof that your program does
what it's supposed to do -

that it does so in bounded time,
in bounded space,

which means you're making
a lot less mistakes.

But, on the other hand,
it's like a major...

a major project to write a small program
in such a language.

And, there is a reason that
not everyone is writing their web servers

in Coq or whatever.

And, Rust kind of falls in the middle.

It does have quite a bit
of static guarantees

and it helps quite a lot.

But, it still aims to be ergonomic,
like easy to program in,

where you don't have to

spend too much time
working on these things.

And I... one way to see the history
of programming languages is kind of...

one aspect of it at least is

that we've been finding better
and better vocabulary to...

to describe these things we know about
our program to the compiler

in a way that's actually convenient.

So, if you have a really terrible
type system,

that's often worse than
no type system at all.

If I have to choose to write something
in Java or Javascript,

[I'll just take Javascript,
thank you very much.]

But, we're getting better at this,

and Rust is making
a big contribution here,

and like bringing a real,
modern type system

to the systems programming space.

And the ownership model is - I think -

just really, really good.

I unfortunately wasn't on the team
anymore when this was introduced,

so I can't take any credit for it, but..

I think that this is
the most exciting part of Rust

and it's exactly this kind of thing
where you...

where the compiler knows
what you're trying to do and

tells you when you're
violating your model.

So, back to typestate.

It looked somewhat like this.

You could define predicates,

which is this "pure function not empty"
at the top

The extra information that
the compiler had about your values

came in the form of this predicate hold

But, these were actually just predicates
written in normal Rust code

that were supposed to be pure.

There was a concept of an fx system
at that point - which is also gone now.

But, they just took a value and said:
"Okay, I hold or I don't hold."

And, then you could define
for your functions,

preconditions and postconditions.

So, you could say, for example,

this function "last" here
demands that its first argument

has the "not empty" predicate
holding on it,

because you can't take the last element
from an empty array...

Then, before you could pass
such a value to such a function

you'd have to convince the compiler
that this predicate held at this point.

For some things,
this worked relatively well:

the compiler was very clever
in propagating its information

through the control flow graph
and like taking it

from the post conditions
of the functions you called.

But here, you have, for example,

I create an array
and then I want to pass it to last.

But, it's not okay: I first have to...

check that it's not empty.

And, this is actually...a sink check
would insert a runtime test,

call to the predicate,
and then panic if it failed.

But actually, I mean,
this array isn't empty:

this is very easy to prove.

But, because the compiler
only saw these predicates as

like opaque pieces of code,

it couldn't actually reason about them -

it could only take what you told it.

Like, if you checked
there was variance of check,

one of them which just was
like an unsafe form of

"just believe me: this holds."

So, that might also have been
appropriate here

because I'm really sure that
this array is not empty.

And then, there was a one version that

ensured that the compiler already
statically knew at this point

that something else... that it was
kind of an assertion of "okay...

"I must notice at this point,
but don't insert a runtime check.

"I want to have a static error
if it's not provable."

But, in my experience,
the effect of this system

was mostly that you would be
littering your code with check statements

and they would also panic at runtime.

So, the amount of static guarantees
wasn't very great

because often... usually the compiler
couldn't really help a lot with

like reasoning about when
they actually held and when they didn't.

It was in the compiler
for a long time still

but eventually it was dropped because
it was just not pulling its weight.

So, in terms of experiments
in good expressive ways

to express these kind of things,
I think this was a failed experiment.

It existed in some research
languages before

but it's never really made it into
a big mainstream type language -

for good reason, I think.

So, so much for that.

Next topic is "structural typing."

So, in typing systems,
you have two concepts

where structural typing is...

say you have a function type,
which has a few arguments types,

and a return type,
and you want to compare it

to another function type.

So, you're just going to look at
the fields in the function:

does it have the same amount
of arguments,

are its arguments of compatible types,
is it return type of compatible type?

And, that's structural.

On the other hand,
there is a nominal typing,

where you just say:
"Where is this type declared?

"What's the name of this type?" -
and it has to be the same.

So, Rust's structs currently work
this way,

as do enums -
two types are only compatible

if they are actual instances of
the thing that was declared

in the same point in the code.

Initially, structs were structural types.

So, this curly braces thing there
is syntax for a struct type,

with two fields - x and y -
of type "float."

And, the type declaration
just defines an alias for the type.

This is just like a name for the type
record with two float fields.

So, if I define a function,
which takes an argument of this point,

I can call it with just a record
constructed on the fly

without any record name involved.

Records themselves don't have a name:

they just have a structure
in this system, and...

it's kind of nice and lightweight
and minimal,

and often you don't even bother
to give your record a name

if you only use it a few times.

So, where you would now
probably use it triple,

you could use a record
with nice descriptive field names.

I kind of liked it for programming with...

But, I'll come back later
to why this part was removed.

Another aspect of this was object types,

whereas structure types
were only compatible

if they had actually
the exact same fields and...

in the same order.

They weren't reordered
because of C compatibility

and they had to be the exact same
to be able to compile it efficiently

because then all code
that interacted with such a record

knew how it was laid out in memory.

Objects were a more dynamic feature,
and here...

any object type that has a subset
of the fields -

fields are always methods -
so they're always functions.

This object type has, its compatible,
so I could,

if I define the type - a collection of T -
with these two -

it's probably not a very great abstraction
but just bear with me -

with a length and
an item accessor method,

you could take any object that has
I don't know what kind of...

methods with also these two,

and you could treat it
as a collection of T.

So, these were both the types
of the concrete objects

and also serve the role of interfaces,
which is kind of nice in terms of

how many concepts you need
to do object-oriented programming.

And, you could also even use it
as a kind of...

checked duck typing,
where you define your function

and you just say:
"I'm only going to call length

"on the thing that I'm getting,"

and then, anything that had a length
method could be passed in -

you don't even need to formally define
an interface name or anything,

it's just all like structural by name.

So...

One implication of this was

that because code that used them
didn't know their size,

they always had to be heap allocated.

I think they even always
had to be garbage collected.

Any calls to them will be going
through a dispatch table - a vtable.

So, they're so much more heavyweight...

compared to the rest of the language.

We were finding that, in the compiler,

we were kind of shying away from them

and that we absolutely needed
polymorphism because

they were more heavyweight
than necessary in many situations.

Then, at some point...

Well, there was also

a lot of machinery involved
in actually doing this,

like upcasting to a type with less methods

because then you needed to
allocate a new vtable,

preferably statically,
which forwarded to the alt object

and created a wrapper.

It was conceptually simple
but not terribly simple to implement.

And then, at some point...

we got more [Haskell?] people
on the team

and we all started agitating for

a typed class kind of implementation

interface thing that we ended up with now.

And, because no one really
liked these objects very much,

we migrated to that.

And, I think they just fit
with the language much better.

They don't require you
to put something on the heap.

They don't require indirect calls unless
you actually are using polymorphism.

So, I think that's a win.

But, now that we had implementations,
which effect a specific type,

structural records
also became problematic,

because if you're using a record that
happens to have the same shape

in two completely independent contexts

and they both define say
a two string implementation of it,

then these will clash,

even though the actual usages
have nothing to do with each other,

they will both be trying to
implement the same interface -

the same trait on it.

That doesn't work, so...

Well, no one cared that much
about structural records either,

so they became nominal
for this reason at that point.

And now, of course,

people say functions
are still structural -

they don't make much sense
in any other way.

But, the like heavy emphasis
on structural typing was abandoned...

again for good reasons.