Research Collaboration Do's and Don'ts (Josh Angrist, Guido Imbens, Isaiah Andrews)

♪ [music] ♪

- [Narrator] Welcome
to Nobel Conversations.

In this episode,
Josh Angrist and Guido Imbens,

sit down with Isaiah Andrews
to discuss the key ingredients

in their Nobel-winning
collaboration.

Josh and Guido, first,
congratulations on the Nobel Prize!

Thank you.

- [Isaiah] The work you did
together, particularly the work

on the Local Average
Treatment Effect,

or LATE framework

was cited as one of the big reasons
you won the prize.

At the same time, you only
overlapped at Harvard for a year--

if I'm remembering correctly--

it would be great to hear
a bit more

about how you started
this collaboration

and what made your working
relationship productive.

Are there ways in which you felt
like you complimented each other,

what got things started
on such a productive trajectory?

Your job talk, as I recall, Guido,
it wasn't very interesting

but I think it was
a choice-based sampling--

It was. It was.
[laughter]

I was a very marginal hire there

because they didn't
actually interview me

on the regular job market,

but I think they were
very desperate to get someone else

to actually teach their courses.

It was after they had
a couple of seminars already

and they're still looking
in econometrics,

- so Gary called me and kind of--
- [Josh] Gary Chamberlain?

Gary Chamberlain called me and
interviewed me over the telephone.

He said, "Okay, well,
why don't you come out

and give a talk?"

- [Josh] I remember this talk
a little bit.

I remember the dinner that
you and Gary and I had.

I remember not being very excited
about your job market paper,

but I saw that Gary was and
luckily, Gary's view prevailed...

Yes.

- [Josh] ...and Harvard
made you an offer

and I think we started talking to
each other pretty pretty soon

after you arrived
in the fall of 1990, right?

As I said, I came and I didn't
have a very clear agenda.

I was a little intimidated
getting there.

But Gary kind of said,
"No, you should talk to Josh."

You should go to the labor seminar,
kind of see what these people do.

They're doing very
interesting things there."

I listened to Gary.

As we did.

As we did in the those days
and ever since.

I think it helped that
we were neighbors.

We both lived in Harvard's
junior faculty housing,

partly because housing costs
were very high in Cambridge

relative to our salary,
which was very low.

I think it also made a difference,

neither of us came from Cambridge,

so there were a lot of MIT people

who already had
their whole networks,

kind of our collaborators.

♪ [music] ♪

- [Josh] Well, I think
we figured out

a mode of working together, also.

We had kind of a regular date,
so we were neighbors

and we often did
our laundry together.

We didn't have laundry
machines at our apartments.

But we used to do our laundry
and we were talking

and you had a way
of fairly systematically,

addressing questions that
would come up in our discussions

and the one thing that
I was very impressed by,

our early interaction,
is you would follow up.

You would write some things down.

Looking back at those days,
sort of clearly,

I just had a lot more time
to actually think.

I look at my junior
colleagues now--

[Guido] You don't have time
to think now.

[Josh] No, but for me that is
kind of one thing,

but I feel now a lot of
my junior colleagues

don't actually have a lot
of time to think.

People are just doing
so many projects,

and it's actually so hard

and there's so much pressure
on people to publish.

I remember spending a lot of time
sitting in my office

and thinking,
"Wow, what shall I do now?"

[laughter]

But it would give me a lot of time

to actually think about
these problems

and trying to figure it them out

and I could actually go to seminars

and then the next day have coffee
or lunch with Josh or Gary

and actually talk about
those things.

- [Isaiah] You guys
weren't actually

at Harvard together all that long,

you started working
together pretty quickly.

Were you both in the mindset that
you were looking for co-authors,

or looking for a particular type
of types of co-authors at the time

or was it more sort of
fortuitous than that?

- [Josh] I think we were lucky.

I don't remember
that I was looking.

I think, it was more fortuitous.

I said I came in,
I'd done my job market paper,

and another paper for my thesis

and I was just very happy
to come to Harvard

and suddenly there were all these
seminars to go to,

and lots of interesting people
to talk to,

but it wasn't a very
conscious thing on my part.

Looking back, I think there
was a moment for me,

where I was discussing
instrumental variables,

potential outcomes,
treatment effects with Guido

and we had
a pretty good discussion,

but then he also sent me some notes

and the notes were very methodical
write-up of our discussion

and what you thought,

we had been concluding
in a fairly formal way

and I thought,
"Well, that's great."

Talk is cheap, right,
but with somebody...

- [Guido] Yeah, but--
- ...really writes out their story.

- [Guido] For me, it really helps
writing things down

and I do remember working with Josh

and sitting in my office
and writing things out

and you guys have all
had the discussions with Gary

where afterwards we need
to then sit down

and actually write things up

to figure out exactly
what was going on.

I think the other thing
we had, Guido,

is we had some
very concrete questions

that came from applications.

- [Guido] Yeah.

A lot of econometrics, in my view,

that we were schooled in
was about models,

here's a model and what can
you say about this model?

I think we were thinking about,
here's a particular scenario,

draft eligibility is an instrument
for whether you serve in the Army.

What do we learn from that?

- [Guido] That's right.

That's right, and that's sort of
where your influence

on the way I do research now
is still very clear--

♪ [music] ♪

- [Isaiah] Zooming out
a little bit, just thinking about

when you guys started
working on this,

when you started working together,

any thoughts for folks

who are just interested in
finding productive

co-authors being productive?

I mean, Guido already mentioned
the importance of having time,

right, which it is.

It is very easily not to have
a lot of time to think--

You definitely have to make time.

That's a great question
though, Isaiah,

and I tell my students that

you should pick your co-authors
as carefully,

maybe more carefully
than you pick your spouse.

You want to find co-authors who,

you have some complementarity

and that's what makes
a strong relationship.

You don't want to work
with somebody

who sees the world exactly like you

and as much as Guido
and I agree about things,

we often disagree
about things to this day

and it's fruitful to have
those discussions

and we had complimentary skills.

I was very empirical.
I'm not really an abstract thinker.

Guido was great at figuring out
what the principles were.

Yeah, that's right and I totally
agree, kind of [inaudible].

These are incredibly
important relationships

and you see a lot of
people working together

and not necessarily
working very well

and then it's very hard often
to get out of this relationship.

A good partnering is a
beautiful thing, like a marriage.

It produces wonderful children,

the fruits of the scholarship are
potentially wonderful

and they exceed the capacity of
the partners to do it on their own

but a bad co-authorship
can be very destructive

and time consuming and painful,
just like a bad marriage.

Arguments may start about
who did what when

and intellectual property
type issues,

especially when it when
it goes a little sour

and somebody thinks the other party
is not pulling their weight.

There's more co-authorship
now in economics,

I think that's been documented,
much more.

- [Guido] Yes.

There's more teams
and there's larger teams

and I think that's great,
I love working on teams.

We do work on schools
with big teams.

I work often with PI teammates
like Parag Pathak and David Autor

and then a team of
graduate students,

but I see that the students
are not always,

in some ways they're a little
too promiscuous,

in my view, in their partnering.

They don't think it through.

It's difficult to think it through.

I think, for me, working
with people always has involved

spending a lot of one-on-one
time with people,

you need to figure out
how they think

and what kind of problems
they're interested in

and how they think about
these problems,

how they like to write,
to make that--

And it takes some maturity on
everybody's part.

- Yes. Yes.
- [Isaiah] In what sense?

Just in the sense of knowing
what's going to work for them,

knowing when things are
versus aren't working?

- [Josh] Maturity in the
sense of having some judgment

to be able to face it honestly,
if it's not going well,

sometimes you have to have
some difficult discussions.

Is it worth continuing?

"I was hoping you would do this,
and you didn't,"

maybe it turns out
there's some feeling

in the other direction,
the same way.

And Josh is very good
[chuckles]

in the being honest,
part from the beginning,

- [Josh] For better or worse.

- [Guido] I would write this stuff
and then I remember

the first version of the paper
with Rubin,

Josh was in Israel at the time,

Don and I were in Cambridge

and so I would talk
with Don regularly,

but Don wasn't really doing
much writing in those days,

I would write things
and then I would fax them to Josh

and they would come back,

the first page
just one big cross, "No,"

second page, one big line, "No"

and that would go for awhile
but he still does that.

I sent him the first draft
of my Nobel lecture,

and Josh goes,
"No, no!"

I've gotten some PDF comments
like that from Josh, very helpful.

Omit needless words.

I have few co-authors
who are willing to do that.

Especially as you get older,
it's harder to put up with that.

I would find it harder now to start
working with people who did that

early on in a co-author
relationship.

It's also very hard because
you need to have enough trust.

Josh, for being willing
to be very critical,

he was also willing
to admit being wrong.

♪ [music] ♪

- [Josh] But you have to be on
the lookout for good partners,

somebody who can help you
answer questions

that you can't answer yourself.

I think there's a natural tendency
for people to gravitate

to people who are similar
in outlook and skills

and that's not as useful.

- [Guido] Josh is right, nowadays
it's very tempting

to find people who think
about the same problems

you're already thinking about,
who think along the same lines

and that may not lead
to very novel stuff.

But at the same time finding people

who actually have
very different ideas,

it's going to take a lot of time.

Guido, you mentioned in passing
how working with Josh

has influenced how you do research,

could you say a little more
about that?

I'd also be interested
to hear from Josh,

did working with Guido influence
the way that you do research?

- [Guido] Nowadays, I'm much
more conscious of the fact that,

for me, good economic research

comes out of talking to people
doing empirical work,

and it's really not reading
econometrica

or the reading the stats journals,

but it's actually talking to people
doing empirical work,

going to the empirical seminars.

When I was at Berkeley,

David Carr and Raj Chetty,
as colleagues there

and I would talk to them
and listen to them,

trying to figure out

how are they solving their problems
and other things there

where I'm not really quite happy
with the way they're doing things

and trying to look for
methodological problems,

where there's some more
general solutions possible.

I tried to tell it to my students
that I encourage them to work

as research assistants also,

for the people doing empirical work
at Stanford.

There was no subbing what I learned
while I was in graduate school,

but it really came out of
working with Josh,

as well as talking to Gary,

Gary was always encouraging
of doing that

and because he done that himself,

he'd worked with on empirical
problems with Zvi Griliches

early in his career.

Yeah.

Well, I became more more interested
in the econometric theory

through our interaction,

and I think empiricists are often
impatient with econometric theory,

partly because empirical work
is very time-consuming,

and you may have a sense
that something is

convincing and sensible

and you haven't really fully
made the case for that,

but you're convinced

and that motivates you
to pursue it,

like the draft lottery story.

I was pretty sure that was
worth doing

and I came away from
working with Guido

seeing that there was
the potential to say something

more than just about
that particular problem,

and I think over the those early
years in the 90s,

our thinking evolved together

that there's actually
a framework here,

a way to solve a lot of problems

and I think that that is the power
of the LATE framework,

is it answers a lot of questions.

♪ [music] ♪

- [Isaiah] In some sense,
did you find that,

email versus facts
versus in-person,

the medium mattered
to how collaboration went

or they're ways that you felt like

it was the most useful
to collaborate?

To me, I think
what matters most is,

initially you have a period of--

We needed that initial period,

that was very intense with
almost daily interaction

and we also became friends.

You don't develop the kind of
friendship, electronically usually

[laughter]

but once you have that foundation
you can be pen pals

and we did use email,
though it wasn't as useful then

but it worked,
we definitely had a lot of faxes.

I still have these faxes,
long faxes

and then in the summer,
I would come to Cambridge,

usually to the NBR meetings
and hang around for a few weeks

and you visited me in Israel.

I visited in Israel.

But yeah, there was good foundation
from that that year

and in some sense that was enough.

Nowadays,

I have the co-authors
in lots of different places,

but it's always been important

to spend some time with people
in the same place,

so you understand how they work,
how they think,

even to the point that,

you know when
they actually respond,

whether they respond quickly

or whether that means

they're not actually
doing anything,

or that mean they're thinking hard
about a problem

and they just take longer,

but you do need to
develop some understanding there.

♪ [music] ♪

- [Isaiah] We've talked about
how your collaboration started,

maybe just to step back slightly,
were they're sort of features about

the environment at Harvard
or in Cambridge, at the time,

which you felt contributed to it?

Coming from Brown,

I felt it was very
intimidating place

because it clearly was a very,
very impressive set of people.

Zvi Griliches was there,
Dale Jorgensen--

Gary, Jerry Hausman, Whitney Newey,
sometimes Jamie Robins.

I mean, my view of that
in retrospect,

I can't say I loved every
minute of every talk

I ever gave in that Workshop,

but that was the highest powered,

that was the group
you wanted to reach

and you would get extraordinarily
insightful feedback,

even if it wasn't always
easy to swallow.

Yeah, and I have for a while,

I would basically give
a talk every semester

because we didn't have
any money to invite people.

Gary would say,
"Well, why don't you give a talk?"

[laughter]

That was the arena for young people
with our interest.

- [Guido] Yeah, it was really
very impressive,

but it was also quite tough--

It was intimidating.

People there had very strong
views on what they thought was

the way you should do econometrics,

the way the direction
things should go,

now, I would think things were
getting a little stale,

that in fact, we were bringing in
a lot of the new ideas...

- [Josh] Yeah.

...and that wasn't necessary
immediately appreciated.

[laughter]

- [Josh] But that's okay.
- And that's fine.

We were pushed
and a lot of great discussions

in that workshop about
what should we make of LATE

but there were other questions
that were just as interesting,

like the role of
the propensity score,

that was a big deal in the 90s

and econometrics was
moving towards that

and there were a lot
of great questions.

Yeah,

I learned a huge amount
there from the time I spent--

- [Josh] I think the other thing
that Guido and I

both benefited from is we both,

not at the same time,
but in early in our careers,

taught econometrics
with Gary Chamberlain,

and that was like
an apprenticeship for us, I think.

I taught a mixed graduate,
undergrad 1126,

I don't know if they still have
that number...

- [Isaiah] Mmhmm, they do.

...very interesting course
that it had

both graduate
and undergraduate enrollment

and it was relatively applied
for an econometrics class,

and I learned a lot by teaching
that with Gary.

But in that sense,
Harvard was a great place,

very flexible there.

The other thing I remember
about Harvard is,

well I had very good students,

I taught a lot of
wonderful students

who went on to have
wonderful careers.

Also, Harvard as an institution,

you're probably are aware of this,
Isaiah,

as a junior faculty member,
they didn't then ask much of us,

other than teaching our classes.

We didn't have administrative
concerns, to speak of.

I think I went to two
faculty meetings

in my two years at Harvard

and so we're left--

You were given a lot of freedom
and flexibility.

I went to the chair said,

"Can I teach this course
with Rubin?"

I think it was Friedman
at the time. It was like, "Fine."

It wasn't really any concern about
what what it was about

and again, that was a very
intimidating experience,

but it was a great experience.

♪ [music] ♪

- [Narrator] If you'd like to
watch more

Nobel Conversations,
click here,

or if you'd like to learn more
about econometrics,

check out Josh's
"Mastering Econometrics" series.

If you'd like to learn more about
Guido, Josh, and Isaiah,

check out the links
in the description.

♪ [music] ♪