Research Collaboration Do's and Don'ts (Josh Angrist, Guido Imbens, Isaiah Andrews)

♪ (music) ♪

(narrator) Welcome to Nobel conversations.

In this episode,
Josh Angrist and Guido Imbens,

sit down with Isaiah Andrews
to discuss the key ingredients

in their nobel-winning collaboration.

Josh and Guido, first congratulations
on the Nobel Prize!

Thank you.

(Isaiah) The work you did together,
particularly the work

on the local average treatment effect,
or late framework

was cited as one of the big reasons
you won the prize.

At the same time, you only
overlapped at Harvard for a year--

if I'm remembering correctly--

it would be great to hear a bit more
about how you started this collaboration

and sort of what made your working
relationship productive.

Are there ways in which you felt like
you complimented each other,

sort of what got things started
on such a productive, trajectory.

Your job talk, as I recall Guido,
it wasn't very interesting

but I think it was
a choice-based sampling--

It was. It was.

(laughter)

I was a very marginal hire there
because they didn't actually interview me

on the regular job market,
but I think they were very desperate to get

someone else to actually teach that
course.

It was after they had
a couple of seminars already

and it was still looking in econometrics,
so Gary called me and kind of--

Gary Chamberlain?

Gary Chamberlain called me and
interviewed me over the telephone.

He said, "Okay, well, my don't you come
out and give a talk?"

I remember this talk a little bit.

I remember the dinner that
you and Gary and I had.

I remember not being very excited
about your job market paper,

but I saw that Gary was and luckily,
Gary's view prevailed...

Yes.

...and Harvard made you an offer

and I think we started talking
to each other pretty pretty soon after

you arrived in the fall of 1990, right?

Now as I said, I came and
I didn't have a very clear agenda.

I was a little intimidated getting there.

But Gary kind of said, "No, you should talk to Josh."

You should go to the labor seminar,
kind of see what these people do.

They're doing very interesting things there."

I listened to Gary.

As we did.

As we did in the those days and ever since.

I think it helped it, we were neighbors.

So we both lived in Harvard's
junior faculty housing,

partly because housing costs
were very high in Cambridge

relative to our salary,
which was very low.

I think it also kind of made a
difference, neither of us came from Cambridge,

so there were a lot of MIT people
who kind of already had their whole networks,

kind of our collaborators.

♪ (music) ♪

(Josh) Well, I think we had figured out
a mode of working together also.

We had kind of a regular date,
so we were neighbors

and we often did our laundry together.

We didn't have laundry
machines at our apartments.

But we used to do our laundry
and we were talking

and you had a way of very systematically,

addressing questions that
would come up in our discussions

and the one thing that I
was very impressed by,

our early interaction,
is you would follow up.

Yeah,

You would write some things down.

Looking back at those days,
sort of clearly,

just had a lot more time to actually think

-- I mean, I look at my junior college now--
-- You don't have time to think now.

(Guido) No, but for me that is kind of one thing,

but I feel now a lot of my junior colleagues

don't actually have a lot of time to think.

People are just doing so many projects,
and it's actually so hard

and there's so much pressure on people
to publish that.

I remember spending a lot of time sitting
in my office and thinking,

"Wow, what shall I do now?"

(laughter)

But it would give me a lot of time
to actually think about these problems

and trying to figure it them out

and I could actually go to seminars

and then the next day have coffee
or lunch with Josh or Gary

and actually talk about those things.

(Isaiah) You guys weren't actually at
Harvard together all that long,

so you started working
together pretty quickly.

Were you both in the mindset that
you were looking for co-authors,

or looking for a particular type
of types of co-authors at the time

or was it more sort of fortuitous than that?

(Josh) I think we were lucky.

I don't remember I was that I was looking

Now that I think, it was more fortuitous.

I said I came in,
I'd done my job market paper,

and another paper for my thesis

and I was just very happy to come to Harvard

and suddenly there were all these
seminars to go to,

and lots of interesting people to talk to,

but it wasn't a very
conscious thing on my part.

Looking back, I think there
was a moment for me,

where I was discussing
instrumental variables,

potential outcomes,
treatment effects with Guido

and we had a pretty good discussion,

but then he also sent me some notes

and the notes were very methodical
write-up of our discussion

and what you thought,

we had been concluding in a fairly formal way

and I thought, "Well, that's great."

Talk is cheap, right, but with somebody..

- (Guido) Yeah, but--
- ...really writes out their story.

(Guido) For me, it really helps
writing things down

and I do remember working with Josh

and sitting in my office and writing things out

and you guys have all
had the discussions with Gary

where afterwards we need to then sit down

and actually write things up

to figure out exactly what was going on.

I think the other thing we had, Guido,

is we had some very concrete questions

that came from applications.

(Guido) Yeah.

A lot of econometrics, in my view,

that we were schooled in
was about models,

Here's a model and what can you say about this model?

I think we were thinking
about, here's a particular scenario,

draft eligibility is an instrument
for whether you serve in the Army.

What do we learn from that?

(Guido) That's right.

That's right, and that's sort of where your influence

on the way I do research now is still very clear--

♪ (music) ♪

(Isaiah) I guess zooming out a little bit,
just thinking about

when you guys started working on this,

when you started working together,

any thoughts for folks

who are just interested in
finding productive

co-authors being productive?

I mean, Guido already mentioned
the importance of having time,

right, which it is.

It is very easily not to have a lot of time to think--

You definitely have to make time.

That's a great question, though, Isaiah,

and I tell my students that
you should pick your co-authors

as carefully maybe more carefully than you pick your
spouse.

You want to find co-authors who,

you have some complementarity

and that's what makes a strong relationship.

You don't want to work with somebody
who sees the world exactly like you

and as much as Guido and I agree about things,

we often disagree about things to this day

and it's fruitful to have those discussions

and we had complimentary skills.

I was very empirical.
I'm not really an abstract thinker.

Guido was great at figuring out what the principles were.

Yeah, that's right and I totally
agree, kind of [a different spot.]

These are incredibly
important relationships

and you see a lot of
people working together

and not necessarily working very well

and then it's very hard often to get out of this relationship.

A good partnering is a
beautiful thing, like a marriage.

It produces wonderful children,

the fruits of the scholarship are
potentially wonderful

and they exceed the capacity of the
partners to do it on their own

but a bad co-authorship can be very
destructive and time consuming and painful,

just like a bad marriage.

Arguments may start about who did what when

and intellectual property type issues,

especially when it when it goes a little sour

and somebody thinks the other party
is not pulling their weight.

There's more co-authorship
now in economics,

I think that's been documented, much more.

(Guido) Yes.

There's more teams
and there's larger teams

and I think that's great,
I love working on teams.

We do work on schools with big teams.

I work often with PI teammates
like Parag Pathak and David Autor

and then a team of graduate students,

but I see that the students are not always,

in some ways they're a little too promiscuous,

in my view, in their partnering.

They don't think it through.

It's difficult to think it's through.

I think, for me, working
with people always has involved

spending a lot of one-on-one
time with people,

you need to figure out how they think

and what kind of problems are interested

and how they think about these problems,
how they like to write, to make that--

And it takes some maturity on
everybody's part.

Yes. Yes.

In what sense?

Just in the sense of knowing what's going to work for them,

knowing when things are
versus aren't working?

(Josh) Maturity in the
sense of having some judgment

to be able to face it honestly,
if it's not going well,

sometimes you have to have some difficult
discussions.

Is it worth continuing?

"I was hoping you would do this, and you didn't,"

maybe it turns out there's some

feeling in the other direction, the same way.

And Josh is very good
(chuckles)

in the being honest,
part from the beginning,

(Josh) For better or worse.

(Guido) I would write this stuff and then I remember the

first version of the paper with Reuben,

Josh was in Israel at the time,

Don and I were in Cambridge
and so I would talk with Don regularly,

but Don wasn't really doing
much writing in those days,

I would write things and then I would fax them to Josh

and they would come back,
first page just one big cross, No,

second page, one big line, No

and that would go for awhile

but he still does that.

I sent him the first draft of my Nobel lecture,

and Josh goes, No, no!

I've gotten some PDF comments like that from Josh, very helpful.

Omit needless words.

I have few co-authors
who are willing to do that.

Especially as you get older,
it's harder to put up with that.

I would find it harder now to start working with people who did that

early on in a co-author relationship.

It's also very hard because you need to have enough trust.

Josh, for being willing to be very critical,

he was also willing to admit being wrong.

♪ (music) ♪

(Josh) But you have to be on
the lookout for good partners,

somebody who can help you answer
questions that you can't answer yourself.

I think there's a natural
tendency for people to gravitate

to people who are similar in outlook and skills

and that's not as useful

Josh is right, nowadays it's very tempting

to find people who think about the same problems

you're already thinking about,
who think along the same lines

and that may not lead to very novel stuff.

But at the same time finding people
who actually have very different ideas,

it's going to take a lot of time.

Guido, you mentioned in passing how working with Josh has influenced

how you do research,

could you say a little more about that?

I'd also be interested to hear from Josh,
did working with Guido

influence the way that
you do research?

(Guido) Nowadays, I'm much more conscious
of the fact that, for me,

good economic research comes out of
talking to people doing empirical work,

and it's really not reading econometrica

or the reading the stats journals,

but it's actually talking to people
doing empirical work,

going to the empirical seminars.

When I was at Berkeley,

David Carr and [inaudible] as colleagues there

and I would talk to them and listen to them,

trying to figure out

how are they solving their problems
and other things there

where I'm not really quite happy with the way they're doing
things

and trying to look for methodological problems,

where there's some more general solutions possible.

I tried to tell it to my students
that I encourage them to work

as research assistants also,

for the people doing empirical work at Stanford.

There was no [subbing] but that I
learned while I was in graduate school,

but it really came out of working with Josh.

as well as talking to Gary,

Gary us was always encouraging of doing that

and because he done that himself,

he'd worked with on empirical problems with
Zvi Griliches

early in his career.

Yeah.

Well, I became more more interested
in the econometric theory

through our interaction,

and I think empiricists are often impatient
with econometric theory,

partly because empirical work is
very time-consuming,

and you may have a sense that something is

convincing and sensible

and you haven't really fully made the case for that,

but you're convinced

and that motivates you to pursue it,
like the draft lottery story.

I was pretty sure that was
worth doing

and I came away from working with Guido

seeing that there was the potential to say something

more than just about that particular problem,

and I think over the those early
years in the 90s,

our thinking evolved together
that there's actually a framework,

a way to solve a lot of problems

and I think that that is the power of
the late framework,

is it answers a lot of questions in some sense.

♪ (music) ♪

In some sense, did you find that,

email versus facts versus in -person,
the medium mattered

to how collaboration went

or they're ways that you felt like it
was the most useful to collaborate?

To me, I think what matters most is,
initially you have a period--

We needed that initial period,
that was very intense with almost

daily interaction and we also became friends.

You don't develop the kind of friendship,
electronically usually

but once you have that foundation you can be pen pals

and we did use e-mail,
though it wasn't as useful then

but it worked,
but we definitely had a lot of faxes.

I still have these faxes, long faxes

and then in the summer, I would come to Cambridge,

usually to the NBR meetings
and hang around for a few weeks

and you visited me in Israel.

I visited in Israel.

But yeah, there was good foundation from that that year

and in some sense that was enough.

and nowadays,

I have the co-authors
in lots of different places,

but it's always been important

to spend some time with
people in the same place each year.

You understand how they work, how they think,

even to the point that,

you know when they actually respond,
whether they respond quickly or whether that means,

they're not actually doing anything

or that mean they're thinking hard about a problem

and they just take take longer.

but you do need to
develop some understanding there.

♪ (music) ♪

We've talked about
how your collaboration started,

maybe just to step back slightly
were they're sort of features about

the environment at Harvard or in Cambridge,
at the time, which you felt like contributed to it?

Coming from Brown,

I felt it was very intimidating place
because it clearly was a very, very

impressive set of people.

Zvi Griliches was there, Dale Jorgensen--

Gary, Jerry Hausman, Whitney Newey, sometimes Jamie Robins.

I mean, my view of that in retrospect,

I can't say I loved every
minute of every talk

I ever gave in that Workshop,

but that was the highest powered,
that was the group you wanted to reach...

(Guido) Yeah.

and you would get

extraordinarily insightful feedback,
even if it wasn't always easy to swallow.

Yeah, and I have for a while,

I would basically give a talk every semester

because we didn't have any money
to be inviting people.

Gary would say, "Well, why don't you give a talk?"

(laughter)

That was the arena for young people
with our interest.

(Guido) Yeah, it was really very impressive,

but it was quite tough--

It was intimidating.

People there had very strong
views on what they thought was

the way you should do econometrics,
the way the direction things should go,

now, I would think things were
getting a little stale that in fact,

we were bringing in a lot
of the new ideas...

(Josh) Yeah.

...and that wasn't necessary
immediately appreciated.

(Josh) But that's okay.
- And that's fine.

We were pushed and a lot of great discussions
in that workshop about

what should we make of late?

But there were other questions
that were just as interesting,

like the role of the propensity score -

that was a big deal in the 90s

and econometrics was moving towards that

and there were a lot of great questions.

Yeah,

I learned a huge amount
there from the time I spent--

(Josh) I think the other thing that Guido and I

both benefited from
is we both,

not at the same time, but in
early in our careers, taught

econometrics with Gary Chamberlain,

and that was like an
apprenticeship for us, I think.

I taught a mixed graduate undergrad 1126,

I don't know if they still have that number,...

(Isaiah) Ahuh, they do.

...very interesting course that it had both
graduate and undergraduate enrollment

and it was relatively applied for an
econometrics class,

and I learned a lot by teaching that with Gary.

But in that sense, Harvard was a great place, very flexible there.

The other thing I remember about Harvard is,

well I had very good students,

I taught a lot of wonderful students
who went on to have wonderful careers.

Also, Harvard as an institution,
you're probably are aware of this, Isaiah,

as a junior faculty member, they didn't then ask much of us,

other than teaching our classes.

We didn't have administrative concerns, to speak of.

I think I went to two faculty
meetings in my two years at Harvard

and so we're left--

You were given a lot of freedom and flexibility.

Yeah. Yeah. So I went to the chair said,
you know, can I teach this course Reuben?

And I think it was Friedman
at the time. It was like fine.

Yeah,

it wasn't really any concern about what
what it was about and again that was

very intimidating, experience,
but it was a great experience.

If you'd like to watch more
Nobel conversations, click here.