-
Not Synced
- [Narrator] On his quest
to master econometrics,
-
Not Synced
Grasshopper Kamal has
made great progress
-
Not Synced
stretching his capabilities
and outsmarting his foes.
-
Not Synced
Alas, today he's despondent.
-
Not Synced
For one challenge remains unmet.
-
Not Synced
Kamal cannot yet decode
the scriptures of academic research,
-
Not Synced
journals like
"The American Economic Review"
-
Not Synced
and "Econometrica."
-
Not Synced
These seemed to him to be inscribed
in an obscure foreign tongue.
-
Not Synced
- [Kamal] Ugh, what the... ?
-
Not Synced
- [Narrator] These volumes are
opaque to novice, Kamal,
-
Not Synced
but can be deciphered with study.
-
Not Synced
Let us learn to read them together.
-
Not Synced
Let's dive into the West Point study,
-
Not Synced
published in the "Economics
of Education Review."
-
Not Synced
This paper reports
on a randomized evaluation
-
Not Synced
of student electronics use
in Economics 101 classrooms.
-
Not Synced
First, a quick review
of the research design.
-
Not Synced
- Okay.
-
Not Synced
- [Josh] 'Metrics masters
teaching at West Point,
-
Not Synced
the military college that trains
American Army officers
-
Not Synced
designed a randomized trial
to answer this question.
-
Not Synced
These masters randomly assigned
West Point cadets
-
Not Synced
into Economics classes
operating under different rules.
-
Not Synced
Unlike most American colleges,
-
Not Synced
the West Point default
is no electronics.
-
Not Synced
For purposes of this experiment,
some students were left
-
Not Synced
in such traditional
technology-free classes,
-
Not Synced
no lap tops, no tablets
and no phones!
-
Not Synced
[voice echoes]
-
Not Synced
This is the control group,
or baseline case.
-
Not Synced
Another group was allowed
to use electronics.
-
Not Synced
This is the treatment group,
subject to a changed environment.
-
Not Synced
The treatment in this case
is the unrestricted use
-
Not Synced
of laptops or tablets in class.
-
Not Synced
Every causal question
has a clear outcome,
-
Not Synced
the variables we hope to influence
defined in advance of the study.
-
Not Synced
The outcomes in the West Point
electronics study
-
Not Synced
are final exam scores.
-
Not Synced
The study seeks to answer
the following question,
-
Not Synced
what is the causal effect
of classroom electronics on learning
-
Not Synced
as measured by exam scores?
-
Not Synced
- Economics journal articles
usually begin with a table
-
Not Synced
of descriptive statistics,
giving key facts
-
Not Synced
about the study sample.
-
Not Synced
- Oh my gosh, I remember this table,
so confusing!
-
Not Synced
Columns 1 to 3 report mean,
or average, characteristics.
-
Not Synced
These give a sense
of who we're studying.
-
Not Synced
Let's start with column 1
which describes covariates
-
Not Synced
in a control group.
-
Not Synced
Covariates are characteristics
of the control and treatment groups
-
Not Synced
measured before
the experiment begins.
-
Not Synced
For example, we see the control group
has an average age a bit over 20.
-
Not Synced
Many of these covariates
are dummy variables.
-
Not Synced
A dummy variable can only have
two values, a zero or a one.
-
Not Synced
For example, student gender
is captured by a dummy variable
-
Not Synced
that equals one for woman
and zero for man.
-
Not Synced
The mean of this variable
is the proportion female.
-
Not Synced
We also see that the control group
is 13% Hispanic
-
Not Synced
and 19% had prior military service.
-
Not Synced
The table notes are key.
-
Not Synced
Refer to these
as you scan the table.
-
Not Synced
These notes explain what's shown
in each column and panel.
-
Not Synced
The notes tell us, for example,
that standard deviations
-
Not Synced
are reported in brackets.
-
Not Synced
Standard deviations tell us how
spread out the data are.
-
Not Synced
For example, a standard deviation
of 0.52 tells us that most
-
Not Synced
of the control group's GPAs
fall between 2.35,
-
Not Synced
which is 0.52 below
the mean GPA of 2.87,
-
Not Synced
and 3.39, which is 0.52 above 2.87.
-
Not Synced
A lower standard deviation
would mean the GPAs were
-
Not Synced
more tightly clustered
around the mean.
-
Not Synced
- [Kamal] Yeah, but they're missing
from most of the variables.
-
Not Synced
- [Narrator] That's right.
-
Not Synced
Masters usually omit
standard deviations for dummies
-
Not Synced
because the mean of this variable
determines its standard deviaton.
-
Not Synced
This study compares two treatment
groups with the control group.
-
Not Synced
The first was allowed free use
of laptops and tablets.
-
Not Synced
The second treatment
was more restrictive,
-
Not Synced
allowing only tablets placed
flat on the desk.
-
Not Synced
The treatment groups
look much like the control group.
-
Not Synced
Hmmm...
-
Not Synced
This takes us to the next feature
of this table, columns 4 through 6
-
Not Synced
use statistical tests to compare
the characteristics
-
Not Synced
of the treatment and control group
before the experiment.
-
Not Synced
In column 4, the two treatment
groups are combined.
-
Not Synced
You can see that the difference
in proportion female
-
Not Synced
between the treatment
and control group is only 0.03.
-
Not Synced
The difference is not
statistically significant.
-
Not Synced
It is the sort of difference
we can easily put down
-
Not Synced
to chance results
in our sample selection process.
-
Not Synced
- [Kamal] Hmm, how do we know that?
-
Not Synced
- [Narrator] Remember
the rule of thumb?
-
Not Synced
Statistical estimates
that exceed the standard
-
Not Synced
by a multiple of 2
in absolute value
-
Not Synced
are usually said
to be statistically significant.
-
Not Synced
The standard error is 0.03,
-
Not Synced
same as the difference
in proportion female.
-
Not Synced
So the ratio of the latter
to the former is only 1,
-
Not Synced
which of course is less than 2.
-
Not Synced
- [Kamal] Uh huh. So none
of the treatment/control differences
-
Not Synced
in the table are more than twice
their standard errors.
-
Not Synced
- [Narrator] Correct.
-
Not Synced
The random division of students
appears to have succeeded
-
Not Synced
in creating groups
that are indeed comparable.
-
Not Synced
We can be confident therefore
that any later differences
-
Not Synced
in classroom achievement
are the result of the experimental
-
Not Synced
intervention rather
than a reflection
-
Not Synced
of preexisting differences.
-
Not Synced
Ceteris paribus achieved!
-
Not Synced
- [Kamal] Cool. Wait,
what about the bottom,
-
Not Synced
the numbers with the stars?
-
Not Synced
Those differences are a lot more
than double the standard error.
-
Not Synced
- [Narrator] Good eye, Kamal!
-
Not Synced
The table has many numbers.
-
Not Synced
Those in Panel B are important too.
-
Not Synced
This panel measures the extent
to which students in treatment
-
Not Synced
and control groups actually use
computers in class.