9:59:59.000,9:59:59.000
One simple vitamin can reduce your risk of heart disease.

9:59:59.000,9:59:59.000
Eating chocolate reduces stress in students.

9:59:59.000,9:59:59.000
New drug prolongs lives of patients with rare disease.

9:59:59.000,9:59:59.000
Health headlines like these are published[br]every day,

9:59:59.000,9:59:59.000
sometimes making opposite claims [br]from each other.

9:59:59.000,9:59:59.000
There can be a disconnect between broad,

9:59:59.000,9:59:59.000
attention-grabbing headlines [br]and the often specific,

9:59:59.000,9:59:59.000
incremental results of the medical [br]research they cover.

9:59:59.000,9:59:59.000
So how can you avoid being [br]misled by grabby headlines?

9:59:59.000,9:59:59.000
The best way to assess a headline’s [br]credibility

9:59:59.000,9:59:59.000
is to look at the original [br]research it reports on.

9:59:59.000,9:59:59.000
We’ve come up with a hypothetical research[br]scenario

9:59:59.000,9:59:59.000
for each of these three headlines.

9:59:59.000,9:59:59.000
Keep watching for the explanation[br]of the first example;

9:59:59.000,9:59:59.000
then pause at the headline to [br]answer the question.

9:59:59.000,9:59:59.000
These are simplified scenarios.

9:59:59.000,9:59:59.000
A real study would detail many more [br]factors and how it accounted for them,

9:59:59.000,9:59:59.000
but for the purposes of this exercise,

9:59:59.000,9:59:59.000
assume all the information [br]you need is included.

9:59:59.000,9:59:59.000
Let’s start by considering the [br]cardiovascular effects

9:59:59.000,9:59:59.000
of a certain vitamin, Healthium.

9:59:59.000,9:59:59.000
The study finds that participants taking[br]Healthium

9:59:59.000,9:59:59.000
had a higher level of healthy cholesterol [br]than those taking a placebo.

9:59:59.000,9:59:59.000
Their levels became similar to those of [br]people with naturally high levels

9:59:59.000,9:59:59.000
of this kind of cholesterol.

9:59:59.000,9:59:59.000
Previous research has shown that people [br]with naturally high levels

9:59:59.000,9:59:59.000
of healthy cholesterol have lower [br]rates of heart disease.

9:59:59.000,9:59:59.000
So what makes this headline misleading:

9:59:59.000,9:59:59.000
Healthium reduces risk of heart disease.

9:59:59.000,9:59:59.000
The problem with this headline is that the[br]research didn’t actually investigate

9:59:59.000,9:59:59.000
whether Healthium reduces heart disease.

9:59:59.000,9:59:59.000
It only measured Healthium’s impact

9:59:59.000,9:59:59.000
on levels of a particular [br]kind of cholesterol.

9:59:59.000,9:59:59.000
The fact that people with naturally high [br]levels of that cholesterol

9:59:59.000,9:59:59.000
have lower risk of heart attacks

9:59:59.000,9:59:59.000
doesn’t mean that the same will be true

9:59:59.000,9:59:59.000
of people who elevate their cholesterol[br]levels using Healthium.

9:59:59.000,9:59:59.000
Now that you’ve cracked the [br]case of Healthium,

9:59:59.000,9:59:59.000
try your hand at a particularly alluring [br]mystery:

9:59:59.000,9:59:59.000
the relationship between eating chocolate[br]and stress.

9:59:59.000,9:59:59.000
This hypothetical study [br]recruits ten students.

9:59:59.000,9:59:59.000
Half begin consuming a [br]daily dose of chocolate,

9:59:59.000,9:59:59.000
while half abstain.

9:59:59.000,9:59:59.000
As classmates, they all follow [br]the same schedule.

9:59:59.000,9:59:59.000
By the end of the study, the chocolate [br]eaters are less stressed

9:59:59.000,9:59:59.000
than their chocolate-free counterparts.

9:59:59.000,9:59:59.000
What’s wrong with this headline:

9:59:59.000,9:59:59.000
Eating chocolate reduces [br]stress in students

9:59:59.000,9:59:59.000
It’s a stretch to draw a conclusion about [br]students in general from a sample of ten.

9:59:59.000,9:59:59.000
That’s because the fewer participants are[br]in a random sample,

9:59:59.000,9:59:59.000
the less likely it is that the sample will[br]closely represent

9:59:59.000,9:59:59.000
the target population as a whole.

9:59:59.000,9:59:59.000
For example, if the broader population of [br]students is half male and half female,

9:59:59.000,9:59:59.000
the chance of drawing a sample of 10

9:59:59.000,9:59:59.000
that’s skewed 70% male and [br]30% is about 12%.

9:59:59.000,9:59:59.000
In a sample of 100 that would be less than[br]a .0025% chance,

9:59:59.000,9:59:59.000
and for a sample of 1000,

9:59:59.000,9:59:59.000
the odds are less than 6 x 10^-36.

9:59:59.000,9:59:59.000
Similarly, with fewer participants,

9:59:59.000,9:59:59.000
each individual’s outcome has a larger [br]impact on the overall results—

9:59:59.000,9:59:59.000
and can therefore skew big-picture trends.

9:59:59.000,9:59:59.000
Still, there are a lot of good reasons for[br]scientists run small studies.

9:59:59.000,9:59:59.000
By starting with a small sample,

9:59:59.000,9:59:59.000
they can evaluate whether the results are [br]promising enough

9:59:59.000,9:59:59.000
to run a more comprehensive, [br]expensive study.

9:59:59.000,9:59:59.000
And some research requires very specific[br]participants

9:59:59.000,9:59:59.000
that may be impossible to [br]recruit in large numbers.

9:59:59.000,9:59:59.000
The key is reproducibility—

9:59:59.000,9:59:59.000
if an article draws a conclusion [br]from one small study,

9:59:59.000,9:59:59.000
that conclusion may be suspect—

9:59:59.000,9:59:59.000
but if it’s based on many studies [br]that have found similar results,

9:59:59.000,9:59:59.000
it’s more credible.

9:59:59.000,9:59:59.000
We’ve still got one more puzzle.

9:59:59.000,9:59:59.000
In this scenario, a study tests a new drug[br]for a rare, fatal disease.

9:59:59.000,9:59:59.000
In a sample of 2,000 patients,

9:59:59.000,9:59:59.000
the ones who start taking the drug upon [br]diagnosis live longer

9:59:59.000,9:59:59.000
than those who take the placebo.

9:59:59.000,9:59:59.000
This time, the question [br]is slightly different.

9:59:59.000,9:59:59.000
What’s one more thing you’d like to know [br]before deciding if the headline,

9:59:59.000,9:59:59.000
New drug prolongs lives of patients[br]with rare disease, is justified?

9:59:59.000,9:59:59.000
Before making this call,

9:59:59.000,9:59:59.000
you’d want to know how much the drug [br]prolonged the patients’ lives.

9:59:59.000,9:59:59.000
Sometimes, a study can have results that,

9:59:59.000,9:59:59.000
while scientifically valid, don’t have [br]much bearing on real world outcomes.

9:59:59.000,9:59:59.000
For example, one real-life clinical trial [br]of a pancreatic cancer drug

9:59:59.000,9:59:59.000
found an increase in life expectancy—[br]of ten days.

9:59:59.000,9:59:59.000
The next time you see a surprising medical[br]headline,

9:59:59.000,9:59:59.000
take a look at the science [br]it’s reporting on.

9:59:59.000,9:59:59.000
Even when full papers aren’t [br]available without a fee,

9:59:59.000,9:59:59.000
you can often find summaries of [br]experimental design

9:59:59.000,9:59:59.000
and results in freely available abstracts,

9:59:59.000,9:59:59.000
or even within the text [br]of a news articles.

9:59:59.000,9:59:59.000
It’s exciting to see scientific research [br]covered in the news,

9:59:59.000,9:59:59.000
and important to understand [br]the studies’ findings.