1
99:59:59,999 --> 99:59:59,999
Health headlines like these are published
every day,

2
99:59:59,999 --> 99:59:59,999
sometimes making opposite claims 
from each other.

3
99:59:59,999 --> 99:59:59,999
There can be a disconnect between broad,

4
99:59:59,999 --> 99:59:59,999
attention-grabbing headlines 
and the often specific,

5
99:59:59,999 --> 99:59:59,999
incremental results of the medical 
research they cover.

6
99:59:59,999 --> 99:59:59,999
So how can you avoid being 
misled by grabby headlines?

7
99:59:59,999 --> 99:59:59,999
The best way to assess a headline’s 
credibility

8
99:59:59,999 --> 99:59:59,999
is to look at the original 
research it reports on.

9
99:59:59,999 --> 99:59:59,999
We’ve come up with a hypothetical research
scenario

10
99:59:59,999 --> 99:59:59,999
for each of these three headlines.

11
99:59:59,999 --> 99:59:59,999
Keep watching for the explanation
of the first example;

12
99:59:59,999 --> 99:59:59,999
then pause at the headline to 
answer the question.

13
99:59:59,999 --> 99:59:59,999
These are simplified scenarios.

14
99:59:59,999 --> 99:59:59,999
A real study would detail many more 
factors and how it accounted for them,

15
99:59:59,999 --> 99:59:59,999
but for the purposes of this exercise,

16
99:59:59,999 --> 99:59:59,999
assume all the information 
you need is included.

17
99:59:59,999 --> 99:59:59,999
Let’s start by considering the 
cardiovascular effects

18
99:59:59,999 --> 99:59:59,999
of a certain vitamin, Healthium.

19
99:59:59,999 --> 99:59:59,999
The study finds that participants taking
Healthium

20
99:59:59,999 --> 99:59:59,999
had a higher level of healthy cholesterol 
than those taking a placebo.

21
99:59:59,999 --> 99:59:59,999
Their levels became similar to those of 
people with naturally high levels

22
99:59:59,999 --> 99:59:59,999
of this kind of cholesterol.

23
99:59:59,999 --> 99:59:59,999
Previous research has shown that people 
with naturally high levels

24
99:59:59,999 --> 99:59:59,999
of healthy cholesterol have lower 
rates of heart disease.

25
99:59:59,999 --> 99:59:59,999
So what makes this headline misleading:

26
99:59:59,999 --> 99:59:59,999
Healthium reduces risk of heart disease.

27
99:59:59,999 --> 99:59:59,999
The problem with this headline is that the
research didn’t actually investigate

28
99:59:59,999 --> 99:59:59,999
whether Healthium reduces heart disease.

29
99:59:59,999 --> 99:59:59,999
It only measured Healthium’s impact

30
99:59:59,999 --> 99:59:59,999
on levels of a particular 
kind of cholesterol.

31
99:59:59,999 --> 99:59:59,999
The fact that people with naturally high 
levels of that cholesterol

32
99:59:59,999 --> 99:59:59,999
have lower risk of heart attacks

33
99:59:59,999 --> 99:59:59,999
doesn’t mean that the same will be true

34
99:59:59,999 --> 99:59:59,999
of people who elevate their cholesterol
levels using Healthium.

35
99:59:59,999 --> 99:59:59,999
Now that you’ve cracked the 
case of Healthium,

36
99:59:59,999 --> 99:59:59,999
try your hand at a particularly alluring 
mystery:

37
99:59:59,999 --> 99:59:59,999
the relationship between eating chocolate
and stress.

38
99:59:59,999 --> 99:59:59,999
This hypothetical study 
recruits ten students.

39
99:59:59,999 --> 99:59:59,999
Half begin consuming a 
daily dose of chocolate,

40
99:59:59,999 --> 99:59:59,999
while half abstain.

41
99:59:59,999 --> 99:59:59,999
As classmates, they all follow 
the same schedule.

42
99:59:59,999 --> 99:59:59,999
By the end of the study, the chocolate 
eaters are less stressed

43
99:59:59,999 --> 99:59:59,999
than their chocolate-free counterparts.

44
99:59:59,999 --> 99:59:59,999
What’s wrong with this headline:

45
99:59:59,999 --> 99:59:59,999
Eating chocolate reduces 
stress in students

46
99:59:59,999 --> 99:59:59,999
It’s a stretch to draw a conclusion about 
students in general from a sample of ten.

47
99:59:59,999 --> 99:59:59,999
That’s because the fewer participants are
in a random sample,

48
99:59:59,999 --> 99:59:59,999
the less likely it is that the sample will
closely represent

49
99:59:59,999 --> 99:59:59,999
the target population as a whole.

50
99:59:59,999 --> 99:59:59,999
For example, if the broader population of 
students is half male and half female,

51
99:59:59,999 --> 99:59:59,999
the chance of drawing a sample of 10

52
99:59:59,999 --> 99:59:59,999
that’s skewed 70% male and 
30% is about 12%.

53
99:59:59,999 --> 99:59:59,999
In a sample of 100 that would be less than
a .0025% chance,

54
99:59:59,999 --> 99:59:59,999
and for a sample of 1000,

55
99:59:59,999 --> 99:59:59,999
the odds are less than 6 x 10^-36.

56
99:59:59,999 --> 99:59:59,999
Similarly, with fewer participants,

57
99:59:59,999 --> 99:59:59,999
each individual’s outcome has a larger 
impact on the overall results—

58
99:59:59,999 --> 99:59:59,999
and can therefore skew big-picture trends.

59
99:59:59,999 --> 99:59:59,999
Still, there are a lot of good reasons for
scientists run small studies.

60
99:59:59,999 --> 99:59:59,999
By starting with a small sample,

61
99:59:59,999 --> 99:59:59,999
they can evaluate whether the results are 
promising enough

62
99:59:59,999 --> 99:59:59,999
to run a more comprehensive, 
expensive study.

63
99:59:59,999 --> 99:59:59,999
And some research requires very specific
participants

64
99:59:59,999 --> 99:59:59,999
that may be impossible to 
recruit in large numbers.

65
99:59:59,999 --> 99:59:59,999
The key is reproducibility—

66
99:59:59,999 --> 99:59:59,999
if an article draws a conclusion 
from one small study,

67
99:59:59,999 --> 99:59:59,999
that conclusion may be suspect—

68
99:59:59,999 --> 99:59:59,999
but if it’s based on many studies 
that have found similar results,

69
99:59:59,999 --> 99:59:59,999
it’s more credible.

70
99:59:59,999 --> 99:59:59,999
We’ve still got one more puzzle.

71
99:59:59,999 --> 99:59:59,999
In this scenario, a study tests a new drug
for a rare, fatal disease.

72
99:59:59,999 --> 99:59:59,999
In a sample of 2,000 patients,

73
99:59:59,999 --> 99:59:59,999
the ones who start taking the drug upon 
diagnosis live longer

74
99:59:59,999 --> 99:59:59,999
than those who take the placebo.

75
99:59:59,999 --> 99:59:59,999
This time, the question 
is slightly different.

76
99:59:59,999 --> 99:59:59,999
What’s one more thing you’d like to know 
before deciding if the headline,

77
99:59:59,999 --> 99:59:59,999
New drug prolongs lives of patients
with rare disease, is justified?

78
99:59:59,999 --> 99:59:59,999
Before making this call,

79
99:59:59,999 --> 99:59:59,999
you’d want to know how much the drug 
prolonged the patients’ lives.

80
99:59:59,999 --> 99:59:59,999
Sometimes, a study can have results that,

81
99:59:59,999 --> 99:59:59,999
while scientifically valid, don’t have 
much bearing on real world outcomes.

82
99:59:59,999 --> 99:59:59,999
For example, one real-life clinical trial 
of a pancreatic cancer drug

83
99:59:59,999 --> 99:59:59,999
found an increase in life expectancy—
of ten days.

84
99:59:59,999 --> 99:59:59,999
The next time you see a surprising medical
headline,

85
99:59:59,999 --> 99:59:59,999
take a look at the science 
it’s reporting on.

86
99:59:59,999 --> 99:59:59,999
Even when full papers aren’t 
available without a fee,

87
99:59:59,999 --> 99:59:59,999
you can often find summaries of 
experimental design

88
99:59:59,999 --> 99:59:59,999
and results in freely available abstracts,

89
99:59:59,999 --> 99:59:59,999
or even within the text 
of a news articles.

90
99:59:59,999 --> 99:59:59,999
It’s exciting to see scientific research 
covered in the news,

91
99:59:59,999 --> 99:59:59,999
and important to understand 
the studies’ findings.