-
Belle Gibson was a happy young Australian.
-
She lived in Perth
and she loved skateboarding.
-
But in 2009, Belle learned that she had
brain cancer and four months to live.
-
Two months of chemo
and radiotherapy had no effect.
-
But Belle was determined.
-
She'd been a fighter her whole life.
-
From age six, she had to cook
for her brother, who had autism,
-
and her mother,
who had multiple sclerosis.
-
Her father was out of the picture.
-
So Belle fought, with exercise,
with meditation,
-
and by ditching meat
for fruit and vegetables.
-
And she made a complete recovery.
-
Belle's story went viral.
-
It was tweeted, blogged about,
shared and reached millions of people.
-
It showed the benefits of shunning
traditional medicine
-
for diet and exercise.
-
In August 2013, Belle launched
a healthy eating app,
-
The Whole Pantry,
-
downloaded 200,000 times
in the first month.
-
But Belle's story was a lie.
-
Belle never had cancer.
-
People shared her story
without ever checking if it was true.
-
This is a classic example
of confirmation bias.
-
We accept a story uncritically
if it confirms what we'd like to be true.
-
And we reject any story
that contradicts it.
-
How often do we see this
-
in the stories
that we share and we ignore?
-
In politics, in business,
in health advice.
-
The Oxford Dictionary's
word of 2016 was "post-truth."
-
And the recognition that we now live
in a post-truth world
-
has led to a much needed emphasis
on checking the facts.
-
But the punch line of my talk
-
is that just checking
the facts is not enough.
-
Even if Belle's story were true,
-
it would be just as irrelevant.
-
Why?
-
Well, let's look at one of the most
fundamental techniques in statistics.
-
It's called Bayesian inference.
-
And the very simple version is this:
-
We care about "does the data
support the theory?"
-
Does the data increase our belief
that the theory is true?
-
But instead, we end up asking,
"Is the data consistent with the theory?"
-
But being consistent with the theory
-
does not mean that the data
supports the theory.
-
Why?
-
Because of a crucial
but forgotten third term --
-
the data could also be consistent
with rival theories.
-
But due to confirmation bias,
we never consider the rival theories,
-
because we're so protective
of our own pet theory.
-
Now, let's look at this for Belle's story.
-
Well, we care about:
Does Belle's story support the theory
-
that diet cures cancer?
-
But instead, we end up asking,
-
"Is Belle's story consistent
with diet curing cancer?"
-
And the answer is yes.
-
If diet did cure cancer,
we'd see stories like Belle's.
-
But even if diet did not cure cancer,
-
we'd still see stories like Belle's.
-
A single story in which
a patient apparently self-cured
-
just due to being misdiagnosed
in the first place.
-
Just like, even if smoking
was bad for your health,
-
you'd still see one smoker
who lived until 100.
-
(Laughter)
-
Just like, even if education
was good for your income,
-
you'd still see one multimillionaire
who didn't go to university.
-
(Laughter)
-
So the biggest problem with Belle's story
is not that it was false.
-
It's that it's only one story.
-
There might be thousands of other stories
where diet alone failed,
-
but we never hear about them.
-
We share the outlier cases
because they are new,
-
and therefore they are news.
-
We never share the ordinary cases.
-
They're too ordinary,
they're what normally happens.
-
And that's the true
99 percent that we ignore.
-
Just like in society, you can't just
listen to the one percent,
-
the outliers,
-
and ignore the 99 percent, the ordinary.
-
Because that's the second example
of confirmation bias.
-
We accept a fact as data.
-
The biggest problem is not
that we live in a post-truth world,
-
it's that we live in a post-data world.
-
We prefer a single story to tons of data.
-
Now, stories are powerful,
they're vivid, they bring it to life.
-
They tell you to start
every talk with a story.
-
I did.
-
But a single story
is meaningless and misleading
-
unless it's backed up by large-scale data.
-
But even if we had large-scale data,
-
that might still not be enough.
-
Because it could still be consistent
with rival theories.
-
Let me explain.
-
A classic study
by psychologist Peter Wason
-
gives you a set of three numbers
-
and asks you to think of the rule
that generated them.
-
So if you're given two, four, six,
-
what's the rule?
-
Well, most people would think,
it's successive even numbers.
-
How would you test it?
-
Well, you'd propose other sets
of successive even numbers:
-
four, six, eight or 12, 14, 16.
-
And Peter would say these sets also work.
-
But knowing that these sets also work,
-
knowing that perhaps hundreds of sets
of successive even numbers also work,
-
tells you nothing.
-
Because this is still consistent
with rival theories.
-
Perhaps the rule
is any three even numbers.
-
Or any three increasing numbers.
-
And that's the third example
of confirmation bias:
-
accepting data as evidence,
-
even if it's consistent
with rival theories.
-
Data is just a collection of facts.
-
Evidence is data that supports
one theory and rules out others.
-
So the best way to support your theory
-
is actually to try to disprove it,
to play devil's advocate.
-
So test something, like four, 12, 26.
-
If you got a yes to that,
that would disprove your theory
-
of successive even numbers.
-
Yet this test is powerful,
-
because if you got a no, it would rule out
"any three even numbers,"
-
and "any three increasing numbers,"
-
it would rule out the rival theories,
but not rule out yours.
-
But most people are too afraid
of testing the four, 12, 26,
-
because they don't want to get a yes
and prove their pet theory to be wrong.
-
Confirmation bias is not only
about failing to search for new data,
-
but it's also about misinterpreting
data once you receive it.
-
And this applies outside the lab
to important, real-world problems.
-
Indeed, Thomas Edison famously said,
-
"I have not failed,
-
I have found 10,000 ways that won't work."
-
Finding out that you're wrong
-
is the only way to find out what's right.
-
Say you're a university
admissions director
-
and your theory is that only
students with good grades
-
from rich families do well.
-
So you only let in such students.
-
And they do well.
-
But that's also consistent
with the rival theory.
-
Perhaps all students
with good grades do well,
-
rich or poor.
-
But you never test that theory,
because you never let in poor students,
-
because you don't want to be proven wrong.
-
So, what have we learned?
-
A story is not fact,
because it may not be true.
-
A fact is not data,
-
it may not be representative
if it's only one data point.
-
And data is not evidence --
-
it may not be supportive
if it's consistent with rival theories.
-
So, what do you do?
-
When you're at
the inflection points of life,
-
deciding on a strategy for your business,
-
a parenting technique for your child
-
or a regimen for your health,
-
how do you ensure
that you don't have a story,
-
but you have evidence?
-
Let me give you three tips.
-
The first is to actively seek
other viewpoints.
-
Read and listen to people
you flagrantly disagree with.
-
Ninety percent of what they say
may be wrong, in your view.
-
But what if 10 percent is right?
-
As Aristotle said,
-
"The mark of an educated man
-
is the ability to entertain a thought
-
without necessarily accepting it."
-
Surround yourself with people
who challenge you
-
and create a culture
that actively encourages dissent.
-
Some banks suffered from groupthink,
-
where staff were too afraid to challenge
management's lending decisions,
-
contributing to the financial crisis.
-
In a meeting, appoint someone
to be devil's advocate
-
against your pet idea.
-
And don't just hear another viewpoint --
-
listen to it, as well.
-
As psychologist Stephen Covey said,
-
"Listen with the intent to understand,
-
not the intent to reply."
-
A dissenting viewpoint
is something to learn from,
-
not to argue against.
-
Which takes us to the other
forgotten terms in Bayesian inference.
-
Because data allows you to learn,
-
but learning is only relative
to a starting point.
-
If you started with complete certainty
that your pet theory must be true,
-
then your view won't change --
-
regardless of what data you see.
-
Only if you are truly open
to the possibility of being wrong
-
can you ever learn.
-
As Leo Tolstoy wrote,
-
"The most difficult subjects
-
can be explained to the most
slow-witted man
-
if he has not formed
any idea of them already.
-
But the simplest thing
-
cannot be made clear
to the most intelligent man
-
if he is firmly persuaded
that he knows already."
-
Tip number two is "listen to experts."
-
Now, that's perhaps the most
unpopular advice that I could give you.
-
(Laughter)
-
British politician Michael Gove
famously said that people in this country
-
have had enough of experts.
-
A recent poll showed that more people
would trust their hairdresser --
-
(Laughter)
-
or the man on the street
-
than they would leaders of businesses,
the health service, and even charities.
-
So we respect a teeth-whitening formula
discovered by a mom,
-
or we listen to an actress's view
on vaccination.
-
We like people who tell it like it is,
who go with their gut,
-
and we call them authentic.
-
But gut feel can only get you so far.
-
Gut feel would tell you never to give
water to a baby with diarrhea,
-
because it would just
flow out the other end.
-
Expertise tells you otherwise.
-
You'd never trust your surgery
to the man on the street.
-
You'd want an expert
who spent years doing surgery
-
and knows the best techniques.
-
But that should apply
to every major decision.
-
Politics, business, health advice
-
require expertise, just like surgery.
-
So then, why are experts so mistrusted?
-
Well, one reason
is they're seen as out of touch.
-
A millionaire CEO couldn't possibly
speak for the man on the street.
-
But true expertise is found on evidence.
-
And evidence stands up
for the man on the street
-
and against the elites.
-
Because evidence forces you to prove it.
-
Evidence prevents the elites
from imposing their own view
-
without proof.
-
A second reason
why experts are not trusted
-
is that different experts
say different things.
-
For every expert who claimed that leaving
the EU would be bad for Britain,
-
another expert claimed it would be good.
-
Half of these so-called experts
will be wrong.
-
And I have to admit that most papers
written by experts are wrong.
-
Or at best, make claims that
the evidence doesn't actually support.
-
So we can't just take
an expert's word for it.
-
In November 2016, a study
on executive pay hit national headlines.
-
Even though none of the newspapers
who covered the study
-
had even seen the study.
-
It wasn't even out yet.
-
They just took the author's word for it,
-
just like with Belle.
-
Nor does it mean that we can
just handpick any study
-
that happens to support our viewpoint --
-
that would, again, be confirmation bias.
-
Nor does it mean
that if seven studies show A,
-
and three show B,
-
that A must be true.
-
What matters is the quality,
-
and not the quantity of expertise.
-
So we should do two things.
-
First, we should critically examine
the credentials of the authors.
-
Just like you'd critically examine
the credentials of a potential surgeon.
-
Are they truly experts in the matter,
-
or do they have a vested interest?
-
Second, we should pay particular attention
-
to papers published
in the top academic journals.
-
Now, academics are often accused
of being detached from the real world.
-
But this detachment gives you
years to spend on a study.
-
To really nail down a result,
-
to rule out those rival theories,
-
and to distinguish correlation
from causation.
-
And academic journals involve peer review,
-
where a paper is rigorously scrutinized
-
(Laughter)
-
by the world's leading minds.
-
The better the journal,
the higher the standard.
-
The most elite journals
reject 95 percent of papers.
-
Now, academic evidence is not everything.
-
Real-world experience is critical, also.
-
And peer review is not perfect,
mistakes are made.
-
But it's better to go
with something checked
-
than something unchecked.
-
If we latch onto a study
because we like the findings,
-
without considering who it's by
or whether it's even been vetted,
-
there is a massive chance
that that study is misleading.
-
And those of us who claim to be experts
-
should recognize the limitations
of our analysis.
-
Very rarely is it possible to prove
or predict something with certainty,
-
yet it's so tempting to make
a sweeping, unqualified statement.
-
It's easier to turn into a headline
or to be tweeted in 140 characters.
-
But even evidence may not be proof.
-
It may not be universal,
it may not apply in every setting.
-
So don't say, "Red wine
causes longer life,"
-
when the evidence is only that red wine
is correlated with longer life.
-
And only then in people
who exercise, as well.
-
Tip number three
is "pause before sharing anything."
-
The Hippocratic oath says,
"first, do no harm."
-
What we share is potentially contagious,
-
so be very careful about what we spread.
-
Our goal should not be
to get likes or retweets.
-
Otherwise, we only share the consensus,
we don't challenge anyone's thinking.
-
Otherwise, we only share what sounds good,
-
regardless of whether it's evidence.
-
Instead, we should ask the following:
-
If it's a story, is it true?
-
If it's true, is it backed up
by large-scale evidence?
-
If it is, who is it by,
what are their credentials,
-
is it published,
how rigorous is the journal?
-
And ask yourself
the million-dollar question:
-
If the same study was written by the same
authors with the same credentials,
-
but found the opposite results,
-
would you still be willing
to believe it and to share it?
-
Treating any problem --
-
a nation's economic problem
or an individual's health problem,
-
is difficult.
-
So we must ensure that we have
the very best evidence to guide us.
-
Only if it's true can it be fact.
-
Only if it's representative
can it be data.
-
Only if it's supportive
can it be evidence.
-
And only with evidence
can we move from a post-truth world
-
to a pro-truth world.
-
Thank you very much.
-
(Applause)