The Joy of Stats

Documentary which takes viewers on a rollercoaster ride through the wonderful world of statistics to explore the remarkable power they have to change our understanding of the world, presented by superstar boffin Professor Hans Rosling, whose eye-opening, mind-expanding and funny online lectures have made him an international internet legend.

The world we live in is awash with
data that comes pouring in
from everywhere around us.

On its own this data
is just noise and confusion.

To make sense of data, to find the
meaning in it, we need the powerful
branch of science - statistics.

Believe me there's nothing
boring about statistics.

Especially not today
when we can make the data sing.

With statistics we can
really make sense of the world.

And there's more.

With statistics, the data deluge, as
it's being called, is leading us

to an ever greater understanding
of life on Earth
and the universe beyond.

And thanks to the incredible
power of today's computers,

it may fundamentally transform the
process of scientific discovery.

I kid you not, statistics is
now the sexiest subject around.

Did you know that there is
one million boats in Sweden?

That's one boat per nine people!

It's the highest number of
boats per person in Europe!

Being a statistician,
you don't like telling
your profession at dinner parties.

But really,
statisticians shouldn't be shy

because everyone wants to
understand what's going on.

And statistics gives us a
perspective on the world we live in

that we can't get in any other way.

Statistics tells us whether
the things we think
and believe are actually true.

And statistics are far more useful
than we usually like to admit.

In the last recession there
was this famous call-in
to a talk radio station.

The man complained, "In times like
this when unemployment rates are up
to 13%, income has fallen by 5%,

"and suicide rates are climbing, and
I get so angry that the government

"is wasting money on things like
collection of statistics."

I'm not officially a statistician.

Strictly speaking,
my field is global health.

But I got really obsessed with stats
when I realised how much people

in Sweden just don't know
about the rest of the world.

I started in our medical
university, Karolinska Institutet,

an undergraduate course
called Global Health.

These students coming to us actually
have the highest grade you can get

in the Swedish college system,

so I thought, "Maybe they know
everything I'm going to teach them."

So I did a pre-test when they came,
and one of the questions

from which I learned a lot
was this one -

which country has the highest
child mortality of these five pairs?

I won't put you at test here,
but it is Turkey

which is highest there, Poland,

Russia, Pakistan, and South Africa.

And these were the result of
the Swedish students.

A 1.8 right answer
out of five possible.

And that means there was a place for
a professor of International Health
and for my course.

But one late night
when I was compiling the report,
I really realised my discovery.

I had shown that Swedish
top students know statistically

significantly less about
the world than the chimpanzees.

Because the chimpanzees
would score half right.

If I gave them two bananas
with Sri Lanka and Turkey,

they would be right
half of the cases,
but the students are not there.

I did also an unethical study
of the professors of
the Karolinska Institutet,

that hands out the Nobel Prize
for medicine, and they are on par
with the chimpanzees there.

Today there's more information
accessible than ever before.

'And I work with my team at
the Gapminder Foundation

'using new tools that help everyone
make sense of the changing world.

'We draw on the masses of data
that's now freely available

'from international institutions
like the UN and the World Bank.

'And it's become my mission to
share the insights

'from this data with anyone who'll
listen, and to reveal how statistics
is nothing to be frightened of.'

I'm going to provide you a view of

the global health situation
across mankind.

And I'm going to do that in
hopefully an enjoyable way,
so relax.

So we did this software
which displays it like this.

Every bubble here is a country -

this is China, this is India.

The size of the bubble
is the population.

I'm going to stage a race between
this sort of yellowish Ford here

and the red Toyota down there
and the brownish Volvo.

The Toyota has a very bad start
down here, and United States,

Ford is going off-road there,

and the Volvo is doing quite fine,
this is the war.

The Toyota got off track, now Toyota
is on the healthier side of Sweden.

That's about where I sold
the Volvo and bought the Toyota.

AUDIENCE LAUGH

This is the great leap forward,
when China fell down.

It was the central planning
by Mao Zedong.

China recovered and said, "Never
more stupid central planning,"

but they went up here.

No, there is one more inequity,
look there - United States

They broke my frame. Washington DC
is so rich over there,

but they are
not as healthy as Kerala in India.
It's quite interesting, isn't it?

LAUGHTER AND APPLAUSE

Welcome to the USA,
world leaders in big cars

and free data.

There are many here who share
my vision of making public data
accessible and useful for everyone.

The city of San Francisco
is in the lead, opening up
its data on everything.

Even the police department is
releasing all its crime reports.

This official
crime data has been turned

into a wonderful interactive map by
two of the city's computer whizzes.

It's community statistics in action.

Crimespotting is
a map of crime reports from the
San Francisco Police Department

showing dots on maps
for citizens to be able to see

patterns of crime around their
neighbourhoods in San Francisco.

The map is not just about individual
crimes but about broader patterns
that show you where crime is

clustered around the city, which
areas have high crime,

and which areas have
relatively low crime.

We're here at the top of
Jones Street on Nob Hill...

..quite a nice neighbourhood.

What the crime maps show us
is the relationship between

topography and crime.

Basically the higher up the hill,
the less crime there is.

You cross over the border

into the flats...

Essentially as soon as you get
into the lower lying areas of Jones
Street the crime just skyrockets.

We're here in
the uptown Tenderloin district.

It's one of the oldest and densest
neighbourhoods in San Francisco.

This is where you go to buy drugs.

Right around here.

We see lots of aggravated assaults,
lots of auto thefts.

Basically a huge part of the crime
that happens in the city happens
in this five or six block radius.

If you've been hearing police sirens
in your neighbourhood,

you can use the map to find out why.

If you're out at night in
an unfamiliar part of town,

you can check the map
for streets to avoid.

If a neighbour gets burgled,
you can see -

is it a one-off or has there been
a spike in local crime?

If you commute through a
neighbourhood and you're worried

about its safety, the fact that we
have the ability to turn off all

the night-time
and middle-of-the-day crimes

and show you just the things that are
happening during the commute,

it is a statistical operation.
But I think to people that are
interacting with the thing

it feels very much more like they're
just sort of browsing a website
or shopping on Amazon.

They're looking at data
and they don't realise
they're doing statistics.

What's most exciting for me
is that public statistics

is making citizens more powerful and
the authorities more accountable.

We have community meetings that
the police attend

and what citizens are
now doing are bringing printouts

of the maps that show where crimes
are taking place,

and they're demanding services
from the police department

and the police department is now
having to change how they police,

how they provide policing services,

because the data is showing
what is working and what is not.

People in San Francisco
are also using public data

to map social inequalities
and see how to improve society.

And the possibilities are endless.

I think our dream
government data analysis project

would really be focused on
live information,

on stuff that was being reported
and pushed out to the world over
the internet as it was happening.

You know, trash pickups,
traffic accidents, buses,

and I think through the kind of
stats-gathering power

of the internet
it's possible to really begin
to see the workings of the city

displayed as a unified interface.

So that's where we are heading.

Towards a world of free data
with all the statistical
insights that come from it,

accessible to everyone, empowering
us as citizens and letting us
hold our rulers to account.

It's a long way from
where statistics began.

Statistics
are essential to us to monitor
our governments and our societies.

But it was our rulers up
there who started

the collection of statistics in the
first place in order to monitor us!

In fact the word 'statistics'
comes from 'the state'.

Modern statistics
began two centuries ago.

Once it got going,
it spread and never stopped.

And guess who was first!

The Chinese have Confucius,
the Italians have da Vinci,

and the British have Shakespeare.

And we have the Tabellverket -

the first ever systematic
collection of statistics!

Since the year 1749
we have collected data

on every birth, marriage and death,
and we are proud of it!

The Tabellverket recorded
information

from every parish in Sweden.

It was a huge quantity of data and
it was the first time any government

could get an accurate
picture of its people.

Sweden had been the greatest
military power in Northern Europe,

but by 1749 our star
was really fading

and other countries
were growing stronger.

At least we were a large power,

thought to have 20 million people,
enough to rival Britain and France.

But we were in for a nasty surprise.

The first analysis
of the Tabellverket

revealed that Sweden
only had two million inhabitants.

Sweden was not just a power
in decline, it also had
a very small population.

The government was horrified
by this finding -
what if the enemy found out?

But the Tabellverket also showed
that many women died in childbirth
and many children died young.

So government took action
to improve the health of the people.

This was the beginning
of modern Sweden.

It took more than 50 years before
the Austrians, Belgians, Danes,

Dutch, French, Germans, Italians

and, finally, the British,
caught up with Sweden
in collecting and using statistics.

It was called political arithmetic.
It was a lovely phrase
that was used for statistics.

Governments could have much more
control and understanding of

the society - how it was working,
how it was developing

and essentially
so they could control it better.

It wasn't just governments who
woke up to the power of statistics.

Right across Europe, 19th
century society went mad for facts.

And, despite its late start,
Britain,

with its Royal Statistical Society
in London,

was soon a statisticians' nirvana.

I love looking at old copies of
the Royal Statistical Society journal

because it's full of such odd stuff.

There's a wonderful paper
from the 1840s

which shows a map of England and
the rates of bastardy in each county.

So you can identify very quickly the
areas with high rates of bastardy.

Being in East Anglia it always
makes me slightly laugh that Norfolk

seems to top the "bastardy league"
in the 1840s.

One of the founders of
the Royal Statistical Society

was the great
Victorian mathematician
and inventor Charles Babbage.

In 1842 he read the latest
poem by an equally great Victorian,
Alfred Tennyson.

Vision of Sin contained the lines:

"Fill the cup, and fill the can

"Have a rouse before the morn

"Every moment dies a man
Every moment one is born."

So keen a statistician was Babbage
that he could not contain himself.

He dashed off a letter to Tennyson

explaining that because of
population growth,

the line should read,

"Every moment dies a man
and one and a 16th is born."

I may add that
the exact figure is 1.067,

but something must be
conceded to the laws of metre.

In the 19th century, scholars all
over Europe did amazing work

in measuring their societies.

They were hoovering up
data on almost everything.

But numbers alone
don't tell you anything.

You have to analyse them,
and that's what makes statistics.

When the first statisticians
began to get to grips with

analysing their data

they seized upon the average, and
they took the average of everything.

What's so great
about an average is that

you can take a whole mass of data
and reduce it to a single number.

And though each of us is unique,
our collective lives produce

averages that can
characterise whole populations.

I looked in my local newspaper
one week and saw a pensioner

had accidentally put her foot on
the accelerator

and crushed her friend
against a wall.

Devastating, hideous,
horrible thing to happen.

And then there was a second one about
a young man who didn't have

a driving licence, was driving a car
under the influence of drugs
and alcohol

and he bashed into a pedestrian
and killed him.

What's remarkable, absolutely
remarkable, if you look at the number

of people who die each year
in traffic crashes,
it's nearly a constant.

What?

All these individual events,
somehow when you sum them all up
there's the same number every year.

And every year, two and a half
times as many men

die in traffic crashes
as women, and it's a constant.

And every year the rate in Belgium
is double the rate in England.

There are these
remarkable regularities.

So that these individual
particular events sum up
into a social phenomenon.

Let's see what Sweden have done.

We used to boast about fast social
progress, that's where we were....

'In my lectures, to tell stories
about the changing world,

'I use the averages
from entire countries,

'whether the average of income,
child mortality, family size

'or carbon output.'

OK, I give you Singapore.
The year I was born,

Singapore had twice the child
mortality of Sweden, the most
tropical country in the world,

a marshland on
the Equator, and here we go.

It took a little time for them
to get independent,

but then they started to grow
their economy,

and they made the social investment,
they got away malaria,

they got a magnificent health system
that beat both US and Sweden.

We never thought it would happen
that they would win over Sweden!

LAUGHTER AND APPLAUSE

But useful as averages are,
they don't tell you the whole story.

On average, Swedish people have
slightly less than two legs.

This is because few people
only have one leg or no legs,

and no-one has three legs.

So almost everybody in Sweden
has more than
the average number of legs.

The variation in data is just
as important as the average.

But how do you get
a handle on variation?

For this, you transform
numbers into shapes.

Let's look again at the number of
adult women in Sweden

for different heights.

Plotting the data as a shape
shows how much their heights

vary from the average
and how wide that variation is.

The shape a set of data makes
is called its distribution.

This is the income distribution
of China, 1970.

This is the income distribution
of the United States, 1970.

Almost no overlap,
and what has happened?

China is growing,
it's not so equal any longer,

and it's appearing here
overlooking the United States.

Almost like a ghost, isn't it?

It's pretty scary.

Rrrr!

LAUGHTER

The statisticians
who first explored distribution

discovered one shape
that turned up again and again.

The Victorian scholar
Francis Galton

was so fascinated he built
a machine that could reproduce it,

and he found it fitted so many
different sets of measurements

that he named it
the normal distribution.

Whether it was people's arm spans,
lung capacities,

or even their exam results,

the normal distribution shape
recurred time and time again.

Other statisticians soon found
many other regular shapes,

each produced by particular kinds
of natural or social processes.

And every statistician
has their favourite.

The Poisson distribution, the Poisson
shape is my favourite distribution.

I think it's an absolute cracker.

The Poisson shape
describes how likely it is

that out-of-the-ordinary things
will happen.

Imagine a London bus stop where
we know that on average

we'll get three buses in an hour.

We won't always get
three buses, of course.

Amazingly, the Poisson shape will
show us the probability

that in any given hour we will get
four, five, or six buses,

or no buses at all.

The exact shape changes
with the average.

But whether it's how many people
will win the lottery jackpot

each week,

or how many people will phone
a call centre each minute,

the Poisson shape
will give the probabilities.

The wonderful example where this was
applied to in the late 19th century

was to count each year the number of
Prussian officers,

cavalry officers, who were kicked
to death by their horses.

Now, some years there were none,
some years there were one,

some years there were two,
up to seven, I think,
one particularly bad year.

But with this distribution,
however many years there were

with nought, one, two, three,
four Prussian cavalry officers

kicked to death by their horses,
beautifully obeyed
the Poisson distribution.

So statisticians use shapes to
reveal the patterns in the data.

But we also use images of all kinds

to communicate statistics
to a wider public.

Because if the story in the numbers

is told by a beautiful and clever
image, then everyone understands.

Of the pioneers
of statistical graphics,
my favourite is Florence Nightingale.

There are not many people who realise
that she was known

as a passionate statistician
and not just the Lady of the Lamp.

She said that "to understand God's
thoughts, we must study statistics,

"for these are
the measure of His purpose."

Statistics was for her a religious
duty and moral imperative.

When Florence was nine years old
she started collecting data.

Her data was different
fruits and vegetables she found.

Put them into different tables.

Trying to organise them
in some standard form.

And so we have one of Nightingale's
first statistical tables

at the age of nine.

In the mid 1850s Florence
Nightingale went to the Crimea to
care for British casualties of war.

She was horrified by
what she discovered.

For all the soldiers being blown
to bits on the battlefield,
there were many, many more soldiers

dying from diseases they caught
in the army's filthy hospitals.

So Florence Nightingale
began counting the dead.

For two years she recorded
mortality data in meticulous detail.

When the war was over she persuaded
the government to set up

a Royal Commission of Inquiry,

and gathered her data
in a devastating report.

What has cemented her place in
the statistical history books

are the graphics she used.

And one in particular,
the polar area graph.

For each month of the war,
a huge blue wedge represented

the soldiers who had died
from preventable diseases.

The much smaller red wedges were
deaths from wounds,

and the black wedges were deaths
from accidents and other causes.

Nightingale's graphics were so clear
they were impossible to ignore.

The usual thing around
Florence Nightingale's time

was just to produce tables and
tables of figures - absolutely
really tedious stuff that,

unless you're an absolutely dedicated
statistician,

it's really quite difficult to spot
the patterns quite naturally.

But visualisations, they tell a
story, they tell a story immediately.

And the use of colour
and the use of shape can
really tell a powerful story.

And nowadays of course
we can make things move as well.

Florence Nightingale would have
loved to have played with...

She would have
produced wonderful animations,
I'm absolutely certain of it.

Today, 150 years on,
Nightingale's graphics

are rightly regarded as a classic.

They led to a revolution
in nursing, health care

and hygiene in hospitals worldwide,
which saved innumerable lives.

And statistical graphics has
become an art form of its very own,

led by designers who are
passionate about visualising data.

This is the Billion Pound-O-Gram.

This image arose out of frustration

with the reporting of billion pound
amounts in the media.

£500 billion pounds for this war.

£50 billion for this oil spill.

It doesn't make sense -
the numbers are too enormous
to get your mind round.

So I scraped all this data
from various news sources
and created this diagram.

So the
squares here are scaled according
to the billion pound amounts.

When you see numbers visualised
like this

you start to have a different
relationship with them.

You can start to see the patterns,
and the scale of them.

Here in the corner,
this little square - £37 billion.

This was the predicted cost
of the Iraq war in 2003.

As you can see it's grown
exponentially over the last few years

and the total cost now is
around about £2,500 billion.

It's funny because when
you visualise statistics

you understand them,
and when you understand them

you can really start to put things
in perspective.

Visualisation is right at
the heart of my own work too.

I teach global health.

And I know having the data
is not enough -

I have to show it in ways people
both enjoy and understand.

Now I'm going to try something
I've never done before.

Animating the data in real space,

with a bit of technical
assistance from the crew.

So here we go.

First, an axis for health.

Life expectancy
from 25 years to 75 years.

And down here an axis for wealth.

Income per person -
400, 4,000, 40,000.

So down here is poor and sick.

And up here is rich and healthy.

Now I'm going to show you the world

200 years ago, in 1810.

Here come all the countries.

Europe, brown;
Asia, red; Middle East, green;

Africa south of the Sahara,
blue; and the Americas, yellow.

And the size of the country bubble
shows the size of the population.

In 1810, it was pretty crowded
down there, wasn't it?

All countries were sick and poor.

Life expectancy
was below 40 in all countries.

And only UK and the Netherlands were
slightly better off. But not much.

And now I start the world.

The industrial revolution makes
countries in Europe and elsewhere

move away from the rest.

But the colonized countries
in Asia and Africa,

they are stuck down there.

And eventually the Western countries
get healthier and healthier.

And now we slow down to show
the impact of the First World War

and the Spanish flu epidemic.

What a catastrophe!

And now I speed up through
the 1920s and the 1930s and,

in spite of the Great Depression,

Western countries forge on towards
greater wealth and health.

Japan and some others try to follow.

But most countries stay down here.

And after the tragedies
of the Second World War,

we stop a bit to look
at the world in 1948.

1948 was a great year.

The war was over,

Sweden topped the medal table at
the Winter Olympics and I was born.

But the differences between
the countries of the world

was wider than ever.

United States was in the front.

Japan was catching up.

Brazil was way behind,

Iran was getting a little richer
from oil but still had short lives.

And the Asian giants...

China, India, Pakistan, Bangladesh,
and Indonesia,

they were still
poor and sick down here.

But look what was about to happen!
Here we go again.

In my lifetime, former colonies
gained independence and then finally

they started to get healthier
and healthier and healthier.

And in the 1970s, then countries
in Asia and Latin America

started to catch up
with the Western countries.

They became the emerging economies.

Some in Africa follows,

some Africans were stuck in civil
war, and others were hit by HIV.

And now we can see the world
in the most up-to-date statistics.

Most people today
live in the middle.

But there is huge difference
at the same time

between the best-off countries
and the worst-off countries.

And there are also huge
inequalities within countries.

These bubbles show country averages
but I can split them.

Take China. I can split it
into provinces.

There goes Shanghai...

It has the same health
and wealth as Italy today.

And there
is the poor inland province Guizhou,

it is like Pakistan.

And if I split it further, the rural
parts are like Ghana in Africa.

And yet, despite the enormous
disparities today,

we have seen 200 years
of remarkable progress!

That huge historical gap between
the west and the rest is now closing.

We have become an entirely
new, converging world.

And I see a clear trend
into the future.

With aid, trade, green
technology and peace,

it's fully possible
that everyone can make it

to the healthy, wealthy corner.

Well, what you've just seen
in the last few minutes

is a story of 200 countries
shown over 200 years and beyond.

It involved plotting
120,000 numbers.

Pretty neat, huh?

So, with statistics, we can begin
to see things as they really are.

From tables of data to averages,
distributions and visualisations,

statistics gives us a
clear description of the world.

But, with statistics, we can
not only discover WHAT is happening

but also explore WHY,

by using the powerful analytical
method - correlation.

Just looking at one thing at a
time doesn't tell you very much.

You've got to look at the
relationships between things,

how they change,
how they vary together.

That's what correlation is about.

That's how you start trying
to understand the processes

that are really going on
in the world and society.

Most of us today would recognise
that crime correlates to poverty,

that infection correlates
to poor sanitation,

and that knowledge of statistics
correlates

to being great at dancing!

Correlations can be very tricky.

I got a joke about
silly correlations.

There was this American who
was afraid of heart attack.

He found out that
the Japanese ate very little fat

and almost didn't drink wine,

but they had much less
heart attacks than the Americans.

But, on the other hand,
he also found out that the French

eat as much fat as the Americans
and they drink much more wine but
they also have less heart attacks.

So he concluded that what kills you
is speaking English.

# Smoke, smoke,
smoke that cigarette

# Puff, puff, puff and if you
smoke yourself to death... #

The time, the pace,
the cigarette. Weights Tilt.

The best example of a really
ground-breaking correlation

is the link that was established
in the 1950s between
smoking and lung cancer.

Not long after the Second World War,
a British doctor, Richard Doll,

investigated lung cancer patients
in 20 London hospitals.

And he became certain
that the only thing they had
in common was smoking.

So certain,
that he stopped smoking himself.

But other people weren't so sure.

A lot of the discussion
of the early data,

linking smoking to lung cancer, said,
"It's not the smoking, surely,

"that thing we've done all our lives,
that can't be bad for you.

"Maybe it's genes.

"Maybe people who are genetically
predisposed to get lung cancer

"are also genetically
predisposed to smoke."

"Maybe it's not the smoking,
maybe it's air pollution -

"that smokers are somehow
more exposed to air pollution
than non-smokers.

"Maybe it's not smoking,
maybe it's poverty."

So now we've got three alternative
explanations, apart from chance.

To verify his correlation
did imply cause and effect.

Richard Doll created the biggest
statistical study of smoking yet.

He began tracking the lives
of 40,000 British doctors,

some of whom smoked
and some of whom didn't,

and gathered enough data

to correlate the amount
the doctors smoked

with their likelihood
of getting cancer.

Eventually, he not only
showed a correlation between
smoking and lung cancer,

but also a correlation
between stopping smoking
and reducing the risk.

This was science at its best.

What correlations do not replace
is human thought.

You've got to think
about what it means.

What a good scientist does,
if he comes with a correlation,

is try as hard as she or he
possibly can to disprove it,

to break it down, to get rid of it,
to try and refute it.

And if it withstands
all those efforts at demolishing it

and it is still standing up then,
cautiously, you say, "We really
might have something here."

However brilliant the scientist,
data is still the oxygen of science.

The good news is that the more we
have, the more correlations we'll
find, the more theories we'll test,

and the more discoveries
we're likely to make.

And history shows how our total sum
of information grows in huge leaps
as we develop new technologies.

The invention of the
printing press kicked off the first
data and information explosion.

If you piled up all the books that
had been printed by the year 1700,

they would make 60 stacks
each as high as Mount Everest.

Then, starting in the 19th century,

there came a second information
revolution with the telegraph,

gramophone and camera.
And later radio and TV.

The total amount
of information exploded.

And by the 1950s
the information available to us all
had multiplied 6,000 times.

Then, thanks to the computer and
later the internet, we went digital.

And the amount of data we have now
is unimaginably vast.

A single letter printed in a book
is equivalent to a byte of data.

A printed page
equals a kilobyte or two.

Five megabytes is enough for
the complete works of Shakespeare.

10 gigabytes - that's a DVD movie.

Two terabytes
is the tens of millions of photos
added to Facebook every day.

Ten petabytes is the data recorded
every second by the world's
largest particle accelerator.

So much
only a tiny fraction is kept.

Six exabytes is what you'd have
if you sequenced the genomes
of every single person on Earth.

But really, that's nothing.

In 2009, the internet
added up to 500 exabytes.

In 2010, in just one year, that will
double to more than one zettabyte!

Back in the real world, if we
turned all this data into print
it would make 90 stacks of books,

each reaching from here
all the way to the sun!

The data deluge is staggering,
but, with today's computers

and statistics,
I'm confident we can handle it.

When it comes to all the data
on the internet,

the powerhouse
of statistical analysis

is the Silicon Valley giant Google.

The average person over their
lifetime is exposed to about 100
million words of conversation.

And so if you multiple that by the
six billion people on the planet,

that amount of words is about
equal to the number of words

that Google has available
at any one instant in time.

Google's computers hoover up
and file away every document,
web page, and image they can find.

They then hunt for patterns and
correlations in all this data,

doing statistics on a massive scale.

And, for me, Google has one project
that's particularly exciting -
statistical language translation.

We wanted to provide access
to all the web's information,
no matter what language you spoke.

There's just so much information
on the internet,

you couldn't hope to translate it all
by hand into every possible language.

We figured we'd have to be able
to do machine translation.

In the past, programmers
tried to teach their computers

to see each language as a set of
grammatical rules - much like the
way languages are taught at school.

But this didn't work because no set
of rules could capture a language

in all its subtlety and ambiguity.

"Having eaten our lunch
the coach departed."

Well, that's obviously incorrect.

Written like that it would imply
that the coach has eaten the lunch.

It would be far better to say...

"having eaten our lunch
we departed in the coach."

Those rules are helpful and they are
useful most of time, but they don't
turn out to be true all the time.

And the insight of using statistical
machine translation is saying,

"If you've got to have all these
exceptions anyways, maybe you can get
by without having any of the rules.

"Maybe you can treat everything
as an exception." And that's
essentially what we've done.

What the computer is doing when
he's learning how to translate

is to learn correlations
between words

and correlations between phrases.

So we feed the system very large
amounts of data

and then the system is seeing that
a certain word or a certain phrase

correlates very often
to the other language.

Google's website currently
offers translation between
any of 57 different languages.

It does this purely statistically,
having correlated a huge collection
of multilingual texts.

The people that built the system
don't need to know Chinese

in order to build the
Chinese-to-English system,
or they don't need to know Arabic.

But the expertise that's needed is
basically knowledge of statistics,

knowledge of computer science,
knowledge of infrastructure

to build those very large
computational systems
that we are building for doing that.

I hooked up with Google
from my office in Stockholm
to try the translator for myself.

'I will type...
some Swedish sentences.'

OK.

Sveriges...

..guldring i orat.

OK. So it says, "Sweden's finance
minister has a ponytail
and a gold ring in your ear."

I guess it probably means
in his ear. 'That's exactly
correct, it's amazing!

'He comes from the Conservative
party, that's the kind
of Sweden we have today.

'I will type one more sentence.'

'I sitt samkonade...'

partnerskap...

nya biskop.

"In his same-sex partnership
has Stockholm's new bishop
and his partners a three-year son."

It's almost perfect,
there's one important thing -

it's HER,
it's a lesbian partnership.

OK, so those kinds of words his
and her are one of the challenges

in translation
to get really those right.

Especially when it comes
to bishops one can excuse it!

'Right, right.'

I guess more often than not
it would probably be a "his".
'I will write one more sentence.'

Nar Sverige deltar
I olympiader ar malet

'inte att vinna
utan att sla Norge.'

OK. "When Sweden is taking part
in Olympic goal is not
to win but to beat Norway."

'Yes! This is what it is!

'But they are very good
in Winter Olympics, so we
can't make it, but we are trying.'

Ah, very good, very good.

'This is absolutely amazing, you
know, and I was especially impressed

'that it picks up words like
"same-sex partnership"
which are very new to the language."

'The translator is good, but
if they succeed with what's next,
that'll be remarkable.'

One of the exciting possibilities

is combining the machine
translation technology with
the speech recognition technology.

Now, both of these
are statistical in nature.

The machine translation relies
on the statistics of mapping
from one language to another,

and similarly speech recognition
relies on the statistics of mapping
from a sound form to the words.

When we put them together,

now we have the capability
of having instant conversation

between two people
that don't speak a common language.

I can talk to you in my language,

you hear me in your language
and you can answer back.

And in real time we can
make that translation,

we can bring two people together
and allow them to speak.

The internet is just one
of many technologies created
to gather massive amounts of data.

Scientists studying
our earth and our environment

now use an incredible range
of instruments

to measure the processes
of our planet.

All around us are sensors
continuously measuring temperature,
water flow, and ocean currents.

And high in orbit are satellites
busy imaging cloud formations,
forest growth and snow cover.

Scientists speak
of "instrumenting the earth".

And pointing up to the skies
above are powerful new telescopes
mapping the universe.

What's happening in astronomy
is typical of how profoundly

this new torrent of data
is transforming science.

Astronomers are now addressing many
enduring mysteries of the cosmos

by applying statistical methods
to all this new data.

The galaxy is a very big place and
it's got billions of stars in it,

and so to put together a coherent
picture of the whole galaxy requires
having an enormous amount of data.

And before you could do
a large sky survey with
sensitive, digital detectors

that meant that you could map many,
many stars all at once,

it was very difficult to build up
enough data on enough of the galaxy.

In the past, large surveys
of the night sky had to be done

by exposing thousands
of large photographic plates.

But these surveys could take
25 years or more to complete.

Then, in the 1990s, came digital
astronomy and a huge increase

in both the amount
and the accessibility of data.

The Sloan Sky Survey
is the world's biggest yet,
using a massive digital sensor

mounted on the back
of a custom-built telescope
in New Mexico.

It's scanned the sky night
after night for eight years,

building up a composite picture
in unprecedented resolution.

The Sloan is some of the best,
deepest survey data
that we have in astronomy.

Both on our own galaxy and
on galaxies further away from ours.

All the Sloan data
is on the internet,

and with it astronomers
have identified millions of hitherto
unknown stars and galaxies.

They also comb the database
for statistical patterns

which will prove, disprove,
or even suggest new theories.

So we have this idea that galaxies
grow, they become large galaxies like
the one we live in, the milky way,

not all at once, or not smoothly,
but by continuously incorporating,

basically cannibalising,
smaller galaxies.

They dissolve them
and they become part
of the bigger galaxy as it grows.

It's a startling idea,
and, in the Sloan data,
is the evidence to support it.

Groups of stars that came
from cannibalised galaxies

stand out in the Sloan data
as statistically different
from other stars

because they move
at a different velocity.

Each big spike
on one of these distribution graphs

means Professor Rockosi has found
a group of stars all travelling
in a different way to the rest.

They are the telltale
patterns she's looking for.

The evidence is accumulating
that, in fact, this really is
how galaxies grow,

or an important way
in which how galaxies grow.

And so this is an important part
of understanding how galaxies form,
not only ours but every galaxy.

The more data there is,
the more discoveries can be made.

And the technology
is getting better all the time.

The next big survey telescope
starts its work in 2015.

It will leave Sloan in the dust!

Sloan has taken eight years to cover
one quarter of the night sky.

The new telescope will scan
the entire sky, in even greater
resolution, every three days!

The vast amounts of data
we have today allows researchers
in all sorts of fields

to test their theories
on a previously unimaginable scale.

But more than this,
it may even change
the fundamental way science is done.

With the power of today's computers
applied to all this data,

the machines might even be able
to guide the researchers.

We're at a potentially
profoundly important

and potentially one of the most
significant points in science,

and certainly one of
the most exciting,

where the potential to transform
not just how scientists do science
but even what science is possible.

And what will power
that transformation

of both how science is done
and even what science is possible

is going to be computation.

Many of the dynamics of the natural
world, like the interplay between
the rainforests and the atmosphere,

are so complex that we don't
as yet really understand them.

But now computers are generating
literally tens of thousands
of different simulations

of how these
biological systems might work.

It's like creating thousands
of hypothetical parallel worlds.

Each and every one
of these simulations

is analysed with statistics
to see if any are a good match
for what is observed in nature.

The computers can now
automatically generate,

test and discard hypotheses
with scarcely a human in sight.

This new application of statistics
will become absolutely vital
for the future of science.

It's creating a new paradigm,
if you like,

in science, in the way
in which we can do science,

which is increasingly...

Which one might characterise as...
data-centric or data driven

rather than being hypothesis-driven
or experimentally-driven.

So, it's exciting times
in terms of the science,

in terms of the computation
and in terms of the statistics.

Now, if all that sounds a bit
abstract and theoretical to you,
how about one final frontier?

Could statistics even make
sense of your feelings?

In California - where else? -
one computer scientist

is harvesting the internet to try
to divine the patterns of our
innermost thoughts and emotions.

This is the madness movement.

The madness movement represents
a skyscraper view of the world.

Each of these brightly coloured dots
is an individual feeling

expressed by someone out there
in a blog or a tweet.

And when you click on the dot
it explodes to reveal the
underlying feeling of that person.

This is what people say
they're feeling today.

Better...safe...

crappy...

well...

pretty...special...

sorry...alone...

So, every minute, We Feel Fine
crawls the world's blogs,

takes all the sentences
that start with the words
"I feel" or "I am feeling",

and puts them in a database.

We collect all the feelings
and we count the most common.

They are better...bad...

good...right...

guilty...sick...

the same...like shit...

sorry...well...

and so on.

And we can take a look at any
one feeling and analyse it.

Right now a lot of people
are feeling happy.

We can take a look at all the
people who are happy and break it
down by age, gender or location.

Since bloggers have public profiles
we have that information and
so we can ask questions like,

"Are women happier than men?"
or, "Is England happier
than the United States?"

We find that, as people get older,
they get happier.

And, moreover, we find that
for younger people they associate
happiness more with excitement,

and, as people get older,
they associate happiness
more with peacefulness.

And we also find that women feel
loved more often than men,
but also more guilty.

While men feel good more often
than women, but also more alone.

As people lead more and
more of their lives online,
they leave behind digital traces,

and with these digital traces
we can begin to statistically analyse
what it means to be human.

So where does all of this leave us?

We generate unimaginable
quantities of data
about everything you can think of.

We analyse it to reveal
the patterns.

And now not only experts
but all of us can understand
the stories in the numbers.

Instead of being
led astray by prejudice,

with statistics at our fingertips,
our eyes can be open
for a fact-based view of the world.

So, more than ever before, we can
become authors of our own destiny.

And that's pretty
exciting isn't it?!

# 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,
12, 13, 14, 15, 16, 17, 18, 19, 20

# 1, 22, 3, 24, 25, 26, 27, 28, 9,
30, 31, 32, 3, 34, 35, 36, 7

# 38, 39, 40, 41, 42, 3,
44, 45, 46, 47

LYRICS DEGENERATE INTO GIBBERISH

GIBBERISH DEGENERATES INTO NOISE

# 100. #