What is Information Theory

Imagine Alice has an idea

and she wants to share it.

there're so many ways to share an idea

She could draw a picture, make an engraving, write a song

send a telegraph or an e-mail

but how are these things different, and more importantly

why are they the same

this story is about the fundamental particle of all forms of communication

It begins with a special skill you likely take for granted, language

All language allows you to take a thought or mental object

and brake it down into a series of conceptual chunks.

These chunks are externalized using a series of signals or symbols.

Humans express themselves using a variation in sound and physical action

As the chirping birds and dancing bees

and man-made machines exchanging a dancing stream of electrical vibrations.

Even our bodies are built according to instructions stored microscopic books known as DNA.

All are different form up one thing, information

In simplest terms information is what allows one mind to influence another

It's based on the idea of communication as selection

information no matter the form can be measured using a fundamental unit.

In the same way, we can measure the massive different object using a standard measures

such as kilograms and pounds

This allows us to precisely measure and compare the way

to say rocks, water or wheat using a scale

Information too can be measured and compared

using a measurement called entropy.

Think a bit as information scale

We intuitively know that a single page from

some unknown book has less information than the entire book

We can describe exactly how much using the unit called the bit

a measure of surprise

So no matter how are us wants to communicate a specific message

hieroglyphics, music, computer code.

each would contain the same number of bits though in different densities

and a bit is linked to a very simple idea

The answer to a yes or no question

think of it as the language of coins

so how is information actually measured

does information have a speed limit? a maximum density?

Information theory holds the exciting answer to these questions.

It's an idea over three thousand years in the making.

but before we can understand this, we must step back.

and explore perhaps the most powerful invention in human history, the alphabet

and for this we return to the cave