How AI could compose a personalized soundtrack to your life

Meet AIVA, an artificial intelligence that has been trained in the art of music composition by reading more than 30,000 of history's greatest scores. In a mesmerizing talk and demo, Pierre Barreau plays compositions created by AIVA and shares his dream: to create original live soundtracks based on our moods and personalities.

About two and a half years ago,
I watched this movie called "Her."

And it features Samantha,
a superintelligent form of AI

that cannot take physical form.

And because she can't
appear in photographs,

Samantha decides to write a piece of music

that will capture a moment of her life
just like a photograph would.

As a musician and an engineer,
and someone raised in a family of artists,

I thought that this idea of musical
photographs was really powerful.

And I decided to create an AI composer.

Her name is AIVA,
and she's an artificial intelligence

that has learned the art
of music composition

by reading over 30,000 scores
of history's greatest.

So here's what one score
looks like to the algorithm

in a matrix-like representation.

And here's what 30,000 scores,

written by the likes
of Mozart and Beethoven,

look like in a single frame.

So, using deep neural networks,
AIVA looks for patterns in the scores.

And from a couple of bars
of existing music,

it actually tries to infer what notes
should come next in those tracks.

And once AIVA gets good
at those predictions,

it can actually build a set
of mathematical rules

for that style of music

in order to create
its own original compositions.

And in a way, this is kind of
how we, humans, compose music, too.

It's a trial-and-error process,

during which we may not
get the right notes all the time.

But we can correct ourselves,

either with our musical ear
or our musical knowledge.

But for AIVA, this process
is taken from years and years of learning,

decades of learning as an artist,
as a musician and a composer,

down to a couple of hours.

But music is also a supersubjective art.

And we needed to teach AIVA

how to compose the right music
for the right person,

because people have different preferences.

And to do that, we show to the algorithm
over 30 different category labels

for each score in our database.

So those category labels are like mood

or note density or composer
style of a piece

or the epoch during which it was written.

And by seeing all this data,

AIVA can actually respond
to very precise requirements.

Like the ones, for example,
we had for a project recently,

where we were commissioned
to create a piece

that would be reminiscent
of a science-fiction film soundtrack.

And the piece that was created
is called "Among the Stars"

and it was recorded
with CMG Orchestra in Hollywood,

under great conductor John Beal,

and this is what they
recorded, made by AIVA.

(Music)

(Music ends)

What do you think?

(Applause)

Thank you.

So, as you've seen, AI can create
beautiful pieces of music,

and the best part of it

is that humans can actually
bring them to life.

And it's not the first time in history

that technology has augmented
human creativity.

Live music was almost always
used in silent films

to augment the experience.

But the problem with live music
is that it didn't scale.

It's really hard to cram a full symphony
into a small theater,

and it's really hard to do that
for every theater in the world.

So when music recording
was actually invented,

it allowed content creators,
like film creators,

to have prerecorded and original music

tailored to each and every frame
of their stories.

And that was really
an enhancer of creativity.

Two and a half years ago,
when I watched this movie "Her,"

I thought to myself
that personalized music

would be the next single biggest change
in how we consume and create music.

Because nowadays, we have
interactive content, like video games,

that have hundreds of hours
of interactive game plays,

but only two hours of music, on average.

And it means that the music
loops and loops and loops

over and over again,
and it's not very immersive.

So what we're working on
is to make sure that AI can compose

hundreds of hours of personalized music

for those use cases
where human creativity doesn't scale.

And we don't just want
to do that for games.

Beethoven actually wrote a piece
for his beloved, called "Für Elise,"

and imagine if we could
bring back Beethoven to life.

And if he was sitting next to you,
composing a music for your personality

and your life story.

Or imagine if someone like
Martin Luther King, for example,

had a personalized AI composer.

Maybe then we would remember

"I Have a Dream" not only
as a great speech,

but also as a great piece of music,
part of our history,

and capturing Dr. King's ideals.

And this is our vision at AIVA:

to personalize music
so that each and every one of you

and every individual in the world

can have access to a personalized
live soundtrack,

based on their story
and their personality.

So this moment here together at TED
is now part of our life story.

So it only felt fitting that AIVA
would compose music for this moment.

And that's exactly what we did.

So my team and I worked on biasing AIVA
on the style of the TED jingle,

and on music that makes us feel
a sense of awe and wonder.

And the result is called
"The Age of Amazement."

Didn't take an AI to figure that one out.

(Laughter)

And I couldn't be more proud
to show it to you,

so if you can, close your eyes
and enjoy the music.

Thank you very much.

(Music)

[The Age of Amazement
Composed by AIVA]

(Music ends)

This was for all of you.

Thank you.

(Applause)