Composing for choir, orchestra—and video projections | Jocelyn Hagen | TEDxMinneapolisSalon

What if classical composers conceived of music and visuals simultaneously, resulting in an immersive audio-visual experience for concert-goers? Composer Jocelyn Hagen is exploring the new frontier of video syncing technology with a new work for choir, orchestra, and a new member of the orchestra—video projections. The first movement of her symphony The Notebooks of Leonardo da Vinci, commemorating the 500th anniversary of the death of da Vinci, was performed on the TEDxMinneapolis stage by a four part choir, chamber ensemble, and Ion Concert Media video technician Scott Winters, led by conductor Kathy Romey. Jocelyn Hagen composes magical, dramatic and deeply moving music. A majority of her work features our most universal instrument: the human voice. Her commissions include Conspirare, The Minnesota Orchestra, the American Choral Directors Associations of Minnesota, Georgia, Connecticut and Texas, the North Dakota Music Teachers Association, Cantus, the Boston Brass, the Metropolitan Symphony Orchestra, and The Houston Chamber Choir. Recently, Jocelyn was inspired by the notebooks of Leonardo da Vinci and has created an original score synced with video and animations from da Vinci’s writing. Jocelyn uses multimodal and compositional narrative to deepen our understanding and allow us to “rehear music” in a new and unique way.

This talk was given at a TEDx event using the TED conference format but independently organized by a local community. Learn more at https://www.ted.com/tedx

(Chamber ensemble tuning)

(Music: flute solo)

(Music continues)

(Single clarinet joins in)

(Single violin pizzicato joins in)

(Single oboe joins in)

(Music continues)

(Strings join in)

(Music continues)

(Music continues)

(Music stops)

"A painting is a poem seen but not heard,

a poem is a painting heard but not seen.

Hence these two poems, or two paintings,

have exchanged the senses
by which they pierce the intellect."

Leonardo da Vinci wrote those words

in one of his many notebook pages
over 500 years ago.

These words served as inspiration for me

in the creation of my
new multimedia symphony -

"The Notebooks of Leonardo da Vinci."

I was inspired to create this work

after seeing an exhibit
of the Codex Leicester

at the Minneapolis Institute of Art

in 2015.

And as the composer

who is the creative force behind
both the music and the visual component,

I have designed an opportunity

in which the music serves
as the foundation for the digital media.

As audience members,

we are conditioned to expect a certain
relationship between film and video.

We all know how powerful
a visual experience can be

when it is supported by adequate sounds

that heighten our emotions
at exactly the right moment.

What is more empowering
than a triumphant line played by the horn

or as sad and isolating as a solo violin.

Film and TV directors

know that great music
is a wonderful asset to their productions,

and they spend millions
of dollars each year

on the creation of new music

or the licensing
of just the right pop song

to get the audience
at their emotional peak.

The world of classical or concert music,
though, in which I work,

is still trying to figure out

how to utilize video in the best way
for their audiences.

Composers have tried

various visual additions
and multimedia productions

throughout the years -

some more successful than others.

What many of them are missing
is the vital connection

that film directors
have been capitalizing on for years:

the ability to sync the film and music,
to, in essence, perform together.

A few years ago, I was introduced
to a new software called Musaic,

created by Ion Concert Media,

which allows for the syncing of video
to live music performance

without the use of any kind
of mechanical metronome.

In the past and still often today,

if a conductor wants to sync a video
to a live music performance,

they would have to use a click track.

Imagine a little clock inside your ear
giving you the time with each beat -

tick, tick, tick, tick,
tick, tick, tick, tick …

It is hard for even the most seasoned
professional musicians

to perform expressively
with a metronome in their ear.

And part of what makes
live performance so magical

is the fact that
each performance is unique,

and great conductors

know when a beautiful moment
should be held a little longer

or if the excitement a section
needs to push just a little bit faster.

Playing with a click track

does not allow performers
to follow their musical instincts,

and it leads to dry,
mechanical performances.

Tick, tick, tick, tick …

A few years ago,

I brought my boys to Orchestra Hall
for a performance of Home Alone -

(Laughter)`

great movie and a wonderful
soundtrack by John Williams -

but I'm not so sure my boys even knew
there was a live orchestra up there.

It's too hard to compete
with Kevin McCallister,

(Laughter)

and that's because the film
is leading the music,

and that's how we're
programmed to digest it.

Movie scores are built to elicit emotion,
and they are very good at that,

but only the best of them participates
in the storytelling in a deeper way,

and when they do,
it elevates the entire experience.

But what I am exploring and creating

is a different relationship
between film and music,

in which the music is leading the film.

The combination of my music

and the wonderful cinematography
by my collaborators,

animator Joseph Mattoon
and filmmaker Isaac Gayle,

goes beyond simple sound and sight.

Adding a new dimension to the work
can be likened to dance.

The effect feels
more organic and intuitive,

and connects us all to a broader,
more present sense of time.

The music is meeting the film
on an equal ground,

and they coalesce into one
blended immersive performance.

I can create this way

because I have a video technician
using the Musaic software

as a member of the ensemble.

I'd like to say "hi" to Scott.

See Scott over there? Wave to Scott.

(Cheers) (Applause)

And he is following the conductor,

just like the singers
standing next to him,

so when the conductor
decides to slow down or speed up,

or hold something a little bit longer,

he adjusts the film accordingly.

In my multimedia symphony,

the film will always adjust
to what needs to happen musically.

What you just saw and heard

was the beginning
of the first movement of the symphony,

a movement titled "Painting and Drawing,"

and it begins with a duet
between the flute and the animation,

but as an audience, we take them in
as a single thought, a single idea,

recognizing the fluid connection

between what we see, what we hear,
and what we feel.

These two instruments are linked
and incomplete without each other.

What you just were witness to
was the beginning of a thought or an idea.

It came in bits and pieces,

there was a mistake,

there was silence -

as Leonardo's brain was thinking
about what to write next –

and then the idea begins to grow
and make sense,

and the music begins to flow,

and the handwriting begins to flow,
just like creative ideas flow.

And it wasn't just handwriting
in all those notebook pages;

I was amazed to discover

how much Leonardo da Vinci

studied the angles of light
and the perception of an object.

His notebook pages

are filled with geometry,
perfect circles, straight lines,

and I thought,
"How do I represent that musically?"

(Music)

(Music continues)

(Music ends)

As Leonardo draws
those long lines across the page,

I brought those lines into the strings.

You can even see it
in the notation of the music:

those long sustained notes

are drawn across the strings
of the instruments,

just like his hand across the page.

And then of course he practiced -

he drew sketches

of people, plants, animals,
buildings, inventions, water,

even grotesque faces and a few dragons.

I wanted to highlight
some of these beautiful portraits

in the final section of music.

This is when the choir enters,

singing an English translation of one
of his many musings from the notebooks.

♪ O painter ... ♪
(Choir sings with accompaniment)

♪ O painter ... ♪

♪ A painter is not admirable ♪

♪ unless he is universal. ♪

♪ O painter ... ♪

♪ O painter ... ♪

♪ O painter ... ♪

♪ A painter is not admirable ♪
♪ A painter is not admirable ♪ (In canon)

♪ unless he is universal. ♪
♪ unless he is universal. ♪ (In canon)

♪ A painting is a poem ♪

♪ seen but not heard, ♪

♪ a poem is a painting ♪

♪ heard but not seen. ♪

♪ Hence these two poems, ♪
♪ Hence these two poems, ♪ (In canon)

♪ two paintings, ♪
♪ two paintings, ♪ (In canon)

♪ have exchanged the senses ♪
♪ have exchanged the senses ♪ (In canon)

♪ by which they pierced ♪
♪ by which they pierced ♪ (In canon)

♪ the intellect. ♪ (In unison)
(Music ends)

I love the correlation of the human voice
and the human face.

I'd like to thank my collaborators
up here today with me:

Conductor Kathy Romi,
Scott Winters from Ion Concert Media

as well as all my friends and colleagues
who are up here performing with me today.

In 2019, we celebrate
the 500th anniversary

of the death of Leonardo da Vinci.

It has been four years
of planning and creating,

but I'm proud to say that this symphony

will be performed over a dozen times
across the United States this year.

You all -
(Applause)

Ah … (Laughs)

(Applause continues)

Thanks.

(Applause continues)

You all were the first to see this.

(Audience) Whoo!

And now, the first movement
of "The Notebooks of Leonardo da Vinci."

Thank you.

(Applause)

(Music: flute solo)

(Music continues)

(Single clarinet joins in)

(Single violin pizzicato joins in)

(Single oboe joins in)

(Strings join in)

(Harp joins in)

(Music continues)

(Music continues)

(Music continues)

(Choir joins in)
♪ O painter ... ♪

♪ O painter ... ♪

♪ A painter is not admirable ♪

♪ unless he is universal. ♪

♪ O painter ... ♪

♪ O painter ... ♪

♪ O painter ... ♪

♪ A painter is not admirable ♪
♪ A painter is not admirable ♪ (In canon)

♪ unless he is universal. ♪
♪ unless he is universal. ♪ (In canon)

♪ A painting is a poem ♪

♪ seen but not heard, ♪

♪ a poem is a painting ♪

♪ heard but not seen. ♪

♪ Hence these two poems, ♪
♪ Hence these two poems, ♪ (In canon)

♪ two paintings, ♪
♪ two paintings, ♪ (In canon)

♪ have exchanged the senses ♪
♪ have exchanged the senses ♪ (In canon)

♪ by which they pierced ♪
♪ by which they pierced ♪ (in canon)

♪ the intellect. ♪ (In unison)

(Instruments continue playing)

(Music continues)

(Music ends)

(Applause) (Cheers)