Augmented reality storytelling how it will change the way we play forever | Devon Lyon | TEDxSalem

This talk was given at a local TEDx event, produced independently of the TED Conferences. Storytelling combined with interactive media can be the beginning of a revolution in creative content creation as well as consumption according to filmmaker Devon Lyon.

Devon Lyon is an award-winning writer, director and producer with over 200 commercials under his belt. He is also a co-creator of the transmedia Angel Punk story world, which will be featured in an upcoming edition of “Dark Horse Presents” and published as an urban fantasy novel. When Devon isn't working, his seven-year-old daughter Dagny is leading him on crazy adventures around what former Oregon governor Tom McCall affectionately called the Sunset Empire.

About TEDx, x = independently organized event In the spirit of ideas worth spreading, TEDx is a program of local, self-organized events that bring people together to share a TED-like experience. At a TEDx event, TEDTalks video and live speakers combine to spark deep discussion and connection in a small group. These local, self-organized events are branded TEDx, where x = independently organized TED event. The TED Conference provides general guidance for the TEDx program, but individual TEDx events are self-organized.* (*Subject to certain rules and regulations)

It's pretty interesting being at The
Elsinore and actually seeing the theater

from this perspective because my first
memory of when I was seven or eight

years old was actually of going to the
movie theater was here at The Elsinore

so that's pretty neat and now I have a 7
year old daughter and until very recently,

we would go on a hikes in a
forest designed to spot fairies.

You know, the little mythical creatures of human
form but with magical powers.

Now of course, my daughter made sure that we

were looking for the winged variety
native to Oregon's Central Valley and coastal forests.

This was our version of
bird-watching and I'm serious.

It was looking for those little imagine moments
of the buggers whipping through the

forest trailing their fairy dust. It was
just fantastic, but that was it, right?

It was those few imagined moments of us
together in that forest. Now of course,

there's nothing wrong with traditional
imaginative play. In fact, it's critical

but I actually think it can be so much
more and I'll come back to fairy hunts

in a forest in just a few minutes.

But, I think TED talk should have at least some
impressive sounding facts and statistics

to ground what would otherwise be just
really a purely fun and imaginative talk

into at least something plausible. So did
you know that the art of moving pictures

has been around for about a hundred and twenty years?

So the art of visual linear storytelling and our passive consumption

of those stories has been refined over a
good amount of time and we at least as

Americans spend an awful lot of our time
passively consuming those stories.

The average American will buy about four
tickets to the traditional movie theater

a year and the average American over the
age of two will spend about thirty-five

hours a week or about 1800 hours a year
watching TV and for the slightly less

passive activity of playing video games,
the average American eight to eighteen

will spend about thirteen hours a week
or six hundred and seventy hours a year

playing video games so it's an awful lot
you know, so we're spending an awful lot

of time doing just that.

I think actually when I when I listen to speeches or when
I read about the subject matter of film

or television, an awful lot of time is
unfortunately spent on things like the

newest distribution model for film,
bigger screens at our local multiplex,

greater immersive surround sound, bigger
4k televisions, or Ultra HD and those

sorts of things and those are fun, right?
But, they have nothing to do with storytelling.

They're fun to geek out on
and my friends and I do an awful lot of

just that but fundamentally they're
simply the technical delivery of

substance and not the substance itself.
I've used the word "passive" a few times already

and I think the word passive is

important because although when we're
watching film and TV, we may jump in our

seats at a scene in a scary movie or we
may slap our knees or laugh out loud

when Will Ferrell rocks us with a
magical moment of comedy gold,

we're still fundamentally just passively
consuming those stories, right? Now of course,

traditional film and TV has moved
us to action, has brought us to tears, and

has changed the way we think because
fundamentally it's powerful and it works

but I actually believe that we are on
the threshold of a brand new genre of storytelling,

and I call it Augmented Storytelling.

But before we go into the
real specifics of Augmented Storytelling,

I think it would be helpful to
understand what it is not and you've

heard a little bit about this from some
of the other speakers today but first,

it's not virtual reality. Think of
virtual reality as a computer simulated reality

where the user wears a
head-mounted display and this is a

picture of the Oculus Rift and the
Oculus Rift is currently the darling of

the virtual reality hardware world and
was a recent purchase by Facebook for

over two billion dollars. Then the other
thing, there is a sketch of my daughter,

Dagny and how she would look wearing the
sort of unwieldy Oculus Rift on her head,

so in virtual reality, the computer
simulates the entire world and then projects it

onto screens embedded in the
head-mounted unit and the head-mounted

unit does other things like track your
head and positional data but no matter

how photorealistic that environment
might be, you're in virtual reality.

You are only virtually there. Your real self
back in reality is most likely sitting

maybe standing in a room somewhere
tethered to a computer. Next is Google Glass,

and unless you've been living
under a virtual reality rock for the

last few years, you have a general sense
of what Google Glass is, right? It's a pair

of glasses but with an optical
head-mounted display and unlike virtual reality,

Google Glass allows the user to
operate mostly normally in the real

world and that's because Google Glass
overlays text in information and

graphics on top of real reality. Then,
there's Augmented Reality Games and for

those of you who have followed this in
the past, or those of you who might be

interested to learn more about A.R.G's in
the future, I encourage you to at least

read about the two I'm going to mention.
First, was the "I love Bees" campaign that

led up to at the time what was a very
big launch of a retail video game called Halo 2.

The second which I don't have a
neat graphic for it was called The Beast

and it led up to the 2001 Stanley
Kubrick/Steven Spielberg film, Artificial Intelligence.

And, the great thing about
Augmented Reality Games is it allows the

player to engage, and participate, and
often shape the story in a nonlinear way

and how is this done? It's done because
the storytellers break the story up into

little bits and pieces like a puzzle and
they sprinkle them around the internet

and other media and even tangibly..
tangibly out in the real world, so when

you're playing an A.R.G, you go out and you
put.. you find the bits and pieces of the

puzzle of the story and you assemble it
and what's really neat is depending on

the order in which you find these clues,
these bits of the story, you're going to

put the story together in a way that's
wholly unique to you.

So that's what Augmented
Storytelling is not, right? And now,

so imagine the near future where there's
this convergence, where you have the

technology between virtual reality, you
have the Google Glass like overlays and

you have some of the underpinnings in
the structure of Augmented Reality Games

and they've come together to form what I
call Virtual Storytelling. So, what is

excuse me, Augmented Storytelling. What
is Augmented Storytelling. Well, I think

the easiest way in some ways to
understand it is to actually go back to

the beginning and it's a misty morning
on that Oregon coast and it's early but

the Sun is up and the light dapples through the trees
above me and my daughter.

She's cold, so she's zipping up her coat as I pull out
our Augmented Storytelling units,

I clip a belt on her waist that carries a

compute.. a black padded computer that's
probably no bigger than three or four

iPhones stacked together and a single
cord runs up the back and attaches to a set of clear glasses.

Then, I clip on my belt. You see, like Google glass, we are
present in the environment around us.

We can see each other, we're together, and
when we're present like I said.

So, I press the power button on her unit and I

press the power button on mine and
otherwise not interfering with the world around us,

is what will be the beginning of our faerie fantasy story.
Because you see, this forest that

we're standing in or a clearly defined
area of the forest has been previously scanned and plotted.

It has hidden GPS units throughout as well as tracking markers.

Our units and our computers also
also carry GPS and our head

units carry positional and tracking data
as well so that the computer not only

knows where we are within this forest, it
knows where we are in relation to

everything tangible in this forest and
like Google Glass overlays, the graphics

and the interactivity is allowed to be
that on top of this reality. So my

daughter looks up at me and I down at
her and we kind of shrug because we've

never experienced this story before so
we're not sure how it even begins.

Then, she spots something deep in the
forest and sure enough whipping through

the trees and coming out as fast as a
beautiful little fairy princess and

before we can even react, it's upon us.
She's frantic, and she's scared, she's

being hunted, and she's telling us this
because we can hear it in the earbuds.

She's being hunted by a troll and needs
our help and then zip, she's off in a way.

My daughter and I look at each other. We
smile and we take off after her.

Our fairy fantasy story that we're out doing
together has begun. Now, imagine the

creative freedom this will allow storytellers and imagine

as a consumer the adventures you're
going to be able to have with friends

and family out in the real world not
passively sitting back on a couch.

Storytellers can also write the stories
to shift and change and split based on

the decisions that are made in the
moment by the people playing or

experiencing the story like me and my daughter.

And once that forest has been scanned and plotted,
others can write stories to it as well.

Historians could walk us through what it
might be like to experience a Native

American camp as it existed hundreds of
years ago. Military strategists could simulate battles.

Play it at night and adults could experience a horror survival game.

But you know, not everything has to be frenetic fast paced
like a video game type of story.

It could be slow and plotted like a
novel broken into chapters and played on

weekends over many months because really
the possibilities are endless.

So for those of you who are interested to learn
more and follow this in the future,

I encourage you of three things. First of
course, you can check out Google Glass that exists.

It's expensive, but it exists.
Second, follow the upcoming retail

release of the Oculus Rift. You'll be
able to probably actually go to a

a Frys or something and try it on your
head at some point. And, third, if you have

a smart phone, I encourage you to go to
the App Store and actually check out

type in "virtual reality" or "augmented
reality games." There's some examples some

of them are hokey, some of them are fun,
but they'll all give you a general

understanding of what virtual reality
and augmented reality apps are like right now.

But augmented storytelling and
the technological underpinnings of that

can do so much more and be so much more
personal. So, I'd like to leave you with this thought.

My grandfather suffers from dementia and
although we're blessed that he is able to live with my mother,

of course she cannot spend every moment of the day with him.

How fantastic will it be when grant.. the
rooms that grandfather lives in have

been scanned and plotted and he can wear
one of these devices and still be

present in the environment that he is
most comfortable in. A companion can be

programmed to interact with him at a
basic level and play the tile and

card games that he loves so much,

and I'm not talking about a creepy attempt at a
photorealistic rendering of a human.

I don't think that would be appropriate,
but how about something simple like a

robotic orb that could float around the
environment that has a cartoon rendering

of a face and could interact with them
at a basic level when no one else is there.

I believe in those moments that he
lives in everyday because he lives in

the moment to moment that that would
help improve his quality of life and

actually depending on the quality of the
games and where the person like someone

who suffers from dementia is at on the
continuum, I actually think it has the

potential to improve their mental
functions as well. So, thank you for

spending a few minutes with me imagining
the near future and I encourage you to

stay engaged and evolve with it as it
becomes reality. Thank you.