How volumetric video brings a new dimension to filmmaking
-
0:01 - 0:03I love making movies.
-
0:05 - 0:09Motion pictures have been in existence
for more than a hundred years. -
0:10 - 0:14Filmmaking hasn't changed
for the dimensional mindset. -
0:15 - 0:19Placing the camera in a scene
and pressing "record" hasn't changed. -
0:19 - 0:23Filmmaking is still a frontal experience,
-
0:23 - 0:26and creating the film has the possibility
-
0:26 - 0:30to follow the same direction
of the content creation. -
0:30 - 0:33We still stand in front of a flat image,
-
0:33 - 0:35watching the fiction.
-
0:36 - 0:38There's nothing wrong with it.
-
0:38 - 0:41I love watching movies
and going to the theaters. -
0:41 - 0:43The experiences can be
such emotional experiences. -
0:44 - 0:47The art and craft of emotional
experiences within a frame -
0:47 - 0:50can be so strong
to drive a stronger emotion. -
0:51 - 0:54The question we're asking is,
-
0:54 - 0:58How the experience of motion pictures
can exist beyond the flat screen. -
0:59 - 1:00How can we start creating content
-
1:00 - 1:03for the next generation
of content experiences? -
1:04 - 1:07Traditionally, when we imagine a scene,
-
1:07 - 1:09we look at the frame and the composition.
-
1:10 - 1:13We have to think about how we create
depth and parallax -
1:13 - 1:18using foreground, background elements
as the camera moves. -
1:18 - 1:22With the technology today
and devices of VR glasses, -
1:22 - 1:25AR glasses, smart devices,
-
1:25 - 1:29allowing three-dimensional
and full navigation in space, -
1:29 - 1:31we have the possibility
to enable audiences -
1:31 - 1:34to experience content
from multiple perspectives. -
1:35 - 1:38What we have to think about
is how we take this technology, -
1:38 - 1:40all the capabilities,
-
1:40 - 1:45and enable the experience
to move farther away inside the scene. -
1:45 - 1:49Now we're not talking about video games
or computer-generated actors, -
1:49 - 1:52which look tremendously realistic.
-
1:52 - 1:55We're talking about real actors
and real performance, -
1:55 - 1:57performing onstage.
-
1:58 - 2:02We have to start thinking
how we capture the actors -
2:02 - 2:04and how we capture the real scene
-
2:04 - 2:06in order to immerse inside.
-
2:06 - 2:10Now, we're familiar
with the 360-degree video, -
2:10 - 2:12where you place a camera inside the scene
-
2:12 - 2:16and you can create this beautiful
panoramic image all around you, -
2:16 - 2:18but from the same aspect,
-
2:18 - 2:20filmmaking is still frontal.
-
2:20 - 2:23In order to emerge fully inside the scene,
-
2:23 - 2:27we will need to capture the light
from all the possible directions. -
2:27 - 2:32We will have to surround the scene
with an enormous amount of sensors, -
2:32 - 2:36with all possible capabilities
to capture the light -
2:36 - 2:38and enable us to emerge
inside afterwards again. -
2:40 - 2:41Now, in this setup,
-
2:41 - 2:44there's no more foreground or background
-
2:44 - 2:45or a camera placed in space
-
2:46 - 2:49but hundreds of sensors
capturing the light -
2:49 - 2:52and capturing the motion
from all the possible directions. -
2:53 - 2:55With the new technological advancements,
-
2:55 - 2:58we can start looking at 3D photography,
-
2:58 - 3:01capturing the light
from multiple perspectives, -
3:01 - 3:03enabling us to reconstruct the object.
-
3:04 - 3:06This is like photography in 3D space.
-
3:06 - 3:09Now, with these
technological advancements, -
3:09 - 3:13we can record video
not just as a flat image -
3:13 - 3:14but as a volume.
-
3:14 - 3:17This is what we call "volumetric video,"
-
3:17 - 3:21and it has the capability
to record every action of the scene -
3:21 - 3:22as a full three-dimensional volume.
-
3:23 - 3:25Now, what is a voxel?
-
3:26 - 3:29A voxel is like a three-dimensional pixel,
-
3:29 - 3:35but instead of being a flat image square,
staying light and colored, -
3:35 - 3:38it's like a three-dimensional
cube in space, -
3:38 - 3:40with x, y, and z positions.
-
3:41 - 3:45This enables us to create
a full capture of the scene -
3:45 - 3:46from any perspective.
-
3:47 - 3:52Now this renders
a fully light-immersive scene -
3:52 - 3:54from multiple perspectives.
-
3:54 - 3:59This capability requires an insane amount
of information to be processed. -
3:59 - 4:04We will have to capture the light
from an enormous amount of cameras -
4:04 - 4:05to create this information.
-
4:07 - 4:10Now, in order to do such a thing,
-
4:10 - 4:15we would need a setup that would host
a numerous amount of cameras -
4:16 - 4:18installed in a stage
-
4:18 - 4:23and a stage big enough in order to fit
a full cinematic experience. -
4:24 - 4:29Now that sounds like a crazy idea,
but that's exactly what we did. -
4:30 - 4:31For the last three years,
-
4:31 - 4:35we have been building
a huge volumetric camera chamber. -
4:36 - 4:39It's 10,000 square feet of a stage,
-
4:39 - 4:42enabling to capture the action
from any location. -
4:43 - 4:45We have deployed hundreds of cameras,
-
4:45 - 4:48sending a tremendous amount of information
-
4:48 - 4:52to a huge data center
powered by Intel supercomputers. -
4:53 - 4:56The ability to have this 10,000 feet
-
4:56 - 4:59enables us to fit any kind of action,
-
4:59 - 5:01any kind of performance.
-
5:01 - 5:04It is the size of an average
Broadway stage. -
5:05 - 5:07We call it Intel Studios,
-
5:07 - 5:10and it's the largest volumetric
stage in the world, -
5:10 - 5:14with the objective
of enabling and exploring -
5:14 - 5:17the next generation
of this immersive media filmmaking. -
5:18 - 5:21Now, to test these ideas,
-
5:22 - 5:27we were thinking about what we can do
as the first scene to try it out. -
5:28 - 5:29So we chose the Western scene.
-
5:30 - 5:35We brought horses, set designers, dirt,
-
5:35 - 5:39everything needed to create
the full scene of a Western. -
5:39 - 5:42But this time, there was no camera inside.
-
5:42 - 5:46There was nothing really moving
besides all the cameras -
5:46 - 5:48installed outside.
-
5:48 - 5:51The challenge
of the actors was tremendous. -
5:51 - 5:57They have to perform a flawless action
visible from all the directions. -
5:58 - 6:02There's no possibility to hide a punch
or not show the action. -
6:03 - 6:05Everything is captured
and everything is seen. -
6:06 - 6:09The output of the capture --
-
6:09 - 6:11this is our future capture --
-
6:11 - 6:14opened our eyes for
the immense capabilities. -
6:14 - 6:18It was like a full 3D scan
of the entire scene. -
6:18 - 6:21We were able to move around
and travel in the space. -
6:22 - 6:23The thing about this,
-
6:23 - 6:26it's not anymore about perceiving
the light emitted from a screen -
6:26 - 6:29but now traveling inside the light,
-
6:29 - 6:31traveling inside the scene.
-
6:31 - 6:34This obviously opens possibilities
for an enormous amount -
6:34 - 6:37of storytelling
and methodologies of creation. -
6:38 - 6:41This is the possibilities
of your personal narrative, -
6:41 - 6:44the possibility of creating
your own story inside, -
6:44 - 6:46or maybe following other stories.
-
6:47 - 6:50Let's take a look
at one of the last renders and see. -
6:50 - 6:51(Music)
-
6:51 - 6:54What you're seeing here
is full volumetric video, -
6:54 - 6:56and there's no physical
camera in the scene. -
6:56 - 6:58(Music)
-
6:58 - 7:00We have the full control
-
7:00 - 7:01(Music, sounds of combat)
-
7:02 - 7:03of space and time.
-
7:03 - 7:10(Music, sounds of combat)
-
7:14 - 7:17Now, again, no physical camera was here.
-
7:17 - 7:20Everything was captured surrounding.
-
7:20 - 7:22Now, this is very nice,
-
7:22 - 7:26but what if we wanted to see the scene,
maybe, from the eyes of the horse? -
7:27 - 7:29Well, we can do that as well.
-
7:29 - 7:30(Horse galloping)
-
7:30 - 7:34So what you're seeing right now
is the same action, -
7:34 - 7:37but this time, we're watching
exactly from the eyes of the horse. -
7:37 - 7:41The possibilities are, well, unlimited.
-
7:41 - 7:42(Applause)
-
7:42 - 7:44Thank you.
-
7:44 - 7:47(Applause)
-
7:47 - 7:51So this is all great
for creators and storytellers. -
7:51 - 7:53It really opens a huge canvas
-
7:53 - 7:56for a different type
of storytelling and moviemaking. -
7:56 - 7:58But what about the audience?
-
7:58 - 8:00How can the audience
experience this differently? -
8:01 - 8:03In order to [create] our explorations,
-
8:03 - 8:05we partnered with Paramount Pictures
-
8:05 - 8:09in order to explore immersive media
in a Hollywood movie production. -
8:09 - 8:11Together with the director Randal Kleiser,
-
8:11 - 8:16we reimagined the iconic movie of 1978,
-
8:16 - 8:17"Grease."
-
8:17 - 8:20Some of you know it, some of you don't.
-
8:20 - 8:23A 40-year-old movie, amazing experience.
-
8:23 - 8:30And our goal was really to look at how
we can take this iconic action and dance -
8:30 - 8:33and bring it deeper into the experience,
-
8:33 - 8:35bring it deeper into the audience.
-
8:35 - 8:37Imagine that you can
not just watch the movie -
8:37 - 8:41but get inside it
and dance with the actors -
8:41 - 8:42and dance with the performance.
-
8:42 - 8:48Now we're breaking, really,
the traditional 2D mindset of thinking, -
8:48 - 8:51and bringing a much richer
possibility of moviemaking -
8:51 - 8:52and content creation.
-
8:53 - 8:55But why watch it on the screen?
-
8:56 - 8:58Let's try to bring these actors
here on the stage. -
9:05 - 9:07So they're not going to really come --
-
9:07 - 9:08I'm going to use an iPad.
-
9:08 - 9:09(Laughter)
-
9:09 - 9:11Sorry.
-
9:11 - 9:15I'm going to use an iPad
in order to bring in augmented reality. -
9:15 - 9:20Now, obviously, these devices
have their own limitations -
9:21 - 9:23in terms of the data-computing process,
-
9:23 - 9:25so we have to reduce
the amount of resolution. -
9:27 - 9:31So what I'm doing now,
I'm placing here a marker, -
9:31 - 9:36so I'll be able to position exactly
where I want everyone to appear. -
9:43 - 9:44OK.
-
9:44 - 9:45I think we have them here.
-
9:46 - 9:52(Applause)
-
9:52 - 9:54John Travolta, or --
-
9:54 - 9:55(Laughter)
-
9:55 - 9:56a version of him.
-
9:57 - 9:58Let's take a look.
-
9:58 - 10:00(Video) Female: Hey.
-
10:00 - 10:01Male: And that is how it's done.
-
10:01 - 10:02Female: Your turn.
-
10:02 - 10:04Male: Hey, guys! Check this out.
-
10:04 - 10:07(Song: "You're the one that I want")
-
10:07 - 10:09Danny: Sandy!
-
10:09 - 10:11Sandy: Tell me about it, stud.
-
10:12 - 10:16(Singing) I got chills.
They're multiplying -
10:17 - 10:21And I'm losing control
-
10:21 - 10:25'Cause the power you're supplying
-
10:25 - 10:27It's electrifying!
-
10:27 - 10:28(Video ends)
-
10:28 - 10:34(Applause and cheers)
-
10:34 - 10:35Diego Prilusky: Thank you.
-
10:35 - 10:37(Applause and cheers)
-
10:38 - 10:39So as you can see,
-
10:40 - 10:44we can watch and experience content
in the traditional way -
10:44 - 10:45or in an immersive way.
-
10:46 - 10:48Really, the possibilities are open.
-
10:49 - 10:52We're not trying to change
or replace movies. -
10:53 - 10:54We're enhancing them.
-
10:54 - 10:59The technologies enable new possibilities
to start thinking beyond the flat screen. -
11:00 - 11:04We're in immersive and really
exciting times in filmmaking. -
11:04 - 11:07We're at the threshold of a new era.
-
11:07 - 11:10We're opening the gates
for new possibilities -
11:10 - 11:12of immersive storytelling,
-
11:13 - 11:18and exploration and defining
what immersive media filmmaking means. -
11:19 - 11:21We're really just at the beginning,
-
11:21 - 11:24and we invite you all to join us.
-
11:24 - 11:25Thank you.
-
11:25 - 11:30(Applause)
- Title:
- How volumetric video brings a new dimension to filmmaking
- Speaker:
- Diego Prilusky
- Description:
-
In this talk and tech demo, filmmaker Diego Prilusky introduces the next chapter in moviemaking: volumetric video, a 360-degree experience powered by hundreds of cameras that capture light and motion from every angle. Check out how this technology creates a new type of immersive storytelling -- and catch Prilusky's reshoot of an iconic dance number from "Grease" captured with volumetric video.
- Video Language:
- English
- Team:
- closed TED
- Project:
- TEDTalks
- Duration:
- 11:43
Erin Gregory edited English subtitles for How volumetric video brings a new dimension to filmmaking | ||
Erin Gregory edited English subtitles for How volumetric video brings a new dimension to filmmaking | ||
Erin Gregory edited English subtitles for How volumetric video brings a new dimension to filmmaking | ||
Erin Gregory approved English subtitles for How volumetric video brings a new dimension to filmmaking | ||
Erin Gregory edited English subtitles for How volumetric video brings a new dimension to filmmaking | ||
Krystian Aparta accepted English subtitles for How volumetric video brings a new dimension to filmmaking | ||
Krystian Aparta edited English subtitles for How volumetric video brings a new dimension to filmmaking | ||
Joseph Geni edited English subtitles for How volumetric video brings a new dimension to filmmaking |