Return to Video

How volumetric video brings a new dimension to filmmaking

  • 0:01 - 0:03
    I love making movies.
  • 0:05 - 0:09
    Motion pictures have been in existence
    for more than a hundred years.
  • 0:10 - 0:14
    Filmmaking hasn't changed
    for the dimensional mindset.
  • 0:15 - 0:19
    Placing the camera in a scene
    and pressing "record" hasn't changed.
  • 0:19 - 0:23
    Filmmaking is still a frontal experience,
  • 0:23 - 0:26
    and creating the film has the possibility
  • 0:26 - 0:30
    to follow the same direction
    of the content creation.
  • 0:30 - 0:33
    We still stand in front of a flat image,
  • 0:33 - 0:35
    watching the fiction.
  • 0:36 - 0:38
    There's nothing wrong with it.
  • 0:38 - 0:41
    I love watching movies
    and going to the theaters.
  • 0:41 - 0:43
    The experiences can be
    such emotional experiences.
  • 0:44 - 0:47
    The art and craft of emotional
    experiences within a frame
  • 0:47 - 0:50
    can be so strong
    to drive a stronger emotion.
  • 0:51 - 0:54
    The question we're asking is,
  • 0:54 - 0:58
    How the experience of motion pictures
    can exist beyond the flat screen.
  • 0:59 - 1:00
    How can we start creating content
  • 1:00 - 1:03
    for the next generation
    of content experiences?
  • 1:04 - 1:07
    Traditionally, when we imagine a scene,
  • 1:07 - 1:09
    we look at the frame and the composition.
  • 1:10 - 1:13
    We have to think about how we create
    depth and parallax
  • 1:13 - 1:18
    using foreground, background elements
    as the camera moves.
  • 1:18 - 1:22
    With the technology today
    and devices of VR glasses,
  • 1:22 - 1:25
    AR glasses, smart devices,
  • 1:25 - 1:29
    allowing three-dimensional
    and full navigation in space,
  • 1:29 - 1:31
    we have the possibility
    to enable audiences
  • 1:31 - 1:34
    to experience content
    from multiple perspectives.
  • 1:35 - 1:38
    What we have to think about
    is how we take this technology,
  • 1:38 - 1:40
    all the capabilities,
  • 1:40 - 1:45
    and enable the experience
    to move farther away inside the scene.
  • 1:45 - 1:49
    Now we're not talking about video games
    or computer-generated actors,
  • 1:49 - 1:52
    which look tremendously realistic.
  • 1:52 - 1:55
    We're talking about real actors
    and real performance,
  • 1:55 - 1:57
    performing onstage.
  • 1:58 - 2:02
    We have to start thinking
    how we capture the actors
  • 2:02 - 2:04
    and how we capture the real scene
  • 2:04 - 2:06
    in order to immerse inside.
  • 2:06 - 2:10
    Now, we're familiar
    with the 360-degree video,
  • 2:10 - 2:12
    where you place a camera inside the scene
  • 2:12 - 2:16
    and you can create this beautiful
    panoramic image all around you,
  • 2:16 - 2:18
    but from the same aspect,
  • 2:18 - 2:20
    filmmaking is still frontal.
  • 2:20 - 2:23
    In order to emerge fully inside the scene,
  • 2:23 - 2:27
    we will need to capture the light
    from all the possible directions.
  • 2:27 - 2:32
    We will have to surround the scene
    with an enormous amount of sensors,
  • 2:32 - 2:36
    with all possible capabilities
    to capture the light
  • 2:36 - 2:38
    and enable us to emerge
    inside afterwards again.
  • 2:40 - 2:41
    Now, in this setup,
  • 2:41 - 2:44
    there's no more foreground or background
  • 2:44 - 2:45
    or a camera placed in space
  • 2:46 - 2:49
    but hundreds of sensors
    capturing the light
  • 2:49 - 2:52
    and capturing the motion
    from all the possible directions.
  • 2:53 - 2:55
    With the new technological advancements,
  • 2:55 - 2:58
    we can start looking at 3D photography,
  • 2:58 - 3:01
    capturing the light
    from multiple perspectives,
  • 3:01 - 3:03
    enabling us to reconstruct the object.
  • 3:04 - 3:06
    This is like photography in 3D space.
  • 3:06 - 3:09
    Now, with these
    technological advancements,
  • 3:09 - 3:13
    we can record video
    not just as a flat image
  • 3:13 - 3:14
    but as a volume.
  • 3:14 - 3:17
    This is what we call "volumetric video,"
  • 3:17 - 3:21
    and it has the capability
    to record every action of the scene
  • 3:21 - 3:22
    as a full three-dimensional volume.
  • 3:23 - 3:25
    Now, what is a voxel?
  • 3:26 - 3:29
    A voxel is like a three-dimensional pixel,
  • 3:29 - 3:35
    but instead of being a flat image square,
    staying light and colored,
  • 3:35 - 3:38
    it's like a three-dimensional
    cube in space,
  • 3:38 - 3:40
    with x, y, and z positions.
  • 3:41 - 3:45
    This enables us to create
    a full capture of the scene
  • 3:45 - 3:46
    from any perspective.
  • 3:47 - 3:52
    Now this renders
    a fully light-immersive scene
  • 3:52 - 3:54
    from multiple perspectives.
  • 3:54 - 3:59
    This capability requires an insane amount
    of information to be processed.
  • 3:59 - 4:04
    We will have to capture the light
    from an enormous amount of cameras
  • 4:04 - 4:05
    to create this information.
  • 4:07 - 4:10
    Now, in order to do such a thing,
  • 4:10 - 4:15
    we would need a setup that would host
    a numerous amount of cameras
  • 4:16 - 4:18
    installed in a stage
  • 4:18 - 4:23
    and a stage big enough in order to fit
    a full cinematic experience.
  • 4:24 - 4:29
    Now that sounds like a crazy idea,
    but that's exactly what we did.
  • 4:30 - 4:31
    For the last three years,
  • 4:31 - 4:35
    we have been building
    a huge volumetric camera chamber.
  • 4:36 - 4:39
    It's 10,000 square feet of a stage,
  • 4:39 - 4:42
    enabling to capture the action
    from any location.
  • 4:43 - 4:45
    We have deployed hundreds of cameras,
  • 4:45 - 4:48
    sending a tremendous amount of information
  • 4:48 - 4:52
    to a huge data center
    powered by Intel supercomputers.
  • 4:53 - 4:56
    The ability to have this 10,000 feet
  • 4:56 - 4:59
    enables us to fit any kind of action,
  • 4:59 - 5:01
    any kind of performance.
  • 5:01 - 5:04
    It is the size of an average
    Broadway stage.
  • 5:05 - 5:07
    We call it Intel Studios,
  • 5:07 - 5:10
    and it's the largest volumetric
    stage in the world,
  • 5:10 - 5:14
    with the objective
    of enabling and exploring
  • 5:14 - 5:17
    the next generation
    of this immersive media filmmaking.
  • 5:18 - 5:21
    Now, to test these ideas,
  • 5:22 - 5:27
    we were thinking about what we can do
    as the first scene to try it out.
  • 5:28 - 5:29
    So we chose the Western scene.
  • 5:30 - 5:35
    We brought horses, set designers, dirt,
  • 5:35 - 5:39
    everything needed to create
    the full scene of a Western.
  • 5:39 - 5:42
    But this time, there was no camera inside.
  • 5:42 - 5:46
    There was nothing really moving
    besides all the cameras
  • 5:46 - 5:48
    installed outside.
  • 5:48 - 5:51
    The challenge
    of the actors was tremendous.
  • 5:51 - 5:57
    They have to perform a flawless action
    visible from all the directions.
  • 5:58 - 6:02
    There's no possibility to hide a punch
    or not show the action.
  • 6:03 - 6:05
    Everything is captured
    and everything is seen.
  • 6:06 - 6:09
    The output of the capture --
  • 6:09 - 6:11
    this is our future capture --
  • 6:11 - 6:14
    opened our eyes for
    the immense capabilities.
  • 6:14 - 6:18
    It was like a full 3D scan
    of the entire scene.
  • 6:18 - 6:21
    We were able to move around
    and travel in the space.
  • 6:22 - 6:23
    The thing about this,
  • 6:23 - 6:26
    it's not anymore about perceiving
    the light emitted from a screen
  • 6:26 - 6:29
    but now traveling inside the light,
  • 6:29 - 6:31
    traveling inside the scene.
  • 6:31 - 6:34
    This obviously opens possibilities
    for an enormous amount
  • 6:34 - 6:37
    of storytelling
    and methodologies of creation.
  • 6:38 - 6:41
    This is the possibilities
    of your personal narrative,
  • 6:41 - 6:44
    the possibility of creating
    your own story inside,
  • 6:44 - 6:46
    or maybe following other stories.
  • 6:47 - 6:50
    Let's take a look
    at one of the last renders and see.
  • 6:50 - 6:51
    (Music)
  • 6:51 - 6:54
    What you're seeing here
    is full volumetric video,
  • 6:54 - 6:56
    and there's no physical
    camera in the scene.
  • 6:56 - 6:58
    (Music)
  • 6:58 - 7:00
    We have the full control
  • 7:00 - 7:01
    (Music, sounds of combat)
  • 7:02 - 7:03
    of space and time.
  • 7:03 - 7:10
    (Music, sounds of combat)
  • 7:14 - 7:17
    Now, again, no physical camera was here.
  • 7:17 - 7:20
    Everything was captured surrounding.
  • 7:20 - 7:22
    Now, this is very nice,
  • 7:22 - 7:26
    but what if we wanted to see the scene,
    maybe, from the eyes of the horse?
  • 7:27 - 7:29
    Well, we can do that as well.
  • 7:29 - 7:30
    (Horse galloping)
  • 7:30 - 7:34
    So what you're seeing right now
    is the same action,
  • 7:34 - 7:37
    but this time, we're watching
    exactly from the eyes of the horse.
  • 7:37 - 7:41
    The possibilities are, well, unlimited.
  • 7:41 - 7:42
    (Applause)
  • 7:42 - 7:44
    Thank you.
  • 7:44 - 7:47
    (Applause)
  • 7:47 - 7:51
    So this is all great
    for creators and storytellers.
  • 7:51 - 7:53
    It really opens a huge canvas
  • 7:53 - 7:56
    for a different type
    of storytelling and moviemaking.
  • 7:56 - 7:58
    But what about the audience?
  • 7:58 - 8:00
    How can the audience
    experience this differently?
  • 8:01 - 8:03
    In order to [create] our explorations,
  • 8:03 - 8:05
    we partnered with Paramount Pictures
  • 8:05 - 8:09
    in order to explore immersive media
    in a Hollywood movie production.
  • 8:09 - 8:11
    Together with the director Randal Kleiser,
  • 8:11 - 8:16
    we reimagined the iconic movie of 1978,
  • 8:16 - 8:17
    "Grease."
  • 8:17 - 8:20
    Some of you know it, some of you don't.
  • 8:20 - 8:23
    A 40-year-old movie, amazing experience.
  • 8:23 - 8:30
    And our goal was really to look at how
    we can take this iconic action and dance
  • 8:30 - 8:33
    and bring it deeper into the experience,
  • 8:33 - 8:35
    bring it deeper into the audience.
  • 8:35 - 8:37
    Imagine that you can
    not just watch the movie
  • 8:37 - 8:41
    but get inside it
    and dance with the actors
  • 8:41 - 8:42
    and dance with the performance.
  • 8:42 - 8:48
    Now we're breaking, really,
    the traditional 2D mindset of thinking,
  • 8:48 - 8:51
    and bringing a much richer
    possibility of moviemaking
  • 8:51 - 8:52
    and content creation.
  • 8:53 - 8:55
    But why watch it on the screen?
  • 8:56 - 8:58
    Let's try to bring these actors
    here on the stage.
  • 9:05 - 9:07
    So they're not going to really come --
  • 9:07 - 9:08
    I'm going to use an iPad.
  • 9:08 - 9:09
    (Laughter)
  • 9:09 - 9:11
    Sorry.
  • 9:11 - 9:15
    I'm going to use an iPad
    in order to bring in augmented reality.
  • 9:15 - 9:20
    Now, obviously, these devices
    have their own limitations
  • 9:21 - 9:23
    in terms of the data-computing process,
  • 9:23 - 9:25
    so we have to reduce
    the amount of resolution.
  • 9:27 - 9:31
    So what I'm doing now,
    I'm placing here a marker,
  • 9:31 - 9:36
    so I'll be able to position exactly
    where I want everyone to appear.
  • 9:43 - 9:44
    OK.
  • 9:44 - 9:45
    I think we have them here.
  • 9:46 - 9:52
    (Applause)
  • 9:52 - 9:54
    John Travolta, or --
  • 9:54 - 9:55
    (Laughter)
  • 9:55 - 9:56
    a version of him.
  • 9:57 - 9:58
    Let's take a look.
  • 9:58 - 10:00
    (Video) Female: Hey.
  • 10:00 - 10:01
    Male: And that is how it's done.
  • 10:01 - 10:02
    Female: Your turn.
  • 10:02 - 10:04
    Male: Hey, guys! Check this out.
  • 10:04 - 10:07
    (Song: "You're the one that I want")
  • 10:07 - 10:09
    Danny: Sandy!
  • 10:09 - 10:11
    Sandy: Tell me about it, stud.
  • 10:12 - 10:16
    (Singing) I got chills.
    They're multiplying
  • 10:17 - 10:21
    And I'm losing control
  • 10:21 - 10:25
    'Cause the power you're supplying
  • 10:25 - 10:27
    It's electrifying!
  • 10:27 - 10:28
    (Video ends)
  • 10:28 - 10:34
    (Applause and cheers)
  • 10:34 - 10:35
    Diego Prilusky: Thank you.
  • 10:35 - 10:37
    (Applause and cheers)
  • 10:38 - 10:39
    So as you can see,
  • 10:40 - 10:44
    we can watch and experience content
    in the traditional way
  • 10:44 - 10:45
    or in an immersive way.
  • 10:46 - 10:48
    Really, the possibilities are open.
  • 10:49 - 10:52
    We're not trying to change
    or replace movies.
  • 10:53 - 10:54
    We're enhancing them.
  • 10:54 - 10:59
    The technologies enable new possibilities
    to start thinking beyond the flat screen.
  • 11:00 - 11:04
    We're in immersive and really
    exciting times in filmmaking.
  • 11:04 - 11:07
    We're at the threshold of a new era.
  • 11:07 - 11:10
    We're opening the gates
    for new possibilities
  • 11:10 - 11:12
    of immersive storytelling,
  • 11:13 - 11:18
    and exploration and defining
    what immersive media filmmaking means.
  • 11:19 - 11:21
    We're really just at the beginning,
  • 11:21 - 11:24
    and we invite you all to join us.
  • 11:24 - 11:25
    Thank you.
  • 11:25 - 11:30
    (Applause)
标题:
How volumetric video brings a new dimension to filmmaking
演讲者:
Diego Prilusky
描述:

In this talk and tech demo, filmmaker Diego Prilusky introduces the next chapter in moviemaking: volumetric video, a 360-degree experience powered by hundreds of cameras that capture light and motion from every angle. Check out how this technology creates a new type of immersive storytelling -- and catch Prilusky's reshoot of an iconic dance number from "Grease" captured with volumetric video.

more » « less
Video Language:
English
Team:
TED
项目:
TEDTalks
Duration:
11:43

English subtitles

修订 Compare revisions