11.1: Capture and Live Video - Processing Tutorial

Okay, so, we are now ready to move on

to something, pretty,
different and exciting

um, and what that is, is
looking at moving images

so we spent all this time saying, okay,
I could have a blank PImage

and i could figure out some
interesting algorithmic way

to set all the pixels of that image

or, I could have an image thats coming
from a file, like frog.jpg

and i could load that image, and i'll
have all the access to all the pixels

and i could draw it on the screen
and i could manipulate it

and do all sorts of stuff
what if, however,

we could have a camera,
this is my weird drawing of a camera

and what if i could pull in a live image
from a camera into my Processing sketch

What could I do with that? what if I had a
you know, a movie file?

could I pull in that movie and play back
that movie in my Processing sketch?

speeding it up, slowing it down,
going in reverse

how could I-, how does all this
stuff work?

How do we make this happen? So, let's
first just take the basic transition from

flat image file to camera and the way
that we're going to do this

is with Processing's Capture object
So, the Capture object, is part of,

Processing's video package
Now, I just realized,

there's kinda a crucial component to this,
if you are using, and again, this part of

the video, sadly, is going to just
become obsolete in a little while

but, if you are using the 3.0 alpha series
and, hopefully, there will be a beta soon

or a final version, this will be true for
all the 3.0 version's of Processing

Let's come over here and look, uh, here,
if i go to, Import Library

let me just zoom in here
Import Library >> Add Library

and i pull this up, you won't, by default,
have the video library, so I'm going to

search under Processing, and this is what
you're looking for, you wanna find

in this contributions manager, 'Video'
library by The Processing Foundation

so, in order to run any of these examples,
or, to work with live video

or recorded movies in Processing, you need
this particular video library

in all the previous versions of Processing
prior to 3.0, this library came with

Processing, but its quite a large... large
library, also it's maintained and updated

on a separate schedule, so by pulling it
in separately, all you need to do is,

Hey! Whoa! I haven't actually- Amazing!
I forgot that i just did a fresh install

of Processing here, so, this is-,
we're going to test this out

you just wanna hit this 'install' button
and, uh, while it's downloading

and should be ready to go, and now,
we're going to have this library!

yeah! and then it'll say 'remove', so, if
you don't have the video library,

make sure you get it (button click)
okay, back over here!

Once you have the video library, you
now have access to the Capture object

you also have access to the Movie object

so, Capture being live video from some
camera, that's connected to your computer

Movie object, you'll use for video files,
that you're loading and playing

back in Processing, so

here's the thing, the thing that's
the most wonderful about this

is that, both of these things are PImage's
essentially, all the functionality; image,

pixels, width, height, copy
any piece of functionality, you learned,

and practiced, and did with PImage, you
could do exactly the same functionality

with Capture and Movie

the only thing that's different, is when
you load a PImage, like frog.jpg,

those pixels are never gonna change,
unless you change them, right?

that file, that image is a
flat, static image

those pixels are the pixels

you could do all the same operations! but
with a Capture object or a Movie object,

those pixels might be changing according
to some schedule, of course,

we have total control over that schedule,
we could say, give me fresh pixels from

the camera, or stop giving me fresh
pixelsfrom the camera, or,

advance the next frame of the movie, or,
don't advance the next frame of the movie,

so this is an added piece of the puzzle,
that we'll be able to work with

as we start using these classes, so, let's
sort of, uh, let's look and see how this

would work, there's a couple little
nuances here, so, i'm back over here,

and i'm in a sketch, so, this is just a,
kind of a simple processing sketch

that I think we did at some point in an
earlier video, I'm loading an image,

I'm drawing it on the screen,
I'm using a couple things;

#1 - you can see I can dynamically resize
the image as i draw it, using the mouse,

or I could also tint it, meaning,
change its color, so,

what I want to emphasize in this-,
just first part of this video,

is let's make this the Capture object from
the camera, I have a laptop in front of me

its not all magic, let's get that image
from that camera, and let's see that there

and do all the same stuff for it,
so, instead-, the first step is,

instead of a PImage, I wanna say,
Capture and I'm gonna say video

now notice, as soon as I do that, I get
an error message, it says, um,

if I look at the errors, you can see,
the class Capture does not exist,

now this needs a little bit of work here,
but it's actually giving me a suggestion

which is nice, this is-, Capture and Movie
are both part of that Processing video

library that we just installed, so even
though I've installed the library

I have to-, in my code, make an explicit
reference that i'm going to use it, so,

I'm just gonna-, Processing will
add this for you automatically,

in a bunch of different ways,
but I'm just gonna do it manually,

so I just need to have an import statement
at the top; import processing.video.*;

now I have my Capture object, now,

you might think, you would say
something like: "I said loadImage(),

so, loadCapture (should work) or
something? createCapture, connectVideo,

startVideo", um, the way that
this works however, is,

there is no function like that, we need to
use the tools and syntax of

object oriented programming,
so lets come back over here

and if I have a video object, the syntax
is to say, I wanna make a new...

Capture object, and I need to figure out,
what goes in there, so,

what might be some parameters
that you need

when you're about to connect to
the live camera?

maybe you might say:
"Ah! camera over there!

I would like a very high resolution image"
or "a very low resolution image"

so, some parameters we're going to
put in there as the width and the height

of our requested, uh-, images
that we'll get from the camera

another piece of information that we
might put over here, is... frame rate

do i want to get images from the camera
at, 30 frames? 60 frames? 120 frames?

15 frames?

now, with both of these-, frames per
second, so, with both of these,

certain cameras are only
going to support

certain resolutions and
certain frame rates,

so, a lot of this really depends
on what you've got,

and I'm going to show
you a way to figure out

what your camera makes available to you

but generally speaking, 640x480, 30 FPS,
those are, kinda good numbers to use,

and in fact, this (meaning >> FPS)
is an optional parameter

you're just gonna get the default frame
rate, if you don't request a specific one,

which is usually 30 frames per second

now, there's another argument
that needs to go in here

and this is one of those tricky one
that appears, when you're using-,

a lot of times when you're using a
Processing library

One of the things we need to do, is, we
need to say: "A Capture object camera",

"you're (referring to camera) going
to have new images available"

"you're looking at the room,
you're looking at me,"

"you know where I want you
to give those images?"

"I want you to give those images to me"

who's 'me'?
this particular Processing sketch,

'this' sketch,

so the keyword 'this' needs to go in here,

now we could have a much longer discussion
about what the keyword 'this' means

and Java, and then we can sit here and
complain for a while

about how confusing these, like, archaic
concepts, and like, old-fashioned

languages like Java are

but for-, we don't really need to
worry about it too much

to make all this stuff work, we just need
to remember the keyword 'this' goes there

and I think it's kinda useful, to kind of
think about the fact that 'this'

is referring to this Processing sketch

I don't want the camera images to go to
some other program on the computer

I want them to go to this Processing
sketch, and we're telling the library that

this-, this happens in a lot of other ways
behind the scenes in Processing,

but you don't always have
to use that keyword 'this'