Linear Algebra: Subspace Projection Matrix Example

Let's say I've got some subspace
V, which tends to be

our favorite letter for
subspaces, and it's equal to

the span of two vectors in R4.

Let's say that the first vector
is 1 0 0 1, and the

second vector is 0 1 0 1.

That is my subspace V.

And you can see that these
are going to be a basis.

That these are linearly
independent.

Two vectors that are linear-- or
any set of vectors that are

linearly independent and that
span a subspace are a basis

for that subspace.

You can see they are linearly
independent.

This guy's has a 1 here.

There's no way you can take some
combination of this guy

to somehow get a 1 there.

And this guy has a 1 here.

There's no way you can get some
linear combination of

these zeroes here
to a 1 there, so

they're linearly dependent.

You can also call this
a basis for V.

Now, given that, let's see
if we can find out the

transformation matrix for the
projection of any arbitrary

vector onto this subspace.

So let's say that X-- we're
dealing in R4 here, right?

Let's say that x is a member
of R4, and I want to figure

out a transformation
matrix for the

projection onto V of x.

Now, in the last video, we came
up with a general way to

figure this out.

We said if A is a transformation
matrix-- sorry.

If A is a matrix who's columns
are the basis for the

subspace, so let's say A is
equal to 1 0 0 1, 0 1 0 1.

So A is a matrix whose columns
are the basis for our

subspace, then the projection
of x onto V would be equal

to-- and this is kind of hard.

The first time you look at, it
gives you a headache, but

there's a certain pattern or
symmetry or a way of-- you

could say it's A times, you're
gonna have something in the

middle, and then you have A
transpose times your vector x.

And the way I remember it is in
the middle, you have these

two guys switched around.

So then you have A transpose
A, and you take

the inverse of it.

You probably won't be using
this in your everyday life

five or ten years from now, so
it's OK if you don't memorize

it, but temporarily, put this
in your medium-term memory

because it's a good thing
to know for doing these

projection problems.

So if we want to find the
general matrix for this

transformation, we just have to
determine what this matrix

is equal to, and that's just a
bunch of matrix operations.

So that's A.

What is A transpose?

A transpose is going to be equal
to just all the rows

turn into columns.

So the first column becomes
the first row.

So it becomes 1 0 0 1.

The second column becomes
the second row 0 1 0 1.

That's what A transpose is.

Now, what's A transpose A?

To figure out that, I want to
figure out what A transpose

times A is.

So let me multiply A
transpose times A.

So I'll rewrite A right here.

1 0 0 1, 0 1 0 1.

This is giving us some
good practice on

matrix-matrix products.

This is going to be
equal to what?

Well, first of all, this is
a 2-by-4 matrix, and I'm

multiplying it by a 4-by-2
matrix, so it's going to be a

2-by-2 matrix.

So the first entry is
essentially the dot product of

that row with that column.

So it's 1 times 1 plus 0
times 0 plus 0 times 0

plus 1 times 1.

So it's just going to be 2 for
that first entry right there.

And then you take the dot
product of this guy with this

guy right here.

So it's 1 times 0, which is 0,
plus 0 times 1, which is 0,

plus 0 time 0, which is 0, plus
1 times 1, which is 1.

Now, we do this guy dotted with
this column right there.

0 times 1 is 0 plus 1 time 0 is
0 plus 0 times 0 is 0 plus

1 times 1 is 1.

And then finally,
this row dotted

with this second column.

Second row, second column.

0 times 0 is 0, 1 times 1
is 1, 0 times 0 is 0,

1 times 1 is 1.

So we have 1 times
1 plus 1 times 1.

It's going to be 2.

It's going to be equal to 2.

So this right here
is A transpose A.

But that's not good enough.

We need to figure out what the
inverse of A transpose A is.

This is A transpose A.

But we need to figure out
A transpose A inverse.

So what's the inverse of this?

So let me write it here.

The inverse A transpose
A inverse is going

to be equal to what?

It's 1 over the determinant
of this guy.

What's the determinant here?

It's going to be 1 over the
determinant of this.

The determinant is 2 times 2,
which is 4, minus 1 times 1.

So it's 4 minus 1, which is 3.

So 1 over the determinant times
this guy, where if I

swap these two, so I swap the
1's-- sorry, I swap the 2's.

So this 2 goes here, and then
this orange 2 goes over here.

And then I make these
1's negative.

This becomes a minus 1 and
this becomes a minus 1.

We learned that this is a
general solution for the

inverse of a 2-by-2 matrix.

I think it was 10 or 11 videos
ago, and you probably learned

this in your Algebra II class,
frankly, but there you go.

We have A transpose A inverse.

So we have this guy.

We have this whole guy here
is just this matrix.

I could multiply the 1/3 into
it, but I don't have to do

that just yet.

But let's figure out the
whole matrix now.

The whole A times this
guy, A transpose A

inverse times A transpose.

Let me write it this way.

So the projection onto the
subspace V of x is going to be

equal to A.

1 0 0 1-- let me write a little
bit bigger like this.

So 1 0 0 1, 0 1 0 1 times A
transpose A inverse, right?

A times A transpose A inverse,
which is this guy right here.

Let's just put the 1/3 out
front just because

that's just a scalar.

I'll put the 1/3 out front
times this guy.

This A transpose A inverse
is 1/3 times 2

minus 1, minus 1, 2.

And then I'm going to multiply
it times A transpose.

And all that times
our vector x.

So A transpose is right there.

It is 1 0 0 1, 0 1 0 1.

And then all of that's going
to be times your vector x.

So we still have some
nice matrix-matrix

products ahead of us.

Let see if we can do these.

So the first one, let's just
multiply these two guys.

I don't think there's any
simple way to do it.

This is a 2-by-2 matrix and this
is a 2-by-4 matrix, so

when I multiply them,
I'm going to end up

with a 2-by-4 matrix.

Let me write that 2-by-4
matrix right here.

And then I can write this
guy right here.

1 0 0 1, 0 1 0 1.

And then I have the 1/3 that
was from A transpose A

inverse, but I put the scaling
factor out there.

And all of this is equal to the
projection of x onto V.

So let's do this product.

So this first entry is going to
be 2 times 1 plus minus 1

times 0, so that is just 2.

Then you're going to have 2
times 0 plus minus 1 times 1.

Well, that's minus 1.

Then you have 2 times 0
plus minus 1 times 0.

Well, that's just 0.

And then you're going to
have 2 times 1 plus

minus 1 times 1.

That's 2 minus 1.

That's just 1, right?

2 times 1 plus minus
1 times 1.

Fair enough.

Now, let's do the second row.

Minus 1 times 1 plus 2 times
0, so that's just minus 1.

Minus 1 times 0 plus
2 times 1.

Well, that's just 2.

Minus 1 times 0 plus
2 times 0.

That's just 0.

Minus 1 times 1 plus
2 times 1.

Well, that's minus 1 plus
2, so that is 1.

Almost there, and, of course, we
have to multiply it times x

at the end.

That's what the transformation
is.

But this right here is our
transformation matrix.

One more left to do.

Let's hope I haven't made any
careless mistakes and that I

won't make any when doing
this product.

This is going to be a little
more complicated because this

is a 4 by 2 times a 2 by 4.

I'm going to end up with
a 4-by-4 matrix.

Let me give myself some
breathing room here because

I'm going to generate a 4-by-4
matrix right there.

And so what am I going to get?

So this first entry is going
to be 1 times 2 plus

0 times minus 1.

So it's just going
to be equal to 2.

The next entry: 1 times-- this
row times any column here is

just going to be the first entry
in the column because it

gets zeroed out.

So 1 times 2 plus 0 times
minus 1 is just 2.

1 times minus 1 plus 0 times
2 is just minus 1.

1 times 0 plus 0 times 0 is 0.

1 times 1 plus 0 times
1 is just 1.

When you take this row and you
multiply it times these

columns, you literally just
got your first row there.

Now, let's do this row
times these columns.

Now, you've got a 0 here, so
you're going to have a 0 times

the first entry of all
of these and a 1

times the second one.

So 0 times 2 plus 1 times
minus 1 is minus 1.

0 times minus 1 plus
1 times 2 is 2.

You're just going to get
the second row here.

2 0 1.

That actually makes sense,
because if you just look at

this part of the matrix, it's
the 2-by-2 identity matrix.

So, anyway, that's a little hint
why this looks very much

like that, but we're just
going to go through this

matrix product.

Now, you multiply this-- let me
do it in a different color.

You multiply this guy times
each of these columns.

That guy dotted with that is
just going to 0 because this

guy's essentially the 0 row
vector, so you're just going

to get a bunch of zeroes.

And then, finally, this last
row, it's 1 times the first

entry plus 1 times
the second entry.

So this guy's going to be 2
plus minus 1, which is 1.

Minus 1 plus 2, which is 1.

0 plus 0, which is 0.

And then 1 plus 1, which is 2.

And all that times x.

And there you have it.

This is exciting!

The projection onto V of x
is equal to this whole

matrix times x.

So this thing right here, I
could multiply the 1/3 into

it, but we don't have
to do that.

That'll just make it a little
bit more messy.

This thing right here is the
transformation matrix.

As you can see, since we're
transforming-- remember, this

projection onto V,
this is a linear

transformation from R4 to R4.

You give me some member of R4,
and I'll give you another

member of R4 that's in my
subspace that is the

projection.

So this is going to be
a 4-by-4 You can

see it right there.

Anyway, hopefully, you found
that useful to actually see a

tangible result.

R4 is very abstract, so this
would even be beyond our

three-dimensional programming
example.

We're dealing with a more
abstract data set where we're

interested in finding
a projection.