< Return to Video

Linear Algebra: Subspace Projection Matrix Example

  • 0:01 - 0:04
    Let's say I've got some subspace
    V, which tends to be
  • 0:04 - 0:08
    our favorite letter for
    subspaces, and it's equal to
  • 0:08 - 0:11
    the span of two vectors in R4.
  • 0:11 - 0:18
    Let's say that the first vector
    is 1 0 0 1, and the
  • 0:18 - 0:26
    second vector is 0 1 0 1.
  • 0:26 - 0:28
    That is my subspace V.
  • 0:28 - 0:30
    And you can see that these
    are going to be a basis.
  • 0:30 - 0:33
    That these are linearly
    independent.
  • 0:33 - 0:36
    Two vectors that are linear-- or
    any set of vectors that are
  • 0:36 - 0:39
    linearly independent and that
    span a subspace are a basis
  • 0:39 - 0:40
    for that subspace.
  • 0:40 - 0:42
    You can see they are linearly
    independent.
  • 0:42 - 0:43
    This guy's has a 1 here.
  • 0:43 - 0:45
    There's no way you can take some
    combination of this guy
  • 0:45 - 0:47
    to somehow get a 1 there.
  • 0:47 - 0:48
    And this guy has a 1 here.
  • 0:48 - 0:50
    There's no way you can get some
    linear combination of
  • 0:50 - 0:51
    these zeroes here
    to a 1 there, so
  • 0:51 - 0:54
    they're linearly dependent.
  • 0:54 - 1:00
    You can also call this
    a basis for V.
  • 1:00 - 1:04
    Now, given that, let's see
    if we can find out the
  • 1:04 - 1:09
    transformation matrix for the
    projection of any arbitrary
  • 1:09 - 1:11
    vector onto this subspace.
  • 1:11 - 1:14
    So let's say that X-- we're
    dealing in R4 here, right?
  • 1:14 - 1:21
    Let's say that x is a member
    of R4, and I want to figure
  • 1:21 - 1:24
    out a transformation
    matrix for the
  • 1:24 - 1:29
    projection onto V of x.
  • 1:29 - 1:34
    Now, in the last video, we came
    up with a general way to
  • 1:34 - 1:35
    figure this out.
  • 1:35 - 1:38
    We said if A is a transformation
    matrix-- sorry.
  • 1:38 - 1:44
    If A is a matrix who's columns
    are the basis for the
  • 1:44 - 1:52
    subspace, so let's say A is
    equal to 1 0 0 1, 0 1 0 1.
  • 1:52 - 1:55
    So A is a matrix whose columns
    are the basis for our
  • 1:55 - 2:00
    subspace, then the projection
    of x onto V would be equal
  • 2:00 - 2:02
    to-- and this is kind of hard.
  • 2:02 - 2:05
    The first time you look at, it
    gives you a headache, but
  • 2:05 - 2:08
    there's a certain pattern or
    symmetry or a way of-- you
  • 2:08 - 2:11
    could say it's A times, you're
    gonna have something in the
  • 2:11 - 2:18
    middle, and then you have A
    transpose times your vector x.
  • 2:18 - 2:20
    And the way I remember it is in
    the middle, you have these
  • 2:20 - 2:22
    two guys switched around.
  • 2:22 - 2:26
    So then you have A transpose
    A, and you take
  • 2:26 - 2:27
    the inverse of it.
  • 2:30 - 2:32
    You probably won't be using
    this in your everyday life
  • 2:32 - 2:34
    five or ten years from now, so
    it's OK if you don't memorize
  • 2:34 - 2:37
    it, but temporarily, put this
    in your medium-term memory
  • 2:37 - 2:40
    because it's a good thing
    to know for doing these
  • 2:40 - 2:41
    projection problems.
  • 2:41 - 2:46
    So if we want to find the
    general matrix for this
  • 2:46 - 2:49
    transformation, we just have to
    determine what this matrix
  • 2:49 - 2:53
    is equal to, and that's just a
    bunch of matrix operations.
  • 2:53 - 2:54
    So that's A.
  • 2:54 - 2:56
    What is A transpose?
  • 2:56 - 3:05
    A transpose is going to be equal
    to just all the rows
  • 3:05 - 3:06
    turn into columns.
  • 3:06 - 3:09
    So the first column becomes
    the first row.
  • 3:09 - 3:12
    So it becomes 1 0 0 1.
  • 3:12 - 3:18
    The second column becomes
    the second row 0 1 0 1.
  • 3:18 - 3:19
    That's what A transpose is.
  • 3:19 - 3:21
    Now, what's A transpose A?
  • 3:21 - 3:25
    To figure out that, I want to
    figure out what A transpose
  • 3:25 - 3:27
    times A is.
  • 3:27 - 3:29
    So let me multiply A
    transpose times A.
  • 3:29 - 3:31
    So I'll rewrite A right here.
  • 3:31 - 3:36
    1 0 0 1, 0 1 0 1.
  • 3:36 - 3:38
    This is giving us some
    good practice on
  • 3:38 - 3:40
    matrix-matrix products.
  • 3:40 - 3:42
    This is going to be
    equal to what?
  • 3:42 - 3:46
    Well, first of all, this is
    a 2-by-4 matrix, and I'm
  • 3:46 - 3:49
    multiplying it by a 4-by-2
    matrix, so it's going to be a
  • 3:49 - 3:52
    2-by-2 matrix.
  • 3:52 - 3:55
    So the first entry is
    essentially the dot product of
  • 3:55 - 3:58
    that row with that column.
  • 3:58 - 4:06
    So it's 1 times 1 plus 0
    times 0 plus 0 times 0
  • 4:06 - 4:08
    plus 1 times 1.
  • 4:08 - 4:11
    So it's just going to be 2 for
    that first entry right there.
  • 4:11 - 4:14
    And then you take the dot
    product of this guy with this
  • 4:14 - 4:15
    guy right here.
  • 4:15 - 4:21
    So it's 1 times 0, which is 0,
    plus 0 times 1, which is 0,
  • 4:21 - 4:26
    plus 0 time 0, which is 0, plus
    1 times 1, which is 1.
  • 4:26 - 4:33
    Now, we do this guy dotted with
    this column right there.
  • 4:33 - 4:40
    0 times 1 is 0 plus 1 time 0 is
    0 plus 0 times 0 is 0 plus
  • 4:40 - 4:42
    1 times 1 is 1.
  • 4:44 - 4:48
    And then finally,
    this row dotted
  • 4:48 - 4:50
    with this second column.
  • 4:50 - 4:51
    Second row, second column.
  • 4:51 - 4:56
    0 times 0 is 0, 1 times 1
    is 1, 0 times 0 is 0,
  • 4:56 - 4:57
    1 times 1 is 1.
  • 4:57 - 5:00
    So we have 1 times
    1 plus 1 times 1.
  • 5:00 - 5:01
    It's going to be 2.
  • 5:01 - 5:03
    It's going to be equal to 2.
  • 5:03 - 5:08
    So this right here
    is A transpose A.
  • 5:08 - 5:09
    But that's not good enough.
  • 5:09 - 5:13
    We need to figure out what the
    inverse of A transpose A is.
  • 5:13 - 5:14
    This is A transpose A.
  • 5:14 - 5:17
    But we need to figure out
    A transpose A inverse.
  • 5:17 - 5:19
    So what's the inverse of this?
  • 5:19 - 5:20
    So let me write it here.
  • 5:20 - 5:26
    The inverse A transpose
    A inverse is going
  • 5:26 - 5:27
    to be equal to what?
  • 5:27 - 5:29
    It's 1 over the determinant
    of this guy.
  • 5:29 - 5:30
    What's the determinant here?
  • 5:30 - 5:33
    It's going to be 1 over the
    determinant of this.
  • 5:33 - 5:38
    The determinant is 2 times 2,
    which is 4, minus 1 times 1.
  • 5:38 - 5:41
    So it's 4 minus 1, which is 3.
  • 5:41 - 5:47
    So 1 over the determinant times
    this guy, where if I
  • 5:47 - 5:53
    swap these two, so I swap the
    1's-- sorry, I swap the 2's.
  • 5:53 - 6:01
    So this 2 goes here, and then
    this orange 2 goes over here.
  • 6:01 - 6:05
    And then I make these
    1's negative.
  • 6:05 - 6:10
    This becomes a minus 1 and
    this becomes a minus 1.
  • 6:10 - 6:12
    We learned that this is a
    general solution for the
  • 6:12 - 6:14
    inverse of a 2-by-2 matrix.
  • 6:14 - 6:17
    I think it was 10 or 11 videos
    ago, and you probably learned
  • 6:17 - 6:19
    this in your Algebra II class,
    frankly, but there you go.
  • 6:19 - 6:22
    We have A transpose A inverse.
  • 6:22 - 6:24
    So we have this guy.
  • 6:24 - 6:26
    We have this whole guy here
    is just this matrix.
  • 6:26 - 6:29
    I could multiply the 1/3 into
    it, but I don't have to do
  • 6:29 - 6:30
    that just yet.
  • 6:30 - 6:32
    But let's figure out the
    whole matrix now.
  • 6:32 - 6:35
    The whole A times this
    guy, A transpose A
  • 6:35 - 6:37
    inverse times A transpose.
  • 6:37 - 6:39
    Let me write it this way.
  • 6:39 - 6:50
    So the projection onto the
    subspace V of x is going to be
  • 6:50 - 6:51
    equal to A.
  • 6:54 - 7:02
    1 0 0 1-- let me write a little
    bit bigger like this.
  • 7:02 - 7:12
    So 1 0 0 1, 0 1 0 1 times A
    transpose A inverse, right?
  • 7:12 - 7:17
    A times A transpose A inverse,
    which is this guy right here.
  • 7:17 - 7:19
    Let's just put the 1/3 out
    front just because
  • 7:19 - 7:21
    that's just a scalar.
  • 7:21 - 7:25
    I'll put the 1/3 out front
    times this guy.
  • 7:25 - 7:29
    This A transpose A inverse
    is 1/3 times 2
  • 7:29 - 7:33
    minus 1, minus 1, 2.
  • 7:33 - 7:35
    And then I'm going to multiply
    it times A transpose.
  • 7:38 - 7:40
    And all that times
    our vector x.
  • 7:40 - 7:42
    So A transpose is right there.
  • 7:45 - 7:52
    It is 1 0 0 1, 0 1 0 1.
  • 7:52 - 7:56
    And then all of that's going
    to be times your vector x.
  • 7:56 - 7:59
    So we still have some
    nice matrix-matrix
  • 7:59 - 8:00
    products ahead of us.
  • 8:00 - 8:04
    Let see if we can do these.
  • 8:04 - 8:07
    So the first one, let's just
    multiply these two guys.
  • 8:10 - 8:12
    I don't think there's any
    simple way to do it.
  • 8:12 - 8:16
    This is a 2-by-2 matrix and this
    is a 2-by-4 matrix, so
  • 8:16 - 8:18
    when I multiply them,
    I'm going to end up
  • 8:18 - 8:23
    with a 2-by-4 matrix.
  • 8:23 - 8:27
    Let me write that 2-by-4
    matrix right here.
  • 8:27 - 8:30
    And then I can write this
    guy right here.
  • 8:30 - 8:35
    1 0 0 1, 0 1 0 1.
  • 8:35 - 8:38
    And then I have the 1/3 that
    was from A transpose A
  • 8:38 - 8:41
    inverse, but I put the scaling
    factor out there.
  • 8:41 - 8:43
    And all of this is equal to the
    projection of x onto V.
  • 8:43 - 8:45
    So let's do this product.
  • 8:45 - 8:49
    So this first entry is going to
    be 2 times 1 plus minus 1
  • 8:49 - 8:52
    times 0, so that is just 2.
  • 8:52 - 8:57
    Then you're going to have 2
    times 0 plus minus 1 times 1.
  • 8:57 - 8:59
    Well, that's minus 1.
  • 8:59 - 9:02
    Then you have 2 times 0
    plus minus 1 times 0.
  • 9:02 - 9:04
    Well, that's just 0.
  • 9:04 - 9:08
    And then you're going to
    have 2 times 1 plus
  • 9:08 - 9:09
    minus 1 times 1.
  • 9:09 - 9:10
    That's 2 minus 1.
  • 9:10 - 9:12
    That's just 1, right?
  • 9:12 - 9:15
    2 times 1 plus minus
    1 times 1.
  • 9:15 - 9:15
    Fair enough.
  • 9:15 - 9:17
    Now, let's do the second row.
  • 9:17 - 9:22
    Minus 1 times 1 plus 2 times
    0, so that's just minus 1.
  • 9:22 - 9:24
    Minus 1 times 0 plus
    2 times 1.
  • 9:24 - 9:26
    Well, that's just 2.
  • 9:26 - 9:28
    Minus 1 times 0 plus
    2 times 0.
  • 9:28 - 9:30
    That's just 0.
  • 9:30 - 9:34
    Minus 1 times 1 plus
    2 times 1.
  • 9:34 - 9:39
    Well, that's minus 1 plus
    2, so that is 1.
  • 9:39 - 9:42
    Almost there, and, of course, we
    have to multiply it times x
  • 9:42 - 9:43
    at the end.
  • 9:43 - 9:44
    That's what the transformation
    is.
  • 9:44 - 9:47
    But this right here is our
    transformation matrix.
  • 9:47 - 9:48
    One more left to do.
  • 9:48 - 9:51
    Let's hope I haven't made any
    careless mistakes and that I
  • 9:51 - 9:54
    won't make any when doing
    this product.
  • 9:54 - 9:55
    This is going to be a little
    more complicated because this
  • 9:55 - 9:59
    is a 4 by 2 times a 2 by 4.
  • 9:59 - 10:04
    I'm going to end up with
    a 4-by-4 matrix.
  • 10:04 - 10:07
    Let me give myself some
    breathing room here because
  • 10:07 - 10:11
    I'm going to generate a 4-by-4
    matrix right there.
  • 10:11 - 10:13
    And so what am I going to get?
  • 10:13 - 10:18
    So this first entry is going
    to be 1 times 2 plus
  • 10:18 - 10:20
    0 times minus 1.
  • 10:20 - 10:23
    So it's just going
    to be equal to 2.
  • 10:23 - 10:29
    The next entry: 1 times-- this
    row times any column here is
  • 10:29 - 10:32
    just going to be the first entry
    in the column because it
  • 10:32 - 10:32
    gets zeroed out.
  • 10:32 - 10:36
    So 1 times 2 plus 0 times
    minus 1 is just 2.
  • 10:36 - 10:40
    1 times minus 1 plus 0 times
    2 is just minus 1.
  • 10:40 - 10:43
    1 times 0 plus 0 times 0 is 0.
  • 10:43 - 10:47
    1 times 1 plus 0 times
    1 is just 1.
  • 10:47 - 10:49
    When you take this row and you
    multiply it times these
  • 10:49 - 10:52
    columns, you literally just
    got your first row there.
  • 10:52 - 10:58
    Now, let's do this row
    times these columns.
  • 10:58 - 11:00
    Now, you've got a 0 here, so
    you're going to have a 0 times
  • 11:00 - 11:02
    the first entry of all
    of these and a 1
  • 11:02 - 11:03
    times the second one.
  • 11:03 - 11:07
    So 0 times 2 plus 1 times
    minus 1 is minus 1.
  • 11:07 - 11:09
    0 times minus 1 plus
    1 times 2 is 2.
  • 11:09 - 11:11
    You're just going to get
    the second row here.
  • 11:11 - 11:14
    2 0 1.
  • 11:14 - 11:16
    That actually makes sense,
    because if you just look at
  • 11:16 - 11:19
    this part of the matrix, it's
    the 2-by-2 identity matrix.
  • 11:19 - 11:22
    So, anyway, that's a little hint
    why this looks very much
  • 11:22 - 11:24
    like that, but we're just
    going to go through this
  • 11:24 - 11:25
    matrix product.
  • 11:25 - 11:28
    Now, you multiply this-- let me
    do it in a different color.
  • 11:28 - 11:32
    You multiply this guy times
    each of these columns.
  • 11:32 - 11:34
    That guy dotted with that is
    just going to 0 because this
  • 11:34 - 11:37
    guy's essentially the 0 row
    vector, so you're just going
  • 11:37 - 11:39
    to get a bunch of zeroes.
  • 11:39 - 11:45
    And then, finally, this last
    row, it's 1 times the first
  • 11:45 - 11:47
    entry plus 1 times
    the second entry.
  • 11:47 - 11:51
    So this guy's going to be 2
    plus minus 1, which is 1.
  • 11:51 - 11:53
    Minus 1 plus 2, which is 1.
  • 11:53 - 11:55
    0 plus 0, which is 0.
  • 11:55 - 11:57
    And then 1 plus 1, which is 2.
  • 11:57 - 11:59
    And all that times x.
  • 11:59 - 12:01
    And there you have it.
  • 12:01 - 12:04
    This is exciting!
  • 12:04 - 12:08
    The projection onto V of x
    is equal to this whole
  • 12:08 - 12:10
    matrix times x.
  • 12:10 - 12:15
    So this thing right here, I
    could multiply the 1/3 into
  • 12:15 - 12:16
    it, but we don't have
    to do that.
  • 12:16 - 12:18
    That'll just make it a little
    bit more messy.
  • 12:18 - 12:21
    This thing right here is the
    transformation matrix.
  • 12:25 - 12:28
    As you can see, since we're
    transforming-- remember, this
  • 12:28 - 12:33
    projection onto V,
    this is a linear
  • 12:33 - 12:38
    transformation from R4 to R4.
  • 12:38 - 12:41
    You give me some member of R4,
    and I'll give you another
  • 12:41 - 12:45
    member of R4 that's in my
    subspace that is the
  • 12:45 - 12:46
    projection.
  • 12:46 - 12:49
    So this is going to be
    a 4-by-4 You can
  • 12:49 - 12:50
    see it right there.
  • 12:50 - 12:52
    Anyway, hopefully, you found
    that useful to actually see a
  • 12:52 - 12:53
    tangible result.
  • 12:53 - 12:55
    R4 is very abstract, so this
    would even be beyond our
  • 12:55 - 12:58
    three-dimensional programming
    example.
  • 12:58 - 13:00
    We're dealing with a more
    abstract data set where we're
  • 13:00 - 13:02
    interested in finding
    a projection.
Title:
Linear Algebra: Subspace Projection Matrix Example
Description:

more » « less
Video Language:
English
Team:
Khan Academy
Duration:
13:04

English subtitles

Revisions Compare revisions