Main content

## Orthogonal projections

Current time:0:00Total duration:13:04

# Subspace projection matrix example

## Video transcript

Let's say I've got some subspace
V, which tends to be our favorite letter for
subspaces, and it's equal to the span of two vectors in R4. Let's say that the first vector
is 1 0 0 1, and the second vector is 0 1 0 1. That is my subspace V. And you can see that these
are going to be a basis. That these are linearly
independent. Two vectors that are linear-- or
any set of vectors that are linearly independent and that
span a subspace are a basis for that subspace. You can see they are linearly
independent. This guy's has a 1 here. There's no way you can take some
combination of this guy to somehow get a 1 there. And this guy has a 1 here. There's no way you can get some
linear combination of these zeroes here
to a 1 there, so they're linearly dependent. You can also call this
a basis for V. Now, given that, let's see
if we can find out the transformation matrix for the
projection of any arbitrary vector onto this subspace. So let's say that X-- we're
dealing in R4 here, right? Let's say that x is a member
of R4, and I want to figure out a transformation
matrix for the projection onto V of x. Now, in the last video, we came
up with a general way to figure this out. We said if A is a transformation
matrix-- sorry. If A is a matrix who's columns
are the basis for the subspace, so let's say A is
equal to 1 0 0 1, 0 1 0 1. So A is a matrix whose columns
are the basis for our subspace, then the projection
of x onto V would be equal to-- and this is kind of hard. The first time you look at, it
gives you a headache, but there's a certain pattern or
symmetry or a way of-- you could say it's A times, you're
gonna have something in the middle, and then you have A
transpose times your vector x. And the way I remember it is in
the middle, you have these two guys switched around. So then you have A transpose
A, and you take the inverse of it. You probably won't be using
this in your everyday life five or ten years from now, so
it's OK if you don't memorize it, but temporarily, put this
in your medium-term memory because it's a good thing
to know for doing these projection problems. So if we want to find the
general matrix for this transformation, we just have to
determine what this matrix is equal to, and that's just a
bunch of matrix operations. So that's A. What is A transpose? A transpose is going to be equal
to just all the rows turn into columns. So the first column becomes
the first row. So it becomes 1 0 0 1. The second column becomes
the second row 0 1 0 1. That's what A transpose is. Now, what's A transpose A? To figure out that, I want to
figure out what A transpose times A is. So let me multiply A
transpose times A. So I'll rewrite A right here. 1 0 0 1, 0 1 0 1. This is giving us some
good practice on matrix-matrix products. This is going to be
equal to what? Well, first of all, this is
a 2-by-4 matrix, and I'm multiplying it by a 4-by-2
matrix, so it's going to be a 2-by-2 matrix. So the first entry is
essentially the dot product of that row with that column. So it's 1 times 1 plus 0
times 0 plus 0 times 0 plus 1 times 1. So it's just going to be 2 for
that first entry right there. And then you take the dot
product of this guy with this guy right here. So it's 1 times 0, which is 0,
plus 0 times 1, which is 0, plus 0 time 0, which is 0, plus
1 times 1, which is 1. Now, we do this guy dotted with
this column right there. 0 times 1 is 0 plus 1 time 0 is
0 plus 0 times 0 is 0 plus 1 times 1 is 1. And then finally,
this row dotted with this second column. Second row, second column. 0 times 0 is 0, 1 times 1
is 1, 0 times 0 is 0, 1 times 1 is 1. So we have 1 times
1 plus 1 times 1. It's going to be 2. It's going to be equal to 2. So this right here
is A transpose A. But that's not good enough. We need to figure out what the
inverse of A transpose A is. This is A transpose A. But we need to figure out
A transpose A inverse. So what's the inverse of this? So let me write it here. The inverse A transpose
A inverse is going to be equal to what? It's 1 over the determinant
of this guy. What's the determinant here? It's going to be 1 over the
determinant of this. The determinant is 2 times 2,
which is 4, minus 1 times 1. So it's 4 minus 1, which is 3. So 1 over the determinant times
this guy, where if I swap these two, so I swap the
1's-- sorry, I swap the 2's. So this 2 goes here, and then
this orange 2 goes over here. And then I make these
1's negative. This becomes a minus 1 and
this becomes a minus 1. We learned that this is a
general solution for the inverse of a 2-by-2 matrix. I think it was 10 or 11 videos
ago, and you probably learned this in your Algebra II class,
frankly, but there you go. We have A transpose A inverse. So we have this guy. We have this whole guy here
is just this matrix. I could multiply the 1/3 into
it, but I don't have to do that just yet. But let's figure out the
whole matrix now. The whole A times this
guy, A transpose A inverse times A transpose. Let me write it this way. So the projection onto the
subspace V of x is going to be equal to A. 1 0 0 1-- let me write a little
bit bigger like this. So 1 0 0 1, 0 1 0 1 times A
transpose A inverse, right? A times A transpose A inverse,
which is this guy right here. Let's just put the 1/3 out
front just because that's just a scalar. I'll put the 1/3 out front
times this guy. This A transpose A inverse
is 1/3 times 2 minus 1, minus 1, 2. And then I'm going to multiply
it times A transpose. And all that times
our vector x. So A transpose is right there. It is 1 0 0 1, 0 1 0 1. And then all of that's going
to be times your vector x. So we still have some
nice matrix-matrix products ahead of us. Let see if we can do these. So the first one, let's just
multiply these two guys. I don't think there's any
simple way to do it. This is a 2-by-2 matrix and this
is a 2-by-4 matrix, so when I multiply them,
I'm going to end up with a 2-by-4 matrix. Let me write that 2-by-4
matrix right here. And then I can write this
guy right here. 1 0 0 1, 0 1 0 1. And then I have the 1/3 that
was from A transpose A inverse, but I put the scaling
factor out there. And all of this is equal to the
projection of x onto V. So let's do this product. So this first entry is going to
be 2 times 1 plus minus 1 times 0, so that is just 2. Then you're going to have 2
times 0 plus minus 1 times 1. Well, that's minus 1. Then you have 2 times 0
plus minus 1 times 0. Well, that's just 0. And then you're going to
have 2 times 1 plus minus 1 times 1. That's 2 minus 1. That's just 1, right? 2 times 1 plus minus
1 times 1. Fair enough. Now, let's do the second row. Minus 1 times 1 plus 2 times
0, so that's just minus 1. Minus 1 times 0 plus
2 times 1. Well, that's just 2. Minus 1 times 0 plus
2 times 0. That's just 0. Minus 1 times 1 plus
2 times 1. Well, that's minus 1 plus
2, so that is 1. Almost there, and, of course, we
have to multiply it times x at the end. That's what the transformation
is. But this right here is our
transformation matrix. One more left to do. Let's hope I haven't made any
careless mistakes and that I won't make any when doing
this product. This is going to be a little
more complicated because this is a 4 by 2 times a 2 by 4. I'm going to end up with
a 4-by-4 matrix. Let me give myself some
breathing room here because I'm going to generate a 4-by-4
matrix right there. And so what am I going to get? So this first entry is going
to be 1 times 2 plus 0 times minus 1. So it's just going
to be equal to 2. The next entry: 1 times-- this
row times any column here is just going to be the first entry
in the column because it gets zeroed out. So 1 times 2 plus 0 times
minus 1 is just 2. 1 times minus 1 plus 0 times
2 is just minus 1. 1 times 0 plus 0 times 0 is 0. 1 times 1 plus 0 times
1 is just 1. When you take this row and you
multiply it times these columns, you literally just
got your first row there. Now, let's do this row
times these columns. Now, you've got a 0 here, so
you're going to have a 0 times the first entry of all
of these and a 1 times the second one. So 0 times 2 plus 1 times
minus 1 is minus 1. 0 times minus 1 plus
1 times 2 is 2. You're just going to get
the second row here. 2 0 1. That actually makes sense,
because if you just look at this part of the matrix, it's
the 2-by-2 identity matrix. So, anyway, that's a little hint
why this looks very much like that, but we're just
going to go through this matrix product. Now, you multiply this-- let me
do it in a different color. You multiply this guy times
each of these columns. That guy dotted with that is
just going to 0 because this guy's essentially the 0 row
vector, so you're just going to get a bunch of zeroes. And then, finally, this last
row, it's 1 times the first entry plus 1 times
the second entry. So this guy's going to be 2
plus minus 1, which is 1. Minus 1 plus 2, which is 1. 0 plus 0, which is 0. And then 1 plus 1, which is 2. And all that times x. And there you have it. This is exciting! The projection onto V of x
is equal to this whole matrix times x. So this thing right here, I
could multiply the 1/3 into it, but we don't have
to do that. That'll just make it a little
bit more messy. This thing right here is the
transformation matrix. As you can see, since we're
transforming-- remember, this projection onto V,
this is a linear transformation from R4 to R4. You give me some member of R4,
and I'll give you another member of R4 that's in my
subspace that is the projection. So this is going to be
a 4-by-4 You can see it right there. Anyway, hopefully, you found
that useful to actually see a tangible result. R4 is very abstract, so this
would even be beyond our three-dimensional programming
example. We're dealing with a more
abstract data set where we're interested in finding
a projection.