Orthogonal projections
-
Projections onto Subspaces
-
Visualizing a projection onto a plane
-
A Projection onto a Subspace is a Linear Transforma
-
Subspace Projection Matrix Example
-
Another Example of a Projection Matrix
-
Projection is closest vector in subspace
-
Least Squares Approximation
-
Least Squares Examples
-
Another Least Squares Example
Another Example of a Projection Matrix Figuring out the transformation matrix for a projection onto a subspace by figuring out the matrix for the projection onto the subspace's orthogonal complement first
⇐ Use this menu to view and help create subtitles for this video in many different languages.
You'll probably want to hide YouTube's captions if using these subtitles.
- Let's say I have a subspace v that is equal to all of the
- vectors-- let me write it this way-- all of the x1, x2, x3's,
- so all the vectors like this that satisfy x1 plus x2 plus
- x3 is equal to 0.
- So if you think about it, this is just a plane in R3, so this
- subspace is a plane in R3.
- And I'm interested in finding the transformation matrix for
- the projection of any vector x in R3 onto v.
- So how could we do that?
- So we could do it like we did in the last video.
- We could find the basis for this subspace right there.
- And that's not too hard to do.
- We could say x1, if we assume that, let's say, that x2 and
- x3 are kind of free variables, then we could say that x1 is
- equal to minus x2, minus x3.
- And then let's just, just so we can write it in, kind of,
- our parametric form, or if we can write our solution set as
- the combination of basis vectors, we can say x2 is
- equal to, let's say it's equal to some
- arbitrary constant, C2.
- And let's say that x3 is equal to some
- arbitrary constant, C3.
- Then we can say that v, we can rewrite v, we could say that v
- is-- I'll do it here-- v is equal to the set of all x1's,
- x2's, and x3's that are equal to C2 times-- so x1 is equal
- to minus-- let me rewrite this with a C2--
- this is equal to C2.
- This is equal to C3.
- So x1 is equal to minus C2, minus C3.
- So x1 is equal to minus 1 times C2, plus C3 times what?
- Plus C3 times minus 1.
- And then what is x2 equal to?
- x2 is just equal to C2.
- So it's 1 times C2, plus 0, times C3.
- x3 is just equal to C3, so it's 0 times C2,
- plus 1, times C3.
- And so this is another way of defining our subspace.
- All of the vectors that satisfy this is equal to this
- definition here.
- It's all the vectors whose components satisfy, or that
- lie in this plane, whose entries lie in that plane.
- And that's for any real numbers right there.
- Or another way of writing this, is v is equal to the
- span of the vectors minus 1, 1, and 0, and the vector minus
- 1, 0, and 1.
- Just like that.
- And we know that these are actually a basis for v because
- they're linearly independent.
- There's no way I can take linear combinations of this
- guy and make the second entry equal a 1 here.
- And likewise there's no way I can take linear combinations
- of this guy and make this third entry equal a 1 here.
- So these are also a basis for v.
- So given, that just using the technique we did before, we
- could set some vector, we could set some matrix A equal
- to minus 1, 1, 0, and then minus 1, 0, and 1.
- And then we can figure out that the projection of any
- vector x in our 3 onto v is going to be equal to, and we
- saw this, it's going to be equal to A times the inverse
- of A transpose A.
- All of that times A transpose and all of that times x.
- And you can do it.
- You have A here.
- You can figure out what the transpose of A is very easy.
- You can take A transpose A, then you can invert it.
- And it'll be very similar to what we did in the last video.
- It'll be a little less work, because this is a 3 by 2
- matrix, instead of a 4 by 2 matrix.
- But you saw it is actually a lot of work.
- It's very hairy and you might make some careless mistakes.
- So let's figure out if there's another way that we can come
- up with this matrix right here.
- Now we know that if x is a member of R3, that x can be
- represented as a combination of some vector v, that is in
- our subspace, plus some vector w, that is in the orthogonal
- complement of the subspace, where v is a member of our
- subspace, and w is a member of the orthogonal complement of
- our subspace.
- Now by definition, that right there is the projection of x
- onto v, and this is the projection of x onto the
- orthogonal complement of v.
- So we can write that x is equal to the projection onto v
- of x, plus the projection onto v's orthogonal complement, or
- the orthogonal complement of v of x.
- So this is by definition, that any member of R3 can be
- represented this way.
- Now if we want to write this as matrix vector products, and
- two videos ago I showed you that these are linear
- transformations.
- So let me write that here.
- So they're linear transformations.
- So they can be written as matrix vector products.
- You see that right there.
- Let me define this matrix, I don't know, let me call this
- matrix T, let me just call it T.
- And let me do another.
- Let me do a letter, let me do B.
- And let's say that the projection onto the orthogonal
- complement of v of x, let's say that that's equal to some
- other matrix C, times x.
- And we know this is a linear transformation, so it can be
- represented as some matrix C times x.
- So what are these going to be equal to?
- Well x, if I want to write it as a linear transformation of
- x, I could just write it as the 3 by 3 identity matrix,
- times x, right?
- That's the same thing as x.
- That's going to be equal to the projection of x onto v,
- well that's just the same thing as B times x.
- And then plus the projection of x onto v's orthogonal
- complement, well that's just C times x.
- Plus C times x.
- And if you want to factor out the x on this side, we know
- that the matrix vector products exhibit the
- distributive property, so we could write that the identity
- matrix times x is equal to B plus C, times x.
- Or another way to view this equation is that this matrix
- must be equal to these two matrices.
- So we get that the identity matrix in R3 is equal to the
- projection matrix onto v, plus the projection matrix onto v's
- orthogonal complement.
- Remember, the whole point of this problem is to figure out
- this thing right here, is to solve or B.
- And we know a technique for doing it.
- You take A transpose, you can do this whole thing, but that
- might be pretty hairy.
- But maybe it's easy to find this guy.
- Maybe, I don't know.
- It actually turns out in the video, this one will be easy.
- So if it's easy to find this guy, we can just solve for B.
- If we subtract C from both sides, we get that B is equal
- to I, is equal to the identity matrix, minus the
- transformation matrix for the transformation onto v's
- orthogonal complement.
- So let's see what this is.
- Let's see if we can figure out what C is right there.
- So let's go back to our original.
- So remember-- let me rewrite the problem actually--
- remember that v was equal to, essentially it's equal to all
- of the x1's, x2's, x3's that satisfy x1 plus x2, plus x3 is
- equal to 0.
- Or another way to say it is that all the x1's, x2's, and
- x3's that satisfy the equation 1, 1, 1, times x1, x2, x3 is
- equal to the 0 vector.
- Or this case it'll just be 0.
- We could write the 0 vector just like that.
- So 1 times x1, plus 1 times x2, plus 1 times x3 is going
- to equal the 0 vector.
- This is another way to write v.
- Now all of the x's that satisfied this right here,
- what is that?
- This is saying that v is equal to the null space of this
- matrix right there.
- The null space of this matrix is all of the vectors that
- satisfy this equation.
- So v is equal to the null space-- let me write it this
- way-- the null space of 1, 1, 1, just like that.
- Up here we, kind of, figured out v in kind of the
- traditional way.
- We figured out that v is the span of these things, but now
- we know that's the same thing is the null space of 1, 1, 1.
- These two statements are equivalent.
- Now we at least had a hunch that maybe, you know, we could
- figure out, straight up, this B here by doing all of this A
- transpose and, you know, by doing all of
- this silliness here.
- But our hunch is maybe if we could figure out the
- transformation matrix for the orthogonal complement of v
- right there, that then we could just apply this, kind
- of, that we can just solve for B given that the identity
- matrix minus this guy is going to be equal to B.
- So let's see if we can figure out the projection matrix, if
- we can figure out the transformation matrix for the
- orthogonal projection, for x onto the orthogonal
- projection of v.
- So this is v.
- What is v compliment?
- v compliment is going to be equal to the orthogonal
- complement, or v perp is going to be equal to the orthogonal
- complement of the null space of this matrix right here.
- Which is equal to what?
- Remember, the null space, its orthogonal complement-- a null
- space's orthogonal complement is equivalent to the row space
- or the column space of A transpose.
- We saw that multiple times.
- Or you could say the orthogonal complement of the
- row space is the null space.
- We've seen this many, many times before.
- So the orthogonal complement of this guy is going to be the
- column space of his transpose.
- So the column space of the transpose of this guy.
- So it's 1, 1, 1, just like that.
- Or we can write that v's orthogonal complement is equal
- to the span of 1, 1, 1.
- The column space of this matrix, we only have one
- column in it, so its column space is going to be the span
- of that one column.
- So just to visualize what we're doing here, that
- original equation for v, that satisfies that, that's just
- going to be some plane in R3.
- That is v right there.
- And now we just figured out what v's
- orthogonal complement is.
- It's going to be a line in R3.
- It's going to be all of the linear
- combinations of this guy.
- So it's going to be some line in R3.
- I haven't drawn it.
- You know this is going to be tilted more, and so is this,
- but it's going to be some line.
- So this is the orthogonal complement of v.
- So let's see if we can figure out.
- So remember, the projection-- let me do it this way.
- So this is the basis for v's orthogonal complement.
- So let's construct some matrix.
- I don't know, let me use a letter that I
- haven't used before.
- Let me construct some matrix D, whose columns are the basis
- vectors for the orthogonal complement of v.
- Well, there's only one basis factor, so
- it's going to be that.
- And we learned, in the last video and the video before
- that, that the projection of any vector in R3 onto v's
- orthogonal complement is going to be equal to D times D
- transpose D inverse, times D transpose, times x.
- Or another way to view it is that this thing right here,
- that thing right there is the transformation matrix for this
- projection.
- That is the transformation matrix.
- matrix
- So let's see if this is easier to solve this thing than this
- business up here, where we had a 3 by 2 matrix.
- That was the whole motivation for doing this problem.
- To figure out the projection matrix for v's subspace, we'd
- have to do this with the 3 by 2 matrix.
- It seems pretty difficult.
- Instead, let's find the projection matrix to get to
- the production onto v's orthogonal
- complement, which is this.
- So what is D transpose?
- So D transpose is just going to be equal to 1, 1, 1.
- What is D transpose times D?
- Well, that's D transpose.
- This is D, just like that.
- So what is this going to be equal to?
- This is just the dot product of that and that.
- 1 times 1, plus 1 times 1, plus 1 times 1, it equals 3.
- So this thing right here is equal to a 1 by 1 matrix 3.
- So let's write it down.
- So this is equal to D-- which is this matrix, 1, 1, 1--
- times D transpose D inverse.
- So D transpose D is just a 1 by 1 matrix.
- And we're going to have to invert it.
- Actually, I've never defined the inverse of a 1 by 1 matrix
- for you just now, so it's just mildly exciting.
- Times D transpose.
- So D transpose looks like this, 1, 1, 1.
- And then all of that's times x.
- But this is the transformation matrix right there.
- Now what is the inverse of a 1 by 1 matrix?
- Now you just have to remember that A inverse times A is
- equal to the identity matrix.
- If we're dealing with a 1 by 1 matrix, then I'm just trying
- to figure out what, let's say, what matrix times 3 is going
- to be equal to the 1 by 1 identity matrix.
- So let's say that 3 inverse times 3 has to be equal to the
- identity matrix.
- 1 by 1 identity matrix.
- Well, the only matrix that's going to make this work out,
- to get this entry I'll just take this guy's entry times
- that guy's entry, is going to be this guy right here.
- The inverse of this 1 by 1 matrix has to
- be the matrix 1/3.
- 1/3 times 3 is equal to 1.
- This is almost trivially simple, but this is the
- inverse, that right there is the inverse matrix, for the 1
- by 1 matrix 3.
- So this right here is just 1/3.
- And we could actually just take that out.
- It's a 1 by 1 matrix, which is essentially
- equivalent to a scalar.
- So this is going to be equal-- let me just draw a line here--
- this thing is equal to 1/3-- actually, I don't want to
- confuse you.
- Let me rewrite it.
- So we get the projection of any vector in R3 onto the
- orthogonal complement of v, is equal to 1/3, that's 1/3,
- times the vector 1, 1, 1, times-- sorry, or wait, that
- is a vector or the matrix 1 on 1-- times that matrix
- transposed, 1, 1, 1.
- And then all of that times x.
- And you can see, this is a lot simpler than if we have to do
- all of this business with this matrix.
- That's a harder matrix to deal with.
- This 1, 1, 1 matrix is very easy.
- Now what is this going to be equal to?
- This is going to be equal to 1/3 times, we have a 3 by 1
- times a 1 by 3 matrix, so it's going to
- result in a 3 by 3 matrix.
- And what do we get?
- So this first entry is going to be 1 times 1, which is 1.
- The second entry is going to be 1 times 1, which is 1.
- The third entry is going to be 1 times 1, which is 1.
- I think you see the pattern.
- The second grow, first column, 1 times 1, which is 1.
- So this is going to be a 3 by 3 matrix of 1's.
- So just like that we were able to get-- that was a pretty
- straightforward situation-- we were able to get the
- projection matrix for any vector in R3 onto v's
- orthogonal complement.
- Now, we know that this thing right here is our original C
- that we said.
- And we said that the identity matrix-- we wrote it up here.
- Let me refer back to what I wrote way up here.
- We said, look, the identity matrix is equal to the
- transformation matrix for the projection onto v, plus the
- transformation matrix for the projection onto v's orthogonal
- complement.
- Or we can write that the transformation matrix for the
- projection onto v is equal to the identity matrix minus the
- transformation matrix for the projection onto v's orthogonal
- complement.
- So if we say that the projection onto v of x is
- equal to B times x, we know that B is equal to the 3 by 3
- identity matrix, minus C, and this is C right there.
- So B is equal to the identity matrix-- so that's just 1, 0,
- 0, 0, 1, 0, 0, 0, 1-- minus C, minus 1/3, times 1, 1, 1, 1,
- 1, 1, 1, 1, 1, just like that.
- And what is this going to be equal to?
- Let's see, let's, in our heads, multiply this out.
- All of these entries are going to be 1/3 essentially, if we
- multiply this out like that.
- So if we have 1 minus 1/3.
- I could write it out like that.
- It's 1/3, 1/3, 1/3.
- Everything is 1/3.
- 1/3, 1/3, 1/3, 1/3, 1/3, 1/3, and this just becomes a 1.
- So 1 minus 1/3 is 2/3.
- And all of the 1's minus 1/3 are going to be 2/3, so we
- could just go down the diagonal.
- And then the 0's minus 1/3 are going to be minus 1/3.
- Minus 1/3, minus 1/3, minus 1/3.
- You have minus 1/3, minus 1/3, and minus 1/3.
- And just like that, we've been able to figure out our
- projection, our transformation matrix, for the projection of
- any vector x onto v, by essentially finding this guy
- first, for finding the transformation matrix for the
- projection of any x onto v's orthogonal complement.
- Anyway, I thought that was pretty neat.
- And you could rewrite this as v equal to 1/3 times 2, 2, 2,
- 2's along back the diagonals and then you have minus 1's
- everywhere else.
- Anyway, see you in the next video.
Be specific, and indicate a time in the video:
At 5:31, how is the moon large enough to block the sun? Isn't the sun way larger?
|
Have something that's not a question about this content? |
This discussion area is not meant for answering homework questions.
Discuss the site
For general discussions about Khan Academy, visit our Reddit discussion page.
Flag inappropriate posts
Here are posts to avoid making. If you do encounter them, flag them for attention from our Guardians.
abuse
- disrespectful or offensive
- an advertisement
not helpful
- low quality
- not about the video topic
- soliciting votes or seeking badges
- a homework question
- a duplicate answer
- repeatedly making the same post
wrong category
- a tip or feedback in Questions
- a question in Tips & Feedback
- an answer that should be its own question
about the site
Share a tip
Suggest a fix
Have something that's not a tip or feedback about this content?
This discussion area is not meant for answering homework questions.