Another example of a projection matrix

Let's say I have a subspace v that is equal to all of the vectors-- let me write it this way-- all of the x1, x2, x3's, so all the vectors like this that satisfy x1 plus x2 plus x3 is equal to 0. So if you think about it, this is just a plane in R3, so this subspace is a plane in R3. And I'm interested in finding the transformation matrix for the projection of any vector x in R3 onto v. So how could we do that? So we could do it like we did in the last video. We could find the basis for this subspace right there. And that's not too hard to do. We could say x1, if we assume that, let's say, that x2 and x3 are kind of free variables, then we could say that x1 is equal to minus x2, minus x3. And then let's just, just so we can write it in, kind of, our parametric form, or if we can write our solution set as the combination of basis vectors, we can say x2 is equal to, let's say it's equal to some arbitrary constant, C2. And let's say that x3 is equal to some arbitrary constant, C3. Then we can say that v, we can rewrite v, we could say that v is-- I'll do it here-- v is equal to the set of all x1's, x2's, and x3's that are equal to C2 times-- so x1 is equal to minus-- let me rewrite this with a C2-- this is equal to C2. This is equal to C3. So x1 is equal to minus C2, minus C3. So x1 is equal to minus 1 times C2, plus C3 times what? Plus C3 times minus 1. And then what is x2 equal to? x2 is just equal to C2. So it's 1 times C2, plus 0, times C3. x3 is just equal to C3, so it's 0 times C2, plus 1, times C3. And so this is another way of defining our subspace. All of the vectors that satisfy this is equal to this definition here. It's all the vectors whose components satisfy, or that lie in this plane, whose entries lie in that plane. And that's for any real numbers right there. Or another way of writing this, is v is equal to the span of the vectors minus 1, 1, and 0, and the vector minus 1, 0, and 1. Just like that. And we know that these are actually a basis for v because they're linearly independent. There's no way I can take linear combinations of this guy and make the second entry equal a 1 here. And likewise there's no way I can take linear combinations of this guy and make this third entry equal a 1 here. So these are also a basis for v. So given, that just using the technique we did before, we could set some vector, we could set some matrix A equal to minus 1, 1, 0, and then minus 1, 0, and 1. And then we can figure out that the projection of any vector x in our 3 onto v is going to be equal to, and we saw this, it's going to be equal to A times the inverse of A transpose A. All of that times A transpose and all of that times x. And you can do it. You have A here. You can figure out what the transpose of A is very easy. You can take A transpose A, then you can invert it. And it'll be very similar to what we did in the last video. It'll be a little less work, because this is a 3 by 2 matrix, instead of a 4 by 2 matrix. But you saw it is actually a lot of work. It's very hairy and you might make some careless mistakes. So let's figure out if there's another way that we can come up with this matrix right here. Now we know that if x is a member of R3, that x can be represented as a combination of some vector v, that is in our subspace, plus some vector w, that is in the orthogonal complement of the subspace, where v is a member of our subspace, and w is a member of the orthogonal complement of our subspace. Now by definition, that right there is the projection of x onto v, and this is the projection of x onto the orthogonal complement of v. So we can write that x is equal to the projection onto v of x, plus the projection onto v's orthogonal complement, or the orthogonal complement of v of x. So this is by definition, that any member of R3 can be represented this way. Now if we want to write this as matrix vector products, and two videos ago I showed you that these are linear transformations. So let me write that here. So they're linear transformations. So they can be written as matrix vector products. You see that right there. Let me define this matrix, I don't know, let me call this matrix T, let me just call it T. And let me do another. Let me do a letter, let me do B. And let's say that the projection onto the orthogonal complement of v of x, let's say that that's equal to some other matrix C, times x. And we know this is a linear transformation, so it can be represented as some matrix C times x. So what are these going to be equal to? Well x, if I want to write it as a linear transformation of x, I could just write it as the 3 by 3 identity matrix, times x, right? That's the same thing as x. That's going to be equal to the projection of x onto v, well that's just the same thing as B times x. And then plus the projection of x onto v's orthogonal complement, well that's just C times x. Plus C times x. And if you want to factor out the x on this side, we know that the matrix vector products exhibit the distributive property, so we could write that the identity matrix times x is equal to B plus C, times x. Or another way to view this equation is that this matrix must be equal to these two matrices. So we get that the identity matrix in R3 is equal to the projection matrix onto v, plus the projection matrix onto v's orthogonal complement. Remember, the whole point of this problem is to figure out this thing right here, is to solve or B. And we know a technique for doing it. You take A transpose, you can do this whole thing, but that might be pretty hairy. But maybe it's easy to find this guy. Maybe, I don't know. It actually turns out in the video, this one will be easy. So if it's easy to find this guy, we can just solve for B. If we subtract C from both sides, we get that B is equal to I, is equal to the identity matrix, minus the transformation matrix for the transformation onto v's orthogonal complement. So let's see what this is. Let's see if we can figure out what C is right there. So let's go back to our original. So remember-- let me rewrite the problem actually-- remember that v was equal to, essentially it's equal to all of the x1's, x2's, x3's that satisfy x1 plus x2, plus x3 is equal to 0. Or another way to say it is that all the x1's, x2's, and x3's that satisfy the equation 1, 1, 1, times x1, x2, x3 is equal to the 0 vector. Or this case it'll just be 0. We could write the 0 vector just like that. So 1 times x1, plus 1 times x2, plus 1 times x3 is going to equal the 0 vector. This is another way to write v. Now all of the x's that satisfied this right here, what is that? This is saying that v is equal to the null space of this matrix right there. The null space of this matrix is all of the vectors that satisfy this equation. So v is equal to the null space-- let me write it this way-- the null space of 1, 1, 1, just like that. Up here we, kind of, figured out v in kind of the traditional way. We figured out that v is the span of these things, but now we know that's the same thing is the null space of 1, 1, 1. These two statements are equivalent. Now we at least had a hunch that maybe, you know, we could figure out, straight up, this B here by doing all of this A transpose and, you know, by doing all of this silliness here. But our hunch is maybe if we could figure out the transformation matrix for the orthogonal complement of v right there, that then we could just apply this, kind of, that we can just solve for B given that the identity matrix minus this guy is going to be equal to B. So let's see if we can figure out the projection matrix, if we can figure out the transformation matrix for the orthogonal projection, for x onto the orthogonal projection of v. So this is v. What is v compliment? v compliment is going to be equal to the orthogonal complement, or v perp is going to be equal to the orthogonal complement of the null space of this matrix right here. Which is equal to what? Remember, the null space, its orthogonal complement-- a null space's orthogonal complement is equivalent to the row space or the column space of A transpose. We saw that multiple times. Or you could say the orthogonal complement of the row space is the null space. We've seen this many, many times before. So the orthogonal complement of this guy is going to be the column space of his transpose. So the column space of the transpose of this guy. So it's 1, 1, 1, just like that. Or we can write that v's orthogonal complement is equal to the span of 1, 1, 1. The column space of this matrix, we only have one column in it, so its column space is going to be the span of that one column. So just to visualize what we're doing here, that original equation for v, that satisfies that, that's just going to be some plane in R3. That is v right there. And now we just figured out what v's orthogonal complement is. It's going to be a line in R3. It's going to be all of the linear combinations of this guy. So it's going to be some line in R3. I haven't drawn it. You know this is going to be tilted more, and so is this, but it's going to be some line. So this is the orthogonal complement of v. So let's see if we can figure out. So remember, the projection-- let me do it this way. So this is the basis for v's orthogonal complement. So let's construct some matrix. I don't know, let me use a letter that I haven't used before. Let me construct some matrix D, whose columns are the basis vectors for the orthogonal complement of v. Well, there's only one basis vector, so it's going to be that. And we learned, in the last video and the video before that, that the projection of any vector in R3 onto v's orthogonal complement is going to be equal to D times D transpose D inverse, times D transpose, times x. Or another way to view it is that this thing right here, that thing right there is the transformation matrix for this projection. That is the transformation matrix. matrix So let's see if this is easier to solve this thing than this business up here, where we had a 3 by 2 matrix. That was the whole motivation for doing this problem. To figure out the projection matrix for v's subspace, we'd have to do this with the 3 by 2 matrix. It seems pretty difficult. Instead, let's find the projection matrix to get to the production onto v's orthogonal complement, which is this. So what is D transpose? So D transpose is just going to be equal to 1, 1, 1. What is D transpose times D? Well, that's D transpose. This is D, just like that. So what is this going to be equal to? This is just the dot product of that and that. 1 times 1, plus 1 times 1, plus 1 times 1, it equals 3. So this thing right here is equal to a 1 by 1 matrix 3. So let's write it down. So this is equal to D-- which is this matrix, 1, 1, 1-- times D transpose D inverse. So D transpose D is just a 1 by 1 matrix. And we're going to have to invert it. Actually, I've never defined the inverse of a 1 by 1 matrix for you just now, so it's just mildly exciting. Times D transpose. So D transpose looks like this, 1, 1, 1. And then all of that's times x. But this is the transformation matrix right there. Now what is the inverse of a 1 by 1 matrix? Now you just have to remember that A inverse times A is equal to the identity matrix. If we're dealing with a 1 by 1 matrix, then I'm just trying to figure out what, let's say, what matrix times 3 is going to be equal to the 1 by 1 identity matrix. So let's say that 3 inverse times 3 has to be equal to the identity matrix. 1 by 1 identity matrix. Well, the only matrix that's going to make this work out, to get this entry I'll just take this guy's entry times that guy's entry, is going to be this guy right here. The inverse of this 1 by 1 matrix has to be the matrix 1/3. 1/3 times 3 is equal to 1. This is almost trivially simple, but this is the inverse, that right there is the inverse matrix, for the 1 by 1 matrix 3. So this right here is just 1/3. And we could actually just take that out. It's a 1 by 1 matrix, which is essentially equivalent to a scalar. So this is going to be equal-- let me just draw a line here-- this thing is equal to 1/3-- actually, I don't want to confuse you. Let me rewrite it. So we get the projection of any vector in R3 onto the orthogonal complement of v, is equal to 1/3, that's 1/3, times the vector 1, 1, 1, times-- sorry, or wait, that is a vector or the matrix 1 on 1-- times that matrix transposed, 1, 1, 1. And then all of that times x. And you can see, this is a lot simpler than if we have to do all of this business with this matrix. That's a harder matrix to deal with. This 1, 1, 1 matrix is very easy. Now what is this going to be equal to? This is going to be equal to 1/3 times, we have a 3 by 1 times a 1 by 3 matrix, so it's going to result in a 3 by 3 matrix. And what do we get? So this first entry is going to be 1 times 1, which is 1. The second entry is going to be 1 times 1, which is 1. The third entry is going to be 1 times 1, which is 1. I think you see the pattern. The second grow, first column, 1 times 1, which is 1. So this is going to be a 3 by 3 matrix of 1's. So just like that we were able to get-- that was a pretty straightforward situation-- we were able to get the projection matrix for any vector in R3 onto v's orthogonal complement. Now, we know that this thing right here is our original C that we said. And we said that the identity matrix-- we wrote it up here. Let me refer back to what I wrote way up here. We said, look, the identity matrix is equal to the transformation matrix for the projection onto v, plus the transformation matrix for the projection onto v's orthogonal complement. Or we can write that the transformation matrix for the projection onto v is equal to the identity matrix minus the transformation matrix for the projection onto v's orthogonal complement. So if we say that the projection onto v of x is equal to B times x, we know that B is equal to the 3 by 3 identity matrix, minus C, and this is C right there. So B is equal to the identity matrix-- so that's just 1, 0, 0, 0, 1, 0, 0, 0, 1-- minus C, minus 1/3, times 1, 1, 1, 1, 1, 1, 1, 1, 1, just like that. And what is this going to be equal to? Let's see, let's, in our heads, multiply this out. All of these entries are going to be 1/3 essentially, if we multiply this out like that. So if we have 1 minus 1/3. I could write it out like that. It's 1/3, 1/3, 1/3. Everything is 1/3. 1/3, 1/3, 1/3, 1/3, 1/3, 1/3, and this just becomes a 1. So 1 minus 1/3 is 2/3. And all of the 1's minus 1/3 are going to be 2/3, so we could just go down the diagonal. And then the 0's minus 1/3 are going to be minus 1/3. Minus 1/3, minus 1/3, minus 1/3. You have minus 1/3, minus 1/3, and minus 1/3. And just like that, we've been able to figure out our projection, our transformation matrix, for the projection of any vector x onto v, by essentially finding this guy first, for finding the transformation matrix for the projection of any x onto v's orthogonal complement. Anyway, I thought that was pretty neat. And you could rewrite this as v equal to 1/3 times 2, 2, 2, 2's along back the diagonals and then you have minus 1's everywhere else. Anyway, see you in the next video.