Projections onto subspaces

Many videos ago we introduced the idea of a projection. And in that case we dealt more particularly with projections onto lines that went through the origin. So if we had some line-- let's say L-- and let's say L is equal to the span of some vector v. Or you could say, alternately, that L is equal to the set of all multiples of v, such that the scalar factors are just any real numbers. These are both representations of lines that go through the origin. We defined a projection of any vector onto that line. Let me just draw it real fast. So let me see, we draw some axes. So that is my-- I want to draw it a little bit straighter than that-- that is my vertical axis and that is my horizontal axis. Just like that and let's say I have some line that goes through the origin. Let's say-- that doesn't go through the origin-- let's say that that line right there goes through the origin. So that is L. We knew visually that a projection of some vector x onto L-- so let's say that that is a vector x. Visually, if you were to draw-- if you have some light coming straight down it would be the shadow of x onto L. So this right here, that right there, was the projection onto the line L of the vector x. And we defined it more formally. We kind of took a perpendicular. We said that x minus the projection of x onto L is perpendicular to the line L, or perpendicular to everything-- orthogonal to everything-- on the line L. But this is how at least I visualize this. It's kind of the shadow as you go down onto the line L. And this was a special case, in general, of projections. You might notice that L is going to be a valid subspace. You could prove it to yourself. It contains the zero vector. It goes through the origin. It's closed under addition-- any member of it plus any other member of it is going to be another member of it. It's closed under scalar multiplication-- you can take any member of it and scale it up or down, it's still going to be an L. So this was a subspace when we defined this. And just as a bit of a reminder of what it was, we were able to figure out what this projection is for some line L. If you have some spanning vector, the projection onto this line L that goes through the origin of the vector x, we figured out was x dot your spanning vector for your line, so x dot v over v dot v, which is really just the length of v squared. So all of this was a number and you want it to be in the same direction as your line. It's going to be another vector in your line. So it's going to be times the vector v. So it's just going to be a scaled up or scaled down version of your spanning vector. Maybe your spanning vector is like that. And really any vector in your line could be a spanning vector. Any vector other than the zero vector. Now that was a projection onto a line which was a special kind of subspace. But now we're going to broaden our definition of a projection to any subspace. So we already know that if-- let me draw a little dividing line to show that we're doing something slightly different-- if v is a subspace of Rn then v complement is also a subspace space of Rn. So the orthogonal complement of v is also a subspace. And let's say we have some members, or let me write it this way. If we have these two subspaces-- you have a subspace and you have this orthogonal complement-- we already learned that if you have any member of Rn-- so let's say that x is a member of our Rn-- then x can be represented as a sum of a member of v and a member of the orthogonal complement of v. Where-- let me write this-- the vector v is a member of the subspace v and the vector w is a member of the orthogonal complement of the subspace v. Just like that. We saw this several videos ago. We proved that this was true for any member Rn. Now given that, we can define the projection of x onto the subspace v as being equal to, just the part of x -- these are two orthogonal parts of x-- we define the projection onto v as a part of x that came from v. It's equal to just that vector v. Alternately you could say that the projection of x onto the orthogonal complement of-- sorry I wrote transpose-- the orthogonal complement of v is going to be equal to w. So this piece right here is a projection onto the subspace v. This piece right here is a projection onto the orthogonal complement of the subspace v. Now what I want to do in this video is show you that these two definitions-- that this definition right here which is then in conjunction with this right here-- this is the equivalent to what we learned up here if the subspace v that we're dealing with is a line. Because this was a valid subspace. But not all subspaces are going to be lines. And to see this we can revisit an example that we saw several videos ago. Several videos ago we had this matrix here A. This 2 by 2 matrix. And then we had this other vector b that was a member of the column space of A. We did this problem to show you that the shortest solution to this right here was a unique member of the row space. Hopefully that gets your memory on track for this problem when we first did it. But let me graph it and show you that for the solution of that problem we could have just as easily taken a projection onto a subspace. Let me graph everything in this problem. This might help you remember also about the problem. So let me draw my axes just like that. So the first thing we learned-- you know you could solve this but I already did this in a video. I think it was two or three videos ago-- the null space of A, or all of the x's that satisfy Ax is equal to zero, is a span of the vector 2, 3. So you go 2 to the right. 1, 2. And then you go 3 up. 1, 2, 3. And so it's the span of this vector. And so the span of that vector is just all the points. Well that vector specifies that point. But if you scale this vector up and down you're going to specify all of the point on this line. All the points on that line. Let me draw it like that. That's good enough. It shouldn't curve down like that at the end. So let me draw that a little straighter. So this is the null space. That is our null space of that matrix right there. And then the row space was a span of the vector 3, minus 2. You see that right here. 3, minus 2 is the first row. This guy is just a multiple of that one. That's why we don't have this guy right here in the span as well. And if we were to graph it, 3, minus 2. You go out 3, then you go down 1, 2. it would be the span of this vector right there. Let me draw it like that. Now you take all of the scalar multiples of that vector and you put those vectors in standard position. They're going to specify, or their tips are going to be on points along this line right there. Along that line right there. I'm trying to make sure I draw them orthogonally. So this right here is the row space. That right there is the row space of A which is the same thing as a column space of A transpose. And we know that these guys are each other's orthogonal complements. We know, we've seen this in multiple videos, that the null space of A is the orthogonal complement of the row space. And we also know that the orthogonal complement of the null space is equal to the row space. Everything in this is orthogonal to everything in that. Everything in that is orthogonal to everything in this. You can see it here in this graph. That these two spaces, which are represented by these lines that go through the origin, are orthogonal. And it makes sense that any-- we said at really the beginning of the video-- that anything in R2 in this situation, can be represented as some sum of a unique member of our row space and a unique member of its orthogonal complement. Let's say I have that point right there. How could I represent it as a sum of a member of this and a member of that? Well if I go along this guy, I have this vector right here. I have that vector right there along that line. And then I have this vector right here. If I were to shift it-- this is drawn in standard position, but I can draw a vector wherever I want. These lines are just all of the vectors drawn in standard position with their tails at the origin. But we learned, in really, I think, the first or second vector videos, that I can draw them wherever I want. So if I add this vector and that vector, I can shift this vector over and this vector will be right there. And there you have it. I took an arbitrary point in R2 and I can represent it as a sum of a member of my row space and a member of the row space's orthogonal complement or the null space. But just to review, what we originally did in that problem is we looked at the solution set of this. We said the solution set of this looks like this. It has a particular solution plus members of your null space, plus homogeneous solutions. We've seen that multiple videos ago. So 3, 0-- it looks like this-- plus members of the null space. So your solution set is going to be parallel to this but shifted to the right by 3. So it looks-- let me draw it a little neater than that-- Let me draw it like that. And then it goes down like, the second part I didn't draw-- there you go. Oh that's not good either. Maybe I'm being too picky. OK, so this is your solution set. And if you remember in that video we said, hey there's some member of this solution set that is also a member of our row space and that member of the solution set that is a member of our row space is going to be the shortest solution. And we saw that. You can see it visually right here. Right? This vector right here. It is in our row space. It is a member of our row space. And it also specifies a point on our solution set. And you could see visually that it's going to be the shortest solution. And one way you could think about it is, this is the projection-- let me pick a good, different, new color-- any solution on our solution set-- let me see right there-- let's say that that is some arbitrary solution on our solution set. Right? That's going to be a point in R2 and any point in R2 can be represented as a sum of some vector in our row space and some vector in our null space. And so if I have this vector right here, how can I do that? Well, I could represent it as a sum of this guy right here and then this vector right here. That vector right here. And this vector right here is clearly a member of my null space. I just shifted over. This line is only when I draw in standard position. This vector right here-- I'm just showing it heads to tails-- if I add this member of my row space to this member of my null space, I get an arbitrary solution to my solution set. And if you think about it, the projection of my arbitrary solution onto my row space will be this guy right here. And that just comes from our-- well there are two ways to think about it-- we could say that this is the solution right here. We could say our solution right here is equal to some member of my row space plus some member of my null space. This is the row space. That is the null space. And so by the definition of a projection onto a subspace I just gave you, we know that the projection of this solution onto my-- let me write a little bit-- onto my row space of my solution, is just equal to this first thing. It's equal to the component of it that's in my row space. It's other component, we could call it, is in the orthogonal complement of my row space. Or it's in my null space. So this is just going to be equal to the R vector. Now, I want to show you that that is essentially equal to the definition that we did before. That this is completely identical to the definition of a projection onto a line because in this case the subspace is a line. So let's find a solution set. And the easiest one, the easiest solution that we could find is if we set C as equal to 0 here. We know that x equals 3, 0 is one of these solutions. So x equals 3, 0 looks like that. So we know x equals 3, 0 is a solution. And what we want to do is we want to find the shortest solution. Or we want to find the projection of x onto the row space. Or if we wanted, we could also think of it is a projection of x onto this line. This line is equal to the row space. So let's do that. And I'm doing this to show you that this definition of a projection onto a subspace that I've just introduced you to in this video, it is completely identical to the definition, or it's not identical, it's consistent with the definition of a projection onto a line. Although this is more general because a subspace doesn't have to be a line. But in this case it is a line. So let's do that. So the projection of the vector 3, 0 onto our row space, which is a line so we can use that formula, it is equal to 3, 0 dot the spanning vector for our row space, right? Dot the spanning vector for our row space. So it's 3, minus 2. There's a bunch of spanning vectors for your row space. This is just the one we happened to pick. So dot 3, minus 2 all over the spanning vector dotted with itself. 3, minus 2 dot 3, minus 2. And then this is just going to be one big scalar and then we want to multiply that-- or essentially scale up-- our actual spanning vector by that. So this is a projection of this solution onto my row space, which should give me this vector right here. Because we're just taking a projection onto a line, because a row space in this subspace is a line. And so we used the linear projections that we first got introduced to, I think, when I first started doing linear transformations. So let's see this is 3 times 3 plus 0 times minus 2. This right here is equal to 9. This is 3 times 3 plus minus 2 times minus 2. So that's 9 plus 4. That's 13. So it's 9/13 times this vector right here. So it's going to be equal to 9/13 times the vector 3, minus 2. Which is equal to the vector 27/13 and then minus 18/13, which is this vector right here. We got this exact answer when we first did it, although we just didn't use the projection onto a line. But now we see that this is exactly consistent with what we did before. We just used the projection onto a line. And we see that this is consistent with our new, broader definition of a projection. Here we were able to do it because we did it onto a line. But here I'm calling a projection onto any subspace. We know how to do it if it's a line, but so far I've just kind of defined it onto an arbitrary subspace. But I haven't giving you a nice mathematical, I guess, or computational way to figure out what this is going to be if this isn't a line. In fact, I haven't even shown you when this is general, whether this is definitely a linear transformation. We know that when you take the projection onto the line it's a linear transformation. But I haven't shown you that when we take a projection onto an arbitrary subspace that it is a linear projection. I'll do that in the next video.