Projection is closest vector in subspace

Let's say I've got some subspace, v, that's a plane in R3. So let me see if I can draw it respectably well. Let me draw some plane in R3. Maybe it looks something like that. That is my subspace. I think that's good enough. Let me see if I can draw it a little bit better than that. There you go. So that is my plane in R3. This is v. This is a subspace. And let's say that I have some other vector, x, any vector in R3. So my vector x looks like this. That is my vector x. Now what I want to show you in this video is that the projection of x onto our subspace-- and let's say that this is our 0 vector right there. I want to show you that the projection of x onto our subspace is the closest vector in our subspace to x. So let me draw that out, and maybe it'll make a little more sense. So the projection of x on to the subspace will look something like this. It will look something like that. That right there, that green vector right there, is the projection of the vector x onto our subspace v. That's our vector x. Now, let's take some arbitrary other vector in our subspace. Let's just take this one. This is just some other arbitrary vector in our subspace. Let me draw a little bit differently. We draw it like that. Let's call that vector v. That's clearly another vector on our subspace. It lies on that plane. What I want to show you is that the distance from x to our projection of x on to v is shorter than the distance from x to any other vector. From the distance from x to any other vector. And obviously, the way I've drawn it, it looks pretty clear that this line is shorter than that line. But that was just a particular choice that I picked. Let's prove it that's general. So what I want to prove is that the distance between x and its projection onto the subspace-- and the way we can get that is essentially just take the length of the vector of x minus the projection of x onto my subspace. This length right here is this length right here, is this length right here. So x minus the projection of x onto v, that's going to be this vector right there. Let me do that in a different color. Don't want to reuse colors too often. That's going to be that vector right there. We could call that vector a. It's clearly in the orthogonal complement of v, because it's orthogonal to this guy. And that's the definition of a projection, actually, so this is equal to a. My claim, what I want to show you, is that this distance a is shorter than any distance here, is less than or equal to the distance between x and v, where v is any member. So that's this distance, right here. This vector right here, that distance right-- let me draw this vector. The vector x minus v looks like this. It looks like that. That is the vector x minus v, right? If you take v plus x minus v, you're going to go to x. So what I want to show is that this distance, the length of a, of the difference between x and its projection, is always going to be less than the distance between x and any other vector in the subspace. So that is x minus v. So let's see if we can prove that. So let's take the square of this distance. So the square of x-- actually, let me do that. Yeah, let me write that way. So we want to concern ourselves with the square of the distance of x minus v, where x is some vector in R3, and v is some vector in R3 that's also a member of our subspace. It sits on this plane. So what's the square of this going to be? Well, x minus v is equal to this vector. Let me draw a new vector here. It is equal to this-- wait, let me draw it in in this yellow. It's equal to this vector. It's equal to this yellow vector plus a. Right? x minus v is-- this magenta vector that starts here and goes there-- clearly equal to this yellow vector plus this orange vector. So let me call that yellow vector b. Now, what is b equal to? Well, b is going to be equal to this vector, this green vector, which is the projection of x onto v, minus this purple vector. Minus this mauve vector, I guess. Minus v. That's what b is. So we could write x minus v as being equal to the sum of the vector b plus the vector a. So x minus v is equal to b plus a. And if we're taking the length of x minus v squared, that's the same thing as the length of b plus a squared. And that's just equal to b plus a dotted with b plus a, which is the same thing as b dot b. Let me write that a little bit neater. Let me write it down here. This is going to be equal to, I'll switch colors, b dot b. Right, b dot b plus b dot a plus a dot b. So plus 2 times a dot b, plus a dot a. Now, a and b are clearly orthogonal. b is the difference between two vectors in our subspace. The subspace is closed under addition and subraction. So b is a member of our subspace. a is orthogonal to everything in our subspace, by definition. So since a is clearly orthogonal to b, a is-- by definition-- going to be in the orthogonal compliment of the subspace. This is going to be 0. And then this right here will simplify to the length of b squared. And then this right here is going to be plus the length of a squared. So we get the distance between x and some arbitrary member of our subspace squared is equal to the length of b, right here, plus the length of a squared. Now, a was a distance between our vector x and our projection, right? That's what the definition of a was. a was the distance between our vector x and our projection. Now, this number right here is going to be at least 0 or positive. So this right here is definitely going to be greater than or equal to a squared. Or another way to say it is that the distance between x minus v squared is definitely going to be greater than or equal to the distance of a squared. Or the distance between x minus v-- this is still going to be a positive quantity, length is always going to be positive-- is greater than or equal to the length of vector a. Or what's that length of vector a? a is just this thing right here. So let's write our result. The length of the vector x minus v, or the distance between x and some arbitrary member of our subspace, is always going to be greater than or equal to the length of a, which is just the distance between x and the projection of x onto our subspace. So there you have it. We've shown, and the original graph kind of hinted at it, that the projection of x onto v is the closest vector in our subspace to x. It's closer than any other vector in v to our arbitrary vector in R3, x. And we've proven it right there.