Introduction to projections

Let's say I have a line that goes through the origin. I'll draw it in R2, but this can be extended to an arbitrary Rn. Let me draw my axes. Those are my axes right there, not perfectly drawn, but you get the idea. Let me draw a line that goes through the origin here. So that is my line there. And we know that a line in any Rn-- we're doing it in R2-- can be defined as just all of the possible scalar multiples of some vector. So let's say that this is some vector right here that's on the line. We can define our line. We could say l is equal to the set of all the scalar multiples-- let's say that that is v, right there. So it's all the possible scalar multiples of our vector v where the scalar multiples, by definition, are just any real number. So obviously, if you take all of the possible multiples of v, both positive multiples and negative multiples, and less than 1 multiples, fraction multiples, you'll have a set of vectors that will essentially define or specify every point on that line that goes through the origin. And we know, of course, if this wasn't a line that went through the origin, you would have to shift it by some vector. It would have to be some other vector plus cv. But anyway, we're starting off with this line definition that goes through the origin. What I want to do in this video is to define the idea of a projection onto l of some other vector x. So let me draw my other vector x. Let's say that this right here is my other vector x. Now, a projection, I'm going to give you just a sense of it, and then we'll define it a little bit more precisely. A projection, I always imagine, is if you had some light source that were perpendicular somehow or orthogonal to our line-- so let's say our light source was shining down like this, and I'm doing that direction because that is perpendicular to my line, I imagine the projection of x onto this line as kind of the shadow of x. So if this light was coming down, I would just draw a perpendicular like that, and the shadow of x onto l would be that vector right there. So we can view it as the shadow of x on our line l. That's one way to think of it. Another way to think of it, and you can think of it however you like, is how much of x goes in the l direction? So the technique would be the same. You would draw a perpendicular from x to l, and you say, OK then how much of l would have to go in that direction to get to my perpendicular? Either of those are how I think of the idea of a projection. I think the shadow is part of the motivation for why it's even called a projection, right? When you project something, you're beaming light and seeing where the light hits on a wall, and you're doing that here. You're beaming light and you're seeing where that light hits on a line in this case. But you can't do anything with this definition. This is just kind of an intuitive sense of what a projection is. So we need to figure out some way to calculate this, or a more mathematically precise definition. And one thing we can do is, when I created this projection-- let me actually draw another projection of another line or another vector just so you get the idea. If I had some other vector over here that looked like that, the projection of this onto the line would look something like this. You would just draw a perpendicular and its projection would be like that. But I don't want to talk about just this case. I want to give you the sense that it's the shadow of any vector onto this line. So how can we think about it with our original example? In every case, no matter how I perceive it, I dropped a perpendicular down here. And so if we construct a vector right here, we could say, hey, that vector is always going to be perpendicular to the line. And we can do that. I wouldn't have been talking about it if we couldn't. So let me define this vector, which I've not even defined it. What is this vector going to be? If this vector-- let me not use all these. We know we want to somehow get to this blue vector. Let me keep it in blue. That blue vector is the projection of x onto l. That's what we want to get to. Now, one thing we can look at is this pink vector right there. What is that pink vector? That pink vector that I just drew, that's the vector x minus the projection, minus this blue vector over here, minus the projection of x onto l, right? If you add the projection to the pink vector, you get x. So if you add this blue projection of x to x minus the projection of x, you're, of course, you going to get x. We also know that this pink vector is orthogonal to the line itself, which means it's orthogonal to every vector on the line, which also means that its dot product is going to be zero. So let me define the projection this way. The projection, this is going to be my slightly more mathematical definition. The projection onto l of some vector x is going to be some vector that's in l, right? I drew it right here, this blue vector. I'll trace it with white right here. Some vector in l where, and this might be a little bit unintuitive, where x minus the projection vector onto l of x is orthogonal to my line. So I'm saying the projection-- this is my definition. I'm defining the projection of x onto l with some vector in l where x minus that projection is orthogonal to l. This is my definition. That is a little bit more precise and I think it makes a bit of sense why it connects to the idea of the shadow or projection. But how can we deal with this? I mean, this is still just in words. How can I actually calculate the projection of x onto l? Well, the key clue here is this notion that x minus the projection of x is orthogonal to l. So let's see if we can use that somehow. So the first thing we need to realize is, by definition, because the projection of x onto l is some vector in l, that means it's some scalar multiple of v, some scalar multiple of our defining vector, of our v right there. So we could also say, look, we could rewrite our projection of x onto l. We could write it as some scalar multiple times our vector v, right? We can say that. This is equivalent to our projection. Now, we also know that x minus our projection is orthogonal to l, so we also know that x minus our projection-- and I just said that I could rewrite my projection as some multiple of this vector right there. You could see it the way I drew it here. It almost looks like it's 2 times its vector. So we know that x minus our projection, this is our projection right here, is orthogonal to l. Orthogonality, by definition, means its dot product with any vector in l is 0. So let's dot it with some vector in l. Or we could dot it with this vector v. That's what we use to define l. So let's dot it with v, and we know that that must be equal to 0. We're taking this vector right here, dotting it with v, and we know that this has to be equal to 0. That has to be equal to 0. So let's use our properties of dot products to see if we can calculate a particular value of c, because once we know a particular value of c, then we can just always multiply that times the vector v, which we are given, and we will have our projection. And then I'll show it to you with some actual numbers. So let's see if we can calculate a c. So if we distribute this c-- oh, sorry, if we distribute the v, we know the dot product exhibits the distributive property. This expression can be rewritten as x dot v, right? x dot v minus c times v dot v. I rearranged things. We know that c minus cv dot v is the same thing. We could write it as minus cv. This is minus c times v dot v, and all of this, of course, is equal to 0. And if we want to solve for c, let's add cv dot v to both sides of the equation. And you get x dot v is equal to c times v dot v. Solving for c, let's divide both sides of this equation by v dot v. You get-- I'll do it in a different color. c is equal to this: x dot v divided by v dot v. Now, what was c? We are saying the projection of x-- let me write it here. The projection of x onto l is equal to some scalar multiple, right? We know it's in the line, so it's some scalar multiple of this defining vector, the vector v. And we just figured out what that scalar multiple is going to be. It's going to be x dot v over v dot v, and this, of course, is just going to be a number, right? This is a scalar still. Even though we have all these vectors here, when you take their dot products, you just end up with a number, and you multiply that number times v. You just kind of scale v and you get your projection. So in this case, the way I drew it up here, my dot product should end up with some scaling factor that's close to 2, so that if I start with a v and I scale it up by 2, this value would be 2, and I'd get a projection that looks something like that. Now, this looks a little abstract to you, so let's do it with some real vectors, and I think it'll make a little bit more sense. And nothing I did here only applies to R2. Everything I did here can be extended to an arbitrarily high dimension, so even though we're doing it in R2, and R2 and R3 is where we tend to deal with projections the most, this could apply to Rn. Let me do this particular case. Let me define my line l to be the set of all scalar multiples of the vector-- I don't know, let's say the vector 2, 1, such that c is any real number. Let me draw my axes here. That's my vertical axis. This is my horizontal axis right there. And so my line is all the scalar multiples of the vector 2 dot 1. And actually, let me just call my vector 2 dot 1, let me call that right there the vector v. Let me draw that. So I go 1, 2, go up 1. That right there is my vector v. And the line is all of the possible scalar multiples of that. So let me draw that. So all the possible scalar multiples of that and you just keep going in that direction, or you keep going backwards in that direction or anything in between. That's what my line is, all of the scalar multiples of my vector v. Now, let's say I have another vector x, and let's say that x is equal to 2, 3. Let me draw x. x is 2, and then you go, 1, 2, 3. So x will look like this. Vector x will look like that. Well, let me draw it a little bit better than that. Vector x will look like that. That is vector x. But what we want to do is figure out the projection of x onto l. We can use this definition right here. So let me write it down. The projection of x onto l is equal to what? It's equal to x dot v, right? Where v is the defining vector for our line. So it's equal to x, which is 2, 3, dot v, which is 2, 1, all of that over v dot v. So all of that over 2, 1, dot 2, 1 times our original defining vector v. So what's our original defining vector? It's this one right here, 2, 1. So times the vector, 2, 1. And what does this equal? When you take these two dot of each other, you have 2 times 2 plus 3 times 1, so 4 plus 3, so you get 7. This all simplified to 7. And then this, you get 2 times 2 plus 1 times 1, so 4 plus 1 is 5. So you get 7/5. That will all simplified to 5. That was a very fast simplification. You might have been daunted by this strange-looking expression, but when you take dot products, they actually tend to simplify very quickly. And then you just multiply that times your defining vector for the line. So we're scaling it up by a factor of 7/5. So multiply it times the vector 2, 1, and what do you get? You get the vector-- let me do it in a new color. You get the vector, 14/5 and the vector 7/5. And just so we can visualize this or plot it a little better, let me write it as decimals. 14/5 is 2 and 4/5, which is 2.8. And this is 1 and 2/5, which is 1.4. And so the projection of x onto l is 2.8 and 1.4. So 2.8 is right about there, and I go 1.4 is right about there, so the vector is going to be right about there. I haven't even drawn this too precisely, but you get the idea. This is the projection. Our computation shows us that this is the projection of x onto l. If we draw a perpendicular right there, we see that it's consistent with our idea of this being the shadow of x onto our line now. Well, now we actually can calculate projections. In the next video, I'll actually show you how to figure out a matrix representation for this, which is essentially a transformation.