Defining the angle between vectors

A couple of videos ago we introduced the idea of the length of a vector. That equals the length. And this was a neat idea because we're used to the length of things in two- or three-dimensional space, but it becomes very abstract when we get to n dimensions. If this has a hundred components, at least for me, it's hard to visualize a hundred dimension vector. But we've actually defined it's notion of length. And we saw that this is actually a scalar value. It's just a number. In this video, I want to attempt to define the notion of an angle between vectors. As you can see, we're building up this mathematics of vectors from the ground up, and we can't just say, oh, I know what an angle is because everything we know about angles and even lengths, it just applies to what we associate with two- or three-dimensional space. But the whole study of linear algebra is abstracting these ideas into multi-dimensional space. And I haven't even defined what dimension is yet, but I think you understand that idea to some degree already. When people talk about one or two or three dimensions. So let's say that I have some vector-- let's say I have two vectors, vectors a and b. They're nonzero and they're members of Rn. And I don't have a notion of the angle between them yet, but let me just draw them out. Let me just draw them as if I could draw them in two dimensions. So that would be vector a right there. Maybe that's vector b right there. And then this vector right there would be the vector a minus b. And you can verify that just the way we've learned to add and subtract vectors. Or you know, this is heads to tails. So b plus a minus b is of course, going to be vector a. And that all just works out there. To help us define this notion of angle, let me construct another triangle that's going to look a lot like this one. But remember, I'm just doing this for our simple minds to imagine it in two dimensions. But these aren't necessarily two-dimensional beasts. These each could have a hundred components. But let me make another triangle. Well, it should look similar. Say it looks like that. And I'm going to define the sides of the triangles to be the lengths of each of these vectors. Remember, the lengths of each of these vectors, I don't care how many components there are, they're just going to be your numbers. So the length of this side right here is just going to be the length of a. The length of this side right here is just going to be the length of vector a minus vector b. And the length of this side right here is going to be the length of vector b. Now the first thing we want to make sure is that we can always construct a triangle like that. And so under what circumstances could we not construct a triangle like this? Well we wouldn't be able construct a triangle like this if this side. if b, if the magnitude-- so let me write this down. It's kind of a subtle point, but I want to make this very clear. In order to define an angle, I want to be comfortable that I can always make this construction. And I need to make sure that-- let me write reasons why I couldn't make this construction. Well what if the magnitude of b was greater than, or the length of vector b was greater than the length of vector a plus the length of vector a minus b? In two dimensions, I could never draw a triangle like that then because you would have this length plus this length would be shorter than this thing right here. So you could never construct it. And I could do with all the sides. What if this length was larger than one of these two sides? Or what if that length was larger than one of those two sides? I could just never draw a two-dimensional triangle that way. So what I'm going to do is I'm going to use the triangle-- the vector triangle inequality to prove that each of these sides is less than or equal to the sum of the other sides. I could do the same thing. Let me make the point clear. I could show that if a, for whatever reason, was greater than the other sides plus b, then I wouldn't be able to create a triangle. And the last one of course is if a minus b, for whatever reason, was greater than the other two sides, I just wouldn't be able to draw a triangle in a plus b. So I need to show that for any vectors, any real vectors-- nonzero, real vectors that are members of Rn-- that none of these can ever happen. I need to prove that none of those can happen. So what does the triangle inequality tell us? The triangle inequality tells us that if I have the sum of two vectors, if I take the length of the sum of two vectors, that that is always going to be less than-- and these are nonzero vectors. This is always going to be less than or equal to the sum of each of their individual lengths. So let's see if we can apply that to this triangle right here. So what is the magnitude, the length of a? Well I can rewrite vector a. What is vector a equal to? Vector a is equal to vector b plus vector a minus b. I mean I'm just rewriting the vector here. I'm just rewriting a here as a sum of the other two vectors. Nothing fancy there. I haven't used the triangle inequality or anything. I've just used my definition of vector addition. But here now, if I put little parentheses here, now I can apply the triangle inequality. And I say, well, you know what? This is going to be, by the triangle inequality, which we've proved, it's going to be less than or equal to the lengths of each of these vectors. Vector b plus the length of vector a minus b. So we know that the length of a is less than the sum of that one and that one. So we don't have to worry about this being our problem. We know that that is not true. Now let's look at b. So is there any way that I can rewrite b as a sum of two other vectors? Well sure. I can write it as a sum of a plus, let me put it this way. If that vector right there is a minus b, the same vector in the reverse direction is going to be the vector b minus a. So a plus the vector b minus a. That's the same thing as b. And you can see it right here. The a's would cancel out and you're just left with the b there. Now by the triangle inequality, we know that this is less than or equal to the length of vector a plus the length of vector b minus a. Now you're saying hey, Sal, you're dealing with b minus a. This is the length of a minus b. And I can leave this for you to prove it based on our definition of vector lengths, but the length of b minus a is equal to minus 1 times a minus b. And I'll leave it to you to say that look, these lengths are equal. Because essentially-- I could leave that, but I think you can take that based on just the visual depiction of them that they're the exact same vectors, just in different directions. And I have to be careful with length because it's not just in two dimensions. But I think you get the idea and I'll leave that for you to prove that these lengths are the same thing. So we know that b is less than the length of those two things. So we don't have to worry about that one right there. Finally, a minus b. The magnitude or the length of vector a minus b. Well I can write that as the length of-- or I can write that as vector a plus vector minus b. If we just put a minus b right there and go in the other directions, we could say minus b, which would be in that direction plus a would give us our vector a minus b. Actually, I don't even have to go there. That's obvious from this. I just kind of put the negative in the parentheses. Well the triangle inequality, and this might seem a little mundane to you, but it really shows us that we can always define a regular planar triangle based on these vectors in this way. It tells us this is less than or equal to the length of our vector a plus the length of minus b. And I just said and you could prove it to yourself, that this is the same thing as the length of b. So we just saw that this is definitely less than those two. This is definitely less than those two. And that is definitely less than those two. None of the reasons that would keep us from constructing a triangle are valid. So we can always construct a triangle in this way from any arbitrary nonzero vectors in Rn. We can always construct this. Now, to define an angle, let me redraw it down here. Let me redraw the vectors, maybe a little bit bigger. That's vector a. This is vector b. And then let me just draw it this way. This is the vector right there. That is the vector a minus b. And we said we're going to define a corresponding regular, run of the mill, vanilla triangle whose lengths are defined by the lengths of the vectors, by the vector lengths. So this is the length of b, that side. This is the length of a minus b. And then this is the length of a. Now that I know that I can always construct a triangle like this, I can attempt to define-- or actually, I will define my definition of an angle between two vectors. So we know what an angle means in this context. This is just a regular, run of the mill, geometric triangle. Now, my definition of an angle between two vectors I'm going to say-- so this is what I'm trying to define. This is what I'm going to define. These can have arbitrary number of components, so it's hard to visualize. But I'm going to define this angle as the corresponding angle in a regular, run of the mill triangle where the sides of the run of the mill triangle are the two vectors and then the opposite side is the subtraction, is the length of the difference between the two vectors. This is just the definition. I'm defining this, the angle between two vectors in Rn that could have an arbitrary number of components, I'm defining this angle to be the same as this angle, the angle between the two sides, the two lengths of those vectors in just a regular, run of the mill triangle. Now, what can I do with this? Well, can we find a relationship between all of these things right here? Well sure. If you remember from your trigonometry class, and if you don't, I've proved it in the playlist. You have the law of cosines. And I'll do it with an arbitrary triangle right here just because I don't want to confuse you. So if this is side a, b, and c and this is theta, the law of cosines tells us that c squared is equal to a squared plus b squared minus 2ab cosine of theta. I always think of it as kind of a broader Pythagorean theorem because this thing does not have to be a right angle. It accounts for all angles. If this becomes a right angle, then this term disappears and you're just left with the Pythagorean theorem. But we've proven this. This applies to just regular, run of the mill triangles. And lucky for us, we have a regular, run of the mill triangle here. So let's apply the law of cosines to this triangle right here. And the way I drew it, they correspond. The length of this side squared. So that means the length of a minus b squared. Length of vector a minus vector b, that's just the length of that side. So I'm just squaring that side. It equals the length of vector b squared plus the length of vector a squared minus 2 times the length of-- I'll just write two times length of vector a times the length of vector b times the cosine of this angle right here. Times the cosine of that angle. And I'm defining this angle between these two vectors to be the same as this angle right there. So if we know this angle, by definition, we know that angle right there. Well, we know that the square of our lengths of a vector when we use our factor definition of length, that this is just the same thing as a vector dotted with itself. So that's a minus b dot a minus b. It's all going to be equal to this whole stuff on the right-hand side. But let me simplify the left-hand side of this equation. a minus b dot a minus b, this is the same thing as a dot a-- those two terms-- minus a dot b. And then I have minus b dot a. Those two terms right there. And then you have the minus b dot minus b. That's the same thing as a plus b dot b. Remember, this is just a simplification of the left-hand side. And I can rewrite this. a dot a, we know that's just the length of a squared. a dot b and b dot a are the same thing, so we have two of these. So this right here, this term right there will simplify to minus 2 times a dot b. And then finally, b dot b. We know that that's just the length of b squared. I just simplified or maybe I just expanded-- that's a better word. When you go from one term here to three terms, you can't say you simplified it. But I expanded just the left-hand side and so this has to be equal to the right-hand side by the law of cosines. So that is equal to-- I almost feel like instead of rewriting it, let me just copy and paste it. What did I just do? Copy, edit. Copy and paste. There you go. I don't know it that was worth it. But maybe I saved a little bit of time. So that is equal to that right there. And then we can simplify. We have a length of a squared here, length of a squared there. Subtract it from both sides. The length of b squared here, length of b squared there. Subtract it from both sides. And then, what can we do? We can divide both sides by minus 2 because everything else has disappeared. And so that term and that term will both become 1's. And all we're left with is the vector a dot the vector b. And this is interesting because all of a sudden we're getting a relationship between the dot products of two vectors. We've kind of gone away from their definition by lengths. But the dot product of two vectors is equal to the product of their lengths, their vector lengths. And they can have an arbitrary number of components. Times the cosine of the angle between them. Remember, this theta, I said this is the same as when you draw this kind of analogous, regular triangle. But I'm defining the angle between them to be the same as that. So I can say that this is the angle between them. And obviously, the idea of between two vectors, it's hard to visualize if you go beyond three dimensions. But now we have it at least, mathematically defined. So if you give me two vectors we can now, using this formula that we've proved using this definition up here, we can now calculate the angle between any two vectors using this right here. And just to make it clear, what happens if a is a-- and maybe it's not clear from that definition, so I'll make it clearer here that by definition, if a is equal to some scalar multiple of b where c is greater than 0, we'll define theta to be equal to 0. And if c is less than 0, so a is collinear, but goes in the exact opposite direction, we'll define theta to be equal to 180 degrees. And that's consistent with what we understand about just two-dimensional vectors. If they're collinear and kind of the scalar multiples the same. That means a looks something like that and b looks something like that. So we say oh, that's a 0 angle. And if they go the other way, if a looks something like-- this is the case where a is just going in the other direction from b. a goes like that and b goes like that, we define the angle between them to be 180 degrees. But everything else is pretty well defined by the triangle example. I had to make the special case of these because it's not clear you really get a triangle in these cases because the triangle kind of disappears. It flattens out if a and b are on top of each other or if they're going in the exact opposite direction. So that's why I wanted to make a little bit of a side note right there. Now, using this definition of the angle between the vectors, we can now define the idea of perpendicular vectors. So we can now say perpendicular vectors-- this is another definition-- and this won't be earth shattering, but it kind of is because we've generalized this to vectors that have an arbitrary number of components. We're defining perpendicular to mean the theta between-- two vectors a and b are perpendicular if the angle between them is 90 degrees. And we can define that. We can take two vectors, dot them. Take their dot product. Figure out their two lengths and then you could figure out the angle between them. And if it's 90 degrees. you can say that they are perpendicular angles. And I want to be very clear here that this is actually not defined for the 0 vector right here. So this situation right here, not defined for the 0 vector. Because if you have the 0 vector, then this quantity right here is going to be 0 and then this quantity right here is going to be 0. And there's no clear definition for your angle. If this is 0 right here, you did 0 is equal to 0 times cosine of theta. And so if you wanted to solve for theta you'd get cosine of theta is equal to 0/0, which is undefined. But what we can do is create a slightly more general word than the word perpendicular. So you have to have a defined angle to even talk about perpendicular. If the angle between two vectors is 90 degrees, we're saying by definition, those two vectors are perpendicular. But what if we made the statement and we can-- if you look at them, if the angle between two vectors is 90 degrees, what does that mean? So let's say that theta is 90 degrees. Let me draw a line here. Let's say that theta is 90 degrees. Theta is equal to 90 degrees. What does this formula tell us? It tells us that a dot b is equal to the length of a times the length of b times cosine of 90 degrees. What's cosine of 90 degrees? It's 0. You can review your unit circle if that doesn't make a lot of sense. But that is equal to 0, so this whole term is going to be equal to 0. So if theta is equal to 90 degrees, then a dot b is equal to 0. And so this is another interesting takeaway. If a and b are perpendicular, then their dot product is going to be equal to 0. Now if their dot product is equal to 0, can we necessarily say that they are perpendicular? Well what if a or b is the 0 vector? The 0 vector-- let me call it z for 0 vector. Or I could just draw. The 0 vector dot anything is always going to be equal to 0. So does that mean that the 0 vector is perpendicular to everything? Well no. Because the 0 vector I said, we have to have the notion of an angle between things in order to use the word perpendicular. So we can't use the 0 vector. We can't say just because two vectors dot products are equal to 0 that they are perpendicular. And that's because the 0 vector would mess that up because the 0 vector is not defined. But if we say, and we have been saying, that a and b are nonzero, if they are nonzero vectors, then we can say that if a and b are nonzero and their dot product is equal to 0, then a and b are perpendicular. So now it goes both ways. But what if we just have this condition right here? What if we just have the condition that a dot b is equal to 0? It seems like that's kind of just a simple, pure condition. And we can write a word for that. And these words are often used synonymously, but hopefully you understand the distinction now. We can say that if two vectors dot product is equal to 0, we will call them orthogonal. As I always say, spelling isn't my best subject. But this is kind of a neat idea. This tells us that-- well, that all perpendicular vectors are orthogonal. And it also tells us that the 0 vector is orthogonal to everything else. To everything, even to itself. The 0 dot 0 vector you still get 0. So by definition, it's orthogonal. So for the first time probably in your mathematical career, you're seeing that the words-- you know, every time you first got exposed to the words perpendicular and orthogonal in geometry or maybe in physics or wherever else, they were always kind of the same words. But now I'm introducing a nice, little distinction here and you can kind of be a little smart aleck with teachers. Oh, you know, it's perpendicular only is the vectors aren't-- if neither of them are 0 vector. Otherwise, if their dot product is 0, you can only say that they're orthogonal. But if they're nonzero you could say that they're orthogonal and perpendicular. But anyway, I thought that I would introduce this little distinction for you in case you have someone that likes to trip you up with words. But it also I think highlights that we are building a mathematics from the ground up and we have to be careful about the words we use. And we have to be very precise about our definitions. Because if we're not precise about our definitions and we build up a bunch of mathematics on top of this and do a bunch of proofs, one day we might scratch our heads and read some type of weird ambiguity. And it might have all came out of the fact that we weren't precise enough in defining what some of these terms mean. Well anyway, hopefully you found this useful. We can now take the angle or we can now determine the angle between vectors with an arbitrary number of components.