Vector triangle inequality

In the last video, we showed you the Cauchy-Schwarz Inequality. I think it's worth rewriting because this is something that's going to show up a lot. It's a very useful tool. And that just told us if I have two vectors, x and y, they're both members of Rn. And they're both nonzero vectors. And that was an assumption we had to make when we did the proof, otherwise there was a potential of dividing by one of their magnitudes. So that would've been a big no-no. But if we assume that they're both nonzero, then we can say that the absolute value of their dot products is going to be less than or equal to the products of their individual lengths. So that's the length of vector x and we defined that a couple of videos ago. And then this is the length of vector y. And of course, this is just a regular number and then each of these are just regular numbers. They're not vectors once you take a length. The length of a 50 dimensional vector could just be the number 3. It's just a scalar value. So this is just scalar multiplication here. And we also learned that the only time that this inequality turns into an equality is the situation where x is equal to some scalar multiple of y. And so in some textbooks you'll say-- and this has to be a nonzero scalar multiple. But that's a bit obvious. I told you that x and y are nonzero. So if this was 0 then x would be 0. And we already said that x is not 0. But if you want to say there you could say that you know c also is going to be nonzero. But that essentially just falls out of this information there. But if this is the case and if and only if this is the case, then we can say that the absolute value of the dot product of the two vectors is equal to the product of their lengths. Now, this is all just a review of what I did in the last video. Now what else can we do that's useful with it? So let's just play around a little bit. I can't claim to be experimenting, I know where this is going to go. Let's see what happens if I were to take the length of x plus y. So I'm going to add the two vectors and then take the length of that vector squared. Well we know from a couple of videos ago that the length squared can also be rewritten as the dot product of a vector with itself. This right here, x plus y, I know it looks like two vectors. But it's two vectors added to each other. So it's really a vector. x plus y is a real vector. I could graph x plus y. So the length of x plus y squared, I can rewrite it as the dot product of that vector with itself. So x plus y dot x plus y. And all of these are vectors. These aren't just numbers. And this is the dot product. It's just not normal multiplication. But we saw two videos ago that the dot product has the distributive and the associative and the commutative properties just like regular scalar multiplication. So you can kind of FOIL this out if that's how you remember multiplying your binomials. Or I will think of it more as just doing the distributive property twice. So this can be rewritten as x dot x. Actually, let me write it as the distributive property because that's sometimes not obvious to a lot of people. So let me write this x as a yellow x and let me write this whole term as x plus y. So this right here can be rewritten as x dot-- so this x dot this x plus y. And then it would be plus this y dot-- I want to just switch colors. Plus this y dot x plus y. It's good to see that when you're multiplying these, you're just applying the distributive property. All I did is I distributed this term along each of these terms into sum right here. So then I got this. And then I can distribute each of these into the sum. So then this becomes-- I'll be careful with the colors-- x dot x plus x dot y. Maybe this was a little bit overkill, but I think it's good to see that this isn't just some magic here. And we're just using the exact properties that we proved with the dot product. So that's that right there. And then it's plus y dot x. Plus this yellow y dot the yellow x. Sorry, dot the blue y. So the magnitude or the length of our vector x plus y squared can be rewritten like this. And I'll just switch back to one color. So this equals that and all of that-- what does this equal? This is equal to x dot x. What's x dot x? x dot x is just the magnitude. So let me write this is just equal to the magnitude of our vector x. I should stop using the word magnitude. The length of our vector x squared. And then I have two terms here. I have an x dot y and a y dot x. We know that x dot y and y dot x are really the same thing. We showed that order doesn't matter when you take the dot product, just like it doesn't matter with regular multiplication. So these are really the same terms. So we could write plus 2 times x dot y. And then finally, we have that last term sitting here. We have this y dot y. y dot y is the same thing as the length of our vector y squared. Now, let's see if we can break out our Cauchy-Schwarz Inequality. Or maybe Schwarz, I don't know if I'm pronouncing it right. But x dot y. t We have the absolute value of x dot y here. But we know that just x dot y is going to be-- it has to be less than or equal to the absolute value of x dot y. Why is that? Well this could be negative. I could show you examples of dot products that are negative. In fact, if x has all positive terms and y has all negative terms, you're going to have a negative dot product. So this could be negative or it could be positive. If it's positive the absolute value-- their equal to each other. If this is negative, than this absolute value is definitely going to be greater than it. We can add to the Cauchy-Schwarz Inequality and this is a bit obvious. We could say look, we could add a little x dot y is less than or equal to the absolute value of x dot y. Which is less than or equal to the length of x times the length of y. So x dot y is definite-- this, the dot product of x with y is definitely less than it's absolute value of that. Which is definitely less than the lengths of those two multiplied. So if I rewrite this, this statement right here is definitely less than or equal to this exact statement. But if I replace these with the lengths of the vectors. So that is definitely less than or equal to-- I'm just rewriting this x squared and I'll write the plus 2 there. Plus 2. But I want to make it very clear what I'm replacing here. And then I have the plus length of my vector y there squared. Now this I'm saying, this is definitely less than the absolute value of x dot y. Which is definitely less, by the Cauchy-Schwarz Inequality, definitely less than the product of the two lengths. So I'm just replacing this with the product of their two lengths. So I'm going to put the length of x times the length of y. And since this is the same as that, this is the same as that. But this is definitely less than this. This whole term has to be less than this whole term. Now let me just remind you what we were doing. I said that this thing that I wrote over here, this is the same as that. So this thing up here, which is the same as that, which is less than that also. So we can write that the magnitude of x plus y squared and not the magnitude, the length of the vector x plus y squared is less than this whole thing that I wrote out here. Or less than or equal to. Now, what is this thing? Remember, I mean this might look all fancy with my little double lines around everything. But these are just numbers. This length of x squared, this is just a number. Each of these are numbers and I can just say hey, look, this looks like a perfect square to me. This term on the right-hand side is the exact same thing as the length of x plus the length of y. Everything squared. If you just squared this out you'll get x squared plus 2 times the length of x times the length of y plus y squared. So our length of the vector x plus y squared is less than or equal to this quantity over here. And if we just take the square root of both sides of this, you get the length of our vector x plus y is less than or equal to the length of the vector x by itself plus the length of the vector y. And we call this the triangle inequality, which you might have remembered from geometry. Now why is it called the triangle inequality? Well you could imagine each of these to be separate side of a triangle. In fact, let's draw it. We can draw this in R2. Let me turn my graph paper on. Let me see where the graphs show up. If I turn my graph paper on right there, maybe I'll draw it here. So let's draw my vector x. So let's say that my vector x look something like this. Let's say that's my vector x. It's a vector 2, 4. So that's my vector x. And let's say my vector y-- well I'm just going to do it head to tail because I'm going to add the two. So my vector y-- I'm going to do it in nonstandard position. Let's say it's look something like-- let's say my vector y looks something like this. Draw it properly. That's my vector y. What does x plus y look like? And remember, I can't necessarily draw any two vectors on a two-dimensional space like this. I'm just assuming that these are in R2. But this is just to give you the idea. So then this is their sum, right? You took from the tail of x to the head of y. So this vector right here is the vector x plus y. And that's why it's called the triangle inequality. It's just saying that look, this thing is always going to be less than or equal to-- or the length of this thing is always going to be less than or equal to the length of this thing plus the length of this thing. And that's kind of obvious when you just learn two-dimensional geometry. That look, this is a much more efficient way of getting from this point to this point than going out here and then going out here. And then, what's the case in which this length is equal to these lengths? Well if you keep flattening this triangle out and you go to the extreme case where maybe the vector x looks like this. And if the vector y is just kind of going in the exact same-- vector y is going in the exact same direction, maybe it's going a little bit further. This is vector x, this is vector y. Now x plus y will just be this whole vector. Now that whole thing is x plus y. And this is the case now where you actually-- where the triangle inequality turns into an equality. That's why that little equal sign is there. The extreme case where essentially, x and y are collinear. And why does that work out? We can even go back to our math and understand that. So let me turn my graph off. We can go back to our math here. If I go back to this point, remember, right here I made the statement, look, this thing is definitely less than this thing over here. But what if I made an assumption? What if I said that x is equal to some scalar times y? And actually, I have to be careful because just some scalar times y-- remember, our Cauchy-Schwarz Inequality said that look, the inequality turns into an equality if x is some nonzero scalar of y. And then we can apply this. We can say that the absolute value of x dot y is the same as this over here. But I don't have the absolute value of x dot y here. I don't know that this is positive. I can say definitively that this is a positive quantity because I took the absolute value of it. No absolute value here. So the only way that I can assure that this is a positive quantity, that this is the same thing as the absolute value of x dot y is to enforce-- if I'm kind of going to go down this road, is to enforce that this term right here, that c be positive. Because if c is positive, then x dot y, if x dot y then that would be the same thing as cy dot y. Which we know is just the same thing as c times the magnitude of y squared. And the only way that I can ensure that this expression right here is equal to the absolute value of x dot y, the only way I can assure this is that c is positive. If c is negative, then this is going to be a negative number while this is a positive. So if I assume that this is positive, then I can say that x dot y is equal to the absolute value of x dot y. And since it's a scalar multiple, then I could say that that term is equal to, not just less than or equal to, the magnitude of x's and y's. So hopefully I'm not confusing you. So all I'm saying is, if I can assume that x is some positive scalar multiple of y, that this wouldn't be a less than sign. Then I could say that x dot y is the same thing as the absolute value of x dot y since this is positive. And if it's the same thing as the absolute value of x dot y and it's some scalar multiple of each other, than we could go down this other route. We could say that this thing here-- I don't want to get too messy. We could say that this is equal to that. If this is equal to that, then this would have become an equal sign, not a less than or equal to sign. And then we would have had the limiting case-- I don't want to call it the limiting case. But we could say that x plus y-- we would've done the same work, but we would've had an equal sign the whole way. Would equal the length of x. The length of x plus y would equal the length of x plus the length of y in the situation where x is equal to some positive scalar times y. So c is greater than 0. These two imply each other. And we saw that geometrically. I lost my axes here, but we see that the only time that the length of x plus y is equal to the length of x plus the length of y is when they're collinear. Over here this plus this is clearly-- you can just visually look at it-- longer than this right there. So you might be saying Sal, once again, this linear algebra's a little bit silly. We learned the triangle inequality in eighth or ninth grade. Why did you go through all of this pain to redefine it? And this is the interesting thing. What I just drew here and this is what you learned in ninth grade geometry. This is just in R2. This is just your Cartesian coordinates, or I don't want to use the word dimension too much because we're going to define that formally. But this is kind of your two-dimensional space that's going on. What's interesting or what's useful about linear algebra is, we've just defined the triangle inequality for arbitrarily large vectors, or vectors that have an arbitrarily high number of components. Each of these, these don't have to be in R2. This is true if we're in R100 where every vector has a hundred components to it. We've just defined some notion of the triangle inequality. We've abstracted well beyond just our two-dimensional Cartesian coordinates. Well beyond even our three dimensions to essentially, n dimensional space. And I haven't defined dimensions yet, but I think you're starting to appreciate what they are. But anyway, hopefully you found that useful. We can now take this result. And actually, that result with this result and define what the notion of an angle between two vectors are. Once again, you know, on some levels you're like well, why do we have to define an angle? Isn't an angle just-- isn't that just an angle? Well yeah, we know what an angle is in two dimensions, but what does an angle mean when you abstract things to n dimensions? Or when you're in Rn. So that's what we'll talk about in the next video.