If you're seeing this message, it means we're having trouble loading external resources on our website.

If you're behind a web filter, please make sure that the domains *.kastatic.org and *.kasandbox.org are unblocked.

Main content
Current time:0:00Total duration:10:35

Proof (part 1) minimizing squared error to regression line

Video transcript

In the last video, we showed that the squared error between some line, y equals mx plus b and each of these n data points is this expression right over here. In this video, I'm really just going to algebraically manipulate this expression so that it's ready for the calculus stage. So we can actually optimize, we can actually find the m and b values that minimize this value right over here. So this is just going to be a ton of algebraic manipulation. But I'll try to color code it well so we don't get lost in the math. So let me just rewrite this expression over here. So this whole video is just going to be rewriting this over and over again. Just simplifying it a bit with algebra. So this first term right over here, y1 minus mx1 plus b squared, this is all going to be the squared error of the line. So this first term over here, I'll keep it in blue, is going to be if we just expand it, y1 squared minus 2 times y1 times mx1 plus b, plus mx1 plus b squared. All I did is I just squared this binomial right here. You can imagine if this was a minus b, it would be a squared minus 2ab plus b squared. That's all I did. Now I'll just have to do that for each of the terms. And each term is only different by the x and the y coordinates right over here. And I'll go down so that we can kind of combine like terms. So this term over here squared is going to be y2 squared minus 2 times y2 times mx2 plus b plus mx2 plus b squared. Same exact thing up here. Except now it was with x2 and y2, as opposed to x1 and y1. And then we're just going to keep doing that n times. We're going to do it for the third, x3, y3, keep going, keep going. All the way until we get the this nth term over here. And this nth term over here when we square it is going to be yn squared minus 2yn times mxn plus b, plus mxn plus b squared. Now, the next thing I want to do is actually expand these out a little bit more. So let's actually scroll down. So this whole expression, I'm just going to rewrite it, is the same thing as-- and remember this is just the squared error of the line. So let me rewrite this top line over here. This top line over here is y1 squared. And then I'm going to distribute this 2y1. So this is going to be minus 2y1mx1, that's just that times that. Minus 2y1b. And then plus, and now let's expand mx1 plus b squared. So that's going to be m squared x1 squared, plus 2 times mx1 times b plus b squared. All I did, if was a plus b squared, this is a squared plus 2ab plus b squared. And we're going to do that for each of these terms. Or for each of these colors, I guess you could say. So now let's move to the second term. It's going to be the same thing. But instead of y1's and x1's, it's going to be y2's and x2's. So it is y2 squared minus 2y2mx2 minus 2y2b plus m squared x2 squared, plus 2 times mx2b plus b squared. And we're going to keep doing this all the way to get the nth term. I guess color we should say. So this is going to be yn squared minus 2ynmxn. And you don't even have to think. You just have to kind of substitute these with n's now. We could actually look at this. But it's going to be the exact same thing. Minus 2ynb plus m squared xn squared, plus 2mxnb plus b squared. So once again, this is just the squared error of that line with n points. Between those n points and the line y equals mx plus b. So let's see if we can simplify this somehow. And to do that what I'm going to do is I'm going to kind of try to add up a bunch of these terms here. So if I were to add up all of these terms right here, if I were to add up this column right over there, what do I get? It's going to be y1 squared plus y2 squared all the way to all the way to yn squared. That's those terms right over there. So I'm going to have that. And then have this common 2m amongst all of these terms over here. So let me write that down. So then you have this 2m here, 2m here, 2m here. Let me put parentheses around here. So you have these terms all added up. Then you have minus 2m times all of these terms. Actually, let me color code it so you see what we're doing. I want to be very careful with this math so nothing seems too confusing. Although this is really just algebraic manipulation. If I had all of these up, I get y1 squared plus y2 squared all the way to yn squared. I'll put some parentheses around that. And then to that, we have this common term, we have this minus 2m, minus 2m, minus 2m. And so we can distribute those out. And so I should actually write it like this. So we have a minus 2m, once we distribute it out up here, we're just going to be left with a y1x1. Or maybe I can call it an x1y1. That's that over there with the 2m factored out. Let me do that in another color. I want to make this easy to read. Plus x2y2. Plus xnyn. Well we're going to keep adding up-- we're going to do this n times. All the way to plus xnyn. This last term over here, ynxn, same thing. So that's the sum. So this stuff over here, the sum of all of this stuff right over here, is the same thing as this term right over here. And then we have to sum this right over here. And you see again, we can factor out here a minus 2b out of all of these terms. So we have minus 2b times y1 plus y2 plus all the way to to yn. So this business. These terms right over here, when you add them up, give you these terms, or this term, right over there. And let's just keep going. And in the next video, we're probably going to run out of time in this one, I'll simplify this more and clean up the algebra a good bit. So then the next term, what is this going to be? Same drill. We can factor out an m squared. So we have m squared times times x1 squared plus x2 squared-- actually, I want to color code them, I forgot to color code these over here. Plus all the way to xn squared. Let me color code these. This was a yn squared. And this over here was a y2 squared. So this is exactly this. So in this last step we just did, this thing over here is this thing right over here. And of course we have to add it. So I'll put a plus out front. We're almost done with this stage of the simplification. So over here, we have a common 2mb, so let's put a plus 2mb times, once again, x1 plus x2 plus all the way to xn. So this term right over here this is the exact same thing as this term over here. And then finally, we have a b squared in each of these. And how many of these b squared do we have? Well we have n of these lines, right? This is the first line, second line, then bunch, bunch, bunch all the way to the nth line. So we have b squared added to itself n times. So this right over here is just b squared n times. So we'll just write that as plus n times b squared. Let me remind ourselves what this is all about. This is all just algebraic manipulation of the squared error between those n points and the line y equals mx plus b. It doesn't look like I've simplified it much. And I'm going to stop in the video right now. In the next video, we're just going to take off right here and try to simplify this thing.