Main content

# Introduction to residuals and least-squares regression

In linear regression, a residual is the difference between the actual value and the value predicted by the model (y-ŷ) for any given point. A least-squares regression model minimizes the sum of the squared residuals.

## Want to join the conversation?

- WHERE does the -140 +14/3x come from??(27 votes)
- where did -140 come from?(8 votes)
- That's the point at which the regression line meets the y-axis, the y-intercept.(1 vote)

- my question is the same as the first 2 previous...Please explain(5 votes)
- where did you get for 140 for y(3 votes)
- If you extend the y-axis, the y-intercept (the point where the line first hits the y-axis) will be approximately -140. Also, the slope of the line is 14/3. So, if you transcribe the above into the equation of the line- y=mx+b, you get y=-140+14/3x(7 votes)

- I'm unenthused with the flow of this lesson. The first three videos are great. Calculate the residuals. Then it suddenly jumps to "as you know, the z-scores are...". The residual idea is a very basic concept that we are learning in Algebra right now. The next step needs to be to define Least Squares Regression and have them do some calculations by having their graphing calculator generate a LSRL. I wish there were a video for that. Seems to get complicated fast, preventing me from using this for basic introduction (for obvious reasons we aren't able to learn this together in the classroom right now)(3 votes)
- Mr. Armerding, you are not up with the time. Because Sal has added a few videos explaining what
is at https://www.khanacademy.org/math/ap-statistics/bivariate-data-ap/least-squares-regression/v/calculating-the-equation-of-a-regression-line.**Least Squares Regression**(6 votes)

- Where did the -140 and the 14/3 come from? Thank you. (time stamp: 2.27min)(5 votes)
- -140 + 14/3x is the equation of the linear line y^(y hat).

to dive more into it:

-140 is the**y-intercept**of the linear line y^

// here's how i get to that conclusion:

as you can see the line passes x-intercept at x = 30 and continues to go down. even though the continuation can't be seen but we can guess that -140 is indeed the y-intercept.

14/3 is the**slope**of the linear equation y^.

// here's how i get to that conclusion:

slope formula is y2-y1/x2-x1. i will use 2 points from the line: (51,100) and (30,0) => 0-100/30-51 = 100/21 = 4.7; 14/3 = 4.66667 = aprx. 4.7)(2 votes)

- I have the same question...how did he get -140+14/3x. If Sal calculated it before...it should be said. How am I to grasp and understand what to be done(5 votes)
- Did Sal pre-calculate the equation? I can't ever do it that fast!

Why are mx and b switched in places?(3 votes)- Linear equations can be written as y=a+bx. This is how I learned them growing up in another country. Start at a and then go up from there.(4 votes)

- how did you find the slope of 14/3x(3 votes)
- You divide the change in y by the change in x. If you take two points with coordinates (a,b) and (c,d), the slope will be (d-b)/(c-a).(4 votes)

- How to mathematically calculate the -140, in the equation -140+14/3x(4 votes)

## Video transcript

- [Instructor] Let's say
we're trying to understand the relationship between
people's height and their weight. So what we do is we go
to 10 different people, and we measure each of their heights and each of their weights. And so on this scatter plot here, each dot represents a person. So, for example, this dot
over here represents a person whose height was 60
inches, or five feet tall. So that's the point 60 comma, and whose weight, which we have on the
y-axis, was 125 pounds. And so when you look at this scatter plot, your eyes naturally see
some type of a trend. It seems like, generally speaking, as height increases,
weight increases as well. But I said generally speaking. You definitely have circumstances where there are taller
people who might weigh less. But an interesting question
is can we try to fit a line to this data? And this idea of trying to fit a line as closely as possible to as
many of the points as possible is known as linear, linear regression. Now, the most common technique
is to try to fit a line that minimizes the squared
distance to each of those points, and we're gonna talk more
about that in future videos. But for now, we want to get
an intuitive feel for that. So if you were to just eyeball it and look at a line like that, you wouldn't think that it would
be a particularly good fit. It looks like most of the
data sits above the line. Similarly, something like this
also doesn't look that great. Here most of our data points
are sitting below the line. But something like this
actually looks very good. It looks like it's getting
as close as possible to as many of the points as possible. It seems like it's describing
this general trend. And so this is the actual regression line. And the equation here, we would write as, we'd write y with a little hat over it. And that means that we're
trying to estimate a y for a given x. It's not always going to be
the actual y for a given x. Because, as we see, sometimes the points aren't sitting on the line. But we say y hat is equal to, and our y-intercept, for this
particular regression line, it is negative 140 plus the slope 14 over three times x. Now, as we can see, for
most of these points, given the x-value of those points, the estimate that our
regression line gives is different than the actual value. And that difference between the actual and the estimate from the regression line is known as the residual. So let me write that down. So, for example, the
residual at that point, residual at that point is going to
be equal to, for a given x, the actual y-value minus the estimated y-value from the regression line for that same x. Or another way to think about it is, for that x-value, when x is equal to 60, we're talking about the
residual just at that point, it's going to be the actual y-value minus our estimate of what the y-value is from this regression
line for that x-value. So pause this video, and see if you can
calculate this residual. And you could visually imagine it as being this right over here. Well, to actually calculate the residual, you would take our actual
value, which is 125, for that x-value. Remember, we're calculating
the residual for a point. So it's the actual y there minus, what would be the estimated
y there for that x-value? Well, we could just go to this equation and say what would y hat
be when x is equal to 60? Well, it's going to be equal to, let's see, we have negative 140
plus 14 over three times 60. Let's see, 60 divided by three is 20. 20 times 14 is 280. And so all of this is going to be 140. And so our residual, for this point, is going to be 125 minus
140, which is negative 15. And residuals indeed can be negative. If your residual is negative,
it means, for that x-value, your data point, your actual
y-value, is below the estimate. If we were to calculate the residual here or if we were to calculate
the residual here, our actual for that x-value
is above our estimate, so we would get positive residuals. And as you will see later
in your statistics career, the way that we calculate
these regression lines is all about minimizing the
square of these residuals.