Main content

## Statistics and probability

### Course: Statistics and probability > Unit 13

Lesson 2: Comparing two means- Statistical significance of experiment
- Statistical significance on bus speeds
- Hypothesis testing in experiments
- Difference of sample means distribution
- Confidence interval of difference of means
- Clarification of confidence interval of difference of means
- Hypothesis test for difference of means

© 2023 Khan AcademyTerms of usePrivacy PolicyCookie Notice

# Confidence interval of difference of means

Sal uses a confidence interval to help figure out if a low-fat diet helps obese people lose weight. Created by Sal Khan.

## Want to join the conversation?

- Well I understand Z Scores and Normal Distribution but i'm having a hard time understanding Confidence Intervals.(3 votes)
- Let's say that I run a sandwich shop. From experience, I know that the number of sandwiches I sell is normally distributed with a mean of 100 and a standard deviation of 10. There are two things that can go wrong with my store -- either I don't have enough customers to meet my labor costs or I run out of the bread and have to turn away all my late customers. So when I use my z-score stuff to realize that there is a 99.75% probability that I will sell between 70 and 130 sandwiches, that's important enough that we want to call [70,130] a 99.75% confidence interval. Then I conclude that if I order 130 rolls and schedule enough workers for a 70 worker day that I should expect only about one "bad day" per year.(8 votes)

- Okay, so we can make a sample distribution for the people who took the low fat diet and one for those who didn't. But why do we find their difference? My guess is that by finding the difference we get how much more effective is the former than the latter. But then how does the confidence interval part fit into this picture.

I'm looking for an answer that gives me an intuition more than a technical answer, if possible...(3 votes)- The confidence interval is there to account for the randomness associated with any sort of trial. People lose weight at different rates, some likely stick to the diet better than others, and people just experience general fluctuations in weight that may have effected their starting or ending weights. All these things add "noise" to the measurement and the confidence interval is a way to show how much noise is associated with a given result.

If the results of a study are well within that noise, it makes the difference much less credible than one where the noise is small relative to the difference in results.(3 votes)

- @ Andrew M:

Yes, I have encountered the CLT. Why exactly does it apply here? I thought the CLT said that given a sufficiently large sample size n, the distribution of the means of the samples would be approximately normally distributed. Why should the differences between two distributions of sample means be normally distributed? Those aren't means.

The only thing I can think of intuitively is that maybe if you add two normal r.v.s together, the resulting distribution is normal? Is this true? Otherwise, what parts of the CLT correspond to the difference of sample means?

Thanks!

Beth

Thanks!(4 votes)- Great! Thank you very much.(1 vote)

- Was this example a real study?(5 votes)
- Can we use the same calculation if the data is not normally distributed? What about if the sample sizes are not equal?(3 votes)
- The sample sizes do not need to be equal. See from about8:50where he explains calculation of variance of the distribution of the differences between the two populations.(1 vote)

- Why are you dividing by sample size for variance here? My understanding was that this is only for the standard deviation of a sample distribution of sample means and that does not seem to be the case here. You just have 2 samples of size n/m. Can you please clarify?(2 votes)
- mu x1 is the sample std dev which was given in the qn right, why are we again approximating using population std dev and dividing by 100?(2 votes)
- When to use a Z Table?

I understand the rule for 1 data set; If the sample size is less than 30, use a T-table.

But for 2 data sets, each data set sample size could be different and you could have a scenario one data set sample size is above 30 and the other is below 30. My guess is that a conservative answer would be to use a T-table if either data set is below 30. Is that a good rule of thumb or are there other factors that come into play?(2 votes) - At about12:16, why doesn't 4.67/10 + 4.04/10 work?(1 vote)
- The variance of the sum (or difference) of two random variables is additive. So for random variables X and Y:
`V( X + Y ) = V(X) + V(Y)`

`V( X - Y ) = V(X) + V(Y)`

This comes from the mathematical theory. I'm not sure where (or if) this is covered on Khan Academy.

Then, the 4.67/10 and 4.04/10 are the standard errors (standard deviation) of the two sample means. In order to be able to add them, we first need to convert them into variances by squaring them.(3 votes)

- were we asked to make a 95% C.I.? Or did we choose that? and if we did, why 95% and not any other percentage?(2 votes)

## Video transcript

We're trying to test whether
a new low-fat diet actually helps obese people
lose weight. 100 randomly assigned people are
assigned to group one and put on the low-fat diet. Another 100 randomly assigned
obese people are assigned to group two and put on a diet of
approximately the same amount of food, but not
as low in fat. So group two is the control,
just the no diet. Group one is the low
fat group, to see if it actually works. After four months, the mean
weight loss was 9.31 pounds for group one. Let me write this down. Let me make it very clear. So the low fat group, the mean
weight loss was 9.31 pounds. So our sample mean for group
one is 9.31 pounds, with a sample standard deviation
of 4.67. And both of these are obviously
very easy to calculate from the
actual data. And then for our control group,
the sample mean, 7.40 pounds for group two. With a sample standard deviation
of 4.04 pounds. And now, if we just look at it
superficially, it looks like the low-fat group lost more
weight, just based on our samples, than the
control group. If we take the difference
of them. So if we take the difference
between the low-fat group and the control group, we get 9.31
minus 7.40 is equal to, let's get the calculator out, 1.91. So the difference of our
samples is 1.91. So just based on what we see,
maybe you lose an incremental 1.91 pounds every four months
if you are on this diet. And what we want to do in this
video is to get a 95% confidence interval around
this number. To see that in that 95%
confidence interval, maybe, do we always lose weight? Or is there a chance that we can
actually go the other way with the low-fat diet? So in this video, 95%
confidence interval. In the next video, we'll
actually do a hypothesis test using this same data. And now to do a 95% confidence
interval, let's think about the distribution that we're
thinking about. So let's look at the
distribution. Of course we're going to think
about the distribution that we're thinking about. We want to think about the
distribution of the difference of the means. So it's going to have
some true mean here. Which is the mean of
the difference of the sample means. Let me write that. It's not a y, it's
an x1 and x2. So it's the sample mean of x1
minus the sample mean of x2. And then this distribution right
here is going to have some standard deviation. So it's the standard deviation
of the distribution of the mean of x1 minus the
sample mean of x2. It's going to have some standard
deviation here. And we want to make an
inference about this. Or I guess, the best way to
think about it, we want to get a 95% confidence interval. Based on our sample, we want to
create an interval around this, where we're confident that
there's a 95% chance that this true mean, the true mean
of the differences, lies within that interval. And to do that let's just think
of it the other way. How can we construct an interval
around this where we are 95% sure that any sample
from this distribution, and this is one of those samples,
that there is a 95% chance that we will select from this
region right over here. So we care about a 95% region
right over here. So how many standard deviations
do we have to go in each direction? And to do that we just have
to look at a Z table. And just remember, if we have
95% in the middle right over here, we're going to have 2.5%
over here and we're going to have 2.5% over here. We have to have 5% split
between these two symmetric tails. So when we look at a Z table,
we want the critical Z value that they give right
over here. And we have to be
careful here. We're not going to look up 95%,
because a Z table gives us the cumulative probability
up to that critical Z value. So the Z table is going to
be interpreted like this. So there's going to be some Z
value right over here where we have 2.5% above it. The probability of getting a
more extreme result or Z score above that is 2.5%. And the probability of getting
one below that is going to be 97.5%. But if we can find whatever Z
value this is right over here, it's going to be the same
Z value as that. And instead of thinking about
it in terms of a one tail scenario, we're going to think
of it in a two tail scenario. So let's look it up for
97.5% on our Z table. Right here. This is 0.975, or 97.5. And this gives us
Z value of 1.96. So Z is equal to 1.96. Or 2.5% of the samples from this
population are going to be more than 1.96 standard
deviations away from the mean. So this critical Z value right
here is 1.96 standard deviations. This is 1.96 times the standard deviation of x1 minus x2. And then this right here is
going to be negative 1.96 times the same thing. Let me write that. So this right here,
it's symmetric. This distance is going to be
the same as that distance. So this is negative 1.96 times
the standard deviation of this distribution. So let's put it this way,
there's a 95% chance that our sample that we got from our
distribution-- this is the sample as a difference of
these other samples. There's a 95% chance that 1.91
lies within 1.96 times the standard deviation of
that distribution. So you could view
it as a standard error of this statistic. So x1 minus x2. Let me finish that sentence. There's a 95% chance that 1.91,
which is the sample statistic, or the statistic that
we got, is within 1.96 times the standard deviation
of this distribution of the true mean of of the
distribution. Or we could say it the
other way around. There's a 95% chance that the
true mean of the distribution is within 1.96 times the
standard deviation of the distribution of 1.91. These are equivalent
statements. If I say I'm within three feet
of you, that's equivalent to saying you're within
three feet of me. That's all that's saying. But when we construct it this
way, it becomes pretty clear, how do we actually construct
the confidence interval? We just have to figure out
what this distance right over here is. And to figure out what that
distance is, we're going to have to figure out what the
standard deviation of this distribution is. Well the standard deviation of
the differences of the sample means is going to be equal to,
and we saw this in the last video-- in fact, I think I have
it right at the bottom here-- it's going to be equal
to the square root of the variances of each of those
distributions. Let me write it this way. So the variance, I'll
kind of re-prove it. The variance of our distribution
is going to be equal to the sum of the
variances of each of these sampling distributions. And we know that the variance
of each of these sampling distributions is equal to the
variance of this sampling distribution, is equal to the
variance of the population distribution, divided
by our sample size. And our sample size in
this case is 100. And the variance of this
sampling distribution, for our control, is going to be equal
to the variance of the population distribution for
the control divided by its sample size. And since we don't know
what these are, we can approximate them. Especially, because our n is
greater than 30 for both circumstances. We can approximate these with
our sample variances for each of these distributions. So let me make this clear. Our sample variances for each
of these distributions. So this is going to be our
sample standard deviation one squared, which is the sample
variance for that distribution, over 100. Plus my sample standard
deviation for the control squared, which is the
sample variance. Standard deviation squared
is just the variance divided by 100. And this will give us the
variance for this distribution. And if we want the standard
deviation, we just take the square roots of both sides. If we want the standard
deviation of this distribution right here, this is the variance
right now, so we just need to take the square roots. Let's calculate this. We actually know these values. S1, our sample standard
deviation for group one is 4.67. We wrote it right
here, as well. It's 4.76 and 4.04. The S is 4.67, we're going
to have to square it. And the S2 is 4.04, we're going
to have to square it. So let's calculate that. So we're going to take the
square root of 4.67 squared divided by 100 plus 4.04
squared, divided by 100. And then close the
parentheses. And we get 0.617. Let me write it right here. This is going to be
equal to 0.617. So if we go back up over here,
we calculated the standard deviation of this distribution
to be 0.617. So now we can actually calculate
our interval. Because this is going
to be 0.617. So if you want 1.96 times that,
we get 1.96 times that 0.617, I'll just write the
answer we just got. So we get 1.21. So the 95% confidence interval
is going to be the difference of our means, 1.91, plus or
minus this number, 1.21. So what's our confidence
interval? So the low end of our confidence
interval, and I'm running out of space, 1.91 minus
1.21, that's just 0.7. So the low end is 0.7. And then the high end, 1.91
plus 1.21, that's 2.12. let me just make sure of that. My brain sometimes doesn't work
properly when I'm making these videos. 3.12. So just to be clear, there's not
a pure 95% chance that the true difference of the true
means lies in this. We're just confident that
there's a 95% chance. And we always have to put a
little confidence there, because remember, we didn't
actually know the population standard deviations, or the
population variances. We estimated them
with our sample. And because of that, we don't
know that it's an exact probability. We just have say we're confident
that it is a 95% probability. And that's why we just say it's
a confidence interval. It's not a pure probability. But it's a pretty neat result. So we're confident that there's
a 95% chance that the true difference of these two
samples-- and remember, let me make it very clear, the expected
value of the sample means is actually the same thing
as the expected value of the populations. And so, what this is giving us
is actually a confidence interval for the true difference
between the populations. If you were to give everyone,
every possible person, diet one. And every possible
person diet two. This is giving us a confidence
interval for the true population means. And so when you look at this,
it looks like diet one actually does do something. Because in any case, even at the
low end of the confidence interval, you still have
a greater weight loss than diet two. Hopefully, that doesn't
confuse you too much. In the next video, we're
actually going to do a hypothesis test with
the same data.