Main content
Course: Statistics and probability > Unit 12
Lesson 2: Error probabilities and power- Introduction to Type I and Type II errors
- Type 1 errors
- Examples identifying Type I and Type II errors
- Type I vs Type II error
- Introduction to power in significance tests
- Examples thinking about power in significance tests
- Error probabilities and power
- Consequences of errors and significance
© 2024 Khan AcademyTerms of usePrivacy PolicyCookie Notice
Type 1 errors
A Type 1 error occurs when the null hypothesis is true, but we reject it because of an usual sample result. Created by Sal Khan.
Want to join the conversation?
So what is the exact relationship between Type 1 error and the p-value?(2 votes)
This might sound confusing but here it goes: The p-value is the probability of observing data as extreme as (or more extreme than) your actual observed data, assuming that the Null hypothesis is true.
A Type 1 Error is a false positive -- i.e. you falsely reject the (true) null hypothesis. In addition, statisticians use the greek letter alpha to indicate the probability of a Type 1 error.
So, as you can see, the two terms are related but not "exactly related" since there is some slight difference in their definitions...
Anyway, inn practice, you would determine your level of significance beforehand (your alpha) and then reject the null hypothesis if your p-value turns out to be smaller than alpha.
Is this helpful?(9 votes)
- Which error does "power" correspond to? And what does that mean?(3 votes)
The power of a test is 1-(type 2 error). Keeping in mind that type 2 error is the probability of failing to reject H0 given that H1 is true. So the power of a test tells us something about how strong the test is, that is how well the test can differentiate between H0 and H1. To improve the power of a test one can lower the variance or one can increase alfa (type 1 error).
Power curves shows the power of the test given different values of H1. The longer H1 is from H0 the easier it is to differen(4 votes)
- can I see an example of
the type 1 error being worked out from using a worded statement?(2 votes)
Here's an easy example of a Type I error. Suppose you are tested for an extremely rare disease that affects only 1 in a million people. The test is 99.9% accurate. Your test comes back positive. It would almost certainly be a Type I error to conclude you have that disease. Here's why.
0.1% of the time, the test produces the wrong answer. Thus, out of a million people, you would get 1000 false positives. You would expect to get 1 person with the disease that has positive test.
Thus, you would reasonably expect 1000 of the 1001 positives to be false positives. This makes it nearly certain that you don't have the disease.
Thus, you almost certainly made a Type I error if you assumed you had the disease. The null hypothesis is that you don't have the disease and you almost certainly falsely rejected that hypothesis.
Therefore, the accuracy of a test must be in keeping with how likely it is that the hypothesis is true. A condition that affects half the population can reasonably be tested with a procedure that is 99.9% accurate. But a rare disease requires much greater accuracy because the false positives would tend to be far more common than true positives without extreme accuracy.
[Actually, I simplified the math a bit, you should be even more dubious about test than I indicated because a sample of a million people is not large enough to reasonably expect 1 person to have the disease. With 1 million people, you'd only have a 63% chance of someone having the disease, you would need about 2.5 million people to have a greater than 90% chance of having someone with the disease.](6 votes)
- How do I calculate power? Some of the other answers mention beta but don't say how to calculate either beta or power.(3 votes)
Where exactly is the null hypothesis explained? I wish there was some sort of logical flow to these videos...(3 votes)
I think I understand everything else, but where does the 0.5% come from? why that exact value?(2 votes)
I assume you mean the 0.05 (or 5%). That's basically just an arbitrary number. A famous Statistician by the name of R.A. Fisher commented that about 1 in 20 - which is 0.05 - is a convenient value to use. He used and apparently recommended that when writing some books. I guess it caught on and stuck.
There's nothing special about 0.05. Some professions aren't that strict, some are far more strict. If I recall correctly, physicists use something like 0.0001. And people who sort through dozens or hundreds of genes to try and find out what genes may influence some condition will use something even smaller.(3 votes)
I don't understand, first of all, what is a null hypothesis?(3 votes)- What's the difference between critical value and significance level ? OR are they same?(3 votes)
Level of significance is alpha, or the level of rarity you're setting as the benchmark to decide that the null hypothesis is false. (Alpha also happens to be the probability of type I error.) The critical value is the z-score corresponding to that benchmark, and it helps you decide whether or not to reject the null. For example, if you're doing a two-tailed test and you set alpha at .05, the critical values of z are 1.96 and -1.96. If the z-score you obtain for your data is greater than 1.96 or less than -1.96, you can reject the null. Hope this helps!(1 vote)
- Do Type 1 and Type 2 errors arise in estimation, are they part of confidence levels in a given sample or are they a feature of hypothesis testing?(2 votes)
They're just a feature of hypothesis testing. The reason you might do it can vary, but fundamentally, it comes down to this: It's rare that you wind up with a 100% chance of any result, so there's usually at least a small chance that you have made a Type 1 error.(1 vote)
Where is the video for type 2 errors?(2 votes)
Video transcript
I want to do a quick video on
something that you're likely to see in a statistics class,
and that's the notion of a Type 1 Error. And all this error means is that
you've rejected-- this is the error of rejecting-- let
me do this in a different color-- rejecting the
null hypothesis even though it is true. So for example, in actually all
of the hypothesis testing examples we've seen, we start
assuming that the null hypothesis is true. We always assume that the
null hypothesis is true. And given that the null
hypothesis is true, we say OK, if the null hypothesis is true
then the mean is usually going to be equal to some value. So we create some
distribution. Assuming that the null
hypothesis is true, it normally has some mean value
right over there. Then we have some statistic and
we're seeing if the null hypothesis is true, what is
the probability of getting that statistic, or getting a
result that extreme or more extreme then that statistic. So let's say that the statistic
gives us some value over here, and we say gee, you
know what, there's only, I don't know, there might be a 1%
chance, there's only a 1% probability of getting a result that extreme or greater. And then if that's low enough of
a threshold for us, we will reject the null hypothesis. So in this case we will--
so actually let's think of it this way. Let's say that 1% is
our threshold. We say look, we're going
to assume that the null hypothesis is true. There's some threshold that
if we get a value any more extreme than that value, there's
less than a 1% chance of that happening. So let's say we're looking
at sample means. We get a sample mean that
is way out here. We say, well, there's less
than a 1% chance of that happening given that the null
hypothesis is true. So we are going to reject
the null hypothesis. So we will reject the
null hypothesis. Now what does that
mean though? Let's say that this area, the
probability of getting a result like that or that much
more extreme is just this area right here. So let's say that's 0.5%, or
maybe I can write it this way. Let's say it's 0.5%. And because it's so unlikely to
get a statistic like that assuming that the null
hypothesis is true, we decide to reject the null hypothesis. Or another way to view it is
there's a 0.5% chance that we have made a Type 1 Error in
rejecting the null hypothesis. Because if the null hypothesis
is true there's a 0.5% chance that this could still happen. So in rejecting it we would
make a mistake. There's a 0.5% chance we've
made a Type 1 Error. I just want to clear that up. Hopefully that clarified
it for you. It's sometimes a little
bit confusing. But we're going to use what we
learned in this video and the previous video to now tackle
an actual example.