Introduction to Type I and Type II errors

what we're going to do in this video is talk about type 1 errors and type 2 type 2 errors and this is in the context of significance testing so just as a little bit of review in order to do a significance test we first come up with a null and an alternative hypothesis and we'll do this on some population in question these will say some hypotheses about a true parameter for this population and the null hypothesis tends to be kind of what was always assumed or the status quo while the alternative hypothesis hey there's news here there's there's there's something alternative here and to test it and we're really testing the null hypothesis we're going to decide whether we want to reject or fail to reject the null hypothesis we take a sample we take a sample from this population using that sample we calculate a statistic we calculate a statistic that's trying to estimate the parameter in question and then using that statistic we try to come up with the probability of getting that statistic the probability of getting that statistic that we just calculated from that sample of a certain size given if we were to assume that our null hypothesis if our null hypothesis is true and if this probability which is often known as a p-value is below some threshold that we set ahead of time which is known as the significance level then we reject the null hypothesis let me write this down so this right over here this is our p-value this should be all be review we introduce it in other videos we have seen in other videos if our p-value is less than our significance level then we reject reject our null hypothesis and if our p-value is greater than or equal to our significance level alpha then we fail to reject fail to reject our null hypothesis and when we reject our null hypothesis some people say that might suggest the alternative hypothesis and the reason why this makes sense is if the probability of getting this statistic from a sample of a certain size if we assume that the null hypothesis is true is reasonably low if it's below a threshold maybe this threshold is 5% if the probability of that happening was less than 5% then hey maybe it's reasonable to reject it but we might be wrong in either of these scenarios and that's where these errors come into play let's make a grid to make this clear so there's the reality let me put reality up here so the reality is there's two possible scenarios in reality one is is that the null hypothesis is true and the other is that the null hypothesis is false and then based on our significance test there's two things that we might do we might reject the null hypothesis or we might fail to reject the null hypothesis and so let's put a little grid here to think about the different combinations the different scenarios here so in a scenario where the null hypothesis is true but we rejected that feels like an error we shouldn't reject something that is true and that indeed is a type one error type one error you shouldn't reject the null hypothesis if it was true and you could even figure out what is the probability of getting a type one error well that's going to be your significance level because if your null hypothesis is true let's say that there your significance level is 5% well five percent of the time even if your null hypothesis is true you're going to get a statistic that's going to make you reject the null hypothesis so one way to think about the probability of a type one error is your significance level now if your null hypothesis is true and you fail to reject it well that's good this we could write this as this is a correct correct conclusion the good thing just happened to happen this time now if your null hypothesis is false and you reject it that's also good that is the correct correct conclusion but if your null hypothesis is false and you fail to reject it well then that is a type 2 error that is a type 2 error now with this context in the next few videos we will actually do some examples where we try to identify one whether the an error is occurring and what is that error is a type 1 or a type 2