Learn how to use a P-value and the significance level to make a conclusion in a significance test.
This article was designed to provide a bit of teaching and a whole lot of practice. The questions are ordered to build your understanding as you go, so it's probably best to do them in order. Onward!

We use pp-values to make conclusions in significance testing. More specifically, we compare the pp-value to a significance level α\alpha to make conclusions about our hypotheses.
If the pp-value is lower than the significance level we chose, then we reject the null hypotheses H0H_0 in favor of the alternative hypothesis HaH_\text{a}. If the pp-value is greater than or equal to the significance level, then we fail to reject the null hypothesis H0H_0—this doesn't mean we accept H0H_0. To summarize:
p-value<αreject H0accept Hap-valueαfail to reject H0\begin{aligned} p \text{-value} < \alpha &\Rightarrow \text{reject } H_0 \Rightarrow \text{accept }H_\text{a} \\\\ p \text{-value} \geq \alpha &\Rightarrow \text{fail to reject } H_0 \end{aligned}
Let's try a few examples where we use pp-values to make conclusions.

Example 1

Alessandra designed an experiment where subjects tasted water from four different cups and attempted to identify which cup contained bottled water. Each subject was given three cups that contained regular tap water and one cup that contained bottled water (the order was randomized). She wanted to test if the subjects could do better than simply guessing when identifying the bottled water.
Her hypotheses were H0:p=0.25H_0: p=0.25 vs. Ha:p>0.25H_\text{a}: p>0.25 (where pp is the true likelihood of these subjects identifying the bottled water).
The experiment showed that 2020 of the 6060 subjects correctly identified the bottle water. Alessandra calculated that the statistic p^=2060=0.3ˉ\hat p=\dfrac{20}{60}=0.\bar3 had an associated P-value of approximately 0.0680.068.
Question A (Example 1)
What conclusion should be made using a significance level of α=0.05\alpha=0.05?
Choose 1 answer:
Choose 1 answer:
Question B (Example 1)
In context, what does this conclusion say?
Choose 1 answer:
Choose 1 answer:
Question C (Example 1)
How would the conclusion have changed if Alessandra had instead used a significance level of α=0.10\alpha=0.10?
Choose 1 answer:
Choose 1 answer:

Example 2

A certain bag of fertilizer advertises that it contains 7.25 kg7.25\text{ kg}, but the amounts these bags actually contain is normally distributed with a mean of 7.4 kg7.4\text{ kg} and a standard deviation of 0.15 kg0.15\text{ kg}.
The company installed new filling machines, and they wanted to perform a test to see if the mean amount in these bags had changed. Their hypotheses were H0:μ=7.4 kgH_0: \mu=7.4\text{ kg} vs. Ha:μ7.4 kgH_\text{a}: \mu \neq 7.4\text{ kg} (where μ\mu is the true mean weight of these bags filled by the new machines).
They took a random sample of 5050 bags and observed a sample mean and standard deviation of xˉ=7.36 kg\bar x=7.36\text{ kg} and sx=0.12 kgs_x=0.12\text{ kg}. They calculated that these results had a P-value of approximately 0.020.02.
Question A (Example 2)
What conclusion should be made using a significance level of α=0.05\alpha=0.05?
Choose 1 answer:
Choose 1 answer:
Question B (Example 2)
In context, what does this conclusion say?
Choose 1 answer:
Choose 1 answer:
Question C (Example 2)
How would the conclusion have changed if they had instead used a significance level of α=0.01\alpha=0.01?
Choose 1 answer:
Choose 1 answer:

Ethics and the significance level α\alpha

These examples demonstrate how we may arrive at different conclusions from the same data depending on what we choose as our significance level α\alpha. In practice, we should make our hypotheses and set our significance level before we collect or see any data. Which specific significance level we choose depends on the consequences of various errors, and we'll cover that in videos and exercises that follow.
Loading