If you're seeing this message, it means we're having trouble loading external resources on our website.

If you're behind a web filter, please make sure that the domains *.kastatic.org and *.kasandbox.org are unblocked.

Main content

Course: Digital SAT Math > Unit 11

Lesson 10: Evaluating statistical claims: advanced
Test prep
Digital SAT Math
Advanced: Problem solving and data analysis
Evaluating statistical claims: advanced
© 2024 Khan AcademyTerms of usePrivacy PolicyCookie Notice

Evaluating statistical claims | Lesson

Google Classroom
A guide to evaluating statistical claims on the digital SAT

What are evaluating statistical claims problems?

We routinely conduct research to answer questions such as "how many residents are in favor of a new law" or "is a new medical treatment effective?" While research results can give us powerful insights, we must carefully consider how the research is conducted, which in turn affects what conclusions can be drawn.
For example:
  • If a survey was given to individuals of one ethnicity, then the results of the survey are not representative of individuals of other ethnicities.
  • If a medical treatment is effective when tested on mice, we cannot conclude that the treatment is just as effective on humans without additional testing.
We won't be required to perform any calculations for these problems. Instead, we'll be asked to read fairly lengthy descriptions and then make logical observations or draw valid conclusions.
In this lesson, we'll learn to:
  1. Recognize good and bad sampling methods
  2. Draw valid conclusions from the results of surveys and experiments
You can learn anything. Let's do this!

What are some good and bad sampling methods?

Reasonable samples

Khan Academy video wrapper
Reasonable samplesSee video transcript

Examples of bias in surveys

Khan Academy video wrapper
Examples of bias in surveysSee video transcript

Example of "undercoverage" bias

Khan Academy video wrapper
Example of undercoverage introducing biasSee video transcript

Sampling methods and their implications

Ideally, a
provides information about a
without having to survey the entire group.
To make valid conclusions about a population, we need a sample that recreates the characteristics of the entire population on a smaller scale.
A good sample is representative and random.
  • Representative means that the sample includes only members of the population being studied.
  • Random means that every member of the population being studied has an equal chance to be selected for the sample.
Bad sampling methods include those that:
  • Gather data from outside the population being studied
  • Gather data that overrepresent or underrepresent a subgroup of the population (not random)

Try it!

try: identify flaws in sampling method
A school district employs 2,000 teachers in its 40 elementary, middle, and high schools. A high school teacher working for the district believes that teacher job satisfaction varies greatly from school to school and wants to estimate the proportion of all teachers in the district who are satisfied with their job.
Match each of the following sampling methods to the reason it's flawed.
1

What are some different types of studies, and what conclusions can we draw from the results?

Types of statistical studies

Khan Academy video wrapper
Types of statistical studiesSee video transcript

Correlation and causality

Khan Academy video wrapper
Correlation and causalitySee video transcript

Identifying study types

Khan Academy video wrapper
Worked example identifying experimentSee video transcript

Drawing conclusions from study results

Sample surveys

We can draw conclusions about only the population from which the random sample was selected.

Controlled experiments

To understand the conclusions we can draw from controlled experiments, we must first understand the difference between correlation and causation.
  • Correlation means there is a relationship or pattern between the values of two variables.
  • Causation means that one event causes another event to occur.
You may have learned about controlled experiments and the scientific method in more detail in your science classes. For the SAT, the key takeaway is that a
is needed to establish a causal relationship.

Try it!

try: extend sample results to a population
A youth activist group surveyed a random sample of 500 teenagers between the ages of 16 and 17 in the U.S. to assess their opinions about lowering the voting age to 16. The survey showed that the majority of those sampled were in favor of lowering the voting age to 16. Based on the results, the majority of which of the following populations in the U.S. are most likely in favor of lowering the voting age to 16 ?
Choose 1 answer:

try: determine whether a study establishes a causal relationship
An ice cream company randomly selected 100 participants to test their new line of ice cream. For the study, each participant was given a pint of the new ice cream. The results showed that 98% of the participants reported enhanced moods after consuming the ice cream.
Did the study have a control group?
Do the results of the study show that consuming the new ice cream is associated with reports of enhanced moods in the study participants?
Do the results of the study show that the new ice cream is more effective at enhancing the moods of people than other ice cream brands?

Your turn!

Practice: draw conclusion from sample data
A study was done on the lengths of frogs in a pond. A random sample of frogs were caught and tagged in order to ensure that none were measured more than once. The sample contained 50 American bullfrogs, of which 40% were shorter than 7 inches. Which of the following conclusions is best supported by the sample data?
Choose 1 answer:

Practice: draw conclusion from research results
A study was conducted to determine if a new treatment is successful in treating
. 500 participants were selected at random from a large population of people with insomnia. Half of the participants were randomly assigned to receive the treatment, and the other half did not receive the treatment. The resulting data showed that participants who received the treatment slept significantly better than those who did not. Based on the design and results of the study, which of the following in an appropriate conclusion?
Choose 1 answer:

Want to join the conversation?

Log in