Main content
Course: Integrated math 3 > Unit 10
Lesson 2: Observational studies and experimentsAppropriate statistical study example
Sal determines if a statistical study was a sample study, an experiment, or an observational study. Created by Sal Khan.
Want to join the conversation?
I feel like this question is set up to trick the user. In the first paragraph, Alma does an EXPERIMENT (but does not perform the experiment properly since there is no control group). Then in the second paragraph she takes a SAMPLE of the petri dish. So there are actually two types of study being used.
Then Alma samples 300 bacteria from the dish, but how do we know how many were in the dish to begin with? I would guess billions of bacteria, but there's no indication of how many there were in total.
Very confusing(13 votes)
You're right. It WAS an experiment as well as sample study. Sal should correct this video. Experiment is a broad term and there are many different types of experiments. The experiments he was talking about are only one specific type of experiment. A good reliable type but not the only kind.(6 votes)
What is Simpson's Paradox?(4 votes)
It is the apparent paradox you get when comparing two groups and a trend you see between the two groups over a set of samples disappears or often reverses when all the data is combined.
For example, if you look at batting averages (example borrowed from wikipedia) you see that although David Justice had a higher batting average than Derek Jeter in both 1995 and 1996, Jeter actually has the higher average over the combined two years.
...........................1995............1996...............Combined
Derek Jeter 12/48 .250 183/582 .314 195/630 .310
David Justice 104/411 .253 45/140 .321 149/551 .270(7 votes)
This is an explanation of one of the answers to the questions in types of
statistical studies:
"Even though experiments suggest causation, it would be too far-reaching to conclude that "meditation reduces work stress," which implies that meditation would reduce work stress for anyone who practiced it"
I don't understand why this conclusion is not allowed(3 votes)
I'm not really sure why your answer was wrong. If it was a randomized experiment, then concluding causality is generally acceptable. My only thought is that perhaps one of the other answers was slightly different.
For some comparison, I went through a number of the questions, and at times for a "randomized experiment" type of question there were two answers that were very similar, but one was a bit more general in terms of how they phrased the results. For instance, some of the questions were about "7-11 year old children" eating more food when watching TV commercials featuring snacks. Two of the answers had the proper conclusion, but one of them phrased it in terms of 7-11 year old children, while the other phrased it more broadly (I think it was all children, or all people).
In your comment you didn't fully clarify the question and answers. Could it be that the question described a more specific population (e.g., adults, or adult men, etc.)? The conclusion of our study can only apply to the same population from which the sample was drawn.
Without seeing the exact text of the question and answers (I looked a bit through the questions for it), I'm not sure that I could provide a better answer.(2 votes)
What's the difference between a sample study and an observational study ?(1 vote)
in which exposure and outcome are determined simultaneously for each subject.(1 vote)
- This is some explanation from the questions in types of statistical studies:
"Even though experiments suggest causation, it would be too far-reaching to conclude that "meditation reduces work stress," which implies that meditation would reduce work stress for anyone who practiced it."
bold Why can't you that "meditation reduces work stress"(1 vote)- The statement "meditation reduces work stress" implies a causal relationship between meditation and reduced work stress for anyone who practices it. However, while experiments can suggest causation, observational studies cannot establish causation. Even if a correlation is found between meditation and reduced work stress in an observational study, it doesn't necessarily mean that meditation directly causes the reduction in work stress. There could be other confounding variables at play.(1 vote)
- Couldn't we describe it as a faulty experiment ? Sal's seemed to do so , when he was answering the second question(1 vote)
- Yes, Sal's comment suggests that the study could be considered a faulty experiment because it lacked a control group. Without a control group, it's difficult to attribute the observed effects solely to Alma's antibiotic, as there could be other factors influencing the outcomes.(1 vote)
- If she is finding a correlation between the effectiveness of the bacteria and the death percentage isn't this study an observational study?(1 vote)
- While Alma's study does involve observing the effectiveness of her antibiotic, it is not an observational study in the traditional sense. Observational studies typically involve passive observation of variables without intervention. Alma's study involves actively applying her antibiotic and measuring its effects, making it more akin to an experimental study.(1 vote)
Hi!
In what cases should an observational study or a sample study be performed instead of an experiment?(1 vote)
1. ethical reasons
some studies (many in medical senses) cannot be allowed to perform to human subjects. thus observational study could and should be the next best option. but in some cases, there aren't enough observational data yet. then you could do sample studies to get enough data for each of variables you're curious of if which have a correlation. and then move into the observational study
2. physical impossibility
if you were interested in the relationship between the aging rate of stars in the universe and the aging rate of living organisms on the planets around them, you better rely on both types of study (observational and sample). because first, you can't experiment with the whole universe due to its sheer scale. second, you can't control the aging of the stars or the organisms in either way. and the third reason for this study to have to rely on some type of sampling is same as above, you may have not enough datapoints (especially for the aging rate of living organism part cause we have one and only data on this very earth yet)
in short, some experiments can be performed but won't be for the ethical reason. while a few cannot be performed physically in the first place
q. i know the aging rate example sounds so unrealistic that gives you some perception that most realistic experiments could be performed in a physical manner. if you do feel this way, what would be a realistic case in which we literally can't perform an experiment thus have to rely on the other options like observations and samplings?(1 vote)
- In this case, a more appropriate experiment would also include a control group on a accepted antibiotic...(1 vote)
- Yes, a more appropriate experiment would include a control group receiving an accepted antibiotic to compare the effectiveness of Alma's antibiotic against a known standard. This would help determine whether Alma's antibiotic is more effective than existing treatments.(1 vote)
- You can determine a sample study by someone's confidence level. You can determine a experiment by measuring a centrality, such as the mean, median, or proportion; and a measure of variability, such as the standard deviation. You can determine observational study by having researches observe the effect of a risk factor, diagnostic test, treatment or other intervention without trying to change who is or isn't exposed to it.(1 vote)
- Determining the type of study (sample study, experiment, or observational study) involves considering the research design and methodology rather than someone's confidence level. In experiments, researchers manipulate variables and measure outcomes to establish causation. Sample studies involve estimating population parameters based on sample data. Observational studies involve observing the effects of variables without intervention. These distinctions are based on the nature of the research process rather than specific statistical measures.(1 vote)
Video transcript
Voiceover:Alma has developed a new kind of antibiotic. For the antibiotic to be
sufficiently effective, it has to kill at least 90% of bacteria when applied to a
harmful bacteria culture. She applied her antibiotic to a petri dish full of harmful bacteria, waited for it to take effect, and then tried to estimate the percentage of dead bacteria in it. She took a random sample of 300 bacteria and found that 94% of them were dead. Then she calculated the margin of error and found that the true percentage of dead bacteria is most
likely to be above 90%. So, what's happening over here, she's trying to figure out what percentage of the total population of bacteria died. Maybe there's something
about this bacteria, maybe when you look at it from, you know, the naked eye, you can't tell whether the bacteria died or not. So, she decides to estimate
the true percentage by sampling 300 individual bacterium, or by sampling ... I always forget the singular case. By sampling 300 bacteria, and then in her sample she found that 94% of them were dead. Then the margin of error tells us, because the margin of
error says it's unlikely, or that it's very likely that the true percentage is above 90%. That means that given that you sample 300 bacteria, it's very unlikely that
the true percentage is below 90%. So, she could feel reasonably confident that in her petri dish, more than 90% of the population did indeed die. Now, let's answer these questions. What type of statistical
study did Alma use? Well, she used a ... she's trying to estimate a parameter for population, in this case, the parameter was the percentage of all of the bacteria that died. She couldn't observe that directly, so instead, she took a random sample of the bacteria in the petri dish, and she used ... she calculated the static for them, 94% of them were dead, and that's her estimate for the population parameter. The percentage of the
population that died. So this is ... when you're using a ... when you're using a random sample to generate a statistic, which estimates a
parameter for a population, that's a sample study. So, she ran a sample study. Now the next question is. Is the study appropriate
for the statistical question it's supposed to answer? So what was the question that she's trying to answer? Well, at least the was it's written, it seems like she's trying to answer whether or not her antibiotic works, whether it's an effective antibiotic, whether it's capable of killing bacteria. You might be tempted to
say, "Okay, well look. It looks like it killed ... it killed more than 90% of the bacteria, or very likely it killed more than 90% of the bacteria given ...
given the sample size, and the margin error and all that." Even if it is indeed the case, that 95% of all the bacteria died, it doesn't necessarily
mean that it was caused by the antibiotic, maybe it was caused by the plastic in the petri dish. Maybe the air in the petri dish was too cold or went bad, or maybe it was handled in a weird way. Or, maybe that bacteria was just a bad ... a bad culture, and it somehow it just spontaneously died on its own. She can't say with confidence that it was definitely ... it was definitely the antibiotic. In order for her to make that statement, she would have to run a proper experiment. She would have to have a control and a treatment group, where everything is equal except for the treatment group has the treatment. So if she had 2 petri dishes that were kept in the same conditions with the same lightning, the same air, the same material that the bacteria is growing on, everything the same, except for the treatment group, has the antibiotic applied to it. Then she saw that in the treatment group that most of the bacteria died while in the control group, most of the bacteria didn't die, then she could say, "Okay. It looks like the antibiotic caused the bacteria to die. That there was actual causality here." So, she would have had
to run an experiment. The most appropriate statistical study, or the most appropriate study would have been a proper
controlled experiment. Where you have a control group, where they don't have the antibiotic, and a treatment group, where they do have the antibiotic. Let's see what are the choices here. Where they say is the study appropriate? So, yes because she is appropriate study. No, I don't like that answer. No, because she can't know for certain that the true percentage of dead bacteria is above 90%. Well, I'm not going to click on that, because even if she knew for certain that the true percentage of the dead bacteria were 95%, she can't feel confident that it was due to the antibiotic. Once again, it could be caused by the air conditioner. It could have been
caused by the petri dish. It could have been caused by the lighting in the room. So, no, because the study didn't have a treatment and a control group. Yeah, I would go with
that one right over there. Yes, because she found that the antibiotic killed more than 90% of harmful bacteria. Once again, even if she knew for sure that more than 90% of the population had been killed, she doesn't know that it was caused by the antibiotic. It could have been caused by a whole bunch of things. If she had a controlled group that had the exact same conditions and the bacteria didn't die, then she could feel better that it was the bacteria death was due to the antibiotic.