If you're seeing this message, it means we're having trouble loading external resources on our website.

If you're behind a web filter, please make sure that the domains *.kastatic.org and *.kasandbox.org are unblocked.

Main content

Introduction to experiment design

Explanatory and response variables. Random sampling and random assignment (including block design). Placebos and blind/double-blind experiments.

Want to join the conversation?

  • aqualine ultimate style avatar for user Isaac
    At , Sal mentions that you can use "block design" to randomly divide patients with common properties, such as sex, evenly throughout the control and experiment groups. Isn't that the same as using stratified sampling in a sample study?
    (7 votes)
    Default Khan Academy avatar avatar for user
  • blobby green style avatar for user jnapireagri
    how to control error using blocking
    (4 votes)
    Default Khan Academy avatar avatar for user
    • aqualine ultimate style avatar for user Astero
      You use blocking to minimize the potential variables(also known as extraneous variables) from influencing your experimental result.

      Let's use the experiment example that Mr.Khan used in the video. To verify the effect of the pill, we need to make sure that the person's gender, health, or other personal traits don't affect the result. We want to test if the pill would be effective for everyone in general. This is where blocking method comes in; we need to divide the samples into two groups in a way that each group is similar to each other in terms of aforementioned factors(gender, etc.). From this, you can ensure that the result will be affected minimum by the unexpected factors, hence controlling the error.
      (6 votes)
  • duskpin ultimate style avatar for user Igor
    Is there a reason the KA video about correlation of A1C with blood sugar can't be found on this site anymore?
    (It is still present on YouTube though)
    https://www.youtube.com/watch?v=MOH33-jFOwo
    (5 votes)
    Default Khan Academy avatar avatar for user
  • blobby green style avatar for user Mir Vg
    How could you confirm your experiment is replicable?, do you have to do another experiment with different populations or just mention on the potential paper that it is necessary to do more experiments considering other samples?
    (3 votes)
    Default Khan Academy avatar avatar for user
    • starky seedling style avatar for user deka
      it is usual that other researchers who have no direct interests in and are independent from one study would do the similar experiments, following the procedures and parameters published in the original paper.

      then they check if their results would be compatible with the proposed ones.

      and as you said, it's almost impossible to do exactly the same experiments with the same subjects again, espeically when it comes to the clinical cases like this. thus the purpose of replicating an experiment is not in duplicating the experiment and getting the exact same data itself, but in confirming general trends with similar conditions not by the authors of the paper but by the others (usually peer researchers).
      (4 votes)
  • blobby green style avatar for user MaKala Bradley
    At I start getting confused
    (4 votes)
    Default Khan Academy avatar avatar for user
  • marcimus pink style avatar for user Dhaliwal, Sharon
    What is the difference between block design and stratified sampling?
    (3 votes)
    Default Khan Academy avatar avatar for user
    • duskpin ultimate style avatar for user Rod Redline
      Block design is when you have a sample and you, out of chance, have two or more categories that the individuals belong to, and then you have to divide those evenly into the control and treatment groups. Stratified sampling is when you, ahead of time, purposefully pick a fixed number from each category.
      Mainly, the difference is that stratified sampling is on purpose, while block design happens because it was random.
      (1 vote)
  • blobby green style avatar for user cedraper19
    whats the difference between block and strat
    (1 vote)
    Default Khan Academy avatar avatar for user
  • scuttlebug blue style avatar for user kadynce
    how to control error using blocking
    (1 vote)
    Default Khan Academy avatar avatar for user
  • blobby green style avatar for user Keylaa
    Is it really important to make a graph and if so why.
    (1 vote)
    Default Khan Academy avatar avatar for user

Video transcript

- [Instructor] Let's say that we've come up with a new pill that we think has a good chance of helping people with diabetes control their blood sugar. When someone has diabetes, their blood sugars unusually high, that damages their body in a bunch of different ways. So we want to conduct an experiment to test if this pill really can help people lower their blood sugar. So the first thing we need to think about is how do we even measure or test whether peoples' blood sugar is getting lower. Well, for our experiment, what's typically done is we measure folks hemoglobin A1C. You don't have to worry too much about this in the context of statistics. But hemoglobin A1C test is a way that's typically used to measure your average blood sugar over the last three months, and we have whole videos on Khan Academy explaining how that works. So our hope would be that our pill lowers peoples' blood sugar which shows up as a lowered A1C. Now we have terms for this. The thing that is causing something else to change, we call this the explanatory variable. Explanatory variable, and the thing that might get changed by that explanatory variable, depending on whether you take the pill or not, we call that our response. Response variable. So now let's actually conduct the experiment. So what we would do is we would go to the population. Population of diabetics, and we would want to take a random sample from that population of diabetics. A reasonably large one, and later in statistics we talk about what a good size sample might be. But let's say that we randomly sample. Randomly sample 100 folks. So we randomly sample 100 folks from that population of diabetics, and then you would want to assign these folks randomly to two different groups. One would be your control group, and this would be the group of people who won't take the new medicine, and then you would have your treatment group. These are the groups of folks who will be given the new medicine. The treatment group. Now in some cases, you can just randomly assign these 100 folks between these two groups, and one way to do it is you could give all of them a random number between one and 100 and then the top 50 go into treatment and the bottom 50 go into the control, or there's, you can use a computer to randomly assign folks. Now sometimes you might want to be a little bit more sophisticated than that. For example, there might be evidence that someone's sex might somehow influence how they respond to a drug. So what you could do is something called block design where, let's say, this group just happens to have 60 females and 40 males. One block design. You can randomly assign, but you can do it in a way that you can ensure that both of these groups have the same proportions of male and females. So for example, if you have 60 females here, you can ensure that 30 of them end up in the control, and 30 end up in the treatment. But you would assign those, those 60 females randomly between these two groups, and similarly, you can do block design of these 40 males, 20 end up in the control, and 20 end up in the treatment. So once you have folks in both of these groups, what you would probably want to do is measure their A1C at the beginning. You can view that as a baseline, and then, over the course of the experiment, you would give the pill to the treatment group, and in the control group, you might be saying, well, we would just wouldn't do anything. But the best practice is actually to give a pill that looks just like the real thing to the control group. This is known as a placebo, and the reason why we do that is there's definitely evidence that when people think they are taking a pill that might help them, that even psychologically it can have an effect on them, and sometimes it helps them. This is known as the placebo effect, and not only would you give both groups a pill that looks the same, even though this one in the treatment group actually has the medicine in it, you also would not want to tell folks which group they are in. When you don't tell them which group they're in, that's known as a blind experiment, and you probably also don't want to tell the people who are administering the experiment which group they are administering, and that's called a double blind. So even the doctors or the nurses that are administering the experiment, when they're giving a pill to the control group, they don't know that that pill is the placebo, and you might say, well, why is it important for an experiment to be blind, or especially double blind. Well, that avoids, one, any type of psychological effect on the, from the point of the patient. Or from the, say the caregivers in this situation, so that they don't kind of give it away. They don't tell these folks, hey, you're actually just pretending to take a pill, and so that ensures that we minimize the amount of influence or bias that might happen. You might even have a triple blind experiment where even the folks who are analyzing the eventual data from this experiment don't know whether they're analyzing the data from the control or the treatment. They just compare the two different groups. But anyway, you do, you people take the medicine and the placebo over the course of the experiment. Maybe this lasts for three months, and then you would want to measure their A1C later, and then you would see their change in the A1C. Now if you saw that there wasn't really a difference in the change in A1C between the control and the treatment group, then you'd say, well, that probably means that my pill didn't work. Now if you do get a greater reduction in the treatment group, and you do the statistical analysis which we will learn in statistics and you show that, hey, there's a very low probability that's happened purely due to chance. Well then, you've got something. You could probably conclude that there is a causal connection between taking the pill and lowering your A1C level. But once again, you cannot be 100% sure, and so, this is why it's very important for people to be able to replicate your experiment. Because what you'd want to do either yourself or other researchers might want to conduct the experiment with different sample sizes and different countries and different populations, maybe with different ages at different times of year to ensure that they continue to see this result.