If you're seeing this message, it means we're having trouble loading external resources on our website.

If you're behind a web filter, please make sure that the domains ***.kastatic.org** and ***.kasandbox.org** are unblocked.

Main content

Current time:0:00Total duration:8:34

in this video we're going to think about what makes a poll or a survey credible because remember the whole reason why we're going to do a poll or survey is we want to understand public opinion but if it's not statistically credible if we can't believe what it's saying if we don't understand exactly what it's saying then we might not have a good sense of what public opinion actually is so the first really important thing if you're running any type of survey whether it's in politics or government or not is that you are taking a random random sample what does that mean well we go into a lot more depth in it in our statistics content on Khan Academy but it means that you should look at the people the voting population and it does get a little bit nuanced and tricky on who are likely voters but you would go to say the voting population or the people in a in an area whose opinion you care about and you would want any one of them to have an equal shot of being selected for your survey so good examples of random samples is maybe you take you have a random phone number generator of likely voters in your district and you call them up maybe and even then you might say well hey maybe certain types of people pick up the phone and certain people don't or certain people have a phone certain people don't so you have to be very careful about this design but examples of non random sample would be good to go hang outside of the Democratic Party headquarters in your district and just survey people coming out of that building that would be very not random and you would get very skewed results now the next thing that you would want when you are taking a survey is that you want a large population size now typically speaking it's going to be at least 500 folks but you're going to see a lot of surveys that are about 500 to a thousand participants and sometimes in statistics I'll say you're n which is the number of people you surveyed is 500 to a thousand and this is so you have a good chance of getting close to the true public opinion we're going to talk in a second about margin of error the larger this is and especially if you're doing a true random sample this is so larger larger it reduces the you'll have a lower margin margin of error and I'll explain in depth what a margin of error is in a second now another key thing is whatever you asked in your survey it needs to be the language needs to be as neutral and unbiased as possible let's say there's a new proposition you could get very different results if the wording of the question is do support funds for those in need or if you said do you support funds for those who are not working those could get very very different results and this is there's really an art to trying to get a neutral unbiased question there last but not least you want to make it very transparent to the public how you conducted your poll so that they can decide for themselves how credible your poll actually is so with that out of the way let's look at real poll results and then I'm going to dig into what the idea of margin of error is because you'll see this a lot when you read the news so this is from the sun-sentinel in Florida they have a senatorial race going on and it says despite millions of dollars in television ad spending Florida's US Senate race between Bill Nelson and Rick Scott hasn't budged a Florida Atlantic University poll shows Republican Scott with 44% of voters surveyed Democrat Nelson with 40% so the first thing you might say is well what about this only adds up to 84% what about the other 16% well those could be undecided voters or maybe there's a third candidate there that's a four-point advantage in Scott's favor but it's within the surveys margin of error which means the race could be tied or Scott could have a lead the FAU Business and Economics polling initiative survey of 800 Florida registered voters was conducted online and through automated calls to people with landline telephones researchers said it had a margin of error of plus or minus three percentage point so let's think about whether this is a credible poll and what inferences we can make from it so the first thing is were they transparent about their methods and it seems like there were this whole this paragraph right over here they're very transparent about their methods how many people did they survey they say how they actually conducted the survey and so check on transparency and then because they were transparent we can check whether these other things are true so first random sample they don't go into a lot of depth on how it was conducted online but if it is a neutral site where any Florida voter is equally likely to show up well then that might be a good random sample now you have to be careful with online because who has access to online who does not have access to online who might go to whatever site it is being conducted on so this this has a couple of question marks right over here and then they say through automated calls to people with landline telephones so once again this seems reasonably random it's along the lines that I talked about earlier maybe they had some random phone number picker in Florida and they call those folks but maybe you know there's certain people who have landlines a lot of people now only have mobile phones there's certain people who pick up and might answer things and certain people who might not so as you can see there's an art to getting a truly random sample but it you can tell that the Florida Atlantic University group tried to get a random sample the next was the large population size well they said a survey of 800 Florida voters so that's pretty good and it's not just 800 for 800 sake it's the number of voters that's gonna drive the margin of error so this is a good time to say what is the margin of error so a margin of error is going to be associated with a confidence level it's typically going to be a 95% confidence level if they don't tell you otherwise and so what this means is is that 95% of the time when you take a random sample of 800 Florida residents you will get a result that is within three percentage points of the true result remember there's some true result we don't know unless we could perfectly get into everyone's head and this poll is a way of trying to estimate that so one way to think about it within three percent of what you got Republican Scott is really you could create a confidence interval where you could say the confidence interval is going to be three percent less than this which is 41 percent all the way to three percent more than this which is forty-seven percent and one way to think about it is there's a 95% chance that this will overlap with the true result and you might say hey well the low-end of this range is still higher than Democrat Nelson but that confidence interval applies to Democrat Nelson as well so Democrat Nelson's confidence interval would be between thirty-seven percent three percent less than this and forty three percent three percent more than this there is still a reasonable chance maybe the Democrats at the higher end of this range which could still mean that the Democrat could win even though the headline numbers show a four-point advantage for the Republican and outside of the statistical error the margin of error we're talking about you have to remember that this poll is before the actual election people change their minds there's a whole campaign going on so even if you were to get the exact result this is just a snapshot in time it could change on election day now this last question where they asked a neutral unbiased question we don't know exactly at least just from this article that they might have published it in the details of the poll but if they ask something like who will you vote for on the election on this day that would be very neutral and unbiased but if they say are you going to vote for the public servant who has served Florida for many years or that Democrat guy well that would be not unbiased

AP® is a registered trademark of the College Board, which has not reviewed this resource.