If you want your research to be statistically reliable — meaning that you can depend on it to accurately represent a particular audience, guide your strategic direction, etc. — you need to have a certain number of respondents.
So the simple answer to this question is 400. But of course, that’s not the only answer.
Without going into the math, if you’re looking at a large population (like college-bound students, or people who own an RV, or companies who buy printer paper), then 400 respondents will give you a margin of error under 5 percentage points at a 95% level of confidence.
What that means is that if you ran the survey again and again with a similar group of respondents, 95% of the time you’d end up with results that were within 5 percentage points of the ones you got the first time. That’s a well-accepted level of statistical reliability.
My sister Sarah is the statistical brains behind our work at Audience Audit, and she puts it this way: The spoonful of soup you’ve tasted is enough to know what any spoonful from the pot is going to taste like.
But of course, the answer isn’t always just 400.
If you’re in a B2B situation where there are only 200 companies that use the type of product you sell (like some kind of scientific testing equipment, for example) then if you get survey responses from 135 of them you’re within that same error margin of 5 percentage points at 95% confidence. But it may be harder to get 68% of the potential respondents to participate in your survey in this case than it is for a B2C company to get 400 respondents out of the millions of qualified respondents out there.
And even if you have a large population, if you know you want statistical reliability even when you’re just looking at a portion of the respondents (such as GenX participants, for example) then you should plan on getting at least 400 of those so you can compare their demographics, preferences and other feedback and still have statistically reliable results.
And if you’re conducting research to share as thought leadership, having 1,000 respondents may be more likely to gain you speaking engagements or press coverage than if you have 400, because it sounds like a much more impressive number (even if you really only need 400).
So the starting point is 400, but your situation and goals may require fewer respondents or more. Talk to your research partner about what you are hoping to accomplish and how your respondent strategy should work to help you achieve that goal.
