How Large Does My Sample Need to Be?
Sample size is an important consideration when you are planning your research project. For basic inferential purposes, or the ability to generalize the findings in your sample to the population it came from, the sample should have at least 30 participants. This number is based on the Central Limit Theorem, which is the rule that governs the normal curve. Among other things, this theorem says that as long as samples from a population are at least 30, the average of those samples will be the same as the population average. In other words, if your sample is at least 30, it will be like the population you get it from. This is important because it saves us from having to conduct our research on an entire population to get trustworthy outcomes. But making sure our sample is like our population is only the beginning. We also have to consider two other things: power and effect size.
Without going too deep into either of these (I’ll do that in later posts), power is simply the ability of your statistical test to find significant results if they are there. If the sample size is too small, the test won’t be powerful enough to find the results you are looking for. Effect size is the strength of the effect of your treatment. For example, suppose the fictitious dentist in a previous post found that making reminder calls to his patients really was effective in having people show up for their appointments. The phone calls would be his “treatment”, and even though we find that they work, he would also want to know how well they work. This is effect size. The effect size is what it is, and can’t be changed by sample size, but a larger sample can give enough power to the statistical test to find not only significance (if it is there) but also a smaller effect size. With smaller samples, we might only be able to see an effect if it is big. In short, larger sample sizes are better for both power to detect significance, and power to detect even smaller effects.
So how large does your sample need to be? It depends on the type of test you will use, and the effect size you want. If you are happy with finding only a medium effect size (the typical choice), you can get by with a smaller sample than if you wanted to be able to detect a small effect. Though the formulas for calculating needed sample size are usually pretty complicated, there are online sample-size calculators you can use. One popular one can be found at: www.danielsoper.com . On that website, choose “statistics calculators”, then simply scroll down the page to “sample size calculators” and fill in the blanks. Once you know your required sample size, try to shoot for a sample that is considerably larger to allow for non-respondents. How many people can you expect to respond? That is a hard question to answer. In 2008, Manfreda and colleagues1 estimated that the response rate for online surveys was about 11%, and was lower than that of any other method (surveys mailed to people’s homes, for example). But there are many factors that affect response rates, and with enough diligence in using follow-up contact methods you can get enough people to respond to make your research results useful. So don’t let Manfreda’s estimation scare you away!
1. Manfreda Katja, Lozar, Michael Bosnjak, Jernej Berzelak, Iris Haas, and Vasja Vehovar (2008). ‘‘Web Surveys Versus Other Survey Modes: A Meta-Analysis Comparing Response Rates.’’ International Journal of Market Research 50:79–104.