It is often said that the most accurate poll is an election and that everything else is only a guess, with varying degrees of accuracy. However, through the science of statistical analysis, we can come very close to estimating a result with a relatively small sample size. For example, using a sampling calculator like the one found here, we can see that by surveying 1,067 people out of a population of 2 million, we can be 95% positive that their answers reflect the entire population (within a variance plus or minus of 2%).
So, how many people would we need to poll in order to get an accurate read of a local race of some 8,000 electors? Unfortunately, despite the smaller number of electors in this case, the sample still needs to be relatively large to achieve a similar level or accuracy. Given an electorate of 8,000, a sample of 942 people provides a +/- of 2% error rate – which is sufficient for our purposes.
Unfortunately, a professional poll of 20 questions to a sample size of 700-900 people, which tests a number of very subtle preferences, can cost upwards of $25,000. This would, of course, destroy a budget for a mid-size race at the municipal level.
Something has to give. A smart fellow once said that everything is a function of time, money and quality – so if we don’t have unlimited time or money, it follows that we will have to sacrifice some quality.
But, is that a problem? For the purposes of testing a message, or just getting a rough idea of the public perception on a candidate or hot button issue, we don’t always need to be within a 2% accuracy – especially if this is for self-consumption, and filtered through the prism of your own experiences.
A sample size of 250 people on a population of the same 8,000 people would be accurate within 6% (using the same calculator as before). This can still be very effective in getting a general sense of where people prioritize key issues, or in testing name recognition of a particular candidate. For example, if the name being tested is found to be 20 points ahead of the nearest competitor, then the error rate means a lot less; while still giving you a general sense of the candidate’s popularity.
There are also considerations in the methods used to collect the data. The strongest and most widely accepted method is by calling people at home, and verifying key demographic information to ensure the sample is statistically balanced to represent the public. For example, you need to ensure you have enough representation from people on the basis of geography, income levels, age and gender.
Moreover, a professional polling project that is statistically accurate, and considered scientific, will often throw out a large number of the data elements they collect in order to “balance” the sample set. If they have interviewed too many males they simply ignore a certain set of male data to help balance it out in accordance with the population size and demographic.
Within the industry, pollsters suffer through an endless struggle over the science of just how accurate each other’s methods can be. It is clear that the questions over the accuracy of different polls will continue to be debated; however, when you are in the privacy of your own home, and deciding whether or not to launch your campaign, the most important question is whether or not you have a chance at winning. Does it hurt to run a more affordable poll using methods which might blur the results slightly, and garner less than 5% accuracy, if it helps determine you are “in the running” or “not on the radar”?
When running a campaign, ANY extra information you can glean about your chances in your chosen seat is an essential part of your campaign arsenal. There are plenty of affordable ways to get polls done that can help you achieve a confidence level you can live with.