3-30-12 Conducting a reliable survey

A lot of non-reliable information is being passed off as a “survey.”  With Survey Monkey and other simple ways of sending out questionnaires available on line, it’s easy to believe that if you just send out a survey on the internet, it will produce reliable information.

However, to be a “random sample” survey with a measurable degree of accuracy, the sample must be pre-selected using a random number system.  If someone chooses to respond to an on-line survey (like Survey Monkey), by definition this is not a statistically valid process.

Telephone interview surveys are still considered to be the most cost-effective way of getting statistically valid information.  There is a margin of error, based upon the size of the sample (generally, a minimum sample of 200 is the industry standard for reliable data about any population segment).  Overall, random sample telephone interview surveys provide reasonably accurate information about the population from which the sample is drawn.

With the advent of cell phones, I became concerned about what this means for the traditional method of doing random sample surveys.  I had a conversation a few days ago with Don Morgan, the President of GMA Research, the company that has done most of the data collection for my surveys for several years.  It turns out that a survey firm, like GMA Research, can now purchase cell phone lists in the same way they have purchased land-line telephone lists, from which the random samples are drawn.  Although some do have cell phones with a prefix that is different from where they reside, most people have cell phones that have been issued in their region.