Sample | Methods | Response Rates

Ten Steps to Better Decisions

Our whole approach -- market research at market speed -- centers on speeding up the cycle of information that informs our clients' decisions. This means executing the market intelligence loop at a faster pace than competitors, seeing what is important before others do, and automating a proven step-by-step research process.

High confidence quality results demand a speedy but rigorous step-by-step process. Do-it-yourself research is especially vulnerable to having data rendered useless, in part or whole, because a step was left out, or ignored in the research process. Here are the steps we adhere to for Wilson Research Group projects:

1. Define goals, sample, method, response rates, costs and schedule
2. Sample development and preparation
3. Questionnaire design and development

     Clear questions
     Clear instructions
     Well organized sequence of questions
     Respondent consideration (length, time, non-forced answers, etc.)

4. Online survey programming and special formatting
5. Survey testing in "live" online format, and data file testing
6. Invitation design and development, list management, and tracking as needed
7. Survey fielding with early and ongoing monitoring
8. Data cumulation and data cleaning
9. Online Reporting and Analysis

     Early live results and monitoring of respondents
     Interactive ExecStats reports
     Online crosstabulations
     Online Verbatim by Segment reports

10. Offline Reporting and Analyses
     Executive training
     WinCross tabulation reports
     SPSS statistical package reports
     Excel exports and reports
     Case Example reports
     Verbatim /Open end reports

Sample

Sampling is the single most important component of high confidence market research. A sample must represent the larger target audience (download our Sample Size and Confidence Interval calculator to estimate your sample requirements). Random or scientifically selected samples are the best for achieving high confidence.

Samples come from customer lists, industry SIC code lists, online panels, magazine subscriber lists, website visitor lists, government sources, newsletter subscriptions, privately built groups, and many other sources.

  • Customer samples are the best source for understanding your customers' needs and opinions.
  • Panel samples have the advantage of predetermined availability and predictability of response rates.
  • Live samples, such as conference attendees, Web-site visitors, or technical support callers can be "intercepted" and provided an opportunity to give feedback on their experiences on a continued basis providing long term audience intelligence.
  • Convenience samples are lists of people who are conveniently available, who are members of your target group, but may not be fully representative of the entire group.
  • Industry samples can be derived from generally available lists of various market segments. How well these lists represent the industry varies as to how the lists were constructed.
  • Methods

    Determining how best to reach the respondent is a key step in the process and determines the methodology you use. Some audiences can only be reached through certain channels.

  • Online Surveys: If your respondents are 100% available online, this is the fastest and least expensive methodology.
  • Telephone Surveys: Handy for more general consumer or general population surveys. Most people can be reached by phone, but not all, and barriers to reaching your target audience by phone are increasing.
  • Panel Surveys: Panels are effective if the panel developer is reputable and employs proven methods to ensure against "professional" survey takers, "straight liners", and such. Panels can reach audiences not available any other way. However, panels are often even more costly than telephone surveys, because of the time and incentives it takes to build and maintain them.
  • Paper Surveys: Sometimes it is easiest and most appropriate to use a simple paper survey. Paper surveys take longer and involve paper, envelopes, printing, and postage costs. Although less used today, this method is true and tried and still yields high value results.
  • Combination: Sometimes a combination of the above methods works well. A postcard or telephone invitation to take an online survey is a method we sometimes recommend. A follow-up postcard as a reminder, instead of an email has certain advantages. An email reminder to a paper survey also works.

Response Rates

Another key to having high-confidence surveys is the response rate.

Many strategies (and incentives) for achieving a high response rate are available. Success in obtaining high response rates revolves around a few principles:

  • Assuring the respondent you will read, look through and use their feedback
  • Making sure the survey is designed with the respondent in mind (clear questions, clear instructions, interesting questions, graphics, good formatting, clear emphases, automated )
  • Incentives are small inducements or rewards for participating
  • Disclosing the full purpose and reasons for the research
  • Attention given to every detail in the process
  • Understanding the respondents' interests and motivations
  • Placing trust in the respondent
  • Letting the respondent know the consequences of the research

It takes only about 384 randomly selected respondents to project to a universe of 1 million people at 95% +/- 5% confidence (a rule of thumb standard). This is why sampling is a cost-efficient and highly productive way to learn about broad characteristics of a population.