Quality - What's Love Got To Do With It?

I attended the Marketing Research Association's Annual Conference last week in New York City. The MRA Conference is a great opportunity to gather with colleagues from across the industry to discuss the state and future of MR. This year's keynote was particularly inspiring. Robin Pearl, vice president of Market Research at Estée Lauder, taking her lead from Tina Turner, asked, "What's Love Got To Do With It?" when it comes to achieving high-quality market research. Robin addressed the quality of sampling, questionnaire content and analysis.

Many issues are affecting sampling. Neither of the most popular quantitative methodologies – telephone interviews and Internet-panel interviews – now accurately represents the population. The 18- through 24-year-old demographic frequently uses only cell phones, putting them out of reach of most automated dialers, which only dial land lines. On the other side, the 55+ demographic is less likely to use the Internet. In addition, different ethnicities are harder to reach by different methods. As a result, Estée Lauder has encountered some of the following quality issues:

* Errors in Weighting - One survey reported in error that few women under the age of 25 used lipstick. This error was a result of the fact that the results had not been weighted to reflect demographics at all and had simply underrepresented that demographic. Robin also reported occasions where weights were applied to the final results rather than to the screener, providing results that did not accurately reflect that different demographic segments did in fact have different propensities of usage. When weights were applied, some research firms showed a willingness to weight a sample size of 50 up to 250 for a particular segment; only minor weighting should be applied, to prevent distorting the results.

* Oversurveying – Robin pined for the days when screeners typically excluded respondents who had participated in another survey within three to six months of the current survey. She joined a number of Internet panels and was shocked to soon be receiving 10 invitations a day. Professional respondents may in fact differ attitudinally from casual respondents.

* Panel Overlap - Estée Lauder is concerned about professional respondents being double counted. In one survey Robin sponsored, there was significant overlap of respondents across panels, to the point where through different panels the same household completed the same survey on the same day.

Read the full article here:

Source: http://blog.vovici.com/vovici_blog/2008/06/quality---whats.html