This article originally appeared in Quirk’s Research Review, May 2005, page 40.

As a client or vendor, I’ve monitored or conducted tens of thousands of phone-based and in-person groups and individual interviews (not to mention the Web, since that’s another article). It often seems as if the fundamentals have been waylaid by some common seductions, spawning “we already knew all this” complaints. The thesis of these six deadly sins is that qualitative research is at best natural conversation and at worst performance art. Since you probably don’t want to be a starving artist…

[Henceforth, “researcher” signifies either the interviewer/moderator or a person that is selling, using or analyzing the study; in context, it should be obvious which. “Client” refers to both external clients and internal customers: marketers who need useful business intelligence for informed decision-making.]
Over the next month or so I’ll discuss each of these six sins, so stay tuned.

The first sin:  Ask me no questions, I’ll tell you no lies.

Managing client expectations begins at the pitch stage, when researchers may allow clients to believe that flying or calling coast-to-coast ensures “nationally representative findings.” You can characterize L.A./Phoenix/Chicago/Houston/Miami/Philadelphia results as “geographically diverse” but to act as if these respondents can represent all the potential respondents not there is counterproductive. Self-selection, recruitment and interviewer interaction biases are more prominent in any qualitative study, just as interviewer-assisted quantitative studies engender greater interaction bias than self-administered studies. If you need a nationally representative sample, you need quantitative data. Multi-city qualitative studies cannot substitute for quantitative. “Semantics,” you say. “We know qualitative is directional.” In my next post I’ll explain why this isn’t so.