Out-of-Focus Qualitative Research

Hope you caught the Frontline episode on The Persuaders last night in the US. While recognizing that my particular field of anthropology - economic anthropology - has greatly helped to influence marketing studies (see Journal of Consumer Research), marketers still rely almost exclusively on focus groups, which have been eschewed by methodolocially mature social sciences. Why? Well, first check out this compendium of thought by a UK research firm, including a link to the article, "Lies, Damn Lies and Focus Groups?" by Daniel Gross on Slate.

My thoughts on focus groups are as follows:

1) They might have "face validity," but certainly not "content validity" as a meaningful measure of consumer thought. Groups are small, selected by those willing to give up several hours for pay, and individuals are taken to be a representative spokespersons for whatever racial/ethnic or sociodemographic groups are to be included. They therefore gloss over differences within groups based on sampling protocol and harken back to the worse days of ethnographic research in which the colonial anthropologist could "pick a native...any native" as an all-knowing omniscient revealer of a foreign culture.

2) All small groups have their own dynamics, especially with people unknown to one another. There are always leaders and followers, so how are we to ascertain whether the group might be shifted in one direction or another by one or two members during the discussion? This is also related to the problem of participants saying what they think the focus group moderators want them to say. Call this the unwitting malfeasance problem.

3) A standard methodological principle of anthropology is that you want informants to speak in their own terms about some cultural frame or topic. Your goal is to understand that, not force them to learn "anthrospeak," to help you complete your study. Focus groups are all about coordinating with marketing departments; using the measure to reinforce decisions already reached behind closed doors. Remember the riddle of Clever Hans the Counting Horse. He really wasn't counting; his owner was unconsciously twitching when he reached the number that had been called out.

4) Focus groups usually are conducted in business-like offices with large, imposing tables, and one-way mirrors. People just don't buy things in that environment. Better to follow them around their daily lives, making observations, and correlating behaviors with what informants have told you about their preferences. This naturalistic component of ethnography is the cornerstone for successful qualitative research seeking to understand behavior, not verbal concepts.

5) How many focus groups fail to make viable recommendations? This is almost never reported in the literature, due in part due to business competition and proprietary knowledge limitations.

6) If you conducted several dozen focus groups, with different people and moderators, how reliable are the findings? I would guess that any correlation of results would be poor, at best. This speaks to the external validity of this methodology.

In short, focus groups are the crutch of qualitative marketing research. Good for generating ideas, but very poor as a meaningful test in the public at large. This is yet another area where anthropologists and sociologists can step in. I think the PBS Frontline special did a good job of showing the public how problematic this type of research can be.