Thursday, May 15, 2014

Are you giving research responders what they want?

For the most part, researchers live off of surveys. The vast majority of researchers specialize or have basic skills in writing surveys and analyzing survey data. But in the last few years, many more tools have become readily available to us.

We recently conducted a quick survey (there’s the first bias!) asking people about their opinions to a number of different common research methods (‘What is your opinion about participating in these kinds of market research studies? Love, Like, Neutral, Dislike, Hate’). 

Obviously, surveys were the favored method with 80% of more than 370 responders giving a top 2 box response. Far behind surveys were communities at 49%, an interesting result given that communities are a relatively new phenomenon. In third place at 44% was online focus groups. And in fourth place, mobile surveys received only a 32% approval rating, likely the result of our reliance on ancient survey designs and our inability to keep up with the advanced features of smartphones. In last place, face to face research generated approval from only 23% of people.

One of the main things we worry about as researchers is how satisfied people are with the research process. We know that as satisfaction declines, so do response rates. Well, one key feature of satisfaction is ensuring that we allow people to participate in research using a method they like.

Let’s consider 18 to 24 year olds, the most coveted, fawned after group of people that researchers want to listen to. Though 80% of people approved of surveys, only 75% of young people did. What they wanted more than any other age group was mobile surveys (56% vs 32%). We’d better step up our mobile survey skills and technology!

What about people who have a lot of education, another highly sought after group? While 73% of them approve of surveys, they are also more interested in focus groups than average (39% vs 31%). Granted, getting people to attend a focus group may be difficult but there is an important group of people who like them and should be offered them.

Try an online focus group (51% vs 44%) if you’d like to interest full-time employed people. Interested in higher-income people? Forget surveys and go straight to communities (57% vs 49%). Need to listen to people from different ethnicities? Try mobile surveys to interest African American people (41% vs 32%), communities to interest Asian people (62% vs 49%), or focus groups to interest Hispanic people (31% vs 44%).

So who hates face to face research more than researchers? With just 13% approval, don’t even think about conducting face to face research with women aged 18 to 24.

Annie Pettit, PhD is the Chief Research Officer at Peanut Labs, a company specializing in self-serve panel sample. Annie is a methodologist focused on data quality, listening research, and survey methods. She won Best Methodological Paper at Esomar 2013, and the 2011 AMA David K. Hardin Award. Annie tweets at @LoveStats and can be reached at

No comments: