The 15 Year-Old Marketing Research Time-Capsule?

Feature Friday: The 15 Year-Old Marketing Research Time-Capsule?

Guest Post by Chris Ruby

More than 30 years ago while I was in the 3rd grade, our entire student populous over at my elementary school participated in a time-capsule ceremony with the teaching staff and administration. I and other members of the student council picked out items that represented the early 80’s, placed them all in a metallic themed lunch-box along with a small note, then proceeded to bury the time-capsule that we created with shovels full of dirt in a large grassy area that was in front of the school’s administration offices.

As our principal patted down the pile he said, “Sometime from now, far off in the future, when construction workers are tearing down this school for whatever reason, they will find this time-capsule that we have just buried. As they open it, start exploring all the little trinkets inside and then stumble upon our “greetings” letter, they will get a glimpse into the year 1981. Will they recognize the world that we lived in? Will they recognize the things they see? Who knows? But this time-capsule represents us. It represents the here and now. And it will give evidence that we as a community existed.”

It was an eventful moment in my life and I recently started to think to myself, “What if I had created a Marketing Research Time-Capsule when I started my career in 1997 and placed within it the Marketing Research methodologies and principles that were in use at the time?” It was an interesting concept and my mind continued to ponder, “What if the Marketing Research time-capsule was discovered and opened today, 15 years later? Would Marketing Researchers recognize and still be using the same methodologies and principles?” It was such an entertaining thought that I decided to play along and give a brief overview of the methodologies and technologies I experienced in the late 90’s, as if I actually buried the time-capsule and then compare them to today’s methodologies and technologies, as if I actually discovered the time-capsule.

To begin, the first paradigm I experienced in the early stages of my career can be summarized as followed: Long Survey + Large Sample Size = Reliable & Valid Study. I believe I experienced this because as market researchers we want our research to be reliable and therefore valid. We achieve this through collecting and analyzing lots of data. It’s in our blood. Now add in the adage that you need to interview 1000 respondents in order to obtain a margin of error of ±3% and the next thing you know, you have a long, expensive study with a 30 minute to 1 hour questionnaire.

The second paradigm I experienced at the onset of my career was the utilization of paper surveys & telephone interviewing centers during the data collection process. A majority of the research projects I managed early on were conducted through either in-person interviews (either through a one-on-one interview or focus group study) or over the telephone. In-person interviews took place either on-site, at local events, in homes or at malls & shopping centers while telephone interviews were handled by an interview center that utilized computerized CATI systems. Each study’s respondents were typically either purchased through a list or were intercepted at various locations. At that point in time the Do Not Call Registry did not exist.

Online surveys at that point were beginning to emerge along with the dot com era. Invitations to participate in an online survey were usually sent via email, where the respondent would click on a link within the email and be taken to an alternate site where the survey would begin.

The last main paradigm I experienced was the belief that marketing research projects cannot be combined or synthesized. Each had its own problem definition, its own design and its own sample that allowed a researcher to perform a strict statistical analysis and come to conclusions and recommendations that were only relevant to that particular study. Combine data from another study with different parameters, different variables and a different sample and voila…you now have an unreliable, invalid study that has broken the principles of marketing research and statistics.

Now forward fifteen years and ask yourself, “Do these marketing research paradigms still exist today?” Well, to some extent they still do, but things are on the move as our marketing research landscape is changing.

As proof to my argument, I recently attended the Future of Consumer Intelligence 2013 Conference in San Francisco, CA where there was a buzz about Online Consumer Panels, Social Media Research and Mobile Marketing Research (all of which were nonexistent when I started my career.) There were 3 specific speakers that seemed to address the paradigms that were infused in me over a decade and a half ago.

The first was Bill MacElroy, Co-founder and Chairman of Socratic Technologies, Inc., a San Francisco-based research agency focused on computer-based and interactive marketing research. Bill provided evidence that survey response rates are at an all-time low and successful survey completion rates drop dramatically when the survey is over 15 minutes long and continues to drop the longer the survey.

Bill’s answer to this dilemma is a fun, interactive, online gaming technology that both visually and psychologically entertains respondents while filling out a Choice-Modeling survey. His short online survey and gaming technology combination had me asking myself, “What’s the point of fielding a long, non-entertaining survey if it drives down the response rate, the completion rate and the validity of responses while increasing the overall cost of the study?”

When I started my career, I was trained to design long, clean looking, likert-scaled, mutually-exclusive surveys for respondents. But to design a survey to be short and entertaining? Can’t say that thought really crossed my mind in 1997, but in today’s world of instant gratification and mobile devices, it should be on all of our minds if we want respondents to continue to participate in our marketing research.

The second and probably most compelling speaker was Dr. Larry Friedman, Chief Research Officer for TNS. During his presentation, Dr. Friedman presented evidence that the way we have conducted research in the past doesn’t necessarily correlate with purchasing behavior. This was surprising news that had my stomach turning with butterflies as I was listening, since the whole purpose of conducting marketing research is to capitalize on marketing opportunities that enhance sales. His main point was that as we move forward in a world of social media and online communities, synthesizing data from multiple sources with smaller sample sizes will be the key to developing a clearer picture of consumer behavior and insights, thereby allowing us to correlate our research with purchasing behavior.

The last speaker, Jared Heyman, Founder of Infosurv & CrowdMed, inadvertently addressed some of my early paradigms. Jared presented on the topic of “The Wisdom of Crowds” where the joint opinions of small crowds can actually match those of “experts” in a field. His CrowdMed website allows random people to overview the symptoms of a medical case and then make an educated guess as to the diagnosis of the illness. His research shows that crowds of people can actually guess the correct medical diagnosis, similar to an “expert” in the medical field.

So how does this relate to marketing research and the marketing research time-capsule? It relates in as much that we quite often want to sample an “expert” respondent. Think of it in terms of segmentation and the core buyer. The person who shops more frequently, uses a product more frequently, spends more on a product than the average person. Is this someone we would like to talk to as a market researcher? Do we screen out other people in order to talk to the core buyer? Certainly, because they are considered an “expert” consumer. And according to the Pareto Rule, 80% of a company’s sales will come from 20% of its “expert” customers.

Needless to say, it was surprising to hear that a crowd of non-experts could match the opinions of an expert. But does this mean that a group of non-core customers could just as well predict the opinions of core customers? The wisdom of crowds may suggest that they can and it is uncertain how we as market researchers can and will respond to this.

However, what I am certain of after attending the Future of Consumer Intelligence conference is that the Marketing Research paradigms of old are changing. Our industry is in a state of flux as we come up with new standardized methodologies and principles that address social media, mobile and online community research that focuses on smaller, more interactive surveys. It’s apparent that those who are willing to change will move ahead in the future of our industry. And those who are unwilling to change will sadly be left behind.