Message Testing Strategy
Messaging is a continuously evolving “game” between marketers and consumers. Marketers communicate a message to their audience – which they believe at that time is the most effective (most motivating) message. Consumers, in response to the marketing message, either respond positively or negatively (which is often measured after-the-fact by a single behavior change or outcome like $-Sales).
There’s a serious problem with measuring the effectiveness of a message using only post-production measures (behavior change/outcome) – creative production is very expensive. Few can afford to be “wrong.”
What if I told you there is a way to test what messaging is most effective before you spend a dime on your advertising campaign (including creative production)? This may sound like cheating, but in reality, it’s just smart marketing. We frequently help companies with message testing, so let us share with you how it works and how you can leverage it.
(Background – our company’s roots stem from advertising, on the client and advertising agency side. We’ve used our experience in advertising to develop a proven process for optimizing messaging.)
Every day consumers make choices – most are not conscious rational decisions. These unconscious emotional drivers are hard to understand using direct questioning approaches because consumers struggle to self-report the impact (weight) of unconscious elements. In other words, consumers don’t know how emotional elements play into their decision-making, so why do we expect them to be able to tell us directly? Direct questioning approaches fall short because they can only evaluate part of the decision – the rational side.
“If you ask a rational person and rational question, you’ll get a rational answer.”
We have developed a tool called IdeaMap® that enables us to identify the optimal messaging…without asking direct questions. Imagine a deck of cards. Imagine you are dealt a 5-card hand. You know in that moment if it is a “good” or “bad” hand. You evaluate the hand as a whole. Consumers evaluate messaging the same way – as a whole.
To drive this point home, think about travel sites like Orbitz and Priceline. When booking travel plans, consumers are faced with a series of elements all at once (price, flight time, number of stops, comfort level, etc.). Consumers don’t sit and weigh out each element in their mind and figure out which is most important – they simply evaluate the options as a whole and make a choice. By analyzing the results of customer choices, these travel providers better understand which elements are driving the decision-making process.
IdeaMap® teases out the individual drivers of peoples’ decisions by varying the different parts of a message and observing their choices. This approach is based on the fundamental communications theory (stimulus –> response). For example, think of a hypothetical comedian named George. When George prepares for a primetime show he refines his presentation at several small shows. When he comes on stage at a small venue he doesn’t say, “Hi, my name is George. I’m a funny person.” Rather, he tests alternative ways of presenting a joke (length of pregnant pause, punch line, etc.) and measures the response through laughter.
We consume messaging/communications the same way we consume a comedian’s jokes – one at a time. A good example is television. During a commercial break advertisements are shown one at a time. Two ads aren’t shown side-by-side. Just like the comedian, the most motivating message is found by testing variations of a message against a single desired response (e.g. laughter). So, when testing messaging before investing in creative production, the question should be, “Which stimulus is necessary to elicit the desired response?”
- Direct questioning approaches are only good for evaluating rational factors. They struggle with understanding the unconscious emotional drivers.
- We evaluate stimuli (e.g. marketing messages, product concepts) as a whole – not in parts.
- We consume marketing messages one at a time – not in a “comparison” setting.
Under the hood (How does IdeaMap® work?)
Short test concepts are made up of the different “elements” of the message (e.g. Product description, rational benefit, emotional benefit, call-to-action, etc.). Within each element are a number of alternative statements. For example, alternative statements under the messaging element “Reason to Believe” for an ice cream company could be:
- We’ve made premium ice cream since 1907
- Made from 100% organic milk
- Named the #1 ice cream by Ice Cream Magazine
IdeaMap® is rooted in Conjoint Analysis. Conjoint Analysis determines cause and effect through an experimental design. In other words, each messaging element (and alternative) is systematically rotated to ensure each element (and alternative) is shown an equal number of times. An experimental design enables us to understand the relative weight of each messaging element.
IdeaMap® presents these messaging concepts one at a time and asks the respondent to rate the message according to a consistent test question or statement (desired response). For example, a respondent is presented a message from a shipping company and asked to rate it on a scale from 1-9 (1 = disagree completely, 9 = agree completely), based on the statement “This would fit my holiday shipping needs.”
The respondent is simply asked to react to the message (stimuli). Throughout the rest of the study they are presented with different combinations of messaging elements. By analyzing how they responded to a specific element over time, we can tease out the weight (importance) of the element.
Where the magic happens
In my opinion, the most powerful feature of this approach is being able to identify “1+1 = 3 relationships.” Some elements have a greater impact when paired together. It’s important to understand the interactive effects at play. By using an experimental design, IdeaMap® can simulate the interactive effects between messaging elements. What does this mean for you? It means you have the ability to not only understand which messaging elements are driving decisions but which mix of elements creates the optimal message (on a predictive basis).
Traditional copy testing (after some creative production) is usually about fine-tuning a message that’s almost “out the door.” The problem with testing so close to campaign launch is that it assumes the core points of a message are “right.” IdeaMap offers a different approach to creative development. IdeaMap® establishes the parameters of the “sandbox” the creative team should play in.
“If we communicate these core messaging elements then we know our desired response will improve by 18%.”
At this point, it’s in the hands of the creative team to execute the core messages. It’s a strategic approach to message testing and it enables the creative team to confidently execute their work – knowing there’s hard data behind what they’re communicating.
Applying this in your world
The first step in wrapping your head around this approach is recognizing that consumers, when asked directly, cannot report the full story behind their decision-making. Therefore, to understand the drivers behind decisions, it’s better to focus on rotating stimuli and measuring its effect on a desired outcome.
The right messaging can have a dramatic impact on the effectiveness of a campaign. It’s easy to fall into the “guessing game” trap. Statistically backing up your messaging strategy not only gives you more confidence about your campaign but it’s proven to be more effective.
Learn more about IdeaMap®