A detailed picture of the impact the commercial had, and how that compares with other Super Bowl commercials, is provided on charts like the following that show at a glance the strengths and weaknesses that account for different levels of performance.

BRC's Advertising Response Model (ARM) gives a quick visual picture of how an ad or commercial performed. It uses colors instead of numbers to show the results of 50 diagnostic measures providing a detailed view of what is driving the success of the advertising - or holding it back.

Adjectives that respondents can use to describe the advertisement are listed down the left. The color shows if the number that used that word to describe the advertisement was in the top third, the middle third or the bottom third of the scores for that word among the commercials that constitute BRC's norms. Top third means the most favorable third. For negative adjectives that means the third that checked the word the least.

Adjectives are combined into sub-clusters and major clusters that reflect the basic types of reactions people have to almost all advertising. The more emotional, affective, and executionally oriented reactions are on the top half of the chart. The more rational, message oriented reactions are toward the bottom. Measures of overall impact are toward the right.

BRC's ARM has a high level of intuitive face-validity. It shows advertising the way most people understand it. But it also has a solid foundation in theory and basic research. The split between emotional and rational reactions is based on the work of Petty and Cacioppo on central and peripheral processing . The way the basic factors feed into attitudes toward the ad and attitudes toward the product are based on the causal modeling of Lutz and his colleagues. The adjectives were drawn originally from the work of Wells, Levitt and McConville and enlarged in collaboration with Alex Biel and David Aaker. The clusters are based on years of factor analysis by Clark Levitt.

The percent reached and affected, the most critical measures of a commercial’s overall perform-ance, are shown in a clear, straightforward manner by the following three bars that provide the basis of all BRC charts comparing the basic overall performance of commercials.

The next chart shows the following situation: Commercial X was best at capturing attention. However Commercial Y was better liked and better at getting the name across, so it reached and actually affected the greatest number. Both were average or better in each of these measures. Commercial Z was below average in all comparisons. Advertising Response Models for these commercials would provide the diagnostics that show what accounts for these differences in performance.

Averages are shown for all commercials, as well as for commercials in the same category, because both are relevant. Every commercial is in competition with every other commercial on the air for attention. But once it gets noticed, its potential effect on market share is determined by how its impact compares to the impact of other commercials in the same category.

The percent that recognize a commercial tends to level out after it has had 300-400 GRPs of exposure. So all commercials that are past that threshold of sensivity to additional exposure can be validly compared as shown on this chart.

If that is your reaction, consider this: Pretesting is an attempt to predict how a commercial will perform when it is actually aired. It is conducted under artificial conditions. What we are offering is a post test. It measures how your commercial actually performed after it was aired and people had a chance to see it under real world conditions.

How long since you checked to be sure your pretesting is predicting correctly?

You will have a hard time finding a less expensive opportunity for checking. It is worth doing because there are things pretesting doesn’t catch. For example, if you are like most advertisers, your pretesting involves "forced exposure" (people are asked to look at your commercial). That makes it impossible to measure one of its most important attributes: its ability to capture attention. A review of the types of information we are collecting is on the next page. Check to see if there are other areas where our post test can make important additions to your knowledge of how your advertising is performing.

Everything asked in the questionnaire serves a specific function. The following recap reviews the basic reasons each type is included.

How many noticed the commercial? The percent that recognize it is the best way of finding out. Most ad tracking surveys are still conducted by phone. Respondents are asked if they recall any recent advertising for the brand. Those who say "yes" may be thinking of last year’s advertising, or even a competitor’s advertising. But, when you show it and ask if they remember seeing THAT advertising, you get a massive increase in accuracy. The greatest danger any advertisement ever faces is being ignored. Separating out those who ignored the commercial, or were never exposed to it, is the critical first step in determining if it had any effect on those who were reached.

Did they know who it was for? Ever talk about a commercial that had a great punch line, but you couldn’t remember who it was for? We all have. For the advertiser, that’s deadly. BRC’s branding question shows how well creatives met the branding challenge, and what proved most successful in making the name stick in people’s minds.

Did they like the commercial? The first two measures only show if a commercial had a chance to work its magic. They show if it achieved cognitive awareness: if it "got inside their skull." Likability, and the diagnostics that show what they liked and didn’t like, are the first that show if it had an actual effect on those who noticed it.

How did they react to the commercial? Once researchers thought the most important question was "What was the main point?" When we found likability was the best of all measures we started to realize if they got the point, but didn’t believe it, or didn’t feel it was important or relevant, or the ad made a bad impression on them, it didn’t lead to sales. We realized it is their reaction to the message that is important, and that is why we obtain all the detailed diagnostics shown previously on the ARM chart.

Did the commercial increase salience and buying intentions? Here we measure the "lift" in unaided awareness of the brand, and intentions to buy the brand, found among those who noticed the commercial. They show how well it performed in these more tangible measures of impact.

This line of questioning is in line with the findings of two industry-wide studies to identify the most valid ways of measuring the effects of advertising: the Advertising Research Foundation’s Validity Study, and the confirming evidence from "Adworks 1," a follow up study by IRI and 40 major advertisers. It avoids measures they have shown don’t work well: like recall, feedback and persuasion measures. It uses those that have demonstrated the highest validity: likability, diagnostics and recognition.

In the late 90"s and early 2000’s we conducted four years of parallel testing that shows our online testing of all Super Bowl commercials produced the same basic results as the conventional testing we had been using for over a decade. Details have been presented in a number of talks and an article in Quirk’s that we would be happy to send to any who are interested. The first was a small initial test in 1999. The encouraging results led to much larger tests in 2000 and 2001. Our final experimenting and fine-tuning was conducted in 2002. Originally, we expected to end up with a procedure where the offline segment of the public was surveyed in the conventional manner. What we found was that split sample approaches produced the same results as the approach we are now using which is entirely online.

At the conclusion of this year's Super Bowl BRC will have completed its 17^th consecutive year surveying people after the game to see which commercials they still recognized. We show if the commercials were noticed and remembered, or ignored and forgotten. When a commercial has passed these two critical tests of attention getting power and the ability to get across the name of the advertiser - factors that are ignored in most other testing procedure - we then start analyzing people's reactions. We measure 50 different dimensions of entertainment value, empathy, relevance and overall impact. The commercials are now tested using online questionnaires. For the first 7 years the same type of photoboard-recognition test was conducted by mail. Then we conducted 4 years of parallel testing using both the internet and mail, to ensure both produced the same winners and losers, and that we could produce comparable scores for all years. Results from the 2008 Super Bowl will be available approximately mid-February. Costs are outlined below on some of the more popular ways of obtaining information:

Did they fail to follow specific instructions?

Did they straight-line answers?

Did zip code and state match?

Did they say they might buy a non-existent brand?

The proportion of the sample these measures showed to be of questionable quality were usually small and the effect they did have was not very dramatic.

 

 

[ About BRC ] [ What we do ] [ How we do it ] [ Why we do it ] [ What's New ] [ Demo Surveys ] [ SuperBowl ]