Articles Papers & Talks

 

by Don Bruzzone

The Effect of Online Sample Quality on Measures of Ad Performance; (May 2008) a test of eight measures cited as likely indicators of poor quality included as part of BRC's 17th annual Super Bowl Study tracking the impact of all 55 2008 Super Bowl commercials. The study finds only three have a measurable effect on ad tracking results: speeding, straight lining and not following unexpected instructions.

(Available at http://www.thearf.org/assets/bruzzone-on-research-quality: as an attachment to an interview with Don Bruzzone posted on the Advertising Research Foundation website in May 2008 under the title "Some online research panelists are poison that affect research quality:  Research from ARF member Bruzzone Research explains why and what to do about them".)

(Another review of this study, and another interview with Don Bruzzone about it, is available at http://www.surveysampling.com/?q=en/myssi/dataqualitymyths )

Kaiser Permanente's "Thrive" Campaign: "There are Hispanics...and then there are Hispanics":  a talk given at the Annual Convention of the Advertising Research Foundation in New York, March 22, 2006, with Jack Bookbinder, PhD, head of advertising research at Kaiser Permanente.  The results they presented were from a large continuing ad tracking study BRC has been conducting among the general public, Hispanics and company decision makers. It compares the impact their massive $100+ million "Thrive" campaign is having on each segment of their target market.  It shows how figures for the number of Hispanics reached and affected by the advertising can change when samples are drawn from different sources, and how those problems were solve

(Later published in the April 2006 issue of Quirks. To see a full copy of the talk click here.)

(For a summary of what else was presented at the March 2006 ARF Convention see BRC's Top 10 Insights for the conference by clicking here.  Note Item 5 for a comparison of Coke's findings with ours.)

"Top 10 Insights from the 2005 Super Bowl Commercials"  This latest 2 page summary of what BRC has learned from the annual post-testing of all Super Bowl commercials that it has been conducting for the past 14 years can be seen by clicking here.

"Pre-post vs. Post-only Studies"  Quirk's Marketing Research Review, April 2005.  A review of the pros and cons of using before and after studies to measure the effect of advertising instead of a single survey afterwards.  Recognition based tracking makes it possible to measure the "lift" in awareness or purchase interest among those who saw the advertising.  The growing use of lift measures makes single surveys more valid.  A copy is available by clicking on the following:

( www.quirks.com/articles/a2005/20050402.aspx?searchID=21484759&sort=5&pg=1)

"What We Learned From The 2004 Super Bowl Commercials"  This was covered in Advertising Research Foundation Web Casts by Don Bruzzone on March 19 and 30 2004 and at talks he gave at Toronto's PMRS and San Francisco's AMA.  It was based on BRC's annual syndicated post-testing of all Super Bowl commercials, studies the firm had been conducting for the previous 13 years.  For a recap of the top 10 insights from this 2004 study, click here.

"OOH vs. Other Media.  Getting Comparable ROIs from Tracking."  Ann Breese, Marketing Research Director of Starbucks and Don Bruzzone presented a case history June 20, 2003, at the ESOMAR/ARF Worldwide Conference on measuring media audiences and the effect advertising has on them.  Advertisers have long been frustrated by available measures of  OOH. (Out-of-home: billboards, transit cards, etc.) They can't be directly compared to measures of the number reached and affected by other media.  The talk shows when Starbucks' TV, radio, magazine and OOH advertising were all subjected to the same recognition based tracking, differences in the additional buyers attracted by each execution in each media could be directly compared in terms of the cost per buyer added.  In this case OOH compared quite favorably to TV, radio and print.  The conference was in the US this year - Hollywood.  (More on this talk, comments on it by ARF President Jim Spathe, and recaps of other talks at this session are in one of BRC's "Top 10 Insights" that can be reached by clicking here.)

(Later published in the April 2004 issue of Quirks, a copy of which can seen by clicking below.)

(http://www.quirks.com/articles/a2004/20040401.aspx?searchID=21437548&sort=5&pg=1)

Testing storyboards online instead of in focus groups:  Jack Bookbinder from Kaiser Permanente and Don Bruzzone paired up to discuss this topic before the Advertising Research Foundation in New York on October 8, 2002.  Both have decades of experience testing storyboards in focus groups, and have just finished experimenting with an online approach.  Testing storyboards in focus groups is one of the most common procedures used in advertising research.  It is also one of the most controversial.  Results presented in this talk help answer a number of key questions:  

  • Testing them online can be better, faster and cheaper.

  • Online respondents are likely to be more representative than the folks you see in focus groups.

  • They could visualize storyboards as finished commercials.

  • Their written comments were as complete and rich in insights as those from qualitative research

  • Online testing of storyboards is most appropriate in selecting the best execution of a given message. 

(For BRC's "Top Ten Insights" recapping other talks at this conference, click here.  A condensed version of this paper was published in Bob Lederer's Research Conference Report, January 2003) 

How to keep respondents interested in long online surveys:  BRC did a great deal of experimenting converting the syndicated tracking of all Super Bowl commercials it has conducted since 1992 to an online study. Don Bruzzone described how they got the average respondent to spend 30 minutes reviewing Super Bowl commercials by telling how many were left and asking if they wanted to continue. It was at the IIR conference on Web-based Surveys in San Francisco, June 19-21, 2002.  BRC's "Top 10 Insights" from other speakers at the conference can be viewed by clicking  here.

Marketing Mix Modeling:  Don Bruzzone Chaired a two-day IIR Conference devoted to Marketing Mix Modeling, March 21-22, 2002 in Chicago, IL.  It was the first on the subject to stress the importance of advertising.  Different ways of incorporating advertising into the model were compared.  One of BRC's "Top 10 Insights" recapping the conference in two pages is available and can be accessed by clicking here.

"Modeling Advertising?  First determine if it was good, bad or average."  A talk given at the IIR Conference on Marketing Mix Modeling in Chicago, March 21-22, 2002.  A major reason marketing mix models don't fit historic data as well as they could is that advertising is only represented by Dollars or GRPs.  That is the same as saying bad advertising contributes as much to sales as good advertising.  This presentation stressed the importance of making that portion of a marketing mix model as realistic as possible, and reviews the best ways of doing it.

    (Later published in Quirk's, March 2003, accessible at:)    

    http://www.quirks.com/articles/a2003/20030302.aspx?searchID=21438181&sort=5&pg=1

"Tracking Super Bowl Commercials Online", a talk given by Don Bruzzone October 30, 2001 at the Advertising Research Foundation conference in Chicago tracing the evolution of the firm's long term tracking of Super Bowl commercials during the previous three years into a highly effective online format. The talk also reviews the results obtained from the online version of BRC's recognition based tracking with its conventional tracking.  

"Why Tracking Should Replace GRPs"  (TiVo, Replay, channel proliferation and a host of other changes are making it very difficult to measure reach, overlap and frequency in the traditional ways.  However, when you ask people "Have you seen THIS commercial?" - that's when you learn the real answers.)  A talk given by Don Bruzzone at the Advertising Research Foundation Annual Convention, March 6, 2001 

"Advertising Movies on the Super Bowl, What Works and What Doesn't," a talk by Don Bruzzone and Dan Rosen, Senior Vice President of Warner Bros. given November 1, 2000 at the ARF Entertainment Marketing and Research Conference, Beverly Hills, 

    (Later published in Quirk's, April 2001, accessible at:)

    http://www.quirks.com/articles/a2001/20010305.aspx?searchID=21438395&sort=5&pg=1

"Brand Extensions: Their Impact on Overall Brand Equity," a talk by Don Bruzzone and Ann Breese, Marketing Research Director of Starbucks given October 18, 2000 at the Advertising Research Foundation, New York. 

"Track the Effect of Advertising Better, Faster, and Cheaper Online", Advertising Research Foundation Annual Convention, New York, March 7, 2000

    (Later Published in Quirk's, July 2000, accessible at:)

    http://www.quirks.com/articles/a2000/20000713.aspx?searchID=21438461&sort=5&pg=1

"Improving the Fit of Marketing Mix Models With Data on Ad Quality", Institute for International Research, Ft. Lauderdale, Florida, October 25, 1999

"ROI & Marketing Mix Models", SfAMA, San Francisco, April 3, 1999

"Super Bowl ROI: 30s vs. Other Lengths", Advertising Research Foundation, New York, October 26, 1998

"Comparing the ROI of Print, Radio, and TV Ads",  a talk by Don Bruzzone & Lizabeth Reyer, Research Director, Blue Cross Blue Shield of MN,  Advertising Research Foundation, New York, October 28, 1998

(Later published in Quirk’s, March 1999, accessible at:)

http://www.quirks.com/articles/a1999/19990307.aspx?searchID=21438543&sort=5&pg=1

"Linking Tracking to Pretesting with an Advertising Response Model",  a talk by Donald E. Bruzzone & Deborah J. Tallyn, Director of Marketing Research, Pacific Bell, at the Advertising Research Foundation in New York, November 12, 1996

 (Later published in the Journal of Advertising Research, 6/97)

Earlier Articles Coauthored by Don Bruzzone

David A. Aaker & Donald E. Bruzzone (1981), "Viewer Perceptions of Prime Time Television Advertising", Journal of Advertising Research 10/81

David A. Aaker & Donald E. Bruzzone (1985), "Causes of Irritation in Television Advertising", Journal of Marketing, 49, 47-57

Stayman, Aaker & Bruzzone (1989), "Types of Commercials Broadcast in Prime Time: 1976-1986", Journal of Advertising Research 6/89

Donald E. Bruzzone & R. Paul Shellenberg (1996), "Super Bowl Advertising: What Really Works", Quirk's Marketing Research Review, 3/96

Articles showing BRC research results

TV Dimensions 2008, Ed Papazian, Publisher/Editor, "Consumer Evaluations of TV Ad Campaigns: A Thirty-Year History" (Pages 317-321)  This article in "the bible" of TV statistics is based on BRC's extensive database showing how thousands of ads and commercials have performed among various types of people.  It is an updated version of a series of papers that have been appearing in TV Dimensions for years.  It compares results through 2007 with the results from the following paper: BRC's first presentation of its data on ad performance in 1979.  

(Donald E. Bruzzone (1979), "The Case for Testing Commercials by Mail",  Proceedings of the 25th Annual Conference of the Advertising Research Foundation, New York, October 23, 1979)    

TV Dimensions 2008, Ed Papazian, Publisher/Editor,  "An Update on How TV Commercial Awareness and Attribute Ratings Vary by Product Class", (Pages 322-325)  This is also an updated version of papers that have been appearing for years in this publication.  It shows that currently commercials for Food and Soft Drinks are performing best while commercials for Automobiles and Household Non-Durables are the poorest performers.

Horovitz (1998), "Super Bowl Myths, Realities,"  USA Today, January 5, 1998 (A humorous recap of what BRC  learned testing all Super Bowl commercials since 1992.)

Biel & Bridgwater (1990), "Attributes of Likable TV Commercials", Journal of Advertising Research, June 1990  (This study continues to be one of the most widely quoted sources of information on what likability really means.  BRC conducted it for Alex Biel, David Ogilvy's head researcher, using BRC photo board questionnaires sent by mail.)

Dubow (1995), "Advertising Recognition and Recall by Age,"  Journal of Advertising Research, September/October 1995  (BRC supplied recognition from a set of 80 commercials that showed the average recognition scores were highest among teens and decreased monotonically with increases in age. Dubow is a former head of advertising research at Coke.)

Aaker & Stayman (1990), "Measuring Audience Perceptions of Commercials and Relating Them to Ad Impact,"  Journal of Advertising Research, September 1990  (Multivariate analysis of BRC's adjective list shows how likability and other diagnostic measures combine to drive the impact of advertising.)

Aaker & Norris (1982),  "Characteristics of TV Commercials Perceived as Informative,"  Journal of Advertising Research, April 1982 (An early study of BRC's database by U C Berkeley academics.) 

Books and texts that have described and commented
on BRC's recognition-based methodology

Ed Papazian (2008), "TV Dimensions 2008", Media Dynamics, Inc., pages 317-325 (See articles listed above in the previous section)

Grover & Vriens (2007), "The Handbook of Marketing Research", Chapter 23 "Ad Testiing" by Baldinger & Cook, page 504 cites Bruzzone Research as one of the 13 major companies involved in Ad Testing and Ad Tracking whose work the chapter is based on.

Chitty. Barker and Shimp (2008) "Integrated Marketing Communications, Second Edition", Cengage Learning

Terence A. Shimp (2006), "Advertising Promotion, Seventh Edition",  Thompson South West, pages 332-338.

David Aaker’s autobiography “From Fargo to the World of Brands” (2005): This widely published author of five of the best selling books on Brand Equity and Brand Management describes his collaborations with Don Bruzzone starting in the early eighties (page 219) with reflections on what was learned from the articles listed above that they co-authored.

Giep Franzen (1994), "Advertising Effectiveness,"  NTC Publications Ltd, page iv.

William Wells, John Burnett, Sandra Moriarty (1992), "Advertising Principles and Practice: Second Edition,"  Simon & Schuster,  pages 605-613.

David A. Aaker, Rajeev Batra, John G. Myers (1992), "Advertising Management: Fourth Edition,"  Prentice Hall, pages 233, 239, 408, 409, 555.

Michael L. Rothschild (1987), "Advertising,"  D.C. Heath & Co., pages 343, 359, 362.

David A. Aaker, George S. Day (1986),  "Marketing Research: Third Edition,"  Wiley, pages 568 - 569, 582..

About BRC ] What we do ] How we do it ] Why we do it ] What's New ] Demo Surveys ] SuperBowl ]