MSI Blue Ribbon Panel Report

MSI BLUE RIBBON PANEL REPORT

Charting the Future of Marketing Mix Modeling Best Practices REPORT PREVIEW

AUTHORS

ORIGINAL DOCUMENT

Neeraj Arora ARTHUR C. NIELSEN, JR. CHAIR AND PROFESSOR, UNIVERSITY OF WISCONSIN-MADISON

Ron Berman ASSOCIATE PROFESSOR, UNIVERSITY OF PENNSYLVANIA

Elea McDonnell Feit ASSOCIATE PROFESSOR, DREXEL UNIVERSITY

Dominique Hanssens DISTINGUISHED RESEARCH PROFESSOR, UCLA; PAST MSI EXECUTIVE DIRECTOR

Alice Li ASSOCIATE PROFESSOR, THE OHIO STATE UNIVERSITY

Mitchell Lovett BENJAMIN FORMAN PROFESSOR, UNIVERSITY OF ROCHESTER

John Lynch UNIVERSITY OF COLORADO DISTINGUISHED PROFESSOR, MSI EXECUTIVE DIRECTOR

Carl Mela T. AUSTIN FINCH PROFESSOR, DUKE UNIVERSITY; PAST MSI EXECUTIVE DIRECTOR

Kenneth C. Wilbur PROFESSOR OF MARKETING & ANALYTICS, UNIVERSITY OF CALIFORNIA, SAN DIEGO

PREVIEW DOCUMENT EDITED BY MSI

Keith M. Smith RESEARCH DIRECTOR, MSI

Earl Taylor CHIEF KNOWLEDGE OFFICER, MSI

MMM IMPLEMENTATION GUIDELINES: STATISTICAL ISSUES FOLLOWING FROM MMM DESIGN CHOICES

MARKETING MIX MODELS (MMM) VERSUS MEDIA MIX MODELS (mMM)

OMITTED VARIABLES BIAS

FACTORS AFFECTING THE UNCERTAINTY OF EFFECT ESTIMATES OF A MARKETING MIX TOOL . . .

DATA STRUCTURE AND GRANULARITY: EFFECTS ON THE VALIDITY, RELIABILITY AND STABILITY OF COEFFICIENTS IN MMM MODELS

MODEL SELECTION FOR MMM

MARGINAL RETURN ON ADVERTISING AND OPTIMAL SPENDING

Implementation Guidelines Available in the Full Report

(FULL REPORT AVAILABLE TO MSI MEMBERS)

OVERVIEW

The objectives of this Marketing Science Institute (MSI) white paper are to outline how best practices in Marketing Mix Models (MMMs) have evolved in response to a rapidly changing marketing landscape, to outline opportunities for innovations, and to offer means to assess those innovations. The report intends to identify challenges for sophisticated MMM users, and to introduce MMMs to executives with a more passing familiarity in modeling who might introduce MMMs in their firms. This report is the first of several resources MSI plans to offer its members to support both groups.

In response to media digitization, firms have increasingly employed models requiring individual level data, such as Multi-Touch Attribution (MTA) models, to apportion credit for a sale among various touchpoints along a customer journey.1 However, new privacy regulations (GDPR, CCPA) and Apple and Google platform privacy policies eliminating 3rd party cookies disrupt the ability of marketers to use individual-level customer data to understand the business consequences of their marketing actions.

In response to these new privacy policies, many firms are pursuing or expanding the use of MMMs

that rely on more aggregate data to assess and predict the effects of changes in spending on different marketing activities and to optimize those activities. These models explain changes in sales over time and across geographic locations, or stores, due to variation in spending on marketing mix elements.

MMMs were developed decades ago by marketing scholars affiliated with MSI and by industry solutions innovators.2 3 4 5 6 7 8 9 10 MMMs are resurgent because of the need to rely on aggregate data due to restrictions on collecting and using individual-level data.

MMMs face new challenges and opportunities with increasing fragmentation in how marketing dollars are spent across new and older communication channels and with the proliferation of “omnichannel” routes to distribution. How can managers know if channels are synergistic rather than substitutes for each other in driving demand? How can firms leverage new indicators of brand preference from user-generated text and images to better measure marketing effectiveness or integrate large-scale, online surveys into the MMM? What data systems integration will be required, and is it worth the investment?

Many consultants and suppliers are competing to offer updated MMM solutions to MSI member companies (e.g., AI-based and SaaS variants), all claiming certain advantages.11 12 Companies sometimes have difficulty sorting through competing vendor claims and innovations. Model details by solutions providers are sometimes a “black box”, and the marketing and C-suite executives trying to assess the claims are not statisticians. Internally, the marketing executives making recommendations based on MMMs need tools to help others in their organizations align to make best use of MMMs.

To address the changing MMM landscape, the Marketing Science Institute has initiated an MMM Industry Challenge, partnering with leading academic and industry authorities on these models and with experts inside MSI member firms.

This preview report begins by defining MMMs. It then identifies three prototypical ways MMMs are being used in industry, as uncovered by a series of industry interviews. Building on these use cases, the report discusses design elements necessary for successful MMM implementation, and it identifies promising innovation opportunities. It then discusses the challenges that corporate experts involved with the MSI MMM initiative find most pressing. It concludes with details on next steps for firms wishing to join the initiative to chart the future of MMM models. The full report provides a more detailed exploration of each of the above elements.

The full report (available to MSI member companies) is supplemented with an MMM Implementation Guideline covering technical details about MMMs. This Implementation Guideline explains why different design choices affect the quality of usable insights from an MMM. The point of the Implementation Guideline is to allow analysts better ways to communicate with marketing users and C-suite investors in marketing.

To address the changing MMM landscape, the Marketing Science Institute has initiated an MMM Industry Challenge, partnering with leading academic and industry authorities on these models and with experts inside MSI member firms.

DEFINITION: WHAT IS MMM?

A marketing mix model (MMM) is a model that predicts customer demand based on a broad set of marketing and other business drivers. MMMs quantify the impact of changes to those drivers to reveal actions a business can take, in order to increase favorable business outcomes by changes in the firm’s marketing mix (product, price, distribution, promotion). A media mix model (mMM) is a specific type of MMM where firm actions are restricted to marketing media actions. actions. The parameters of MMMs and mMMs are estimated from real-world data and/or experiments.

“Customer” here refers to the ultimate buyer of the product or service, which can be an

organization, a household, or an individual. “Demand” can refer to sales, consumption, new customers acquired, profit, long term value of customers, or any relevant KPI.

To make demand predictions, MMMs use data from the past to explain why demand was high or low. The highs and lows are statistically tied to changes in marketing actions by the firm as well as changes in uncontrollable marketplace factors.

Unlike MTAs and other individual level modeling approaches, MMMs and mMMs often operate on aggregate and less granular data (e.g., weekly advertising and sales) not subject to privacy concerns.

USE CASES: WHAT IS MMM USED FOR?

Companies use MMM for a variety of purposes, which can be conveniently organized in three use cases, going from least to most advanced: scorecards, forecasting, and optimization. Scorecards are descriptive and backward facing, forecasting is predictive, and optimization is prescriptive. Within the full report (available to MSI member companies), we provide examples that are tactical (e.g., allocating an advertising budget) and strategic (e.g., setting investment levels across marketing levers or across brand portfolios) for each of the three use-cases.

Each use case is affected by recent changes in the marketing landscape. For example, machine learning affords new means of specifying and calibrating predictive models. New KPIs suggest alternative benchmarks for business management such as social media measures of consumer sentiment.

Overall, the intended use cases can affect how the MMM is specified or implemented. MMMs are critically important for resource allocation adjustments within budget cycles. They also play a key role in getting internal buy-in for marketing actions, and operational aspects of the business.

A detailed exploration of the three MMM Use Cases is available in the full report.

DESIGN CONSIDERATIONS AND OPPORTUNITIES FOR INNOVATION

Marketing Mix Models differ in scope and objective, depending on the breadth, the frequency and the granularity of the data, and the intended use cases of the MMM.

MMM DESIGN CONSIDERATIONS: INPUTS, OUTPUTS, AND MODELS

All MMMs share a common architecture, in that, they process inputs using a model to generate outputs for the use cases. The modeler should work with the marketing manager to elicit their mental models of what drives the business in identifying the inputs, outputs, and model specifications.

In designing an MMM, the analyst will have to consider inputs, outputs, and model specification in conjunction with one another. For example, the marketing manager may want to understand the effect of a control variable like “influencer marketing”, but the firm may not have sufficient data on the spending or reach. Maintaining data sources, doing ongoing validation of the model and improving the model’s mathematical formulation are key components to building and maintaining a strong MMM capability and modeling pipeline.

USER REPORTED CHALLENGES AND INNOVATION OPPORTUNITIES

Specific challenges with MMM come up in the context of the three major use cases. The committee organized the challenges along different MMM design elements. Solving these challenges provides opportunities for successful innovation, which are available to MSI members who can access the complete report.

It may not be obvious to CMOs making investments based on MMM models or managers using these models why design elements are so critical. Marketing Mix Modelers recommend resource allocations based on a set of coefficients in statistical models. The design challenges in the full report can lead to random errors in forecasts or to model misspecification—so that the models are not correctly capturing the effects of incremental investment in some marketing action. As a result, misallocations of marketing spend can occur with an adverse effect on ROI.

A detailed exploration of specific design considerations for inputs, outputs, and the model is available in the full report.

A detailed exploration of specific challenges and innovation opportunities in MMM is available in the full report.

NEXT STEPS FOR THE MSI AND MEMBER FIRMS

The Marketing Science Institute will continue to work with the Blue-Ribbon task force of academics to develop tools to help our industry and our members improve their Marketing Mix Models. Our overall initiative has three phases:

Phase I: What is current MMM practice?

Phase II: What makes for a successful MMM?

Phase III: A processes for validating MMMs

We are now working on the Phase II task of coming to a consensus on desired or required elements in an MMM model mapped to given use cases. We aim to set industry standards for best practices. We will then develop an MMM Checklist for companies to document their MMM operations, and for MSI members to evaluate their own MMM operations by benchmarking their use cases against best practices and (confidentially) sharing areas that are challenging. Member firms can contract with MSI to facilitate their internal discussions on using checklist results to improve their internal MMM model pipelines. MSI will ask companies to (anonymously) share checklists to allow an aggregated summary report on the state of the industry. We will also be developing resources to help firms develop an MMM model pipeline if they have not developed that competence to date.

We plan to work with MSI members on a third validation phase. MSI members will contribute to the initiative through direct academic-member interactions. This will allow MSI to provide feedback to participants of how well their (or their vendor’s) MMM pipeline and strategy address member use cases and what, if any, changes may be desirable.

CONCLUSION

Successful Marketing Mix Model (MMM) implementation puts demands on data, models, users, and the C-suite. Innovation opportunities exist where there are gaps between the desired and actual levels.

•Data and methods need to be matched to the needs of the use case.

•Models need to be validated according to scientific standards, consistency with marketing principles and econometric standards for evaluating predictive accuracy.

•Managers need to appreciate the strengths and limitations of MMM models, especially in the face of modeling innovations and new media channels.

•Investors need to understand MMM and offer strategic guidance on the choice of KPIs.

If your firm wishes to take part in the MMM Industry Challenge, reach out to MSI Research Director Keith Smith: research@msi.org

For complete access to the MSI Blue Ribbon Panel Report: Charting the Future of Marketing Mix Modeling Best Practices, contact our membership team at msimembership@msi.org.

For additional MSI resources on this topic available to the public, see below.

• Marketing Return on Investment: Seeking Clarity for Concept and Measurement

• Drivers of Metric Use in Marketing Mix Decisions: An Investigation across the G7, BRIC, and MIST Countries

• Executives, Investors, and Academics Assessments of Marketing Performance: Trade-offs between Metric Type, Uncertainty, and Performance

• Close Enough? A Large-Scale Exploration of Non-Experimental Approaches to Advertising Measurement

• Reconciling the Customer Equity and the Brand Equity Perspectives on the Financial Value of Marketing

ACKNOWLEDGEMENTS

The MSI Academics contributing to this report thank the following industry participants for their insights about key challenges and where the MSI MMM Industry Challenge can provide the most value to businesses:

Mike Anderson FISERV

Pablo Azar PNC

Sam Barrows AIRBNB

Neha Bhargava META

Rob Cederbaum AT&T

Linsha Chen AIRBNB

Karen Chisholm PERNOD RICARD

Scott Collins WAYFAIR

Rob Corbin WAYFAIR

Casey Cowgill GOOGLE

Martyn Crook

UNITEDHEALTH GROUP

Tina Daniels GOOGLE

Asaf Davidov NETFLIX

Agustin De Dios Iglesias COLGATE-PALMOLIVE

Shaun Desmond NIKE

Kristin Federico VANGUARD

Saleel Gadgil DELL

Pierre Gardan DOMINO’S

Matt Gray ASTRAZENECA

Zhiying Gu AIRBNB

Michael Guber AT&T

Matthew Hanlon NIKE

Ryan Hays ADOBE

Lea Hinderling GOOGLE

Wayne Huang NETFLIX

Matt Kane NIKE

David Kaul GOOGLE

Divya Kaur KINESSO

Suki Lau WAYFAIR

Christina Liao FISERV

Ross Link

MARKETING ATTRIBUTION LLC

Stephen Mangan GOOGLE

Satya Menon KANTAR

Russ Messner VANGUARD

Tony Michelini CITI

Linette Mookanamparambil AT&T

Patrick Moriarty KANTAR

Jessica Nguyen META

Carter Noordsij WAYFAIR

Greg Pharo COCA-COLA

Connor Richmond WAYFAIR

Michael Schoen TRANSUNION

Emiko Seale TRANSUNION

Sandipan Sinha

COLGATE-PALMOLIVE

Samuel Sokolowski DOMINO’S

Jim Spaeth SEQUENT PARTNERS

Alice Sylvester

SEQUENT PARTNERS

Marc Vermut TRANSUNION

Jing Wang VANGUARD

Uri Weg PINTEREST

Hannah Wilder DOMINO’S

Helen Wolf

COLGATE-PALMOLIVE

Brock Wright PERNOD RICARD

George Wu AT&T

The MSI Academics would also like to thank Keith M. Smith, Research Director, Marketing Science Institute, Scott McDonald, President & CEO, Advertising Research Foundation, the MSI and ARF staff, and attendees at the MSI Summit at UCLA in February of 2023, and the MSI Analytics Conference at the University of Pennsylvania in May of 2023.

REFERENCES

1 Li, H. (Alice) and P.K. Kannan (2014), “Attributing Conversions in a Multichannel Online Marketing Environment: An Empirical Model and a Field Experiment,” Journal of Marketing Research, 51(1), 40–56. https://doi.org/10.1509/jmr.13.0050

2 Little, John D. C. (1975), “BRANDAID: A Marketing-Mix Model, Part 1: Structure,” Operations Research, 23(4), 628–655. https://doi.org/10.1287/opre.23.4.628

3 Boulding, William, Enkyu Lee, and Richard Staelin (1994), “Mastering the Mix: Do Advertising, Promotion, and Sales Force Activities Lead to Differentiation?” Journal of Marketing Research, 31(2), 159–172. https://doi.org/10.2307/3152191

4 Link, Ross (1995), “Are Aggregate Scanner Data Models Biased?” Journal of Advertising Research, 35(5), RC8–RC8.

5 Dekimpe, Marnik G. and Dominique M. Hanssens (1995), “The Persistence of Marketing Effects on Sales,” Marketing Science, 14(1), 1–21. https://doi.org/10.1287/mksc.14.1.1

6 Mela, Carl. F., Sunil Gupta, and Donald R. Lehmann (1997), “The Long-Term Impact of Promotion and Advertising on Consumer Brand Choice,” Journal of Marketing Research, 34(2), 248–261. https://doi.org/10.1177/002224379703400205

7 Spaeth, Jim, and William Cook (2001), “Maximizing Marketing ROI: Final Report,” White paper by American Productivity & Quality Center and ARF.

8 Van Heerde, Harald. J., Carl F. Mela, and Puneet Manchanda (2004), “The Dynamic Effect of Innovation on Market Structure,” Journal of Marketing Research, 41(2), 166–183. https://doi.org/10.1509/jmkr.41.2.166.28669

9 Srinivasan, Shuba and Dominique M. Hanssens (2009), “Marketing and Firm Value: Metrics, Methods, Findings, and Future Directions,” Journal of Marketing Research, 46(3), 293–312. https://doi.org/10.1509/jmkr.46.3.293

10 Ataman, M. Berk, Harald Van Heerde, and Carl F. Mela (2010), “The Long-Term Effect of Marketing Strategy on Brand Sales,” Journal of Marketing Research, 47(5), 866–882. https://doi.org/10.1509/jmkr.47.5.866

11 Neff, Jack (2022, October 6), “Adobe Uses AI to Speed Up Marketing Mix Modeling,” Ad Age. https://adage.com/article/data-driven-marketing/adobe-uses-ai-speedmarketing-mix-modeling/2439531

12 Zappa, Joseph (2022, November 7). “Keen Brings SaaS to Marketing Mix Modeling,” Street Fight. https://streetfightmag.com/2022/11/07/keen-brings-saas-tomarketing-mix-modeling/