Delivering Big Data ’s Potential: A Marketer’s Primer

Delivering Big Data’s Potential: A marketer’s primer Bryan Smith, PhD Director Marketing Strategy & Analysis, SapientNitro San Francisco Bryan is a computational research scientist with 15+ years of experience uncovering insights buried in massive data.

You’ve heard it at conferences, read about it the trade rags, and followed the debate on your favorite blogs – Big Data is here to intuit your customer’s most intimate thoughts, predict his/her behavior, and solve all your marketing problems.

Odds are, your organization has spent significant time, money and effort debating Big Data opportunities, conducting workshops, developing proof of concepts, selecting vendors, and hiring (or at least attempting to hire) a team of data scientists and engineers.

With a bit of luck, these efforts have built your capability to tackle the first three Vs of Big Data: Volume (using larger data sets), Velocity (acting on signals more quickly), and Variety (leveraging diverse data sources).

The ROI deficit is partially explained by obvious factors identified by Gartner and Wikibon: immature technology, lack of skilled data science resources, and poorly defined business use cases.

But, what about the all-important fourth V… where is the Value the Big Data hype promises to deliver?

According to recent research1,2 the average return per dollar invested in Big Data to date is just 55 cents, yet investments in data-driven decision making are projected to provide larger ROI than many other marketing investments. So what’s going wrong? What actions can you take to deliver on Big Data’s potential?

However, in our work with clients, we’ve observed some more basic pitfalls – poor communication between marketing and data science teams, a dearth of understanding of one another’s perspectives, and hit-or-miss methods for commissioning and managing data science work. This article hopes to address these issues by unpacking basic concepts for marketers to communicate with data science professionals and practical advice on managing data science efforts to yield maximal returns.

1 2

Enterprises Struggling to Derive Maximum Value from Big Data: http://wikibon.org/wiki/v/Enterprises_Struggling_to_Derive_Maximum_Value_from_Big_Data Gartner Survey Reveals That 64 Percent of Organizations Have Invested or Plan to Invest in Big Data in 2013: http://www.gartner.com/newsroom/id/2593815

129

delivering big data’s potential: A marketer’s primer

Learning to Communicate: A Data Scientist’s Glossary We’ve developed the following functional glossary to help demystify some of the common terms used in data science practice, connecting them to some practical ways we use them in our work.

Big Data comprises both a technology stack component and the machine-learning algorithms running on top. Given that Big Data core technology stacks are being increasingly commoditized (e.g. Cloudera, MapR, HortonWorks, etc.,) our focus here is on the machine learning aspects of the broader Big Data domain.

Machine Learning Machine learning is the process of training computers how to “learn” interesting patterns in data and how to adapt automatically when data change over time. We feed the machine-learning algorithms data at training time to build models, and those models are then used to inform marketing and operations decisions, ideally in an adaptive manner. It comes in two different flavors: supervised learning and unsupervised learning. These two methods differ in the data required to make them work and the insights that they yield.

Applying Big Data in Retail: Visualizing Cross-Category Sales Activity

The images show a comparison to two cross-channel sales scenarios derived from clickstream data. The image on the left represents relatively low cross-channel sales, while the image on the right represents relatively higher cross-channel sales, for example, after optimizations were made to the site.

This interactive tool – based on terabytes of data – can identify categories which retailers might want to cross-promote which are not intuitive. For example, using this method, we could discover correlations between buyers who purchased tent hardware, and, who also purchased batteries. A secondary benefit is that these data unlock opportunities for brands to offer additional advertising slots to buyers.

delivering big data’s potential: A marketer’s primer

Fig. 1 Supervised Learning Training (old data)

Predicting (new data)

algorithm

input

measurements/output

model

model

input

estimated outputs

Supervised Learning Supervised learning algorithms require both input (e.g., user clickstream) and output (e.g., user purchases) data, commonly referred to in statistics as the independent and dependent variables, respectively. The general purpose is to create a model for how the output relates to the input. Once we have a “good” model based on past clickstreams and purchases, we can employ that model to estimate purchase propensity using input data from new users.

Supervised Learning Use Cases Applying Supervised Learning algorithms to marketing problems can generate powerful predictive models, which can be used and refined over time. These models can generate deep insights and guide marketing strategy, and also allow you to test ‘what-if’ scenarios.

1. Conversion probability: How likely is a specific shopper to purchase based on user data (demographic, geographic, social media profile) and purchase history?

2. Click-through rate prediction: What are the estimated click-through rates on new content, based on user profile data, time of day, product category, content publisher identity, and historical click-through data?

3. Churn probability: What are the online behaviors which best predict the loss of a customer? Or, conversely, what are the characteristics of customers most likely to stay with your business?

4. High-value user acquisition: Who are the most valuable customers over the long-term to your business? Marketers can then apply these characteristics to shape marketing strategy as part of a new user acquisition campaign.

Technical Deep-Dive: Two Explanatory Models Algorithm #1 “Classifier” A classifier is a supervised-learning algorithm concerned with assigning data to one or more pre-defined categories. It is particularly useful for interpreting data with discrete categories – such as purchased or not purchased, or click or not clicked.

In the example above, we defined two specific categories, “purchase” or “no purchase,” which may be encoded as numeric values of 1 and 0, respectively.

We would use a classifier to understand how the data we’ve collected relate to these two distinct “classes.”

If certain combinations of user demographic data, marketing channel identity, and content relevance are correlated with conversion probability, we may be able to discover those relationships using this type of model.

Algorithm #2 “Regression” Regression models are used when we want to understand “continuous” outputs, such as the actual amount of money a customer is likely to spend over their lifetime. So, rather than the discrete “purchase” and “no purchase” categories described in a classifier, a regression model can be used to estimate how combinations of data may be predictive of actual dollar amounts spent by your users. Regression models are also supervised, meaning we need to train them using historical examples of both the input data as well as the actual purchase data for the same set of users.

delivering big data’s potential: A marketer’s primer

Fig. 2 Unsupervised Learning Training (old data)

Applying (new data)

algorithm

ungrouped input

model

grouped output

unknown input

group assignment

model

Unsupervised Learning Unsupervised learning, often called data mining, occurs when we are concerned with understanding patterns and relationships within the input data itself, regardless of any particular output. Given that we do not need to use any particular output data to train these kinds of models, unsupervised learning may seem like the path to magical customer insight. In practice, successful application of these methods requires clearly defined questions and hypotheses in order to generate value.

Unsupervised Learning Use Cases Applying unsupervised learning models to marketing problems generate different types of insights – namely, they don’t try to determine causality, but rather they only generate characteristics of the segments.

1. Bottom-up segmentation: Identify naturally occurring “clusters” of users within the data; key inputs are data about users and their behavior. This information can be used in targeting campaign efficacy assessment, etc.

2. Product recommendations: Suggest products that are likely to be enjoyed by a user by using collaborative filtering (a variant of pattern mining) combined with user preference or ranking data.

3. Experience personalization: Establish user personas using clustering algorithms applied to user data, to be employed in personalization strategies.

4. Dynamic site optimization: Dynamically optimize a site experience by using frequent pattern mining combined with clustering algorithms, taking into account both the user’s current clickstream behavior as well as a persona grouping to which that individual belongs.

5. Cross-Category promotions: Using basket analysis (frequent items mining), identify items or item categories that often occur together in transaction data. Use the resulting information for promotions, sales and inventory management optimizations.

delivering big data’s potential: A marketer’s primer

Technical Deep-Dive: Two Explanatory Models Algorithm #1 “Cluster Analysis” Clustering algorithms are used to group objects, such as users or products, into clusters (or segments) based on some notion of similarity. Whereas top-down segmentation as executed by a marketing expert enables precise targeting, there is also significant value in understanding the naturally occurring groupings (personas, user archetypes, etc.) in a user database. For this type of exploration, we rely on cluster analysis.

Algorithm #2 “Frequent Pattern Mining” Market basket analysis, likely a concept many readers have exposure to, is a type of frequent pattern mining. The famous (though likely apocryphal) association between beer and diapers in retail consumer’s carts is an illustrative example of using market basket analysis to identify product associations that are not necessarily intuitive.

Collaborative filtering algorithms, used by Netflix recommendations and Amazon’s “users who bought X also bought Y” are other examples. It is worth noting that many of the algorithms in a data scientist’s toolbox may be described in terms of “frequent pattern mining,” but the above examples define a general distinction.

Putting the Models to Work: The Intersection of Business and the Data Scientist Now that you are armed with a knowledge of some commonly used vocabulary of the data scientist, you should understand some differences in perspectives between typical marketing resources and the data scientist.

Marketers are familiar with the world of creatives and creativity. The cadence of data science work does have some elements in common with creative work. Data science is more ambiguous in nature and quality results may take longer to develop than initially anticipated. In fact, in the same way that a creative team needs a clear brief, data science teams need concrete questions provided by the marketer to direct their efforts. Left without direction, or with a vague mandate to go produce some insights, we’ve seen data science teams work feverishly for months, only to produce insights with limited practical utility.

Unlike creative however, data science also has elements of scientific research, which makes progress through experimentation. Carefully designed experiments typically prove or disprove hypotheses, but can also sometimes be inconclusive. A series of false hypotheses or failed experiments can lead to dead ends and the need to backtrack. It may seem from the perspective of the marketer that the data science team is flailing, not making progress at all, when in reality this iterative process may be a normal progression as these methods are initially adopted. While a deadline on a creative team may work to fuel the creative process, a deadline on a data science team will only limit the time taken on a set of experiments, which may not guarantee better insight or deliver a business result more quickly.

delivering big data’s potential: A marketer’s primer

Essential Marketing Questions to Ask Your Data Team 1. What attributes define my most valuable customers?

Best Practices in Managing the Data Science Team How do you work with data professionals in your organization? To answer the question, let’s take the hypothetical example of a brand that needs a datadriven cross-category sales and promotions retail strategy. Here are some best practices:

2. What customer segments am I most likely to lose over the next 3 months?

3. What features of my product resonate with various segments of my user base?

4. How are my marketing campaigns being received in social media?

5. Are there identifiable differences among various groups who like or dislike my products?

6. Why was the click-through rate of my most recent campaign so high/low?

1. Ask specific questions: Being clear about the expected outcomes of your data science efforts may seem obvious, yet too often we see insufficient time devoted to identifying and prioritizing the questions for the data science team.

Leveraging data to optimize cross-selling activities requires a careful consideration of factors ranging from your marketing mix to your supply chain. All of these complex systems and more may come into play. Left undirected, a data science team may decide to tackle any or all of these systems that may not align with your marketing objectives. Questions that seem related to the general problem may in fact require distinct approaches to research and answer.

In the context of our example (a retail brand driving cross-category sales and promotions), you may want to specify questions including: • What categories occur most often together in user’s carts?

7. Can I move stale inventory by identifying cross-promotion opportunities from existing user behaviors?

8. Are there any readily-identifiable common attributes of my most successful salespeople?

• Is there any geographic variability with category associations?

• How can we measure cross-category promotions activity across geographies?

• Are there any additional specific sources of data we can integrate to better inform our models?

• What categories of items show up consistently together organically (in the absence of any known cross promotions)?

delivering big data’s potential: A marketer’s primer

2. Create a sense of urgency: Our earlier comments about data science being like research are not meant to imply that an approach to Big Data should come with no deadlines and open wallets.

We’ve found an effective technique for creating a sense of urgency to meet deadlines is to create a backlog of well defined and time boxed “experiments,” in which the expected results are well understood by both the marketing and the data science team. Setting specific objectives and timelines around data source identification, data access processes, and information security issues should be included.

For our hypothetical situation, we will need to know how to access historical sales data, promotion activities, and product catalogue data. With data in hand, we need a simple analysis of item and category co-occurrence. Given our understanding of this initial model, we need to make decisions about what specific questions to approach next.

The ideal length of time per experiment is four to six weeks, seldom exceeding eight weeks.

Be prepared for failure. Expect it. The goal is to learn, acknowledge failures, and reprioritize the backlog. This approach borrows in many ways from Agile software development.

3. Define the owners: For the questions your teams will be answering, both within your team and the data science team.

• Who is the ultimate owner?

• Who owns data provisioning and access?

• Who will use the results of the cross-category analysis?

• Is this effort intended for the marketing team, promotions, purchasing, or someone else?

4. Create an integrated view of your data science efforts, and keep control of your information: Maintaining access to your information and your semiprocessed data is not just a matter of IT security. Rather, it’s about ensuring efficient access to the types of data that will be required to generate insights you seek.

Due to hype, available options and vendor proliferation, organizations are outsourcing not only the collection and storage of their data, but are giving up access to raw data streams.

The perceived convenience and initial cost savings of leaving your data collection and management processes to third-party vendors could end up proving costly in the long run, as demands for those data evolve over the lifetime of an organization. Simply, these pitfalls can be avoided, and outsourcing benefits fully realized, by involving data science and governance in any decisions where data will be housed outside the enterprise.

The bond between marketing and data science has deep roots. Direct marketers have leveraged data science for decades.

But digital media has changed the dynamics of this partnership – offering massive data streams that promise to provide insight into not only what customers are saying and doing, but potentially also what they are thinking.

As George Dyson asked and answered, “But, how can the machine know what I think?” The machine does not need to know what you think – no more than one person ever really knows what another person thinks.

A reasonable guess at what you are thinking is good enough. That’s the promise of data science for marketers. Let’s work on that ROI.

• Are there multiple consumers of the insights? If so, does the output need to be specifically crafted for these various users?

This is a critical stage of the process to actively manage and assess, with the risk being the possibility that brilliant insights are being delivered into a void of inaction.

135