Test and Learn Final Analysis

Page 1

Taubman /!measurement report! CLT Draft!


agenda! •

measurement strategy!

digital channel overview!

survey recommendations!

usage by channel!

value of shopper by channel!

social recommendations!

social analysis!

sms/push messaging!

roll out strategy / plan!

!

! ! 2!


overview!

3!


measurement strategy / test & learn! Business Objectives"

Drivers of measurement strategy!

Drive Traffic (Local & Tourism)

Induce Trial of Stores

Measurement Plan"

Business Forecasting"

Engagement

(visits, sales & advocacy)

Advocacy

Channel Improvement

Visits & Intent to Visit

ROI

Key Performance Indicators!

Channel & System ROI!

Channel Value

Share News Purchases & Intent to Purchase Enhance Shopping Experience

Add Digital Benefits to Sponsorships

Favorability

Number of Stores Visited

4!

Klout Score &! Alexander Babbage! Study!


measurement / digital channel overview! Channel KPIs

Engagement!

Control Groups

Desktop Site

Mobile Site

Registration & Sign-in / Favoriting / Specific content views / Return visits / Communication opt-ins!

Advocacy NPS Current Content sharing! (likelihood to Shopper ! recommend)! Survey & Site Survey! Monthly Center Visits & Intent ZIP Code! to visit! Survey! Purchases & ! Intent to (Baseline)! purchase Desktop Mobile Site Site Survey! Favorability Survey! (brand affinity) Number of stores visited

Social Sites

Email

Facebook fan activity/active users! Interaction rate, demographic info! ! Tweet volume/sentiment, ! fan/follower growth, ! unsub/unfollow rate! Retweets, Facebook posts, sentiment, usage of share within wall posts, Facebook activity! ! Survey!

Twitter

Opt-ins!

Tweets, Open rate! Retweets! CTR’s!

Content Sharing!

SMS

Opt-ins, CTR’s!

Content Sharing!

Facebook survey! Twitter link to site survey! Survey!

Facebook survey! Twitter link to site survey! Sentiment, Facebook survey! Twitter link to site survey! Facebook survey! Twitter link to site survey!

5!

(Smartphone only)!

Email Survey!

Twitter Survey!


measurement / digital channel universes! Universe!

Desktop! Monthly unique visitors!

Email!

Facebook!

SMS!

Twitter!

E-Mail receipts!

Fans!

Messages sent!

Followers !

Universe total!

193,876!

TBD!

41,086 !

468*!

878!

Survey respondents!

1,563!

180!

166!

19!

17!

Response rate!

0.8%!

0.4%!

4%!

1.9%!

Each mall and each channel provided an identical survey:! ! •  Control: Mall intercept! •  Desktop – Site survey link on websites.! •  E-Mail – Take our survey featured as part of e-mails.! •  Facebook – Facebook survey tab and wall posts.! •  SMS – Survey text messages.! •  Twitter – Tweets of links to survey on websites.! *Overall, 921 text messages were sent, but in two rounds (453 and 468) with substantial recipient overlap.!

6!


recommendations!

7!


survey recommendations! Invest in digital. For an initial roll out, leverage the consistencies below while being sensitive to the nuances observed at the center level. As a follow up to this initial test and learn, expand program so to gain incremental learning that will account for the sub segment variances across centers ultimately driving smarter investment and engagement decisions.! !

Observation! Comparing control and test, age and income are similar. The test group (digital channels) skews more toward a female audience than the control. ! Channel usage is similar between groups – most go to the desktop website to find information about their center, followed by facebook and email respectively. Digital consumers have higher digital penetration than control.!

Recommendation! Ensure that digital messaging, content and tone map back to the core demographic.!

!

Invest in digital, where it matters. Focus on website, facebook and email. Connect experiences across all channels. !

Digital respondents will spend 2.7 times the amount of control in the next 30 days. They will also visit more frequently and have higher NPS.! !

Participate in channel allocation (scorecarding) exercises to ensure dollars are allocated against channels and tactics that provide the highest ROI while keeping true to goals and objectives.!

Digital consumers’ website experiences could be enhanced by offering more sales, special offers and sweepstakes on site.! !

Digital consumers spend more. Consider promos and incentives to drive even more engagement and loyalty. Cost will be absorbed by increased frequency and spend. !

8!


survey analysis!

9!


measurement / responses by center! Control!

Test! 41%!

28%! 24%!

23%!

21%!

25%!

22%!

9%!

Beverly Center!

Mall! Beverly Center! Dolphin! Partridge Creek! Twelve Oaks! Total!

Dolphin!

Partridge Creek!

Control! Desktop! Email! Facebook! SMS! 525 ! 159 ! 26 ! - ! 19 ! 451 ! 810 ! 55 ! 23 ! - ! 504 ! 392 ! 104 ! 104 ! - ! 538 ! 378 ! 56 ! 45 ! - ! 2,190 ! 1,739 ! 241 ! 172 ! 19 !

Twelve Oaks!

Twitter! Test Total! 4! 208 ! - ! 888 ! 13 ! 613 ! 2! 481 ! 19 ! 2,190 !

In total, sample size across control and test are similar. Control respondents are equally distributed across centers, while test respondents skew Dolphin, Partridge and Twelve Oaks. Beverly Center seems to be less digitally engaged than others. ! 10!


measurement / respondent demographics: control and test! launch – May 31, 2012!

Income Distribution 30%!

Control!

25%! In total, income age and gender patterns are similar between test and 20%! 15%! control.! •  Income is slightly higher for control ! 10%! 5%! •  Both groups skew female; test 0%! more! •  Both groups median age is mid 30’s!

Gender Distribution Male!

Test!

Medians: Control=$71,517 Test= $68,182

Age Distribution Female!

30%!

Control!

Test!

Medians: Control = 33 Test = 36

25%!

Test!

22%!

78%!

20%! 15%! 10%!

Control!

37%!

63%!

5%! 0%! <18!

11!

18-24!

25-34!

35-44!

45-54!

55-64!

65+!


digital media usage over time"

Time Periods: Jan 16 – Jan 31; Feb 14 – Feb 29; Mar 10 – Mar 19; April 6 – April 19; May 1 – May 9, 2012 !

Website via PC is has the highest penetration, followed by facebook and Email. Emerging channels (mobile, twitter and check-in) have increased since launch of survey.! 12!


digital media usage over time: cross channel Launch – May 31, 2012! %!"!#$

-+9./66:$

5;<=.1$

>?.9:$<,$

5.@7AB0B$

>6C3D7.1$

BC+173?6,.$A$5+/8.7$

EC+<8$

)%"!#$ )!"!#$ (%"!#$ (!"!#$ '%"!#$ '!"!#$ &%"!#$ &!"!#$ %"!#$ !"!#$ *+,$

-./$

0+1$

231$

0+4$

567+8$

Website via PC, facebook and Email are the most popular channels for control respondents as well. Emerging channels appear to be flat since launch. Test respondents have higher digital channel penetration than control (e.g. Website via PC control = 13% ; test = 38.5% in current period)! 13!


channel value / digital channel scorecard: cross channel! control vs. test! KPIs!

Segment!

Sample Size!

Sample Size!

Engagement!

Advocacy! # of Stores Visited!

Amount Spent in Last Visit!

Intent to Revisit!

Value per Shopper !

!

# of Food, Net # of Dept Merchand Total Specialty Restauran Brand # of visits Spend in Digital Promoter Stores ise and Amount Stores ts, and Preference in next 30 next 30 Media Score visited Services Spent in visited Entertain(Q8)! days (Q5)! days**! vs. Total! (Q6)! (Q3)! (Q2)! Last Visit! (Q4)! ment (Q1)!

Control!

N=2,018!

4.14 !

37.90%!

1.15!

3.5!

$204.87 ! $21.36 ! $226.23 !

2.7!

$610.82 !

1.0X!

Test!

n=2190!

4.00 !

42.90%!

3.4!

5!

$435.10 ! $56.79 ! $491.89 !

3.4!

$1,662.48 !

2.7X!

Test v. Control!

8.52%!

-3.50%!

11.66%! 66.18%! 30.00%! 52.91%! 62.39%! 54.01%! 20.59%!

63.26%!

Digital consumers have a slightly lower brand preference yet are slightly more likely to recommend their center than control. Digital consumers are far more engaged than control respondents (+~70% / +30% more stores visited ; +54% more money spent ; + 21% more stores planned in next 30 days). Digital consumers are also more valuable control (likely to spend 2.7x more money in the next 30 days). ! 14!


channel value / digital channel scorecard: control that connect digitally! KPIs!

Measurements!

Control!

Sample EngageSize! ment! Sample Size!

N=2,018!

Connect with TTC via PC n=279! website! Connect with TTC via n=191! Facebook! Connect with n=130! TTC via email!

Advocacy!

# of Stores Visited!

Amount Spent in Last Visit!

Intent to Value per Revisit! Shopper !

Food, Total # of Merchandi Restaurant Amount Specialty se and s, and Spent in Stores Services ! Entertain- Today’s visited! ment! Visit!

!

Brand Preference!

Net Promoter Score!

# of Dept Stores visited!

# of visits Digital Spend in next in next 30 Media 30 days**! days! vs. Total!

4.14!

37.90%!

1.15!

3.5!

$204.87 ! $21.36 ! $226.23 !

2.7!

$610.82 ! 1.0X!

4.35!

41.40%!

1.42!

3.79!

$210.20 ! $37.62 ! $247.82 ! 3.79!

$939.24 ! 1.5X!

4.37!

52.40%!

1.25!

3.39!

$161.67 ! $19.34 ! $181.01 ! 4.41!

$798.25 ! 1.3X!

4.24!

37.20%!

1.3!

2.92!

$169.42 ! $35.19 ! $204.61 !

$818.44 ! 1.3X!

4!

Control respondents who also engage with the top 3 digital channels for their center spend 1.3x to 1.5x more than those who are not digitally engaged.!

15!


measurement / digital channel scorecard: cross channel detail! ! Value Food Chain KPIs!

Sample Size!

Measurements!

Sample Size!

Total!

n=2190!

n/a!

4.0!

n/a!

42.9%!

82%!

3.4!

5.0!

$435.10 !

$56.79 !

$491.89 !

3.4!

$1,662.48 !

Desktop!

n=1739!

n/a!

4.0!

n/a!

38.1%!

81%!

3.7!

5.3!

$460.83 !

$60.64 !

$521.47 !

3.4!

$1,792.64 !

Email!

n=241!

n/a!

4.2!

n/a!

64.7%!

89%!

2.8!

4.3!

$536.12 !

$54.29 !

$590.41 !

3.3!

$1,958.30 !

Facebook!

n=172!

n/a!

4.1!

n/a!

59.3%!

81%!

2.2!

4.0!

$200.03 !

$40.67 !

$240.70 !

3.0!

$732.27 !

SMS!

n=19!

n/a!

3.5!

n/a!

-13.3%!

93%!

1.4!

2.5!

$92.38 !

$54.14 !

$146.52 !

3.3!

$488.39 !

Twitter!

n=19!

n/a!

4.4!

n/a!

50.0%!

85%!

1.7!

3.8!

$100.07 !

$21.00 !

$121.07 !

2.6!

$310.23 !

Engagement!

Advocacy! Favorability!

# of Stores Visited!

Amount Spent in Last Visit!

Enhances Merchandis Food, Actions Brand Net # of Dept # of Specialty shoppers e and Restaurants, Engagement (Avg. Preference Promoter Stores Stores visited Score! experiences Services and EntertainTAT)! (Q8)! Score (Q6)! visited (Q3)! (Q4)! (Q9)! (Q2)! ment (Q1)!

Intent to Value per Revisit! Shopper !

Total # of visits Spend in Amount in next 30 next 30 Spent in days (Q5)! days**! Last Visit!

Desktop web dominate in terms of number of responses. NPS is 42.9% across channels. This is slightly above the control. Email, Twitter, and Facebook skew higher, with SMS skewing lower. On average, consumers will spend just over $1,660 in the next 30 days, well over double that of the control. Email consumers are estimated to spend the most; Twitter the least.!

16!


measurement / digital channel scorecard: cross channel detail! ! How could we make your visit better? 36%!

13%!

•  Sales and special offers are both the top reason for visiting and primary way to make the respondent’s visit better.!

Offer fashion trends; runway videos; hottest gift items!

14%!

More information on events and latest happenings!

More information on new stores/ restaurants!

18%!

Contests, sweepstakes or promotions!

More sales and special offers!

19%!

•  Driving directions / center information and coupons or promotions also top the list.!

Primary Reason for Visit 42%!

26%! 21%! 11%!

To hear about sales and special offers!

To get general To stay current on To hear about new information like events and latest stores/restaurants! driving directions, happenings! store listings, center hours, etc!

17!


social content analysis!

18!


social content recommendations!

Each center’s audience engages with content in different ways, and each center has different offerings that affect the content published on its channels. There were, however, several instances of commonality where optimizations can be made.! Observation! Recommendation! ! On average, seasonal/holiday/occasion posts have higher engagement rates !

Increase amount of seasonal posts at every opportunity to drive engagement!

Based on engagement, users prefer a more even distribution of content than what was actually published!

Distribute content more evenly to improve engagement rates for all content types!

The visual fashion posts are a point of differentiation from competitors!

Publish fashion content and take advantage of FB features: Timeline, polls!

Beverly Center and Dolphin Mall had the highest engagement rates !

Continue using 1st person singular which directly impacts engagement!

Successful competitor Twitter accounts retweet/reply 5-6 times/day on many topics to add followers!

Actively tweet to check-ins, RT retailers posts – particularly giveaways/sweeps!

SMS / Push messaging yields lack luster results/ It reaches a small audience and may lead to consumer dissatisfaction.!

Discontinue SMS/Push messaging at this time.!

19!


social rollout / toolkit recommendation! Option 1!

Price/Month

Main Features •  Scheduling, monitoring +

Hootsuite Enterprise

$2500 - $3500!

managing on all FB and Twitter accounts! •  Basic reporting on Page performance and clicks!

Benefits •  Manage social publishing for all

centers from one place! •  Various levels of admin access

and moderation! •  Snapshot reports for growth

and key influencers! !

Option 2: Option 1 + the following:! Simply Measured

$1750!

•  Custom report templates! •  Automatic updates! •  Aggregate FB and Twitter

data! •  Keyword monitoring!

•  Location-based monitoring

Momentfeed

$1800!

and reporting! •  Create groups for Locations! •  Location ranking! •  Real-time maps!

20!

•  Compare across centers +

! competitors! •  ! Produce reports quickly and ! efficiently! !•  See page and post-level performance!

•  Insight into people checking

in at centers on 4sq + FB! •  Own locations + delete

unofficial ones! •  Add tips, photos, etc. to locations!


facebook / content recommendation! ! The tables outlines the recommendations for optimizations and the corresponding actions/tactics!

Actions! Post Frequency!

•  Continue posting once per day, six times per week. Total post count per month will be 25 posts.!

Post Timing!

•  •  •  •

Post Distribution!

•  See Slide 10 for tactical breakout/schedule!

Voice!

•  Write in 1st Person Singular for best engagement rates!

Beverly Center: Publish all posts between 7pm and 9pm! Partridge Creek: Publish all posts between 2pm and 3pm! Twelve Oaks: Publish all posts in the hour of 4pm! Dolphin Mall: Publish all posts in the hour of 1pm!

21!


facebook / content recommendation! For the centers, in addition to the post distribution table:! Each week, 1 of each post in each content bucket should be “Push” and the other should be “Conversational”! !

Content Bucket!

Week 1! Week 2! Week 3! Week 4!

Totals by bucket!

% of Posts by Content Bucket!

Sales/Events!

2!

2!

2!

2!

8!

32%!

Fashion!

2

2

2

2

8!

32%!

Seasonal!

2

2

2

2

8!

32%!

Alerts!

0

0

1

0

1!

4%!

Totals by Week!

6!

6!

7!

6!

25!

100%!

24%!

24%!

28%!

24%!

100%!

% of Posts!

22!


facebook / content recommendation! Use Facebook Timeline Visual and Promotion Features! •  Pin one post a week to feature at the top of the page: It can be a big sale or event, a red-hot fashion trend or an alert. (Pinned posts stay at the top of the page for one week)! •  Highlight special posts – such as movie premieres, big events or a popular fashion item 2-4 times per month to create strong visual elements (Highlighted posts span the width of the Facebook Page)! •  Competitors are not currently utilizing these features – it will be a point of differentiation! ! !

23!


twitter / content recommendations! Organic growth is slow but, as in the case of the centers, the growth rates are compounding over time. Proactively engage in the channel to add followers.! To add more followers, faster:! •  Retweet content from the retailers in the center, such as their sales, promotions and special events. They will then know about your Twitter account and may start tagging you in tweets to increase your exposure! •  Engage multiple times per day: According to Alexander Babbage Q4 2011 Report, the top malls on Twitter tweet/reply 5-6 times per day on average! •  Reply not only to people using the center’s handle (ex. @BeverlyCenter) but also those using #BeverlyCenter and “Beverly Center”! •  To grow, it takes proactive seeking out of your brand on the platform, and engaging! ! !

24!


twitter / content recommendations! Organic growth is slow but, as in the case of the centers, the growth rates are compounding over time. Proactively engage in the channel to add followers.! •  Retweet content from the retailers in the center, such as their sales, promotions and special events. ! •  Engage multiple times per day: According to Alexander Babbage Q4 2011 Report, the top malls on Twitter tweet/reply 5-6 times per day on average! •  Reply not only to people using the center’s handle (ex. @BeverlyCenter) but also those using #BeverlyCenter and “Beverly Center”! !

Actions/Tactics!

!

Publish Tweets!

Publish 1 Tweet/day, on a rotation of content buckets ! (Day 1: Sales/Events, Day 2: Fashion, Day 3: Seasonal, etc.)!

Retweet and/or reply to retailers and other users’ tweets at least Repurpose Tweets! once per day and up to 5-6 times per day, depending on availability of content!

25!


sms / push messaging! •  During this reporting period we continued following our reduced frequency approach for SMS in an effort to ensure that our approach would yield lower unsubscribe rates from the program.! •  Unfortunately the unsubscribe rates increased sharply when compared to the previous reporting period from 1.16% to 2.42%. ! •  Our three messages over this reporting period featured a store opening, free gift with purchase and 30% off entire store. ! •  The store opening was for Wolford Boutique and this message received a 3.21% unsubscribe rate and three clicks to the mobile site.! •  The free gift with purchase offer was from Ben Sherman and this message received a 2.43% unsubscribe rate along with three clicks to the mobile site.! •  While the 30% off entire store offer was for Custo Barcelona. This was the best performing message with 8 clicks and an unsubscribe rate of 1.58%.! •

!

! Message Theme Send Date

Total

Total

Total

Unsubscribe

bit.ly

Completed

Send

Received

Unsubscribes

Rate

Clicks

Surveys

Wolford Boutique

4/13/12

468

453

15

3.21%

3

N/A

Ben Sherman

5/4/12

453

442

11

2.43%

3

N/A

Custo Barcelona

5/19/12

442

435

7

1.58%

8

N/A

1,363

1,330

33

2.42%

14

Grand Totals:

26!


launch plan and forecast" UNDER CONSTRUCTION!

27!


Channel Allocation Process! Leveraging our test and learn insights, we will create a multi-dimensional analysis to align channel spend with performance expectations and marketing objectives.

! ! Apply Objectives and Set Benchmarks!

Data Analysis, Forecasting, Modeling, etc.!

! The Process!

Ongoing Measurement, Attribution and Optimization!

Ideate, Evaluate and Execute!

Planning! 28!

Ideation!

Measuring!

Situation Analysis!


Details! Situation Analysis!

Ideation!

Planning!

Measuring!

Identify campaign KPI s (metrics for success) for each campaign/channel. !

Define consumer journey and relevant opportunities to reach consumers at each touch point.!

Create measurement plan and define metrics for success.!

Develop a score card which will be used to evaluate new ideas based on criteria that map back to corporate objectives.!

Leveraging the consumer journey, brainstorm list of channels/tactics to achieve performance objectives.!

Score channels / tactics and build preliminary forecast. All channels/tactics that score higher than average are in consideration set.!

Allocate dollars by channel/tactic based on contribution to conversion. !

Create test and learn agenda.!

Partner with creative, technology and media to finalize launch plan.!

Launch!

Monitor in market performance on daily/weekly/monthly basis.!

Test and Learn; optimize for top performance. Identify best creative/offer. !

Reforecast based on current results. Set guidance for total program performance.!

29!


example channel score card! ! Goal! !

!  Test channels to drive branding, awareness and engagement strategies… all leading to conversion.! !  Obtain most engagements at most efficient cost and channel mix!

!  Leveraging best practices, historical performance results, and program specific objectives, we defined a channel score card.! !  Attributes and their values / ranks are reflective of the above, and are customized to help identify the best investment opportunities to drive results.! !  The score card is created at the beginning of the campaign, and can be revised (along with associated forecast) as the campaign objectives shift and/ or as we progress through various phases.! 30!


example forecast! !  Leveraging our score card results, we have allocated at least 80% of our media dollars to channels that scored above average.! !  Note, when doing this, we have to take into account uncontrollable - inventory availability, cost constraints, and diminishing returns; which is channel specific dollar amounts may not mirror score (in other words highest scoring item may not get the most dollars as a result of said constraints).!

Invest in top scoring channels

31!


example launch Make It Real / theplan! map to scale 10/11

11/11

12/11

1/12

Awareness & Satisfaction (Holidays, Giving/Spending, New Year Goals)

PHASE 1

BUILD & SEED

2/12

3/12

4/12

Planning projects, college acceptance letters, cabin fever

5/12

6/12

7/12

Accomplishing Goals

8/12

10/12

Finishing projects, kids to college

11/12

12/12

1/13

Holidays, Giving/ Spending, New Year Goals

2/13

3/13

Planning projects, college acceptance letters, cabin fever

ONGOING CUSTOMER APPRECIATION

PHASE 2

PHASE 3

MAKE IT REAL PLANNING

MAKE IT REAL: GETTING IT DONE

PHASE 4

MAKE IT REAL CELEBRATION

PHASE 5

PERSONAL, INVESTING, BUSINESS, BORROWING

GENERAL

MEDIA

9/12

Paid Ad Buy

Paid Ad Buy

Paid Ad Buy

Paid Ad Buy

Paid Ad Buy

Paid Ad Buy

CITIZENS ONGOING COMMUNICATION STRATEGY

Wednesday, October 5, 2011

32!

6


thank you!

33!


Turn static files into dynamic content formats.

Create a flipbook
Issuu converts static files into: digital portfolios, online yearbooks, online catalogs, digital photo albums and more. Sign up and create your flipbook.