8 0 fb summit berlin tests for success

Tests for success.

Experimental design and online conversion tracking

Nik Shah, May 2014

Nik Shah EMEA Retail Measurement Lead

CTR is not a key measure

“There is no correlation between CTR and ROI%” Nielsen NetEffect meta analysis

Correlation = -.07%

Source: Nielsen, Oct 2011

Moving towards a multi-touch attribution model A last-touch attribution model attributes entire value of a conversion to the last ad clicked or seen

✖ Miscalculates ROI by assuming the consumer doesn’t see or click any other ads along the way

A multi-touch attribution model attributes value to observed touch point along the path to conversion

Provides a more realistic assessment of ROI by capturing a holistic picture of the customer journey across channels

Last Click Attribution First Only Prefer First

Facebook undervalued by

12% to 30%

Divide Equally

using Last Click

Prefer Last U-Shaped

Last Ad measurement undervalued Facebook adver4sing rela4ve to each of the five alternate a:ribu4on models.

Source: Quantifying the Impact of Multi-touch Attribution across hundreds of millions of clicks, Retail and FinServ verticals Kenshoo Research, August 2013

More people using more devices

> 60% of people in the UK use at least two devices every day and over 20% use three devices

> 40% of all online adults sometimes start an activity on one device and finish it on another

Multi-Device Usage Study by GfK, Nov - Dec 2013 (commissioned by Facebook). Survey of 2,018 UK online adults.

Even Multi-touch misses the mobile opportunity Last Click

Multi touch

Multi device

Custom Audiences

Experimental design helps cut through this jungle Nov

Nov

16

17

Usage

36% 64%

desktop

mobile Wiggle Events Calendar Wiggle Super Series

Blog ‘help make London a safer place to cycle’

` Gsearch ‘Evans ride it’ Evans ‘Find a ride’

Nov

19

Blog ‘The Right of Way’

‘London to Paris bike ride

Blog ‘The Right of Way’

6h 01m

19

0.4

15.5

Time per day

Categories visited

Searches per day

Social media visits per day

A typical medical trial Pre

Post

Treatment

P Placebo

Essential elements of a lift-test Create control group

Run campaign to test group

Measure Sales among both groups

€€€€€

Test group

€€ Control group

Ways to create a control group Custom Audiences FB holdout functionality Test group

Remember: Randomisation Statistical power Control group

How to run a campaign

Bid Type

Sufficient reach

Other media

Ways to link sales to exposure

€€€€€

FB conversion pixel

FB mobile app SDK

€€

Direct sharing of anonymised transaction information

Online/Multichannel conversion measurement

SDK

1 CLIENT: Identify target audience, e.g. using Custom Audiences

2 FB: Impose holdout on target audience and run ads only at test group

3 FB/CLIENT: Conversion data passed to FB either via SFTP, conversion pixel or SDK

4 FB: Differences in conversion between test and control groups are analysed and uplifts reported

Key takeaways

Comprehensive cross-device data is needed for proper attribution

To measure true incrementality, you need experiments

FB has great experimental design tools – come use them!