Local Insights
India Marketing Campaign Efectiveness High-Level Cognitive Report
In partnership with
Prepared by Arnabi Konar, UXE, Applause
April 2019 2019 December
Table of Contents Section 1 – Executive Summary Section 2 – Participants Section 3 – Study Results and Recommendations Section 4 – Proposed Solutions Section 5 – Appendix
Facebook Local Insights
Executive Summary Study Overview & Summary of Findings
Facebook Local Insights
Overview
This document presents the fndings and recommendations for the India Marketing Campaign Effectiveness survey cognitive evaluation which took place in November-December, 2019. Study Goals The goal of this study was to assess user's understanding of survey questions and response choices in local language.
Methodology Participants were recruited to mirror target user groups according to the recruitment profle details discussed by the Facebook and Applause teams.There were 10 participants from various corners of India who were interviewed remotely. All the participants have had been using Facebook for not more than 4 years.
Facebook Local Insights
Participants
Facebook Local Insights
Participant Demographics
Facebook Local Insights
Summary of Findings Major Findings ● 5-point option worked well for most users ● Question 5 was most relatable, both Hindi and English ● The users didn’t fnd the english translation of Q1, Q2 are relatable ● C pattern of asking the same questions were best liked ● Q3 is fne, approach doesn’t suit the FB perception of users
Facebook Local Insights
Question Library
Q1
Facebook का इसेमाल करने से आपको ककतना फायदा होता है?
How much value do you get from using Facebook?
Q2
आपके काम की जानकारी पाने मे Facebook से आपको ककतनी मदद ममलती है?
How much does Facebook help you get relevant information?
Q3
उपयोगी जानकारी पाने मे Facebook से आपको ककतनी मदद ममलती है?
How much does Facebook help you get useful information?
Q4
नई चीजे सीखने के मामले मे Facebook से आपको ककतनी मदद ममलती है?
How much does Facebook help you learn new things?
Q5
अपनी पसंद की चीजो के बारे मे जानने मे Facebook से आपको ककतनी मदद ममलती है?
How much does Facebook help you discover more about your interests?
Facebook Local Insights
Frequency Maps
A Lot
Helps a lot
A moderate amount
Helps a moderate amount
Facebook Local Insights
Helps a little
A Lot
Helps a lot
Quite a bit
Helps a moderate amount
Helps a little
Helps a moderate amount
Helps a lot
Helps a little
Helps a moderate amount
Helps a little
Study Results and Recommendations
Facebook Local Insights
5-point option worked well for most users Even people who were fne with 4 point options initially, liked 5 point better When only 4-point is shown, 4 users felt fne but later on chose their appropriate option from 5-point options. ● 5 out of 10 people instantly felt that 4 point options were not enough to express their selection ● Actual user reactions: ○
“Kuch Haad tak aur Thoda mein zyada farak nehin hai” ( Shitu Sahu, Male, 29 years, FB user for 2 years ) Translation: There’s not much diference between ‘helps a moderate amount’ and ‘helps a little’
○
“Bahut Zyada aur Kuch haad tak ke beech mein kuch hona chahiye.” ( Neha Singh, Female, 36 years, FB user for 2-4 years ) Translation: There must be some other option in between ‘A lot’ and ‘helps a moderate amount’
○ 8 out of 10 people actually fnally chose from the 5-point option variant.
Facebook Local Insights
‘Value’ and ‘Faida’ are not relatable People could not relate to the hindi and english question as the same ‘Value’ was more related to ‘Mulya’ and ‘Faida’ to ‘Benefcial’. ● 7 out of 10 people instantly felt that the two questions were diferent ● The frst-hand translation that users did was ‘ How is Facebook benefcial to you?’ ○
“Yeh alag question hai, meaning alag hai dono questions ka!” ( Rajneesh Saharan, Male, 29 years, FB user for 2 years ) Translation: These are two diferent questions altogether!
○
“Value is something relating to adding worth to life and I don’t feel I relate to FB that way” ( Jyoti Bhatnagar, Female, 36 years, FB user for 2-4 years )
Facebook Local Insights
Both the questions portray very diferent meanings Value and Faida not same
Facebook Local Insights
‘Kam ki Jaankari’ is relatable to information pertaining to work Most participants narrated the same as work-related information 7 participants said FB provides them with information but whether they are useful in their work-life/ occupation or not depends! ● 4 out of 10 people had ambiguity about the genuineness of the information provided to them through FB ● It sometimes helps to bold main points within a sentence for improved readability when scanning slides. ○
“Kam ki jaankari matlab information for work” ( Shitu Sahu, Male, 29 years, FB user for 2 years ) Translation: It means information for work and not relevant information.
○
“Kam ki jaankari nehin milti, FB manoranjan ke liye hai.” ( Neha Singh, Female, 36 years, FB user for 2-4 years ) Translation: FB is meant for social Media and not really work!
○
‘What work-related information do you get from facebook? -- intent for most users. Users don’t relate it to Relevant Information.
Facebook Local Insights
C pattern of asking the same questions were best liked 6 people found the C version of question more clear in their content When asked A version, they felt it sounded very needy and in B version they were confused upon how can they judge on people’s behalf? ● 6 out of 10 people found it easy to and clear at intent the version C of each Question ● Actual Reactions: ○
“How would you rate seems like playstore rating pop-ups that I ignore!” ( Aravind Karthik, Male, 18 years, FB user for 2 years )
○
“Logon ko kaisa lagta hai main kaise batau?.” ( Neha Singh, Female, 36 years, FB user for 2-4 years ) Translation: How would I be able to say what people think? (Q type B)
Facebook Local Insights
Diferent approaches used for the same intent Version C is preferred by maximum
A
Facebook Local Insights
B
C
‘Upayogi Jaankari’ is too formal and doesn’t suit the aura of FB (Q3) The word ‘upayogi’ is hardly used in day-to-day conversation When asked most people were not able to translate ‘Upayogi’ properly and even narrated ‘Useful Information’ very diferently. ● 4 out of 10 people found it very formal and couldn’t relate it to FB ● Actual Reactions: ○
“Mujhe yaad bhi nehin aa rha last kab ‘upayogi’ word use kiya tha!” ( Aravind Karthik, Male, 18 years, FB user for 2 years ) Translation: I don’t even remember when did I last use the ‘upayogi’ word..
○
“Upayogi toh nehin hai, par help karta hai connecting people.” ( Anubhuti, Female, 26 years, FB user for 2-4 years ) Translation: Facebook not exactly is ‘Upayogi” but it’s like an aid connecting People.
Facebook Local Insights
‘Nayi Chizein’ are very specifc, FB ofers a lot more While most users agreed to learn varied new things from FB but they have their own challenges Though people fairly related to ‘nayi chizein sikhne ko milti hai’ but with a hitch about what next? It’s like a trailer without the movie. ● 5 out of 10 people users feel they get to see ‘nayi chizein’ but not all of them are useful. ● Actual Reactions: ○
“Nayi chizein mill toh jaatein hai, par usko deep-dive nehin kar sakte!” ( Rishikesh, Male, 34 years, FB user for 2 years ) Translation: We get to learn new things, but can’t learn completely as we can’t deep-dive.
○
“Nayi chizein bahut random hota hai aur unki genuinity ka bhi pata nhi hota?.” ( Anuja, Female, 36 years, FB user for 2-4 years ) Translation: Can’t rely on the genuinity of new things that you get to learn on FB.
Facebook Local Insights
‘Pasand ki chizein’ works well with most users 9 people found ‘pasand ki chizein’ very relatable to the context ‘Pasand ki Chizein’ was well accepted by almost all the participants with or without the hitch of authenticity of the information. ● All 10 people found it easy to relate and clear at intent in the context ● Actual Reactions: ○
“Haan, pasand ki chizein kaf milti hai FB pe, in fact agar main kuch window shop bhi karta hoon toh woh bhi aa jata hai Ads mein!” ( Aashish Jain, Male, 41 years, FB user for 2 years ) Translation: yes, I get to discover things of my interest in FB, it even insists me to shop all the things that i would have window shopped online.
○
“Mere Hobbies ke baarein mein kaf DIY videos miltein hai, jisko main apne hisab se recreate karti hoon?.” ( Neha Singh, Female, 36 years, FB user for 2-4 years ) Translation: It helps me get a lot of related videos to my hobbies, which I modulate as per my wish.
Facebook Local Insights
Question 5 was most relatable, both Hindi and English People in-general found more contextual identifcation with Q4 and Q5, where Q5 was working good for all 10 participants 2 users had problem relating learning new-things through facebook as they felt FB only provides the introductory information, rest is not available in the platform. But about discovering their interests everybody was sanguine about. ● All 10 people could easily associate Q5 with the context set with Facebook ● Actual Reactions: ○
“Facebook mein things that might interest you kaf available hoti hai! Sometimes, unrelated news bhi dikhate hai” ( Aashish Jain, Male, 41 years, FB user for 2 years ) Translation: There’s a lot of available information on Facebook. Sometimes it also provides unrelated news as well..
○
“Mujhe local information and News Facebook se hi mill jata hai?.” ( Anuja, Female, 36 years, FB user for 2-4 years ) Translation: I get local information and News by surfng Facebook itself.
Facebook Local Insights
‘Puri tarah’ as an option leads to the need of ‘Bahut haad tak’ as another option There’s a gap in the fow if ‘Puri tarah’ is followed by ‘Kaf had tak’ ‘Puri tarah’ and ‘To a large extent are not synonymous, instead ‘Bahut had tak’ and ‘To a large extent’ are. Again, ‘Puri tarah’ and ‘Helps Completely’ would have been fne. ● 6 out of 10 people would prefer a 5 point scale for this option arrangement type i.e Option Type A and B. ● Actual Reactions: ○
“Mujhe ‘puri tarah’ hi select karna pad raha hai kyunki koi 85-95% wala option hai hi nehin!” ( Aravind Karthik, Male, 18 years, FB user for 2 years ) Translation: I am being forced to select ‘Puri tarah” because of the lack of apt option
○
“Kaf had tak aur kuch had tak mein zyada farak bhi nhi hai! samajh nehin paa rahi kaunsa choose karoon.” ( Neha Singh, Female, 36 years, FB user for 2-4 years ) Translation: There’s not much diference between ‘to some extent’ and ‘to a moderate extent’.
Facebook Local Insights
Proposed Solutions
Facebook Local Insights
Proposed Solution 1 – Value and Faida are not same
Issue
People could not relate to the hindi and english question as the same Proposed Solution
Impact
Regional Context
Facebook Local Insights
Value leads to a very different context for Indian users, they instantly started relating it to some learning quotient or achievements
Benefts happen to be something that to users meant something that was ‘Faida’ to them
While local Language being the one most identifable to users, 7 users mentioned Benefts to be a better translation than Value.
Proposed Solution 2 – 5-point options work better for perceptual questions
Issue
Impact
Regional Context
Facebook Local Insights
4 point options worked fne with fewer audience and a large portion of users changed choices thereafter While 3/10 people didn't understand the need of 5-point option scale till the time they were shown, 4 felt something was missing while the other 4 changed choices after seeing the 5-point scale. Users felt something was missing but was unable to coin the right ft, but when shown most users aligned with 5-point option scale
Proposed Solution For Hindi, a 5-point scale helped to make the nearest to correct choice
Proposed Solution 3 – Question 5 was most relatable, both Hindi and English
Issue
Impact
Regional Context
Facebook Local Insights
People mostly chose ’Helps to a moderate amount’ in hindi which was ‘bahut madad milti hai’ , but according to them that wasn’t the correct interpretation Absence of option like ‘Helps to a great extent made them chose moderate as they did not want to choose Puri tarah’ which meant completely, as according to them nothing can be absolute and there’s always a scope of betterment. ‘Puri tarah’- Completely ‘Bahut madad milti hai’-Helps to a great extent ‘Helps to a moderate amount’- ‘kuch haad tak madad karta hai’
Proposed Solution Had there been the highest option being ‘To a great extent’ as an interpretation to ‘bahut madad milti hai’ in Q5 (Hindi), Users would choose that and according to them since the question is relative it cannot have an absolute answer like ‘Helps completely’ or ‘Puri Tarah’
Appendix
Facebook Local Insights
Marketing Campaign Objective Drive consideration and drive top-of mind for Facebook among the new-to-internet/Facebook users. ● Defnition of New to Internet People- people with less or equal to two years internet age ● Defnition of New to Facebook User- user with less or equal to two years FB age
Business Platform and Analysis Growth slows down for the blue app in India for 3 consecutive quarters. Time Spent (TS) is also showing a downward trend, while competition (YT & TikTok) continues to grow in users & engagement. Facebook’s analysis confrmed that 57% of India MAP (which is over 192M) have joined Facebook only in the last 2 years, compared to just 18% of US MAP. This large New-to-Facebook cohort consistently underperforms in comparison to the more mature (4+ Years on Facebook) users. ● 35% fewer number of sessions ● 50% lower TS/DAP ● Fastest to churn out from Blue Facebook Local Insights
Context Jio's launch in India disrupted the telecom access landscape leading to an unprecedented surge of people getting on to the internet for the frst time. This rapid growth shifted the composition of Indian consumers dramatically, where in just over two years, a large segment of Indian user base now consists of people that had never been on Facebook or the internet. This New-to-Facebook cohort differs signifcantly from more tenured users in both affnity and engagement. We have an opportunity to address this cohort and establish the value of Facebook to drive positive shifts in sentiment & behaviour. For a consumer entering the internet today with a multitude of apps to choose from- Facebook has to compete with newer apps that are singular in purpose, simple to use, and provide tangible value with instant gratifcation. In this environment, even though the awareness of Facebook is high, its value is obscured, and we are not top of mind when it comes to actual usage.
Facebook Local Insights
Campaign Strategy ● Cultural Tension: In a country of 1.4B peers, if you aren’t constantly moving forward, you’ll be left behind ● Key Message: Facebook opens up an unexpected world of ideas and possibilities through serendipitous discoveries that give you the edge to stay ahead ● Our Benefts to People: Discovery powered by Connectedness ● Proof point: Newsfeed - where discovery manifests through power of people at scale
Brand Strategy ● Core Insight: People are each other’s best resources ● Brand Belief: We can do more together than alone ● Brand Promise: The potential of connection & communities at scale
Facebook Local Insights
Why Newsfeed? 1. Newsfeed is the default experience for the vast majority of our users globally, but India stands out among the top 10 countries on how skewed that is. 40% of India users, spend more than 90% of their time on Newsfeed. 2. Newsfeed is the best surface where the 'Serendipitous Discovery' aspect of our App actually manifests, every other surface is more intentional 3. Newsfeed by itself is the most comprehensive combination of the most salient individual products/features of the app - it shows up videos, photos, status updates, content from pages, news, updates from the people. 4. By virtue of #3 above, our newsfeed is the richest source of information compared to all other competitive apps as well, since it brings together diverse genres - entertainment, news, sports, personal updates, lifehacks, and everything else through the lens of the people we are connected with.
Facebook Local Insights
DISCUSSION GUIDE OUTLINE Section Introduction
Questions
Response choices
Facebook Local Insights
Summary / Goals ●
Brief participant on study goals + procedures
●
Build rapport
●
Present question in local language
●
Ask participant to explain their understanding of the question
●
Assess if anything is unclear to the participant
●
Measure the ease of understanding and clarity of the question.
●
Present the same question in English
●
Explain intent of the question if unclear to participant
●
Assess if the translation carries the same meaning and intent as the English question
●
Capture local text and reasoning for any participant-suggested changes
●
Repeat above for the response choices available per question