THE TEETHING TROUBLES OF SURVEY RESEARCH
In today’s business reality, decisions cannot be based on random, uncontrollable factors such as luck. In this fast-moving environment the chance to fail is greater than ever. Figures reported by the Doblin Group show that
96% of all new product introductions and innovations fail to return
What to expect?
their cost of capital (Marsh, 2012). The market space requires brands to validate their communication and advertising efforts before an actual market launch. Organisations are tracking more and more consumer perceptions on all valuable touch-points. Businesses and marketers are striving for a more
data-driven decision-making process. We need those hard-core numbers to help us select the ideas to take forward and the ones to leave behind. This need for fact-based decision-making is the reason why survey research remains a very powerful and commonly used research method. The 2012 global research report by ESOMAR shows that no less than 76% of all market research projects conducted worldwide are in the field of quantitative research. The power of survey research lies in its validation strength,
surveys helps us provide those (no) go decisions on a brand, product or strategy level. Yet, considering the stimulating innovations that have moved the research industry in the past years, surveys research has been
lagging behind on several important aspects.
1
From boredom to engagement Online surveys are part of an engaging environment - everything online is fun, gamified and interactive. Nonetheless, filling in an online survey is not that much an engaging activity
These people are considered to be stimuli junkies and will stop any activity that does not succeed in crossing the minimum expected engagement level.
and we need to realise that participants are
always just one click away from exiting our surveys. Besides the high drop-out rates, this lack of engagement has an impact on the data quality - researchers are confronted more and more with straight-lining behaviour and the level of detail in participants’ answers is dwindling. This low(er) engagement is at the foundation of what we could call “the global warming of panels”. It is becoming
increasingly harder to attract people to participate in research. Especially considering some important target groups such as youngsters and Millennials who grew up in this fast-pacing environment.
Some research players have turned to gamification as the Holy Grail to enhance participant engagement (Puleston & Sleep, 2011), while others have redefined the research landscape by introducing fresh survey formats (e.g. Google consumer surveys) or by focusing on behavioural data. Yet, is this enough and
what is the ultimate recipe for re-engaging people to participate in survey research?
2
From quarantine to consumer context Survey research insufficiently copes with the complex reality of consumer behaviour. Decisions are influenced by a number of dynamics and it is important that surveys mirror these different aspects
in order to provide valuable input for clients and their business needs. Consumers are social animals Consumers are social animals and our decisions are coloured by group thinking or herd behaviour (Earls, 2009). The majority of consumer decisions are taken in a social setting (both conscious and unconsciously). We tend to copy the behaviour of people around us rather than to use our
own (maybe more rational) information to guide our decisions. Nevertheless, we do not take into account this social dimension in survey research. We keep on conducting surveys in an individualistic setting, where participants are asked to answer one question after another without being able to connect and reflect with other participants.
Consumers are bad witnesses of their own behaviour
Survey research traditionally taps into the so-called “system 2 thinking” of our brain. Nonetheless, the whole thinking behind behavioural economics and the work of Daniel Kahneman (2011) show that our decisions are mainly taken quickly, automatically by the so-called “system 1” side of the brain. We are
not rational thinkers and we use heuristics for our decision-making. One of these heuristics is “emotions”, our decisions are wired by emotions. We thus need to tap more into implicit attitudes and procedural knowledge in our surveys.
The majority of consumer decisions are taken in a certain context or occasion The majority of consumer decisions are taken in a certain context or occasion. It is important to
grasp the contextual background consumers are in when making certain decisions (e.g. consuming or buying a product). An answer to a question might not be as simple as yes or no, but might be explained by ‘yes when’ or ‘yes because’. Context is a better predictor of consumer
behaviour than individual characteristics (Spruit, 2012). We need to get a better understanding of the variations in consumer behaviour depending on the consumer situation or context. Are we losing game to other research methods? Considering the importance of data for the decisionmaking process, the need for accuracy and
projectiveness is present more than ever. Can survey research reclaim its position to provide consistent and rich data for decision-making by capturing the complex consumer reality, while at the same time increasing the engagement level?
A PARADIGM SHIFT Taking into account the above mentioned truths, we have designed a new survey approach. A two-paradigm shift rests at the base of this new survey thinking.
1
The empowered consumer Why is it that we, researchers, fail to create an engaging experience for participants? In
order to get a better understanding of participant engagement, we need to capture the concept of motivation. The self-determination (SD) theory, initially developed by Edward L. Deci and Richard M. Ryan (2000), explains that motivation, or in this case participant motivation, is a continuum and not a fixed state. It relates to a task within a context. This continuum (see Figure 1) has two
This is where participants are mainly driven by internal factors and are therefore more likely to sustain the activity. Yet, the motivation level to
participate in surveys is generally located in the middle of this continuum, where participants are mainly driven by external factor such as incentives. Furthermore, the selfdetermination theory describes how to move people’s motivation along the continuum. To do so, you need to foster feelings of autonomy, competence, relatedness and value.
anchoring points, a-motivation on the one
Interest, enjoyment highly competent
hand and intrinsic motivation on the other. In between lay a number of levels of extrinsic motivation. Naturally, intrinsic motivation is the ideal state.
EXTRINSIC MOTIVATION AMOTIVATION Task is not done properly
Figure 1 – Motivation continuum (SD Theory)
INTRINSIC MOTIVATION
AUTONOMY Autonomy refers to the fact that you allow
participants to choose to do something or not. Traditionally, when filling in a survey, you enter this tunnel experience - getting one question after the other, screen after screen. In order to avoid this, we can develop a modular survey approach dividing the survey into different modules or building blocks. These modules can contain both question types and tools. In this modular survey, participants can choose which
building block to start the survey with. This modularity principle in its pure form may not be realisable for all survey set-ups. Some research projects may need direction and a strict flow in questionnaire design; for these set-ups we can use the idea of ‘perceived non-linearity’. When participants complete a building block, by filling in all the questions within that block, they will unlock a new module.
Combining this gamified element with a nice modular survey layout, where every building block is a visual element, enables participants to break out of the traditional survey tunnel.
Next, in traditional survey research we do not allow participants who are topic, brand- or
Even in the best case, where we put our efforts in creating an engaging experience or where we invest
experience-engaged to collaborate with
in gamification techniques, participants are only
brands or researchers beyond their survey
asked to answer our questions without being given the option to share extra feedback. Yet
participation. Some participants would like to enter an additional collaboration dimension and help out brands, but we do not allow them to do so. A study conducted by InSites Consulting at the end of 2013 showed that
44% of consumers would like to collaborate with brands. Yet, the traditional survey format does not allow participants who have something to say to share their advice with brands or researchers. We basically let
participants go after their survey participation, without truly leveraging their full potential. Why is it that we do not allow engaged participants to go further in their collaboration with the brand?
consumers are valuable brand consultants and we should start leveraging this, even in survey research. This can be done rather easily by giving participants the possibility, after their initial survey participation, to enter a second survey dimension where they can truly collaborate with the brand and researcher.
COMPETENCE The feeling of competence, which is about
showing participants that they are good at something, can be reached by empowering consumers to do more than simply answering a researcher’s questions. We should give participants a role beyond responding, by involving them in tasks that are normally on a researcher’s repertoire. An example here is the use of crowd interpretation (Verhaeghe et al, 2011), where we show the live survey results to participants, for example, and ask them to interpret them by using their own background and knowledge as a reflection point, in order to get a greater understanding of the research results.
This is where participants engage in a coresearcher role. Research conducted by Balcetis and Dunning (2011) revealed that we, as
individuals, fail to take into account the influence of the situation when predicting our own behaviour. By contrast, when predicting the behaviour of others, we correctly take into account the influence of these circumstances. These findings show that by involving consumers in
an interpretive role, we might gain greater understanding.
RELATEDNESS The third dimension stipulated by the self-
Important here would be to allow participants to
determination theory is relatedness, which is about
introduce themselves to one another. The latter
showing participants that other people just
might function as a conversation starter and will
like them are doing this. Filling in a survey is a rather a-social and lonely activity; as a participant
you might even wonder whether you are alone in this. Theories such as Herd by Mark Earls have highlighted the importance of recognising a social dimension in marketing and accordingly marketing research. A first step would be to make
participants aware of the fact that they are not the only ones filling in the survey. This could be done by real-time visualisation of the number of people participating in the survey. The
true feeling of relatedness is fostered when allowing participants to connect and reflect with other participants on the research topic.
show participants that people just like them are
contributing to the research project as well.
VALUE The last feeling we should foster as researchers in order to increase the engagement level is ‘value’. Namely, showing participants that what they
do has a meaning. Traditionally in survey research we do not share with participants who the research is for, in order to avoid any potential bias. If we want participants to become an
active empowered partner in research, we can benefit from introducing the brand behind the research project and by even sharing the objectives of the research. Researchers and brands have to acknowledge that participants can provide valuable feedback to help a brand, yet we often do not allow them to share any with us. Being open and transparent
towards participants shows them that their contribution matters. By uncovering the real research objective participants get a clear idea of the value of their contribution.
Today, consumers expect to go beyond simply ‘responding’. Yet the foundation of survey research, like Pete Comley (2006) describes, is a parentchild relationship between researcher and participant. The sole role of participants is to
respond to a researcher’s questions, without allowing them to ask questions in return. Nonetheless, researchers can benefit greatly from a partner relationship with participants. It is therefore time that we allow participants to play a
more active role in research and become an active empowered partner. This empowerment starts with creating an engaging survey experience for participants by fostering feelings of autonomy, competence, relatedness and value.
2
Moving out of that box Next to the importance of boosting participant engagement, we are feeling
the need to capture a more contextual understanding in survey research. We researchers currently fail to mirror the complex consumer reality in our surveys.
We need to redefine our current survey thinking in order to capture the different dynamics that influence consumer behaviour. The second paradigm shift is a result of inside- the-box thinking. The sole role of participants in survey research is to respond to a researcher’s questions. Traditional survey research is thus mainly about asking questions in an individual setting. Yet, in order to grasp the consumer reality, we might need to go beyond asking questions and to think out of the box. In research we can identify three
supplementary collaboration modes between researcher, brands and consumers: listening, doing and cocreating.
LISTENING The researcher openly listens to and observes
But listening in surveys has moved beyond
the consumer. When looking into the research
capturing behavioural data. It involves every
landscape, we can detect plenty of approaches
attempt where the research participant has the
tapping into observing spontaneous consumer
chance to speak up, to share whatever is on his or
behaviour. This can involve the direct observation
her mind on a certain topic.
of consumers in their natural setting through
(online) ethnography but also listening to what consumers spontaneously share on social media (e.g. social media netnography or monitoring). We also observe an increasing importance towards integrating behavioural data (e.g. purchase data, online clicking behaviour etc.) into research. Also within survey research, examples have popped up where data was enriched with the social graph of research participants (Rodenburgh, 2012) or even physiological data
coming from for example eye tracking or ERP research.
It is about giving people the chance to give
On the other hand, it helps us uncover new insights
answers without asking questions. It is about
into consumer behaviour that we were not
taking a bottom-up approach where the
aware of before, simply because we did not think of
consumer determines the rule of
asking any questions about it.
conducting research. This can be done, for example, by integrating an exit forum at the end of the survey where consumers can share whatever is on their minds, but it could also involve approaches where consumers have the chance to formulate questions for other
research participants (Schillewaert, 2009). The benefits of embracing this mode of ‘just listening’ are twofold: on the one hand it leads to more objective fact-based information
giving us insights into what consumers DO and not only say.
DOING The researcher involves participants in
In cognitive psychology (Schneider and Schiffrin,
different task-based exercises, where they are
1977) we make a distinction between declarative
asked to undertake a certain activity. There are
(knowledge about something) and procedural
generally two kinds of approach within this
collaboration mode. To start with, at times consumers cannot express themselves because they are simply not aware of their own behaviour. We are bad witnesses of our own behaviour. This is in line with the whole thinking around behavioural
knowledge (knowing how to do something). Neuroscience has uncovered that both types of knowledge are stored in different part of our brains and hence require different techniques to trigger recall. However in survey research we do not adapt our approach based on what we need to identify.
economics, which expresses that the majority of
We mainly tap into the declarative brain. Research
what we do is done implicitly. So why ask people
shows that procedural knowledge can be
about it? We need to use new and creative ways
explored by allowing people to do things. This
within survey research that allow us to capture
can be reached by embedding more scenario
this ‘system 1’ thinking. One way to do so is by
thinking and storytelling in research but also by
using time-pressure exercises. As people do not get the time to reflect on their answers, we get a glance of automatic behaviour leading to new information
allowing participants to explain something through visual cues. There already exists a need in survey research to move more towards task-based
(Verhaeghe et al, 2012). A second reason why
research. Or better: it is not what you ask
asking consumers to do things can be more powerful
people, it is what you do with them.
than asking them to explain things has to do with how our brain stores information.
CO-CREATING This is the ultimate collaboration dimension, where
Moreover a great idea can sometimes come from
participants take up a very active role in
somebody who enters the discussion with a fresh
helping a brand beyond just giving their
unbiased perspective. But there is more: by
opinion. Here consumers have the opportunity to
looking at the actions consumers propose
think along with brands and be involved in tasks that
based on the research results, we can learn a
traditionally are considered as being beyond their
great deal also about what they are looking
capability. Examples of co-creating tasks are
for.
involving consumers to shape the solution space by including them in idea brainstorms and asking them for their advice based on survey results. Qualitative research has already a long tradition of co-creation but so far this mode has not entered survey research yet. The benefit of empowering consumers to explore the solution space together with us is twofold: it is a known fact that innovations or campaigns that are co-created with consumers have a bigger impact (Schreier et al, 2012).
These collaboration modes can be plotted against a second dimension representing the inter-
Me: no real inter-consumer interaction is present and the focus here lies on the individual participant.
consumer relations or interactions. Theories such as Herd make us realise that we are more
socially determined than we think we are.
Crowd: there is an interaction between
However, today survey participants function in
participants, yet no real closed feeling of
isolation, not being allowed to reflect and connect
belongingness. We have a group of individuals who
with other participants. We need to move away
from solely looking at the individual
are not yet a team working together towards a
common goal.
respondent and to recognise the value of
Group: close interaction between participants
inter-consumer reactions. This is where our
who share a feeling of belonging to the same
second dimension comes in, a continuum going
community.
from individual to connected in 3 phases: me, crowd and group.
By combining both dimensions we can identify a framework with twelve quadrants (see Figure 2). Traditional survey research primarily focuses on one single cross-point in this framework, namely individual and asking. Yet, survey research can greatly benefit
from going beyond this single-box thinking. This does not imply that we should completely let go of asking questions to participants, this will still remain the core of quantitative survey research. However,
combining the different collaboration modes will allow us to better uncover the underlying dimensions of the research topic. Some aspects require activities beyond asking questions in order to expose them. Moving beyond the border of asking should thus allow us to better capture the complex consumer reality.
Figure 2. Research collaboration framework
EBAY CASE STUDY
1
Project background eBay UK wanted to get a better understanding of why consumers lapse (or decrease in buying
frequency) by exploring the perceived site experience and its drivers and barriers. Besides optimising the current eBay proposition, eBay also wanted to explore a new solution space for online shoppers. The research ran in the UK amon different types of lapsers within the fashion and consumer electronic product categories. A total of 834 consumers took part in the research. The scope of this research was threefold (see Figure 3) :
1. 2. 3.
Describe the behaviour and attitudes related to lapsing and pinpoint areas of improvement for eBay Generate insights into key success factors for eBay to manage lapsing Explore the solution space for lapsing, with a focus on inspiration and findability
Figure 3. Research flow
2
Project methodology The research method used explored the boundaries
After the main survey, participants were invited
of survey research. In order to assess the impact of
to enter “The Village”, a second optional
this new approach we split-ran the survey. Some
survey dimension which allowed engaged
participants got a traditional survey whereas others got the enriched version containing the modular approach and the village dimension. The research
approach is based on our new survey thinking
participants to go further in their collaboration with the eBay brand. After participants filled in the survey, they could thus opt in for this optional part where they could connect
where we go beyond asking questions and
with other participants and reflect on the research
apply the principles of the self-determination
topic together. The Village is a platform
theory to better engage participants. The main
consisting of different buildings, each of which
survey consisted of a modular survey design, in
contains a different task-based element.
which the survey was composed of different building blocks or modules. The self-determination theory claims that autonomy is a key driver of engagement. Therefore, we introduced the concept of perceived
non-linearity by combining the modular survey
approach with a gamified unlocking element.
This study thus went beyond the traditional single-box thinking of individual and asking. The different tools in and after (The Village) the survey can be plotted on our framework (see Figure 4). The survey still consisted of various research questions, yet on the individual dimension we also introduced some task-based exercises. Next to that,
the introduction of The Village and the contagious tool allowed us to involve the crowd through the social dimension embedded in these tools. More detailed information on each of these tools is available in the next section. Figure 4. eBay project framework
3
Project approach Next to the traditional questions, we introduced some new tools in the survey: IMPLICIT MEASUREMENT TOOL The eBay brand image was measured through implicit attitude testing, allowing to recognise natural and potential brand associations. Participants got to see each statement for three seconds and were asked to press the spacebar if they felt the statement could be attributed to eBay. The tool is located in the doing and individual cross-point of the
framework. This Implicit Measurement exercise allows plotting all statements on two dimensions: (1) the percentage of participants
linking the statement to eBay and (2) the time before pressing the spacebar, resulting in four quadrants (see Figure 5):
Natural associations: These are spontaneous associations - the majority of participants link the item with the brand with an above average reaction time.
Potential associations: These items are highly associated with the brand, however they require some reflection (response time is below average). These items trigger some “ahaa”
feeling of recognition, however they could become more natural when investing in additional communication efforts. Figure 5. Implicit Measurement quadrant
Niche associations: These items are only linked with the brand by a few participants, yet the reaction time is above average. These are mainly items that are recognised by a specific target group (e.g. heavy users).
Limits: Few participants link the item to the brand and the reaction time is below average. If an important brand item is contained in this quadrant, the brand needs to invest the required resources to turn the item in question into a potential or natural association.
MY SHOPPING TRIP In order to get an understanding of eBay’s role (and that of its competitors) in the purchase process, we
introduced a task-based question in which participants could reconstruct their purchase process visually. Participants got an empty timeline representing their purchase process from start to finish. They could fill the timeline by dragging icons of sources (information channels or platforms) onto the visual and by explaining for each source how they used it. Furthermore, we asked them to rate each
source on the extent to which it facilitated them and contributed to their final purchase
decision. The outcome of this task-based exercise allowed us to grasp the perceived positioning of eBay in the purchase process and the use in each stage. This gave us the potential to identify and quantify
differences in the search process across
subgroups.
CONTAGIOUS TOOL In order to understand the lapsing behaviour, an
This exercise allows identifying both
important element was to explore the drivers and
spontaneous and prompted drivers for buying on
barriers for buying on eBay. These were
eBay. In addition, the word cloud allowed
measured by means of a new tool which allows
participants to see that other participants
highlighting a so-called ‘contagious effect’. In a first
also contributed to the research; this semi-
phase, participants were asked to write down
social dimension thus taps into the relatedness
what they considered to be ‘reasons for buying
aspect as specified by the self-determination
on eBay’. They could write down spontaneous
theory.
answers, using key words, in five open-answer boxes.
In a second phase we showed them a word cloud
including all answers from all respondents for this question up until that point in the survey. The word cloud combined their own answers with those given by the other participants. In a second phase, the
participants could click on those words in the
word cloud that they felt were also ‘reasons for buying on eBay’.
THE VILLAGE After participants had filled in the survey, they could opt in for The Village where they could
connect with other participants and further collaborate with the eBay brand. The eBay Village consisted of five buildings: the Lounge, the Ideation, the Inspiration wall, the Newspaper stand and the Gallery (see Figure 6).
Figure 6. The Village
The Lounge (1) is the central building of the village where participants can connect with one another, start a discussion on topics created by the researcher and even post topics of their own. In other words: participants can connect with one another, the researcher and the eBay brand. In the Lounge we introduced three featured topics where participants could introduce themselves, give feedback on the survey and share their advice with the eBay brand. The topic in which participants were asked to introduce themselves functioned as a conversation starter and showed participants that people just like them were also contributing to the research project. This introduction topic enhances the feeling of being visible
as a consumer and thus taps into the relatedness aspect as defined by the self-determination theory. In another topic, participants were asked to share their advice with eBay. We did not simply introduce eBay as the brand behind the research project, we also openly shared the objectives of the research. This is the only way to truly encourage consumers in providing valuable feedback to help a brand. Apart from these featured topics,
participants could create their own posts related to the research topic, which allowed them to discuss and interact with other participants. This open social space helped to gain additional insights as it provided us with answers to questions we did not even ask. Figure 7. The Lounge
In the
“Inspiration wall” building (2)
participants were asked to share inspirational
examples they believed eBay could learn from. They could do so by uploading images, YouTube videos and links to interesting webpages. Participants were motivated to look for example in- and outside the online shopping platform environment. A real task-based element which taps into the co-creation dimension of the collaboration framework, where participants were involved in generating inspirational output eBay could learn from. In
this exercise we emphasise the value aspect, as explained by the self-determination theory, by openly sharing the objective for eBay. The result is their own inspirational board
with explanations on how eBay should implement this in order to optimise its current offering. In the Gallery (5) building, participants could view the work of others, Like it and comment on it.
Figure 8. Inspiration Wall
In the
Ideation building (3) participants
were asked to brainstorm and share ideas on
three topics related to inspiration and findability, two areas in which eBay wanted to optimise their current offering. Besides posting their own ideas, participants could see what other people posted and Like it or comment on it. This idea sharing allows involving participants in discovering the solution space. This task-based
element taps into the co-creation collaboration dimension, where participants
Positioning this as a real task-based challenge, participants had to pretend they were going to buy that item, so we invited them to visit eBay and other platforms and to write a review about their experience. The strength of this exercise is that we are not focusing on recall, which is visible in the added value from positioning this as an in-the-moment task-based challenge. In the
view the reviews of others, Like them and comment on them.
are asked to think along with brands and tap into the solutions space. The output of this exercise is the creation of ‘idea cards’, which combine a consumer idea with an inspirational visual that can be used in future workshops or strategic meetings. In the Newspaper stand
(4)
participants were asked to write a critical review of their shopping experience. For this challenge, we asked participants to think of a fashion/ consumer electronics item that they would like to buy in the very near future.
Gallery (5) participants could
Figure 9. The Gallery
4
Research findings IMPACT ON PARTICIPANT ENGAGEMENT Introducing a modular survey approach showed to have a positive impact on the participants’
satisfaction level. The use of a second optional survey dimension, The Village, also contributed to a positive increase in satisfaction level. Statistics from SSI, our sampling partner for this project, showed that the extent to which participants were extremely satisfied increased with 30% in comparison with the benchmark condition. In addition, the interest to participate again in this type of research projects doubled, compared with the benchmark setting. Overall, participant
experience was very positive: “Definitely loved the new take on surveys you guys used. And I very much prefer this method.” Nevertheless, the major positive impact of this new approach was visible in the contribution level throughout the different task-based elements in The Village.
The Village was optional and only available after participants completed a 15-20-minute survey. We know from previous research that the maximum attention span is about 20 minutes, yet 10% of the participants actively participated in these additional contextual tasks. Taking into account that these contextual tasks are used to get additional sensing and understanding, such proportion is well suited for this purpose. Additional experiments showed that this proportion varies depending on the research topic, the active participation rate ranging from 10% to 25%.
Overall, all active participants are
characterised by a high topic and brand identification level and they are mainly driven by intrinsic motives to participate in research.
The output of the task-based elements showed that participants went beyond the expected in their involvement. In the Newspaper stand, for example, we noticed that the level of detail in their stories was significantly higher compared to what we receive in traditional surveys. This can be explained by the increased engagement level entailed by
the task-based character of the challenge and the social visibility the platform creates. Overall, participants in the new-style survey reported a significant 15% increase in ‘feeling understood’ in comparison with the benchmark condition. Next, participants felt that eBay was
more open to their suggestions and feedback, compared to the results of the traditional survey approach. This shows that survey research can have an impact on the consumer brand perception. Considering that surveys are yet another touch point between consumers and brands makes this another interesting finding.
IMPACT FOR THE RESEARCHER The enriched set-up used a modular design. As the questionnaire design required a fixed order, we used the concept of perceived non-linearity. The order of the different questions was the same for both the traditional and the enriched survey design. The enriched survey only differed through its design and gamified unlocking element. In a first dimension we needed to understand
whether the latter would lead to different results. Comparing the data from the traditional survey approach with those of the modular design showed that the new survey design did not lead to significant deviations in the data. These findings highlight that a modular approach, combined with a nice design, could be a first step towards increasing the participant motivation level as a result of an increase in perceived autonomy. This does not mean that from now on we can ask hundreds of questions in a single survey as long as we introduce the modularity concept.
Researchers and research users should still take into account the natural limit of acceptable interview length for participants. Next, the output of the contextual
tasks allowed us to form more tangible recommendations. In a first analysis phase we analysed all questions included in both the traditional and enriched survey designs (excluding any new tools and task-based challenges). This formed the basis of our traditional research report and conclusions. The second analysis phase included all enriched elements and tools. Comparing both reports taught us that not only we had more data, but more importantly that the data was also richer. The contextual output from the new tools and challenges, composed of consumer visuals, stories and ideas, allowed us to bring more
sensing and understanding into the research results. In addition, the involvement of consumers in shaping the consumer space and the possibility to share their advice and feedback, allowed us to shape very tangible recommendations for improvement.
IMPACT FOR EBAY The impact for eBay was threefold: gaining contextual understanding, the ability to uncover new insights and achieving more actionable research output.
1.
A first key benefit was the addition of contextual
understanding to the survey data. Survey research helps to answer the predefined research questions and objectives; however these answers often lead to additional questions. There is a need to combine the
survey output with some additional sensing and understanding. The task-based elements in The Village allowed eBay to grasp the contextual space consumers are in, leading to a better understanding of the ‘why’ behind the survey data. The ‘my shopping trip’ exercise in the survey allowed to identify and quantify the role of eBay and its competitors in the purchase process across target groups. The output of this
exercise showed where eBay enters the flow, what its role is and how it was evaluates in terms of facilitating power for the final purchase decision.
In The Village we challenged participants to
The survey highlighted that eBay was lagging
pretend entering a new purchase mission, by visiting eBay and other e-commerce platforms and to write a review about the experience. These review stories allowed us to
behind on some important buyer perception items in comparison with its competitors. The output from the Inspiration
get a detailed understanding of the perception and performance of the different platforms. Yet the real value came to life when combining the consumer stories and visual output of the review exercise with the quantification of the ‘my shopping trip’ exercise. Not only did we get
clear view of eBay’s role in the purchase process for the different subgroups, we also got a clear sensing and understanding of how eBay could improve its role. Additionally, the visual output from these contextual tasks (user-generated pictures) proved to be stronger at conveying a message.
wall, where participants were asked to upload inspirational pictures and videos that eBay could learn from, visualised the same critical aspects as mentioned in the survey. Yet, the visual storyline from this exercise helped convey the message in a much more impactful way. Images are very powerful to convey a similar message because of their emotional load.
2.
Another key benefit resulting from this new
Next, the social dimension in our survey set-
approach was the ability to discover new
up allowed for eBay to capture the contagious effect arguments might trigger.
things, applying new tools that went beyond the single-box thinking, thus allowing to uncover new insights. The Implicit Measurement approach allowed for eBay to distinguish which category items evoke spontaneous and potential brand associations. The tool allowed eBay to identify the items that
currently form a barrier and need closer attention. Apart from measuring the associations towards the eBay brand, we repeated the same exercise for eBay’s main competitors. This approach allowed us to define a new performance indicator of brand strength: brands for which the items were located more towards the right upperside of the framework have a clearer brand image than those with items located more towards to lower bottom-side. This allowed eBay to get an understanding of its brand strength relative to its competitors.
This is an important aspect to understand, as this effect can have an influence on consumer perception and reflects the actual consumer reality. The word cloud exercise, in which participants could indicate other words in a prompted setting, allowed uncovering this effect. 2% spontaneously indicated PayPal as a driver for buying on eBay, yet the contagious effect showed that this increases to 52% in a prompted setting. Using PayPal in a communication campaign might thus trigger understanding and recognition amongst people. This contagious
effect is important for eBay to recognise and to take into account for future communication efforts.
3.
A third and last interesting benefit is that the
co-creation tasks led to more actionable research output. Some of the task-based challenges allowed participants to help shape recommendation for eBay. In the ideation tool, participants were asked to brainstorm on potential ideas related to inspiration and findability. The outcome of this exercise was 50 idea discussions, which allowed us to create idea cards for eBay. These cards, formulating tangible instructions for improvement, could then be used in future workshops or strategic meetings. Next, the
output from the Lounge, where participants could share their advice with eBay, allowed to shape very tangible recommendations based on consumer feedback.
5
To conclude Is it ‘game over’ for surveys? Reflecting on the approach and learnings described above, we can conclude that the answer is definitely “no”. Yet, we need to realise that it is high time to walk away from our traditional survey thinking.
We need to invest in a survey approach
which empowers consumers. Consumers expect to go beyond responding, so why not allow them to do so? We have given some examples of how you can foster the feelings of autonomy, competence, relatedness and value in order to boost participant motivation. Yet, we, researchers and brands, can do many other things. We need to acknowledge that we have to step away from a 20-minute survey where the only thing participants can do is answer the researcher’s questions. An engaged consumer will
contribute better to research, will provide better and richer answers and is more likely to share his experience with others. Considering the fact that surveys are yet another touch-point between consumers and brands, this will result in a win-win-win for all of us. In addition,
we need to go beyond that single-box thinking in survey research. Each set-up should ideally combine all four collaboration modes and allow consumer to connect and reflect with one another.
The activities and tools used in each of these collaboration dimensions will depend on the research objective, yet only combining the ‘asking’ dimension with other modes will allow to uncover the real consumer reality. Traditional survey research is still far too
focused on the individual consumer, yet this illustrative case shows that we can benefit greatly from introducing a social dimension in survey research. The framework can be used beyond survey research. Online research communities, for example, mainly touch upon the group dimension (shared feeling of belongingness amongst participants), with the different community activities tapping into the different collaboration modes. This framework thus allows us to think beyond the
boundaries of traditional research methods and will extend the possibilities of true fusion research. We should move beyond silo thinking and realise that, in fact, it is not only about thinking out of our ‘individual asking’ box, it is about realising that there really should be no box at all in research. It is about grasping
consumer reality to the fullest and using creative methods to reach that goal.
I would like to take this opportunity to thank some people who helped me throughout all phases of this project. Firstly, I would like to thank Annelies Verhaeghe for all her support, creative thoughts and knowledge sharing. I also thank Ioana Joanta and Dieter
Verschueren for their contributions in developing the tools we used for this project. A special thank you goes to Joost Van Eyck, Helmont Siau and Ken Vanderbeken, all part of the development team at InSites Consulting, for creating The Village platform. Also to Hannes Willaert, the creative brain behind the look and feel of the modular survey design and The Village. This project would not have been possible without the collaboration opportunity from eBay and Barbara Langer in particular. Also great thanks to
Christophe Vergult and Annelies Verhaeghe for managing the content side of this project. Last but not least I would like to thank our sampling partner SSI, who helped us with the recruitment for this study, and in particular Pete Cape, for sharing his knowledge on participant motivations.
Katia Pallini
Annelies Verhaeghe
Barbara Langer
Senior Research Innovation Consultant InSites Consulting
Managing Partner InSites Consulting
Head of Market Insights eBay Europe
Thank you!
@InSites marketing@insites-consulting.com www.facebook.com/insitesconsulting www.slideshare.net/InSitesConsulting
www.insites-consulting.com