For IKEA, the yearly Catalogue is the main communication channel
with existing and potential customers globally. This case study show
What to expect?
how the 2013 edition of the Catalogue and possible covers for the 2014 edition were evaluated qualitatively around the world, through ‘Market Research Online Communities’ (or ‘Consumer Consulting Boards’) in five different countries.
The paper also shows how ‘Online Communities’ are becoming a
true ‘Fusion Research’ tool more and more which allows for ‘triangulation’ on different levels (data sources, research methods, research environments, theories and investigators),
leading to more valid research results, fresh inspiration and a deeper understanding of the issue researched. Best practices concerning: moving an existing qualitative project online, creating internal buy-in for emerging research methods, engaging internal audiences with research findings, running communities in different cultures and reactivating an MROC over time, are shared as well.
The IKEA Catalogue
Evaluating an icon of a global brand
IKEA has the vision ‘to create a better
A few weeks after the global distribution, a large quantitative
everyday life for many people’ by ‘offering a wide range of well-designed, functional home furnishing products at prices so low that as many people as possible will be able to afford them’. For IKEA, their yearly
study was conducted to evaluate the new edition on key performance indicators, to measure the impact it had in the market and to benchmark the performance with previous editions. The findings of these studies serve as input for the creative team that works on the latest edition.
Catalogue is one of the main channels of communication with existing and potential
For the 2013 edition, significant changes were made to
customers globally: one Catalogue showcasing
the format (slightly bigger), content (offering more inspiration
one product offering to serve ‘the many people’ around the globe. Every year, IKEA
pages, including more storytelling and a different way of
conducts a ‘global’ qualitative study to understand how people feel and think about the latest edition of the Catalogue and what they do with it (this was done before the actual launch).
1). Furthermore, a complementary mobile application was
picturing the products) and structure of the Catalogue (Figure launched. These structural changes made it even more important to deeply understand people’s emotional and
rational reactions to the latest edition.
From good to great
Over the past years the feeling had grown within IKEA that the qualitative evaluation of the Catalogue, done
through offline focus groups in different countries around the world, could potentially be done differently and better and in a fresher way. Here is a shortlist of the key issues that the business owners and the research team at IKEA were encountering:
1
Focus groups were only giving a snapshot of reality : the first reactions to the new Catalogue. No real insights were gained on people’s second thoughts and their behaviour afterwards (how they are using the Catalogue and how that use evolved over time).
2
Secondly, given the limited time spent with consumers and a limited portfolio of research techniques possible,
focus groups did not bring that much fresh and inspiring information to the table . Every year, a new edition of the Catalogue was only judged partially (due to the limits of the method) and in the same way, leading to conclusions that were very similar year after year.
3
Another issue flagged by internal stakeholders was the limited number of participants per session in a
focus group and the dominance of certain individuals in the discussion. Communities give access to more opinions of people with a wide range of profiles. And due to the longer time period, all participants are given an equal chance to give their opinion.
4
Fourthly, the project owners experienced that the quality of a focus group was largely
determined by the quality and experience of the moderator. There was probably interviewer bias as well. Country differences were possibly due to differences in moderation style, rather than real differences between cultures.
5
Finally, despite all the new technologies available to follow offline focus groups from home as a client,
internal stakeholders do not follow sessions that often.
Conclusion: Focus group transcripts are experienced as not that convenient and pleasant to go back to the real discussion for inspiration or to justify an element when creating or making decisions. The result is that internal stakeholders
are less confronted with the ‘voice of the customers’ during and after the project.
Given this situation, the research team at IKEA was on
Given these objectives, a Market Research Online
the look-out for a method enabling them to
Community (MROC) (De Ruyck et al, 2010) was chosen
understand both the emotional and rational reactions to the 2013 Catalogue and the mobile application. Furthermore, in order to really understand
as the backbone of this project in favour of traditional focus groups (See Figure 2). Online closed platforms
to have a dialogue and work together with
the Catalogue’s impact, it was also important to gain
consumers in five different countries around the
insights into how people use the Catalogue on a daily
world (Germany, Italy, Poland, the US and China)
basis after it landed in their homes.
seemed to be the answer to the above issues. But why
was this really a better option?
A simple, but very useful framework to evaluate a new research method and to demonstrate its effectiveness internally is to check if it is providing ‘automatical’, ‘informational’ and/or ‘transformational’ benefits
1
Automational: doing things faster and more cost-efficient
A research community indeed takes more time to set
Secondly, the set-up cost of a community project is
up, but once created it can be reused without losing
higher than that of an ad hoc qualitative project, but once
the time of a classical set-up phase. This is a plus in
established and in use it becomes a cost-efficient tool:
the process of crafting a new Catalogue: the platform
more and different research methods can be
cannot only be used to evaluate the current issue, it
combined within the same budget.
can also be used as a source of feedback and inspiration during the process of creating the next edition. For this project five communities with a 3-
week duration were created. Eight weeks later the platforms were reopened for one week to get feedback on the next stage - the creation of the cover for the 2014 edition.
The @Home Community
2
Informational: obtaining a better data quality and deeper insights
Communities are characterised by the fact that you can work with more people of different profiles - in
this case with 50 participants in each of the five countries. We included three different profiles: potential, existing and lapsed customers. The result was five MROCs to hear the opinions of ‘the many people’ in one single project, on one platform. The ‘research on research’ we did in the past taught us that for most participant profiles and research objectives it is best to run a community in the mother tongue of the participants. Firstly, by doing so they will
discuss more and they will post in a more nuanced and emotional way . Secondly, our experience showed that it is a must for the community to be moderated by a native who knows the language, the
local culture and the local market. All of this will lead to more relevant and to-the-point customer dialogues. That is why we made the choice to set up a separate MROC in each of the five countries. This approach also allows us to run the whole project at once and in parallel. While being in contact with participants via a local moderator, you still grasp the advantages of having a central/global project team and content overview when
‘connecting the dots’ on a global level. By working with a single master topic guide (which is adapted to the local reality and culture), interviewer bias is partly eliminated. The project is run by a team of different people who work together both on country and global level.
3
Transformational: doing things which were not possible before
Evaluating the Catalogue in an MROC
Another advantage of having
To obtain a holistic view on a
gives us the opportunity to work
more time with participants is
participant’s use and perception of the
with the same participants over a
‘consecutive learning’. One can
Catalogue, a whole range of
longer period of time (in this case:
build further on what one
research techniques was plugged
three weeks of 24/7 contact) in the
learned previously. Or stimuli
in into the community platform:
comfort and context of their own living
material can be tested, adapted
observational tasks, an online diary, a
environment (home). In this project, we
and tested again in only a matter
collage tool, mini-surveys, creative
followed the evolution in
of days with the same group of
exercises, group discussions and even
perception and usage of the
consumers.
implicit/emotion measurement tools.
Catalogue over time (before the
Communities allowed us to blend
Catalogue arrived, the first flip-through
different research methods, giving
moments and the different reading
us depth and breadth in terms of the
sessions). This is something that is
insights we found.
hard to realise cost-efficiently with traditional methods.
The research team did not have a hard time convincing the internal clients to make the switch, as it was clear that the time
had arrived for something different and better than offline focus groups. The advantages of using a community in the evaluation of the Catalogue (and the IKEA Catalogue App) were very clear as well. The questions arising among the business owners were more about the practical side of things, e.g. how to choose and optimally combine the different
research options available on the platform, how to safely distribute large numbers of Catalogues to different parts of the world before the actual launch and how to make sense of a big pile of information.
A research community as the backbone for fusion research
In order to bring fresh and unique insights to the table and to get a valid, clear and holistic view on people’s perception and use of the 2013 Catalogue, we used the principle of ‘triangulation’ (Guin et al, 2012) to create a true Fusion Research tool: investigating the same subject or issue from different angles and by doing so
create a more adequate and deeper understanding. We applied ‘triangulation’ on five different levels:
1
Data triangulation: by including both existing, lapsed and non-customers in the evaluation, we get data on all three groups and a full view on how different receivers of the Catalogue react to it: what do they feel, think and do?
2
Method triangulation: by combining different observational, qualitative (both interviewing and discussions) and quantitative research techniques (both explicit and implicit measurement) we tackle the same issue from
different angles, leading to a holistic view of it.
3
Environmental triangulation: the mobile application used on the community platform allows participants to not only take part on the main research platform from behind their PC, but to also provide us with more personal and contextual information - in this case on how they use the Catalogue on a day-to-day basis in their homes and over time.
4
Theory triangulation: as we know that ‘people think less than we think they think’, it is important to take into account both ‘System One’ and ‘System Two’ thinking when asking people to evaluate the Catalogue. That is why we included both an implicit and an explicit test of the 2014 cover during the one-week reactivation of the community.
5
Investigator triangulation: by asking the local community moderators, the global project team at agency side, the client-side researchers, the internal client and the research participants to analyse certain parts of the data, we made sure we got everything out of it and our final conclusions included different points of view from people with different backgrounds. The local moderators keep an eye on aspects specific to the local culture and market of their country, the global research team looks for the global consensus, the client-side researchers and internal client frame the results within the business context and participants help us to close the final blind spots we have. In this project we asked ambassadors from the five different country communities to join forces in an English-speaking Global Room where a discussion took place concerning the cover test during the reactivation, to see if and how it was possible to come to one cover for all countries. The view of the participants helped us to see which cultural differences could be won over.
Applying the principle of ‘triangulation’ asks for more time and man-hours. One needs to make the trade-off between cost and getting additional understanding. In this project, it was a must to get a 360° view on how the Catalogue was perceived and used over time. It was only by making use
of a ‘Research Community’ and applying the principles of ‘triangulation’ to it, that we gained a fresh and complete view on the evaluation of the 2013 Catalogue.
Next we give examples of each of the different tools used to bring Fusion Research into practice
Phase 1: Evaluation of the 2013 Catalogue during three-week communities in five different countries
During the first week the goal was to ‘meet the reader’
After the Catalogue was dropped of at the
and understand the actual and aspirational
participants’ homes (in preview, two weeks before the
behaviour of the participants. Who is the reader
real global launch), we assessed the perception,
and what are his/her expectations towards the brand
satisfaction and level of engagement the 2013
and the Catalogue? To get there we conducted
Catalogue evoked. First impressions and second
amongst other activities a mix of an ethnographic task
thoughts on particular aspects of the Catalogue were
to get a view of their house and insights into how they
researched (structure, pictures, stories, the Ikea
live, a mood board exercise in which we asked the
Catalogue App, etc.). It was important to understand
members to map their feelings about the IKEA brand
how the Catalogue offers both inspiration and
and creative tasks on the forum of the platform (e.g.
information to the reader and if that was done
Tell us ‘Your IKEA Catalogue story’).
to the right extent. Moreover, we wanted to get a grip on the life cycle of the Catalogue. We did so by adding three specific exercises:
A photo safari, in which we asked the participants to spot the Catalogue in their house. We wanted them to imagine that the Catalogue could speak and tell the story of his past, current and future life. During the last two weeks of the community, a calendar tool was added to the platform to get day-to-day insights in the usage and the emotions it evoked. This was done in a private part of the community (participants could not read the responses of others), which aimed at avoiding bias. Finally the Ikea Catalogue App was tested by understanding the expectations of these types of apps, asking them to use the app and to evaluate it afterwards on its relevance, user-friendliness and discuss possible improvements for the next version.
During the final week, we investigated whether the Catalogue was meeting expectations, giving us first clues
on the impact the latest Catalogue had on the brand perception and shopper behaviour. In other words, what impact did it have on the business: was it attracting people to the shop, raising interest in home furnishing and was there a positive impact on the perception of the brand, especially among lapsed customers?
Phase 2: Reactivation of the different communities (after eight weeks of inactivity) to test the overall theme and the first cover ideas for the 2014 Catalogue This was done in a two-step approach during one week: A mini-survey (N=226, response rate > 70%) to test six options for the new cover (illustrating the theme) Each participant randomly evaluated three different covers (resulting in at least 90 participants for each cover tested). The questionnaire was a mix of explicit questions (fit of the covers with brand and mission statements) and emotional/implicit measurement:
• Two-second test: each participant saw a cover for only two seconds. Afterwards, via an open-ended question, the members of the communities were to write down what they saw, which emotions they had felt and what they remembered. This was done three times for each participant in randomised order, to exclude order effect. We were measuring ‘stomach impact’ here explicitly, though spontaneously.
• Implicit measurement: each participant saw the same three cover pages for 10 seconds. Afterwards, the members had to execute an ‘implicit measurement’ assignment where 15 emotions were shown rapidly (one second) in randomised order. Participants had to press the space bar each time they associated the emotion with the cover they just saw. Here we measured unconscious emotional reactions to seeing the cover page.
By combining the breadth of the ‘stomach impact’ on the X axis (the share of people who associate a given cover with the emotion, via implicit measurement) with the depth on the Y axis (for those who have associated the emotion with the cover, how strong is that association, i.e. how rapidly was the association made). To make
interpretation easier, the scale of the Y axis was reversed (= the higher, the faster) (see Figure 3). Results of these measurements were thrown back into the discussion during phase 2, in order to fully understand them.
Continuing with a discussion on the forum: in the survey, the participants were confronted with different cover ideas After two days they were asked which of the covers they still remembered. We used a tool in which you first have to answer yourself, before the others can see your answer (which ensures that answers are not biased). We did this to assess the ‘stomach impact’ on the mid-long term. After the spontaneous recall test, we
conducted the spontaneous associations test: upon showing the different covers, participants were asked to share their initial thoughts, without prompting. Via this exercise we got an understanding of the key associations made with each cover. Finally, by asking indirect questions (e.g. tell the story of the cover),
participants were motivated to elaborate on the indirect impact of the cover. Finally, the most remarkable
results of the survey were shared and the participants were invited to comment on them in order to maximise our understanding. ‘Battle of the covers’ across the five countries in order to find creative ideas that would have appeal across the globe All participants of the five communities got the opportunity to take part in this discussion in English
in a central room. This way, cultural differences and local preferences were unveiled.
Reactivating a research community
The previous paragraphs informed us about the fact that a
The answer to the first question is ‘yes’: almost all
community provides one with automational,
participants from the communities in the US, Poland
informational and transformational benefits and that
and Germany took part again. For Italy (78%) and
it is a great tool to apply ‘triangulation’ to. The
China (70%), the reactivation was a bit less
examples provided in the previous paragraph on how we evaluated the 2013 Catalogue and pre-tested the themes
successful, although this was neither expected nor communicated at the start of the community.
and cover for the 2014 edition show that it can be a handy
Cultural differences in commitment are
tool while creating a new Catalogue as well. A community
probably at the basis of the lower numbers in
does not need to be ‘always-on’. You can perfectly align it
both countries. For all the questions on the different
with business planning. In this case the communities were
communities, we had at least 30 posts, which is what
used to draw learnings from the previous edition and
we needed to reach our saturation effect (Schillewaert
to get feedback on first ideas for the new one. We can
et al, 2011). Across the five communities we had
imagine situations later on in the creation process where we
2,807 interactions in a week’s time, which is 38%
might want to reactivate the communities for a short period
of the 7,261 we generated in the initial three weeks.
of time. Is that possible? Are participants indeed willing to
This is perfectly in line with what one may expect.
participate again and what does it take to encourage them to
do so?
How did we get to these results?
By making the participants feel part of the company as real ‘consultants’. Before the first phase of the project a kick-off was organised in a 30-minute chat session in smaller groups, during which we explained who the client was, what the goal of the project was and what was in it for them. During the first phase, participants received weekly newsletters on the progress of the project
and what the company was learning from the discussion. After the first three weeks and the workshop at the company side the participants got an update on how the debrief in Sweden went and pictures from the office were the Catalogue is created and a word from the team behind it. It is by doing so
and by keeping your promises in terms of incentives that you create a strong relationship with the members.
Global MROC projects: best practices
The fundamentals of the community approach work on a global scale. Just like the brands we are working for, we need to localise our way of working from country to country. In order to fully understand to what extent localisation of our methodology is required, we conducted several studies with moderators from our ‘Global Community Moderator Network’ (recruited in 30 different countries) and with local research participants. This way we co-created best practices for the different markets we are operating in. We found that it is important to
adapt your community to the local culture on five aspects. The direction the adaptations are made in can almost always be explained by the work of the Dutch academic researcher Geert Hofstede and his five dimensions to explain cultural differences between countries. Next we
explain the different dimensions in more detail and we also add some striking examples.
1
Reason to participate: intrinsic vs. extrinsic motivation From the first e-mail invitation for joining the
We also noticed that some countries are more
community onwards, it needs to be clear
extrinsically motivated than others. This is especially
what is in it for the participants. In
the case in the US and in Eastern European countries, but
almost all countries, the main reason to
for different reasons. Americans consider it normal that
participate is the possibility to have an
influence on the future of a brand or a product.
there is a payment to reward performance. In most Eastern European countries on the other hand an (monetary) incentive is perceived as a nice extra on top of their monthly income. Furthermore, in Poland, it is a must to gain the ‘trust’ of the members - trust in the fact that the agency or company behind the community will not harm them in any way and also in the fact that they will really get their incentive. The preferred type of incentive
differs from country to country. It is an illusion to think that ‘PayPal fits all’. In Asian countries such as China, the intrinsic part is important: they like to be connected with aspirational brands and share their wisdom.
2
Conversation guide: empowering vs. directive
A different culture also means different
Others prefer talking about the group, which is
attitudes and values, leading to a different
considered to be a safer option. See it as a
way of reacting to certain questions, tasks and
projective technique to let people talk about
exercises that moderators want participants to
their own situation, free of any pressure. The
perform. Some cultures, for instance, love to share
same holds for co-creation exercises. It is not a
a lot of details about themselves and their lives.
given in every culture that people are used to taking initiative. They feel better when they are only asked to give feedback about what already exists. It is important to map the country that
one is working in on those two axes (‘me’ vs. ‘we’ and ‘feedback’ vs. ‘co-creation’) and to adapt
the way of writing and (re)mixing topics for the conversation guide to it.
3
Role of the moderator: facilitator vs. authority One does not only need to adapt the way of
But it is also expected from the moderator that he/she
inviting and incentivising the members, the
is steering the discussion in the right direction when it
medium of data collection and the nature of the
is going off topic. In Poland on the other hand, the
topics in the conversation guide. The role of
moderator needs to be strict and almost literally direct
the moderator is also perceived differently
the members to the next question or task they need
from one country to the next. In Italy a moderator needs to facilitate and start the discussion. His/her role lies more in the background.
to look into. At the same time the moderator in Brazil is a social peer, he/she needs to be a formal professional in China and a like-minded person to exchange wisdom with in India. It is crucial to
know and manage all these different
expectations when running (multi-country or global) ‘Consumer Consulting Boards’.
4
Gamification: playful vs. serious Adding elements of ‘gamification’ to the community brings more richness to the table. In our research-on-research among our moderators, we learned that the level of and
the intensity with which you gamify your ‘Consumer Consulting Board’ need to
differ between countries. In Germany, for example, it is wise to limit it to a minimal level as it is culturally less accepted.
The elements described on the previous slides show that in multi-country projects you need to
start from a master conversation guide, which is important to make sure that there is a uniform way of working and that you exclude the effects of interviewer bias. But adaptations both in
content and style of the topics and in the way of moderating will be amongst other key elements in making the community a real success. Furthermore, it is wise to plan several debriefs between the different local moderators facilitated by the global research team: to challenge each other’s conclusions, let them go back to the results of their own country and come back with deeper and richer understanding in several iterative loops.
Creating a dialogue between the internal and external world
Only the business owners were following the
Previous ‘research-on-research’ (De Ruyck, 2011) and
community closely, although it should be convenient
our experience during this project have taught us that
as no travelling is required and one can take a
confronting stakeholders with real stories by real
look at the discussion on the community when
people is very impactful as an illustration of the
one feels like doing so. As a researcher there is a
main conclusions. They become alive. It is also an
clear need to give intermediate updates to the different
advantage that stakeholders can go back to the
stakeholders, as they do not follow the discussion
community based on the final report and read
spontaneously and there is a lot of information to
exactly what and how customers put it during the
digest. Moreover, this gives the business owners the
community. Finally, it is great that a community can be
opportunity to finalise the topic guide for the upcoming
reopened when you need it, perfectly in line with
days: going deeper into certain elements that are
the business planning.
really interesting or pushing the discussion in a new direction.
The research results have led to significant changes to the Ikea Catalogue App. We now know that the new
concept of the Catalogue was a big step into a new and right direction and improvements will be made for the 2014 edition, based on the research.
The future for research communities
The power of ‘Research Communities’ as a methodology lies in the fact that you have the ability
to work with more people, over a longer period of time, and that you get to know a lot from all kinds of different angles by combining different research tools and methods. The latter especially is still untapped potential in most communities. This case study describes how
fusing observational, qualitative and quantitative research methods lead to a deeper understanding and new insights. The case also demonstrates that ‘Structural Collaboration’ with consumers over time is valuable and leads to more impactful communication tools.
References
De Ruyck, T. et al (2010). ‘How Fans Become Future Shapers of an Ice-cream Brand’, Proceedings ESOMAR Qualitative Guin L., Diehl D. and McDonald D. (2012), Triangulation: Establishing the validity of qualitative studies, IFAS Schillewaert, N. et al (2011). ‘The Darkside to Crowd-sourcing in Online Research Communities’, CASRO Journal De Ruyck, T. et al (2011). ‘Engage, Inspire, Act: 3 Stepstones towards Developing more Impactful Products’, Proceedings ESOMAR Congress
Tom De Ruyck
Frédéric Gennart
Pieter De Vuyst
Frank Naessens
Head of Consumer Consulting Boards InSites Consulting
Global Market Research Consultant Inter IKEA Systems
Senior Research Manager InSites Consulting
Senior Research Consultant InSites Consulting
Want to know more about Consumer
Tom De Ruyck Head of Consumer Consulting Boards
Consulting Boards? +32 9 269 14 07 tom@insites-consulting.com
Thank you!
@InSites marketing@insites-consulting.com www.facebook.com/insitesconsulting www.slideshare.net/InSitesConsulting