FIVE METRICS THAT MAKE OR BREAK MOBILE APPS
APPS ARE TAKING OVER THE WORLD Ever since their advent in 2007, new generation smart phones
Starbucks gets 14% of its in-store U.S. payments from its mobile
and apps that run on them have become an indispensable part
app, and 25% of Starbucks' loyalty program transactions are made
of life for a quarter of the world’s population. Mobile apps are
via the mobile app.
becoming the de facto gateway for users to consume Internet services. App publishers are working hard to create apps that have high retention. Localytics studied the average number of times an app was used. Between 2011 and 2014, the percentage
Developing effective apps that provide best user experience and have high retention is both a science and an art
of apps used 11 or more times comprises nearly 40% of all apps up from 26% four years ago. Apps that have been runaway
Creating successful apps that have high retention is both a science
success are those that engage users, and provide a great user
and an art. An analysis of successful app adoption and retention
experience. Starbucks is an example of a successful mobile app
reveals that they are created from a clear strategy and rooted in user
that has a high retention. This is a result of a deep understanding
behavior research. App publishers usually measure a few key
of user behavior and leveraging this understanding to provide an
metrics that provide vital information on app adoption and retention
unbeatable user experience. Starbucks’ mobile strategy is to
to tweak their apps so that they provide the best user experience,
increase engagement with their customers who throng its cafĂŠs.
and become sticky.
Five Metrics that Make or Break Mobile Apps
1
THE ART AND SCIENCE OF USER BEHAVIOR RESEARCH The app provides the opportunity for the publisher to literally be
app is not context aware of the transactions of the user in the web
in the hands of users all day long. When the closeness of interac-
application.
tion is this intimate, the app must enhance user experience and loyalty. Most app strategies focus on achieving the largest
Apart from strategy, it is important to research user behavior.
possible engaged users who use the app on a regular basis over
Insights from user behavior research along with strategic goals are
an extended time period. This signifies user stickiness that is
critical inputs while designing and developing the app. For example,
often a result of high perceived value and great user experience.
it is critical that users start on the right note when they first open the
The measure of user engagement is often specific to the
app. User research is used to identify how much the user wants to be
Internet services industry vertical. For example, a mcommerce
taught on using the app, and how much the user wants to self-dis-
app measures engagement by number of users who purchase,
cover. Analytics is used to determine user segments with specific
average purchase size, and the point at which people abandon
initial user preferences based on demographic data, and this
their shopping cart. Two aspects that are increasingly a
information is used to customize the app for a user.
must-have in mobile apps are cross-platform compatibility and seamless access anywhere anytime. Once the app is developed and introduced in the market, it is imporApp publishers realize that they need to develop mobile apps to
tant to measure and monitor a few key metrics. These metrics
reach a wider base of users across platforms, and those who
provide insights on how well the app is performing, and form the
switch platforms and devices. The users demand and expect all
basis for mid-course corrections. UX researchers at Google have
the apps that are available to them in one platform are also
developed a strategic framework to define and choose appropriate
available in other platforms. Not only that, users demand the
metrics categories that reflect the quality of user experience. This is
same overall user experience is maintained across different
known by the acronym HEART.
platforms. App publishers are increasingly adopting cross-plat-
Happiness: Measure of user attitudes. E.g. user experience
form development technologies to ensure their apps are
related metrics
developed on code bases and technology that allow the app to
Engagement: Level of user involvement. E.g. frequency of use
be distributed and deployed across multiple mobile platforms
Adoption: Active users of the app. E.g. value metrics like
and devices.
visitation rate Retention: The rate at which existing users are returning. E.g.
Users today have multiple screens. This means that they are
number of active users
constantly on the app from a mobile phone or tablet. Or they can
Task success: This includes traditional behavioral metrics of
be on the web portal on a computer. For example, a user of an
user experience. E.g. session length related to time required
ecommerce service can start adding an item to the shopping
for completing a task
cart while accessing the portal on a computer at work in the morning. The user can modify the shopping cart by adding
This framework is applicable in multiple use contexts including
another product by using the mobile app during lunch. Finally,
mobile apps. The Google UX team has used the HEART framework to
the user can access the app from a tablet post dinner to check
a wide range of projects within Google, and found it useful in different
out, and make the payment. The user expects the app in the
use contexts. We believe that the HEART categorization is apt for
smart phone, in the tablet and the web portal all seamlessly
mobile apps since it provides a collectively exhaustive set of catego-
work in tandem to provide a seamless anytime anywhere experi-
ry list that serves an important design guideline for app developers.
ence. User experience with the mobile app drops if the mobile
Five Metrics that Make or Break Mobile Apps
2
FIVE CORE METRICS THAT ALWAYS MATTER A plethora of metrics that aligns with the HEART framework can be measured. While many of them are nice to measure, we believe there are few that are a must have to get a good grip of adoption and retention.
Value The most important metric is value. Often, this metric gets
Internet services industry domain. For example, it is visitation rate
dubbed as “old marketing” and is not given the importance it
and value of purchases in a mcommerce app or number of ads
should. The value metric is comparative. In contexts where the
streamed in a news app. According to Compete, during April –
app is the only channel for the customer of Internet services, the
September 2013, Amazon had an average of 7.16 visits/month on
value of different segments of users is analyzed. Where the app
mobile vs. 3.39 visits/month on desktop . This provides a context to
is one of many channels, the value of customers from the mobile
Amazon’s strategy to introduce the Fire mobile phone preinstalled
channel is compared to value of customers from other channels.
with the Firefly app to retain mobile users, and make them stickier.
It is useful to identify the segments of users who provide the highest value to the publisher. Measuring value depends on the
Five Metrics that Make or Break Mobile Apps
3
Retention Rate and Frequency of Use A direct measure of retention is the percentage of users using the app say thirty days or ninety days after installing the app. This is a good measure to judge if the user finds the app engaging and also whether the app is used frequently on a continuous basis. A related metric is frequency of use per week that is the number of times the user uses the app in a week. Flurry analyzed a large universe of apps for retention rate at thirty days and frequency of use per week. The summary of their findings provides interesting insights:
Retention rate over 30 days
Frequency of use per week
Examples of app categories
Impact on retention
>35%
>5
Communication and News
Deliver enduring value over time
<35%
>5
Deliver value in bursts
<35%
<5
Streaming Music, Dating and Social Games Personalization (e.g. screens)
>35%
<5
Airline, hotel or rental car-booking
Used infrequently but deliver very high value
Once-and-done
Across all categories of apps, the average retention rate over 30 days is 54%, over 60 days is 43%, and over 90 days is 35%. The average frequency of use per week across all categories is 3.7
Session Length
A more engaged user likely to have a high retention rate is one who spends more time with the app. This is the first level of measurement for user experience and stickiness. While a long session length is a good measure of user engagement in most Internet services contexts, it may not be so in a few. For example, a long session length for a taxi finding app will mean that the app is not effective in quickly finding the user a taxi. Many mobile app sessions last less than one minute, a few sessions last longer than ten minutes, and the average session length comes is just over five minutes.
Five Metrics that Make or Break Mobile Apps
4
User Experience One of the criticisms of the session length metric is that users
Other measures require a detailed feedback from users on aspects
may be spending time on the app not doing what they want to
like usability of the app, endurability of experience and novelty of
do but figuring out how to do it. The time spent trying to figure
experience. Some app developers use in-app feedback especially to
out how to do something does not lead to a good user experi-
report about crashes, bugs and UI. The most important aspect of
ence or retention. Publishers analyze the reviews that users
measuring user experience is to take action â&#x20AC;&#x201C; redesign or reengi-
provide on the app stores to understand user sentiment. A quick
neer the app to improve user experience.
measure of user experience is the ratings score on app stores.
Active Users Active users are how many unique users use the app on a daily
the highest ratio of daily active users to monthly active users of
or monthly basis. Daily active users are important since they
72%. This is a measure of the high level of stickiness of users.
are of the core group of users that the app publishers need to
Communication apps have a ratio of about 50%, and other apps a
engage and retain. An interesting example is Whatspp. It has
ratio between 10% and 20%.
The focus of these metrics is on adoption and retention. They help the app publisher to fine-tune strategy to provide value to users, and also derive value from them.
THREE USER RETENTION GOALS THAT ALSO MATTER App developers need to focus on three important retention
Stickiness: Gamification is a good strategy to engage users to
goals - user retention, existing user retention and stickiness.
make them sticky to the app, and increase usage. This should be based on a deep and contextual understanding of user
New user retention: App developers need to ensure that the
behavior insights. These insights should translate into design
app is engaging and impactful in a super short time frame of
elements in the app that targets the userâ&#x20AC;&#x2122;s intrinsic
the first sixty seconds of first use to retain the new user.
motivation.
Achieving high new user retention should be a specific focus of ethnographic user research and app strategy.
Connecting user behavior insights to the strategic business goals, and tracking key metrics ensure that the app is what both the user
Existing user retention: Fatigue with the app prevents exiting
and the publisher want. User behavior research is essential to
users from returning to the app. The app developer should
unravel counter-intuitive user behavior nuggets. These insights
leverage smart analytics and user research to determine what
along with a focused app strategy including gamification are essen-
new to showcase and how to showcase it to the user. While
tial to increase customer engagement with the app, and motivate
gamification is gaining popularity as a strategy to engage more
them to consume more of the services provided by the app. This
with existing users, another effective way to engage is through
leads to higher adoption and retention of apps.
targeted in-app messaging.
User behavior research is essential to improve app adoption and retention
Five Metrics that Make or Break Mobile Apps
5
EMERGING TRENDS IN ANALYTICS AND USER BEHAVIOR Insights from Ethnographic User Research Ethnographic user research helps discover hidden use cases
users want to know about the latest styles introduced from the app
and usage patterns that often provide deep user insights. It
while the users may equally value information on nearest store.
helps identifying user needs that are not satisfied by the mobile
This insight is provided by ethnographic user research where the
app, and is used to assess the market for features that are not
researcher notes that the user searches for the fashion brand in the
yet in the mobile app. These insights are increasingly used to
map app since the app does not provide that feature.
develop or modify the mobile apps that have high adoption and retention. For example, a fashion brand may believe that the
Gamification Gamification is the use of game like rewarding for completing a
Gamification provides insights and data to sales teams that help
task or friendly competition to trigger the usersâ&#x20AC;&#x2122; intrinsic
identify and focus on important behavioral traits. Gamification lets
motivation. User motivation to use apps with gamification
every member of the sales team identify and use the key behavioral
aspects comes from friendly competition that positively
traits. Once the members experience the reward and recognition,
motivates humans. Gamification usually does this in the form
adoption of the desired traits increases. This improves the overall
of leaderboards. If the user has a competitive spirit, gamifica-
performance of sales teams. The focus of gamification in the near
tion in the form of leaderboards in apps can create a great user
future will move away from the implementation of low hanging
experience for users, and aid app retention. Another domain
fruits like leaderboards and badges, and focus on behavioral design
where gamification is making inroads is in customer relation-
aspects like co-opetition.
ship management (CRM). According to a salesforce.com study, most gamification initiatives in CRM are successful.
User Analytics in Mobile Enterprise Application Platform (MEAP) by
future. We expect leading MEAPs to provide integrated user analyt-
established companies often leverage a MEAP. A good example
ics, and also integrate with a broad ecosystem of third party mobile
is the context of extending the customer portal as a mobile app
app analytics providers. The MEAPs are likely to include an analyt-
in B2C contexts like banking, telecom services or any subscrip-
ics dashboard for collecting user feedback and analyzing usability
tion based services. The existing links in the web portal to the
metrics. We expect MEAPs to offer libraries of metrics that can be
back-end systems like customer accounts, service provision-
collected from the appsâ&#x20AC;&#x2122; usage. While it is a wish list now, MEAPs in
ing, billing, etc. need to be replicated in the mobile app. MEAPs
the future must leverage analytics to suggest modifications or
are efficient development environment to convert existing
modify the app to increase user adoption and retention.
Mobile
application
development
and
maintenance
enterprise web applications into mobile apps. Most MEAPs do not collect metrics on adoption on retention or provide integrated user analytics. This is likely to change in the near
Five Metrics that Make or Break Mobile Apps
6
ARICENTâ&#x20AC;&#x2122;S EXPERIENCE Aricentâ&#x20AC;&#x2122;s mobile enablement services include mobile app development, testing, advanced visualization, enterprise mobility enablement, location-based services, app deployment on cloud, and UX design for the Internet of Things. As part of its mobile enablement
-
services, Aricent advises clients to choose appropriate third party analytics solutions and metrics to measure, and integrates these solutions with the mobile app.
REFERENCES 1. http://www.emarketer.com/Article/Smartphone-Users-World wide-Will-Total-175-Billion-2014/ 1010536 2. http://info.localytics.com/blog/app-retention-improves 3. http://www.paymentssource.com/news/starbucks-mobile-app-now-used-in-14-percent-of-uspayments-3017341-1.html 4, https://blog.compete.com/2013/12/18/mobile-vs-desktop-traffic-top-retailers/ 5. http://www.flurry.com/bid/90743/App-Engagement-The-Matrix-Reloaded#.VA7QJU3lo2w 6. http://sequoiacapital.tumblr.com/post/77211282835/ four-numbers-that-explain-why-facebook -acquired
Aricent is the worldâ&#x20AC;&#x2122;s #1 pure-play product engineering services firm. The company has 20-plus years experience co-creating ambitious products with the leading networking, telecom, software, semiconductor, Internet and industrial companies. The firm's 10,000-plus engineers focus exclusively on software-powered innovation for the connected world. frog, the global leader in innovation and design, based in San Francisco is part of Aricent. The companyâ&#x20AC;&#x2122;s key investors are KKR & Co. and Sequoia Capital. info@aricent.com
Š 2015 Aricent. All rights reserved. All Aricent brand and product names are service marks, trademarks, or registered marks of Aricent in the United States and other countries.