USER EXPERIENCE I’M A CHIEF EXPERIENCE OFFICER
SLA
xLA
USER EXPERIENCE
I’m a Chief Experience Officer Written by Marco Gianotten
In the past, innovations like the PC and mobile phone first made their entrance on the business market. Only much later, following substantial price decreases, they became commonplace among consumers. Now the reverse is true: companies mainly follow trends that emerge in the world of consumer technology. The consequence: end users place high demands on Corporate IT and behave like business-consumers. Many CIOs realise that user experience (UX) will play a prominent role within IT performance management. This chapter discusses the why and how. We live in an economy that increasingly
companies rely on the Net Promoter Score: the
revolves around perception: what customers
extent to which customers recommend the
feel, think and experience is the driving force.
company or product to friends and colleagues.
Social media, comparison sites and user change the way companies and customers
IT has a major impact on customer experience
interact, not only digitally but also physically.
User eXperience (UX) plays a prominent role in
In the retail world, the credo is increasingly
the services and extends to the design, the
‘bricks and clicks’: strong growth in online
marketing and the management of customer
retail is regularly associated with investments
channels, such as websites and contact centres.
in the conversion of counters, shops, show-
Services and products may in no way harm the
rooms and bank branches into experience cen-
customer experience. Designing, testing and
tres and flagship stores. The experience
improving the usability of websites is common-
economy might look like a fad, but the way in
place in the world of e-commerce. Companies
which managers’ measure the performance of
do their utmost to help customers as much as
their own organisations has already changed
they can during the orientation and sales pro-
significantly. At more and more companies,
cess and actively try to avoid ‘shopping cart
customer service is no longer judged in terms
abandonment’. In e-commerce, there is a direct
of efficiency (for example, the number of min-
relationship between usability and commercial
utes per customer contact or costs per FTE)
ratios, such as average revenue per order and
but in terms of customer experience. Many
repeat visits. Mobilising positive customer
reviews feed the experience economy and
02 USER EXPERIENCE WWW.GIARTE.COM
User experience is more than just technology. The technical performance of systems is often different from what users actually experience. There may be a considerable difference between the absolute availability of core systems (usually far above 99 percent on the basis of 168 hours per week) and the perception of the users. What do they notice in practice of less than 1 percent technical downtime?
3 April 2013 produced more than twentythousand tweets (#ING) in just a few hours.
The employee is also a consumer In the experience economy, consumers hate it when companies do not do what they promise, if helpdesks do not really help or are hard to reach, or if the information provided is inadequate or appears to be incorrect. The experience economy also hurtles into the world of business IT. The modern consumer is not only found in the high streets or online stores, but also within your own organisation. There they
Health insurer Agis measured how its IT users experience availability. Their experience was much worse than one would suspect from the technical availability. Every minute of downtime was magnified, especially during peak periods; systems’ slow responsiveness was seen by users as a form of unavailability.
are ‘dressed’ as employees and end users, or the internal IT customers. As end users, they adopt more and more consumer behaviour: IT systems should ‘just work properly’ and the IT department ‘should be easy to do business with’. End users hate it when they have to call back because the problem is not completely solved, because they have to keep on repeating what the problem is, or because they are always
Poor usability of business applications leads to longer learning curves, lower labour productivity and dissatisfaction. Usability sometimes also comes under pressure with off-the-shelf software. For example, Microsoft was harshly criticised during the introduction of its software package for Windows 8 because of the high learning curve and an inconsistent user experience: Windows 8 works differently on a tablet than on a desktop.
faced with people who are not empowered to solve the problem.
What does the end user actually want? End users expect user-friendly business applications and portals, a decisive service desk and smart mobile apps that allow them to monitor or control key processes. As organisations discover that working entails more than bricks, brains and bytes (the key elements of the New Way of Working, short NWoW), the expectations of IT are raised. With another way of working for employees, the emphasis shifts to behaviour: the conduct, the content of the
experience produces sales. The growing influ-
work, working together in multidisciplinary
ence of IT on our daily lives as consumers
teams and learning by sharing knowledge in the
makes an ever increasing impression on our
workplace with co-workers. The performance
user experience. If online banking is briefly
of IT is becoming increasingly important here:
unavailable, this immediately causes negative
it affects the personal productivity of employ-
emotions: that are socially contagious. The
ees. Users (and their managers) are therefore
problems with ING’s Dutch internet banking on
becoming more critical and verbal: they con-
03 USER EXPERIENCE WWW.GIARTE.COM
sider properly working IT as a matter of hygiene; problems and obstacles in its use have now become dissatisfiers. If users complain about the sluggishness of a core application such as CRM on their tablets, the IT department does not stand on strong ground if it argues that ‘the application is performing properly according to monitoring in the data centre’. There is no point relying on user experience if the negative user experiences are downplayed or dismissed with the argument that ‘the SLA has been complied with’. In other words: the most important step is to recognise that experience, and thus the perception of the end user, is the truth.
What do IT organisations do? IT organisations are until now used to measuring and managing mainly on the basis of technical performance. With classical office
many CIOs realise that user experience will play a prominent role within performance management
automation, that was hardly ever a problem: IT organisations – or their service providers in the
nical causes because IT had control over the
case of outsourcing – had their own servers,
entire chain. Nowadays more and more func-
networks, workstations and applications. If an
tionality is virtualised: delivered to notebooks,
application on a PC was too slow or hard to
tablets and smartphones – purchased by
access, it was fairly easy to figure out the tech-
employees themselves – by means of web appli-
We are the 99% The end-users 04 USER EXPERIENCE WWW.GIARTE.COM
The new measure for satisfaction. Who dares?
In its new outsourcing contracts, ABN AMRO has decided to rely on four KPIs that improve the cooperation between all parties in all the multi-vendor outsourced IT service supplychains: aka as collaborative KPIs. Two of the four KPIs are subjective in nature: business satisfaction and project excellence. Business satisfaction is the annual score that key decision makers award the IT. Project excellence is the evaluation of completed projects by business sponsors. The satisfaction is measured on three points: a) quality of the delivered functionality, b) lead time and c) value for money. A few years ago, DSM decided to use KPIs in which the voice of the end user is paramount. As a single point of contact, it makes no sense to close an incident administratively (closing a ticket) with the aim of reaching agreements about the average resolution time of the SLA, while the problem is not properly resolved for the user. The opinion of the IT user has become overriding: after the ticket has been closed, he assesses whether he is satisfied with the solution by means of an online survey.
In an updated contract with KPN, ING has decided to work with a ‘new style’ SLA: an experience level agreement (XLA). XLAs are characterised by the lack of penalties, the focus on continuous improvement and a closed loop. If an individual user has a bad experience following a service moment (incident handling, delivery from service catalogue, answer to a question), he or she is called back to resolve the complaint. Even if the cause of the complaint lies with the solution groups of third parties, KPN declares ownership of the entire chain for resolving incidents. This closed circle should lead to a culture of improvement (instead of a claim culture) where the user experience (rather than technical performance) occupies centre stage. As with ABN AMRO, ING also has a collaborative KPI for all parties (thus including ING itself): the satisfaction of employees about their workplace is expressed by a so-called happiness ratio. This is the ratio of users who rate their workplace with a 7 or more to those who award a 5 or less. The value 6 is neutral. The ratio is increased by one point each year.
cations or with mobile apps. With trends like
Classic SLAs are for technocrats
Bring Your Own Device (BYOD) and mobile
The good news is that more and more CIOs
internet, IT loses its absolute control over,
recognise the importance of user experience.
for example, workplace hardware, operating
More and more IT departments are conducting
systems on devices and networks. As a result,
research into user satisfaction. Nevertheless,
control over user experience also declines. But
customer satisfaction and user friendliness are
even without this control over the technology
often the poor relation in IT. After years of
you have to be accountable for managing and
relying on ‘hard’ data, it is not easy to give
improving user experience.
space to subjective indicators. IT departments have difficulty initiating the appropriate
05 USER EXPERIENCE WWW.GIARTE.COM
Focussing on user experience means that SLAs must be adapted to the new reality improvement processes on the basis of results.
It is important to combine soft data (opinions;
The methods used do not always provide any
answers to open and closed questions) with
useful control information; sometimes manag-
hard data (from IT Service Management sys-
ers do not know which buttons they should
tems, such as the active directory, service desk
press. For example, little is known about which
tooling and CMDB) in order to identify causes
leading indicators ultimately affect the lagging
of discontent. In this way, differences become
indicator user satisfaction.
clear between, for example, locations, type of users and type of workplaces. The combined
Focussing on user experience means that
data ensures that you can now provide the
Service Level Agreements (SLAs) must be
answers. For the different categories of inci-
adapted to the new reality: not only managing
dents, what is the critical time period after
on the basis of technical control ratios, but
which the satisfaction plummets? Is there a
also on the basis of output. On the one hand,
connection with working days or periods within
this output affects the impact of IT on the
one day? And with the type of users? In addi-
business and, on the other hand, in the percep-
tion to measuring the user experience, it is also
tion of customers and users. In outsourcing,
important to have access to tooling with which
new agreements – in the direction of XLAs,
user transactions can be measured for each
eXperience Level Agreements – will also
business process by means of a (mobile) device
have to be made.
and divided into components, such as render
User satisfaction 2.0: hard and soft User experience is becoming increasingly
time of the device, response time of the infrastructure and the end-to-end transaction time.
ity of management and the corresponding ser-
Sentiments and discussions: the social part of UX
vice. The measurement of user satisfaction
By analysing the open comments in satisfaction
about the workplace or business applications
research (with the aid of language technology),
will have to be richer and deeper in order to
the sentiments about corporate IT can be made
achieve a targeted improvement of quality.
clearly visible and connections found within
important with both the design and functional-
06 USER EXPERIENCE WWW.GIARTE.COM
Net Promoter Score In recent years, the Net Promoter Score (NPS) has successfully established itself in the row of top KPIs that (top) managers work with. The NPS indicates the extent to which customers are willing to recommend a company or organisation. It is now ten years since the NPS was conceived by Frederick Reichheld. He saw the NPS as a solution for the problem that a high level of customer satisfaction did not appear to predict the loyalty of customers. A disadvantage of working with the Net Promoter Score is that it is not exclusive; the figure is the total of the percentage of fans that awards a 9 or 10 minus the percentage of detractors that awards a 6 or less. The middle group of passives is not included in the NPS score. An NPS of -5 can consist of 45% promoters, 10% passives and 50% detractors, but can also consist of 6% promoters, 88% passives and 11% detractors. These are substantially different results with an identical final score. In addition, managers tend to opt for ‘low hanging fruit’: problem areas where
you can relatively quickly and easily implement improvements in order to quickly realise higher NPS scores. NPS also takes no account of the effect of outliers: an average favourable NPS score produces a distorted image if the 5% most dissatisfied customers furiously stir things up on social media. In other words, the way in which the NPS is used also says something about the management culture within an organisation. Furthermore, considerable doubts are expressed among consumers about the value of the NPS. Kumar et al (2008) published an article in the Harvard Business Review about the results of an analysis carried out among 9,900 customers of a telecom company. It emerged from this that 81% of telecom customers said they would recommend their provider but only 30% of them actually did so. No more than 8% of those who received the recommendation eventually became a profitable customer of the telecom company.
The good news is that UX can clearly be managed and that the results will contribute to a stronger IT-business alignment 07 USER EXPERIENCE WWW.GIARTE.COM
multinationals too. The answer not only lies in
between 1 and 5; the lower the number, the less
measuring and clever analysis; the physical dia-
the effort. The CES is perfectly suitable as a
logue with users is also important to encourage
customer experience metric for IT organisa-
empathy. More and more IT organisations use
tions and can also be used in combination with
customer panels to collect feedback. Rabobank
user satisfaction.
regularly organises IT user panels with a moderator; IT professionals are present as listeners
Another method that is widely used to monitor
and feedback (active listening) what they have
the user experience is recording the user’s
learned and would do differently in the services.
opinion at ‘moments of truth’ (such as after the
UX with Net Promoter Score: sense of non-sense for IT?
delivery of a service or the resolution of an incident) and then enriching this with ticket data (such as the start and stop time, where the
There are various methods and sources avail-
elapsed time is the actual resolution time) in
able for monitoring the customer experience.
order to identify causes of dissatisfaction in the
Some CIOs have implemented the Net
service chain. The recording of these opinions
Promoter Score to measure the extent to which
during ‘moments of truth’ – often by means of
users would recommend their own IT organisa-
automated research following a telephone call
tion. The advantage of the NPS is that this met-
or email exchange – has already been used for
ric is part of the marketing and sales
some time in the case of services provided to
management lingua franca. However, there are
consumers.
also a number of disadvantages attached to not know what they can do to improve the rec-
What is the ambition of the IT organisation?
ommended score. Furthermore, the NPS score
A service entity that mainly looks at the tech-
can be misleading and the question is also what
nology or a team that successfully contributes
IT end users can and should do with the rec-
to the growth and future of the business?
ommendation question, since they do not have
Those who have an eye for the latter, must be
to deal with a free market with multiple provid-
open to matters such as (end) user experience
ers for their IT.
and experience level agreements. The good
working with NPS scores. Managers often do
Measuring user experience with Customer Effort Score
news is that UX can clearly be managed and that the results will contribute to a stronger IT-business alignment.
In addition to the Net Promoter Score, a lot of attention has been paid to a new metric during the past five years or so. This is, more than the NPS, focused on the service that companies provide to their customers: the Customer Effort Score (CES). The CES indicates the amount of personal effort made by a customer to get something done by the company. The CES is applicable to proposals, questions, orders, returns and changing data. The CES is a five-point scale with the result being a number
08 USER EXPERIENCE WWW.GIARTE.COM
GIARTE P.O. Box 890 1000 AW Amsterdam The Netherlands Telephone +31 (0)20 622 3444 info@giarte.com www.giarte.com