Data, Decisions and Design 48 nudges to push BI strategies further
Data, Decisions and Design How to use this deck The deck consists of 48 cards. Each card addresses an idea and offers a hint to help you rethink your organization’s approach to data, decisions and design. Use them for planning, in meetings or in workshops by either:
• Shuffling the deck and drawing 3 cards • or asking each person to draw a card • or dividing out the suits and drawing 1-3 cards from each suit
• or simply picking a card whenever you need to encourage fresh ideas
Conceived and published by Qlik’s Innovation & Design team
Perfect data …
01
Perfect data equals unicorn tears Perfecting data requires herculean efforts, and it becomes harder as data volume and diversity grows. Fact: it simply isn’t possible to make all sources perfect anymore. Even if it was, what would the perfectionist do anyway? Conform data with the risk of over-cleaning or “correcting” errors incorrectly? We have to accept the fact that data is rarely perfectible, and getting less so. Apply the idea of layered quality tolerances to the data flowing into your organization. A Virgin with a Unicorn Domenichino, c. 1604–05
DATA
01
A single version ‌ 02
A single version of the myth The ‘single version of the truth’ is a distraction. Why? Because unless BI models evolve at the speed of business, which they rarely do, they become much less useful to decision makers. People end up managing to a myth that was relevant when the model was built but becomes increasingly less so, to the point where it departs from business realities. Review your BI output’s relevance to management – in particular its metadata.
The Mercator Projection
DATA
02
KPI … 03
Killing Personal Initiative Key Performance Indicators are primarily a mechanism of managerial control. As descriptors they have little to do with analysis, more with situational awareness. Although a common part of BI projects a KPI bias must not be allowed to dominate, as this can kill personal initiative by restricting creativity. Work with decision makers on how approaches to data and discovery might change to support innovation efforts, by moving the focus towards self-service and playing with data.
DATA
03
Context … 04
Context is king Most BI deployments contain only internally sourced data. Result? Data tunnel-vision where decisions made with BI lack external context and are unlikely to yield the best outcome. Context provides orientation and mitigates risk. What data would your analysts add if they could? How fast do they need it? Using external data is not a new idea, but what has changed is speed – not only gaining context but doing so fast enough to impact decisions.
Edward H. Adelson’s checker shadow illusion Squares A and B are the same colour
DATA
04
Reporting … 05
Reporting is not control Financial and management reports give the illusion of control. But reporting a situation isn’t controlling it. Control comes from going beyond “what happened?” description to diagnostic analytics where people ask and answer “why?” questions repeatedly. When they explore root causes, interrelations, trends and shifts in the data iteratively they gain a better understanding of what’s reported. Test if your decision makers are using self-serve enough to address “why?” questions. If they aren’t, why not? Time-and-motion study by Frank Gilbreth (1916)
DATA
05
Ethics …
06
Ethics is a design problem Data doesn’t have ethical dilemmas. Data is just data. However the design process is about exclusion. We select the problems, methods and solutions. When we design with data, whether it’s a chart or an entire information system our choices have biases that may not be apparent to us. Always ask yourself, what are we leaving out of the design and why? Strive to understand who benefits from this exclusion and who suffers.
The Isolator by Hugo Gernsback (c. 1925)
DATA
06
Personal data ‌ 07
Personal data is currency Organizations won’t be able to simply grab people’s personal data for much longer. As society’s data literacy increases people’s understanding of their data's value and currency will become more acute. This new awareness, coupled with an explosion in the variety and depth of the data generated by individuals, will lead to increasing challenges to data access and privacy for analytic usage. How will your data strategy be impacted if your customers turn off the data tap?
EU data protection rights
DATA
07
Data …
08
Data can be viscous Numeric data flows through organizations and is why BI initiatives are quantitatively led. However, optimal decision making requires both quantitative and qualitative input. This is a challenge. Qualitative data is more viscous: thick and slower moving. For example, when researching behaviours, an ethnographic study may describe situations, record conversations and annotate interactions. Does your broad information strategy even consider sources like this? Collaborate with teams working with qualitative data to learn how they analyse and parse information. Martin H. Moynihan’s field notes on Cyanerpes, Panama, 1961
DATA
08
Governance … 09
Governance is more carrot than stick Governance of analytics is not just regulation of data access. Governance is a form of social contract, “of the people, by the people, for the people”, under which they have rights as well as responsibilities. However, these are often unclear to people using data for analysis, as they’ve not been expressed. Draft and discuss a (simple) analytic ‘bill of rights’ that sets out what people can expect to get and what they’re expected to give.
Official draught of the first twelve amendments to the Constitution - from New York Public Library
DATA
09
Analysts analyse best ‌
10
Analysts analyse best what they analyse least Familiarity sometimes breeds contempt. If someone analyses the same data repeatedly the likelihood is that they’ll eventually overlook something important or miss insights. Shake things up. Get them to visualise data in novel forms or use different form factors to challenge boredom and offer another viewpoint. Better still; circulate analysts so they move between functions and teams periodically. This movement may have the additional benefit of raising the level of analytic maturity too.
DATA
10
bitter salty
salty
sour
sour sweet
Facts are ‌ 11
Facts are less permanent than you think The relevance of information quickly decays over time. Our desire to fix data and definitions runs counter to the fact that world it inhabits is moving. The way people use analytic information is dictated by the moment in which that experience takes place. How relevant are the facts you are providing? Check how often your BI users export to Excel – a lot of this activity could be an indicator that the information artefacts available to them have lost currency.
The tongue taste map is a myth
DATA
11
Heuristics …
12
Heuristics can be relied on Heuristic thinking, derived from familiarity and practice, can offer hints on what’s happening, quickly picking up the signals in the data. Of course, this relies on individuals’ learnt understanding of their measures and a keen sense of how accurate they are. When employed as triggers heuristics can be useful as a way of detecting information scent. How can you build out from what decision makers know intuitively in BI apps? What shortcuts do they use to spot what needs deeper analysis? You can use your fingers to estimate how many hours of day light are left – 1 finger is 15 minutes
DATA
12
Doubt …
13
Doubt not belief Doubt everything. To make good decisions people need to be doubtful of the information presented to them, no matter how compelling the format. They need a scientific mind-set that says “this is what we measure and observe now, but it may be erroneous and our hypotheses may be wrong�. Think about how you might use A/B testing and control groups to test the outcomes of decisions influenced by analytics.
From Magic; stage illusions and scientific diversions, including trick photography by Albert A. Hopkins
DECISIONS
13
Storytelling …
14
Storytelling must create debate We use storytelling about data to persuade others to act. But knowledge and real understanding comes through challenging rather than accepting. It’s imperative to design into the technology and processes ways for people to debate and question what is presented to them. This allows them to get access to what’s missing from the picture, to find the untold story, the neglected angle, to ask ‘what did you leave out or underplay?’
Cicero’s Oration against Catiline in the Senate Hans Schmidt (1920)
DECISIONS
14
Persuasion …
15
Persuasion is a dark art When we set out to persuade using data we often end up employing tricks to help ‘facts’ resonate with an audience. We use narrative, comparison, emphasis and visual techniques to convince others of our point of view. Dark art? Maybe, but it’s the normal way people drive to a decision resolution. Persuasive skills are important. Do your analysts know how to set out an argument from data? If not how might you train them to be better storytellers?
Victorian pamphlet promising to teach its middle class readers how to perform magic tricks - British Library
DECISIONS
15
All narrators ‌
16
All narrators are liars When people tell a story with data it reflects their point-of-view, which might make them an unreliable narrator. Unreliability could emerge because they have an agenda or through errors or omissions. Consider how you can use the inherent unreliability in data stories as a trigger to drive greater engagement. Data stories need to be unpicked, people need to discuss, and the narrator questioned to get the best decision outcome. What forum could this happen in?
DECISIONS
16
Data … 17
Data has no agenda When applying metadata models to data (as dimensions, measures, visual forms, attributes etc.) we’re inevitably reflecting a set of preconceived notions. That’s fine, as long as it’s done with awareness of any agendas in the definitions. As such, the metadata model may reflect bias or outmoded structures that in turn constrain thinking about decisions and innovation itself. Could your data be modelled differently to undermine inbuilt assumptions? Plan to review metadata definitions with decision makers.
DECISIONS
17
Culture …
18
Culture eats bananas for lunch Repeating the tired cliché that ‘culture eats strategy for breakfast’ is useless. It’s a vote for apathy. Everybody knows that changing cultures is extremely hard, but people are smart. If they can see that it is to their mutual benefit then they’ll change behaviour. It may be slow but it will change. Further, culture is never monolithic – micro-cultures abound in teams and departments. Build quick wins by matching analytic tech to micro-culture. The laggards will follow.
DECISIONS
18
Artificial intelligence ‌
19
Artificial intelligence thinks differently to us ‘“I’ve never seen a human play this move… So beautiful…” As he [Fan Hui] played match after match with AlphaGo over the past five months, he watched the machine improve. But he also watched himself improve. The experience has, quite literally, changed the way he views the game… In the months since October, AlphaGo has taught him, a human, to be a better player. He sees things he didn’t see before.’ – Wired.com Expect AI to impact BI in unexpected ways.
“My Wife and My Mother-in-Law” by William Ely Hill also known as the “Boring figure” after Edwin Boring
DECISIONS
19
Data literacy … 20
Data literacy could divide us Literacy dictates life chances – 75% of US prison inmates are effectively illiterate. Data literacy – “the ability to read, work with, analyze and argue with data” – will soon dictate organizations’ chances. Hiring data literate staff is a critical path item for any organization that’s going to compete effectively using its information assets. Don’t allow your organization to sleepwalk into having data-haves and data-have-nots. Design education programs that raise general data literacy (not just technical capability).
DECISIONS
20
Moving fast ‌
21
Moving fast requires time to think The operational tempo of business has sped up. Decisions need to be made more quickly in less time, but people still need time to think. How do we solve this paradox? Slice their time differently. Instead of spending too much of their decision window wrangling data give them an experience that collapses the data prep stage (through governed data sets) and frees them to iterate more times through the data by themselves. More iterations in less time.
Ernest K Adams - Electric Metronome
DECISIONS
21
Analysis …
22
Analysis will only get you halfway there Analysis is part of a process, not an end in itself. Findings discovered through analytics are pointless unless they impact a decision. However most BI program managers don’t know whether they’re impacting decisions or not. Establish a periodic survey to explore decision makers’ views gathering both quantitative and qualitative data. At a minimum canvass them on timeliness, relevance, user experience and, critically, their ability to act on the analysis available to them.
Tatlin’s Tower would have required more steel to build it than available in Russia at the time (1919–20)
DECISIONS
22
Heuristics …
23
Heuristics cannot be relied on A tried and tested shortcut to a decision may work passably 9 times out of 10 but be disastrous on the tenth occasion. Managers’ experience-based “rules of thumb” fail if a situation is out of their experience. Further, heuristics are highly subjective and fall prey to cognitive biases. Beware of people using gut feel over data in your organization. Run training to educate managers and analysts in being more data-driven and questioning of their own assumptions.
You can use your fingers to estimate how many hours of daylight are left, but it all depends on where you are
DECISIONS
23
Social …
24
Social is niche Facebook and Twitter have created the illusion of all-encompassing social networks. But in reality it's a lot more tribal. People cluster around ideas, like-minded people and activities. Some may appear in multiple groups, acting as the connecting fabric between disparate teams. To increase analytics adoption, look for the social butteries in your organisation, those who cross silos and connect groups. They are perfect vectors for spreading insight, processes and ideas.
From British Butteries, James Duncan (1840)
DECISIONS
24
People … 25
People are born analysts You cannot not see the snake. Humans evolved to cope with vastly complex information and critical choices. As a result people have outstanding skills in pattern recognition, outlier detection, sorting, categorization and associative thinking. Put trust in your peoples’ capabilities. Think about how you take advantage of their innate analytic skills both via visual and other forms. Free more people to do more analysis more often with more data.
DESIGN
25
Notifications … 26
Notifications are tyrannical Being made aware of changes is undeniably useful. Often the quicker people can respond to fresh information the better. But what happens when everything is changing all the time? When all devices are demanding that people read this, stand up, walk more steps, respond now! Just because a change has occurred do people really need to know about it immediately? Work with those using the information to design rules for what to surface and when.
DESIGN
26
The map … 27
The map is not the territory A map is an artificial representation of a space, created with a specific need or agenda (e.g. a roadmap). How people or objects inhabit and move through a ‘mapped’ space is often different from that which the map suggests. Always look beyond the map to the place. By comparing the geospatial representation to behavioural data you can reveal paradoxes and gaps in your analytic understanding. Find ways to augment the maps you use with other data.
Map of the 1854 cholera outbreak John Snow, C.F. Cheffins (presented 1855)
DESIGN
27
Beauty …
28
Beauty is not truth Humans tend to give more credence to attractive things (the ‘Keats heuristic’), but beauty is not truth. A good looking dashboard or report gets people’s attention and buy in. Consider how you might get beyond that initial attraction. Review the design of some core BI outputs through this lens. How could different forms (search, tables, text) help people overcome their tendency to be misled by aesthetic distraction, and see the reality in the data, beautiful or ugly?
Portrait of John Keats by William Hilton
DESIGN
28
Infographics …
29
Infographics aren’t much use Today’s visualization mavens are as bad as yesterday’s Excel gurus. Specialists designing infographic-style visuals which are just so and then sending them to people to view is much less useful than many people making their own data visualizations. The maven approach is exclusive, not collective and not about engaging as many people with data as possible. Ask the following: What’s the point of ‘just so’ visualizations? Who benefits? How do they work across devices? Phrenological head and chart M. Nutting (c. 1857)
DESIGN
29
Visualization …
30
Visualization is myth making Numbers are compelling. Charts are credible. Seeing shouldn’t be believing, but all too often it is. There are agendas at play in how data is designed to be shown – for example using particular emotive colours or manipulating axes to exaggerate or minimize change. These create a visual mythology. Visualization design decisions can impact how data is regarded. Spend some time looking at the some of your most used visualizations. Are they obfuscating data?
Napoleon’s Russian campaign of 1812 compared to Hannibal’s Punic War - Charles Joseph Minard
DESIGN
30
The map … 31
The map is the territory Signage and way finding is mapping at 1:1 scale. We use in situ hints and markers help us navigate and even leave fresh markers for others to follow. When we work in information spaces these take the form of bookmarks, hyperlinks and annotations. Help people understand the data by designing useful navigational elements as part of the system. Most importantly look at how you might help those who explore data make their marks and highlight the pathways for others.
Transcontinental Air Mail Route Beacon Photograph by Dppowell via Wikipedia
DESIGN
31
Visualization works …
32
Visualization works with one sense alone Not all people are equally visually oriented. Humans use an individual mixture of sensory inputs to learn – auditory/reading, visual and kinaesthetic. Focussing on visualization may leave some people behind and hinder their understanding. Experiment with designs and tech that could reach more people by using information delivery mediums across the learning styles. What about text to speech, 3D printing or large, shared touchscreens? The aim is to remove barriers between people’s brains and data assets.
DESIGN
32
Design thinking ‌
33
Design thinking requires making something Design thinking is practical and not a solo activity. It involves small teams going out and researching a problem, talking to people, formulating hypotheses and iterating on possible solutions. At its heart is imagination and the freedom to ask; “why?”, “what if?” and “what will this change?” When faced with an information design problem try mixing up your teams: bring users, engineers, analysts, designers and support staff together and give them space to try stuff out.
DESIGN
33
Persuasion …
34
Persuasion can be ambient Changing behaviour isn’t always about enforcement. Often a nudge can make a significant impact. Victorian designers added an etched bee to urinals as a visual joke to encourage mens' aim (the latin for bee is apis). Aad Kieboom, who implemented a similar approach at Schiphol Airport, suggested the impact it has on the cleaning budget amounts to a saving of 8%. What sort indicators could you surface to help awareness and shifts in behaviour around analysis?
DESIGN
34
Readers read best …
35
Readers read best what they read most People learn to ‘read’ data and BI through repeated exposure, and they may find that things that don’t conform to their normal expectations feel wrong or awkward. If they are business decision makers (as opposed to analysts) make only small changes to their guided apps and information tools over time. Introducing different formats or styles to communicate data and insights may increase the cognitive load and prove unpopular. List out apps or user roles where consistency of form is a benefit.
Although a compelling idea there is no Cmabrigde Uinervtisy research
DESIGN
35
Gamification …
36
Gamification is not likes Taking static artefacts and scoring people for looking at or liking them will not address the underlying adoption issues facing many BI projects. A culture of analytics has to emerge from improved data literacy and meaningful engagement with data. Behavioural change requires motivation, ability and triggers. Build for learning not just consumption enable skill progression and depth of interaction. If you’re going to gamify BI, approach it via user interaction, not crude scoring.
Parlour game card from the 1920s
DESIGN
36
Self service
37
“People are generally better persuaded by the reasons which they have themselves discovered than by those which have come into the mind of others.� Blaise Pascal
37
Collaboration
38
“Never doubt that a small group of thoughtful, committed citizens can change the world. Indeed, it is the only thing that ever has.� Margaret Mead
38
Certainty
39
“It aint’s what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so.” Mark Twain
39
Facts
40
“Facts and theories are different things, not rungs in a hierarchy of increasing certainty. Facts are the world’s data. Theories are structures of ideas that explain and interpret facts. Facts do not go away while scientists debate rival theories for explaining them.” Stephen Jay Gould
40
Contradiction
41
“That’s your responsibility as a person, as a human being – to constantly be updating your positions on as many things as possible. And if you don’t contradict yourself on a regular basis, then you're not thinking.” Malcolm Gladwell
41
bitter salty
salty
sour
sour sweet
Diversity 42
“Requisite variety: in order to deal properly with the diversity of problems the world throws at you, you need to have a repertoire of responses which is (at least) as nuanced as the problems you face� Dan Lockton
42
Speed 43
“We’ve long believed that when the rate of change inside an institution becomes slower than the rate of change outside, the end is in sight. The only question is when.” Jack Welch
43
Models 44
“Essentially, all models are wrong, but some are useful.� George Box
44
Novelty
45
“In considering any new subject, there is frequently a tendency, first, to overrate what we find to be already interesting or remarkable; and, secondly, by a sort of natural reaction, to undervalue the true state of the case, when we do discover that our notions have surpassed those that were really tenable.� Ada Lovelace
45
Change 46
“You never change things by fighting the existing reality. To change something, build a new model that makes the existing model obsolete.� R. Buckminster Fuller
46
Uncertainty
47
“Underestimating uncertainty can lead to strategies that neither defend against the threats nor take advantage of the opportunities that higher levels of uncertainty may provide.� Hugh Courtney et al
47
Debate
48
“Conversations are the way workers discover what they know, share it with their colleagues, and in the process create new knowledge for the organization.� Alan Webber
48
“Digressions are incontestably the sunshine” Laurence Sterne
Data, Decisions and Design 48 nudges to push BI strategies further Created by James Richardson and Murray Grigo-McMahon. Read more about the ideas in this deck on the Qlik blog at: qlik.com/blog and look out for our Research Digests at: go.qlik.com/innovation-and-design-research-digests Special thanks to: Donald Farmer, all the Innovation & Design team, Jacquelyn Yang and Maria Reyes
Published 2016 by Qlik Technologies Inc. All text is Copyright ©2016 Qlik Technologies Inc. All rights reserved.