Smashing ebooks the mobile book by Byron Sin

É¥F .PCJMF #PPL #Z 4NBTIJOH .BHB[JOF

THE MOBILE BOOK

Published 2012 by Smashing Media GmbH, Freiburg, Germany. Printed in EU. Cover Design and Illustrations created by Mike Kus. Proofreading: Andrew Lobo, Clarissa Peterson, Iris Lješnjanin. Author Bios: Cosima Mielke. Editing and Quality Control: Vitaly Friedman, Iris Lješnjanin. eBook Production: Talita Telma-Stöckle, Andrew Rogerson, Markus Seyfferth. Marketing: Stephan Poppe. Design and Layout: Ricardo Gimenes, Mike Kus, Melanie Lang, Markus Seyfferth. Typefaces used: Elena by Nicole Dotin (Process Foundry), Whitney by Jonathan Hoefler and Tobias Frere-Jones, Skolar by David Březina (Type Together). The Mobile Book was written by Peter-Paul Koch, Stephanie Rieger, Trent Walton, Brad Frost, Dave Olsen, Dennis Kardys, Josh Clark. The extra eBook chapters of the book were written by Greg Nudelman, Rian van der Merwe, Nathan Barry, Arturo Toledo, Sebastiaan de With, Remy Sharp. Reviewers: Andreas Constantinou, Brad Frost, Bruce Lawson, Bryan Rieger, Dave Olsen, Dennis Bournique, Francisco Inchauste, Krijn Hoetmer, Lyza Danger Gardner, Peter Beverloo, Rian van der Merwe, Scott Jenson, Tom Giannattasio, Tim Kadlec, Trent Walton, Vasilis van Gemert. Idea and concept: Vitaly Friedman, Sven Lennartz. All links featured in this book can be found at smashed.by/links. The Mobile Book: smashed.by/the-mobile-book.

FOREWORD

· BY JEREMY KEITH

TABLE OF CONTENTS

Foreword by Jeremy Keith

PART I—THE MOBILE LANDSCAPE

What’s Going On In Mobile? by Peter-Paul Koch

The Future Of Mobile by Stephanie Rieger

PART II—RESPONSIVE WEB DESIGN

Responsive Design Strategy by Trent Walton

129

Responsive Design Patterns by Brad Frost

175

Optimizing For Mobile by Dave Olson

PART III—UX DESIGN FOR MOBILE

225

Hands-On Design For Mobile by Dennis Kardys

289

Designing For Touch by Josh Clark

BY JEREMY KEITH

FOREWORD

HIS BOOK IS AN ARTEFACT OF ITS TIME. There will come a time when this book will no longer be necessary, when designing and developing for mobile will simply be part and parcel of every Web worker’s lot. But that time isn’t here just yet. So in the meantime you’ve got the current state of all things mobile packed together into this single volume. You’ll find technology-agnostic chapters on design patterns for mobiles and touchscreen devices; patterns that will prove useful whether you’re building native apps or publishing on the Web. But, unsurprisingly, the majority of this book concerns itself with responsive design. That’s good. That’s very, very good. I remember when responsive design was first unveiled. It really resonated with me—I’ve always been a big fan of the One Web approach—but I thought it would be a hard sell. At the time, the default strategy for dealing with mobile devices was to build a separate mobile silo, usually in a subdomain sequestered away from the “real” site.

It’s been amazing to see how quickly the practice of responsive design has spread. In retrospect, I shouldn’t be surprised: the silo strategy simply couldn’t scale to cover the multitude of different devices out there. Also, if you’ve ever explained responsive design to a non-techie, their response is usually something along the lines of “Well, that’s just common sense, isn’t it?” It’s certainly sense but it isn’t quite common enough yet. There will come a time when conferences and books targeted specifically at mobile devices will seem quaint and anachronistic ...but we’re not there yet. So enjoy this book—don’t let PPK scare you with his horror stories. And hold on to this book even after you’ve read each and every knowledge-filled chapter. Because this book is an artefact of its time.

ABOUT THE AUTHOR Jeremy Keith has been building websites since the 90s. After a few years of freelancing, he formed the design agency Clearleft with his fellow Web nerds Andy Budd and Richard Rutter in 2005. His message to you: “I know it feels like there is an infinite amount of new stuff to keep up with every single day, but you know what? It’s okay. It’s okay not to know everything.”

6 | FOREWORD

The Mobile Landscape i. ii.

WhatÂ´s Going On In Mobile? The Future Of Mobile

8 | WHATâ&#x20AC;&#x2122;S GOING ON IN MOBILE?

CHAPTER ONE

· BY PETER-PAUL KOCH

WHAT’S GOING ON IN MOBILE?

HE MOST CHILLING MOMENT of my entire 13-year career as a browser tester occurred back in 2009, when I’d been studying mobile for a few weeks. I started my mobile investigation by testing as many browsers as I could, ably helped by the fact that the Vodafone office in Düsseldorf allowed me unlimited access to its huge device library. I asked the Vodafone guys to surprise me with odd devices with a wide range of operating systems, input methods and browsers, and one day I was handed a Samsung F700. I ran the browser through my JavaScript browser detection script and found that it was Opera Mini 2. That surprised me—Opera Mini was at version 3, and version 4 was going to be released soon. Why would the F700 run such an antiquated browser? I jotted down a note with a question mark and went on to the next device. A few weeks later I ran the F700 through the server-side detection script that I’d meanwhile written and found that the browser was NetFront 3.1. I went back to my client-side script and still got BY PETER - PAUL KOCH |

Opera Mini 2. Also, there was a bright red navigation bar that I hadn’t coded into my page. I still remember the chill that crept into my heart. This browser was not obeying the rules. This browser identified itself as Opera Mini 2 on the client and as NetFront 3.1 on the server. That’s not allowed in my book—and I’d never encountered anything like this in the wild. I loudly complained to the Vodafone guys, and one of them said, “Oh, that. It’s content adaptation.” He fiddled with my phone, and the effect disappeared, never to return. From then on, client and server agreed that the Samsung F700 ran NetFront 3.1. It turns out that operator networks run a script that detects your device, and if it decides your device is too old, it automatically switches you to an old version of Opera Mini that runs on their servers and that includes its own navigation bar on any website you visit. That explained the Opera Mini that my JavaScript detection saw. However, the browser’s real HTTP_USER_AGENT header arrives at the server, which was the reason why my server-side script returned NetFront 3.1 as a result. All of it was caused by a SIM card setting. Compare this behavior to normal ISPs. Say you run a slightly antiquated Firefox on a not-too-new desktop computer, and your ISP decides that it’s not good enough for you and switches you to an old Opera running on its servers, including its own navigation bar on any website you visit. People would be livid. It’s not the job of your ISP to change your browser for you. Mobile operators, however, do this sort of thing all the time because they think it’s a service—and because they want to control their customers. They want to avoid the fate of the desktop ISPs, which have become dumb pipes that exist only to move data from A to B. You have to remember this crucial difference between desktop and mobile—it explains many of the operators’ actions. The same Samsung F700 nicely illustrates another crucial fact of the mobile world: fragmentation. When I studied the phone, I had no 10 | WHAT’S GOING ON IN MOBILE?

clue which operating system (OS) it ran. NetFront mostly appears on proprietary Samsung (and Sony Ericsson) OSs, so it was my initial guess. Later I learned that it runs Windows Mobile (Windows Phone’s predecessor), ably disguised by an uninformative browser string and a rather lousy UI that doesn’t resemble Windows at all. If Samsung’s Windows Mobile interface resembled any other Windows Mobile interface, consumers wouldn’t care what kind of phone they buy, and device vendors would become commoditized. Therefore, all device vendors that use someone else’s OS prefer to customize it, whether doing so makes sense or not. This episode taught me several valuable lessons: Ŧ Never assume anything on mobile. Never. Anything. Ŧ There are vastly more browsers on mobile than on desktop. Ŧ Operators and device vendors are afraid of commoditization

and irrelevance and will defend themselves by influencing the overall user experience. Sometimes this is not a good idea. Learn these lessons by heart. You’ll need them. Writing a chapter on the mobile market is challenging because it’s one of the fastest-changing environments ever—faster by far than the Web. Most of what I could write now in the summer of 2012 Thanks to Peter Beverloo, Dennis Bournique, Andreas Constantiwould be outdated by the time nou, Vitaly Friedman, Lyza Danger you read it. So, what I’m going Gardner, Vasilis van Gemert, Krijn to do in this chapter is teach you some fundamentals of the mobile Hoetmer and Bruce Lawson for market, and illustrate them with reviewing this chapter or parts of it. examples from 2011 and 2012. With a bit of luck, the underlying principles will still be valid when you read this, although the examples will likely be outdated. So, here we go. It’s going to be quite a journey. BY PETER - PAUL KOCH |

The Mobile Value Chain Traditionally, operators (called â&#x20AC;&#x153;carriersâ&#x20AC;? in the US) and device vendors have formed the mobile value chain. Recently operating system vendors have entered the value chain, too. In fact, the software layer is rapidly gaining in importance, and thus software vendors such as Google and Microsoft are becoming equal partners to operators and device vendors.

FIGURE 1.1. The mobile value chain consists of operators, device vendors and OS

vendors, all of which compete for consumer attention in order to lure him into their closed silo. Currently the OS vendors succeed best at this, and operators and device vendors are falling behind.

Each link in the chain enhances the value of the others: for example, a mobile network is worthless without mobile phones, and vice versa. Parts of the mobile value chain can act against each other. Each of them has the same goal: commanding consumer mindshare and money. Each of them would love to lock the consumer into a vertical silo of its own making, where anything the consumer does 12 | WHATâ&#x20AC;&#x2122;S GOING ON IN MOBILE?

is controlled by the company and makes money for the company. (Apple is, of course, the most successful example.) At the same time, operators and device vendors fear becoming commoditized, i.e. becoming indistinguishable from their direct competitors. Later on we’ll look at the details of this process in the operator and device vendor worlds. OS vendors have not yet entered the commoditization phase and are becoming more powerful at the time of writing; but in the end, they’ll succumb, too. Especially if the Web becomes the dominant factor in mobile, why would a consumer care if they’re on Android or Windows Phone or BlackBerry? On a large scale, studying the mobile market is mainly the process of predicting which companies—and which parts of the value chain— will be more successful than others in avoiding commoditization. The Nokia E-series running Symbian, aimed at business users. When BlackBerrys became a success, Nokia copied the design. You’ll still find them in many users’ pockets. (Image credit: RafeB, smashed.by/symbian)

This Nokia 2730 runs the S40 feature-phone OS, customized for the Chinese market. Remember also that while Nokia is declining in smartphone sales, it remains strong in the feature phone segment. (Image credit:

(northirish),

smashed.by/nokia2730)

BY PETER - PAUL KOCH |

Operators The operators own and maintain the mobile networks. Until now, they were the winners of the mobile game because they make incredible amounts of money, especially on text messages, and dominate the consumer market by subsidizing devices. Although the operators still control well over half of the money that streams through the mobile value chain, they have no points of differentiation: in the end, the consumer cares very little whether they’re on Vodafone or T-Mobile’s network. Besides, operator profits are falling as a result of changing consumer habits: customers are sending fewer text messages, preferring to use other IM solutions, such as BlackBerry Ping and WhatsApp. So, the operators are in trouble. I expect them to gradually become less important, as other mobile players, especially device, OS and service vendors, win consumer mindshare—and the consumer money flow. OPERATOR SUBSIDIES Subsidies are the most powerful weapon that operators have at their disposal, because the psychological mechanism behind them is so devious that nearly all consumers fall for it. Operators offer phones to consumers for a “lower price” than they’d find elsewhere. If you buy a new high-end Android phone

in the operator’s store, you’d pay only €100 or so, while the normal sales price is more like €600. Of course, the operators don’t give you €500 out of their own pocket: they get it back (and actually earn more) on the two-year contract that you’re obliged to sign. Although buying a smartphone for the full price and a separate contract for connectivity is cheaper in the long run, the psychological difference between €100 and €600 is so huge that most consumers don’t even think about buying a phone anywhere else than in an operator’s store. (My sister saw through the operators’ cunning plan 14 | WHAT’S GOING ON IN MOBILE?

without my having to brief her, and bought her iPhone directly from Apple. I was very proud of her. But she’s an exception.) There’s something you should do every few months. Go to an operator’s store near you, pretend you know nothing about smartphones, and ask for advice on purchasing a phone. The store Note that operator subsidies are forclerk will efficiently steer you bidden by law in certain countries; towards the type of phone that for instance, in Belgium and Italy. the operator wants you to buy. There, operators have much less Store clerks earn a slight power, and high-end smartphone commission on any phone penetration is somewhat lower. they sell, but the exact amount depends on the type of phone. This mechanism enables operators to make sure clerks steer consumers towards a certain type of phone. At the time of writing, that type is always an Android phone, and often a Samsung. Consumers are familiar with the brand, and most Android vendors are able to produce phones fairly cheaply—cheaper than Apple, RIM and the Windows Phone vendors. That frees up extra money for the operators, store clerks, independent resellers and sometimes even consumers. Through this process, operators gain considerable power over device vendors. If the operators decide that they don’t want to sell certain types of phones, they can simply remove them from their stores. Sometimes they’re contractually obliged to offer the phones, but in that case they place the devices at the back of their stores and slash the clerks’ commission, with the obvious result that nobody buys them anymore. This is what’s happening to Nokia at the time of writing, and it’s deadly. When you visit an operator’s store, always note carefully which phones are in the display windows and which ones are in the back. It teaches you a lot about the operators’ current thinking.

BY PETER - PAUL KOCH |

The take-away for Web developers is that, by deciding which phones will be offered to unsuspecting consumers, operators influence the mobile browser market, because those devices’ default browsers will get more market share. Thus, keeping track of operators’ current preferences is important.

Apple Disrupts the Model From the operators’ point of view, this system is ideal. However, Apple is disrupting the model to a certain extent because it controls a large part of consumer mindshare. Some consumers are in love with Apple products and want only an iPhone. They don’t care about any other kind of phone. These consumers go to an operator’s store to buy an iPhone and don’t allow the store clerk to steer them towards any other phone. This deflates a lot of the operators’ power. Although they still make a nice chunk of money on the contracts, the operators do not decide which phones these consumers will buy. Also, Apple has an independent retail outlet in its Apple Stores. That’s not quite as important as owning consumer mindshare, because operator subsidies will still entice clients to buy a phone with a contract, but it’s one more way in which Apple influences consumers without the operators having any say. Only Apple has this particular power. All other device vendors crave it, but so far they haven’t succeeded in capturing consumer mindshare like Apple has. At the moment, Samsung comes close, but its mindshare is to a large extent based on the fact that operators love selling Samsung Android phones. If operators switched away from Android or Samsung, the Korean giant would be in serious trouble. IDENTITY MANAGEMENT AND PAYMENTS Operators hold two trump cards in the mobile value chain, but refuse to play them:

16 | WHAT’S GOING ON IN MOBILE?

Ŧ Operators can establish a consumer’s identity much more easily

than any other party. Ŧ Operators can handle payments much more easily than any other party. Operators can establish your identity by reading your SIM card. Every call or data connection you establish tells the operator who you are (or, rather, which SIM card you’re using, which amounts to the same in most cases). Similarly, because your operator already Granted, establishing identity will sends you a monthly bill, it not work with a prepaid SIM card. would be very easy to add When buying one, consumers may items you’ve purchased online choose not to divulge their identity and make the purchase run (at least, that’s how it works here in through its channel. Holland). Payments will still work Even better, operator billing fine: they’re subtracted from the would be independent of credit user’s prepaid account. cards, which are required for all proprietary payment systems such as Apple’s App Store, Google’s Play market and so on. More than half of the world’s population does not have a credit card, and currently they’re excluded from the payment systems. Of course, credit card holders are generally more affluent and, thus, a more interesting target audience for mobile payments, but the potential of an app sold to 3 billion people for $0.49 is immense. Ask the ringtone vendors. Recently, several operators, including Vodafone and Telefónica, have started to offer payment options, but they are restricted to their own network and are incompatible with each other. That is especially annoying to Web developers, who create apps that can run anywhere but don’t have a payment system that works everywhere. If you want in-app purchasing in a Web app, you need to support Vodafone’s way, Telefónica’s way, T-Mobile’s way, AT&T’s way BY PETER - PAUL KOCH |

and so on. And you have to make sure it works with Google, Facebook, Nokia, Amazon, Apple and BlackBerry, just in case. Hmm, and maybe add Microsoft and Samsung to that list. Anyway, you get the point. Telefónica’s payment system is Payments will remain a probBlue Via; see smashed.by/bluevia. lem for some time to come. We Vodafone offers the OneAPI, need a solution that works on which includes payment options; more than just the creator’s see smashed.by/oneapi. platform, but mobile players currently have little incentive to standardize payments, especially because they’re thinking primarily about closed app stores, not about open Web apps. Operators, which already have billing relationships with just about everyone on the planet, could play a major role in creating payment standards, but at the time of writing there is no evidence that they plan to do so. Keep an eye out for solutions not only from operators and from device and OS vendors, but also from services companies such as Facebook and Amazon. If a payment system works beyond its creator’s platform, you know you’ve found a winner. And if the system also includes ways to pay without a credit card, so much the better.

Device Vendors and Hardware Mobile networks are worthless without mobile phones. Duh. It’s time to turn to device vendors and their role in the mobile market. Device vendors create the mobile hardware, and sometimes the software. Most of them try to cover the entire mobile market, from cheap to expensive, by creating several lines of phones. Obviously, the cheap phones have less powerful hardware and less functionality than the expensive ones.

18 | WHAT’S GOING ON IN MOBILE?

FOLLOWING A PHONE

Before delving into the details, we need to understand the big picture. So, let’s follow a phone from its inception until it ends up in the consumer’s pocket. Suppose Samsung decides to produce a new high-end smartphone. The first item on the agenda is to figure out what kind of components it can afford for a reasonably priced phone while still making a profit. Tightly tied in with this is the selection of an OS. The most obvious choice at the time of writing is Android: it’s a good mature OS that’s also free of royalties, and consumers and operators like it. Still, Samsung could choose another OS like bada or Tizen or maybe Windows Phone. Let’s say Samsung goes for Android. That influences the exact components it needs, as well as its marketing plan. Then the phone gets designed: the hardware, the UX and the changes to the default Android software. Also, Samsung decides which of its own apps it will include as firmware. This is roughly the time at which the phone is announced. Marketing copy is created and disseminated through the usual PR channels. Samsung hopes the pending release triggers a wave of interest, which would help it in the next phase. Then comes the trickiest part of all: negotiating with the operators. Samsung wants the operators to display the new phone prominently and do a good job of marketing it to their customers. The operators will be buying a lot of phones from Samsung to resell them to their customers, and they will want a bulk discount. Then comes the actual production process in Samsung’s factories, with prototypes and final versions. Test units are sent to strategic partners (operators, app developers, Google, maybe some browser-compatibility researchers). After a final feedback round, the phone is shipped to the operators, as well as to independent stores. Usually this is decided country BY PETER - PAUL KOCH |

by country, which is why at the start of their lifecycle, most phones are available in some countries but not in others. Most operators decide to also place their own apps on the phone, next to Samsung’s and the default Android ones. Now the phone’s firmware is complete. Now the operators do their job by enticing consumers to buy the phone with a two-year contract. Simultaneously, Samsung starts up a marketing campaign to firmly lodge the phone in consumers’ awareness. The operators may do the same, depending on how well they think the phone will perform in the market. Then the consumer enters the operator’s store, is gently steered towards the phone by a commission-hungry clerk and eventually buys it. From inception to entrance in the market, the process takes roughly six months. That assumes an existing OS: if the OS is new and untried, the process would take rather a year because several software iterations are needed to get the OS right. THE GLOBAL DEVICE MARKET

What kinds of devices are being sold, and how much of each type? How does that affect mobile Web development? These questions are surprisingly hard to answer. I’ll provide some numbers, and their caveats, later on. Let’s first discuss the global device market qualitatively. In general, the more expensive phones are sold in richer countries, and the cheaper ones in poorer countries. That’s a generalization, though: rich elites in poor countries buy expensive smartphones for status reasons (sometimes in addition to a basic or feature phone that they actually place calls with), and middle-class people from rich countries might not feel the need for a smartphone and instead buy something cheaper. Despite these generalizations, the “global device market” doesn’t exist. Instead, there are dozens of regional markets, and although you can aggregate the data to create global statistics, they don’t tell 20 | WHAT’S GOING ON IN MOBILE?

you anything useful about particular markets. There’s too much difference in demographics, culture and disposable income to define general worldwide rules for the phone market.

Sales, Installed Base and Browsing Market Shares Pure sales statistics are actually not all that important to Web developers. More important is installed base—that is, the types of devices people have in their pockets right now. It’s all well and good to know that sales of Nokia’s Symbian OS are crashing dramatically, but plenty of people still have a Symbian phone in their pockets. Sure, when they buy a new phone, they’re likely to switch to Android, but that hasn’t actually happened yet. If they want to surf now, they’ll fire up a Symbian browser. Is your website ready for that? Even more important than installed base is browser market share. As we’ll see in a moment, Android’s sales share in Q2 2012 was about 67%, while its installed base was about 41%. Still, Android WebKit’s browsing market share was only about 20%. The reasons for this discrepancy are hotly debated. Do Android users genuinely browse less than iOS users? Are the sales numbers wrong? Is there an error in the detection scripts? Although this discussion is interesting in and of itself, in the end we Web developers don’t care about the entire phone market, but only about that part that actually uses its devices for browsing. Later in this chapter we’ll see that the Android WebKit’s browsing market share is roughly equal to Safari’s, so we should pay roughly equal attention to the two browsers, despite Android’s higher sales and installed base shares. Even general reports of browser market shares should be only moderately important to you. In the end, what matters is which browsers people use to visit the website of your client. Other statistics should be used only if your client doesn’t have reliable statistics of their own. BY PETER - PAUL KOCH |

Smartphone Sales Market Share Now, what are the actual numbers as of Q2 2012? SMARTPHONE SALES MARKET SHARES FOR Q2 OF THREE YEARS Vendor

Samsung

Country

Korea

Android, bada,

2012

2011

2010

33%

16%

Windows Phone Apple

iOS

17%

19%

14%

Nokia

Finland

Windows Phone,

15%

39%

11%

Symbian, MeeGo HTC

Taiwan

Android, Windows Phone

ZTE

China

Android

RIM

Canada

BlackBerry

12%

18%

Sony

Japan

Android

Huawei

China

Android

Korea

Android

Motorola

Android

Others

Varies

Android, Symbian

11%

Phones sold

153

108

(in millions) FIGURE 1.2. In 2010, Tomi did not yet count these vendors separately. They are part of “Others.” Source: Tomi Ahonen Consulting: smashed.by/2012-market, smashed.by/2011-market, smashed.by/2010-market.

22 | WHAT’S GOING ON IN MOBILE?

First, a caveat. These numbers are less precise than you’d think. At the time of writing, Samsung is not divulging its exact sales figures—just the total number of all phones it has sold. It’s up to analysts to split up this number into Android, bada, Windows Phone and proprietary OSs, and sometimes analyst houses disagree. So, rather a lot of guesswork is involved in creating these overviews. Nokia was once the undisputed The exact opposite of Nokia is smartphone front-runner—in fact, Motorola, which is strong in the the Nokia Communicator 9000, US but negligible elsewhere. It released in 1996, is considered the used to have market share in the first smartphone. But Android rest of the world but lost it in became more popular, and Nokia 2007 when it couldn’t create a decided on a dramatic change of successor to the wildly popular course by embracing Microsoft’s Motorola Razr—and the iPhone Windows Phone. So far, the results was released. have not been overwhelming. Still, while most of the world witnessed the Nokia tragedy with awe, sadness or glee, the US remained unaffected. Americans hardly know Nokia because it never made inroads in the US market, even though it was overwhelmingly dominant everywhere else. This is a good example of the regional differences that I mentioned earlier in the chapter.

OS Sales Market Share What we Web developers need to know about more than the devicevendor split is the OS split. The browser that a visitor uses depends on the OS, after all.

BY PETER - PAUL KOCH |

SMARTPHONE OS SALES MARKET SHARES FOR Q2 OF THREE YEARS

Android

Developer

Vendors

Google

All but Apple,

2012

2011

2010

67%

45%

18%

Nokia, and RIM iOS

Apple

17%

19%

14%

BlackBerry

RIM

12%

18%

Symbian

Mainly Nokia

Nokia, some

16%

44%

Japanese Windows

Microsoft

Mostly Nokia

bada

Samsung

MeeGo

Nokia

Other

Varies

Many

Phones sold

153

108

Phone

(in millions) FIGURE 1.3. Source: Tomi Ahonen Consulting. Same articles as for the previous table.

Android is clearly the big winner of 2010 to 2012, with the iPhone holding on to its market share but not rising beyond one in five, Windows Phone winning a bit, and the rest slipping. Again, these are global statistics. They don’t necessarily say anything about your country. To illustrate that, here are the OS sales market shares for five countries for June 2012:

24 | WHAT’S GOING ON IN MOBILE?

SMARTPHONE OS SALES MARKET SHARES FOR THE US, AUSTRALIA AND THREE EUROPEAN COUNTRIES, JUNE 2012

Germany France Australia

Android

50%

57%

69%

59%

57%

iOS

37%

26%

17%

15%

31%

BlackBerry

11%

10%

< 0.5%

Symbian

Windows Phone

bada

< 0.5%

Other

< 0.5%

FIGURE 1.4. Source: ComTech, smashed.by/android-rules.

Do you spot the differences? Android’s share fluctuates between 50% and 69%. Samsung’s bada is nowhere to be seen, except in France, where it performs quite decently. BlackBerry holds on to the UK and France but is wiped out elsewhere. And the iPhone is doing relatively poorly in Germany and France. There is no global device market. There is only a collection of regional ones.

OS Installed Base Still, global numbers are the easiest to come by. That’s why the next table is global again. It shows the installed base of the various OSs.

BY PETER - PAUL KOCH |

SMARTPHONE OS INSTALLED BASE FOR Q2 OF THREE YEARS

Q2 2012

Q2 2011

Q2 2010

Android

41%

27%

Symbian

25%

35%

49%

iOS

19%

16%

11%

BlackBerry

10%

12%

14%

bada

Windows Phone

Windows Mobile

Others

10%

Millions of phones

1,059

910

700

FIGURE 1.5. Source: Tomi Ahonen. smashed.by/2012-market, smashed.by/2011-final-numbers.

As you see, Android’s installed base is still below the 50% mark, while Symbian’s is still a quarter. That will change, but only gradually over the next few years. DIFFERENTIATION AND FRAGMENTATION Take another look at the table for device vendor market share. As you see, well over half of the vendors produce Android phones. That being the case, why would consumers care whether they buy a Sony or a ZTE Android device? Device vendors want to stand out from the crowd. They want to differentiate their devices from their competitors’. “Differentiation” is the positive way of looking at this process, while “fragmentation” is the negative way. The word you use depends on whether you think it is a good idea.

26 | WHAT’S GOING ON IN MOBILE?

In general, device vendors and operators love differentiation because they want to stand out. OS vendors and developers hate fragmentation because they prefer a single system to maintain or to test their apps and websites against. As for consumers, they might benefit from less fragmentation, too, but they don’t really understand the issues involved and usually don’t have an opinion.

Android When Google released Android back in 2009, it deliberately allowed fragmentation in order to make Android more interesting to device vendors. Not only was the OS free to use, but anyone could make changes to the system. Specifically, each device vendor could change the UX layer to its heart’s content, and that’s exactly what they all did. HTC Sense, Samsung TouchWiz, MotoBlur and a host of other UX layers were born and added on top of Android. Device vendors could also make changes to apps, which led to differences in the browsers. The default Android WebKit came with every Android installation, but device vendors were allowed to switch a lot of flags on and off, which is the cause of the dreaded Android browser fragmentation. All this changed with Android 3. Google announced that it was going to crack down on fragmentation because that was in its own interest as well as the interest of Android developers. That was bad news for the device vendors. Back in early 2011, I tested two Android 3 tablets at the same time. I found literally no differences, except for a few home screen apps. Subsequently, I even forget who manufactured the tablets—I think it was Samsung and Motorola, but I’m not sure any more—just because there was no differentiation at all. You see? Around the same time, and probably in reaction to Google’s crackdown, some vendors, especially ZTE and Huawei from China, but also Amazon, started to create their own Android branches. Essentially, Google completely lost control over these branches. BY PETER - PAUL KOCH |

The Nokia Lumia is the best-selling Windows Phone device, and it sports an interface that everyone agrees is unique and interesting. (Image credit: vernieman, smashed.by/lumia)

Built by HTC, the Nexus One, Google’s first phone, would become the gold standard of Android programming until the release of its successor, the Samsung-built Nexus S. (Image credit: johanl, smashed.by/htc)

Samsung’s 2012 smash hit: the Galaxy III.” Sporting the latest Android build, the Galaxy III was the default phone for anyone who didn’t pick an iPhone, and Samsung’s market share soared. (Image credit: Sergey Galyonkin, smashed.by/galaxys3)

Then Google acquired Motorola, and the other Android vendors started to get nervous and cast around for alternative OSs. Google had to placate them, and one of the easiest ways to do that was by allowing differentiation once again. Besides, the growth of nonGoogle Android branches meant that Google’s attempt at centralization had failed anyway. I see this as a victory for the device vendors and a defeat for Google. Device vendors got what they wanted, to the detriment of Google and developers. Basically, Google lost control of Android and won’t ever regain it.

Windows Phone Compare the Android story to Windows Phone. From the outset, Microsoft was very strict in prescribing exactly what kind of hardware its OS could run on and in disallowing any change to the native UX layer. That’s great for developers, but it’s also one of the reasons why Windows Phone hasn’t taken off. If device vendors can’t differentiate themselves from their competitors, why would they use the OS? Especially when they have to pay a hefty royalty fee to Microsoft. So far, Microsoft hasn’t budged: differentiating Windows phones (except by screen size) is still not allowed. It should probably relax its stand, or else Windows Phone will never amount to much. If by the time you read this Windows phones have started to become differentiated, you’ll know that Microsoft has lost this particular battle. ENTERING THE DEVICE MARKET Because the mobile market is clearly the next frontier, company after company is trying to enter it by setting up production of its own mobile devices. In 2011 and 2012 we had rumours of Facebook, Intel, Dell, Lenovo and many others working on their own phones. BY PETER - PAUL KOCH |

In general, all of these efforts are doomed to failure because entering the mobile market is extremely difficult. Designing and building a phone is but one of the challenges. Go through the “Following a Phone” section again, and imagine a company trying to go through all of these steps without any experience or connections in the mobile market. You see the problem? Even if a company managed to do it, it would cost billions and billions of up-front investment. Few companies have pockets so deep and can keep shareholders silent while making these investments. That’s why you should ignore any press release that states that company X, which so far hasn’t played a part in the mobile world, will release phones on date Y. It won’t. The only possibility for new entrants is to buy an existing device vendor outright, including its hardware designs, supply chain, operator relations and marketing. Even that is not certain to work, though. In fact, up until 2012, not a single party has succeeded in taking over a mobile company and its sales share as well. The next few years might see a successful acquisition, with both Nokia and (likely) RIM going up for sale. It’s theoretically possible that each will be bought by a company that knows what it’s doing. Give the acquiring company the benefit of the doubt for about a year or so, and then write them off if nothing happens.

OS Vendors and Software Any phone, even a basic one, needs an OS that makes sure the right things happen when the user presses a button or touches the screen. (Please take a moment to recover from this stunning revelation.) Since 2008, the quality of the OS—the “software layer”—has become more important than the quality of the device, the “hardware layer.” More importantly, many of the traditional mobile powers noticed this too late and are still having trouble adjusting. Nokia and RIM spent 2010 to 2012 improving their software layer. At the 30 | WHAT’S GOING ON IN MOBILE?

time of writing, it’s fairly certain that Nokia failed, while the fate of RIM still hangs in the balance. IMPORTANT OSs Currently, the most important OSs are Android and iOS. Still, the contention that these are the only two OSs that matter is untrue. The mobile world is not like the desktop world of the ’90s, when Windows

and Mac battled it out. Most mobile players have good reason to create and maintain their own OSs, and will continue to do so. It is sometimes said that Windows Phone will be the “third OS” because operators are looking for a competitor to Android and iOS. It’s very unlikely; Windows Phone doesn’t have enough differentiation. Be very critical when you encounter this idea in the wild. In my opinion, the mobile OS market will reach an equilibrium, with about four to six OSs gaining a market share of 5% or higher. Two of those will certainly be iOS and Android, but who the other two to four will be is unclear at the time of writing. OS UPDATES

It usually takes a while before a new OS version is distributed to consumers, and this is a frequent cause of complaints from consumers and developers, who are forced to continue to work on older OSs. OS updates take so long because device vendors and operators want to make sure that their own apps work on their devices. Say you have an HTC Android phone that’s sold by Vodafone. Google delivers the new Android version to HTC, which starts to test the hardware, the UX layer and its own apps. Only when they are all confirmed to work with the new OS, or when new versions have been written, does HTC distribute the Android version. This can easily take a few months. The distribution goes to Vodafone, which also has to test its apps against the new OS. This can again take a few months. Then, finally, BY PETER - PAUL KOCH |

the new Android version is ready to be distributed to HTC phones on the Vodafone network. By that time, the next Android version is almost ready. The irony, of course, is that most consumers hardly use HTC or Vodafone’s apps. Still, the device vendor and operator each need its own point of differentiation. SEVERAL OSs PER VENDOR There is no law that requires a device vendor to use only one OS. Nokia has a long history of producing several OSs by itself. It was the driving force behind Symbian, and its S40 feature phone OS is wholly proprietary. When it became clear that Symbian was too old, Nokia started to develop a new OS: MeeGo. In this sense, Nokia’s 2011 adoption of Windows Phone was a break with the past. Many Android vendors also use Windows Phone, although it seems their hearts aren’t entirely in it. Sales are quite disappointing at the moment, and the lack of potential for differentiation leads to the conclusion that device vendors aren’t too keen on Windows Phone. Samsung created its own OS in bada (meaning “ocean” in Korean), despite using Android and Windows Phone. Everyone assumed that Samsung’s plan was to let bada grow quietly in the shadow of Android until it was good enough to replace Android entirely. At the time of writing, that plan has been cancelled, but Samsung is working on the Tizen OS, which could fill the same role. THE WEB AS AN OS In 2009, Palm astonished the world by announcing it was going to use the Web as its next platform. Native apps would be written in HTML, CSS and JavaScript, and the OS would be called webOS. The plan was great; the execution lousy. Palm didn’t bother to reach out to Web developers, and the marketing was a disaster. webOS disappeared silently. 32 | WHAT’S GOING ON IN MOBILE?

Still, the Web will likely have a future as an OS. Currently, there are two major initiatives for the Web as an OS: Firefox OS by Mozilla, and Tizen by Samsung and Intel. Firefox OS is supported by a slew of worldwide operators, while Tizen is supported mainly by Samsung. The first thing to keep track of here is device vendors or operators that brag about using Firefox OS or Tizen. If one does, then it is expecting the OS to make a difference in the market and can be relied on to support it as much as it is able to. The second thing to keep track of is whether it reaches out to Web developers and gives away a lot of devices. A Web-based OS can be expected to garner interest among Web developers, but they will need actual devices in order to create apps. Also, remember that the Web is the weapon of choice for the losers in the mobile platform game. Platforms with a healthy ecosystem, notably iOS and Android, do not really need the Web—the struggling also-rans are the ones that are likely to bet on the Web, because they have no other choice. This doesn’t mean they’re wrong: the Web remains the only way to create an app for more than one platform. It also means that none of the strongest mobile players will back the Web beyond offering a good browser.

Browsers A modern mobile OS needs a browser, so it’s time to look at this area. I’m not going to give you a long list of browsers and their quirks; not only is my information incomplete, but the list would get outdated very soon. If I were to write about mobile browsing today, I would sing the praises of BlackBerry WebKit and Samsung Dolfin for bada (not to be confused with Dolphin for Android). These two mobile browsers are excellent, but most Web developers have never heard of them. I’m not sure they’re going to be relevant by the time you read this, BY PETER - PAUL KOCH |

so I won’t talk about them. Instead, I’ll give a general overview of mobile browsing. If you’re only used to the simple five-browser ecosystem that exists on the desktop, you’re in for a rough surprise in the mobile market. So far, I can identify about 30 to 35 mobile browsers, ranging from lousy to great. Not all of these browsers are equally important: in fact, about 25 of them are rather marginal. On the desktop, there are four rendering engines: Trident (IE), Gecko (Firefox), WebKit (Chrome and Safari) and Presto (Opera). There used to be many more on mobile, but it’s clear now that none of them can even approach the quality of the desktop ones, and one of the continuing themes of the mobile browser market is the gradual replacement of mobile rendering engines with desktop ones—especially WebKit. There are two important distinctions in the mobile browser world: proxy browsers versus full browsers, and default browsers versus downloadable ones. PROXY BROWSERS Full browsers are browsers as we usually know them: when the user requests a page, the browser fetches the HTML, CSS, JavaScript and other assets via HTTP, and once it has everything, it renders and shows the page. All of these steps take place on the client. Proxy browsers, of which Opera Mini is by far the most important, are different. They work as follows: 1. The user requests a page. This is not a normal HTTP request, but a

special request to a proxy server over an encrypted connection. 2. This proxy server makes the normal HTTP request to the Web server that the user wants to access. It requests the HTML as well as all assets, such as CSS, JavaScript, images and what not.

34 | WHAT’S GOING ON IN MOBILE?

3. The proxy server contains a rendering engine (Presto, in the case

of Opera Mini), which renders the page as usual. 4. The proxy server then compresses the rendered page to a highly compressed “image” of the website. Think of it like a PDF or an image map. It has hotspots that the user can click on, and the user can also select text. Opera’s version is called OBML (Opera Binary Markup Language). 5. The proxy server sends this file to the client, again over an encrypted connection. 6. The client shows the file to the user. The user can scroll and zoom and click on links.

Advantage: Cheap This process serves primarily to save the user money. Because all the client has to do is show static files, allow for clicks and generate a simple UI, it’s fairly light and, thus, able to run even on low-spec’ed phones. So, users do not have to buy an expensive smartphone in order to access the Web. Besides, all the client receives is a highly compressed file, which is much lighter than raw HTML, CSS, JavaScript and image files, and it uses only a single request and 90%: See for instance response. This saves a lot of mobile smashed.by/march-madness data traffic—sometimes up to 90%. under “Pages transcoded.” Also, this will work even on older networks, which is important to operators that don’t want to spend money on upgrading. Proxy browsers serve to make the Web accessible even to low-income users who can’t afford a desktop computer or a smartphone. It is not a surprise, then, that they’re especially popular in the developing world. Opera was the first company to see this niche and has taken the lead in the proxy browsing world mainly because it was the first to enter the market. BY PETER - PAUL KOCH |

Nowadays there are many more proxy browsers. UC, the biggest browser in China, has followed the same path as Opera, as has its domestic competitor QQ. Nokia released the Ovi proxy browser to directly compete with Opera Mini on its own phones. And, as we saw in the introduction to this chapter, some operators decide that you should use a proxy browser, whether you want to or not.

Disadvantage: No Client-Side Interactivity There’s a disadvantage to proxy browsing, too: no client-side interactivity. Proxy browsers support JavaScript, but every time the user fires a JavaScript event (by clicking on an AJAX link or something similar), the client sends a request back to the server for instructions. The server executes the script, fetches new assets if necessary and sends back the updated page, which, as far as the client is concerned, is a completely new page. It’s important to realize that this lack of client-side interactivity is a feature, and not a bug. By giving up client-side interactivity, users save themselves a lot of money. Executing JavaScript costs users money, and some prefer not to pay the price. DEFAULT VS. DOWNLOADABLE Every phone has a default browser, i.e. a browser that’s part of the firmware and that’s usually developed by the OS vendor. In addition, downloadable browsers are available for many, though not At the time of writing, iOS and all, OSs; users may download, BlackBerry do not allow the instalinstall and use them, if they so lation of other rendering engines, so choose. there are no other full browsers for The difference is in market these OSs. Installing proxy clients penetration. Generally speakis allowed, though, so Opera Mini is ing, default browsers have available for both. larger market shares than 36 | WHAT’S GOING ON IN MOBILE?

downloadable ones, because few mobile users bother to install a different browser on their phone—most people aren’t even aware that it’s possible. There’s one exception to this rule: Opera Mini is downloaded a lot in developing countries because people want cheap Internet on their phones. It’s not yet possible to update a default browser without also updating the underlying OS. Thus, iPhone users will get a new Safari when they upgrade to the next iOS version, etc. One advantage of downloadable browsers is that they can be updated independent of the underlying OS.

Android is at the other end of the spectrum: an incredible number of browsers are available, because it’s the easiest platform to port a browser to and, thus, figures in the calculation of every aspiring browser vendor. At the time of writing, I have nine browsers on my Android phones: Android’s default WebKit, Chrome, Dolphin (not to be confused with Dolfin), UC, QQ, Opera Mobile and Mini, Firefox, and NetFront. Except for Firefox and the two Operas, they all use WebKit.

WEBKIT

So many mobile browsers use WebKit as their rendering engine that it’s more efficient to list the ones that do not: Ŧ IE Mobile uses Trident. Ŧ Opera Mobile and Mini use Presto. Ŧ Firefox Mobile uses Gecko. Ŧ Nokia mostly uses WebKit, but occasionally Gecko (for in-

stance, in its Ovi proxy browser). Ŧ Older versions of several browsers (such as BlackBerry, UC, QQ and NetFront) still use old, proprietary and, frankly, quite lousy rendering engines. BY PETER - PAUL KOCH |

Still, the fact that a browser uses WebKit does not mean it’s the same as any other WebKit-based browser. In fact, there are considerable differences between them. Of course, basic stuff is supported in every WebKit-based browser. Features such as the :nth-child CSS selector are supported in WebKit itself, which means it works everywhere (as long as the vendor uses a modern enough WebKit version). Still, WebKit is a rendering engine, not a browser. If you hand it HTML, CSS, JavaScript and images, it will deliver a rendered page. However, it does not contain the modules necessary to request the assets or to actually show the rendered page on the phone’s screen. It depends on the OS for interfacing with the keyboard, mouse and touchscreen. WebKit provides support for hardware-accelerated animations but does not contain the modules that communicate with the GPU and that make sure that hardware animations actually show up on the screen. If you want modern form fields such as <input type= "date">, you must write the date interface yourself. WebKit includes Apple’s JavaScriptCore as the default JavaScript engine, but you may decide to switch to another, such as Google’s V8. Finally, you may use a different WebKit version than the other guy. So, there is no “WebKit on mobile.” A lot of browsers use more or less the same rendering engine but differ a lot in their details. Thus, testing your website in all individual WebKit-based browsers is best. If it works in Safari for iOS, it will not necessarily work in BlackBerry WebKit, or Samsung Dolfin for bada, or Android WebKit, or Chrome, or Obigo WebKit, or Symbian WebKit, or Dolphin for Android, or MeeGo WebKit, or… well, you get the point.

38 | WHAT’S GOING ON IN MOBILE?

OPERA Opera was the first desktop browser vendor to recognize the potential of the mobile world. It released its first mobile browser in 2002 and pioneered the concept of a proxy browser. It was also quick to establish good relations with operators and device vendors. Back in those days there were no real browsers on mobile phones. Mobile websites used either WAP or XHTML-MP (Mobile Profile), which were dumbed-down versions of HTML deemed suitable for mobile. In addition to these formats, Opera could also handle real HTML. That appealed especially to operators, because if their customers would be able to visit real websites, they’d spend more on data plans. Opera Mini and Mobile are In addition to the Opera Mini proxy available for free, and Mini has browser, there’s also Opera Mobile, become very popular in certain which exists for Android and Symcountries, including Russia and bian. It’s not nearly as important as India. Consumers have discov- Mini because it doesn’t have Mini’s ered that with proxy browsers advantages, and few people bother the Web becomes more accesto replace their default full browser sible to them, and they also ad- with another full browser. vise their friends to download Opera Mini. Opera capitalizes on this by making sure its browsers exist for as many platforms as possible. Opera Mini is available for all smartphone OSs except Windows Phone and for a lot of proprietary feature phone platforms. Operator relations are still very important to Opera today. If you look through Opera’s press releases, you’ll find that many of them are about a partnership with operator X that will bring cheap mobile Internet to consumers in country Y. Thus, Opera wins market share, while operators entice people to use more data traffic. All of these strategies have combined to make Opera the second or third most used mobile browser in the world. Its popularity is geoBY PETER - PAUL KOCH |

graphically determined: it is used a lot in poorer countries, where few people can afford modern data-hungry smartphones, and is hardly used in wealthy countries with a lot of iPhones and Androids. READING MOBILE BROWSING STATISTICS I referred above to Opera’s market share. As soon as you hear such a statement, wonder where the author got that data and whether they

are interpreting it correctly. The best statistics on browser market shares are the ones that come from your client’s log files. Study them to find out what kinds of phones are used to visit their website. Be aware that users of some phones might not be able to use the website and, thus, might be underrepresented. I usually look only at statistics for the home page or another important landing page and ignore the other pages. If a certain mobile browser is visiting the home page in decent numbers but is nowhere to be seen elsewhere on the website, then users of that browser are likely encountering a problem that you must solve. Finding and using general worldwide mobile browser market shares is fairly hard. What we need are the mobile browser statistics of a first-rank website such as Google or Yahoo. Unfortunately, these companies keep their statistics a secret. Search engine vendors pay browser vendors a slight commission for every query they send, and they want to hide these vital statistics from their competitors (and from browser vendors). That’s why they do not share the browser make-up of their home page hits. So, we’re reduced to using analytics services that gather these statistics from their clients and share them freely. Unfortunately, Web designers have to sign up for these services and install a counter script, so the statistics will not be representative of the entire Internet. 40 | WHAT’S GOING ON IN MOBILE?

We have the same problem with desktop browser statistics, which are gathered in exactly the same way. The choice is yours, then: either use the statistics, knowing they’re incomplete and biased, or use none at all. To me, any data is better than no data, but your mileage may vary. At the time of writing, I know of three such services, and I encourage you to compare them or even sign up your clients for one of them.)

Why I Prefer StatCounter Ask yourself two questions about mobile browser statistic services. Beware of services that are unclear on either of these points:

1. How does the statistics service treat tablets? Ideally, tablets should be treated as a separate device category, neither desktop nor mobile. If they aren’t, then they should be counted as desktop. Tablets have more in common with desktop computers than with mobile phones.

Ŧ StatCounter

(smashed.by/statcount) Ŧ NetMarketShare (smashed.by/netms) Ŧ Akamai (smashed.by/akamai)

2. What is the geographic spread of the sample? If you’re interested only in US data, then all three sources are fine; but if you want truly global statistics, then the only real option at the time of writing is

With all of these caveats, have a look at some mobile browser statistics on the following page.

StatCounter. Its global spread and the fact that it does not count tablets as mobile devices are the reasons why I use StatCounter, knowing full well that its data is likely not entirely correct. I’d rather use the least bad source than have no data at all.

BY PETER - PAUL KOCH |

MOBILE BROWSER MARKET SHARES FOR Q2 OF 3 YEARS

Browser

Q2 Q2 2012 2011

Q2 2010

Remarks

Safari

24%

22%

28%

For iOS

Android

22%

17%

For Android

Opera

22%

26%

Mostly Opera Mini, on many OSs

Nokia*

11%

17%

15%

Symbian browser and S40 Ovi proxy browser

UC*

Chinese proxy browser

BlackBerry

13%

14%

WebKit-based since OS 6

NetFront

Mainly for Samsung and Sony feature phones

Dolfin

For Samsung bada

Samsung

For Samsung’s non-Android, non-bada phones

For Windows Phone

Others

A lot of minor browsers

FIGURE 1.6. StatCounter’s detection script had an error that counted UC browsers as

Nokia browsers. This error was corrected in Q4 2011, and it’s the reason why UC won and Nokia lost such a large share in 2012. Source: StatCounter, smashed.by/statcount.

Don’t stare yourself blind on tiny differences that are statistically meaningless. StatCounter shows both Safari and Opera (Mini) at roughly 22 to 24%, with one then the other growing slightly larger. These fluctuations mean nothing: the only conclusion you can draw is that, according to StatCounter, Safari and Opera have roughly one quarter of the market each.

42 | WHAT’S GOING ON IN MOBILE?

Trends, on the other hand, are meaningful. BlackBerry’s share dropped sharply in 2012, and this no doubt is an actual trend, even though the percentages that StatCounter gives might not be exact. Finally, the mobile browser market is highly localized. Here are the Q2 2012 statistics for the US, the UK, India and Brazil: MOBILE BROWSER MARKET SHARES FOR Q2 2012 IN FOUR COUNTRIES

Browser

India

Brazil

Safari

49%

43%

Android

40%

22%

23%

Opera

36%

Nokia

19%

18%

< 0.5%

22%

BlackBerry

29%

< 0.5%

NetFront

< 0.5%

10%

Dolfin

< 0.5%

Samsung

< 0.5%

Others

FIGURE 1.7. Source: StatCounter, smashed.by/statcount.

If you can’t get useful data from your client’s log files, use your country’s market shares, not the global ones.

BY PETER - PAUL KOCH |

iOS and Android are the two most important mobile OSs. But theyâ&#x20AC;&#x2122;re not the only ones; about 15% of the smartphones sold in 2012 use other OSs. (Image credit: Daniel Y. Go, smashed.by/lgandroid)

The BlackBerry 9630 is one of the many models aimed at people who type a lot: business people answering mail and youth pinging each other. I hope BlackBerry will survive, but itâ&#x20AC;&#x2122;s not looking good in the end of 2012. (Image credit: Dan_H, smashed.by/blackberry)

Iconic and game-changing, this phone needs no introduction. (Image credit: thronx, smashed.by/iphone)

Developing Mobile Websites Now that we have addressed the mobile market, it’s time for some Web development advice. Because the other chapters in this book are replete with excellent recommendations on various mobile topics, I’ll address only a few general issues. The first bit of advice is the most important: start mobile browser testing today. You likely have an iPhone or Android device; test your current project right now. Also, download Opera Mini and test in that browser, too. True, your single device is not representative of the market as a whole, but it still allows you to start working on the most serious mobile problem and its solution: the small screen and responsive Web design. CREATING A DEVICE LAB The most significant obstacle that beginner mobile Web developers will encounter is finding enough devices to test on. Buying devices is expensive, and you won’t want to shell out thousands of dollars the very first time you undertake a mobile project. Save at least €100 per month to buy devices. This will allow you to buy two devices per year. That’s not really a lot, but it’s better than nothing. The more you save per month, the better, obviously. Make sure to buy a wide range of devices. iOS and Android are the most important ones in the Western world, so you’ll likely start there. Once you have one of each, buy something else that’s important for your market: a BlackBerry in North America and the UK, and Symbian in Europe. True, their sales are dropping like a stone, but these devices are still in quite a few pockets. Also, consider Windows Phone, although at the time of writing its big breakthrough hasn’t happened (yet).

BY PETER - PAUL KOCH |

Make sure to buy at least one non-touchscreen device. Not all consumers want a touchscreen: heavy texters, especially, are addicted to hardware keyboards and will buy accordingly. Your A word about money: I’ve noticed websites should work on these that some Web developers do not devices, too. want to spend anything on devices A second Android would and complain loudly when I mention be useful: as we’ve seen, that the need to buy them. I find that OS has a lot of fragmentation. very odd. You buy a computer and Buy an Android from a differsoftware because you need them ent vendor with a different for your job, and you expect to make Android version and a different the money back from your clients. screen size. That will give you Buying devices works exactly the the greatest scope for finding same way: you invest now in order subtle incompatibilities. to serve your clients better—and If you’re setting up a device make back the money you’ve spent lab anyway, add a few tablets: over time. an iPad, Android tablet and BlackBerry PlayBook. They will give you more screen sizes and more browsers to work with, which never hurts.

OPEN DEVICE LABS Still, if you’re a freelancer who can afford two devices per year, you’ll initially have a lab of only three or four devices: the two professional ones, your personal one and maybe your personal tablet. That’s still not enough. Fortunately, you’re very likely not the only one in your area with this problem. If you find other developers who are building a library of their own, invite them to compare notes and swap devices. Not only will you be able to test your websites on more devices, you will also acquire useful contacts to discuss technical issues and the mobile market in general, and perhaps gain surplus clients. 46 | WHAT’S GOING ON IN MOBILE?

In fact, open device labs are Smashing Magazine has an excelbecoming popular. Usually lent article on setting up a device founded and supported by a lab: smashed.by/open-device-lab. small local company with a The website lab-up.org is dedicated decent set of devices, these labs to making available lists of open deare open to anyone who wants vice labs around the world, and also to test mobile devices, provided links to good resources about open you reserve a slot in advance. device labs. Look there if you need In return, you can leave your one or want to found one. devices there if you don’t need them for a while. Finding out whether your city has one is worth the trouble. If it doesn’t, maybe you should set one up? WORKING WITH PROXY BROWSERS I’ve said it before and I’ll repeat it one more time: test in Opera Mini. A proxy browser doesn’t quite work like the browsers you’re accustomed to, and many users will get their first taste of the Web via a proxy browser. Having at least some experience with them is mandatory. The problem is not in the HTML or CSS—they work pretty much as you’d expect because it’s all static anyway (except for a few dynamic CSS properties such as :focus). It’s in the JavaScript that you’ll encounter the most serious problems, because any JavaScript execution requires a round-trip to the server. Although proxy browsers support JavaScript and, thus, JavaScript events, most of them disallow certain events. For instance, if you have an onscroll event handler, it should fire whenever the user scrolls. But in a proxy browser, that would mean making a server request with every few pixels of scrolling, which would make the page completely unusable. Therefore, proxy browsers disable the scroll event.

BY PETER - PAUL KOCH |

As a rule of thumb, assume that only events that clearly show the user’s intent to load new data will work in proxy browsers. In addition, mouseover is widely supported because so many websites depend on it, and load and unload because they will be processed on the server anyway. Thus, for instance, you can expect click, The definitive guide to JavaScript in Opera Mini can be found at change, focus, submit and the smashed.by/opera-js. like to work, but mouseout, the key events, resize and scroll will not work. I advise you to keep it simple and concentrate on the click event, which always works everywhere. Add submit if you’re working with forms. That’s mostly it, though. OVERCOMING OUTDATED REFLEXES There are some reflexes from traditional “desktop” Web development that we have to let go of. The most important ones are our distrust of browser detection and our unthinking use of JavaScript libraries.

Browser Detection, Good Traditionally, on desktop, browser detection is a no-go for Web developers. If you distinguish between IE and Chrome through their navigator.userAgent, you’ll face the scorn of your peers. Instead, detect the feature you want to use, and make decisions based on the result of that check. I have been preaching feature detection ever since 1998 and played my part in convincing the Web world at large of the evils of browser detection, so it took me a while to acknowledge that the situation on mobile is different. As soon as I started speaking with really knowledgeable people with years of experience in mobile, they all told me that some sort 48 | WHAT’S GOING ON IN MOBILE?

of browser detection is a necessary evil. The more experienced a mobile Web developer is, the more they rely on server-side browser detection. For instance: 1. Suppose you need a “Back” functionality on your website.

Certain OSs, such as Android and BlackBerry, have a native button for that, and inserting your own button would only confuse users. 2. Some Android devices claim to support input type="date" and such, but don’t actually have the native components to fill in a date. 3. BlackBerry 6’s default browser supports touch events and tells you so—even if it is running on a device without a touchscreen. 4. A browser might support a lot of stuff but have such a cheap CPU that everything slows to a crawl, and the user would be better off without the advanced features. In all of these cases, feature Even Modernizr, arguably the best detection won’t help you. Defeature-detection library, cannot tecting the presence of a “Back” detect everything (see button is impossible, and in smashed.by/github-modernizer). the other examples the feature Undetectables for a list of what it detection would return a false can’t detect. If you must use one positive. If one of these cases of these features, then a browser applies to you, it might be time detector might be necessary. to start detecting browsers. There’s a whole ecosystem of device detection services, of which WURFL (smashed.by/wurfl) and Device Atlas (smashed.by/atlas) are the best known. Hand it the UA string of a mobile browser, and it will tell you something about its capabilities. It took me a while to shake off my desktop-formed preconceptions and to recognize that in the mobile world the art of browser BY PETER - PAUL KOCH |

detection is much more evolved than on the desktop, and that it is simply necessary due to the hundreds of devices that might visit our clients’ websites. If you must roll your own detection script, assume that any device is of the worst kind and does not support the feature you want to use unless it appears on your white list. Create this white list yourself by adding the browser UA strings of devices that support the given feature. And yes, this is pretty tricky because you’ll have to test hundreds of devices in order to create a really good white list. That’s why, in general, going with established services is better.

Javascript Libraries, Bad The second outdated reflex is to use a JavaScript library for absolutely everything. This is the sad result of the over-reliance on libraries that we developed in the 2006 to 2011 era, to the extent that some Web developers can’t even write JavaScript anymore. I’ve always had reservations about JavaScript libraries, and they were “Who Killed My Battery?: reinforced by a research paper from Analyzing Mobile Browser April 2012. The researchers measured Energy Consumption,” see the battery use of an Android phone smashed.by/wwwcon. while loading several websites, including Wikipedia, and experimented with redesigning one function of that website. Wikipedia’s accordion script, which uses jQuery, was replaced by a custom-made function, and the measured energy consumption for downloading and rendering the page fell by one third. The size of the download isn’t even the issue: library vendors are well aware of that problem and have taken steps to make their files as light as possible. The problem is that the entire library has to be executed, draining the battery as feature after feature is initialized— and your page might not even use most of the features.

50 | WHAT’S GOING ON IN MOBILE?

If Web developers would just use JavaScript instead of a library for simple tasks, the mobile Web would be a much better place. Complex interfaces with a lot of script-driven interactions are another matter. With them, the judicious use of a carefully crafted JavaScript library might be warranted. On the other hand, it might not. In any case, we have to get rid of the reflexive use of libraries without thinking about the mobile network and battery life. THE MOBILE NETWORK Finally, a few notes about the mobile network. Although TCP/ IP—and, thus, the Web—works fine over the mobile network, it’s nonetheless quite different from the networks we’re used to on the laptop, because it was made for different use cases.

Network Speed Instinctively, Web developers assume that a 3G connection is slower than a Wi-Fi connection. In general, that’s true, but it’s important to realize that this general rule doesn’t always hold. Sometimes a user is in a public space with Wi-Fi—but it’s slow or unreliable Wi-Fi that’s being used by many people simultaneously. In that case, the user’s 3G connection might actually be faster. Do not fall into the trap of assuming that a user on 3G has a slow connection speed, or that a user on Wi-Fi has a fast one. Connection type is not a proxy for connection speed. The roll-out of LTE (or 4G), which is proceeding apace in the US but is going much slower in Europe, will likely improve mobile connection speeds significantly. Nonetheless, all caveats that we discussed will continue to apply, since LTE connections, too, may be overwhelmed by sudden demand for connections, or slow due to technical difficulties. Again, connection type is not a proxy for connection speed. What we really need is a way to figure out the true connection speed. I’d love to have a JavaScript property that gives a user’s estiBY PETER - PAUL KOCH |

mated average connection speed over the last five minutes or so, so that we actually have useful data to decide whether to send highend or low-end images to the device.

Connectivity Mobile networks were set up to accommodate devices that only have to be reachable occasionallyâ&#x20AC;&#x201D;when a device makes or receives a call and when it sends or receives an SMS. Setting up a mobile connection takes some time, Steve Souders has done groundand if the mobile connection breaking research in this area and remains idle for a while, it is found the numbers quoted here closed down in order to save (see smashed.by/mobile-connection). battery life. Chapter 5 contains a lot more inforThis principle also goes mation about the mobile network. for websites: when the browser requests assets, it takes roughly 2 seconds to set up a mobile connection, which then closes down after 5 to 12 seconds of inactivity. Two seconds might not sound like much, but compounded with the normal latency of a mobile network and Web server, it makes for an annoying wait. This is where early Androids went horrendously wrong: they treated the mobile network as if it were a normal fixed landline network and ran countless background services that requested data every few minutes or so. The error was corrected in Android 3. The worst thing you could do is send a request for a tiny bit of data, then wait for the user to do something else, and then send another request for a small bit of data. Every single time the user requests something, they have to wait, and that becomes very annoying after a while. Itâ&#x20AC;&#x2122;s much better to make an educated guess as to which bits of data the user will need, load them while your main website is loading, and then make similar guesses every time the user requests bits of data that you did not foresee. 52 | WHATâ&#x20AC;&#x2122;S GOING ON IN MOBILE?

Tablets Finally, a brief note about tablets, which we haven’t really talked about yet. To me, tablets are part of the desktop market. The only reason they’re often lumped with mobile is that they’re a new category and have touchscreens and run mobile OSs (except for Windows tablets). No one will hesitate between buying a tablet or a mobile phone, but more and more consumers are wondering whether to buy a tablet instead of a laptop. Thus, tablets are competitors of desktop and laptop computers, not of mobile phones. Technically, treating tablets as part of the mobile market might make sense: Safari for iPad is the same browser as Safari for iPhone but doesn’t resemble Safari for Mac much. Still, the fact that some technical aspects of tablets fall in the mobile realm (because we define iOS, Android and BlackBerry as “mobile”) should not blind us to the fact that the tablet market is completely distinct from the mobile one and that the two do not interact to a meaningful degree. Tablets have much larger screens than mobile phones, so much so that non-mobile-optimized websites will usually render decently. That’s why it’s best to start on mobile when you study new devices, and leave tablets aside for the moment. In my opinion, they’re just not different enough from desktops in screen size.

Conclusion We’ve reached the end of our journey through the mobile world. As you’ve seen, it’s quite different than the desktop space. To summarize the most important conclusions of this chapter: Ŧ Never assume anything on mobile. Never. Anything. Ŧ There are vastly more browsers on mobile than on desktop. Ŧ Operators and device vendors are afraid of commoditization

and irrelevance and will defend themselves by influencing the user’s overall experience. Sometimes this is not a good idea. BY PETER - PAUL KOCH |

Ŧ Operators could handle identity and payments much more easŦ Ŧ Ŧ

Ŧ Ŧ Ŧ Ŧ Ŧ Ŧ Ŧ Ŧ

ily than any other party but, mysteriously, don’t do it. Each country is its own device market. Sales market share is less important than installed base, which is less important than browser market share. To avoid commoditization, operators and device vendors try to differentiate themselves from competitors—especially ones that use the same OS. Entering the device market is very difficult and expensive. OS vendors are becoming more important. The Web will become an OS, but currently it’s a weapon in the hands of the losers. Desktop rendering engines are taking over the mobile world. Proxy browsing saves customers money at the cost of clientside interactivity. One WebKit-based browser is not the same as another WebKitbased browser. Create a device lab—by yourself or with other freelancers or small businesses. Use browser detection. Don’t use JavaScript libraries.

Good luck in finding your own way through the mobile landscape. This is Only the start of your journey.

ABOUT THE AUTHOR Peter-Paul Koch is a mobile platform strategist, consultant, and trainer in Amsterdam, the Netherlands. He specializes in HTML, CSS, JavaScript, and browser compatibility.

54 | WHAT’S GOING ON IN MOBILE?

CHAPTER TWO

BY STEPHANIE RIEGER

THE FUTURE OF MOBILE

“

First mobile was the standard old cell phone. You talked into it. The second mobile era was brought to us by the iPhone. You poked at a screen. The third era will bring us a mobile that saves us from clicking on the screen.” – Robert Scoble1

REDICTING THE FUTURE of technology is always hard. Seen from a distance, progress feels almost inevitable. With each passing day, products seem to get better, smaller and cheaper. The transition from one technology to the next is rarely painless (or free), but by the time technologies enter the lives of “normal people,” most major kinks have probably been worked out. Of course, we are not those normal people. We designers seek out technologies well before they reach the mainstream. We dream up crazy ways to use or enhance them. Sometimes we are even called on to define them. It’s therefore not surprising that we sometimes forget how long it can take for a technology to truly matter to normal people. Take the humble touchscreen—a technology that first leapt into the mainstream with the release of the iPhone and is now so common that it’s practically become an expectation on portable devices. The iPhone was obviously not the first device with a touchscreen, but what you might not know is that touchscreen techno-logy has in fact existed for more than 40 years.2 1. “Mobile 3.0 Arrives,” smashed.by/3rd-generation. 2. Wikipedia provides a good overview under smashed.by/history.

BY STEPHANIE RIEGER | 55

Those 40 years have seen a great deal of change, including the rise of the microprocessor, the invention of on-board memory, vast improvements in battery life and, of course, advancements in touch technology itself. Those 40 years also saw the release of many products whose features and capabilities were, in many ways, not all that different from the iPhone’s—features such as the ability to install applications, to surf the “real Web” using a standards-based browser, to control the device or input text using touch, and even to draw or annotate content using the touchscreen. What made the iPhone more successful than many of these products was not its use of a new technology, but the brilliant, thoughtful and fun way it brought so many existing technologies together. This is what made the iPhone great and revolutionized the way we use (and think about) the humble touchscreen. The rise of the touchscreen provides a great example of how challenging predicting the future can be. While it’s easy to identify technologies that will reach (or have already reached) sufficient maturity to be commercialized, it’s far harder to predict which will grow the fastest, what new behaviors will be created, and what new and unexpected products will be spawned in return. It’s also hard to predict where we might get stuck along the way—due to patent wars, “not invented here” syndrome and the reluctance among many companies to cannibalize their existing products by releasing something truly disruptive. In this chapter, I will provide a glimpse of where the future of mobile might lead, and what technologies may lead us there. These include new low-power computer chips, fantastic new display technologies, new APIs and the growing penetration of near field communication (NFC). But more important than the technologies themselves is how they will need to work together, enabling new and exciting ways to do business, to connect with friends and family and to interact with the world around us. 56 | THE FUTURE OF MOBILE

Let’s begin by examining key trends that have already begun to transform our lives and that will continue to shape our future. These trends all revolve around connectivity and the impact that new ways of connecting will have on people’s lives.

Trend: Everyone Will Be Connected Fast low-cost computer chips and open software have taken the world by storm. Each and every day, millions of people purchase a smartphone.3 These devices are cheaper, faster and more versatile than they have ever been. Most are also Internet-enabled and include a modern Web browser. If current trends hold true (and there is currently nothing to indicate that they won’t), we will very soon all be connected. While many of us will not have constant access to broadband or Wi-Fi, most of us will live within reach of a cellular network. We will also own devices that can connect to these networks, and we will have access to reasonable data rates. Many of our devices will be “smart,” but as we will discover in this chapter, the meaning of “smart” will likely change. The impact of all this connectivity can be hard to predict, but it’s sure to affect every aspect of our lives. We already live in a world where, thanks to the software on our mobile phones, getting lost is almost impossible; a world where connecting and collaborating with people across the globe is becoming increasingly cheap and easy; a world where we have the sum of all human knowledge (and cat photos) at our fingertips at any time and any place. The growth in connectivity has clearly revolutionized the way we live. We can only imagine how our lives will change once we are all connected.

3. Pinpointing the most accurate phone activation data can be difficult, so let’s look at just

one platform. As of September 2012 (smashed.by/android-activation), Android alone accounted for 1.3 million activations a day, up from 1 million in July and 700,000 just a few months prior. Given that Android is just one smartphone platform, the total number of daily phone activations could easily be twice that much. BY STEPHANIE RIEGER | 57

Trend: Everything Will Be Connected Did you know that our world already holds more connected devices than people? In fact, Cisco predicts that by 2015 there will be 15 billion connected devices on the planet. 4 If these trends hold true, our future will be populated not just by connected people, but by connected things. These devices can be loosely broken down into two categories: Ŧ connected things with screens, Ŧ connected things (without screens).

Calling a group of devices connected things with screens might seem simplistic, but it’s not far from the truth. A screen is merely a component of a larger product. Each component has a basic unit cost. That cost is added to the bill of materials5 and ultimately passed on to the consumer. As components go, screens are pretty expensive (especially modern ones that respond to touch); so, historically, products haven’t included screens unless they really needed them. Recently, however, the cost of touchscreens has substantially decreased and is expected to continue doing so.6 Connectivity is also a component (or, as “Smart Things” author Mike Kuniavsky would say, maybe even “a material”). Like any other component, connectivity imposes design constraints and affects the cost of the device. In recent years, however, the components that enable connectivity have been bundled have been bundled into tiny chips—some costing as little as $0.20.7 These chips also consume 4. Evans, Dave. “Ten In Ten: Ten Technology Trends That Will Change The World In Ten

Years,” smashed.by/teninten. 5. Wikipedia, “Bill of Materials,” smashed.by/billofmaterials. 6. According to Moore’s law, the number of transistors on a microchip doubles every 24 months. This results either in far more capable chips or in equally capable chips that sell for a fraction of their original cost: smashed.by/mooreslaw. 7. The Guardian, “Flycatcher computer chip,” smashed.by/flycatcher.

58 | THE FUTURE OF MOBILE

very little power. This creates an interesting environment—one where it’s now feasible to embed connectivity into almost any product. But these products are are not computers (at least not in the traditional sense). They don’t necessarily require a screen. They may not even require direct human interaction. These products (and the ecosystem in which they operate) are commonly referred to as the Internet of Things (IOT)—a world inhabited by millions of autonomous connected objects. We will discover later in the chapter that today’s mobile device may not yet be a part of this ecosystem, but will likely play an important role in its future success. CONNECTED THINGS WITH SCREENS We are in the early days of the “connected things with screens” era. We can expect a great deal of experimentation in the coming years, with more than a few products released simply because “we can.” This opportunistic pairing of screens with devices is already well under way in the lifestyle and home appliances sectors.

Connected TVs Connected TVs (increasingly referred to as “Smart TVs”) don’t neatly fit into the “connected things with screens” category, but they will likely play an important role in our future. Television habits have already changed a great deal. Thanks to smartphones and tablets, a large number of consumers already combine television viewing with Internet use. This behavior includes general multitasking, texting friends about a program and Googling products seen on TV.8 Adding connectivity to the TV will not only change how the TV is used, but will also change the role of mobile as a companion device. One of the most obvious uses of mobile is as an enhancement—or

8. Watching TV while using a device is often referred to as “dual-screen behavior.” An Inter-

net search for this term will return the most up-to-date studies.

BY STEPHANIE RIEGER | 59

outright replacement—of the traditional (and much hated) television remote control. Certain manufacturers already offer apps to offload complex interactions such as inputting URLs and credit cards to far more usable smartphone and tablet interfaces. Some of these apps also enable users to initiate streaming of content such as photos and movies from one device to the other. And because many Web browsers now include automatic tab and bookmark synchronization, sharing Web content from device to device should soon be quite straightforward as well. Manufacturers are also introducing Once controlling the TV gesture-and voice-based interacbecomes trivial and we are tions for TVs. This may prove to be able to seamlessly beam (i.e. push or pull) content from one one of those purely opportunistic feature choices that come back device to another, we might to haunt us. Adding a camera and begin to think of the TV in an entirely different way. It could audio-in capabilities makes the TV ideal for video chatting with famsimply become “just another ily and friends, so why not use that screen,” one that is as suitable same camera to enable gestures? for watching a film as it is for After watching an exhaustedcompleting a tax return or looking sales person demonstrate enjoying a video call with gesture-based surfing and data family. input in a Bangkok showroom, I have a feeling that this feature will require Connected Fridges a lot more iteration before we deterConnected fridges are still far mine what it’s truly good for. from mainstream, but new models are appearing with shocking regularity. Some consist of no more than a screen embedded in the fridge door, the way an ice dispenser might be. Conceptually, these “devices” are not that different from the mini-TVs that appeared in kitchens in the early 1980s. Now, instead of simply watching TV, you can surf or send mail while you cook—not exactly 60 | THE FUTURE OF MOBILE

life-changing, given that you can do the same with a tablet (which can be conveniently repositioned as needed). In an attempt to make these products more useful, manufacturers have recently begun embedding fridges with apps that control refrigerator settings or provide recipes.9 The long-term goal is to turn the fridge into a virtual kitchen helper, using sensors to keep track of what’s in the fridge, to suggest meals and to automatically place an order when key ingredients are running low. Connected fridges are a perfect example of a technology that could take years to find its place. While the idea of a fridge that shops for you sounds useful, the technology won’t be useful unless the items in the fridge are discoverable. An entire ecosystem will have to develop before these features are workable. In the meantime, it will be interesting to see how many consumers embrace the “check your mail and watch a movie while cooking” scenario. Tip: Be sure to read the section titled “Control vs. Interoperability” near the end of this chapter for a surprising answer to this question.

In-Car Displays Most car manufacturers are already experimenting with ways to enhance the dashboard using a collection of touchscreens.10 These screens often provide a single interface that groups functions that were previously separate, such as radio, music, navigation, trip computer and vehicle configurations such as those for interior mood lighting. In an effort to improve safety (and to meet state-and countryspecific regulations), manufacturers have also provided connection points for devices. These enable drivers to control common phone functions hands-free and also to share data such as contacts with 9. “The ‘Smart Fridge’ Finds the Lost Lettuce, for a Price”: smashed.by/smartfrigde. 10. Wikipedia, “Connected Car,” smashed.by/connected-car.

BY STEPHANIE RIEGER | 61

the in-car navigation system. This is an interesting extension of the things with screens concept in that it enables drivers (or passengers) to bring their own device and bypass the native controls in favor of context-appropriate ones. These displays will be “mobile,” but not in the way we would normally expect. Location will often play a key role, but a user’s current location could in some cases be far less relevant than their intended destination. Regional searches could, therefore, be far more important than the 1 to 2 kilometer radius that we currently think of when designing services for “local search.” Another long-term consideration will be lack of connectivity. Even if cars come equipped with a SIM card for wireless network access, this service will be prone to interruption as users drive from one region to the next, pass through areas with poor connectivity, or switch from one regional provider to another. Apps that seamlessly manage and negotiate changes in bandwidth will do very well in this environment.

Consumer Customization Things that don’t natively come with screens may also acquire them at a later date—either temporarily or as a permanent improvised enhancement. A quick Google search already uncovers all manner of bracket designed to affix a smartphone or tablet to your wall, desk, or in-car dashboard. As devices get cheaper (and communication protocols are standardized), it might make more sense for a consumer (or service provider) to choose the most appropriate device for their market or price point, install context- or task-specific applications and then simply mount the device in the given environment. Devices used in this way have already begun to appear in retail and hospitality settings, where self-serve tablets are pre-fixed to

62 | THE FUTURE OF MOBILE

tables in lieu of menus and store directories.11 Some of these simply provide information, while others enable customers to choose products and even finalize the transaction. These do-it-yourself scenarios are unpredictable, but they happen every day. They may also provide more value to users than an opportunistically-connected fridge. As a designer, it’s increasingly important to consider that your product will be interacted with in novel and unexpected ways. These scenarios should be embraced and directly considered when designing future products. As illustrated in the examples above, mobile devices may not always be the center of attention. In fact, they are often most useful when interacted with sporadically to enhance everyday interactions with people, places and things. This less-focused mobile behavior will only increase. The smarter the things around us become, the less we will need to spend our days staring at our beloved glowing rectangles. CONNECTED THINGS (WITHOUT SCREENS) Let’s begin by clarifying the difference between connected things and the connected devices we use today. The majority of today’s connected devices are, in effect, small multi-purpose computers. They have sophisticated operating systems and (thanks to third-party applications) offer a seemingly endless range of capabilities. Being connected to the Internet greatly increases their usefulness, but they are still useful in many ways when out of contact with the network. The connected things we will discuss next are not computers in the traditional sense. They are regular objects in which connectivity has been integrated as enablers to larger services. The connectivity (wireless, infrared, etc.) exists for the sole purpose of exchanging data with the cloud or an adjacent device. 11. USA Today, “iPads Increasingly Crop Up on Restaurant Menus for Ordering,”

smashed.by/ipad-restaurant. BY STEPHANIE RIEGER | 63

Connected things may, therefore, not have traditional (i.e. softwarebased) user interfaces. This is because interacting with data while using the object isn’t necessarily the point—it may merely be a very useful side effect. To better understand the relationship these objects may have with mobile devices (and the people who use them), let’s look at a few services that already incorporate connected things.

The Sonos Multi-Room Music System Sonos is a service that enables users to stream music throughout their home.

FIGURE 2.1. The various components of the Sonos system. (Image: Stephanie Rieger)

The system consists of a series of wireless speakers, a tiny control box and an optional handheld remote control. The control box connects wirelessly to your music, which is either stored locally on a computer or network-area storage (NAS) or streamed off the Internet using services such as Spotify and Internet radio. The Sonos system then streams this music to the speakers distributed throughout your home. 64 | THE FUTURE OF MOBILE

Sonos supports the creation of different zones, enabling you to simultaneously play different tracks in different parts of the house. Once the system is set up, the speakers and control box do all the work necessary to negotiate streaming from room to room. Purchasing the Sonos remote control is optional. Users can instead download a smartphone app that enables them to create playlists, adjust the volume and change tracks. Connectivity is an integral component of the Sonos system, but certain components in this system (the speakers, for example) are merely connected objects and communicate only with other devices. Embedding a screen in each speaker (which is likely hidden behind furniture or mounted high up on a wall) wouldnâ&#x20AC;&#x2122;t improve the user experience at all; proxying this interaction through the overall service and then accessing it (from anywhere in the house) using a mobile device makes far more sense. The next example takes the idea of a connected object a step further. In this case, the connected object is a fairly mundane piece of plastic that weighs just a few grams, yet is the heart of an entire service that could ultimately save lives.

The GlowCap Service The GlowCap service is designed to ensure that patients properly take their medication. The service consists of a cloud component and a series of objects that periodically transmit data back to the cloud. These objects are about as far from traditional computers as you can imagine. They are simply smart-connected lids for pill bottles. Pharmacists use these lids when dispensing medication to GlowCap subscribers. Once at the userâ&#x20AC;&#x2122;s home, the lid communicates with a small control box, which in turn connects to the cloud. Each time a patient takes their medication, the GlowCap lid reports the date and time of the action. The service tracks usage, so if BY STEPHANIE RIEGER | 65

the patient forgets, the lid glows and emits a sound as a reminder. If the patient still does not comply, the cap communicates with the control box once again to trigger an email to the patientâ&#x20AC;&#x2122;s doctor or designated family member. Patients and doctors can track their progress using the GlowCap website or smartphone application. Similar to Sonos, the value is in the data and the overall service. Most interaction with the service happens naturally, by simply taking the cap off the bottle. There is no screen- or software-based user interface. This keeps the cost of the lids down and is far more useful than forcing the user to interact with a tiny screen embedded in an object that could easily become FIGURE 2.2. A prescription bottle feawet or end up in the bottom of a turing the GlowCap. (Image: Vitality) purse.

Connected Things Outside the Home The previous examples have described products we use primarily at home, but we can also expect to find an increasing range of connected objects in our city spaces, shops and recreational areas. How we will interact with these objects will vary. Similar to the in-home examples, some objects may not be designed for human interaction at all. Many will simply consist of sensors, actuators and some form of connectivity or communication capability embedded in day-to-day infrastructure, such as lamp posts, public transportation vehicles, product labels and parking spots.

66 | THE FUTURE OF MOBILE

Systems such as these are already being used to monitor and collect a host of infrastructure, process and environmental data in warehouses, logistics operations and major city infrastructure around the world. This data is analyzed in real time, enabling adjustment of traffic patterns on busy streets,12 letting drivers know the cost and location of the nearest parking spot,13 and sensing the temperature of a railway track to predict when a wheel is about to fail.14 These types of connected objects exist simply as part of a large, invisible system designed to improve our lives. So long as they work, we will be happy if we never see them or know they exist. But in other scenarios (such as retail or maybe transportation), discovering and seeing these objects may be the entire point. In Brazil, for example, a retailer by the name of C&A15 has developed connected clothes hangers that display—in realtime—the number of likes that a particular item of clothing has received on Facebook. Clothes hangers are pretty useful on their own, yet, thanks to connectivity (and a small LCD display), these hangers are no longer simply well-designed lumps of wood or plastic. They can now play a more active role in the retail value chain. But is a Facebook-aware clothes hanger truly useful? How could we further improve its value to retailers, as well as to consumers? Imagine for example that your smartphone could directly communicate with the hanger—using it as a proxy to interact with the Facebook service. You could then not only see the number of “Likes”, but also add your own without the need to open (or download) a Facebook app, and then find that particular pair of jeans amongst millions of other products. 12. Wikipedia, “Traffic Light Control and Coordination,” smashed.by/traffic-control. 13. SFpark: smashed.by/sfpark. 14. Information Week, “Union Pacific Delivers Internet of Things Reality Check,”

smashed.by/union-pacific. 15. Mashable, “Clothes Hangers Show Real-Time Facebook Likes,” smashed.by/clothing-rtlikes. BY STEPHANIE RIEGER | 67

Then again, you might not care how those jeans rank on Facebook at all. You may be far more interested in their washing instructions, the materials they were made from or the manufacturer’s ecological footprint. You might also want to know how best to accessorize the jeans with using nearby items. Displaying this level of detail on a tiny 2-inch LCD attached to a clothes hanger would, of course, not be appropriate. Projecting it onto a single point-ofpurchase (POP) or wall-sized display would also be impractical (after all, there could be hundreds of individual clothing items in the store). The most obvious candidate to display this information might instead be the device that shoppers already happen to have with them. But using today’s technologies, it could take several minutes, and many taps or swipes to locate this information on a brand’s website. Most shoppers are also unlikely to download an app just to look up washing instructions for a pair of jeans that they might not end up buying. Wouldn’t it make far more sense for shoppers to simply “ask” the clothes hanger (or the “smart” label attached to the jeans) to provide them with this information, and then to use whatever screen they happen to be carrying to display it? But we’re not quite there yet. The technologies required to enable these types of interactions aren’t terribly complex, but for this to work, they’re all going to have to find ways to work together. Thankfully, the bits and pieces we need to make all this happen are slowly falling into place.

Key IoT Technologies The first thing we will need are communication and identification protocols that enable these smart (but often mundane) objects to exist on the network, to securely transmit data, and to do so using as little power as possible. The Internet’s TCP protocols might seem ideal, but they require far too much processing power. Standards 68 | THE FUTURE OF MOBILE

bodies16 such as IPSO, 6LoWPAN and ZigBee IP have, therefore, been formed to develop a whole new generation of low-power IP (Internet) protocols. As is often the case with standards, potential solutions may come from several groups and it’s impossible to tell which will be adopted long term. What we can only hope is that the successful standards enable simple and flexible communication not just with (and between) objects, but also between the standards themselves. The second critical component will be a UI layer—one that is easy to implement, widely interoperable and (ideally) non-proprietary (i.e. that doesn’t require people to download an app—let alone a different app for each store, brand or location they wish to interact with). A Web standards-based solution would obviously be a strong candidate, but the Web platform will need to evolve to make this possible. There are also technical barriers to overcome if we wish to use standard IP protocols as a communication channel in objects that might not contain a power supply. This brings us to the final key component: some sort of discovery service, one that ideally combines the ability to search for objects and an unobtrusive push-style mechanism to discover objects nearby.17 Let’s now turn to a few technologies that are far closer to becoming a reality, especially for the “real people” we mentioned at the beginning of this chapter.

16. Wikipedia, “IPSO Alliance,” smashed.by/ipso. 17. Scott Jenson, former Creative Director of frog design, has written an excellent series of

articles (http://jenson.org) describing the Internet of Things ecosystem, the challenges of just-in-time discovery and how such a discovery system could work. BY STEPHANIE RIEGER | 69

Future Technologies In this section, we will explore a few additional technologies that will enable the future contexts we’ve discussed: Ŧ future display technologies, Ŧ RFID and NFC, Ŧ new Web and device APIs.

These technologies will enable three key aspects of our future environment: smaller, cheaper and more flexible displays, new ways to spontaneously exchange data between devices, and new methods of communication between software and host devices. FUTURE DISPLAY TECHNOLOGIES The past 10 years have seen an explosion in computerized displays. Screens already pretty much surround us: in our pockets, on billboards in the subway, in shop windows on the street and throughout many of the rooms in our homes. The way we interact with screens is also changing: transitioning from purely indirect manipulation (using a mouse, keyboard or remote control) to a combination of touch, gesture and even voice. We’re still barely getting used to this diversity, yet products are on the horizon that will further challenge how we design for and use the screens that surround us. One of the most exciting changes will likely be the commercial release of flexible displays. Two technologies are likely to dominate this marketplace: flexible e-paper and flexible active-matrix organic light-emitting diode (AMOLED). E-paper has been around for many years18 but has finally entered the mainstream thanks to lower costs, and the success of e-Readers 18. Wikipedia, “Electronic Paper,” smashed.by/epaper.

70 | THE FUTURE OF MOBILE

such as Amazon Kindle. All e-paper devices to date have been rigid, but announcements from the likes of LG suggest that we will see the first commercial release of flexible e-paper as early as 2013. Where these screens will first appear is anyone’s guess, but the magazine industry has expressed great interest in them as a means of embedding interactive “content zones” into existing print media. To date, all commercial implementations of e-paper have also been monochrome. Color technologies do exist, but they are still quite expensive, and color saturation tends to be far lower than on conventional displays. The PPI can also be quite low—averaging about 110 PPI. This is where Flexible AMOLED comes in. Flexible AMOLED displays are said to provide the same power consumption, brightness, color saturation and contrast of AMOLED, while also being flexible, and fully transparent—enabling you to look right through the display. Pixel densities of up to 300 PPI can be achieved, yet the display is light and flexible enough that it can be rolled up like a newspaper. These are exciting developments that will introduce many new contexts of use, and new interaction models. Imagine for example being able to twist or flex a display to indicate intent. This would be similar in principle to the interaction in accelerometer-enabled racing games. Tilt the device to the left and the car goes left; tilt it forward and the car speeds up. Now transpose these interactions to your favorite application on a flexible e-paper device. Bend the paper towards you to scroll up a list. Bend it away from you to scroll down. The more you bend, the faster you scroll. These interactions are disruptive and will introduce many of the same challenges we are currently grappling with on touchscreens:

BY STEPHANIE RIEGER | 71

Ŧ How will we provide appropriate affordances for these interacŦ Ŧ Ŧ Ŧ

tions? How will we minimize accidental triggering? How will we support multi-modal devices, or ensure backwards compatibility. Surely, these interactions won’t be practical on all devices? In which contexts will they be most appropriate? How will we accurately detect these capabilities? What properties will we even need to detect? How will these properties map to existing platforms and standards?

These questions are obviously yet to be resolved and may well prove as disruptive as the recent shift away from fixed layout and pixel measurements in Web design. It’s also worth remembering that we may have a lot of seemingly mundane uses for flexible displays. E-paper technology is already being used as a flexible (i.e. bendable) display on disposable batteries. Being able to determine how much charge is left in your battery might not require sexy gestures, but it certainly is useful! Until these technologies are released, the best way to learn about them is on the Internet. This collection of videos should provide food for thought about regarding the interactions we can expect to see in coming years: Ŧ Nokia Kinetic device, a flexible smartphone:

smashed.by/nokia-kinetic More videos available on Nokia Conversations: smashed.by/nokia-bendy Ŧ PaperPhone, a paper computer prototype from Queens University in Canada: smashed.by/paperphone

72 | THE FUTURE OF MOBILE

Ŧ Samsung flexible-paper AMOLED prototypes:

smashed.by/samsung-flexible Concept video: smashed.by/samsung-concept Ŧ Plastic Logic’s color e-paper technologies, featured in a BBC documentary: smashed.by/plastic-logic Heads-up displays (HUD) are transparent displays that present data without requiring users to look away from their usual viewpoint. To date, HUD technology As display technologies evolve, we has primarily been used to can also expect to see new and novel overlay valuable contextual uses of small displays. A particularly data (such as air speed and interesting small-screen context will altitude) on vehicle windbe the heads-up display (HUD). shields in military and aviation settings. Thanks to improvements in display technologies, affixing tiny HUDs to a simple headpiece or pair of glasses is now also possible. Google has recently showcased an upcoming HUD product, codenamed “Project Glass.”19 At the time of writing, very few details have been released, but early promotional videos suggest that users will be able to record, send and receive data while wearing Google’s futuristic-looking set of glasses. Whatever the final feature set, Project Glass is sure to prompt a great deal of discussion regarding the social and personal repercussions of cheap, unobtrusive HUD technology. Ŧ Privacy

Will it be socially (or legally) acceptable for a person to wander around a city, corporate office, or playground surreptitiously filming or photographing everything around them? 19. For the latest details on the project, be sure to visit the Project Glass page on Google+:

smashed.by/googleglass. BY STEPHANIE RIEGER | 73

Ŧ Safety

In what contexts would it be unsafe to use HUDs? Many countries (and quite a few US states) have already banned the use of handheld phones while driving (although using a phone handsfree is typically permitted). It’s hard to imagine how projecting an email inches from your eyeball could be less distracting. Ŧ Usability

Little information is available regarding the way we will actually use Google’s HUD. Given the size of the display, interactions will likely involve a combination of voice, the odd press of a FIGURE 2.3. A model wearing the Project Glass prototype heads-up hardware button (to turn display. (Image: Google) the device on and off), and simple gestures such as nods of the head. Will these limited (and possibly comical to watch) forms of interaction be sufficient, practical and usable in daily life? RFID AND NEAR FIELD COMMUNICATIONS (NFC) RFID is a means of exchanging information between objects or devices. If you live in a town or city, you may already use RFID daily when taking the bus or train to work. In this scenario, your bus card is an RFID transponder. It contains electronically stored information that will be read by an RFID reader each time you swipe the card when entering the bus. 74 | THE FUTURE OF MOBILE

RFID is also widely used in inventory and logistics to automatically track and identify items in warehouses and shipping depots. In fact, these types of implementations are far more common than in the public transportation scenario. This is because RFID tends to have high associated set-up costs. For large companies such as Amazon that must pick, pack, track and manage millions of stock items, the benefits of implementing RFID-based logistics may be immediate, and easy to meaOne of the most successful rollouts sure. In public transportation of RFID-based payments is the Hong scenarios, the benefits are typi- Kong Octopus Card.21 Implemented cally long-term and the impact in 1997, the system was initially deless immediately measurable. signed to eliminate the need for comThe main problem is that muters to find exact change. Fifteen RFID requires some level of years later, Octopus is a full-fledge ecosystem. Readers must be in- payment alternative, accepted as stalled on platforms and within payment at tens of thousands of cabusses or trains. Customers fés and convenience stores throughmust be notified of the new out the region. You can even use the payment type, and convinced card to pay your electricity bill. to switch from whatever old system they are used to. There may also be external partners, such as local and regional ticket vendors, that will need to be notified of the change, and trained to use the new equipment.20 This can lead to a frustrating chicken- and egg-scenario: investors and stakeholders may not understand the benefits of the technology until a critical mass of usage has been reached, but without money for infrastructure there will be little usage to reach that critical mass.

20. Wikipedia, ”Octopus Card,” smashed.by/octopus.

BY STEPHANIE RIEGER | 75

NFC NFC (a newer subset of RFID) seeks to remove many of the challenges of RFID while also increasing functionality and improving user experience. NFC differs from RFID in that it can be used for one- or two-way communications and has three operating modes: read/write, card emulation and P2P (peer-to-peer). Read/write mode behaves similar to RFID, but with the added benefit of writing capabilities, making it easy for almost anyone to embed retrievable data into an object. A common scenario is a subway poster to which an NFC tag has been attached. The user would simply tap their phone against the poster,21 triggering the transmission of a URL or other information stored in the tag. In this scenario, “write” mode refers to the ability to write the data on the tag before it’s embedded into the poster.22 In the future, we can imagine using read/write tags to provide just-in-time information about objects in shops, retrieve location-specific data about historical monuments, or access critical point-of need specifications in industrial settings. Card emulation mode transforms an NFC device such as a smartphone or tablet into a contactless smart card. This enables contactless payment and ticketing (as described in the RFID transportation payment example), but without costly changes to the existing infrastructure.23 This technology should also make it cheaper and easier to prototype payment scenarios to obtain much-needed buy-in from users and stakeholders. Finally, there is P2P mode, which allows two NFC-enabled devices to communicate, share information and exchange files. In cases where the data is small—for example, a 2 to 3 KB vCard contact— NFC can be used to exchange data during the short period of time 21. See the NFC website for more on NFC smart posters: smashed.by/smart-posters. 22. See the section below entitled “Get Your Hands Dirty” for examples of how to easily

prototype with writeable NFC tags. 23. “NFC Forum: Frequently Asked Questions,” smashed.by/nfc-faq.

76 | THE FUTURE OF MOBILE

that the devices are touching. For much larger data, NFC can be used to open a Wi-Fi or Bluetooth connection with the paired device. The data is then transmitted using the higher data-rate technology. There are many possible uses for P2P mode, such as quick sharing of contact information, in-store coupon delivery, initiation of two-player gaming sessions or connecting a camera and printer simply by touching one to the other. This choice of modes enables a much wider range of uses than the simple payment and tracking scenarios of RFID. Similar to RFID, however, what’s still missing is an ecosystem of readers and opportunities to use them. The device that will enable widespread NFC use will probably be the smartphone. Estimates by Forrester Research suggest that NFC-enabled handsets will reach critical mass (set as 15-25%) in the UK by 2014. But NFC support is only part of the equation. To ensure widespread adoption, NFC FIGURE 2.4. A Samsung read/write NFC needs to prove that it can TecTile. (Image: Samsung Mobile) provide real value to merchants, transaction processors and customers. Until this happens, we can expect to see continued growth in proprietary (non-NFC) digital wallet or payment systems, such as those offered by Starbucks24 and Square.25 These systems are akin to the polyfills that we’re used to on the Web: enabling valuable functionality, introducing behaviors, and creating mental models in preparation of an emerging standard.

24. Starbucks, “Mobile Applications”, smashed.by/starbucks. 25. Square, smashed.by/squareup.

BY STEPHANIE RIEGER | 77

Get Your Hands Dirty The best way to understand the true potential of a technology is to start prototyping. Prototyping an NFC user experience is becoming easier by the day with the release of products such as Samsung’s TecTiles.26 TecTiles are NFC tags that can easily be programmed to perform simple tasks using a mobile phone. Tags can be easily affixed to all sorts of objects (using the sticky backing provided) and programmed to trigger a call, set an alarm, update a Facebook status and much more—all with a single tap of an NFC-enabled phone. An NFC-enabled Android device is required to use TecTiles. You will also need the free TecTiles smartphone app that will enable you to write data and program actions to the read/write tag. NEW WEB AND DEVICE APIS The more devices can exchange data with other devices and platforms, the more useful they ultimately are. The presence (or absence) of an API is, therefore, a key indicator of how a technology may progress. Whether an API is open or proprietary may also impact its usefulness in the long term. New APIs are actively being developed to bridge the communication gap between platforms and devices. Developing APIs is a complex task. Progress is often slow, and there are many false starts. This can be discouraging, but what’s important to remember is that with each new API comes a new opportunity to experiment. This experimentation leads to better understanding of the problem the API is trying to solve, which, in the end, leads to better and more useful specifications.

Device APIs W3C Device APIs27 seek to facilitate deeper integration between 26. smashed.by/tectile 27. smashed.by/dap

78 | THE FUTURE OF MOBILE

Web applications and the capabilities of their host devices. These capabilities include access to the camera, microphone, address book, calendar, and even system information such as network connection and battery level. Device APIs aren’t exactly new. Similar standards were developed in the early 2000s as part of emerging (and now defunct) widget and packaged Web app frameworks such as Opera Widgets and Nokia’s Web Runtime.28 Mozilla has also undertaken the Launched in 2011, the Firefox development of a similar set of Mobile OS (previously named device APIs. 30 These APIs are “Boot to Gecko”) is a lightweight primarily destined for Mozilla’s mobile operating system designed Firefox Mobile OS, but they for lower-end smartphones, such might eventually be impleas those popular in many emergmented in Firefox’s standalone ing economies. Developed using mobile Web browser. open standards, Firefox Mobile OS aims to eliminate the need for apps to be built on platform-specific native APIs. The initiative is backed by leading global network operators including Deutsche29 Telekom, Etisalat, Smart, Sprint and Telefónica. The first FMOS device is expected to be released in 2013.30 Of course, communicating with a camera or calendar will only get us so far. To enable an Internet of Things, we will also need access to sensors. Here again, the native platforms have a head start, but the upcoming Sensor API Working Group31 is working hard to define DOM sensor events and data formats for the browser. 28. An excellent overview of this history and the progress of current Device API initiatives is

available in a slide deck (smashed.by/web11) and video (smashed.by/web11vid) from James Pearce, Head of Developer Advocacy at Facebook. 29. Mozilla Wiki, “WebAPI,” smashed.by/webapi. 30. The Mozilla Blog, “Mozilla Gains Global Support for a Firefox Mobile OS,” smashed.by/mobile-ff. 31. smashed.by/sensor-api BY STEPHANIE RIEGER | 79

These APIs aim to detect ambient temperature (in degrees Celsius), pressure (in kPa), relative humidity (as a percentage, ambient light (in Lux), ambient noise (in decibels adjusted, dBA) and proximity (in centimeters). Also of interest is a new framework called “Web Intents.” While Device and Sensor APIs communicate with device hardware, Web Intents32 looks to enable client-side service discovery and interapplication communication. Through Web Intents, services (i.e. apps) can register their intention (or ability) to handle an action on the user’s behalf. The system then listens for action requests

FIGURE 2.5. An artist’s rendering (from left to right) of the idle screen, inbox,

incoming call screen, gallery, contact details and browser start screen in the upcoming Firefox Mobile OS. (Image: Mozilla) 32. W3C Wiki, “WebIntents,” smashed.by/intents.

80 | THE FUTURE OF MOBILE

(based on verbs such as “share,” “edit” and “view”) and presents the user with the most appropriate services for that action. This should enable far more fluid, seamless and personalized interactions between the user and the platform.

Sensors and the Physical World Building an application that interfaces with your phone’s hardware is one thing, but how do you interface with everyday objects? Until recently, experimenting with the Internet of Things was a fairly specialized domain—home to people who could program, who understood electronics and who were happy to keep a soldering iron on the kitchen table. But as the Internet of Things moves from the domain of geeks into the mainstream, tools are finally emerging to enable almost anyone to write programs that interact with the physical world. “Ninja Blocks” (currently in Alpha) are small computers designed to make it easy to experiment with the Internet of Things without having to deal with electronics or programming. Whereas FIGURE 2.6. A Ninja Block. Samsung’s TecTiles used NFC to em(Image: Ninja Blocks) bed data within everyday objects, Ninja Blocks monitors and captures real-world data (such as temperature or movement) using a series of plug-in sensors. This data can then serve as a trigger for actions such as taking a picture or sending an email. All of this is initiated using a Web service called “Ninja Cloud.” Using the Ninja Cloud interface, controlling your things is as simple as writing a recipe:

BY STEPHANIE RIEGER | 81

FIGURE 2.7. The Ninja Cloud Web interface. (Image: Ninja Blocks)

1. Select an event that you want to watch. Events can be physical

(detected by a sensor) or virtual, such as the arrival of a new tweet. 2. Define a reaction for the event, such as “take a picture” or “update my Facebook status.” The beauty of Ninja Blocks is that it includes a full system: a microcomputer, sensors and an API that almost anyone can experiment with. Many of its components are also open, enabling users with more advanced skills to extend the blocks while taking advantage of the pre-built API. This creates an ideal environment for small teams of designers, developers and students of all disciplines to cheaply prototype products and interactions. Another interesting characteristic of Ninja Blocks is the way the blocks were initially prototyped. Manufacturing plastic blocks may be far cheaper than it once was, but producing products on a small scale still results in a high cost per unit. This makes it prohibitively expensive to design in an agile manner and to generate early and final-resolution prototypes. To alleviate these challenges, the makers of Ninja Blocks chose to print the early prototype blocks “just in time” using a table-top 3-D 82 | THE FUTURE OF MOBILE

printer. This enabled them to experiment and iterate the design using materials similar to those in the final product, but without the high cost and long lead times of a “proper” manufacturing process. And because Ninja Blocks has open-sourced its hardware specifications, it’s easy to imagine other companies, students and even hobbyists taking a FIGURE 2.8. A MakerBot Replicator printer similar approach to prototyp- and collection of 3-D printed objects. (Image: MakerBot Industries) ing custom blocks of their own.

Preparing for the Future We live in exciting, and yet often confusing times. Certain changes come quite quickly (last year, there was one Smart TV in my department store—this year there is almost nothing but). Meanwhile, other changes (the implementation of certain standards and APIs, for example) feel as if they may still take years to materialize. What we can be certain of, however, is that the future will be full of new technologies—each bringing new opportunities to innovate, delight customers and improve people’s lives. Many new products will be developed, but if history is anything to judge by, quite a few of these will fall short of their full potential. Some of the blame can be put down to poor timing (early e-paper eBook readers greatly suffered from a lack of available books), yet much of the blame may well lie with us—the people who design and develop these products. Companies have a long history of trying to own new technologies or ecosystems, often believing that if they lock each bit of a product down, consumers (or partners) will have no choice but to BY STEPHANIE RIEGER | 83

come to them. We are already seeing examples of this mind-set in products such as the connected fridge. CONTROL VS. INTEROPERABILITY I recently spent time reading online comments from more than 60 owners of connected fridges. The fact that 60 such people existed was shocking enough. Connected fridges have long been the butt of jokes in the UX community. Does anyone really need to check mail on a fridge (especially with all of the tablets and other connected devices that we now own)? Yet these 60 families had spent an average of $3000 US dollars (€2500) to buy one of these amazing new contraptions. Reading the comments, I half expected to find nothing but disappointment, yet most people absolutely loved this fridge. The fridge itself was well designed. It had big storage drawers and well-thoughtout controls, but more importantly, people were actively using the built-in touchscreen to access applications. The only recurring complaint was that the “connected” bit of the fridge (the computer on the inside) didn’t play well with anything else in their house. One user was desperate for Bluetooth connectivity (or at the very least an output jack) so she could plug in her favorite speakers to listen to music (the manufacturer had provided speakers, but she didn’t like the sound of them). Another complained that there was no way to connect the fridge to a printer (turns out it’s hard to follow recipes when you’re forced to stare at the fridge the whole time, so this owner wanted to print them out). Others complained that the customizable screensaver software wasn’t accessible “in a normal way” (such as through a file system or cloud-based interface). It was therefore impossible to remove the pre-installed promotional screensaver photos, or to easily create collections of family photos to display on the fridge.

84 | THE FUTURE OF MOBILE

None of the requested features and capabilities were all that strange, yet you can somewhat understand the manufacturer’s reluctance to implement them. You see, this manufacturer also makes printers, speakers and other kinds of electronics equipment—products that would make an ideal up-sell when purchasing a fridge. Why include industry-standard protocols and components, when you can lock users in to your own little connected fridge ecosystem? Connected fridges are an entirely new category of product, so imposing lock-in early on could also guarantee a dominant position in the budding “connected home” market. This type of logic often makes sense on paper but doesn’t map well to the world we now live in. A more flexible and interoperable product will always be more useful to consumers. It can be customized to suit each person’s needs, tastes, or budget, and the flexibility also entices third parties to develop add-ons and accessories. Far from reducing a product’s appeal, the presence of a third-party ecosystem actually enhances it. Would the iPad have been nearly as popular had Apple prevented third parties from developing helpful accessories, such as waterproof cases for the beach, mounts for the car dashboard or rubberized, chewable childproof bumpers? Apple took a big risk in choosing a non-standard connector plug for its first few generations of iOS products—a risk that paid off only because its devices were so incredibly successful. Thousands of products (including cars and hotel bathrooms) now sport an iOS plug, a plug that Apple recently announced it is discontinuing, in favor of yet another non-standard plug design. Will Apple’s products remain iconic enough for these manufacturers to change all their products to meet the new design? One could argue, however, that owning an iPad or connected fridge isn’t critical to a person’s well being. Does it really matter how interoperable it is?

BY STEPHANIE RIEGER | 85

In the grand scheme of things, the answer is probably “No.” But would you feel any differently if we were talking about an entirely different type of connected object? What about a connected diabetic sugar monitor? Or maybe an entire country full of connected railway track safety sensors? Should these products be based on open and inclusive standards, or controlled by whichever company won the initial installation contract (thus forcing taxpayers and consumers to choose between the cost of remaining with a supplier that no longer suits them, and the cost of upgrading specific hardware in order to be able to engage an entirely new supplier)? These are the types of questions we will need to answer if connected objects are to fulfill their true potential and help us lead better lives. TOWARDS THE FUTURE History has taught us that new technologies need at least two things to succeed—a strong ecosystem, and a handful of iconic products that demonstrate why this technology is truly useful (and wonderful), rather than simply being new. History has also taught us that, in many cases, the seeds of these future products were sown many years ago. Most of the technologies we will encounter in the decades to come are already here, just waiting to be discovered and experimented with by a whole new generation of designers and developers. Seek out these new technologies. Experiment with them where you can, and begin to think about the kinds of products and ecosystems that will enable them to shine. Remember as well that the content and services you design today may need to communicate (or at the very least co-exist) with a whole host of new and often unexpected connected things. As our connected fridge example revealed, the key to making friends with these future things will be 86 | THE FUTURE OF MOBILE

flexibility—flexibility that will enable users to decide where and how to access your product, and will enable it to adapt to rapid or widespread changes in context (you will likely experience both). My final advice is this: don’t try to predict the future. Carefully observe what’s happening today, and learn about what is coming next. Think of ways to build flexibility into each and every product you develop. Make the best decisions you can make based on your— or your client’s—circumstances, goals and constraints. And be prepared for change. Lots of work remains to be done to make all of this work, and it’s really just the beginning. As mobile Web designer Brad Frost often says at the end of his presentations: “This is going to be fun!”

“

TThe future is a journey, not a destination.” –Bruce Sterling

BY STEPHANIE RIEGER | 87

ABOUT THE AUTHOR Stephanie Rieger is a writer and designer with a passion for the many ways people interact with technology. With a diverse background, Stephanieâ&#x20AC;&#x2122;s expertise lies in marrying design, technology, and business goals to craft simple, elegant experiences.

88 | THE FUTURE OF MOBILE

Responsive Web Design iii. Responsive Design Strategy iv. Responsive Design Patterns v. Optimization For Mobile

CHAPTER THREE

BY TRENT WALTON

RESPONSIVE DESIGN STRATEGY

AM ALLERGIC TO CEDAR POLLEN. Here in the Texas Hill Country, cedar trees pollenate during the winter, triggering a cedar fever blight that makes over half the population miserable, cranky and desperate to know how bad pollen levels will be each day. Some days it’s so bad that I’ll wear a bandana or surgical mask over my face if I’m going to be outside for too long. And with that, your perception of me as a rugged Texas mountain man is shattered. Alas, moving on… During this season, the first thing I do is check the weather section of my local news website to know the pollen levels for the day. I’ve visited this website every winter day for three years until recently, when the news outlet launched device-specific versions of the website with a completely different information architecture and hierarchy from one device to the next. Even worse, it decided to pare down core content in the various versions from desktop to tablet to mobile. When I pulled up the website on my phone, I got a mobile version with 10 general headlines and no clear path to BY TRENT WALTON | 91

the pollen levels, even after multiple guesses with the drill-down navigation. This “update” wasn’t about optimization, and the overall strategy was ill-conceived. I’ve not been back to the website since. Creating a new version of your website for each new device that hits the market is neither a responsible nor a sustainable practice. This is why a responsive approach has become the default for how I build for the Web. If I were advising a small community center or a mega-corporation on its Web presence, my advice would be the same: responsive Web design is the best, most sensible place to start for any project. In the case of that news outlet, it would have one version to maintain, one version to budget for and one version of consistent content for users. Thus, when new devices hit the market (like seven-inch tablets), it wouldn’t have to segment its Web budget again to start from scratch on a whole new experience. At its core, responsive Web design (RWD) is defined by three key components: flexible grids, flexible images and media queries. In the next three chapters, we will discover that these components are just the tip of the iceberg. And with the ever-increasing number of devices flooding the market, RWD is the most effective way to address them all at once. We will always be able to enhance or optimize further through techniques such as app caching and browser detection, but RWD is definitely the best place to start. So, where should we start? How about with a confession?

From Pixels to Proportions As much as I believed RWD was the brilliant future of our craft, every cell in my Web designer body screamed out in terror when I first saw it in action. I wanted to cower under my desk after reading Ethan Marcotte’s visionary A List Apart article “Responsive Web Design”1 in

1. A List Apart, “Responsive Web Design,” smashed.by/rwd.

92 | RESPONSIVE DESIGN STRATEGY

May of 2010. Ethan had coined the term, connecting the dots from flexible grids to flexible images to media queries. He even built us a demo showing how they work together magically, reflowing the page into any sized viewport. What wasn’t so quick and easy, however, was my acceptance of this new reality. My fear was rooted in how I perceived and understood the Web. Maybe my working style was a lot like yours: my team and I would work on content and design based on the understanding of a single fixed-width layout, usually designed in Photoshop. Designing in Photoshop was where we could exercise complete control, and I took solace in that. What we saw in the PSD was, for the most part, exactly what we’d execute with the front-end code. In this way, pixel perfection was a point of pride for us, and the idea that what we’d be designing would flex, resize and rearrange itself based on the viewport’s size was terrifying. How could we ensure quality? How could we retain control? But that level of control has always been a myth. We all experience this everyday as we work to accommodate varying browser support, font rendering, monitor sizes, screen color set-ups and connection speeds. Moreover, now that users are visiting from pocket-sized to television-sized screens, it’s time to accept that Web design is about so much more than creating desktop-friendly layouts. Ultimately, I realized that we must trade the control we thought we had in Photoshop for a new kind of control—using flexible grids, fluid images and media queries to build not a page, but a network of content that can be rearranged at any screen size to best convey the message. RESPONSIVE WEB DESIGN ISN’T BOLT-ON I’ve found that adapting approaches, workflows and strategies to get the most out of RWD doesn’t typically happen overnight.

BY TRENT WALTON | 93

Unlike easy progressive enhancements for rounded corners (border-radius) and Web fonts, RWD can’t be quickly added, tested and then removed from an existing website. Consider how easy it would be to open a website that you built a year ago and add border-radius to a button. RWD isn’t bolt-on in the same way. It’s not something that can be easily added to a nearly finished website because its flexible foundation requires us to rethink the very units of measure that we build on, from pixels to proportions—things such as ems (based on the current font size), rems (based on the root font size) and percentages. All the things we’ve become accustomed to designing in certain ways (such as calendars, checkout flows, tables and navigation patterns) can still exist, but they must be reconsidered to work optimally in a responsive setting. But before we get too far ahead of ourselves, let’s gain a fresh perspective by briefly looking at each of RWD’s core components.

Flexible Grids If there’s anything I’ve had to learn the hard way, it’s that this core component of RWD—flexible grids—extends beyond grid structure to type, vertical spacing, positioning and even decorative items such as border-radius and text-shadow. Establishing this flexible foundation, regardless of whether you plan to add media queries, is a great way to help future-proof a website, preparing it for proportional scaling. Think about it this way: take a button with a font size of 1 em and 20 pixels of padding. Then, use a media query to increase the font size to 1.5 em at widths greater than 600 pixels. You will have to declare a larger pixel value for the padding if you want the button to grow proportionally. If you had declared 1.250 em instead of 20 pixels

94 | RESPONSIVE DESIGN STRATEGY

FIGURE 3.1. Basic setup for two buttons.

for padding, then that proportional growth would have happened automatically with less code.2 Because I used so many pixel values on the first responsive website that I built, I had to go through each media query and update just about every property to adjust those pixel values to suit. The style sheet just looked wrong. After establishing that flexible foundation, I was able to go back and simply change the body’s font size and watch all of the em-unit-sized pieces of the website scale proportionally. This process facilitated a two-thirds reduction in code, all thanks to our dear friend, the flexible foundation. Think twice about using pixel values in your CSS. Now, on to the grid itself. I’ll assume you’ve already chosen your favorite calculator and have mastered Ethan’s handy-dandy formula:

2. A demo: smashed.by/grids-demo.

BY TRENT WALTON | 95

target ÷ context = result

If you need to find the proportional amount for four equally sized 230-pixel columns within a 1000-pixel container, you’d plug in the following: 230px ÷ 1000px = 0.23 or 23%

Great! So, we’ll set each of the four columns at 23%. Next up, let’s say we have 10-pixel margins on each side. 10px ÷ 1000px = 0.01 or 1%

We’ll plug in the 1% margin and watch it flex—all too easy. But what happens if the container (i.e. the context) is 1080 pixels? 230px ÷ 1080px = 0.21296296296296

OK, I know what you’re thinking, but leave those numbers intact. Resist the urge to trim, as unsightly as they may be. If your calculator gives you 0.21296296296296, don’t shorten the percentage to 21.3%. We want the truest proportional representation possible. Plus, computers are smarter at math than you are, and they know it, so don’t do them any favors. Note: Aside from setting margins to separate grid columns, you could also take advantage of box-sizing: border-box. This allows for the addition of padding and border without expanding the width of the box.3 OK, grids? Flexing. Next, let’s tackle images.

3. Irish, Paul. “{ box-sizing: border-box } FTW,” smashed.by/box-sizing-ftw.

96 | RESPONSIVE DESIGN STRATEGY

Flexible Images (and Media) Implementing flexible images is a breeze. img { width: 100%; }

Here, images are set to fill the width of their containers. But what happens if the image’s actual size is much smaller than the container itself and we are worried about it becoming horribly pixelated? This is where max-width becomes incredibly useful. img { max-width: 100%; }

With max-width, image sizes still fill the width of their containers, but they won’t render larger than their actual size. In other words, a 400-pixel image will scale perfectly to fill a 300-pixel container but will not suffer a resolution blowout if it continues to scale to a 500-pixel container, stopping at 400 pixels. PROTECTING IMAGE ASPECT RATIOS

What happens to an image whose width and height attributes are defined in the HTML when max-width: 100% is applied to them? As Bruce Lawson reminds us in his post on preserving image aspect ratio, 4 we must remember to set height: auto to ensure that images aren’t horrendously squeezed in only one direction at a time. That way we’ll be able to override that static height and enable all images to flex proportionally in all directions. 4. Lawson, Bruce. “Responsive Web Design: preserving images’ aspect ratio,”

smashed.by/asp-ratio. BY TRENT WALTON | 97

EXTENDING THINGS TO MEDIA The convention is to extend this to other forms of media elements by adding to our code to cover design elements such as video and Flash objects. img, embed, object, video { max-width: 100%; }

Great! All our bases are covered. With fluid images being so easy to implement, the rest must be just as easy, right? You’d think so, but gaining fluid-width video embedding isn’t always as automagical as we’d like. Thankfully, Dave Rupert and Chris Coyier have built a solution in the form of a jQuery plugin named FitVids.5 FitVids automates the intrinsic ratio method created by Thierry Koblentz6 to achieve fluid-width videos in a RWD. Essentially, Thierry suggests adding a “magic wrapper” (the parent) around the container for the media element (the child)—a container that proportionally resizes itself according to the width of its parent. This means creating a box with the proper ratio (4:3, 16:9, etc.) and then making the video inside that box stretch to FIGURE 3.2. Fitvids.js let you easily embed vidfit the dimensions of the eos with flexible width. box. With this technique, 5. FitVids.js, smashed.by/fitvids. 6. Koblentz, Thierry. “Creating Intrinsic Ratios for Video,” A List Apart, smashed.by/intrinsic-videos.

98 | RESPONSIVE DESIGN STRATEGY

browsers are able to determine a videoâ&#x20AC;&#x2122;s dimensions based on the width of its containing block. With intrinsic dimensions, a new width will trigger a new calculation of height, allowing videos to resize and scale the way images do. Here is the markup: <div class="wrapper-with-intrinsic-ratio"> <div class="element-to-stretch"></div> </div>

And here is the CSS magic: .wrapper-with-intrinsic-ratio { position: relative; padding-top: 25px; padding-bottom: 56.25%; height: 0; } .element-to-stretch { position: absolute; top: 0; left: 0; width: 100%; height: 100%; background: teal; }

To create a box with an intrinsic ratio, all we need to do is apply top or bottom padding to a div. Whatever the width of the viewport, the teal box should keep its aspect ratio (in this case, 16:9 = 0.5625, which corresponds to 56.25%). Ordinarily, you would also need to BY TRENT WALTON | 99

consider the height of the chrome and player controls (in this case, 25 pixels for the YouTube chrome). ABOUT IMAGE OPTIMIZATION Later chapters will dig into various responsive image techniques and proposals, but be sure that the images you start with are optimized as much as possible. ImageOptim7 has become an indispens-

able part of my workflow. It strips out unnecessary comments and color profiles from PNG, JPG and GIF images. Smush.it8 is a similar tool, but with a Web uploader interface. Another tip is to edit the actual images themselves. Jeremy Keith recently wrote a post9 about his optimization of the 2012 dConstruct website, for which he took the speakers’ photos and blurred the outer edges with a photo editor. Anything that wasn’t the focal point (i.e. the speaker’s face) was blurred. This reduced the number of jagged artifacts in the raster images, thus reducing overall file size by up to 85%. And because we tend to focus on human faces, the technique isn’t really noticeable. Brilliant! MOVING TOWARDS RESOLUTION INDEPENDENCE As easy as it was to gain fluid-width images, reevaluating how we’re

using those images will be important. Let me pose a few problems that could arise. First, using images instead of CSS to create buttons with effects such as borders, shadows and gradients makes it harder to resize and reshape the buttons. The need for this ability increases in a responsive setting because size and layout change from one viewport to another. Embedding text in images (such as JPEGs for advertisements) is always risky, but the negative effects are compounded on fluid-width 7. ImageOptim, smashed.by/imgopt. 8. Smush.it, smashed.by/smush-it. 9. Keith, Jeremy. “dContruct optimisation,” smashed.by/dconstruct.

100 | RESPONSIVE DESIGN STRATEGY

pages. The font size of text embedded in an image will scale with the container’s size. Without proper planning and care, the text could become illegibly small or blurry as the image resizes. With the introduction of high-pixel-density displays (such as Apple’s Retina), the more we can use resolution-independent techniques, the more future-proof our websites will be. Consider icons. If we’re using CSS sprites, we could create multiple versions of a sprite and load an actual-sized or a doublesized version to ensure crisp rendering; but creating multiple-sized assets isn’t as future-proof as seeking true resolution-independence through either SVG sprites or icon fonts. I’ve recently taken to using icon fonts whenever possible. They’re the most easily scalable solution because we’re using only font-size instead of background-position and backgroundsize, which are necessary for sprites. We can change the color on the fly (although fonts are limited to a single color), and they’ll look crisp regardless of the pixel density of the display. Implementing icon fonts is relatively easy. Use a service such as Shifticons10 or Pictos11 or roll your own. Chris Coyier has a nice write-up on icon font usage12 that he’s been updating regularly as best practices are hashed out. Chris maps icons to Unicode symbols and includes the aria-hidden attribute as an accessibility upgrade. Here’s what an implementation might look like if we wanted to add an icon before a link to the weather section of a website: <a href="/"><i aria-hidden="true" data-icon="&#x2601;"></ i><span class="screen-reader-text">Weather</span></a>

10. Shifticons, smashed.by/shifticons. 11. Pictos, smashed.by/pictos. 12. Coyier, Chris. “HTML for Icon Font Usage,” smashed.by/font-usage.

BY TRENT WALTON | 101

And the CSS would be this: @font-face { font-family: 'Icon-Font-Name'; src: url('icon-font.eot'); src: local('☺'), url('icon-font.woff') format('woff'), url('icon-fontweb.ttf') format('truetype'), url('icon-font.svg#webfontIyfZbseF') format('svg'); } [data-icon]:before { font-family: 'Icon-Font-Name'; content: attr(data-icon); }

One other icon font service I’m giddy about is Symbolset.13 In addition to using Unicode and CSS classes, Symbolset also lets designers apply icons to plain-text keywords, replacing them with matching icons. For example, one could replace the word “cart” with an actual shopping cart icon. This method is particularly helpful when an icon will appear without accompanying text, which is fairly common on responsive websites that constrict to a narrow layout and on which space is at a premium.

Media Queries Sooner or later, fluid-layout break columns will become too cramped, navigation items will wrap in an unsightly manner, and paragraphs will become difficult to read. We can use media queries to repair broken layouts by having them target device or browser

13. Symbolset, smashed.by/symset.

102 | RESPONSIVE DESIGN STRATEGY

properties. Media queries have two parts: the media type and the query. Let’s look at an example. @media screen and (min-width: 1000px) { body { font-size: 125%; } }

The screen type is the one we are most familiar with. But we could also target other output media, such as print, projection and tv. Their support is very limited (so far), but keep them in mind nevertheless. For a complete current list, refer to the W3C’s CSS2.1 specification.14 The min-width is the query itself. Width- and height-based queries are common, as are device-width and device-height. For a complete current list of queries and information on whether they support minimum and maximum values, refer to the W3C’s media queries specification.15 With media queries, not only are we able to repair layouts when they break at a given view, but we can harness their power to ensure ideal experiences across the ever-increasing array of device types and screen sizes. Take a basic two-column layout. When the flexible grid scales down to a narrower view—let’s say 600 pixels wide—the columns would get too cramped. We could use media queries to stack the columns on top of each other.16

14. CSS 2.1: Media Types, “Recognized media types,” smashed.by/media-types. 15. CSS3: “Media Queries,” smashed.by/media-queries. 16. Demo: smashed.by/stack-demo.

BY TRENT WALTON | 103

FIGURE 3.3. Maintaining an ideal experience for users on small devices.

HTML: <div id="container"> <div class="row"> <div class="col one"> <p>Nullam quis risus eget urna mollis ornare vel eu leo. Donec id elit non mi porta gravida at eget metus. Curabitur blandit tempus porttitor. </p> </div> <div class="col two"> <p>ullam quis risus eget urna mollis ornare vel

104 | RESPONSIVE DESIGN STRATEGY

eu leo. Donec id elit non mi porta gravida at eget metus. Curabitur blandit tempus porttitor. </p> </div> </div> </div>

CSS: .col { float: left; margin: 2%; width: 46%; } @media screen and (max-width: 600px) { .col { float: none; margin: 0; width: 100%; } }

Success! With this example, we’ve altered the architecture of the page, maintaining an ideal experience for users on small devices and narrow viewports. ESTABLISHING BREAKPOINTS Most importantly, set media query breakpoints based on the content and not any particular device’s screen size. While 480 and 768 pixel breakpoints are common today, they aren’t universal, and they become less and less ubiquitous with each new tablet or smartphone that hits the market. Simply add a breakpoint when

BY TRENT WALTON | 105

the layout breaks. Use media queries to prevent navigation items from wrapping awkwardly, columns from cramping and text from becoming difficult to read at any given viewport size. With all of these possibilities and problems to solve, managing media queries can quickly become unwieldy. Because of this, I try to consolidate media queries around a few key breakpoints. For example, If I realize that I added breakpoints to accommodate content at 650 pixels and again at 670 pixels, then I might seek to unify those changes under just one. Sure, we ought to be willing to extend things beyond that, but consolidating will help everyone—from copywriters to designers to developers—to maintain and communicate about the website. As you can imagine, the more media queries and breakpoints you add, the more difficult keeping track of them will be. Even when a flexible foundation is firmly in place, the CSS code can pile up. While there are infinite ways to wrangle things, below are a few that have worked for me.

Top Down Regardless of what view I design first, I always structure my CSS for mobile first by coding for the narrowest view first, and then adding in min-width media queries to build towards a full-width desktop view. For this, the simplest method, everything should live on one style sheet. Another benefit to ordering things in this way is that it reinforces a mobile-first strategy any time the website undergoes maintenance or gets a new feature. This limitation of space can serve as an editor of sorts, continually honing a website’s objectives and message. Module-Based Organization For large websites, having everything in one file can make it difficult to navigate the code as well as to keep your head wrapped 106 | RESPONSIVE DESIGN STRATEGY

around which media queries affect the layout at which times. In situations such as these, I’ll first establish CSS for the basic layout and typography. Then, I’ll import style sheets for each part of the page that I plan to modularize. For example, a news website could have sheets like headlines.css, weather.css, newsfeed.css, menu.css and ads.css.

Breakpoint-Based Organization Another option is to import style sheets for each key breakpoint. This can help designers mentally separate key shifts in the layout that occur at various screen sizes. It would look something like style. css, 540.css, 720.css, 1000.css and so on. I prefer starting with (and importing) narrow one-column views as the base and building up from there. Of course, when a website goes into production, it’s not uncommon for all of the files to be merged into one and minified. Media Query Splitting Another approach, outlined by Simurai,17 is to set some breakpoints 1 pixel apart and to split the styles instead of overriding from one media query to the next. This would alleviate the need to reset something like box-shadow: 0 1px 3px #333 to box-shadow: none when one is set in an alternate media query. To my mind, this technique is probably most useful in conjunction with a mobile-first, minimumwidth approach, but it would surely ease some of the mental strain from tracking media queries during development. There is an infinite number of ways to go about organizing media queries, so don’t feel like you need to lock into a single one. To complement a good system of organization, there are some useful tools to keep track of which media queries are triggered and when:

17. “Media Query splitting,” smashed.by/mq-splitting.

BY TRENT WALTON | 107

Ŧ Responsive.is (smashed.by/inbrowser)

Resizes a website in-browser to typical preset device sizes and orientations. Ŧ Responsivepx (smashed.by/respx) Remy Sharp’s tool helps you pinpoint pixel values for when a responsive layout breaks. Ŧ Aptus (smashed.by/aptus) A Mac app that acts as a dedicated browser for testing a website at any size. My favorite part is that it stores custom breakpoints so that you can quickly preview and diagnose layout issues in one click. VERTICAL MEDIA QUERIES If you’re like me, most of your media query attention centers on width. But what can be done with height-based media queries? I’ve recently found a few uses. One of my favorite things to do with media queries is to increase the font size in larger views. One problem I’d run into, however, was that I didn’t want short but wide screens to trigger the largest of font sizes, because then everything would just look awkward. I used a min-height media query to solve that: @media screen and (min-width: 1000px) and (min-height: 700px) { body { font-size: 137.5%; } }

With this, only views wider than 1000 pixels and taller than 700 pixels would get the extra-large type. Another helpful use of this I’ve found is to manage the fold. I’m not trying to incite a debate on 108 | RESPONSIVE DESIGN STRATEGY

whether or not the fold exists, but let’s suppose you’d like to ensure that certain important pieces of content are visible upon initial loading without any scrolling. You could use vertical media queries to reduce vertical padding or margins or even to crop images on the page.

RELATIVE UNITS FOR MEDIA QUERIES Up to this point, we’ve been using pixel values for our media queries. I do this regularly when building out a website, but recently I’ve been completing the pixel-value purge and using em units for media queries as well. As Lyza Gardner of Cloud Four points out,18 this can help facilitate proportional scaling, which is especially handy when users zoom in the browser. To do this, we’d simply calculate the em-unit value based on the baseline body font size, which is 100% or 1 em. Suppose we start from a baseline of 16 pixels, and we have the media query @media all and (min-width: 400px) in our CSS. To calculate the appropriate em values, we would just calculate 400 pixels ÷ 16 pixels = 25, resulting in @media all and (min-width: 25em). The difference is that if users zoom in (i.e. make the text larger in their browser), then they would no longer be using the 16 pixels baseline that we used for pixel-based media queries. With the em-based media query, we’re better equipped for the whatever baseline they define in their browser. My favorite tool for quick calculations is PXtoEM.19 MEDIA QUERY SUPPORT Browser support for media queries is great until you get to IE 8 and below. But never fear. We’ve got a few options for handling this, which I choose from depending on a project’s scope and requirements. 18. Gardner, Lyza. “The EMs have it: Proportional Media Queries FTW!,” Cloud Four,

smashed.by/queries-ftw. 19. PXtoEM, smashed.by/pxtoem. BY TRENT WALTON | 109

Scott Jehl’s Respond.js20 was built as part of the Filament Group’s work for the Boston Globe. The group has kindly placed it on GitHub as a polyfill for min-width and max-width media queries. Another option is to structure CSS mobile first, using min-width media queries to re-architect the layout from single-column narrow views up to the full-width desktop. IE 6 through 8 will get that mobile single-column narrow view, but with your amazing design chops, I’m sure that’s nothing to frown about. A third option is to build fixed-width layouts for older browsers, with targeted LT (i.e. less than) IE 9 style sheets that mimic the layout at full width (or some arbitrary container width, such as 960 pixels). I like this option because it helps separate the layouts for older browsers from all of the progressive enhancement we do for the main version. Because so many of the newer CSS properties we enjoy are supported only by browsers that also support media queries, we can push things further with the main style sheet.

Getting to Work We’ve broken down each of the three core responsive components and looked at tips for each of them. Now that the dots have been connected, let’s dive into examining how teams can begin to work together to build their next website responsively. DECOMPARTMENTALIZE THE WORKFLOW In the production model that I was used to, a project was passed down the assembly line from content to design to development to launch. While it might not have been the best approach, it worked because each step was a prerequisite for the next. The content would be set; the designers would lay things out; and then the developers would code according to the image mockups. 20. Respond.js, smashed.by/github-respond.

110 | RESPONSIVE DESIGN STRATEGY

With RWD, we’re designing that aforementioned network of content, rather than a page, so I now favor an approach with rapid iteration and a lot of show and tell. This requires all team members to work even harder to foster healthy collaborative relationships. If you’re a designer but don’t yet know how to code, I’d recommend learning how to use basic browser development tools, such as WebKit’s Web inspector. Requesting bumps in pixel values in the margin or padding will mean less to those who code in percentages and em units. If you’re a developer working with designers, make sure they get easy access to view the website while in production. With RWD, screenshots won’t fully convey the progress that’s been made or make it easy for collaborators to understand where things need to be improved. RWD will further blur the line between the Web designer and developer’s roles, so be willing to expand your skill set by taking advantage of these collaborative working situations. BUILDING THAT NETWORK OF CONTENT Now that your workflow is in order and everybody loves each other, what’s next? One of the biggest challenges for a team new to RWD (and we’re all new) is discussing the content and hierarchy for a layout that has become a moving target. It’s easy to revert to thinking about content arrayed across a full-width desktop view and to forget that space will come at a premium in alternate views. Because of this, I like to start with low-fidelity mockups. These could be paper sketches, gray-box wireframes, anything you want. The trick is to keep them simple, because at this point it’s less about how things look and more about providing visuals to generate a discussion about how content should fit in the space at any given width and how to maintain the hierarchy.

BY TRENT WALTON | 111

There’s no time like the present to abandon static ways of thinking about how layout and content interact. Test ideas by quickly sketching alternate views, trying to poke holes in the plan. After all, what might seem like a great idea for a full-width layout might not make sense at narrower views. This will also help the team pinpoint where the layout might run out of space sooner than later. Perhaps the content needs to be edited down. Maybe the game plan needs to be rethought. This is one of my favorite parts of the process. It’s laid back and collaborative, so invention is bound to happen. DESIGNING RESPONSIVELY

Once the team has all of its best guesses for Plan A down on paper, it’s time to start designing. Similar to the hierarchy sketching stage, picking any view and designing it is OK. I like to start with the full-width desktop view and then, before getting too far, gather the team for a review. As mentioned earlier, I still favor a mobile-first approach from a content and coding perspective; I’ve just found that my team likes to start at full size during the design phase. If we find that we’re designing ourselves into a responsive corner, we’ll quickly sketch out how things could shift and reflow at various widths. It’s also not uncommon for coding to begin at this point. If there’s something we’re unsure we’ll be able to develop, then we take to websites such as JS Bin21 and CodePen22 to experiment with potential solutions. For some designers, Photoshop may not be necessary for anything other than asset creation (logos, textures, etc.). Designing in the browser is both a viable and a smart option, especially in a responsive scenario. I’ve recently noticed that we rarely wait until Photoshop comps are completed before getting into the code. Because any one comp can capture only a sliver of the vast array of 21. JS Bin, smashed.by/jsbin. 22. CodePen, smashed.by/code-pen.

112 | RESPONSIVE DESIGN STRATEGY

what users will experience at various screen sizes, the only true way to evaluate a responsive design is in the browser. Because of that, Samantha Warren has created Style Tiles,23 which can help teams collaborate on a design without committing to a series of fully fleshed out Photoshop mockups.

FIGURE 3.5. Style Tiles, a new design process for responsive websites,

smashed.by/style-tiles.

Any time I have presented a handful of breakpoint-based comps for a page, I find myself fielding a series of “What if” questions that wouldn’t have been asked had the design been presented in the browser. Style Tiles could be used to get approval on core design elements, before the process of developing a fully formed layout begins. Those elements can then inform the process once approval has been gained. This is why gradually transitioning the team to a more browsercentric workflow can pay off in the long run. Full disclosure: I still like my Photoshop comps for quick ideas and brainstorming because I find myself in such a different mental mode in an image editor than in a code editor. However, once ideas take shape, I transition to code

23. Style Tiles, smashed.by/style-tiles.

BY TRENT WALTON | 113

as soon as possible to really kick the tires. I also find that saving all of the precise nudging and typesetting for code makes the most sense. After all, you’re not gaining any ground by perfecting a JPG layout when it’s going to wind up in code anyway. Save all the blood, sweat and tears for the browser.

Tending to the Network As we further develop the HTML and CSS for our responsive project, a few things can be done to help preserve the hierarchy and keep this painstakingly developed network of content intact.

FIGURE 3.6. An introduction to content choreography on Trent Walton’s

blog: smashed.by/cont-chor.

CONTENT CHOREOGRAPHY Earlier, we used media queries to re-stack elements on the page when columns became cramped in narrow views. Taking things further, we shouldn’t just consider the space available; we must also consider the content. Imagine a four-column website at full width: as the view narrows, four will become three, then two, then one. A common solution is to stack them on top of each other in chunks. 114 | RESPONSIVE DESIGN STRATEGY

Now, there’s a good chance this rearrangement is consistent with the content hierarchy on the page, but what happens if the first column is really tall? Is the content in column two less important than all of the content in column one? In some cases it might be nice to interdigitate content by folding elements into each other as the viewport narrows. On the next page are a few techniques we can use to better choreograph how content flows from one view to the next. No, these aren’t names for 1950s dance moves; they’re just horribly coined terms from yours truly. Hey, what do you want from me? Imagine this for that local news outlet’s website I referred to at the beginning, with the forecast (including pollen levels) in the top right. In a narrower view, we’ll want the forecast to be directly below the headline graphic, instead of being shuffled down to the bottom of the first column.

FIGURE 3.7. Often elements are stacked

on top of each other in smaller views.

FIGURE 3.8. Sometimes it might be more

useful to interdigitate content by folding elements into each other as the viewport narrows.

FIGURE 3.9. For example, the “weather”

section could be placed under the main headlines rathen than pushed to the bottom of the page.

BY TRENT WALTON | 115

The Bunch and Slide One way to handle this is by grouping columns in the markup in a particular way. We could create a row for content that we want to bunch together (in this case, the headlines and local weather). Then, before continuing with the rest of the content, weâ&#x20AC;&#x2122;d simply close the row and start with yet another container row for the rest of the content down the page.24

FIGURE 3.10. In this case headlines and weather are bunched together in

one row, while the news container fills out the rest of the page.

24. Demo: smashed.by/bunch-slide.

116 | RESPONSIVE DESIGN STRATEGY

<div id="container"> <div class="row"> <div class="headlines col"> <p>Insert headline code here.</p> </div> <div class="weather col"> <p>Insert local weather here.</p> </div> </div> <div class="row"> <div class="news col"> <p>Insert additional news stories here.</p> </div> </div> </div>

And the CSS: .col { float: left; display: inline; } .headlines { width: 70%; } .weather { width: 30%; } .news { width: 70%; }

BY TRENT WALTON | 117

@media screen and (max-width: 33.750em) { .headlines { width: 100%; } .weather { width: 100%; } .news { width: 100%; } }

The Slide and Float Another option is to use media queries to manage floats. What if our objective were to place local weather above the headline graphic in one-column views because some twerp kept filling out a feedback form complaining about not being able to find pollen levels? First, we’d reorder the HTML to reflect our new change in hierarchy.25

FIGURE 3.11. We could place the “weather”

section above the headlines as well.

<div id="container"> <div class="row"> <div class="weather col"> <p>Insert local weather here.</p>

25. Demo: smashed.by/hierarchy.

118 | RESPONSIVE DESIGN STRATEGY

</div> <div class="headlines col"> <p>Insert headline code here.</p> </div> </div> <div class="row"> <div class="news col"> <p>Insert additional news stories here.</p> </div> </div> </div>

Then, the CSS could look something like this: .col { float: left; display: inline; } .headlines { width: 70%; } .weather { width: 30%; float: right; } .news { width: 70%; }

BY TRENT WALTON | 119

@media screen and (max-width: 33.750em) { .headlines { width: 100%; } .weather { width: 100%; } .news { width: 100%; } }

VoilĂ ! For any width greater than 540 pixels (33.750 em), the weather would be floated to the right side of the page, adjacent to the headlines.

The Squishy Flexbox Whereas our first two dance moves techniques allow for minimal content choreography, the sky is the limit with the flexible box layout. Letâ&#x20AC;&#x2122;s revisit the initial markup:26

FIGURE 3.12. In this case headlines and weather are bunched together in one row,

while the news container fills out the rest of the page. 26. Demo: smashed.by/squishy-flx.

120 | RESPONSIVE DESIGN STRATEGY

<div id="container"> <div class="headlines col"> <p>Headlines</p> </div> <div class="weather col"> <p>Weather</p> </div> <div class="news col"> <p>News</p> </div> </div>

Then, in the CSS, we could try this: container { display: -webkit-flex; display: flex; } .col { flex: 1; } .headlines { order: 1; } .weather { order: 2; } .news { order: 3; }

BY TRENT WALTON | 121

@media screen and (max-width: 33.750em) { #container { flex-flow: column wrap; } .headlines { order: 2; } .weather { order: 1; } .news { order: 3; } }

You can rearrange the stacking order of elements on the page by changing the order.27 RESPONSIVE IMAGE HIERARCHY Another important aspect of maintaining hierarchy is physical space. Simply put, the larger something is on the page, the more important we perceive it to be. Letâ&#x20AC;&#x2122;s take this page as an example:

FIGURE 3.13. A layout with a large and 3

column images on wide views.

27. The flexbox specification is still being refined and has recently undergone a massive

overhaul, so implement with care. To learn more, check out the specification at: smashed.by/flexbox. Also, Chris Coyier has a rather nice side-by-side demo comparing the old and new specification: smashed.by/new-flexbox. 122 | RESPONSIVE DESIGN STRATEGY

Obviously, the large panoramic banner is important. It’s higher up on the page and larger than the three images below. With fluid images, what will happen to this layout as it is displayed at narrower views? Perhaps the layout displayed on the right? No way! What happened to our visual hierarchy here? That banner graphic was the largest thing on the page, appropriately sized to suit the importance of the content relative to the rest of the page. In this narrower view, things are out of FIGURE 3.14. A whack and all we see is a tiny strip of image, much layout with a large smaller than the rest of the images on the page. and 3 column images on narrow What can be done to resolve this? Once again views. with the flexible media save is my coworker Dave Rupert.28 We can wrap the panoramic element in a div. Let’s call it .banner-item.29 Then, we’ll use min-height, height and overflow: hidden to keep things proportional. Here’s what the code could look like: .banner-item { overflow: hidden; min-height: 300px; /* 300px is arbitrary. */ } .banner-item img { width: 100%; } /* 600px / 37.500em is about the width "locked in" at */ @media screen and (max-width: 37.500em) { .banner-item img { 28. Rupert, Dave. “Responsive Image Hierarchy,” smashed.by/image-hierarchy. 29. Demo: smashed.by/flexmedia.

BY TRENT WALTON | 123

FIGURE 3.15. A technique that wraps the panoramic element in a div and uses min-height, height and overflow: hidden to keep things proportional.

width: auto; max-width: none; height: 300px; } }

With this media query, we’ve stopped the reduction in height at 300 pixels, maintaining proportional hierarchy. Now that we’ve explored so many ways to approach flexible grids and layouts, let’s look at how this affects text and readability. 124 | RESPONSIVE DESIGN STRATEGY

FLUID TYPOGRAPHY

Amidst all of this, we want to ensure readability across all devices and widths. A key part of this is managing characters per line. In a fluid layout, browser width and typographic measure are linked: the wider the viewport, the more characters per line. The Elements of Typographic Style Applied to the Web30 tells us that 45 to 75 characters per line (including spaces) is generally accepted as safe for comfortable reading. Bearing that in mind, we can do a few things to avoid extra-long lines of text in fluid layouts. First, we could use media queries to adjust the width of the container. As the viewport gets wider, we could decrease the container’s width to give lines of text less distance to span across. Secondly, we could increase the font size in wider views. A trick I like is to place two asterisks in some placeholder text: Lorem ipsum dolor sit amet, consectetur adip*isicing elit, sed do eiusmod *tempor incididunt ut labore et dolore magna aliqua.

As I widen the browser window, if at any point the two asterisks appear on the same line of text, then I know it’s time to increase the font size. These adjustments are easy (usually one CSS property) because I’ve used percentage, em and rem (rather than pixel) units in the CSS.31 To my delight, Josh Brewer and Mark Christian released a jQuery plugin named Responsive Measure,32 which helps to automate this by generating the font size needed to produce the ideal measure for text at a given width.

30. The Elements of Typographic Style Applied to the Web, “Choose a comfortable measure,” smashed.by/comf-measure. 31. This technique is a small part of a post I wrote titled “Fluid Type:” smashed.by/fluid-type. 32. Responsive Measure, smashed.by/resp-measure.

BY TRENT WALTON | 125

This is great for paragraph text, but what if we want the text to behave more like a fluid image, scaling based on the width of the browser window, for more intensely typeset headlines? I had this problem on my own blog, and, fortunately for me, Iâ&#x20AC;&#x2122;m friends with Dave Rupert. He built a jQuery plugin, FitText, to inflate Web type, and we released it on GitHub.33 Sean McBride has a great example of FitText in use.34 There, Sean has applied FitText to an h1 heading whose width he has governed via padding: 5em 40% 2.5em 5%. The 40% padding on the right leaves room for the focal point of the photograph. The text scales to fill the width of the container with the addition of the following script in the head:

FIGURE 3.16. Increasing font size is easy because percentage, em and rem

units have been used here.

33. FitText, smashed.by/fittext. 34. Demo: FitText, smashed.by/fittext-demo.

126 | RESPONSIVE DESIGN STRATEGY

Actually, if you like FitText, then you’ll love Molten Leading.35 Dreamed up by Tim Brown and built by Mat Marquis, Molten Leading automatically adjusts line height to a minimum and maximum range within a minimum and maximum pixel value width range. In other words, it gives us a fluid way to set line height. Even better news is that, one day, things like FitText could be made obsolete by vw, vh and vmin units of measure. It’s pretty simple: Ŧ 1 vw is 1% of the viewport’s width; Ŧ 1 vh is 1% of the viewport’s height; Ŧ vmin is 1 vw or vh, whichever is smaller.

Instead of using JavaScript and the percentage-based widths that FitText requires, these units would do the job all on their own. These units could also be used to more precisely govern typographic measure across all viewport widths once they gain more browser support. CODE-BASED WEB DESIGN STANDARDS As basic design elements such as typography, buttons, navigation and general page structure become finalized, it’s worth considering building a page or an asset library website. This would help get things ready for production, but it could also help establish visual standards and assets for a project.36 In my mind, I picture this as a light version of the fantastically comprehensive Twitter Bootstrap,37 but unique to your project. Having design examples in a live code setting is the most accurate way to test and continually improve a website. The way I see it, using images to show a responsive layout is like test driving a car that’s stuck in park. 35. Molten Leading, smashed.by/molten-leading. 36. You can review some examples of front-end style guides in a Gimme Bar collection:

smashed.by/frontendguides. 37. Twitter Bootstrap, smashed.by/tw-bootstrap. BY TRENT WALTON | 127

Onward! Responsive Web design has restored my sense of wonder for our craft. We’ve reached a new frontier, and the uncharted territory ahead is gnarly. With a spirit of bold adventure, let’s press on. How will online banking, extensive drill-down menus and content management systems look in a responsive setting? Brad Frost, the responsive Web’s faithful docent, will take over in the next chapter, helping us to begin answering some of these questions.

ABOUT THE AUTHOR Trent Walton is one of the founders of paravelinc.com, a small Web shop based out of the Texas Hill Country, where the chicken fried steaks are as big as your face. Together with Dave Rupert and Reagan Ray he is designing and building for the Web since 2002. The most important lesson he has learned in his career is that there is no substitute for being in control of your own destiny.

128 | RESPONSIVE DESIGN STRATEGY

CHAPTER FOUR

· BY BRAD FROST

RESPONSIVE DESIGN PATTERNS

S RESPONSIVE DESIGN CONTINUES TO EVOLVE, we’re confronted with difficult problems about how to create adaptive interfaces that look and function beautifully across many screen sizes and environments. How do we handle navigation that’s four levels deep? What about tables with a ton of columns? How do we deal with carousels? The exciting news is that the community is discovering ways to create responsive layouts, adapt existing design conventions to a multi-screen world, and generate entirely new interaction methods. It’s a great and challenging time to be a Web designer as we dive head first into this multi-device future.

Modularity and Patterns and Style Guides, Oh My! It’s literally impossible to articulate in Photoshop all of the different design outcomes that arise from different device environments, screen sizes and user settings and a host of other variables. So, as Trent mentioned in the last chapter, we must address this by moving away from designing Web pages and toward designing systems. BY BRAD FROST

129

Just as all matter consists of small molecules, which consist of even smaller atoms, Web interfaces are made up of many small components. This means we can break the entire design system down into more modular, more manageable chunks. In doing so, we define the elements of our interfaces, figure out how each element works in a responsive environment, and then build the final system by assembling these units. This type of atomic thinking is certainly different than the way a lot of us are used to crafting Web experiences. That’s where frontend style guides can be a tremendous help. A front-end style guide is a collection of your project’s interface components. In her article “Front-End Style Guides,”1 Anna Debenham explains the many benefits of a front-end style guide: it makes testing easier, establishes a better workflow, offers a shared vocabulary among team members, and serves as a useful reference to keep coming back to. In my expe-

FIGURE 4.1. Anna Debenham explains the many benefits of a front-end style

guide on 24 Ways. 1. Debenham, Anna. “Front-end Style Guides,” smashed.by/front-end-guides.

130 | RESPONSIVE DESIGN PATTERNS

rience, the guide becomes a sort of glue for a project’s team members, communicating to everyone involved how the design system works. With the rise of responsive Web design, style guides are becoming more important than ever. What better way to show how an element will adapt than to show it in its final environment? Thankfully for us, a lot of great work has been done recently in the area of responsive style guides. Starbucks2 graciously published its responsive pattern library, which it used to guide its responsive redesign. Samantha Warren created Style Tiles,3 a design deliverable that captures a project’s fonts, colors and interface elements. Dan Cederholm created Pears,4 an open-source framework for developing one’s own pattern library. And, as it happens, I’ve created an opensource library of responsive patterns.5

Picking Things Apart We need to look at our designs in a more modular way. So, let’s get our hands dirty, pick apart Web interfaces and look at some emerging responsive patterns! We’ll look at some common ways to handle layout, navigation, interactive elements and more. By examining these emerging patterns, we can see what options are already in the wild and use them as a starting point to evolve and innovate. Remember, these techniques are all relatively new, so don’t treat them as gospel, but rather as great starting points for your particular project. There are plenty of opportunities to improve existing patterns and perhaps even create entirely new techniques. These are exciting times, indeed!

2. Starbucks Style Guide, smashed.by/starbucks-styleguide. 3. Style Tiles, smashed.by/style-tiles. 4. Pears, smashed.by/pears. 5. Responsive Design Patterns, smashed.by/rwd-patterns.

BY BRAD FROST

131

Page Layout A logical place to start is with layout, because it is certainly the most affected by the core tenets of responsive design. Historically, we’ve created multi-column designs without batting an eye, but now layouts need to work harder than ever to ensure that users have a legible, functional experience, no matter how they access the Web. So, what do we do with all of these columns and modules? How should they reflow in a responsive environment? Luke Wroblewski answered these questions when he wrote about the various flavors of responsive layout patterns6 in the wild, after having combed through Mediaqueri.es,7 a responsive gallery website. Let’s take a close look at the patterns he discovered in his research. MOSTLY FLUID In this pattern, the layout relies mostly on a fluid grid for most of the adaptation; it stays roughly the same on medium and large

FIGURE 4.2. The mostly fluid layout remains consistent until it appears

on a small screen. See a demo at smashed.by/mostlyfluid. 6. Wroblewski, Luke. “Multi-Device Layout Patterns,” smashed.by/md-patterns. 7. MediaQueri.es, smashed.by/mq-showcase.

132 | RESPONSIVE DESIGN PATTERNS

screens and only dramatically shifts on small screens. This mostly fluid pattern is popular for many responsive websites because it requires maintaining only two major design views, instead of more complex patterns such as the column drop or layout shifter described next. COLUMN DROP The column drop is a popular technique for three-column designs. If the viewport is too small to display three columns side by side, one of the sidebar columns drops beneath the other two columns. The columns eventually all stack above one another on small screens. This pattern can be effective if the content contained in the three columns isnâ&#x20AC;&#x2122;t related, but content hierarchy can become difficult to manage if the content chunks in the columns depend on one another.

FIGURE 4.3. A demo for the column drop layout pattern is available

at smashed.by/col-drop.

BY BRAD FROST

133

LAYOUT SHIFTER

This relatively complex pattern involves creating distinct layouts at each breakpoint. As Luke states in his article:

“

Because this inherently requires more work, it seems to be less popular than the previous two patterns I outlined.

Media queries are certainly a lot of fun, but take care to avoid overcomplicating layouts just for the sake of declaring “Look, ma! It’s responsive!” Once you introduce big changes at every major responsive breakpoint, you are basically creating different states in your layout that might need to be addressed separately and that, thus, could increase complexity. Remember that the goal of responsive design is to create legible, beautiful and functional experiences across as many environments as possible. Sometimes that involves 200 media queries, while other times it involves just 2.

FIGURE 4.4. The layout shifter pattern creates a distinct layout at

each breakpoint. See the demo at smashed.by/layout-shifter.

TINY TWEAKS Speaking of minimalist approaches, the tiny tweaks pattern is as simple as it sounds. Especially suitable for single-column layouts, tiny tweaks maintains its basic structure while subtly adjusting 134 | RESPONSIVE DESIGN PATTERNS

type size, images and other components where appropriate. This pattern is useful for pages that contain only a single piece of content, such as articles, and for one-page linear websites.

FIGURE 4.5. The Tiny Tweaks pattern subtly adjusts type, images and other

components in an otherwise consistent layout: smashed.by/tiny-tweaks.

OFF-CANVAS Luke lays out one of the fundamental problems with the previous patterns:

“

While there’s a lot of variety in the responsive design layout patterns listed above, there’s also some common characteristics. They all tend to stack everything vertically on small screens resulting in long pages full of diverse components.

FIGURE 4.6. Off-canvas layouts move supplementary content off screen, from

where it can be brought in as requested: smashed.by/off-canvas.

BY BRAD FROST

135

Stacking content on top of each other isn’t necessarily a bad thing, but mobile users are typically used to scrolling through a singular content type, such as a social feed, email feed or group of contacts, or diving deep into an article. Introducing multiple content types further down the page requires users to go on a scavenger hunt to find them, and it could lead to frustration and/or confusion when users stumble upon this new, potentially unrelated content. What are the alternatives? The off-canvas pattern8 is a clever solution that moves a page component just off screen and, when requested, exposes the content using a sliding-door effect. This technique was popularized by Facebook’s mobile fly-out navigation.9 The off-canvas approach reserves the visible screen area for core content, while keeping additional content off to the side for easy access. Be sure to check out Jason Weavers and Luke Wroblewski’s excellent off-canvas demos (see the footnote above).

FIGURE 4.7. Facebook’s mobile navigation popularized the off-canvas technique.

8. Wroblewski, Luke. “Off Canvas Multi-Device Layouts,” smashed.by/off-canvas-layouts. 9. Responsive Navigation Patterns, smashed.by/nav-patterns.

136 | RESPONSIVE DESIGN PATTERNS

CONTENT CHOREOGRAPHY

As we just saw with the off-canvas pattern, there’s more to responsive layouts than simply stacking everything on top of each other for small screens. Be sure to check out the techniques that Trent lays out in the previous chapter for thoughtfully reflowing the layout while maintaining a logical content hierarchy. LAYOUT CONSIDERATIONS Whatever approach you decide to take for your responsive layout, focus on delivering a legible, logical experience to your users. Some things to think about: Ŧ

Ensure that content flows logically. As mentioned, users don’t want to sift through a bunch of disparate content to find what they’re looking for.

Treat layout as an enhancement. 10 In his talk “Rethinking the Mobile Web,” Bryan Rieger eloquently stated that, “The absence of support for @media queries is in fact the first @media query.” By constructing layouts mobile-first, we’re able to serve optimized layouts to more mobile platforms, which are less likely to support media queries. This approach also embraces the cascading nature of CSS and results in smaller, more legible, more maintainable CSS.

Let the content, not device dimensions, determine breakpoints. Way too many devices and screen sizes are out there, and new ones pop up every day. Instead of chasing down the screen sizes du jour, establish breakpoints where the design necessitates them, such as when line length becomes uncomfortably long, images become

10. Rieger, Bryan. “Rethinking the Mobile Web,” smashed.by/mobile-web.

BY BRAD FROST

137

unruly or interface elements looked stretched and unnatural. Follow Stephan Hay’s advice: “Start with a fluid layout, start small and expand until the layout looks like shit. Time to insert a breakpoint!” Ŧ

Don’t go overboard. While dramatically changing a layout across screen sizes is certainly fun, don’t feel obligated to insert breakpoints and radical changes just for the sake of being responsive. Instead, do whatever’s required in order to create a legible layout that’s a joy to interact with.

Navigation Navigation is sometimes the most important interaction in a Web experience. I’ve always said that navigation should be like a good friend: there when you need it, just like a friend who’s willing to help you move into a new home. But it should also be considerate enough to give you your space, just as a good friend doesn’t demand your constant attention. The community has been creating some clever solutions for dealing with responsive navigation.11 Let’s look at some of the more popular techniques. TOP NAVIGATION, OR “DO NOTHING” For navigation that contains

only a few links, keeping it at the top of the page might make sense. It’s certainly easy to implement, but it’s not particularly scalable and could

FIGURE 4.8. Top navigation simply

displays links across the top of the page. Demo: smashed.by/topnav.

11. I’ve been researching and collecting them in my very own blog post on Responsive Navigation Patterns smashed.by/nav-patterns.

138 | RESPONSIVE DESIGN PATTERNS

potentially take up a lot of valuable real estate. Remember that we want navigation to be accessible but not take up a lot of space, in order to make room for the core content. THE FOOTER ANCHOR

This clever solution places the navigation in the footer, leaving a simple anchor link in the header that lets users jump to the footer. This clears up room for the core content while still providing quick access to the navigation. It is relatively easy to implement and provides great support. Just keep in mind that a jarring jump to the websiteâ&#x20AC;&#x2122;s footer could disorient some users.

FIGURE 4.9. The footer anchor lets users

jump to the navigation in the footer. Demo: smashed.by/footeranchor.

THE SELECT MENU Another clever approach is to transform navigation links into

a select form element for small screens. This approach saves a lot of real estate; but the select menu pattern could also be confusing because youâ&#x20AC;&#x2122;d be using a form element outside of its natural environment. This approach can FIGURE 4.10. The select menu also be tricky if your navigation converts navigation links into includes submenus (more on com- a drop-down menu: smashed.by/dropdown-menu. plex navigation later). BY BRAD FROST

139

THE TOGGLE

The toggle is one of the more elegant patterns out there. On a small screen, the navigation is collapsed, with only a button (typically denoted by three horizontal lines12 or a “Menu” anchor) that, when tapped, toggles the navigation’s visibility. The result is navigation that saves space but is still easily accessible. We need to make sure that the navigation remains accessible to users with poor JavaScript support. But we can bypass JavaScript altogether with this pattern by using the technique that Aaron Gustafson describes in “Build a Smart Mobile Navigation Without Hacks,”13 employing the :target CSS pseudo-selector to toggle the navigation.

FIGURE 4.11. The toggle collapses navigation items, revealing them

conditionally: smashed.by/togglenav.

12. Moore, Jordan. “The Semantic, Responsive Navicon,” Smashing Magazine,

smashed.by/design-navicon. 13. Gustafson, Aaron. “Build a smart mobile navigation without hacks,” .net magazine, smashed.by/nav-no-hacks.

140 | RESPONSIVE DESIGN PATTERNS

NAVIGATION FLY-OUT

The navigation fly-out is a good implementation of the off-canvas pattern explained in the layout section earlier. Instead of appearing above or below the core content, the navigation exists just to the side, off screen, and slides in when requested. This technique is certainly elegant, but because there are plenty of (literal) moving parts, make sure to test in as many environments as possible to avoid rendering the navigation non-functional, as Stephanie Rieger explains in her article “A Plea for Progressive Enhancement.”14

FIGURE 4.12. The navigation fly-out pattern tucks the navigation off screen and

animates in when requested: smashed.by/navflyout.

PRIORITY+ The name “priority+” was coined by Michael Scharnagl to describe

the technique of exposing what are deemed to be the most important navigation items and tucking away less important ones behind a “More” link. This pattern can be effective for navigation that contains a lot of links at the same hierarchy level. However, deciding which navigation items to prioritize requires making assumptions about the user’s intentions; while we hope those assumptions are correct, the user may be looking for something else.

14. Rieger, Stephanie. “A plea for progressive enhancement,” smashed.by/plea-enhance.

BY BRAD FROST

141

FIGURE 4.13. Priority+ exposes the most important navigation icons and

hides the rest behind a “More” link: smashed.by/priority-nav.

THE HIDE ’N’ CRY This pattern (or should I say anti-pattern?) hides navigation that is deemed unimportant for small-screen users. While this certainly saves real estate, it deprives the increasingly mobile audience of access to those parts of the experience. Remember that small-screen users want to do anything and everything that large-screen users do, provided that it’s presented in a usable way. Don’t penalize users for the device they happen to be browsing with.

Complex Navigation Desktop screens have plenty of room for navigation. In addition, the mouse enables users to hover over parent categories to reveal drop-down menus containing child links. However, the mobile Web environment, with its lack of screen real estate and lack of hovering, poses a significant challenge for implementing complex navigation. Sometimes whittling thousands of pages of content into three little links that fit tidily on a phone’s screen is not realistic. Major organizations, massive e-commerce retailers, universities and other owners of large websites often need complex multi-level navigation. So, once again, the community is rising to the challenge and creating some great solutions for complex navigation for responsive designs.15 Let’s dive in, shall we? 15. Complex Navigation Patterns for Responsive Design, smashed.by/complex-nav.

142 | RESPONSIVE DESIGN PATTERNS

THE MULTI-TOGGLE

The multi-toggle is basically just a series of nested accordions. The user taps on a parent category to reveal child categories below. At a large enough screen size, the navigation converts to the usual multi-level drop-down menus we’re used to seeing. This pattern allows the user to easily scan parent categories, while providing quick access to child elements. It’s also a scalable solution for navigation that is many levels deep.

FIGURE 4.14. The multi-toggle is basically a series of nested accordions:

smashed.by/multi-toogle.

THE OL’ RIGHT TO LEFT While the multi-toggle reveals child categories beneath the parent

category, the ol’ right to left takes a cue from the off-canvas pattern and slides in sub-navigation from off screen to the right. This is definitely one of the more elegant solutions for complex navigation; it is scalable to multiple levels, and it follows a common mobile convention of enabling the user to drill down into an experience with right-to-left animation. Because this approach is relatively intricate, test it in as many environments as possible to ensure the navigation remains accessible. Also, keep in mind that BY BRAD FROST

143

FIGURE 4.15. The ol’ right to left pattern animates submenus in from the

right when requested: smashed.by/right-left.

animation (especially on mobile browsers) can vary extremely in performance (or be absent altogether). SKIP THE SUB-NAVIGATION

Sub-navigation typically contains items that are also included on the parent category’s landing page. Because that content is accessible on the landing page, it’s perfectly viable to simply take smallscreen users straight to the landing page and let them make their next move from there.

FIGURE 4.16. This pattern takes small-screen users to a new page for submenu

items: smashed.by/skip-subnav.

144 | RESPONSIVE DESIGN PATTERNS

This approach frees the user from having to deal with sub-navigation altogether; however, requiring a full page refresh just to get to the next level of navigation isn’t very efficient. CAROUSEL+

In this pattern, parent categories are displayed as a carousel, with sub-menus displayed individually as a list below. The user can swipe horizontally or use right and left arrows to move through the carousel. This pattern is unique; however, while swiping through a carousel can work well on touch devices, it can feel tedious on non-touch ones. Also, by not exposing all parent categories at once, the FIGURE 4.17. Carousel+ is carousel navigauser is forced to interact with tion that exposes sub-navigation options the menu just to see what below the chosen option: smashed.by/carousel-plus. other categories are available to them. NAVIGATION CONSIDERATIONS Whatever navigation method you choose, keep these things in

mind: Ŧ Find the balance between navigation accessibility and unobtru-

siveness. Like a good friend, it should be there when you need it, but not always in your face, demanding attention. Ŧ Because navigating is such an important interaction, test in as many environments as possible. See how the navigation works on a variety of devices and mobile browsers, including proxy BY BRAD FROST

145

browsers. (Proxy browsers offload JavaScript interactions to the server and can yield interesting results; read more about them in Peter-Paul Koch’s introductory chapter.) Ŧ If you’re collapsing the navigation for small screens, be explicit with the navigation icon. An icon consisting of three horizontal bars is becoming the de facto standard.16 Just make sure that whatever icon or wording you use screams “Navigation!” to the user.

Conditional Loading Different environments provide different constraints and opportunities, so it’s essential to understand that responsive Web design does not mean serving up the exact same experience to all users (and adjusting only the layout). Serving hefty desktop designs and functionality to small screens does not make for a good mobile experience, but that is what the majority of responsive design implementations do today, as Guy Podjarny explains in his article “Performance Implications of Responsive Design.”17 Conversely, serving a barebones mobile experience to massive 27-inch screens doesn’t really capitalize on all that extra screen space and processing power. Is there a happy medium? Enter conditional loading.18 It’s not one of the three core ingredients of responsive design, yet this incredibly powerful technique can significantly enhance the performance of a responsive website. Conditional loading entails serving up an initial core experience (i.e. the content that is essential to the experience and that is defined by the URL), while providing links to chunks of supplementary content (things like related posts, products and content; comment modules; social sharing widgets; maps; and other third-party content) and loading them on demand. So how exactly do we do it? 16. Clarke, Andy. “We need a standard show navigation icon for RWD,” smashed.by/navicon. 17. Podjarny, Guy. “Performance Implications of Responsive Design,” smashed.by/rwd-perf. 18. Keith, J. “Conditional Loading for Responsive Designs,” 24ways, smashed.by/cond-loading.

146 | RESPONSIVE DESIGN PATTERNS

CREATING CONDITIONALLY LOADED CONTENT

1. Create a main HTML page, as well as separate HTML pages that contain only supplemental fragments of content. 2. Link to those content fragments on the main HTML page. This will ensure that they remain accessible, even for users with poor or no JavaScript support. 3. Using JavaScript, detect when the user clicks or taps a link to a content fragment or when the screen size warrants loading the supplemental content. 4. Load the relevant content fragment via AJAX.

FIGURE 4.18. This example segments lists of related products and reviews

as their own HTML fragments, which can be pulled in when requested.

By not including everything and the kitchen sink by default, we are able to serve up a relatively lightweight core that ensures fast page loading, while still providing access to non-essential content and functionality. And when the screen becomes large enough, we can dynamically pull in that supplemental content to build the final, full experience.

BY BRAD FROST

147

FIGURE 4.19. Supplementary content is loaded for screens that are large

enough: smashed.by/mobile-first-example.

Conditional loading not only helps performance, but also makes pages more scannable. Earlier, in the layout section, we discussed how sifting through disparate content types can confuse users and make it harder for them to find what they’re looking for. Presenting a link with a heading and/or excerpt, instead of the full content, reduces clutter and enables the user to scan the page more quickly. CONDITIONAL LOADING CONSIDERATIONS Ŧ

Giving users different experiences is alright.

However, the full functionality should remain accessible to everybody in some way, shape or form. For example, it’s perfectly fine to have small-screen users tap to load a blog post’s comments, while loading comments by default for large-screen users. Sure, the experience is a bit different, but access to the comments is what truly matters.

148 | RESPONSIVE DESIGN PATTERNS

Screen size is just one variable.

Conditional loading needn’t be limited to screen size. Touch events, geo-location, CSS animations and a host of other variables can be checked to pull in advanced functionality. Check out Modernizr19 for a broad list of features and capabilities that can be detected. Ŧ

Large screens do not necessary imply a fast connection. A laptop user could easily be on a flaky connection, just as a mobile user may be on a strong Wi-Fi connection. While there’s currently no reliable way to truly detect connection speed, keep an eye out for the emerging navigator.connection20 to detect connection type and speed.

Mind the number of HTTP requests being made. Dynamically pulling in a lot of extra scripts, styles and content chunks can add many HTTP requests and bog down performance. Thankfully, tools such as the Filament Group’s Quickconcat21 can concatenate all of those extra files and minimize the number of requests being made. Ultimately, be prudent with conditional loading, and keep performance at the top of mind.

Look into server-side optimization. Conditional loading doesn’t need to be solely a client-side tech-

nique. In the next chapter, Dave Olsen will explain how RESS (responsive design with server-side components) can be very useful when conditionally loading different views and content for different contexts.

19. Modernizr, smashed.by/modernizr. 20. Calhoun, David. “Using navigator.connection on Android 2.2+,” smashed.by/navcon. 21. Quickconcat, smashed.by/quickconcat.

BY BRAD FROST

149

Images In case you haven’t noticed, the Web is chock-full of images. In fact, images often make up the bulk of a page’s payload, 22 so handling them properly in a responsive design is important. Many factors come into play when serving graphic elements to multiple devices:23242526

“Conditional Loading for Responsive Designs”23 Jeremy Keith provides a great overview of what conditional loading is and how effective it can be for responsive designs.

“An AJAX Include Pattern for Modular Content”24 Scott Jehl explains how his team used conditional loading

Varying screen size We have to deal with small screens, large screens and everything in between.

for non-essential elements on the Boston Globe’s responsive website. Ŧ

“SouthStreet Progressive Enhancement Workflow”25

The growth of high-resolution displays Led by Apple’s Retina displays, high-resolution displays are becoming increasingly common.

The Filament Group has created lot of useful (and opensource!) scripts to help our community execute conditionally loading properly. Ŧ

“Creating a Mobile-First Responsive Web Design”26

Varying bandwidth

My tutorial and demo go

Bandwidth runs the gamut from EDGE to 3G to blazingly-fast FiOS connections.

through some considerations to make when implementing conditional loading.

22. HTTP Archive stats, smashed.by/httparchive-stats. 23. Keith, Jeremy. “Conditional Loading for Responsive Designs,” smashed.by/cond-loading. 24. “An Ajax-Include Pattern for Modular Content,” smashed.by/mod-content. 25. Southstreet, smashed.by/southstreet. 26. “Creating a Mobile-First Responsive Web Design,” smashed.by/mobile-first-example.

150 | RESPONSIVE DESIGN PATTERNS

The need for art direction

Jason Grigsby sums up the issues involved in art direction for responsive images in his post “A Framework for Discussing Responsive Images Solutions:”27 “Authors need to be able to provide different sources for images at different sizes […] based on the judgment of the designer for what is the best image at a particular breakpoint.” Ŧ

Image types Images on the Web appear in a few common ways, e.g. as background images, icons and even good ol’ fashioned <img> elements. Let’s now look at how to handle each image type.

BACKGROUND IMAGES Background images are defined in CSS, which means we can conditionally swap out or introduce background images in media query blocks. We can detect for certain screen dimensions or pixel densities and conditionally serve the appropriate background image for the device at hand. Tim Kadlec has done some extensive testing28 and found that browsers do a good job of ensuring that redundant background images aren’t downloaded. For the record, background images are stylistic elements that

don’t add semantic meaning to a Web page. I’m saying this because (as we’ll find out in a bit) responsive <img> elements can be tricky, and some people mistakenly think they can skirt the issue altogether simply by turning <img> elements into background images. This is not semantically sound, so don’t consider it as an option.

27. Grigsby, Jason. “A framework for discussing responsive images solutions,”

smashed.by/rwd-images-solutions. 28. Kadlec, Tim. “Media Query & Asset Downloading Results,” smashed.by/download-research.

BY BRAD FROST

151

Considerations Ŧ

Don’t load large background images by default.

We should be authoring everything in a mobile-first manner, and background images are no exception. Mobile browsers that don’t support media queries shouldn’t be forced to download large images that were never meant for small screens. Instead, serve smaller background images (or no images at all) by default. Ŧ

Optimize for high-resolution screens. We can create crisp background images for high-resolution screens by detecting pixel density in a media query using the 29 min-device-pixel-ratio property. That is, we include larger images and then use the background-size property to shrink them down to the appropriate dimensions.

ICONS Icons can be found throughout Web interfaces: shopping cart icons, social media icons, arrows, carets and much much more. Icons clarify actions and guide users through tasks. However, their relatively small size means they are quite susceptible to becoming blurry on sharp screens. Also, all of those tiny icons can add a lot of HTTP

requests and bog down performance. Here are some techniques for dealing with these important little elements: Ŧ

Use Unicode characters.

Some common icons have Unicode character equivalents. This eliminates the need to include bitmap images for tiny elements, instead allowing for widely supported vector characters. These Unicode characters are great, but be careful because not every OS

29. Smus, Boris. “High DPI images for variable pixel densities,” smashed.by/high-dpi.

152 | RESPONSIVE DESIGN PATTERNS

supports the same character sets. For example, some BlackBerrys and old Android devices will render an upward-facing caret as a square. Ŧ

Implement icon fonts.

Another trick to avoid using raster graphics is to use icon fonts. Icon fonts are Web fonts that contain vector icons, which you can include on your website via the @font-face declaration. Symbolset30 and Pictos31 are great tools to get up and running with icon fonts. Again, keep in mind that not every device and environment supports @font-face. Ŧ

Use background sprites. Image sprites are raster graphics that are used to reduce HTTP requests and manage interface images all in one place. Maykel Looman nicely sums up the process of producing high-resolution sprites in his article “Using CSS Sprites to Optimize Your Website for Retina Displays:”32

1. Instead of loading all of your assets as separate files, put them in a sprite. 2. Create another version of your sprite that is exactly double in size, with graphics that are exactly double in dimensions. 3. Gather all selectors that reference the sprite, and point them to the @2x sprite in the media query for high-resolution displays. 4. Don’t forget to set the background-size property to translate the dimensions of the @2x sprite.

30. Symbolset, smashed.by/symset. 31. Pictos, smashed.by/pictos. 32. smashed.by/retina-sprites

BY BRAD FROST

153

FIGURE 4.20. Optimizing sprites for high-resolution screens can be

as simple as creating one that’s twice as large as the original.

Sprites are implemented as CSS background images, so proportionally larger sprites may be swapped for them on high-resolution displays. If done correctly, optimizing for high-resolution screens should only require declaring a new larger sprite and sizing the background image accordingly. IMAGE ELEMENTS Web designers need to be able to conditionally load different image

assets based on such conditions as resolution, pixel density, network speed and more. Unfortunately, the lowly <img> element isn’t cut out for such a tall order. The topic of responsive images could easily fill a book on its own. This certainly isn’t the place to get into all of the history, browser behaviors, political hurdles and so on, so I’ll spare you. If you’re curious about why responsive images are so challenging, I recommend checking out Mat Marquis’ article “Responsive Images: How They Almost Worked and What We Need.”33 33. smashed.by/resp-images

154 | RESPONSIVE DESIGN PATTERNS

FIGURE 4.21. Web designers need to be able to load different image

assets for a number of conditions.

So, what are we to do about responsive images? Discussing all current solutions at length wouldn’t be very useful because they are changing and evolving at an amazingly swift pace. So, I won’t discuss them here (although if I had to choose one, I’d put my money on Picturefill34 by Scott Jehl). If you’re looking for a good rundown of the many solutions out there, be sure to check out Chris Coyier’s post “Which Responsive Images Solution Should You Use?”35 RESPONSIVE IMAGES CONSIDERATIONS While responsive image solutions vary in implementation, you

should follow these general guidelines: Ŧ

Load small image assets by default. Keeping with the mobile-first strategy, we need to avoid mak-

ing easy assumptions about our users’ context. By defaulting to small images, we are less likely to serve bulky assets to less capable mobile browsers and constrained environments.

34. Picturefill, smashed.by/picturefill. 35. “Overview of Responsive Images Solutions,” smashed.by/resp-images-solutions.

BY BRAD FROST

155

Make only one HTTP request per image.

Make image content legible on small screens. Make sure that users can make out the content of images on small screens. This sounds obvious but it’s important because images (especially desktop-sized images squished down onto mobile screens) can be difficult to decipher.

Serve crisp images to high-resolution displays In his article “Retina Revolution,”37 Daan Jobsis explains that image compression impacts JPG file size more than image dimensions. Jobsis found that we can serve JPGs with larger dimensions by default to ensure images look crisp on high-resolution screens. But because those larger JPGs are more compressed, there’s no real toll on performance. This technique is promising because it removes the need to maintain and swap out multiple assets for the same image.

Avoid text in images.

In “How Apple.com Will Serve Retina Images to New iPads,”36 Jason Grigsby explains how Apple’s website loads both low- and high-resolution versions of images for Retina screens. Avoid this at all costs, because one of those assets will always be dead weight and result in a big waste of bandwidth.

This practice is commonplace. Besides basic accessibility issues—text rendered in bitmap graphics can’t be searched, read by screen readers, copied and pasted, etc.—it looks especially fuzzy on high-resolution displays.

36. smashed.by/retina-ipads 37. smashed.by/retina-revolution

156 | RESPONSIVE DESIGN PATTERNS

Maintain hierarchy.

Images of different shapes and sizes that look very distinct on large screens run the risk of looking uniform when scaled down to small screens. One image might be a giant full-width banner on large screens, while another might be supplemental and occupy only a third of the width. Maintaining the hierarchy of these images is important, even when they’re being viewed on a small screen. Ŧ

Consider server-side detection. Many client-side responsive image solutions are still being ironed out, so looking into server-side responsive tools might be worthwhile until a long-term solution is in place. Check out Sencha.io Src38 and Adaptive Images.39

Media Objects Media objects, such as videos, slideshows and maps, need to be given many of the same considerations as images because they, too, need to be fluid, legible and performant. In addition to these considerations, we also need to ensure they are functional. VIDEO

Videos and other media objects are different than images in that they don’t maintain their aspect ratio when resized. Besides the myriad of codecs that need to be juggled, the scaling issue presents a big challenge for serving up video to a slew of Web-enabled devices. Thankfully, there are ways to create intrinsic ratios for videos, 40

38. Sencha.io Src, smashed.by/sencha-io. 39. Adaptive Images, smashed.by/adaptive-images. 40. Koblentz, Thierry. “Creating Intrinsic Ratios for Video,” A List Apart,

smashed.by/intrinsic-ratios

BY BRAD FROST

157

and there’s even FitVids.js, 41 a plugin that does the heavy lifting of serving up videos that maintain the correct aspect ratio when resized. (Trent explained how the technique works in the previous chapter of this book.) MAPS More involved media objects such as maps may require more than simple resizing in order to create the best experience in every context. While techniques42 exist to create fluid and scalable embedded maps, I’d argue that embedded maps don’t translate all that well to the mobile context. There are a few reasons why we might not want to serve embedded maps to mobile users. First, mobile viewports are small to begin with, so forcing our users interact with a map that occupies a fraction of that viewport will make for a cramped experience. Secondly, many mobile operating systems (including iOS and Android) intercept links to Google Maps and open the device’s native mapping application, resulting in a much more robust and familiar experience.

FIGURE 4.22. Take context into account when working with maps in order

to create the best possible experience: smashed.by/embedded-maps. 41. Fitvids, smashed.by/fitvids. 42. “Responsive Google Maps,” smashed.by/responsive-maps.

158 | RESPONSIVE DESIGN PATTERNS

Lastly, the performance cost of loading a bunch of external scripts and images might not be worth it, considering the limited user experience that embedded maps ultimately provide. So, what are we to do? This is actually a great opportunity to use conditional loading to serve the best mapping experience for the given context. By default, we could simply include a static image of a map that links to a third-party service, such as Google Maps. Thus, when the image is tapped, most major mobile platforms will fire up the native maps application, where the user will have a richer experience. If that fails, the user is simply taken to the Google Maps website, where they receive a dedicated full-screen mapping experience. We can also detect when the screen is large enough to include an embedded map, and then dynamically load the embedded map experience. This technique comes in handy for features such as store locators and other location-specific needs. While it doesnâ&#x20AC;&#x2122;t make sense in every circumstance, it makes the most of each context, instead of simply creating a one-size-fits-all experience.

Typography Serving a legible reading experience to the multitude of devices that access the Web is critical. Thankfully, responsive techniques give Web designers some level of control over type size, line length, rhythm and more. Much can be said (and has been said) about responsive typography; so, while reducing and increasing font size across resolutions is common, bringing up a few emerging type-related patterns would be worthwhile. FLUID TEXT Fluid images are great because they scale to perfectly fit the size of their containers. However, hypertext just doesnâ&#x20AC;&#x2122;t work that way out BY BRAD FROST

159

of the box. Thankfully, some clever people have devised tools such as FitText43 and SlabText44 to create text that scales proportionally with its container. This enables designers to create striking headings (see Trent Walton’s website45 for some inspiration) and blocks of text that scale with their parent containers. MOLTEN LEADING Another challenge with responsive design is ensuring that type remains legible and beautiful across all screen resolutions. However, font size and line height don’t scale proportionally the way images do. Tim Brown discusses this in his article “Molten Leading (or Fluid Line Height),”46 proposing a pattern that enables designers to control line height fluidly, so that the leading can ebb and flow with the available screen space. CONDITIONAL TEXT Should we apply conditional hiding and showing of text based on screen size? In “Is There Ever a Justification for Responsive Text?”47 James Young describes the pattern and concludes that if text isn’t worth displaying on mobile screens, then it shouldn’t be included anywhere. While I am certainly a fan of content parity, there are cases in

which hiding supplemental information on smaller screens is valid (such as article excerpts, minor meta data, etc.), so long as users are able to accomplish all of their tasks and make informed decisions based on the available text.

43. FitText, smashed.by/fittext. 44. SlabText, smashed.by/slabtext. 45. Trent Waltons website, smashed.by/trentwalton. 46. smashed.by/molten-leading 47. smashed.by/resp-text

160 | RESPONSIVE DESIGN PATTERNS

FIGURE 4.23. The Boston Globe hides supplemental text in its weather widget,

while still getting the point across.

A good example of conditional text in action is the weather module on the Boston Globe’s website, which hides supplemental text (like “Mostly sunny” or “Sunny and cool”) from small screens while still giving users the gist of the weather. Tread lightly when using the conditional text pattern; there’s a fine line between hiding supplemental text and hiding text that’s essential to the experience.

Interactive Components Some website elements contain more moving parts than simple text and images, and putting relatively complex interactive components into a responsive environment can be tricky. Let’s take a look at how the community is handling common interactive components like carousels and accordions. CAROUSELS

We see carousels everywhere on the Web: home page promotional areas, featured stories, product images, related content and so on. They’re a convenient way to save real estate; they establish hierarchy within a group of content elements; and they introduce BY BRAD FROST

161

a sense of motion and interactivity that can really enhance the experience. Let’s look at some ways to implement carousels in a responsive environment.

Fully Fluid The fully fluid carousel matches the visible panel’s width to the screen’s size, similar to a fluid image. While the result looks similar to a static image, more is going on under the hood because the carousel’s logic needs to be adjusted via JavaScript to ensure that the panels advance according to the screen’s size.

FIGURE 4.24. A fully fluid carousel scales each panel to match the con-

tainer’s width: smashed.by/fluid-carousel.

Another consideration is placement of type. A common desktop convention is to overlay text on top of a panel’s images. But small screens mean small images, so setting text over an image could obstruct the image’s content. To avoid this, we can simply stack text underneath the image on small screens to ensure that both the type and the image remain legible. THE REVEAL The reveal pattern exposes more carousel panels when space is available. This technique is used on many fluid desktop websites, such as Amazon’s and Vimeo’s, and on responsive websites, such as Disney’s. 162 | RESPONSIVE DESIGN PATTERNS

FIGURE 4.25. The reveal exposes more panels in the carousel for screens

that can accommodate them: smashed.by/reveal.

The reveal keeps content compact on small screens, while capitalizing on more space by exposing more content. A common complaint against mobile-first responsive design is that large-screen versions of mobile-first designs look vacant. The reveal solves this by introducing more content to fill the space.

Carousel Considerations Ŧ Make sure you actually need one. Many carousels are used to

sweep content under the rug. A carousel should not be a lazy substitute for making important content strategy decisions. Ŧ Cycle through like items. Carousels should be predictable. The user should have a rough idea of what the next panel will be. Instead of cycling through disparate items (“Check out our summer sale!,” “Like us on Facebook!,” “Read our blog!”), give panels a running thread (product images or related news stories or related products, etc.). Ŧ Load only what you need. If a carousel has 20 panels, please don’t load all 20 by default! You don’t know whether the user will even make it to the second panel. Don’t load more than you have to, to keep the experience nice and fast. BY BRAD FROST

163

FIGURE 4.26. Disney, Vimeo and Amazon all use the reveal pattern to

show more related content for screens that can accommodate it.

164 | RESPONSIVE DESIGN PATTERNS

Ŧ Be explicit. Tiny little dots sitting by themselves below a carou-

sel aren’t explicit enough to tell the user that this is a carousel. Add previous and next arrows, and indicate the user’s current position in the carousel. Ŧ Treat touch as an enhancement. Enabling touch gestures to swipe through a carousel is a great way to add some pizazz to the user experience. Remember, though, that not every device or browser supports touch events. Provide additional means of navigation for optimal support. ACCORDIONS Progressive disclosure is an extremely useful technique to help deal with super long page lengths on small screens. Progressive disclosure hints at more content by providing a title or summary of a content chunk that’s hidden or not loaded. Once the user taps the desired section, the section is expanded and the associated content chunk is revealed.

FIGURE 4.27. Wikipedia’s mobile website uses progressive disclosure to

enable users to quickly scan each section.

BY BRAD FROST

165

Accordion to Full Accordions have long been a common interface convention, but they makes even sense for small screens where space is at a premium. Collapsing sections of content makes for more scannable pages on small mobile screens. However, once enough screen space is available to accommodate more content, it might make sense to remove the accordion altogether and simply expose the content in full.

FIGURE 4.28. The accordion-to-full pattern collapses content sections on

small screens and exposes them on large screens: smashed.by/accordion.

What Iâ&#x20AC;&#x2122;ve just described is a basic accordion. Not terribly exciting on its own, but in a responsive environment, an accordion makes sense when space is at a premium. And if the screen is large enough, simply remove the accordion altogether and expose all of the content.

Accordion to Tabs Tabs are another component that work really well for conditionally exposing chunks of content. However, tabs donâ&#x20AC;&#x2122;t always fit right on small screens. Thankfully, accordions work consistently well on small screens. We can combine these two popular patterns, so that an accordion is shown on small screens for a group of content and then converts 166 | RESPONSIVE DESIGN PATTERNS

FIGURE 4.29. The accordion-to-tabs pattern presents an accordion on small

screens and a set of tabs on large screens: smashed.by/acc-to-tabs.

into a series of tabs for large screens. While doing this certainly requires a bit of hoop jumping, the solution ensures that users can efficiently navigate those chunks of content.

Forms Forms add a vital element of interactivity to the Web experience. Because forms are so crucial, they must be as easy to use as possible, no matter how a user accesses the Web. FULLY FLUID

A common challenge on small screens is creating fully fluid form fields while still maintaining a fixed amount of padding. The traditional box model makes this surprisingly difficult. Thankfully, the CSS box-sizing property gives us a way forward. By setting 48 box-sizing to border-box, we can make form fields fully fluid while keeping a fixed amount of padding.

48. Irish, Paul. â&#x20AC;&#x153;* { box-sizing: border-box } FTW,â&#x20AC;? smashed.by/box-sizing.

BY BRAD FROST

167

FIGURE 4.30. The CSS rule box-sizing: border-box makes elements

such as form fields fully fluid, while retaining fixed padding and borders: smashed.by/fluid-form.

TOP-TO-LEFT LABELS Aligning labels to the left of form fields is common on large screens. But the pattern doesnâ&#x20AC;&#x2122;t always translate well to constrained screens. A simple solution is to stack labels on top of form fields on handsets.

FIGURE 4.31. We could stack form labels vertically on small screens

and float them left on large screens: smashed.by/top-left.

168 | RESPONSIVE DESIGN PATTERNS

Considerations Ŧ

Look for alternatives to forms.

Look for opportunities to reduce friction and perhaps remove a form altogether. Social log-in buttons can reduce the pain of logging in on small screens, and geo-location can be used in lieu of address, city and ZIP forms to serve up location-specific information. Ŧ

Reduce the number of fields in a form. Completing forms is a chore, especially if the user is pecking on a mobile keyboard. So, in adapting your big honking form to small screens, look for opportunities to shed extraneous fields.

Use the appropriate input types. The proper HTML5 input type will pull up a contextualized keyboard in many mobile browsers. For example, specifying type="email" will pull up a virtual keyboard containing dedicated .com and @ buttons, which makes completing the form just a little less painful.

Consider different input methods.

Keep in mind that devices with a myriad of input types are used to access the Web: touch, D-pads, scroll wheels and many others. Employing things like the tabindex attribute can make forms more accessible and enjoyable, regardless of input method.

Data Tables Responsive data tables are tricky buggers. A table is one interface component whose semantic meaning is closely tied to its presentation, making it a challenge to translate across many screen sizes.

BY BRAD FROST

169

Chris Coyier has rounded up49 some of the more popular ways our community is dealing with responsive data tables. Let’s look at some of the techniques.

Priority Similar to the priority+ navigation pattern, this one displays columns that are deemed to be most important for small screens, while still giving users a filtering option to see the remaining content. The wider the screen, the more columns are conditionally displayed. The result is a relatively elegant solution that gives users control over what data to view.

FIGURE 4.32. Priority tables expose the most important columns on

small screens, making the remaining ones available on request: smashed.by/display-data.

Horizontal Overflow Another approach taken by data-rich mobile websites such as Wikipedia is to use the overflow-x CSS property, which lets mobile users scroll horizontally to view more columns. Some techniques even dock the left-most column for easy comparison and analysis.

49. Coyier, Chris. “Responsive Data Table Roundup,” smashed.by/resp-tables.

170 | RESPONSIVE DESIGN PATTERNS

FIGURE 4.33. The horizontal overflow pattern lets users scroll horizontally

to see more table columns: smashed.by/overflow-tables.

Definition List One way to deal with tabular data on small screens is to avoid using a table altogether, swapping it with a definition list, which is more scannable on small screens. But prioritize semantics, and use whatever is most appropriate.

FIGURE 4.34. The definition list pattern converts a table into a definition

list, which is friendlier to small screens: smashed.by/table-def-list.

Table to Chart One of the wilder solutions out there is to convert data tables into pie charts on small screens. While not appropriate for every data table, the solution is pretty interesting and visually appealing for some use cases (see image on the next page). BY BRAD FROST

171

FIGURE 4.35. Some tables can be converted into graphical charts on small

screens: smashed.by/table-to-charts.

Opt-Out Responsive Design The last pattern we’ll look at is an interesting one. Up until recently, mobile-optimized websites have been separate and (unfortunately in many cases) inadequate experiences. Because mobile users have been consistently let down by these inadequate experiences, they’ve become conditioned to immediately fish for the “View full site” link upon reaching a “mobile-optimized” experience. Also, some users simply prefer the full zoomed-out view of a Web page to the close-up view provided by a mobile-optimized design. I’ve heard many stories from (and know a few) people who say they’d rather see a bird’s-eye view of a layout, and then pinch and zoom into the desired section. I concede that having a bird’s-eye view is better than having to sift through a load of disparate content (as described in the layout section at the beginning of the chapter). Chris Coyier explains how to execute the opt-out technique in his post “Opt-Out Responsive Design?”50 The technique satisfies the

50. smashed.by/opt-out-rwd

172 | RESPONSIVE DESIGN PATTERNS

user’s desire for an unoptimized mobile experience (hey, they asked for it!). Chris lays out these steps: 1. Query a parameter in the URL string (something like http://yoursite.com/?resp=no). 2. Remove the viewport meta tag and a responsive class from

the body. 3. Adjust the CSS to tweak the desktop view. 4. Provide a way for the user to switch back and forth between the responsive and unresponsive versions. I foresee this technique fading away as (hopefully) more fully functional, robust mobile Web experiences become more common. In the meantime, this could be a viable technique to better serve your users.

Into the Great Wide Open We as Web designers are tasked with crafting experiences that look and function beautifully in a host of contexts, even ones that haven’t been invented yet. That’s certainly a tall order. But by breaking the overall interface into smaller puzzle pieces, we can make these admittedly tough problems a little more bearable. We can always count on change in this ever-shifting Web landscape. Pick this chapter up in a year or two and you’ll see what I mean. As responsive design continues to evolve, we’ll see some techniques and patterns fall away, and the ones that stick around will surely evolve. And that’s OK. While implementations will surely change as time marches on, the need remains to serve the Web in all its glory to more people in more environments than ever. That’s certainly a challenge for us, but also an absolutely amazing opportunity. Here’s to the future! BY BRAD FROST

173

ABOUT THE AUTHOR Brad Frost is a mobile Web strategist and frontend designer at R/GA, where he helps famous brands navigate this ever-changing device landscape. The biggest lesson he has learned in his career is that there is no one right way to do things and that in order to do your best, you have to be yourself. Brad’s motto: “Wake up excited.“

174 | RESPONSIVE DESIGN PATTERNS

CHAPTER FIVE

BY DAVE OLSEN

OPTIMIZING FOR MOBILE

ESPONSIVE DESIGN has given us the power to deliver our content to anyone who happens to be using a device with a browser. Much of the discussion today surrounding responsive design seems to focus primarily on flexible layouts and the decisions on how to implement them. That’s understandable. The push and pull of the browser window provides a visceral and satisfying interaction with a website’s design. Unfortunately, it obscures the larger opportunity that responsive design provides us—namely, the ability to truly deliver our content anywhere. Today, a responsive website may be viewed on devices ranging from a TV box over high-speed Internet connection to an embedded system with a low-powered CPU to a mobile device with slow or metered network access. The latter device is becoming more important by the day. 17% of today’s US population use a mobile device as their primary means of accessing the Internet.1

1. Smith, Aaron. “Cell Internet Use 2012,” smashed.by/pewinternet-reports.

BY DAVE OLSEN

175

To paraphrase Erik Runyon of the University of Notre Dame, because of this shift we should think of “RWD” as standing not only for responsive Web design but also for responsible Web design. We should do everything in our power to make sure that our websites are optimized for performance. It’s the responsible thing to do. Unfortunately, many recent designs and trends show that we have a ways to go in making sure that our websites are truly optimized for performance.

The Web Is Getting Heavier It’s true. The Web has been putting on some serious weight in the last few years. Since 2003, the average weight of a Web page has risen from 300 KB to 1,098 KB, or just a little over a 1 MB (using an average of 85 requests per page to do so!).2, 3 It’s both fascinating and disturbing to me that it now takes that much content to display something as simple as a Web page. Tim Berners-Lee’s first ever Web page, weighing in at a measly 2.5 KB, could be downloaded over 420 times using the same amount of bandwidth as it takes to download today’s average page. Two things have driven this gradual increase in overall page weight: images and JavaScript. Combined, they make up 82% of the 1,098 KB needed to load today’s average page. Images alone account for 64% of that average weight. While these may be the primary technical reason for the increase in page weight, there is a more important reason: ourselves. As designers and developers, we’ve bought, hook, line and sinker, into the notion of an ever faster Internet, an Internet where speeds can and will only increase. That’s because, quite frankly, it’s a handy

2. Grigsby, Jason. “Going Fast on the Mobile Web,” SlideShare, 18 Sep 2008,

smashed.by/slideshare-presentation. 3. HTTP Archive, Trends, 14 Oct 2012, smashed.by/httparchive-trends.

176

OPTIMIZING FOR MOBILE

FIGURE 5.1. This graph of total page transfer sizes between November 2010 and

September 2012 shows that the Web has been putting on some weight over the last few years.

rationalization for the things we want to do. Larger screen resolutions have given us larger canvases to fill with images. New browsers and JavaScript libraries have enabled us to create richer experiences. With an ever faster Internet, why worry about page weight? Surely bandwidth will eventually catch up with what we’re pushing. With mobile, the “ever faster Internet” rationalization ran head long into reality. Bandwidth wasn’t catching up. It actually took a huge step back to the bleep-bloop heyday of the dial-up modem. Unfortunately, with the rollout of supposedly high-speed mobile networks, the “ever faster Internet” rationalization may once again induce us to overlook our bad practices. By wearing our “desktop Web” blinders and focusing solely on bandwidth as the only performance optimization factor, we’re ignoring the true bottleneck of performance on mobile networks: latency.4

4. HTTP Archive, Trends, 14 Oct 2012, smashed.by/httparchive-trends2.

BY DAVE OLSEN

177

LATENCY: THE MOBILE NETWORK KILLER

While mobile network speeds are slower than desktop speeds, they aren’t on the whole that bad. The real Web performance difference between the two is latency. In his article “Latency: The New Web Performance Bottleneck,”5 Ilya Grigorik shows that simply increasing bandwidth on mobile doesn’t lead to correspondingly faster page loading times. After a certain point, page loading time is nearly identical, no matter how fast the network. For example, “upgrading from 5Mbps to 10Mbps results in a mere 5% improvement in page loading time.” On the other hand, lowering network latency always leads to improved page loading times. So, what is latency? Latency is the measure of the amount of time between when a browser requests a resource from a server (such as a CSS file) and when it starts to receive the server’s response. While a cable modem could experience an average latency of 20 milliseconds per resource request, a 3G connection could experience an average latency of 200 milliseconds per resource request.6 That’s 10 times more latency! To illustrate how quickly this lag can affect your Web page’s performance, consider the following. On a cable modem, assuming a browser that can open four concurrent connections,7 latency will account for 0.4 seconds of the total download time of our average page and its 85 requests. That same page loaded over an average 3G connection? Latency alone will account for 4.5 seconds of the total download time.

5. Grigorik, Ilya. “Latency: The New Web Performance Bottleneck,” igvita.com, 19 Jul 2012,

smashed.by/igvita-latency. 6. Podjarny, Guy. “The Mobile Difference in Numbers,” SlideShare, 26 Jun 2012, smashed.by/mobile-difference. 7. On average, most modern desktop/mobile browsers open six concurrent connections, but four is a good number considering a number of legacy browsers. You can find more data about it on Browserscope (smashed.by/browsers-connections).

178

OPTIMIZING FOR MOBILE

OUR PERFORMANCE EXPECTATIONS DON’T CHANGE JUST BECAUSE WE’RE ON MOBILE

Essentially, because of latency, the responsive website that you’ve built will take at least 10 times longer to download on a mobile network than on a wired connection…you know, the kind of wired connection that you do all of your development and testing on. But the latency factor can’t be that big of a deal, right? Users on mobile networks should realize that Web pages will take longer to download on mobile, and they should and will be more forgiving of that kind of thing. Bzzzt. Unfortunately, that’s simply not the case. Equation Research found that 74% of mobile users expect a mobile website to load just as quickly as a desktop website.8 Two years prior, only 58% of users felt that way. Does your website drive business to you? In the same report, it found that 57% of users who had a problem with a website on their mobile phone, including slow loading, wouldn’t recommend that website or service; 46% wouldn’t return. The most striking statistic from the report is the one related to loading time. In 2009, 20% of survey respondents said that if a mobile website took longer than five seconds to load, they’d leave. In 2011, a whopping 74% answered that they’d leave after the same amount of time. As we found earlier, latency alone would take up 4.5 seconds of that 5-second window when loading the average website. So, what do these numbers mean for us, designers and developers, as we develop responsive Web designs? RWD: OUT OF SIGHT AND STILL DOWNLOADING

One of the ugly truths about RWD as a mobile development technique is that it can do a damn good job of hiding page weight. Some 8. Equation Research, “What Users Want From Mobile,” Compuware, Jul 2011,

smashed.by/gomezpdf. BY DAVE OLSEN

179

developers approach RWD as if implementing the technique were simply a matter of sprinkling magical media-query pixie dust on their website and, voilà, they would instantly have a mobile-friendly website. As Jason Grigsby writes:9

“

The way in which CSS media queries have been promoted for mobile hides tough problems and gives developers a false promise of a simple solution for designing for multiple screens.”

With RWD, do you get a layout that is mobile-friendly? Sure. Can you provide a mobile user with a view of your website that shows less content and that is more focused? Yup. Have you delivered fewer assets and, therefore, less weight to the browser in this stripped down view? Sadly, you probably haven’t. From a Web performance perspective, the Achilles’ heel of RWD is over-downloading. Research by both Grigsby and Guy Podjarny shows that the mobile versions of many responsive layouts weigh the same as their full-blown desktop versions. The trap with responsive design is that, while the visual end product may be mobile-friendly, the page is not necessarily performance-friendly for mobile. DISPLAY: NONE: THE DARK MATTER OF RWD While RWD helps to hide page weight in three areas, the most important technique is to use display: none to hide interface elements.

9. Grigsby, Jason. “Performance Implications of Responsive Web Design,” SpeakerDeck, 26

Jun 2012, smashed.by/performance-implications.

180

OPTIMIZING FOR MOBILE

Are assets loaded per website page request on a mobile device larger than on a desktop view? Jason Grigsby's research

Guy Podjarny's research

FIGURE 5.2. These charts show Jason Grigsby (on the left) and Guy Podjarnyâ&#x20AC;&#x2122;s (on

the right) research on responsive website page sizes. Both graphs show whether the assets loaded per website load on mobile are larger or small than the full-blown desktop view. Usually, both have almost the same size.

#related-images { width: 100%; padding: 4px; background: #ccc; display: block; } @media screen and (max-width: 320px) { #related-images { display: none; } }

BY DAVE OLSEN

181

The truth is that, while those elements might not be visible to the end user, the HTML parser will still chunk through them and fill in their content. So, in the case of images included in your page with an img tag, they will still get downloaded, but neither you nor visitors to your website will be able to see them.10 Think of display: none as the equivalent of adding dark matter to your pages. Although the content won’t be seen, it’s there, lurking behind the scene, compromising the performance of your website by invoking extra requests and causing longer download times (remember: latency!). The second technique of RWD is to scale images so that they can flex and fit in a fluid layout. The problem with this is that some developers size their images to look good at large screen resolutions but then scale those images down simply via CSS to fit mobile resolutions—thereby forcing the mobile browser to download unnecessary kilobytes of data. The third technique is to put media queries within link tags. But it won’t prevent browsers from downloading CSS files. Media queries simply help the browser decide the loading priority and then decide which styles to apply to the page. For example: <link rel="stylesheet" media="screen and (max-width: 320px)" href="small.css"> <link rel="stylesheet" media="screen and (min-width: 320px) and (max-width: 760px)" href="medium.css"> <link rel="stylesheet" media="screen and (min-width: 760px)" href="large.css">

10. Tim Kadlec has presented some research on display: none in his article “Media

Query and Asset Downloading Results” (smashed.by/timkadlec2012). The conclusion is unambiguous: If you’re going to hide a content image, don’t do it with display: none. If you want to hide a background image, your best bet is to hide the parent element. To swap background images, define them both inside of media queries.

182

OPTIMIZING FOR MOBILE

On an iPhone, for example, the last CSS file would get the highest downloading priority, as well as have its styles applied to the Web page. The other two CSS files would be downloaded only after the first one has completed. The good news is that they wouldn’t block the rendering of the page.11 When building a responsive website, performance optimization needs to be baked into its design and implementation from the very beginning of the project. Some of these optimizations are quite simple to implement when building the website; others are more involved and will require time to rethink and reevaluate our common coding practices.

Learning Where to Trim the Fat The solutions suggested in the next major section of this chapter (“Mobile Optimization Techniques”) generally fall into one or more of the following four performance-related categories: Ŧ Reduce page weight; Ŧ Reduce the number of requests to the server (which addresses

latency!); Ŧ Optimize when content is loaded so that your website becomes usable more quickly; Ŧ Optimize JavaScript performance. In order to properly apply these techniques to your website, you will first need to find out which ones fit your needs. I highly recommend four tools to better understand your website’s needs.

11. Grigorik, Ilya. “Debunking Responsive CSS Performance Myths,” igvita.com, 14 Jun 2012,

smashed.by/perf-myths. BY DAVE OLSEN

183

WEIGH YOUR WEBSITE WITH MOBITEST

There are many ways to “weigh” a website. In my experience, the easiest and most useful mobile-specific tool is Mobitest.12 It provides not only the average loading time and average size of your website as viewed on a real mobile device, but an actual video of your page’s loading, which helps to visualize the performance of the website. It also provides an HTTP Archive file (discussed in the next subsection), which you can save locally to analyze later on.

FIGURE 5.3. Mobitest, a tool provided by Akamai, helps you discover and visualize

the loading issues for your website.

As of the time of writing, Mobitest has a few downsides. The devices that you can test on are limited to a few selected Android and iOS models. Also, the network connection can’t be modified, so testing latency is out of the question. That said, it’s a great step forward for testing mobile performance. If you’re familiar with WebPageTest,13 which Mobitest is built on, make sure to check out Andy Davies and Aaron Peters’ Velocity Europe talk, “Web Page Test: Beyond the Basics.”14 It provides some great tips and tricks for directly leveraging WebPageTest to your mobile performance testing needs. 12. smashed.by/mobitest 13. smashed.by/webpagetest 14. Slides: smashed.by/slideshare-test

184

OPTIMIZING FOR MOBILE

WATERFALLS: UNDERSTANDING THE HAR FILE

If you’re going to optimize your website, you’ll quickly realize that to get the most out of the process you’ll need to learn how to use an HTTP Archive (HAR) file. While this might sound daunting at first, it’s actually not that hard. If you’ve ever worked with the “Network” panel in either Chrome or Safari’s Web Inspector, then you’ve most likely already seen an example of a waterfall chart based on a HAR file. The following example uses a HAR file saved out of Chrome and uploaded to the PCAP Web Performance Analyzer.15

FIGURE 5.4. An example of a HAR file as viewed on the PCAP Web Per-

formance Analyzer website.

In looking at the overview above, it’s obvious why it’s referred to as a waterfall chart. The two vertical lines on the right should stand out. The blue line denotes the start of the rendering of the Web page. This is when the DOM is loaded and the Web page first becomes available to the end user. It will most likely still shift around a bit as assets continue to be added and modified. The red line corresponds to when the page has finished rendering and the browser’s onload event fires. Let’s look more closely at one of the loaded resources to get a better idea of what the HAR file tells us:

15. smashed.by/pcapperf

BY DAVE OLSEN

185

FIGURE 5.5. A close-up of the options available for an individual entry in a HAR file.

In this example, the blue bar shows the time that the browser took to look up the DNS information for the request. The green bar displays the time it took to connect to the server. The red line shows the time it took to send data to the server. The purple line represents the time that the browser waited to receive the first byte of data back from the server. And the gray shows the rest of the data being downloaded for use. HAR files contain other useful information such as compressed and uncompressed sizes for assets, as well as request and response headers. The PCAP Web Performance Analyzer also provides aggregated performance statistics. THE MOBILE PERFORMANCE BOOKMARKLET

Steve Souders, well known as a Web performance evangelist, has put together the ultimate mobile performance testing bookmarklet, Mobile Perf.16 He also lists a handy collection of Web performance bookmarklets that are useful and easy to use when evaluating performance on mobile, including YSlow,17 DOM Monster,18

16. smashed.by/mobileperf 17. smashed.by/yslow 18. smashed.by/dom-monster

186

OPTIMIZING FOR MOBILE

SpriteMe19 and CSSess.20 Overall, these bookmarklets can go a long way in helping you not only to evaluate the performance of your mobile website, but to take steps to resolve any issues you might find. SIMULATING BANDWIDTH AND LATENCY CONSTRAINTS

Testing responsive designs on wired or wireless laptops can give a false sense of how well your page performs. In order to see how well your website will perform under the suboptimal constraints of mobile networks, use one of the two tools that will allow you to modify your network connection. The first solution is Charles,21 a robust HTTP proxy and HTTP monitoring tool that also provides the ability to shape your network traffic. The best part of Charles, though, is that it allows you to capture and review the network traffic generated by your website. Similar to the previous example with Chrome, you can use it to save a HAR file and then use the PCAP Web Performance Analyzer to see where problems and bottlenecks might appear on higher-latency networks. As of this writing, Charles v3.6.5 costs $50. The second solution is Throttle.22 Throttle was developed at West Virginia University as a free and simple traffic-shaping tool that we could use in our company with Adobe Edge Inspect. Adobe Edge Inspect is a free tool that helps you to preview and remotely debug a responsive design on iOS and Android devices. While it doesnâ&#x20AC;&#x2122;t currently provide a HAR file, it does give us a rough gauge of performance as we work with clients and designers across our organization.

19. smashed.by/spriteme 20. smashed.by/cssess 21. smashed.by/charles 22. smashed.by/throttle

BY DAVE OLSEN

187

Mobile Optimization Techniques A lot of options are out there for improving the performance of your website. To help you decide which techniques would be most effective, I’ve selected some and organized them into three groups: 1. The essentials, 2. Next steps, 3. Deep cuts.

Each set of techniques can be implemented in whole, in part or not at all. This is simply my way of organizing techniques in the order that I think would have the most benefit for any given website. In my own workflow, this categorization has served as a checklist to move through when considering various optimization techniques for our websites. THE ESSENTIALS

Enabling file compression and caching headers on your server are two of the simplest things you can do to improve mobile performance. They really are “must do” performance tweaks, no matter what else you end up implementing. The techniques in this section will help you to: Ŧ Reduce page weight, Ŧ Reduce the number of requests to the server (addresses latency!).

Because these two techniques are implemented on the server, they will immediately improve the performance of every page load. It doesn’t get much easier than that.

188

OPTIMIZING FOR MOBILE

Enable mod_gzip or mod_deflate to Compress Files In the same way that we compress files on our computers to make them easier and faster to send to clients for review, servers can compress Web files before sending them to browsers. Because our mobile users might be surfing with limited bandwidth, we should make the files we send over it as small as possible. Accelerating the downloading of assets is an obvious win, but it also clears the downloads queue more quickly, thus speeding up the entire loading performance of the website. The two compression librar- At West Virginia University we’re using mod_deflate to compress ies for Apache, mod_gzip and our home page and its related mod_deflate, work best on text-heavy files such as HTML, JavaScript and CSS files. As of CSS and JavaScript. Image for- this writing, if those files were not compressed, a visiting user would mats such as JPEG are already need to download a total of 824K compressed, so the libraries in content. Instead, by compresswill have almost no effect on 23 ing the files, a client only needs to them. download 663K of content. That’s a One thing to keep in mind 20% savings in bandwidth use. when using mod_gzip is that it doesn’t work well on files smaller than 500 bytes, nor on files larger than 500 KB. While it will compress small files, the savings are so negligible that the payoff isn’t worth the effort of your server. Conversely, compressing large files on the server could take so long and delay the response of the website enough that, again, it might not be worth the effort. Enabling this feature is very easy. Below are simple configurations that you can use with Apache for mod_gzip and mod_deflates respectively:

23. Simple ways to further compress images are discussed in the “Next Steps” section of

this chapter. BY DAVE OLSEN

189

-- MOD DEFLATE -<IfModule mod_deflate.c> AddOutputFilterByType DEFLATE text/html text/plain text/ xml text/css application/javascript application/x-javascript application/rss+xml application/atom_xml </IfModule>

-- MOD GZIP -<IfModule mod_gzip.c> mod_gzip_minimum_file_size 500 mod_gzip_maximum_file_size 500000 mod_gzip_item_include file \.html$ mod_gzip_item_include file \.css$ mod_gzip_item_include file \.js$ mod_gzip_item_include file \.php$ mod_gzip_item_include file \.py$ mod_gzip_item_include file \.txt$ mod_gzip_item_include file \.xml$ mod_gzip_item_exclude file \.png$ mod_gzip_item_exclude file \.jpg$ mod_gzip_item_exclude file \.gif$ mod_gzip_item_include mime ^text/html$ mod_gzip_item_include mime ^text/plain$ mod_gzip_item_include mime ^text/xml$ mod_gzip_item_include mime ^text/css$ mod_gzip_item_include mime âpplication/javascript$ mod_gzip_item_include mime âpplication/x-javascript$ mod_gzip_item_include mime âpplication/rss+xml$ mod_gzip_item_include mime âpplication/atom_xml$ mod_gzip_item_exclude mime îmage/ </IFModule>

190

OPTIMIZING FOR MOBILE

You can use RedBot 24 to see whether your website is already using compression. If you’re interested in delving deeper into this topic, there is much more to learn about mod_gzip and mod_deflate on their respective websites.25

Make Sure Assets Are Cached by the Browser Whether CSS files, images or JavaScript files, the vast bulk of the content we create for our Web pages is static. Once we’ve launched a website, we tend to change these assets very rarely. For mobile, if we can make sure these static assets are downloaded once and then cached on the device, we get two wins: fewer total requests made in future page loads, and less total content to be downloaded. This way, on subsequent visits, these users will have a primed cache, and their experience will be that much faster and, consequently, better. The simplest trick to make sure users get the primed cache state for their browser is to make sure that your Web server’s Expires header is set and, importantly, that it’s set to an appropriate date far into the future. This expiration date should be no sooner than one month and no longer than one year into the future. If you know that a particular resource will change every two weeks, then you can tweak the Expires header for that asset to make sure it gets refreshed more often. But this exception highlights a serious downside to this type of aggressive caching on the client. The better option would be to rename the resource when it’s updated, which would then generate a new request from the client. The following is a simple mod_expires configuration for Apache that you can use to cache your files for an appropriate length of time.

24. smashed.by/redbot 25. smashed.by/mod_gzip and smashed.by/mod-deflate.

BY DAVE OLSEN

191

<IfModule mod_expires.c> ExpiresActive on ExpiresByType text/html "access plus 5 minutes" ExpiresByType image/jpg "access plus 6 months" ExpiresByType image/jpeg "access plus 6 months" ExpiresByType image/gif "access plus 6 months" ExpiresByType image/png "access plus 6 months" ExpiresByType text/css "access plus 6 months" ExpiresByType text/javascript "access plus 6 months" ExpiresByType application/javascript "access plus 6 months" ExpiresByType application/x-javascript "access plus 6 months" ExpiresByType application/x-icon "access plus 1 year" </IfModule>

The one limiting factor of this performance optimization technique for mobile devices is the size of the device cache itself. For the most part, and especially on older OSes and devices, the cache is relatively tiny. The total cache of Android 2.x’s stock browser can hold only 5.7 MB’s worth of content.26 iOS is better at 50 MB, but that pales in comparison to the kind of cache sizes we have on desktop, especially when you consider that a single website can take up to 4 MB worth of that 50 MB total. Still, attempting to cache files is better than not, especially as mobile browser- and device-makers beef up their cache sizes. One workaround for the cache size limitation is to use localStorage as a file cache. We’ll touch on this technique in more depth in the “Deep Cuts” section.

26. Joshua Bixby, “Early findings: Mobile browser cache persistence and behaviour,”

Web Performance Today, 12 July 2012, smashed.by/webperformancetoday.

192

OPTIMIZING FOR MOBILE

NEXT STEPS

File compression and browser caching are very good first steps, but they’re merely that—first steps. To get your website performing optimally, you can take more advanced steps that, while requiring more work, could have a much more dramatic performance gain for your users. The techniques in this section will help to: Ŧ Reduce page weight, Ŧ Reduce the number of requests to the server (addresses latency!), Ŧ Optimize when content is loaded so that your website becomes

usable more quickly. Two of these techniques, minification and concatenation, should probably only ever be implemented when you are moving from a development environment to a staging or production environment. They can be unwieldy when multiple and frequent updates are required.

Make Your Files Smaller With Minification Minifying code simply means removing extra bytes from a file’s size by taking out unnecessary things such as white space, line breaks and indentation. Of course these things make it easier for us to read a file but they aren’t actually necessary for browsers to parse and understand the code. With minification, the browser downloads less content, translating into quicker overall downloads and asset loading. Thus the rendering time for your Web page will also be faster. If you’ve used any popular JavaScript library (jQuery, for example), then you’ve most likely used the minified version. Because of this, many developers assume that minification can only be used with JavaScript, but, technically, it can be applied to any text-based resource. The most important asset we should consider minifying in production code is CSS. BY DAVE OLSEN

193

A number of resources are available for minifying CSS and JavaScript files. The oldest and possibly best is YUI Compressor27 by Yahoo. If you’d rather not have to worry about downloading and installing it, you can also use an online version of it.28

One File to Rule Them All: Combine Multiple Files with Concatenation Minification’s performance twin is concatenation. If you know that a Web page will need a large number of JavaScript or CSS files, you can not only minify them into smaller individual packets, but also string them together into several larger files to increase performance. For example, rather than needing to download 10 JavaScript files—therefore requiring 10 connections to the server—users might download only 4 separate files that are some combination of the original 10. After Expires headers, concatenation might be the single best technique for opening up latency bottlenecks on mobile networks. Unfortunately, it also comes with some downsides. As it turns out in practice, overoptimizing when using concatenation is possible, too. Unless your JavaScript or CSS files are very small, don’t lump them all together into one big file. By keeping them partially separate, you’ll take advantage of the browser’s capacity to download multiple files at once, on average up to six. Bryan McQuade does a good job of discussing this issue in his Velocity 2011 conference talk, “Website Acceleration With Page Speed Technologies.”29 Just as with minification, YUICompressor is a great resource for concatenating files. If your Web application uses PHP, you can use QuickConcat,30 developed by the Filament Group.

27. smashed.by/yui-compressor 28. smashed.by/yui-online 29. smashed.by/velocity2011 30. smashed.by/quickconcat

194

OPTIMIZING FOR MOBILE

Minify: The PHP Minification and Concatenation Library As recommended earlier in this chapter, minify and concatenate files only when styles and code are production-ready. As we discovered recently, if you happen to use PHP, though, you can take advantage of the great library Minify,31 which minifies and concatenates files on the fly. Let’s look at how we’re using Minify with our CSS files on West Virginia University’s home page: <link rel="stylesheet" href="/min/?b=css&amp;f=base. css,supersized.css,supersized.shutter.css,layout. css,option-3.css">

Minify automagically compresses and minifies the files that we’ve listed in request variable f. We can continue to modify these files as we normally would without having to worry about optimizing them. Minify is also available as a WordPress plugin.32

Inlining Small JavaScript and CSS Files If your website requires only a bit of JavaScript and/or CSS in order to work properly, then consider simply embedding the code on the page, rather than linking to it. Any extra requests are not worth the overhead, performance-wise, for anything less than 4 KB. Just make sure to place the inlined CSS in the head of your document and the inlined JavaScript at the end of your document. Image Optimization As shown earlier, images are the number one source of bloat on the Web. They account for 64% of the average weight of a Web page. If a developer or designer can reduce the footprint of their images, 31. smashed.by/minify 32. smashed.by/minify-wordpress

BY DAVE OLSEN

195

then they’ve taken a big step in optimizing their Web page’s performance. Image optimization is a two-step process. The first thing to do to optimize your images is to choose the appropriate format. For almost all of your images, you can use PNG, with the color depth set to the lowest acceptable level. If you’re optimizing a photograph, use JPEG. Again, focus on finding the lowest acceptable setting for quality. Don’t just set it to some high-quality default. Even when an image is saved with something like Photoshop’s “Save for the Web…,” it can contain extra information, such as unnecessary comments or incorrect color palettes, which makes the file’s size larger than it needs to be. To reduce an image’s size even further, the second thing to do is run it through an optimizer that strips out that extra information. The best Web-based tool for this, in my opinion, is kraken.io.33 Another good resource is Smush.it34 by Yahoo. If you need an image optimization utility that you can use offline and you’re on a Mac, check out ImageOptim.35 On Windows, one of the most helpful offline tools is Radical Image Optimization Tool36 (RIOT). Make sure to play around with the settings to find the combination that gives you both the best file size and the quality you’re looking for. Another way to optimize images is to use sprites, which we’ll cover in the “Deep Cuts” section. If you’re really into image optimization, make sure to read Sergey Chikuyonok’s two articles on the subject, “Clever JPEG Optimization Techniques”37 and “Clever PNG Optimization Techniques,”38 on Smashing Magazine. Both articles give great tips on how to deliver 33. smashed.by/krakenio 34. smashed.by/ysmushit 35. smashed.by/imageoptim 36. smashed.by/riot 37. smashed.by/jpeg-techniques 38. smashed.by/png-techniques

196

OPTIMIZING FOR MOBILE

high-quality and highly optimized images, based on a deep understanding of how each format works.

Defer Loading of Content When Possible All of the techniques listed above, with the exception of concatenation, will help to reduce overall page weight. Another possibility is to prioritize the loading of assets. By prioritizing assets, we aren’t necessarily improving performance through a cache or bandwidth, but we can make the page appear to load faster. The easiest type of asset to defer is a JavaScript file, using the HTML5 defer and async attributes of the script tag. <script defer src="script-1.js"></script> <scrypt async src="script-2.js"></script>

From a downloading and latency standpoint, the defer and async attributes behave very similarly. When the browser parses your Web page and encounters the script tag, it will attempt to download the referenced files. The defer and async attributes tell the browser that the linked files should be executed after the browser has finished loading all of the other content on the Web page. So, which one should we use? When deciding which attribute to use, the question to ask is whether order matters when loading the linked JavaScript files. If it does, then the defer attribute is better. All tags with that attribute will be executed in the same order as the script tags are listed. Suppose you reference two JavaScript files in order: the first references jQuery, and the second references custom JavaScript that uses functionality from jQuery. Because the second script must be loaded after the first, you’d need to use the defer attribute on both in order for your website to work. If order doesn’t matter, then you are better served by the async attribute. BY DAVE OLSEN

197

But what do you use if you need to support an older browser that doesn’t support the defer and async attributes? Then, you can dynamically create script elements using JavaScript. var js = document.createElement('script'); js.src = 'script.js'; var head = document.getElementsByTagName('head')[0]; head.appendChild(js);

Read more about this fallback in Stoyan Stefanov’s article “NonBlocking JavaScript Downloads.”39 The defer attribute is supported by all major desktop and mobile browsers except for the Opera family of browsers.40 The async attribute shares a similar level of support, with the notable exception of Internet Explorer; async support was added in IE 10.41 DEEP CUTS

The previous two sections, “Essentials” and “Next Steps,” cover most of the “traditional” performance optimization techniques for responsive websites. Considering that RWD is itself rather cuttingedge, let’s discuss now a number of HTML5-related and forwardlooking techniques that will improve the performance of your website in mobile browsers.

39. smashed.by/non-blocking 40. It is supported in IE7+, Firefox 14+, Chrome 21+, Safari 5.1+, iOS Safari 5.0+ and Android 3.0+,

but not currently in Opera or Opera Mini: smashed.by/defer-support. 41. Support is very similar to the support of defer outlined above. However, in IE’s case, only IE 10+ supports it: smashed.by/async-support.

198

OPTIMIZING FOR MOBILE

The techniques in this section will help to: Ŧ Reduce page weight, Ŧ Reduce the number of requests to the server (addresses latency!), Ŧ Optimize when content is loaded so that your website becomes

usable more quickly. The primary downside of the techniques listed in this section is a lack of support in some old mobile browsers. Unfortunately, there aren’t many workarounds for them (at least for now), so if you are primarily concerned with legacy mobile browsers, thoroughly study and research the techniques before applying them in your projects.

Correctly Ordering Your CSS and JavaScript Surprisingly, simply ordering your CSS and JavaScript correctly can have a large impact on the performance of your page. Because JavaScript can alter the content and layout of a Web page, in many browsers it will block the rendering of any content that appears after the referenced JavaScript file has been downloaded, parsed and executed. This includes even the downloading of certain content that appears after the JavaScript reference. Let’s look at a quick example of this type of blocking in action. Here is the <head> of a typical website, where we are referencing two style sheets and a JavaScript file. Let’s assume that in the <body>, we reference an image. <head> <script src="jquery.js"></script> <link rel="stylesheet" href="small.css"> <link rel="stylesheet" href="big.css"> </head>

BY DAVE OLSEN

199

Here is the order in which the browser would download these files:

FIGURE 5.6. This waterfall chart shows the image download being blocked by the

JavaScript located in the head.

The JavaScript file blocks the downloading and, importantly, the parsing and rendering of the DOM elements that follow—in this case, an image. A better option would be to put the JavaScript just above the closing body tag. When we do that, we can see that the image loads at the same time as the JavaScript file.

FIGURE 5.7. This waterfall shows that the image is being downloaded in parallel to

the JavaScript file, now that the latter was moved to above the closing body tag.

The only time you’d want to reference a JavaScript file higher up on the page would be if you were using document.write() to inject markup directly in the Web page.

Using localStorage as a Browser Cache As discussed in the “Essentials” section, using the Expires header is the best and simplest way to cache static assets in the browser. But big Web brands such as Google and Bing are now turning to a new technique, localStorage, to cache their static content in the browser. With Bing, Microsoft has reduced its footprint from 200 200

OPTIMIZING FOR MOBILE

KB on the first file request to 30 KB for subsequent requests by using localStorage. localStorage is a part of the W3C’s “Web Storage” specification.42 Essentially, it is similar to browser cookies in that it enables developers to save data in key value pairs that persist in the browser even if it’s closed. Unlike cookies, data stored in localStorage will never be sent across the network to the server. Here’s an example of how you can use localStorage right away. Suppose we want to load jQuery out of localStorage. When the user first loads the Web page, we check to see whether jQuery has already been stored. If it has, then we simply get the data out of localStorage. If it hasn’t, then we need to do an AJAX call to get the data from the server. Once we have it, we can push it into localStorage so that it’s available for future requests. We then push the code into the selected script tag, where it will be parsed.  <script id=”jquery”></script>  <script> // set-up var to hold script var jqfile; // determine if the script has already been stored in localStorage if (‘jqFile’ in window.localStorage) { // it has so grab the file jqFile = window.localStorage. getItem(‘jqFile’); } else {

42. smashed.by/w3webstorage

BY DAVE OLSEN

201

// it hasn’t, AJAX call to get the data from the server jqFile = getJQFile(); // push the data into localStorage window.localStorage.setItem(‘jqFile’,jqFile); } // insert the JavaScript into the appropriate script element document.getElementById(‘jquery’).text = jqFile; </script>

Easy, right? Note that localStorage can only store strings. This means that you can only cache an image if you’ve saved it as its Base64 string equivalent. localStorage is very easy to implement. I expect more and more JavaScript libraries and websites to take advantage of this feature. In the meantime, you can try it out by using a small library such as the Memcache emulator lscache43 or the script-loader basket.js.44 The former was written by Pamela Fox and enables developers to set an expiration time for items stored in localStorage. The latter was written by Addy Osmani and is a proof-of-concept script loader that caches downloaded JavaScript files.

Lazy Loading JavaScript If you have a JavaScript-heavy mobile website, consider implementing a lazy load technique to reduce its startup time. When a browser downloads a JavaScript file, it is also parsed and executed. As discussed earlier, the browser will block the rendering of the Web page until this process is done. In most cases, you probably don’t require all of your downloaded JavaScript to be parsed and executed 43. smashed.by/lscache 44. smashed.by/basketjs

202

OPTIMIZING FOR MOBILE

on page load, so it could impair For me, the most disappointing feature your website’s performance for of HTML5 so far has to be AppCache. Considering that a W3C community no good reason. 45 group45 is dedicated to fixing it, I’m Traditionally, there are two obviously not alone in my feelings. ways to lazy load JavaScript. It used to hold great promise to help The first method, shown earus address Web performance issues, lier, uses JavaScript to insert but it has never lived up to it. One script tags into a document’s issue is that you cannot see any new <head> tag. The second methfiles added to the AppCache until od utilizes XmlHttpRequest after the page reloads, requiring the to download and evaluate entire cache to be downloaded before code. The primary downside to saving it, meaning that users on slow these techniques is that a user connections might never end up with might want to access a bit of a “primed” AppCache. Also, Appfunctionality that hasn’t been Cache is not addressable for granular downloaded yet, so the page updates. Jake Archibald does a good might appear to break or to be job of breaking down other issues with broken already. In response AppCache in his article Application to this issue, the team behind Cache Is a Douchebag.46 My advice Gmail for mobile pioneered regarding AppCache for now is simple: stay away from it. And if you decide a new way to lazy load JavaSto use it, study the issues related to cript that makes a lot of sense 4647 mobile browser cache sizes.47 for mobile developers. With Gmail’s lazy load technique,48 your user will first download all of the Java-Script needed to interact with the page, so all functionality is available on page load. The difference is that much of the downloaded code is commented out, so the browser won’t parse it. Whenever the user 45. smashed.by/w3fixing-appcache 46. smashed.by/appcache-douchebag 47. smashed.by/mobile-cache and smashed.by/mobile-cache-sizes. 48. smashed.by/google-codeblog

BY DAVE OLSEN

203

needs a particular feature that is commented out, the appropriate snippet of code is uncommented and evaluated. When the Google team first tested the technique, they found that loading 200 KB of JavaScript and parsing it during page load added 2600 milliseconds of loading time. For an application like Gmail, which requires only a small subset of its downloaded JavaScript at page load, this delay in rendering was not only unnecessary, but undesirable. With its clever lazy load technique, that same 200 KB of JavaScript ended up adding only 240 milliseconds of time to the page load: a tenfold increase in page loading time! Here is an example: <html> ... <script id="lazy"> // Make sure you strip out (or replace) comment blocks in your JavaScript first. /* JavaScript of lazy module */ </script> <script> function lazyLoad() { var lazyElement = document.getElementById('lazy'); var lazyElementBody = lazyElement.innerHTML; var jsCode = stripOutCommentBlock(lazyElementBody); eval(jsCode); } </script> <div onclick=lazyLoad()> Lazy Load </div> </html>

204

OPTIMIZING FOR MOBILE

This technique probably only makes sense for projects with JavaScript-heavy mobile websites on which developers have done a good job of compartmentalizing their JavaScript.

The Two-Click Social Widget In the fall of 2011, German online magazine Heise rolled out a new set of social media icons. The difference between the new icons and old ones? The user is now required to click twice to “Like” a story, “Tweet it” or “+1” it. Heise loads its own small icon by default, and loads the Facebook, Twitter and Google+ social widgets only when the user explicitly chooses to take that action to share the content. The primary purpose of this move was to make sure that social media networks cannot track their users—thereby keeping Heise in compliance with EU and German privacy laws. The technique has more applications than just addressing privacy concerns, though. This technique would work well with RWD for two reasons. First, it obviously cuts down on bandwidth usage. For example, at the moment of writing, Facebook’s Like button requires approximately 120 KB worth of JavaScript and HTML. Secondly, it cuts down on latency. The Like button requires four separate requests, with each affected by our mobile network latency bottleneck. By cutting out the social media button overhead, Heise’s two-click social widgets49 go a long way to speeding up the initial page loading of content, while still allowing users to share content if they choose to. Embedding Images and Fonts Using the Data URI Scheme In my opinion, one of the most useful features implemented in modern browsers is the data URI scheme. It might be the most underutilized feature actually. The data URI scheme enables developers to embed binary data, such as images and fonts, directly in their Web pages and style sheets. The latter is shown in this example: 49. The code of Heise’s two-click social media widgets is open-sourced: smashed.by/heise.

BY DAVE OLSEN

205

#masthead { background-image: url(data:image/png;base64;iVBORw0KGgoA AAANSUhEUgAAAQwAAAAoCAYAAAALxsQHAAA...); }

The most significant benefit of data URIs is that we can reduce extra HTTP requests. Again, we’re trying to avoid the latency bottleneck whenever possible. That being said, this feature is only efficient for small files (less than a few KB) and for sprites that you use across all breakpoints. The Base64 encoding that’s normally used with the data URI scheme leads to embedded image files that are larger in size than linked ones. Also, by embedding images in the CSS, they are downloaded whether the browser needs to render them or not. The other important downside to this method is that your images will need to be re-encoded and embedded every time they’re changed. Assuming you use this technique only with production websites whose sprites, icons and mini-graphics don’t change very often, this shouldn’t be that much of an issue. The Base64 Image Encoder50 is a handy online tool for converting images to Base64 encoding. And Font Squirrel offers a free tool51 to convert fonts to Base64 encoding.

Using Sprites to Concatenate Images JavaScript and CSS aren’t the only assets that we can concatenate to reduce the total number of requests. Of course, we can also join images together into sprites. Usually, using sprites is a good idea when either of the following is true:

50. smashed.by/base64-image 51. smashed.by/fontsquirrel

206

OPTIMIZING FOR MOBILE

Ŧ A set of images is always loaded together (for example, a set of

icons for actions in a Web application that need to be loaded together as a sprite); Ŧ Very small random images would, on their own, cause a lot of

overhead per request. Also, try to sprite images based on color palette. Basically, only sprite together images that share the same 256 colors or fewer. If a sprite image is made up of images that don’t share the same color palette, it will end up being unnecessarily large because it would be saved with the PNG truecolor type, rather than a simpler color palette. Remember, if your sprite isn’t overly large, you could also use the data URI scheme to embed it in your CSS, thereby eliminating another request. To create sprites, you can use an online service such as SpriteMe or CSS-Sprit.es.52

Avoid Browser Reflow One feature that often gets overlooked when addressing a website’s performance is understanding how the browser “paints” a website in the browser window. Your CSS, JavaScript and inline objects (such as images) all provide the browser with information on how the page should be laid out. Performance problems crop up when we try to dynamically modify the layout too much. This forces the browser window to reflow and recalculate where objects should be laid out as the page is loaded—thereby slowing down the final rendering of the page. The most important thing to do to limit the number of reflows is to make the DOM elements and CSS as lean as possible. The cleaner the DOM is, the fewer the elements that will need to be recalculated and laid out during page loading. In a similar vein, the simpler the

52. smashed.by/css-sprites-gen

BY DAVE OLSEN

207

CSS rules are, the fewer number of times the DOM will need to be modified. But one trap that is often overlooked is that modifying styles via JavaScript can have a heavy-handed effect on reflowing. Take the following bit of code: var el = document.getElementById("foo"); el.style.color = "#fff"; el.style.backgroundColor = "#000"; el.style.borderColor = "#fc0";

This is how some developers might modify style attributes in their JavaScript. Unfortunately, each style declaration causes a reflow. In our example, we’ve reflowed and blocked the rendering of our page three times. A better solution would be to simply add a class name to the element: var el = document.getElementById("foo"); el.className = "bar";

By doing this, the page requires only one reflow. The only gotcha in this scenario is if you’re trying to modify styles that you’ve defined using an ID selector with the newly applied class styles. Because the ID declaration is more specific than the class declaration, the latter won’t overwrite the styles by default. You might need to add the !important declaration in your class styles. The reflow issue is also affected by the dynamic addition of elements to the DOM. For example, rather than adding list items to an unordered list one at a time, we should instead add the list items to a document fragment and then insert that document fragment into the DOM only once.

208

OPTIMIZING FOR MOBILE

KeeKim Heng does a good job of explaining how to properly accomplish this in his article “Speeding Up JavaScript: Working With the DOM.”53

Touching Is Better Than Clicking on Mobile As shown in the previous section, not all mobile performance issues are directly related to traditional factors such as bandwidth and latency. Some issues affect the perceived performance of our websites on mobile. One example of this is the 300-millisecond delay that WebKit browsers add to the execution time of onclick events. The purpose of this delay is to make sure that the user has a chance to finish a gesture such as a double-tap before the event fires. To users who are used to the lightning-quick reaction of other touch events, this delay can prove irritating when the object they’re tapping on obviously requires only a single tap. To get rid of this delay, swap your onclick events for ontouch events on page load. A good resource to learn how to make the swap is Google’s tutorial on creating fast buttons.54 It goes into depth on why you’d want to make the swap and provides sample code so that you can easily implement it. Go Micro and Drop Big Frameworks Just because JavaScript is not the biggest contributor to today’s explosion in page weight, it shouldn’t be given a free pass when looking at areas to optimize for performance. Laying the blame entirely on projects such as jQuery, MooTools and other “everything and the kitchen sink” JavaScript libraries is a bit presumptuous. Without a good review, it’s difficult to say where all of the JavaScript-related weight has come from. That being said, developers should think critically about how they’re using these libraries and about whether smaller, more effective libraries are out there. 53. smashed.by/javascript-dom 54. smashed.by/fast-buttons

BY DAVE OLSEN

209

If you use jQuery merely as a selector tool (for example, $('element')), then you might want to consider libraries such as Zest and Qwery, which are designed specifically for this task. 11 KB (versus 93 KB) can make quite a large difference to the overall weight of a page. Quite a number of these micro-frameworks—available on Microjs55—are focused and small (much smaller than 11 KB even) and might satisfy many of your needs. But page weight isn’t everything, and there is another benefit to using a micro-framework. As we discussed in the lazy load section above, when a browser downloads a framework like jQuery, the browser must parse it so that other scripts can use its functions. This parsing takes time and blocks the rendering of the Web page. By using a (literally) smaller framework, we can reduce the impact of this blocking. For instance, Stoyan Stefanov found that a microframework such as Zepto loads 56 times faster than jQuery on an iPhone 4 running iOS 5.1.1.56 Be critical when reviewing your JavaScript needs. The 11 KB selector micro-framework itself might be overkill, if document. getElementById()is all you really need.

Rethinking Mobile Optimization for RWD While all of these traditional techniques, and even some of the newfangled ones, improve performance with RWD, they still don’t necessarily address the “dark matter” of unnecessary styles, scripts and, most importantly, images. They also don’t deal with the underlying structural performance problem of responsive design: that is, delivering to the end user a single dumb page of markup that may be loading third-party ads and social widgets or rich media such as video. The techniques outlined above will help to alleviate these performance issues, but they don’t really solve them. 55. smashed.by/microjs 56. Stefanov, Stoyan. “JavaScript Performance Patterns,” smashed.by/javascript-patterns.

210

OPTIMIZING FOR MOBILE

There is a new technique that can help us better address these issues. Rather than rely completely on the browser to determine which elements to show and which to load, we can resurrect a method that has been used forever to deliver mobile-friendly content. INTRODUCING RESS: RESPONSIVE DESIGN + SERVER-SIDE COMPONENTS

Let’s start with a case study. RWD has become the go-to mobile solution at West Virginia University—a project that we have worked on recently. Its benefits are obvious: the website will have one URL for mobile and desktop content; it tends to play well with content management systems; and the content manager is able to effortlessly deliver mobile-friendly content. It’s easy on our clients and, for the most part, easy on us. That said, as I’ve deployed complex responsive websites, I’ve found that RWD presents its own challenges. It’s also obvious that I’m not the only one who runs into these problems. Every day you stumble upon new ways to smoothen out some of the rough edges of implementing websites with RWD. After seeing Stephanie and Bryan Rieger’s talk, “Adaptation,”57 at Breaking Development in September 2011 and playing with their Profile tool58 and, most importatly, learning that 82% of Alexa’s top 100 properties still use server-side detection to modify some amount of their content,59 I started to believe that maybe there was a different way to deliver responsive websites. The clincher for me, though, in deciding that server-side detection still holds value and can be combined with RWD was Luke Wroblewski’s article “RESS: Responsive Design + Server Side Components.”60 57. smashed.by/slideshare-adaptation 58. smashed.by/github-profile 59. Cremin, Ronan. “Server-Side Mobile Detection Used by 82% of Alexa Top 100 Sites,”

CircleID, 11 Jan 2012, smashed.by/circleid. 60. smashed.by/ress BY DAVE OLSEN

211

The principal concept that I found so intriguing in Luke’s article is the following (emphasis added):

“

In a nutshell, RESS combines adaptive layouts with server side components (not full-page) optimization. So a single set of page templates define an entire Website for all devices but key components within that site have device-class specific implementations that are rendered server-side.”

Luke is suggesting that our overall layout may be governed by conventional RWD principles, but that, before sending it to the browser, we can intelligently insert small bits of customized content in the layout where appropriate. That way, the final markup is as optimized as possible—thus combining “responsive design” and “serverside components,” as Luke describes the technique. The beauty of RESS is that, rather than forcing developers to choose one technique or another, it attempts to combine the best of both worlds. This enables developers to create unique and flexible solutions by changing markup order, media and even functionality, making the most of the capabilities of the requesting browser. So, how would a website developed using RESS work? AN EXAMPLE RESS SOLUTION: MODIFYING LAYOUT

The performance issues I keep returning to again and again are those related to images. RESS is the perfect tool for addressing these concerns. At its simplest, RESS can help you deliver file sizeoptimized images, but it can also be expanded to deliver completely different layouts and images when customized advanced content design is required. Let’s look at a very simple example that addresses the latter use case.

212

OPTIMIZING FOR MOBILE

The Problem Suppose a client has asked us to develop a website that prominently features three images that they want to be able to swap out on their own after launch (see the image below). The website is required to be mobile-friendly. Heeding the client’s wishes, we’ve developed a design that does, in fact, feature three large images, and we’ve used responsive markup so that the images scale across various screen sizes. This has introduced two problems, though. The design calls for us to deliver desktop-sized images to mobile devices, which is suboptimal for performance (given the downloading and scaling issue we identified earlier), and because we’ve had to stack our layout so that the three modes of transportation remain highlighted according to our client’s wishes, the page is feeling long. The client has directed us to rectify this.

FIGURE 5.8. Our desktop (left) and mobile (right) mockups.

With those constraints, we’ve developed two mockups. On the left is the traditional desktop mockup. On the right is the mobile mockup with the adjusted “choreographed” content—the three other images of different sizes. Once they’re approved, we will obviously need to create different markup and templates for each. The thing is that the markup is really only different in that small bit involving the images. So, how should we go about delivering our responsive design with the markup for the images and with the images themselves, altered BY DAVE OLSEN

213

based on screen size? We have two options for the half of RESS dealing with server-side components: traditional browser detection, and the new kid on the block (discussed further below): server-side feature detection. BROWSER DETECTION ISN’T THE DEVIL

Upon hearing the suggestion that browser detection is a viable mobile solution, I’m sure many of you (or at least more than several) are thinking that I can’t be serious. “Hasn’t it been settled that useragent sniffing is unreliable? Feature detection is the best solution!” The fact of the matter is that browser detection is a tried and true technique when used for general browser classification. Is browser detection infallible? No. Should a browser-detection library be updated regularly just in case? Yes. Despite all the talk that keeping up with the user-agent strings for new browsers on the market is an arms race, feature detection can be plagued with a similar issue. For example, as of this writing, WebKit and Opera each has its own prefixed version of min-device-pixel-ratio that it implements differently (using the formats 1.5 and 3/2, respectively). Mozilla supports its own prefixed version of min-device-pixel-ratio, which matches the WebKit version, but it’s moving to a new method, min-resolution, which Internet Explorer also supports. So, just as with browserdetection libraries, developers will need to maintain and update their feature-detection tests to stay on top of the latest changes to browsers. That being said, the most important criterion in determining the success of either browser detection or feature detection is not in how well either performs its particular type of detection without regular updates, but rather in how each fails and how a website handles that failure. One of the advantages of RESS when using browser detection is that, even if a browser is misclassified, the 214

OPTIMIZING FOR MOBILE

fallback to the default responsive view shouldn’t be so terrible that features are unworkable when presented to the end user. From commercial vendors like DeviceAtlas61 and 51Degrees. mobi62 to open-source projects like WURFL63 and the library that I contribute to, ua-parser,64 we have quite a few options to choose from that are proven, robust and continually updated. But browser detection isn’t the only way to decide which template to deliver when using RESS. Server-side developers can also leverage feature detection to make that determination. TAKING FEATURE DETECTION SERVER-SIDE

While WURFL can be used simply as a browser-detection library, it also offers a wealth of data on mobile devices. For example, a developer could use it to customize a mobile experience based on the availability of the Geolocation API for the visiting device. That said, WURFL tracks only so many device features. As noted earlier, it also requires periodic refreshing to ensure the data is up to date. There are good reasons why feature detection is now seen as the “proper” way to interrogate browsers. One interesting question that WURFL brings up regarding feature detection, though, is how often do we really need to run feature detection? In his article “Cutting the Interrogation Short,”65 Alex Russell suggests, “Feature tests should only ever be run when you don’t know what [user agent] you’re running in.” Essentially, for any particular version of a browser, the vast majority of the browser’s features should not change from user to user. Rather than using feature detection on every page load, we could instead run those tests only once per browser session or—to extend the concept—run 61. smashed.by/deviceatlas 62. smashed.by/51degrees 63. smashed.by/wurfl 64. smashed.by/ua-parser 65. smashed.by/cutting-short

BY DAVE OLSEN

215

the tests only once per unique user-agent string, and then store that information on the server for later retrieval, to be used in our serverside code. Thus can we use server-side feature detection.

The Simple Server-Side Feature Detection Example: ServerSide Breakpoints One of the primary uses of media queries today is to provide breakpoints for RWD. For example, if a particular viewport is 380 pixels or narrower, then the developer might want to show a particular set of styles to make their pages more touch-friendly or to alter the layout in some way. Using that same viewport information, we could also enable the server to decide what type of markup to send or what kind of features to enable. Server-side breakpoints are very easy to set up. The first thing we’ll do is write a cookie, using JavaScript, that tells us the current width of the window. We need to do this only for the first request. After we’ve gathered this information, we can redirect the user so that our server-side code is able to take advantage of the information. var width = (window.innerWidth > 0) ? window.innerWidth: screen.width; document.cookie = "sitewidth=" + width; document.location.reload();

I’m using PHP for this example, but any server-side language that can read a cookie and set a session is capable of taking advantage of this technique. if (isset($_COOKIE["screenwidth"]) { $_SESSION["screenwidth"] = $_COOKIE["screenwidth"]; }

216

OPTIMIZING FOR MOBILE

With the screen width in hand, we can modify parts of our markup and include different types of templates. if ($_SESSION["screenwidth"] <= "380") { include("includes/mobile-nav.inc.php"); } else { include("includes/desktop-nav.inc.php");

} For example, we could include templates for our transportationrelated images, thus addressing the problem we encountered earlier. While the session cookie makes for a very simple solution, it does require a reload for any visitor coming to your website who doesnâ&#x20AC;&#x2122;t already have a browser session open.

The Robust Server-Side Feature Detection Example: ServerSide Modernizr Screen width is only so reliable when it comes to deciding on support for browser features. Having a large number of focused tests would help us make more informed decisions. One tool that many front-end developers use to directly test browser features is Modernizr. As pointed out by Alex, do we really need to run our Modernizr tests on every page load? James Pearce pioneered the use of Modernizr66 on the server in 2010 when he released his PHP library, modernizr-server.67 Using modernizr-server as a base and combining it with ua-parser,68 a browser-detection library, I released my own server-side Modernizr implementation, Detector,69 in January 2012. 66. smashed.by/modernizr 67. smashed.by/modernizr-server 68. smashed.by/ua-parser 69. smashed.by/detector

BY DAVE OLSEN

217

Detector is an attempt to realize Alex’s suggestion. At its simplest, it uses Modernizr to profile each unique user agent that visits a website, stores the information on the server and then gives the serverside code access to the information. Future requests from browsers with that user agent can then rely on the stored data. A very simple way to use the Modernizr data server-side would be to present a YouTube video on the page according to browser support. require("lib/Detector.php"); if ($ua->video->h264 || $ua->video->webm) { print $html5Embed; // YouTube's <iframe> code } else { print $simpleLink; }

In this simple example, we’ve use the video property to decide which type of YouTube video the end user’s browser supports. Alternatively, we could use it to solve our image template problem from earlier. require("lib/Detector.php"); if ($ua->screenattributes->width > 320) { // our images for screens larger than 320px include("templates/desktop_images.html"); } else if ($ua->csstransforms) { // our art-directed images for 320px devices with a modern browser include("templates/mobile_advanced_images.html"); } else { // text links for basic mobile devices include("templates/mobile_basic_images.html"); }

218

OPTIMIZING FOR MOBILE

By combining Modernizr with a logic-less template language such as Mustache, we can build intelligent and easy-to-maintain templates easily. A much longer explanation of how to combine both is available on Detectorâ&#x20AC;&#x2122;s GitHub wiki.70 Another benefit of using Modernizr is that we can easily expand and modify feature detection as appropriate. There is no need to wait and hope that a centralized library will add the feature we want to do template switching with.

Remember the Vary Header When using RESS to modify markup based on the user agent yet serving content from the same URL, be aware of two issues. The first is that content delivery networks (CDNs) could possibly cache content for user agent X and deliver it to user agent Y. The second is that Google has a mobile bot that it uses to index pages that it knows are mobile-friendly. To address both issues, make sure your server is sending the Vary header with the user-agent option turned on. You can use REDbot to review the response headers sent by your server to see if the server is appropriately setting that Vary user-agent option. If it isnâ&#x20AC;&#x2122;t, simply modify your Apache configuration or .htaccess file with the following: Header append Vary User-Agent

With this single line, CDNs will properly cache your content, and Google will know to crawl your website with its mobile bot.

70. smashed.by/detector-tutorial

BY DAVE OLSEN

219

RESS IS NOT JUST A MOBILE VS. DESKTOP SOLUTION

RESS does not have to be a mobile-only solution. Yes, it lends itself to mobile-related issues, but it can be used to deliver optimized experiences to any browser. In the same way that Modernizr.load can help provide polyfills to older browsers, server-side feature detection and RESS can and do offer similar capabilities. They offer a very robust platform for server-side developers who build futurefriendly websites and applications.71 That said, there is still work to do to make these solutions as robust as possible. ALSO, RESS IS NOT A SILVER BULLET

Just as RWD is not a silver bullet for delivering mobile solutions, I don’t want to give the impression that RESS is either. It has its uses— hopefully as shown as above—but it certainly has its limitations. First, there is the maintenance aspect of the solution. If your RESS solution relies on browser detection, then it can be spoofed and will require updates to capture new browsers. Alternatively, if it incorporates feature detection, especially for cutting-edge features, you’ll have to make sure that those detections are also regularly reviewed and updated. Secondly, as pointed out throughout this chapter, avoid latency and extra requests at all costs. Depending on the exact solution you use, RESS might break that rule by requiring at least one extra page load and redirect to “set up” the system. In return for a more optimized experience throughout a user’s visit to a website, this seems like a reasonable tradeoff, but each developer should make that decision on their own. Thirdly, depending on how complicated your templates are, your data might need to be strongly separated from the presentation. This adds another level of complexity. With many content management sys-

71. smashed.by/future-friendly

220

OPTIMIZING FOR MOBILE

tems, this probably happens anyway, but if you’re looking to use this with a simple website, this could add overhead. But the primary issue for me—the one that will affect adoption of this technique by the majority of developers—is that it’s a serverside solution that requires different tools and skill sets than RWD. Because the technique is in its infancy, work is required in the short term to reinvent the RESS wheel in distinct environments. At the end of the day, for the server-side developer, RESS could be a boon for delivering mobile-optimized content. For the front-end developer who has little experience with languages such as (but not limited to!) PHP, Python and Ruby, this technique won’t be as easy to take advantage of.

Wrapping Up Improving Web page performance on mobile is not rocket science. The tried and true optimization techniques of the past, such as minification and file compression, are more important than ever. Image optimization needs to come back to the fore, and, luckily, great tools exist to help with it. HTML5 techniques such as localStorage and touch events show that there are new ways to optimize our websites and pages for speed. While designing for mobile first has become the rage, as evidenced by developers embracing techniques such as RWD, we need to start thinking about optimizing performance for mobile first, too. The former shouldn’t happen without the latter.

BY DAVE OLSEN

221

ABOUT THE AUTHOR Dave Olsen works as a “do it all” at West Virginia University’s Web unit, thinking about and delivering solutions based on new technologies. Although it sounds like a cliché, the most important life lesson he has learned is “Just do it. If you have an idea, just try it out, even if it seems silly. More often than not it’ll work and, more than likely, someone else will want to learn from your failures too.”

222

OPTIMIZING FOR MOBILE

UX Design For Mobile vi. vii.

Hands-On Design For Mobile Designing For Touch

CHAPTER SIX

· BY DENNIS KARDYS

HANDS-ON DESIGN FOR MOBILE

HAT’S THE MOST DIFFICULT CHALLENGE of designing for mobile? It’s a question I’ve found myself returning to over and over again. I’ve considered the types of challenges we face: inconsistent device sizes and capabilities, quirky browsers, irresponsible responsive images and so on. What I’ve realized is that all of these challenges are technical in nature. They are all resolvable problems—problems that we’ll eventually work out. The bigger challenge is one that can’t be solved in code. This challenge will need to be solved collectively, in our minds. We carry with us personal, intelligent technology that’s always connected and always with us. These devices will inevitably continue to shape how we think and behave, transforming the way we access information and interact with the world around us. So, how do we go about recognizing the huge potential of these small devices that we hold in the palm of our hand? To unlock new ways in which mobile technology can serve us and improve our lives, we need to move beyond our inherited understanding of them. BY DENIS KARDYS

| 225

“To understand something is not to be able to define it or describe it. Instead, taking something that we think we already know and making it unknown thrills us afresh with its reality and deepens our understanding of it. For instance, suppose there is a glass here. You might know about a glass. But what if you need to design one? The moment a glass is proposed as an object to be designed, you start thinking about what kind of glass you want to design, and you lose a bit of your understanding of “glass.” Arrayed in order before you are dozens of glass vessels of gradually varying depths, from “glass” to “dish.” What if you are asked to clarify the exact boundary point between one and the other? Faced with the objects, you’re at a loss. And again you become a little less sure of your knowledge of a glass. However, this doesn’t mean that your knowledge has been overturned. Indeed, it’s just the opposite. You’ve become more keenly conscious of glasses than before, when you understood them by simply unconsciously calling them all by the term “glass.” Now you actually understand glasses more realistically.”1 Kenya Hara goes on to say in Designing Design that, “The more firmly we’re convinced that we’ve identified an object, the less precisely we understand it.” Absent design scrutiny, a concept such as “mobile” can feel quite tangible and clear. But examined through a design lens, its definition quickly becomes clouded. Arrayed before you are dozens of devices of gradually varying size and capability, from feature phone to tablet. Does your definition of mobile depend on a particular class of device, a viewport size or its context of use? As the people hired to design for mobile, we are responsible for defining it—even if that means explaining why the answer is, “It depends.”

1. Hara, Kenya. “Designing Design,” Lars Muller Publishers, 2008, smashed.by/hara-design.

226

HANDS-ON DESIGN FOR MOBILE

We’re the ones who must verbalize and visualize both the boundaries and opportunities provided by mobile. However hard it may be, communicating this to clients and team members is very important. This realistic, shared understanding of mobile is what creates the possibility for design innovation, and it could reveal yet-tobe-discovered potential in these ultra-connected, always-with-us devices. But without this depth of understanding, we’re doomed to remain constrained by the conventions and processes that we’ve inherited from other media.

FIGURE 6.1. The concept of mobile seems easy to fathom—until we try to de-

fine it. What constitutes a mobile experience, and where or when does it end?

This chapter is about how we define what we design. It’s about the challenges we face in trying to communicate our design intentions. Finally, it’s about implementing and selling processes that support a realistic understanding of what it means to design with mobile in mind.

Defining the Unknown Tempting as it may be to begin designing with what you know, there is merit in setting aside all presumptions and taking a fresh look. Many design discussions and, ultimately, decisions stem from unvalidated, inaccurate assumptions. We’re all susceptible to this. When it comes to estimating how much we know, we’re pretty bad at it. When it comes to predicting what others know, or how they will behave, we’re even worse. In “Thinking Fast and

BY DENIS KARDYS

| 227

Slow,”2 psychologist Daniel Kahneman talks about the errors in judgment we commonly fall victim to. Some of the more common ones are the overconfidence effect, which is the tendency to inflate our estimation of how much we know, and the false consensus effect, which is the inclination to believe that our own behaviors are the norm and, so, to expect that others will act and think the way we would. Coupled, they induce us to blindly believe that we understand the motives and needs of our target audience. Worse still, we might feel as though we’re representative of the target audience, imposing our own needs and biases onto them. The harm here is that some of these innocent assumptions can set you up for a rude awakening later on, once your app or website is released in the wild. To avoid being misled by assumptions early on, expose as many “unknowns” as you can at the beginning of the project. Watch out for a few in particular. While misconceptions about these exist in all domains, the following are particularly nasty for mobile because they cater to our inherent biases and inherited mental models. 1. Canvas

We believe there is a standard page size on which we can base our designs. 2. Capability

We assume that most people use devices similar to, or with capabilities equal to, our own. 3. Context

We make design decisions based on edge cases, rather than realistic contexts of use.

2. Kahneman, Daniel. “Thinking Fast and Slow, Farrar, Strauss and Giroux,” 2011,

smashed.by/kahnemann-thinking

228

HANDS-ON DESIGN FOR MOBILE

WE THINK OF THE CANVAS AS A PAGE

The paradigm of print, in which design is anchored to the determinate bounds of a page, still manages to influence how we design for the Web, a medium that has evolved well past its infancy. To this day, I get asked by clients what the current standard width for Web pages is. Why do they ask this? Probably because we Web designers have an inglorious history of defining (and redefining) for our clients what that size has been. (I confess to being quite guilty of this myself!) The proliferation of large desktop displays gave us a false sense of security. Our users were able to control their browser size and adjust windows—and even change the default resolution. On mobile, we’re not afforded this luxury. We cannot, regardless of any context, count on a standard predetermined screen size or pixel density. Even in more predictable environments, such as a native application for a particular device, the screen size will inevitably change as the hardware evolves. When it comes to devices, viewports come in all sizes. This issue of canvas-as-page goes beyond our inability to predict screen dimensions. The term “page” itself is loaded with over a thousand years of baggage. It implies a point of existence for each piece of information and an inextricable link between content and predefined layout. This is an outdated model. The information we display in app and Web interfaces isn’t stored on screens; it’s stored in databases. It can be shared and distributed across any number of views simultaneously; different devices pull the information into different containers, which are dynamically assembled into layouts determined by each environment. Trying to define every possible permutation of content can be a futile quest. Pages must give way to systems. WE ASSUME THAT MOST DEVICES ARE AS CAPABLE AS OUR OWN

Flash back to the year 2009. The Nokia 2700 was just released, and Nokia would go on to sell 20 million of the devices. It featured FM BY DENIS KARDYS

| 229

radio, Bluetooth connectivity, multimedia playback, a 2 MP camera and several Web-based applications, including chat, email and a very basic Web browser. If you’ve never heard of this phone, no one would blame you. Most the tech world’s attention was on the iPhone 3S, with its 3 MP camera, multi-touch screen and voice control. Fast forward to today. Both of these phones still exist and are in use! If you happen to own a newer smartphone, perhaps the iPhone 5 or Nokia Lumia, these devices from four years ago might feel like worthless relics. Because of the exponential rate of technological advancement (Moore’s Law),3 features that blow our mind today will in short time be reduced to basic expectations at best or, at worst, become completely obsolete. For example, as location-based technology advances, we’ll see tremendous opportunity to design smarter, more contextually aware experiences. We should pursue these possibilities, continually endeavoring to explore new methods of interaction. At the same time, we need to stay mindful that not all mainstream devices will have the same degree of sophistication. We should remain wary of designing apps and websites that depend entirely on such features when it risks alienating users of assistive technology, those with less capable devices and those who, because of fragmentation, might be running outdated versions of the native OS. Thinking to the future, we should consider not only deprecated smartphones, but also other types of less capable Internet-enabled devices. What form might they take in the years to come? WE CONFUSE EDGE CASES WITH CONTEXT OF USE

Stop me if you’ve heard this one: “We need a mobile app for our users on the go.” It’s easy to get hung up on the notion that our users are perpetually mobile. Frequently, we conclude that because a user

3. Wikipedia, “Moore’s Law,” smashed.by/moore-law.

230

HANDS-ON DESIGN FOR MOBILE

might be busy and on the go, that they must be. Don’t allow edge use cases to drive design decisions. Your average mobile user is not likely “a distracted person using their mobile phone with one hand, standing in the aisle of a speeding train, holding a baby.” Similarly, being on a non-handheld device in no way implies that the user is chained to a desk, deeply focused on interacting with your website. So, if we can’t rely on either case, how do we determine the context of our mobile users? The short answer is that we can’t; it’s an unknown. There is no way to know the environmental factors surrounding our users during every interaction. This doesn’t mean we’re in the dark either. We can model realistic use cases and scenarios by conducting ethnographic research and directly observing our users. At the very least, try to gather information about how mobility affects the priority of tasks that people carry out. Again, don’t let assumptions about a user’s situation lead you to presume what they will or will not do. As Brad Frost wisely reminds us, “Mobile users will do anything desktop users will do, provided it’s presented in a usable way.”

Embracing the Unknown Believing we can predict screen size, type of device and context of use is comforting. We’ve deluded ourselves into thinking that we have access to this information. In fairness, we’ve managed to get along pretty well so far. We’ve been steadily raising the caliber of digital experiences and improving user experiences well beyond where they were just years ago. Except that, instead of platforms converging, mobile has swept in like a force of nature, fracturing any semblance of “sameness” in terms of where and how people rely on technology to engage with companies, products and services. The very things we don’t know—canvas, capability and context— are perhaps the things we as designers crave the most. Knowing them would give us more control over the experiences we want to BY DENIS KARDYS

| 231

enable. Letting go of these assumptions is hard; I still struggle with it daily. Acknowledging the limited amount of contextual information we have access to is scary, too. But I’ve got news: if it’s intimidating for you, imagine how terrified your client must feel. Your client, whether an external or internal stakeholder or your boss, is now confronted by the reality that you must essentially design for the unknown. What started out with the simple phrase “We need an app” has now become a seemingly insurmountable burden on their shoulders. In their terms, this equates to a potentially expensive project with a questionable promise of success—not exactly the reassurance they had hoped for. Can you blame them, then, for demanding extensive and incremental reviews and sign-offs at each and every stage of the project? If you can relate to this burden of the unknown, don’t fret—embrace it! You have become a little less sure of your knowledge of mobile. However, this doesn’t mean that your knowledge has been overturned. Indeed, just the opposite. You’ve become more keenly conscious of mobile than before. Now, your understanding of mobile is more realistic.

Setting Basic Expectations Basic expectations are another form of baggage that mobile designers are plagued by. Basic expectations are the conventions we take for granted, the familiar features and processes that have been adopted so widely that we don’t question them. For example, many of the tools that we use to design interfaces, such as Photoshop, originated as tools for producing print documents. When you open a new document, the first thing Photoshop does is prompt you to set a canvas size, a step we’re so familiar with that we don’t question it. It’s no surprise then that, prompted to set our canvas, we begin to think in terms of a fixed canvas. The software and its “Save for Web” feature then influence us to produce static 232

HANDS-ON DESIGN FOR MOBILE

images. We might believe philosophically in concepts such as mobile first, but if the tools we use are influencing us to make design decisions in a single context, and we are exporting static mockups for our clients to review, then we are reinforcing the very expectations that we need to be resetting! In other words, we must make sure that each part of our process and every deliverable that we produce set proper and realistic expectations. So, letâ&#x20AC;&#x2122;s look at two parts of our process in which we may be unintentionally sabotaging ourselves: 1. Static design documents, 2. Assembly-line design processes.

Tell me if this rings a bell. Your UX team pours over research and user data, producing blueprints for an application. In the course of this, you continually discuss the proposed plans with stakeholders. During these skirmishes, informed recommendation battles subjective opinion, and data-driven scenarios duke it out with hypothetical use cases. Once all stakeholders have been satisficed, sign-off is obtained, and the approved design documents are passed on to the UI designer or whoever is next in line. The process repeats itself, until a finished portrait of the design is approved and development can begin. A while back, the company where I work was hired to build a mobile Web app for a large enterprise technology company. The company had been working closely with its creative marketing partner for months on the design. We became responsible for the development and CMS integration. We inherited a robust stack of documents: source PSDs, annotated wireframes, written specifications. The designs had been reviewed and discussed thoroughly and had made it through the clientâ&#x20AC;&#x2122;s rigorous internal approval process. Despite the design documentation being quite in depth (sometimes BY DENIS KARDYS

| 233

FIGURE 6.2. Designs conceived in a single context often reveal

flaws once they are tested in multiple modes or contexts.

granularly so), problems with the design were revealed right off the bat as we began the programming. For example, the design mockups were shown in the context of an iPhone in portrait mode. Tabbed navigation was fixed to the left side of the screen, but there was no indication of how the navigation should be repositioned if the deviceâ&#x20AC;&#x2122;s orientation was changed to landscape. Notes indicated that a sub-navigation panel should fly out when each tab was tapped, but no cues were given as to what should happen if an item had no sub-navigation or if the panels contained more links than could fit in the visible confines of the viewport. In retrospect, these considerations might seem obvious, but, in defence of the creative team, none of them were apparent in the single context that the team was designing for. All of the teams involved learned a hard lesson: even with annotation, static illustrations of a single context are ineffective at communicating something as complex and variable as the mobile environment. The linear process we were following made things even more complicated. Change orders were required in order to introduce design modifications during the development stage, post-approval. 234

HANDS-ON DESIGN FOR MOBILE

For many designers, me included, the scenario above is a brutally realistic portrait of the design process. Surely, there must be a better way! I believe there is, but let’s start by pinpointing some particular problems with the conventional process and figure out some productive alternatives. STATIC DESIGN DOCUMENTS SET FALSE EXPECTATIONS

From the earliest stages of design exploration, the documents we typically share to help communicate our design intentions—from wireframes to highly detailed renderings of the proposed interface—are static. Because they aren’t clickable, tappable or resizable, interaction can only be hinted at. This means that each information architecture diagram, wireframe and PSD needs to be accompanied by succinct annotation and lengthy phone or in-person walkthroughs to describe the system’s intended behavior. Consider for a moment the time and effort required by both parties to translate these documents. It’s not just your time being spent trying to communicate the implied interactive properties of the design; it’s also the client’s time trying to reinterpret them. Perhaps the biggest issue I have with static deliverables and linear processes is that they build false expectations of the design. You know how when you unwrap fast food, it never looks like the photo that triggered your appetite? The discord felt with that mismatch is the same feeling our clients experience when they make the leap from seeing a static JPG of their website to seeing it in the browser or on the actual device. Each stage of the design process achieves a higher level of fidelity, leading everyone involved to believe that what you show them at the end of the process is exactly what they’ll see in the finished document. But unless you’re showing them HTML in the browser (or natively rendered content), then it’s failing to reflect rather major design considerations, including things such as pixel density, variBY DENIS KARDYS

| 235

able viewport size, CSS support and the annoying nuances of Web or device font rendering. When designing specifically for mobile, the effects are exacerbated. Envisioning how a Photoshop document will translate to an HTML page in a desktop browser requires a fairly light amount of cognitive effort. We can map it pretty easily. But mapping a design from a Photoshop document to a handheld touch-enabled device, where only the slightest portion of the page’s full content is visible at any one time, requires a bit more effort. It can generate more questions than answers. It relies too much on your stakeholders’ imagination. I don’t know about you, but that sounds like trouble. All in all, these deliverables seem better suited to a single context than to a multi-channel, multi-layout system. They tether the design to pages and prearranged layouts, not to reusable modular components. The process plays out like a telephone game, with bits of the intended message making it through at each turn. The lack of cross-role collaboration can lead to siloed thinking and page-centric deliverables. Asking clients to imagine the interactivity makes early design decisions more likely to be based on discussions of hypothetical use.

Design As Hypothesis Any time you present a design, you’re simply presenting a hypothesis—you’re putting forward your guess at the best solution for the challenge at hand. Considering the number of moving pieces, the likelihood that your proposed designs will nail everything is extremely low. When changes to the hypothesis are driven by debate or personal preference, rather than by user testing or structured critique, then we risk signing off on designs prematurely. Unless your process embraces iteration, the shared goal will likely become to not make any changes! The truth is that, whether you spend three days or three months pontificating on hypothetical use cases, there’s 236

HANDS-ON DESIGN FOR MOBILE

really no way to tell what will work and what won’t until the designs are actually tested—through use! Unfortunately, many projects stall on hypothetical debates. How much time gets wasted by disagreement over what users might do or what users might not see? There is a gap between implied use and actual use; naturally, we fill this gap with assumptions about usage, colored by our own biases. Does this mean that all deliverables are bad, that we need to put an end to wireframing and uninstall Photoshop? Not at all. Design artifacts are still invaluable. Robert Hoekman puts it best:4 “The purpose of a design artifact, whether a wireframe, prototype or sketch, is to illustrate our thinking.” Regardless of what type of design document you’re producing, stay focused on its core purpose in your workflow and on what you intend for it to communicate—equally important, make sure to clearly articulate what it isn’t intended to communicate. ASSEMBLY-LINE DESIGN PROCESSES LIMIT SOLUTIONS

On the design assembly line, every person is assigned a role and, ideally, assigned clearly defined parameters for where their work begins and ends. Although there may be team brainstorming sessions, each individual is ultimately responsible for the production of whatever deliverables are assigned to them. In this model, every task depends on the work that’s been generated in earlier stages. Once design decisions are approved, the appropriate documents are handed off to the next team members. These documents—wireframes, mockups, etc.—become the equivalent of IKEA assembly manuals. Once they’re used, they are archived or simply discarded. Although a positive outcome of this model is the potential for clear ownership of each deliverable, ownership of the design is a little more 4. Hoekman, Robert. “The Big Think: Breaking the Deliverables Habit,” Smashing Magazine:

smashed.by/deliverables-habit.

BY DENIS KARDYS

| 237

amorphous. This occurs in situations in which multiple designers are assigned to different roles, all contributing to the end product. We love to champion the ideal that “design is not decoration,” but we designers are as guilty as anyone of assigning the role of decorator to the UI specialist, who may very well be a keen graphic designer with superior visual communication skills. Similarly, we assign the role of implementer to the front-end developer, who, chances are, might have a masterful understanding of grid systems and design as well. The further into the process each designer is introduced, the more removed from design decision-making they end up being. Let’s say that users are confused by the behavior of a particular menu on small-screen devices, and your client wants to explore an alternative solution. Could the front-end developer simply swap the menu, or do any changes need to originate from the original UX designer? How far up the chain do you need to go to push through a simple design change? The front-end developer might be able to quickly take care of all the design and code that is needed. But if that happens, what does it mean for the UX designer who later finds that their plans have been altered? So, while a clear sense of territory is a positive outcome of this model, it’s possibly outweighed by the model’s rigidity, and it is not very conducive to change or cross-discipline collaboration. Because this model is based on sign-offs and hand-offs, making changes to a design that has already been approved could involve a lot of people, which translates to a cumbersome, time-consuming and expensive process. Naturally, this encourages all of the decisions to be made early on and deters changes as the project evolves, creating a funnel-shaped process.

238

HANDS-ON DESIGN FOR MOBILE

DESIGN FUNNELS A design funnel is a process that loads all of the ideation into the early stages of the project, quickly narrowing multiple concepts down to what’s perceived to be the most viable one. Each subsequent stage focuses on refining the initially selected design.

FIGURE 6.3. In the conventional design funnel (“Pugh’s Funnel”), the op-

tions are quickly narrowed down, and then the chosen idea is refined in each successive iteration.

FIGURE 6.4. Buxton’s variation introduces the idea of alternating between con-

cept generation and concept reduction throughout the convergent process.

In “Sketching User Experiences,”5 Bill Buxton proposes a variation based on a simple concept: alternate between concept generation and controlled convergence. Unlike a traditional funnel, which en-

5. Buxton, Bill. “Sketching User Experiences”, Morgan Kaufmann,

smashed.by/buxton-sketching.

BY DENIS KARDYS

| 239

courages ideation only in the earliest phases of a project, this model embraces exploration during each stage. Concept convergence also occurs in each stage, whereby ideas are filtered down by measuring against explicitly defined design goals and principles. Controlling ideation with structured critique moves the design discussion towards actionable, productive feedback. This approach is healthy because it allows a greater number of ideas to be “suggested, proposed, and questioned.”6 The concept that no idea is sacred and that alternative, potentially better ideas may be introduced later in the process can only lead to a stronger product. Because the assembly-line processes that result in a traditional design funnel have to kill so many ideas before they can be explored, there is a danger that they’ll come back to haunt you later. A stakeholder who is soured by the feeling that their idea wasn’t given due consideration might resist alternatives or seek to work in their idea at a later stage. Buxton’s variation, on the other hand, is more accommodating and fits well in a more iterative workflow. So, what does all this talk about assembly lines and funnels have to do with designing for mobile? Simple: I don’t believe we can front-load all of the necessary design decisions at the beginning of a project for any platform, least of all for mobile. This approach is an unfortunate remnant of industrial-era thinking, when products could be sequentially manufactured. We need to guide our clients to a shift in thinking about how we design and build software systems. We need to be able to make critical decisions about how an application behaves in the appropriate context, at different times throughout the project. In her presentation “The Mobile Frontier,” Rachel Hinman says, “Throughout the design process, our work should be situated in the 6. If you’re interested in learning how to craft effective design principles and goals, Adam

Connor and Aaron Irizarri’s blog, “Discussing Design” (smashed.by/discussing-design), is an indispensable resource, filled with tips on managing feedback and facilitating critiques.

240

HANDS-ON DESIGN FOR MOBILE

contexts where it will be used.”7 I agree. In the past, we may have been able to get sign-off with a picture of a website meant for the desktop and then go build it, but that won’t cut it for mobile. She continues, “Our work needs to be conceived of and fleshed out in the real context of use.” Now that’s an idea I can get on board with! MORE PURPOSEFUL DELIVERABLES

Let’s step back and ask why we are spending time creating anything that is not the actual product: i.e. a mobile website or application. At a very high level, we have two major goals: to create something that satisfies a need of whoever will use it, and to satisfy the objectives of whoever is paying for it. In order to meet both of these goals, there needs to be a shared vision. We use design artifacts to articulate our ideas and as a tool for discussion. In Communicating Design,8 author Dan Brown cites three reasons for producing deliverables: 1. Consistency of vision

Make sure that everyone on the team is on the same page so that they can make worthwhile contributions. 2. Accountability

Make sure that everyone understands the decisions that have been made and the implications of those decisions. 3. Traceability

Track the collective history of decisions in the project, to serve as a reminder of the design rationale at crucial points in the project’s evolution.

7. Hinman, Rachel. “The Mobile Frontier,” Breaking Development Conference, April 2012:

https://vimeo.com/44265965. 8. Brown, Dan. “Communicating Design,” New Riders, smashed.by/comm-design. BY DENIS KARDYS

| 241

If we consider interactive prototypes, I would add a fourth item: 4. Validation

A deliverable acts as a proof of concept, validating or disproving design decisions. Using these as a guide, we can start to make more rational decisions about what types of design documentation to use for different types of projects. A native application and a responsive website would likely need drastically different sets of deliverables, even though a person might access both through the same device. We can now measure documentation against slightly more formal criteria: What is the purpose of this deliverable? What is it intended to communicate? And which communication goal does it satisfy?

Every Design Deliverable Should Have A Core Purpose When design deliverables get a bad wrap, it’s usually because too much is invested in them. I am a firm believer in doing the least to communicate the most. Time and again, I see needless documentation listed as a requirement in RFPs, or I see painfully bloated deliverables produced to satisfy a fool’s quest to solve every single problem up front. For example, how many times have you seen a project slow to a crawl because wireframes were burdened to serve too many different purposes? I’ve seen wireframes that serve as a specification document for content, layout, interface patterns and interactive behavior. In a funnel where any deviation from the approved document requires a change order or is met with resistance, is it any wonder that every element of the design is scrutinized under a microscope? Let’s start by demanding that each document we’re tempted to produce can justify its own existence. Then, let’s set some clear boundaries for what that document should define and what it 242

HANDS-ON DESIGN FOR MOBILE

should merely suggest. By articulating the core purpose of a document, we narrow the focus of the conversation surrounding it. Anything indicated in the document beyond what’s defined as its core purpose is simply a cue—a suggestion that will do until a better alternative is revealed. You’ll find a lot of articles about process and workflow, but the reality is that none may fit your exact needs. Instead of preaching about what to do when planning the mobile user experience, I’ll cover some techniques and tools that we’ve found useful, and I’ll establish criteria for when and why to use them. To begin, let’s categorize them: Ŧ Concept generation

Figuring out what to build Ŧ Content planning Figuring out what information to show Ŧ Designing and refining Working out the interface(s)

Concept Generation To develop any sort of mobile application, you need to begin by conceptualizing what you should build: what will this product do, and who should it serve? Basically, you need ideas, and, fortunately, everyone has those! When you’re starting a new project, although it might feel like you’re starting with a blank canvas (or viewport), you really aren’t. You can’t see it, but that canvas is filled with expectations—including your own! The point of concept-generation techniques is to bring all of those ideas and expectations to light, sucking them from the minds of all involved parties and materializing them in a way that exposes them to discussion and critique. So, when we look at methods and tools for concept generation, we can define a couple of their core purposes: BY DENIS KARDYS

| 243

Ŧ Generate as many ideas as possible.

The ideas generated need not all be viable—some may be entirely implausible. The point is to shoot for the maximum number of ideas. Bad ideas are sometimes stepping stones to great ones. In line with Bill Buxton’s variation on the design funnel, the more concepts you have to explore, the better equipped you’ll be to help realize the project’s potential. Because you are looking for quantity over quality in ideation, you certainly want to include as many different informed perspectives as possible. You’ll have opportunities for controlled convergence later. Ŧ Break existing mental models.

The invisible expectations described above are actually mental models. They’re a person’s mental image of how something will work or function, and they’re based on that person’s prior experience with similar systems or devices and on their assumptions and biases. Mental models start to form from the moment that team members begin to envision the project. As planning and discussion go on, the mental images become more defined. This can lead to situations in which you find yourself designing for what’s in the client’s head, instead of designing for the project’s goals and the user’s needs. In order to get people to accept new ideas, you need to exorcise the old ones. This is where sketching comes in. We often think of sketching as a way to generate and communicate ideas, but it can also be a weapon to dismantle them. The goal of sketching in this case isn’t to produce drawings that inform the final design; we are not replacing the internal UX process. The goal is to drive out those stubborn, thorny ideas and make room for new ones. Only then can we look anew and achieve a deeper understanding of what we’re designing. 244

HANDS-ON DESIGN FOR MOBILE

FIGURE 6.5. Sketching is a powerful way to exorcise festering ideas that

could derail a design later on.

SKETCHING TECHNIQUES

Two excellent lightweight activities for simultaneously exploring new concepts and purging ideas that can lead a project astray are sketching and paper prototyping. In Drawing Is Thinking, Milton Glaser says, â&#x20AC;&#x153;Art seems to affect us beneath the surface of our rational or logical mind. It basically moves the mind to action on a different BY DENIS KARDYS

| 245

level, one that is more profound and less describable.”9 He goes on to suggest that drawing is a form of meditation and that it brings the drawer to a greater state of attentiveness. This state of focused attention is what I believe makes sketching such a powerful tool for ideation and communication. Mental models, especially those that pertain to visual interfaces, can be quite difficult to articulate verbally. There is a much freer connection between the hand and the mind, however, which facilitates the translation of ideas from mind to paper. In these cases, a picture truly can be worth a thousand words. The ability to sketch, though, is not the same as the ability to render. With just a few basic shapes and symbols, anyone can develop their ability to sketch. Designer Peiter Buick has written a fantastic article on sketching technique, titled “The Messy Art of UX Sketching,”10 filled with useful information for both those new to sketching and seasoned pros. COLLABORATIVE SKETCHING

While sketching can certainly be a solitary, meditative ideation activity, it also works well as a collaborative brainstorming exercise. Group sketching facilitates the rapid exchange of ideas between team members, providing opportunities for immediate feedback. Although the participants in a sketching workshop might not think of themselves as artists, this depth of focused consideration of the design challenge is crucial. Below are some group sketching exercises that have been particularly helpful in reframing how we think about mobile and device-agnostic design.

9. Glaser, Milton. “Drawing Is Thinking,” Overlook Hardcover, smashed.by/glaser-drawing. 10. Buick, Peiter. “The Messy Art of UX Sketching,” Smashing Magazine:

smashed.by/sketching.

246

HANDS-ON DESIGN FOR MOBILE

Stick-Figure Comic Strips Stick-figure comics are simple to do. Break your group up into teams of two. Give each team a piece of paper filled with square panels, and ask them to draw a crude comic that depicts a member of the target audience opening the website or app and completing an activity. We used this technique recently when planning a responsive redesign for a university website. To give our burgeoning cartoon artists a little more direction, we specified, “Begin your comic strip with a character who is representative of your target audience. Your character’s story does not begin on your website (or in your app). Think of a place where the story begins. What motivates them to visit your website, and what actions do they take when they get there? What device are they using?”

FIGURE 6.6. Drawing comics is a fun way to imagine new scenarios, whimsical

though they may be.

In the university workshop, one team described the story of a teen who is at a coffee shop talking about colleges with her friend. At her friend’s suggestion, she pulls up the university’s website on her BY DENIS KARDYS

| 247

smartphone and explores. She bookmarks the URL and retrieves it later on her tablet, doing a little more research and then, noticing an upcoming group tour date, registering for a visit. Weeks later, while visiting the campus, she uses her phone to take photos and share them to her Facebook page, asking others in her network what they think of the university. These storyboard-type comics are great because they make people turn lists of needs and objectives into narratives. Layers of abstraction peel away as participants give detailed accounts of how they envision individuals using the website. This also moves the conversation in the right direction: instead of focusing on how to arrange content so that it fits on a tiny (or medium or large) screen, they start thinking about behavior. What particular activities do they want to encourage? What situations do they want to account for? In the example above, instead of thinking about a user completing a task in a single session or context, the team describes a scenario in which the website connects experiences in the character’s life that span multiple locations, devices and applications. This type of cross-channel experience is precisely what we want our clients to think about in these exercises.

Sketching Task-Based Interactions The scenarios described in the comic-strip exercise usually get stakeholders excited about the prospect of purposeful interactions with the website or application. Suddenly, instead of a faceless visitor, we’ve got pages of sketches of people with names trying to complete actions. It’s a great time to investigate the following question: What activities do you want your website to support? Ask your group to come up with a short list, focusing only on critical activities. Out of those, select one to work on (you can come back to the others, or you could simply take note and explore them with the rest of your design team as part of your internal process). 248

HANDS-ON DESIGN FOR MOBILE

Typically, a single activity, such as “register for an event,” comprises multiple tasks. Identify those tasks, and list them somewhere for everyone to see. If you’re building a native application, your sketched template should mimic the hardware and screen proportions of the device it’s being designed for. In the case of a mobile Web app or responsive design, you can use a standard 6-up template or download mobile sketch templates, which are freely available.11 (For the more serious sketcher, UI Stencils12 sells pads of mobile templates and drawing stencils.) Give each person five minutes to sketch the different views needed by a user to complete the interaction. Five minutes, you ask?! The extremely short time seems completely unrealistic, I’ll admit. In fact, when participating in this exercise, I almost never complete all six sketches—and that’s OK! When the purpose of a sketching exercise is to ideate, keeping the time limit tight is advantageous. Paradoxically, the more time you give people, the harder they think, causing them to censor their ideas more. Tight time constraints encourage participants to quickly put down any idea that comes to mind. After the time is up, don’t dive into discussion. Instead, immediately pair up each participant, ideally with someone who they don’t work closely with on a daily basis. Ask each person to spend a few minutes reviewing their partner’s ideas, and then spend the next 20 minutes or so collaborating on a new, more detailed sketch that outlines the activity and sub-tasks. Once all of the teams have completed this, go around the room and give each team three minutes to present their concept. Keep the time for feedback equally short (three to four minutes), to keep the conversation on track and to remain focused on critiquing rather than ideation.

11. “UX Sketching and Wireframing Templates for Mobile,” Smashing Magazine:

smashed.by/templates-mobile. 12. smashed.by/uistencils

BY DENIS KARDYS

| 249

Creating Sketchboards Sketching workshops can generate an immense quantity of ideas in a relatively short amount of time. What do you do with them all? As people present their ideas during the course of the workshop, we have them tape them up to a large wide sheet of paper pinned to the wall. This sheet is our sketchboard. As participants critique, we add notes directly onto the paper. At the end of the day, you can just pull down the sketchboard, roll it up and bring it back to your team for discussion. As with all group sketching, the purpose isn’t to replace your design team or UX team’s responsibilities, but to share knowledge and contribute new ideas to the pool. PAPER PROTOTYPES AS A VALIDATION TOOL

Sketching exercises generate a lot of ideas. Confronted with a wall full of sketches, you’ll probably have your personal favorites, and other stakeholders will likely have their own. Assuming you’re not going to move forward with every appealing idea, you’ll need some type of controlled convergence—some way to filter the ideas. This is when an extremely versatile tool such as paper prototypes comes in quite handy. If you’re not familiar with them, paper prototypes are cut-out paper models of your website or application, usually with repositionable elements. When I first learned about them, I made the mistake of writing them off, assuming they were impractical and less effective alternatives to HTML prototypes. I changed my tune after seeing them in action. When you’re designing for mobile, paper prototypes offer some serious advantages over other forms of prototypes, such as sketches, wireframes and HTML. These advantages include the following: Ŧ People can actually pick up, hold and interact with paper mod-

els. 250

HANDS-ON DESIGN FOR MOBILE

FIGURE 6.7. Paper prototypes are an easy way to test ideas in the field. They

enable you to make changes quickly based on immediate feedback.

Ŧ They’re portable, freeing you to test and validate design deci-

sions in the field. Ŧ Paper prototypes are easy to alter, allowing for immediate changes based on user feedback. We recently had an opportunity to redesign the intranet for an airline in Canada. Pilots and flight crew, we learned, carry around heavy duffle bags filled with tiny laptops and bulky flight manuals and folders of paperwork that they need to fill out prior to flying. Mobile is changing the way they work, as manuals are transitioning from paper to electronic format, and their required flight forms can be filled out and submitted online. The airline’s intranet is the hub for these tasks. They also spend a lot of time waiting in airports and lodging overnight in hotel rooms in various cities away from home. Booking hotels, bidding on shifts, checking memos: they BY DENIS KARDYS

| 251

need to be able to access the intranet from anywhere and everywhere. Carrying around tiny laptops certainly does not define their future—mobile technology does. Meanwhile, the same intranet is being used by maintenance workers—people on the floor in hangars repairing jets. Some of their procedures are routine, such as changing tires on planes. For these routine tasks, many of the workers have printouts that they carry around every day to refer to. When an unusual incident occurs, such as when they detect an unfamiliar leak, the worker needs to leave the floor and find a shared work terminal where they can look up the appropriate procedure. How could mobile access change the way these people work? We spent an afternoon with the project team, doing rounds of sketching exercises. We had a lot of ideas about what might work for the pilots, flight attendants and maintenance crew. We also recognized that assuming that our ideas were the right ones would be extremely naive. Quick paper prototypes helped us move from brainstormed sketches to tangible artifacts that we could use to validate our ideas in interviews with our target audiences. Using paper prototypes was extremely liberating here. We had the advantage of being able to field test without getting bogged down by technical issues related to, say, the interviewee’s device or Internet connectivity. Instead, we were able to focus on how each person might complete the task at hand, and even pick up cues on the more ergonomic aspects of the design.

Making Paper Prototypes Creating paper prototypes can be quite easy. The easiest way is simply to take some bristol board or heavyweight card stock and create a sleeve, cutting out a frame for the viewport. Keep the top and bottom of the sleeve open so that you can slide longer sheets of paper (the screens) up and down through the sleeve. You can sketch 252

HANDS-ON DESIGN FOR MOBILE

the interface directly on the “screen,” although you’d have a more useful tool if you broke up the content into individual components and adhered them to the page with a low-tack adhesive such as StudioTac, which is available at most art and hobby stores.

12cm.

14.5cm.

Starting with a heavy cardstock cut out the device along the solid lines.

Fold along the dashed lines, adhering the flap to the backside of the prototype.

account ID

password

Add any important hardware controls or other device specific details.

With users, slide interface mock-ups into the window, testing different screens, flows and application states.

FIGURE 6.8. Building paper prototypes is not as daunting as it seems. You can

make one in no time with some common supplies and in a few simple steps.

StudioTac has enough tack to keep your items in place, while remaining repositionable. You could also simply use tape or even sticky notes. If you aren’t the handiest person in the world, the UX-

BY DENIS KARDYS

| 253

FIGURE 6.9. Interface Origami, a simple technique to recreate common interactions in

native and Web applications.

Pin Mobile Kit for iPhone13 provides a notepad of iPhone templates, along with a collection of generic UI elements on sticky paper, making prep time much quicker than if you were to build each kit yourself. If you feel like you’ve graduated beyond simple cut-and-pasted paper prototypes, Juan Sanchez of Tack has written an article titled “Interface Origami,”14 describing ways to recreate complex paperbased interactions that mimic conventional design patterns (such as accordions) and gestural interactions (such as swiping, pinching and zooming).

Content Planning and Information Architecture In a perfect world, we’d all be working alongside content strategists in each project. But that is simply not the case for many of us, for whatever reason. Whether there is a strategist or not, your project still needs content, and your client still wants to preview it before “committing” to approval. When a project lacks formal content 13. UX Pin Mobile Kit for iPhone: smashed.by/kit-iphone. 14. Sanchez, Juan. “Interface Origami,” smashed.by/origami.

254

HANDS-ON DESIGN FOR MOBILE

strategy documents, such as inventories and page tables, the burden of communicating content decisions is placed on other deliverables. As those requirements sneak silently into other design documentation, the latter’s purpose is confused. I’m sure we’ve all seen cases in which wireframes end up picking up too much of the slack of communication. The wireframe becomes the focal point of conversations about content, labeling, navigation structure, layout, interactive states and even task flows. When a single deliverable is left to resolve so many things, it becomes more and more massive, and discussions surrounding it seem to go on forever. For the sake of approval, our sanity and the future, we need to put more emphasis on managing and prototyping content, independent of layout systems and interaction diagrams. Without digging too far into the realm of content strategy, we can identify a few types of deliverables that will help us communicate what content needs to be shown and how that content will map to different templates and devices. PAGE TABLES

In “Content Strategy for the Web”, Kristina Halvorson describes a page table as a Word document or other text file that “tells you everything you need to know about the content on a specific website page (or content module)—from the objective and source content to the content recommendations and their implications, including requirements for creation, delivery and maintenance.”15 Essentially, what you are doing is liberating content from presentation, which, as it happens, is a rather future-friendly strategy. Let’s say your client is a global provider of machine parts for heavy-equipment manufacturers. Their parts catalog, knowledge

15 Kristina Halvorson, “Content Strategy for the Web,” smashed.by/halvorson-strategy.

BY DENIS KARDYS

| 255

base, specification documents, videos and corporate information need to be distributed across many channels. Their content needs to be available on the Web in a format accessible to a wide range of (mobile) devices. In order to avoid duplication of effort in the

FIGURE 6.9. Page tables are a tool for planning how to adapt content to different

device types and screen sizes.

256

HANDS-ON DESIGN FOR MOBILE

creation and management of content, you will need to make pieces of this content reusable, shared with native “product-finder” applications designed for the iPhone and iPad. Within a year, another version of the application will be deployed to support Android. Without content strategy documents such as inventories and page tables (as well as some system for content management), the majority of your information architecture decisions could end up being made specific to a device or screen size. Decisions about what information to show will conveniently and almost magically seem to match the amount of visible space available on that particular device. We’ve seen this tendency manifest itself in the stereotypical “mobile website,” where the smaller screen seems to urge people to present smaller amounts of content. Correct me if I’m wrong, but I’m pretty sure that no studies have shown any direct correlation between a person’s situational goals or information needs and the size of their mobile device! It’s precisely for this reason that we need to evaluate content and the priority of information objectively, beyond the influence of any one platform. This isn’t to say that platform considerations don’t come into play—they definitely do. In the example above, imagine how you might handle a product listing page. On a device with a large screen, you might have room to display the product’s title, a medium thumbnail and a 200-word description. On a smaller screen, the lengthy descriptions might be cumbersome, so showing only a brief 225-character description might make more sense. But a truncated description would not likely be of much use to anyone. Page tables can aid in the process of planning how to structure, input and deliver your content in a way that adapts intelligently to a range of platforms and screen sizes.

BY DENIS KARDYS

| 257

CONTENT REFERENCE WIREFRAMES

If a page table represents the available content and its objectives, then a content reference wireframe exists as the key that maps content to the different contexts in which it will appear. Stephen Hay compares these to thumbnail sketchesâ&#x20AC;&#x201D;explorations of what should go where. A content reference wireframe is nothing more than a crude approximation of a page, comprising gray boxes with labels that refer to different content items. These boxes are roughly positioned to indicate their general placement on the screen. Because they are quick to produce, you can easily explore variations in layout resulting from breakpoints, device state and application state. Essentially, these are ultra-low-fidelity wireframes whose core purpose is to communicate and validate what content should be present on individual screens in the website or application.

FIGURE 6.10. Youâ&#x20AC;&#x2122;ll have plenty of opportunity to fine-tune the details of the interface

later. Content reference wireframes can be used to map content to templates, allowing you to focus on what content to show, rather than on its exact presentation.

258

HANDS-ON DESIGN FOR MOBILE

Page tables and content reference wireframes are just two of many different types of documentation you might use to plan the content and information architecture. I’m not bringing them up here to paint the ideal workflow (Stephen Hay does that pretty well in his chapter in Smashing Book 3, “Workflow Redesigned: A FutureFriendly Approach”),16 but rather to demonstrate the value of keeping deliverables (and design decision-making) focused on one area at a time. With consensus gained on what information should be displayed in an interface, you the designer have a lot more leeway to suggest which tool makes the most sense to develop the design with: whether it be higher-fidelity wireframes, interactive HTML prototypes or something else entirely. TO WIREFRAME OR NOT TO WIREFRAME?

Earlier I mentioned problems with static deliverables such as wireframes. Does this imply that they should be nixed from the design process? Certainly not. Instead, consider the criteria you use to determine whether, when and how much to wireframe. Wireframes are one of those basic expectations that exist in the conventional design process. But they are easily abused when their purpose is not clearly defined. When dealing with an enterprise client, for example, a design team might be required to create highly detailed wireframe decks with hundreds of pages. However, that level of detail would confuse the client about the type and breadth of feedback that the team is soliciting, making the line between what should and should not be critiqued extremely fuzzy. Let’s set aside the issue of client-mandated design documentation for a moment and examine the end deliverable. Under what circumstances, if any, does it make sense to invest that amount of resources into creating that many annotated pictures of boxes and arrows?! In 16. Hay, Stephen. “Responsive Workflow: A Future-Friendly Approach,” Smashing Book 3, Smashing Media, 2012, smashed.by/smb3. BY DENIS KARDYS

| 259

his article “Ditch Traditional Wireframes,”17 Sergio Nouvel explains the cost of this:

“

At this point, lots of design hours have been invested, and chances are that almost none of the underlying decisions have been properly validated with users: incremental steps done in fidelity. You are basically performing (and sweating) work that is very likely to be completely undone when user feedback arrives.”

Wireframing For Responsive Design Mobile view, tablet view, desktop view—wireframing can easily get out of hand when you are planning a design that is intended to display differently across multiple devices. Showcasing content hierarchy and position can quickly become a daunting and timeconsuming task. Depending on the number of breakpoints in your design, each “page” could end up requiring three or four unique layouts. It’s easy to see how wireframe decks become so bloated. An alternative here would be to start thinking about precisely what needs to be shown in order to communicate layout. For a website with hundreds, even thousands, of pages, trying to create a static representation of each page layout and breakpoint variation is impractical. So, why do we still sometimes do it? On the one hand, you most likely have very interested, invested stakeholders clamoring to see what particular pages will look like. On the other hand, at the risk of being cynical, many agencies aren’t going to say “No” to a client who’s willing to pay more for additional documentation or specifications. If you’re the lucky individual who’s been tasked with creating this grimoire of wireframes, take a step back first. Instead of focusing on all of the different pages and variations that will exist,

17. Nouvel, Sergio. “Ditch Traditional Wireframes,” UX Magazine, smashed.by/uxmag-ditch.

260

HANDS-ON DESIGN FOR MOBILE

identify the key pages and page templates that need to exist on the website. Identify the components that will be reused across the application. Create a simple library, modularizing these components, so that you can refer to them in your other documents. Let the key pages serve as archetypes for the others. Highly detailed wireframes with actual content can show in depth how a fully constructed page might appear. Between page models and modular components, content reference wireframes can indicate how your other pages might be dynamically assembled. Continually question how much you actually need to show, in order to communicate the greater systemâ&#x20AC;&#x2122;s behavior. Perhaps most importantly, remain focused on the purpose of wireframes, and avoid getting bogged down trying to mock up every piece of content. Wireframes are for planning the design system, not for managing content; content management systems are for managing content and assembling pages.

Wireframing For Mobile Websites And Apps Wireframing can help to document the different screens of an app or dedicated mobile website, although Iâ&#x20AC;&#x2122;ve struggled to justify their use in some recent projects. Recently, we were working on a mobile website for a law school. We were able to explore screen layouts and flows using sketches, and then we moved directly from sketches to HTML prototypes. The HTML prototypes took on the traditional role of the wireframe, with the advantage of also allowing us to focus on layout and interactive behavior simultaneously. This was one of our first projects in which we abandoned wireframes, and we didnâ&#x20AC;&#x2122;t ditch them lightly. While we were able to speed up the process and use the time saved to iterate, we were essentially losing a specification document. Fortunately, in this particular case, the self-explanatory nature of the prototypes and the limited number of content views made the HTML prototypes a completely viable substitute. BY DENIS KARDYS

| 261

In cases where interaction is complex and sketches are too rough, but you need more specification about the system’s intended behavior in order to build a prototype, you might want to explore supplementing sketches with use cases. Use cases are step-by-step instructions on how users will interact with an application, complete with exceptions and conditions.

Designing and Refining With Prototypes Todd Zaki Warfel defines prototypes as “a representative model or simulation of the final system. Unlike requirements documents and wireframes, prototypes go further than show and tell and actually let you experience the design.”18 Prototypes serve a very specific purpose: to encourage feedback based on use. They exist to disprove or validate design hypotheses. They take many different forms but generally share certain qualities, including the following: Ŧ They are collaborative, intended to be used outside of any silo Ŧ Ŧ Ŧ Ŧ

you may be designing within. They should be easy to alter, allowing for rapid changes based on immediate feedback. They cut down on the need for descriptive functional annotation and detailed explanation. They exist as proofs of concept and are a tool to help sell your idea. They are meant to be used, enabling you to test your designs.

Aza Raskin, former creative lead at Firefox, says it well:19

“

The goal of prototyping is to convince yourself and others of an idea.”

18. Warfel, Todd Zaki. “Prototyping,” Rosenfeld Media, smashed.by/warfel-prototyping.

19. Aza Raskin, “How to Prototype and Influence People,” smashed.by/influence-people.

262

HANDS-ON DESIGN FOR MOBILE

Given the advantages of prototyping, it’s easy to see that there are some immediate benefits. Namely, compared to static deliverables, the time that the design team has to dedicate to explaining intended behavior is reduced tremendously. It also cuts down on the amount of time that stakeholders have to spend reviewing and interpreting the designs. I suggest getting into prototypes as early in the process as possible. Until your stakeholders or test users actually interact with a prototype of some sort in their hands, most of the feedback you’ll gather will be based on how people think users will act, not on how they will actually act. We talked about paper prototypes earlier. Although paper prototypes do offer cues about content, layout and interactive behavior, their primary purpose (at least in how we’ve been using them) is to validate ideas, helping us decide which ideas are worth pursuing. Through whichever methods you use to determine a direction for your proposed design, you’ll probably want some way to mock it up and test your theories. Then again, there is the alternative: BDUF— big design up front. Proponents of BDUF argue that, with enough research and planning, enough insight exists to spec, design and build it, all in one swoop—do it once and do it right! If you’re wondering whether this is the right approach, ask yourself, “During the development process, have I ever found the opportunity to make changes to the design that would dramatically improve the quality of the final product?” If the answer is “No,” then BDUF might just be the process for you! On the other hand, if information has been uncovered that has guided the design forward, then you should already recognize the value of prototyping! Prototypes help to identify those opportunities as early as possible, while the project is still in a crude enough state to be altered without a significant impact on time or development.

BY DENIS KARDYS

| 263

Because prototypes come in all flavors, I’ve categorized the ones I’m describing according to their intended purpose: behavior and style. PROTOTYPING BEHAVIOR

If you are prototyping behavior, then you’re not overly concerned with details of aesthetics, such as typography and visual style. Rather, these typically low-fidelity prototypes demonstrate functionality and interactive behavior. They might showcase interactivity at the process level—for example, the screens that a user will need to interact with in order to complete a mobile transaction. Or they might demonstrate how a single component works, such as a navigation pattern or a swipeable image carousel. Focusing on the visual aspects of a design can be tempting for clients and designers alike. I call this “design voyeurism,” whereby judgments about the quality of a design are made from afar, irrespective of the actual experience of engaging with it. If you were transpose this to architecture, it would be akin to critiquing an edifice as you approach it. The factors that would immediately shape your judgment of it are the visual qualities, its shape and form, the different surfaces, and how it relates visually to surrounding buildings. Think about how this differs from the perspective you would have as an active participant in the space. The same goes for Web design—and, in the case of mobile-specific design, it’s intensified. Being able to look beyond the surface and evaluate the design as an active end user might engage with it introduces a whole new perspective. A toggle menu navigation pattern might look great on an iPhone, for example, where all of the text fits nicely in the viewport. But what happens on smaller screens where the menu links don’t all fit the viewport? At the beginning of this chapter, I mentioned a scenario in which static visuals for a mobile Web app were accompanied by poorly 264

HANDS-ON DESIGN FOR MOBILE

written specification documents, leaving critical aspects of the design undetermined. By the time the problem revealed itself, the design was so far along the process that revisions would potentially require rethinking the UI. How could using prototypes have revealed the issues sooner or prevented the problematic elements from having made it into the design in the first place? The reason why the obvious question “What happens if you turn it sideways?” never came up is because both teams had been staring at a picture of phone, fixated on the pros and cons of what they were seeing in that particular context. With a prototype, the design team could have crudely mocked up that initial idea into something tangible that could be picked up and held by the teams. Testing it in that context would have revealed the issue, prompting the design team to spend a little more time working out that component of the system’s behavior. So, what does it take to move from static deliverable to interactive prototype? Surprisingly little!

Interactive Wireframes Adding even a small amount of interactivity to your wireframes can go a long way toward reducing the amount of annotation and conversation needed to convey how your proposed design is intended to function. Interactive wireframes can also be used for basic usability testing. The process of creating an interactive wireframe can be insanely simple. Imagine that you’re planning a simple native application. In your wireframing tool of choice (mine is InDesign), create a page for each screen or application state that you want to demo. Export your wireframe to PDF. Using Acrobat, you can create rectangular hotspots anywhere on the screen. Then, assign a link to each hotspot, cross-linking it to another page in the PDF. In doing so, you can easily simulate task flows and test the positioning of icons, buttons and other parts of the interface. BY DENIS KARDYS

| 265

There are a couple of things to consider with this approach. First, PDF hyperlinks don’t work reliably on most mobile devices, so you might end up being limited to testing the mobile design in Acrobat or Reader on the desktop. Not being able to view it in the context of a device in hand, and having to click rather than tap links and buttons, will make this approximation of the actual mobile experience somewhat inaccurate. That being said, it’s a step in the right direction. If you’re not currently prototyping as part of your normal design process, you could use a simple technique like this to ease into, and to start gathering buy-in for, more advanced prototyping techniques or software.

Native Application Prototypes If you’re designing a native application, you might be better off using a different set of tools than you would use for a Web-based mobile application. One of the underlying goals of prototyping is to simulate the final system not only as closely as possible, but also as quickly as possible. You don’t want to waste time recreating buttons, sliders, switches, toolbar icons and other platform-specific design conventions. Not only is recreating the wheel time-consuming, but whenever you break from device conventions in a native application, you risk confusing both your developers and users! For popular UX software such as OmniGraffle, platform-specific stencils and UI libraries are widely available. These are great for mocking up different views, but they won’t get you much further than where you’d be had you shared any other type of document formatted as a PDF. Luckily, some great apps are available to help you quickly put together prototypes of any level of fidelity. FluidUI20 is a subscription-based service (with a free plan as well) for prototyping iOS and Android apps for phones and tab20. FluidUI, smashed.by/fluidui

266

HANDS-ON DESIGN FOR MOBILE

FIGURE 6.11. FieldTest App, one of the many browser-based applications for

rapidly prototyping mobile apps: smashed.by/fieldtest.

lets. It comes with a full library of UI elements and gestural support, and it works equally well for demonstrating screen flows and on-screen interactivity. It also has built-in features for sharing mockups and collecting feedback. FieldTest App is another browser-based application for rapidly prototyping mobile apps. Upon uploading images of your different screens, you can superimpose hotspots that respond to different types of device interaction. More advanced and better for testing than interactive PDFs, it pretty much eliminates the need for Acrobat.

HTML Prototypes The main difference between HTML prototypes and interactive wireframes, Flash-based prototypes (which Iâ&#x20AC;&#x2122;ve deliberately left out of this chapter) and any other methods that people use to simulate mobile or Web application experiences is that the former are created in the same medium that the end product will ultimately be rendered in. You could certainly introduce HTML prototypes at any BY DENIS KARDYS

| 267

point in the process, but if you are building a mobile Web application or a website (mobile, responsive or otherwise), I can’t stress enough the importance of getting into HTML early on. A word of caution, though. Building and writing the HTML and CSS can be problematic if you don’t yet have a clear sense of direction, a firm grasp of the content and a well-planned information architecture. Without this foundation, you could get caught up fiddling around with ideas but not making much progress, or you could end up arbitrarily ping-ponging between concepts. If you have a shared vision of the goals of the project and general agreement about the priorities of content, then HTML prototyping can be a very valuable tool. HTML prototyping makes ideas less hypothetical than any other method, allowing your design decisions to be tested in the actual context that they are being designed for. For the mobile Web, you can get an actual, exact indication of the extent to which the content will fit in any particular viewport at any given time. For touch-enabled applications, you can hold, tap or swipe the interface on an actual handheld device! A recurring theme here has been purposeful deliverables. When you show a client or stakeholder a static design deliverable and then invite feedback on interactive behavior, you’re undercutting your own design argument. Instead of giving your design decisions a chance to stand on their own and prove their effectiveness, you’re putting them on the chopping block, just waiting for somebody to make a remark like, “…But I’m just not sure people will know to click it.” We experienced a case like this while working on wireframes for a project just recently. Our client wanted their audience members to be able to self-select their role, in order to reveal a menu of relevant links and targeted calls to action. We approached it in a few ways and ultimately proposed a standard select box to display role types.

268

HANDS-ON DESIGN FOR MOBILE

Once the user has selected a role, a panel would reveal the targeted links and content. We spent 45 minutes on one call and another 20 on a follow-up call debating the likelihood that a person visiting the website would realize that they could (or should) tap the select box control to reveal more information. To be fair, both we and our client had valid points, so it wasn’t a matter of which option was right, but rather a matter of recognizing futility. Seven people spent 65 minutes, for a total of 455 combined minutes, debating the imaginary behavior of website users. An observation: it would not have taken 455 minutes to run a simple test, using an HTML prototype to determine whether people would find the select box to be intuitive. !e best place to decide about interaction is in the medium where that interaction will occur. HTML prototypes also enable you to test a design more rigorously. Designing components that look perfect and behave seamlessly at predefined screen sizes is relatively easy. But what happens when the design is shown in a different-sized viewport? HTML prototypes enable you to test the awkward spaces between breakpoints; for example, how the design will hold up on screens smaller than 320 × 480 pixels or bigger than 480 × 800. A mobile view of a website sometimes has a drastically different design than a desktop view. Depending on how you’ve set up the media queries, the design would switch from the desktop to mobile view at a particular breakpoint. On paper, this works fine, but the HTML prototypes will reveal any flaws—any places where the design looks incongruous and out of place. Identifying design flaws early on allows for smarter development and resource allocation later on in the project. In fact, these prototypes are your best tool for validating or disproving your design hypothesis. In the event that your design turns out to have problems—which it inevitably will—HTML prototypes are also an extremely malleable format for making changes. Because you are BY DENIS KARDYS

| 269

prototyping in the same medium that browsers render in, prototyping can continue (if desired) well into the development phase.

HTML Prototyping Tools HTML prototyping can be a bit daunting for those who don’t spend the majority of their time writing front-end code. Critics of HTML prototyping argue that it’s too cumbersome to be practical. The perception by many is that, in order to prototype in HTML effectively, you need advanced front-end development skills. While it wouldn’t hurt for a designer to develop those skills, HTML and CSS expertise is certainly not a prerequisite. Software such as Axure21 will help you build an interface by dragging and dropping components into a mobile template. You can then export the project to HTML and test it in the browser or on an actual device. If you aren’t intimidated by code, then dive right in. Starting completely from scratch every time you want to build something in code doesn’t make a lot of sense. Most front-end developers have some sort of prebuilt or custom-built package they use when starting a new project. Frameworks such as Bootstrap22 and Foundation23 provide all of the boilerplates and snippets of code that you need to copy and paste your prototype together. The frameworks are robust and customizable enough to be used to build production-ready HTML for your mobile Web project. If you’re confident enough in your (or your coworkers’) abilities to handcode everything, then you might be able to build your own framework from scratch or build on a solid base. Mobile Boilerplate24 is a helpful baseline template that you can use for your mobile website or application. If you’re focused on creating task-based 21. Axure, smashed.by/axure 22. Bootstrap, smashed.by/bootstrap 23. Foundation, smashed.by/foundation 24. Boilerplate, smashed.by/boilerplate

270

HANDS-ON DESIGN FOR MOBILE

mobile Web applications and want to retain some of the elegance of a native interface, then you’ll also want to spend some time exploring jQuery Mobile.25 All of these options will get you pretty far along, but you can still take it a lot further. If you’re serious about putting together your own framework, then familiarizing yourself with OOCSS and SMACSS might be helpful. Nicole Sullivan coined the term OOCSS, “object-oriented CSS,” to refer to the practice of breaking your HTML and CSS into reusable, customizable objects. Both her framework and Jonathan Snook’s SMACSS, “scalable and modular architecture for CSS,” emphasize scalable and modular code. The beauty of these methods is that they tie in so nicely with the concepts we’ve been talking about: device and layout independence. By using HTML and CSS patterns, you create more modular code, meaning that components exist independent of any particular context. Any component should be able to adapt to fit any space on the screen. Sure, building your code might take a little more time than using prototyping software that outputs HTML, but it could save you time in the long run. Using a framework (custom or otherwise) makes it easy to swap out elements in the design, which is important in the prototyping process. It also improves collaboration between roles. Setting rules for naming conventions, structuring your HTML into reusable chunks and keeping separate CSS files for layout, style, patterns and behavior make it possible for anyone on your team to work on the code at any point, with a shared understanding on how everything is set up. With the markup being turned into reusable shared building blocks, and with the CSS that controls the aesthetics being separate from the CSS that controls the patterns and layout, it becomes possible to prototype the style and layout independently.

25. jQuery Mobile, smashed.by/jquerymobile

BY DENIS KARDYS

| 271

PROTOTYPING STYLE

Prototyping style is a reaction against BDUF and assembly-line processes. When visual design is relegated to the last step in the design process, before production begins, its impact on the overall design and experience is limited by the extent of what can be achieved by selecting typefaces and picking colors. When approved blueprints that describe content, layout and interaction are handed off to the frontend or UI designer, what leeway do they have for visual exploration? Visual design is more than just icing on the cake, yet thatâ&#x20AC;&#x2122;s not how it gets treated in our conventional processes. To realize the value that UI and style exploration can contribute to our designs, we must adopt processes that embrace it earlier on, when it can still have an impact.

Sttyle Tiles, Pattern Libraries And Prototypes It would be counterproductive to get overly literal with the UI at the onset of a project, while content and layout are still being fleshed out and evaluated. Getting a jumpstart on the UI is still possible, though, if you go a bit more abstract with what you are presenting. You might even find that you are able to have equally (if not more) productive conversations with stakeholders about their preferences. Discussing the design of the system as a whole, before attempting to present the design as it appears in a single context, is an approach that has been gaining a lot of momentum. An early advocate of this is Samantha Warren, who proposes Style Tiles26 as a means of building consensus around aesthetics and style. A style tile is a design deliverable consisting of swatches of color, typographic pairings, patterns and any other elements that help convey a visual theme. They do an excellent job of focusing conversation on the broader aspect of the visual designâ&#x20AC;&#x201D;suggesting possibilities for an overall global theme. Early agreement on thematic direction can

26. Samantha Warren, Style Tiles, smashed.by/styletiles

272

HANDS-ON DESIGN FOR MOBILE

FIGURE 6.12. How much or little do you need to show in order to convey the

look and behavior of a design? The team at Sparkbox has been experimenting with Style Prototypes (left) as a means of exploring and communicating design direction before creating fully developed pages (right).

facilitate and speed up the transition to prototyping later. As the products we design begin to reach more and more contexts, our visual designs must become more resilient than ever. We need to consider how the aesthetics we introduce support a cohesiveness that transcends any one device or channel.

BY DENIS KARDYS

| 273

To achieve this cohesion, others have approached the idea of abstraction from a totally different angle. Jeremy Keith suggests using a Pattern Primer27—an interactive guide that documents a website or application’s core components, showing both the UI and the code used. By focusing on the design of these components, you style the bits that ultimately get assembled into the final product. Because they’re created in HTML, they can be demoed in the browser or on a device and can even include interactive behaviors. Building on these ideas, others have experimented with ways to pool together the best aspects of both approaches. The team at Sparkbox uses a deliverable called a Style Prototype to explore stylistic options.28 It’s similar to a Style Tile in how it presents a visual story for each concept being pitched, but it’s put together with HTML and CSS, so that basic interactions and responsive behavior can be demoed. Built in code, this document can be presented in the browser, ideally setting realistic base expectations for any stakeholders who preview it. In our own experimentation, trying this out on content-heavy responsive websites, we’ve found that more detail is sometimes needed. In these cases, with each iteration of the Style Prototype that we deliver, we’ll increase the range of what we show. As additional components are added, the Style Prototype evolves. To get constructive feedback, though, we have had to show some elements in some degree of context. Layout is implied only to the extent that it reflects the template’s overall structure—thus, identifying that there is a header region or icon bar, a main content region and so on. The details within each need not be exact yet. The context that is shown need not be a component’s relationship to the page layout, but rather its relationship to other chunks of content. At the most basic level, it’s the difference between showing h1 to h5 headings in a sequential vertical list, as you might see in a style 27. smashed.by/patternprimer 28. smashed.by/style-pro

274

HANDS-ON DESIGN FOR MOBILE

FIGURE 6.13. By introducing additional components during each iteration,

you show how modular elements can come together harmoniously.

BY DENIS KARDYS

| 275

guide, and placing your h1 next to introductory text, or clustering masthead elements in a realistic way. Because this method allows for exploration independent of the final layout, visual design does not have to happen in its normal sequential order, directly following wireframes. In a recent project where we tried this new approach, the client dropped a new brand requirement on us halfway through the design process. In a linear process, this could have caused a ripple effect of changes to the layout, causing deviation from the already approved wireframes. To make things trickier, the implications of the new branding seemed minor at wider breakpoints—thus, getting approval on a static comp (showing the “desktop view”) would have been easy. But because we were in HTML, we immediately saw that incorporating this new logo would mean trouble at certain mid-range screen sizes. This approach made it easy to explain the issue (by showing it) and to come to a resolution. In the past, the new logo would have been awkwardly shoehorned in, looking like an afterthought at certain screen sizes. By not showing the full designs of particular pages, we reinforce the notion of designing for a system. Because producing style prototypes takes less time, we can present a greater volume of concepts, soliciting feedback and exploring more options during each round of design. At each new iteration, we reveal a bit more and explore a bit more, all while converging conceptually—like Buxton’s variation on the design funnel. Conversely, when the first design concept you show your client is a polished design, it’s inevitable that any critique or change requested by the client will feel like a compromise that waters down your concept. Better to adopt a format that encourages testing and concept generation at each stage, so that each new iteration feels like an evolution.

276

HANDS-ON DESIGN FOR MOBILE

In practice, when designing for a wide range of screen sizes and devices, this approach seems to work well as a replacement for static design comps. We are advancing the style and component UI to the point that it converges seamlessly with other more functional and layout-specific HTML prototypes. While this approach shows early promise, it does have some drawbacks as well. The biggest issue is that finding the happy medium between literal and abstract can be very challenging. If it becomes too representational, your audience will focus on critiquing it as a page; too abstract, and your audience will be confused as to what exactly they are evaluating.

PROTOTYPING FOR NATIVE APPLICATIONS Although you may find some use for Styles Tiles and interactive interface guides when designing native applications, they might also be overkill. Native applications often have their own devicespecific interface patterns and conventions to be followed. The same browser-based tools used to prototype behavior could also be used to mock up finished high-fidelity interfaces. PSD template libraries filled with iOS and Android UI elements are abundantly available for downloading on the Web, including a robust one from Teehan + Lax.29 Regardless of how you opt to mock up your native application designs, always eliminate anything that could set the wrong expectation, and do as much as you can to reveal the different modes of your app. An interface challenge that I continually struggle with when designing any app that needs to be supported on multiple platforms is whether to lean toward consistency with the deviceâ&#x20AC;&#x2122;s native conventions or lean toward consistency with the application itself across devices. When you are focusing on a single platform, using the stencils and UI kits makes a lot of sense. By following inherited 29. Teehan + Lax, smashed.by/tlax

BY DENIS KARDYS

| 277

IDEATION Collaborative sketching

Paper prototyping

Good for: most projects.

Generate a high volume of con-

Narrow down concepts.

cepts. Break mental models and reset

Validate or disprove concepts and

expectations.

scenarios.

Build consensus around design

Collect real-time feedback.

direction.

PROTOTYPING BEHAVIOR Low-fidelity HTML prototypes

Software-based low-fidelity

Good for: responsive design, Web apps

prototypes Ideal for: native apps

Test system and component be-

havior and task flows.

Evaluate design decisions.

Quickly model native device conventions.

Build consensus around features

and functionality.

HIGH-FIDELITY PROTOTYPING HTML prototypes Good for: responsive Software-based prototypes design and Web apps

Good for: native and Web apps

Simulate the behavior and appear-

Recreate and simulate native de-

ance of the final product.

vice functionality and behavior.

Test designs in a realistic context.

Gain approval or sign-off for full

development.

278

HANDS-ON DESIGN FOR MOBILE

CONTENT PLANNING Content reference

Good for: content-driven

wireframes

Medium-fidelity

websites and apps

Good for: responsive design

Audit page content.

Identify page arche-

Map content to tem-

types (templates).

plates.

Page tables

Prioritize information. Establish global tem-

wireframes

Audit component

plates.

variations.

Suggest layout break-

Model key pages.

points. Suggest system and component behavior.

PROTOTYPING STYLE HTML style and

Style Tiles

HTML pattern libraries

Good for: responsive design

interface guides

Rapidly explore and

Develop a visually

Rapidly explore

compare aesthetic

consistent interface

stylistic options in a

alternatives.

system, independent of

realistic context.

Good for: responsive design

layout or device. Establish visual style

Map reusable compo-

Build consensus

before investing in

nents to modular snip-

around design direc-

higher-fidelity proto-

pets of code.

tion.

types. Demonstrate basic interaction. FIGURE 6.14. Clearly define the purpose of each deliverable that you produce in order

to sharpen the focus of design critiques and smoothen the approval process.

BY DENIS KARDYS

| 279

conventions of the platform, you match the user’s existing mental models of the system. If you know that what you’re designing will be deployed across multiple platforms, then you may have a case for some of the more device-agnostic design approaches.

Putting It All Together We’ve covered a lot of territory so far. Given the unique needs of any project, you would not likely need (or want) to fit every technique into every project. Knowing how to decide which tools to use and how to pitch them is key, so let’s put on our project-manager hats for a moment and get logistical. What practical considerations do we need to make when implementing some of these techniques for a non-sequential, iterative project? EXAMINING ALL OF THE PIECES AND THE PURPOSE

You may have noticed a recurring theme throughout this chapter: liberating content from layout, and prototyping behavior independent of style. With every deliverable described, the emphasis has been on identifying its core purpose and restricting the range of what it’s being used to communicate. This may seem a bit counterintuitive, since we’re narrowing the focus of our design artifacts and asking for feedback on pieces rather than on the whole. All of this deconstruction might even seem to fly in the face of holistic design! I’d argue to the contrary. It’s just another shift in how we perceive what we’re building. Rarely do our websites and applications exist in a single context, even applications designed for a particular platform. So, unless we can guarantee that our designs will be presented in a single context, we can no longer evaluate our designs in a single context and call them holistic.

280

HANDS-ON DESIGN FOR MOBILE

By establishing a particular purpose for every design artifact you produce, you move toward a more modular process, one in which you use or lose different parts of the process when it makes sense to. Making them modular also makes them more quantifiable in terms of how much time you need to spend with each technique. Most importantly, modularity fosters collaboration, since you can more easily assign different people to work concurrently on different pieces of a design, everyone moving towards the same goal. CONVERGING PROCESSES

Before examining a complex model of how this might work, let’s start with an extremely basic but effective example of a nonlinear design process: building with LEGOs. Think back to building with LEGOs as a kid, or with your kids if you’re a parent like me. Now, imagine for a moment imposing a linear design process onto your LEGO-building workflow. It seems crazy, right?! This is because we’re all familiar with how fun and fast the real LEGO workflow is. Everyone has a shared idea of what’s being built. The pieces themselves are a mixture of predefined components and custom ones, making it easy for anyone to just start grabbing pieces and building—continually swapping out pieces until things fit just right. When a partially constructed idea starts to show signs of instability, weak parts are swapped out for sturdier ones. As the section that each person is working on evolves, others in the group see and take note, adjusting their design to facilitate the impending assembly—a tower, village, starship or whatever has been imagined. This is an ideal example of a convergent process! It includes ideation, concept exploration, prototyping, testing, revision, collaboration and assembly. We build things that are a bit more complex than LEGO towers, to be sure. But doesn’t it make sense for the different specialists who comprise a design team to work concurrently rather than sequentially? BY DENIS KARDYS

| 281

After all, unlike architects who build with steel and concrete, we work in the extremely malleable medium of pixels and code. When coordinating tasks for a project, instead of thinking about a relay of hand-offs, consider dumping all of the project’s pieces in front of the team at the beginning. How could the pieces be organized in a way that allows tasks related to layout, style and behavior to be carried out concurrently? WORKING WITH YOUR CLIENT

For all intents and purposes, your boss and client are one and the same when it comes to the practical implications of changing or adopting new design methods. There need to be reliable expectations of the outcome, and there need to be reliable expectations of time and cost. If you can’t set those expectations, you’ll be fighting a losing battle. In theory, prototyping is great. In reality, BDUF is practical. If you want to blur the lines a bit and move away from a BDUF or waterfall process, expect some resistance. You’ll need to address some very pragmatic considerations. Most payment schedules are tied to milestones in a project. In order for your company to get paid, the client must acknowledge in writing that satisfactory progress has been made. In order to maintain steady revenue, those milestones might need to be reached at certain points in the project’s timeline; for example, the beginning, middle and end. These periodic approvals also serve to prevent scope creep and wishy-washy decisionmaking. And these logistical considerations ensure that projects can be profitable and be delivered on time and on budget. If you want people to adopt the new approaches that you’re proposing, don’t wait around expecting others to figure out the logistics. Sell your ideas in a way that fits the existing business models— and if that’s not feasible, then consider proposing new ones.

282

HANDS-ON DESIGN FOR MOBILE

So, what can we say are the benefits when selling an appealing alternative to BDUF? 1. It gives the client more opportunities to provide input and feed-

back on the design throughout the process. 2. It shortens the projectâ&#x20AC;&#x2122;s duration by running tasks concurrently

and eliminating needless artifacts. 3. It guides the project to better results by taking a test-driven approach to design decisions. Build on this by suggesting that the pieces that make linear processes so sensible can still exist in a more iterative workflow. You simply need to find a way to do the following: 1. Redefine what your client will need to sign off on at different

points in the process. 2. Allow for iteration and changes to occur later in the project, while limiting the number of revisions in scope. 3. Make sure that changes are controlled and methodical, so that you donâ&#x20AC;&#x2122;t end up investing superfluous resources.

Getting Buy In Some of the ideas mentioned in this chapter, such as incorporating collaborative design workshops into your workflow, might not be met with much resistance. Aside from time, they pose very little risk to anyone. Few would dispute that by weeding out assumptions and implementing a more collaborative ideation process, youâ&#x20AC;&#x2122;re setting up for a better outcome to the design. But gaining support of some of the other ideas presented, such as making heavy use of prototypes and even going so far as to replace or diminish the role of static deliverables, might require a bit more tact. Wireframes and design mockups have been staples of the BY DENIS KARDYS

| 283

interaction and visual design process for many years, and clients and bosses have grown to expect them. In some cases, payment schedules are even tied to particular design hand-offs and sign-offs, so tampering with the process could have implications under the surface. Prototyping also makes people on both sides nervous, because their entire existence revolves around disproving design decisions that others have approved into the design. This leads to inevitable changes, and changes take time, which translates directly to dollars. UNDERSTANDING THE CLIENT’S RESISTANCE TO CHANGE

Adoption of or resistance to change in a process is not purely a logistical matter. There are other reasons why clients and bosses will continue to clamor for static deliverables. Being able to see a finished design early in the project makes the entire process seem more stable. Resolving the most contentious aspect of a project—i.e. what the design will look like—brings a feeling of security. By signing off on this, the client is able to definitively know what they’ll be getting and can make sure that others in their organization know it as well, adding another layer of assurance. Assuming that your goal is to deliver the best experience possible to mobile users and not just get sign-off, the question then becomes, How do we instill such confidence and a sense of security in any new process that we propose? Here are some tips on getting your bosses and other stakeholders on board: 1. Show that it’s viable.

Can you produce a final product that’s equal to or better than what you would have produced through a conventional process, with the same timeline and budget? Although it might take a little guesswork the first couple of times, you should eventually be able to calculate how much time and cost are required by your new methods.

284

HANDS-ON DESIGN FOR MOBILE

2. Bring clarity to the process.

If a client or boss can’t envision how the process will work or what types of deliverables they’ll be able to show their colleagues, then they’ll have trouble committing to any new approach. In the face of uncertainty, people retreat to the security of convention. Show working examples of prototypes and any other deliverables you want to include. 3. Inspire them.

Explain the potential that mobile devices afford the client to connect with their audience. Remind them of the shortcomings of our conventional processes. 4. Start small.

This could mean starting with something like interactive wireframes, or moving straight from sketching to HTML in a small project. Once you have a few quick wins under your belt, you might find that others in your or your client’s organization are championing this more hands-on approach. In our last six months, we’ve already seen a handful of clients eagerly accept our proposition to move away from an assembly line of static deliverables and to get on board with a more hands-on prototype-driven approach. I believe that the reason clients and bosses cling so tightly to conventional processes and deliverables is simply that no one has pitched them a practical business case for a better way to communicate design. But mobile is a powerful wedge here. By promising the client more collaboration, more active involvement in the process and a prototype-based approach to design, we alleviate many of the fears that weigh on the minds of the project’s team members. “Will I like how it looks?” “Will it address the issues we hope it will?” We’re BY DENIS KARDYS

| 285

essentially proposing to cut out inefficient documentation (which drives up costs), give the client more choices and increased involvement (through ideation and prototyping), and schedule tasks concurrently (modular and convergent processes), thus saving time and yielding a more accurate, tappable and testable version of the interface sooner. Who would argue with that?

Designing for Interconnectivity No single workflow works for every person or every project, although I sincerely hope that some of the techniques described here will be helpful to you in upcoming projects. The reality is that the technology we design for changes almost daily, so new tools and methods will always be in demand. We preach the value of fluidity in design, and yet we stick with the same rigid processes that we’ve been using for years. It’s time to apply our design scrutiny to our own processes, ruthlessly abandoning elements that are outdated or inefficient and bravely exploring new ideas. Each document we produce should contribute to a better understanding of the bigger system it’s a part of. We must be careful how we communicate our design intentions. The default choice is to proceed along the same path, opting for status-quo deliverables that continue to hinder our understanding of mobile. Or we could take a step back and ask these questions: Ŧ Do we have a system in place that enables each member of the

team to contribute to their fullest? Or does our workflow needlessly limit people by chaining them to an assembly line? Ŧ What expectation does the design documentation being produced set for the recipient? Does the documentation encourage design testing, or does it tempt them to hypothesize?

286

HANDS-ON DESIGN FOR MOBILE

Ŧ Do the techniques we employ reinforce our understanding of

mobile, or do they provoke new insights into interaction and interconnectivity? We are designers in a time of incredible opportunity. The line that divides us and the technology we surround ourselves with is blurring. Mobile devices make it possible for us to, on a whim, pluck nearly any piece of information we desire out of the air almost instantaneously. They have sensors capable of giving us feedback about our actions and capable of storing pictures and sounds of the experiences we accumulate. Even though we understand that they are simply computers and, thus, entirely replaceable, we still covet them, petting and tapping them, keeping them near our beds at night. The technology is a shell, but we’ve found ways to extend ourselves through it. During a keynote speech by futurist and inventor Ray Kurzweil at SxSW Interactive earlier this year, a man in the audience asked him at what point he thought that man and machine might converge. Kurzweil paused momentarily, reached into his jacket breast pocket and pulled out his iPhone. Looking at it thoughtfully, he said something along the lines that, to him, the distinction between whether this technology exists in our pockets or under our skin is relatively inconsequential at this point. And so where does that leave you and me—people who spend their days designing for mobile, repeatedly learning and unlearning what we know about it so that our understanding of it becomes ever keener? We are the ideators who help others appreciate its potential. We are the designers who mould the task processes that these devices enable. We’re the builders who make it all ubiquitous. With such immense potential, think of all the good that’s possible. Now, it’s just up to you to communicate it.

BY DENIS KARDYS

| 287

ABOUT THE AUTHOR

Dennis Kardys had no experience in HTML and CSS or designing for the Web when he startet at the Web design agency WSOL, so he ended up having to figure it out quite quickly. His passion is to use design to unlock the Web’s potential to enrich the lives of people and to help businesses flourish. The biggest lesson he has learned in his career is not to get attached to your ideas. “Be prepared to kill your darlings. It can be extremely hard to move beyond a good idea and recognize that even though sometimes it might be a great idea, doesn’t mean it’s the right idea.”

288

HANDS-ON DESIGN FOR MOBILE

CHAPTER SEVEN

· BY JOSH CLARK

DESIGNING FOR TOUCH

OUCH CHANGES EVERYTHING. Fingers and thumbs turn desktop conventions on their head, with touchscreens upending whole swaths of settled interface practice. Done right, touch interfaces create the sensation of interacting directly with information, of nudging and manipulating data as if it had actual physical properties. This trick of the eye and mind demands new methods: interface metaphors shift; content itself becomes the interactive control; traditional buttons, menus and tabs become secondary; and the familiar physics of the everyday world intervene. All of this means that the best touch interfaces look and behave differently from those driven by mouse and keyboard. Dennis Kardys wrote earlier that when we design for mobile we have to turn away from familiar conventions. But to what? Best practices continue to evolve as we feel our way through this “Wild West” era of mobile. I’ve got some clear ideas of where it’s headed, but the best place to start is with a new perspective—one that focuses less on visual experiences and more on physical ones. BY JOSH CLARK

| 289

Visual designers often discuss the “look and feel” of their interfaces. When the touchscreen is your palette, the feel takes on literal importance. Touch design introduces genuine ergonomic issues. That means your job as a designer has fundamentally changed.

You’re Not “Just” A Visual Designer Anymore When you venture into touch, you step into the world of industrial design. We typically consider user interface (UI) work to be a pursuit of visual, graphic and information design. All of those disciplines remain central to touchscreen UI, sure, but there’s a crucial new physical dimension. It’s not only how your pixels look, but how they feel in the hand. There are distinct similarities between designing a touch interface and designing a physical device. Because your design is stretched across a physical object and explored by human hands, your work defines how those hands interact with the hardware. You have to consider basic concerns of physical ease and comfort. Fat fingers require more forgiving controls, and they have a pesky habit of blocking the view, too. As the designer, you need to make way for them. So let’s start there, with the limitations of that age-old input device, the finger. Or more specifically: the thumb. The thumb turns out to be the most crucial digit in handheld touchscreen design, no matter how large or small the device.

The Rule of Thumb The phone is unique among other touchscreen gadgets because there are a blessedly limited number of ways to hold the thing. It has three essential grips, and that’s it. You can hold it in one hand and tap away with the other, you can hold it in two hands and work it with your thumbs or you can hold it in just one hand.

290

DESIGNING FOR TOUCH

Of those three fundamental handholds, the one-handed grip is at once the most freeing and the most limiting. It’s freeing because it lets you do things with the other hand—write, sip coffee, hold a baby—a fact that makes it the most common grip. But it’s limiting because working a phone one-handed means working it with your thumb. The opposable thumb is a marvel—they say it’s what separates us from the beasts—but when it comes to driving software, thumbs lack both reach and dexterity. This peculiar combination of freedom and constraint means you should optimize smartphone designs for the one-handed grip. It’s used most often but also requires specific design concessions. Those concessions focus almost entirely on thumb span. While a thumb can manage to sweep most of the screen on all but the most oversized phones, only about a third of the screen is in truly effortless territory—at the bottom of the screen on the side opposite the thumb. When holding a phone in the right hand, for example, the thumb falls naturally in an arc at the bottom left corner of the screen. Place primary tap targets in FIGURE 7.1. The thumb’s hot this thumb-thumping hot zone. zone is the arc of your thumb This is a big reason why the best span on the screen. Favor this designs typically pin toolbars and area for primary controls. navigation to the bottom edge of phone interfaces—precisely the opposite of typical desktop layouts. Desktop software conventions put menus at the top of the screen or window, and websites traditionally position primary navigation at the top of the page. Our limited thumb spans, however, flip that convention on BY JOSH CLARK

| 291

its head, sinking navigation and primary tap targets to the bottom of phone screens. This preference for screen bottom is more important than left versus right. We’re models of ambidexterity when slinging our phones. Right-handed users often favor their left hands (or switch to the left when writing, for example), and lefties likewise often go with their right. A modest majority of users do go with their right hand a bit more often, but it’s not a strong enough trend for designers to fret over. And besides: optimizing for one hand naturally penalizes the other. When in doubt, offer equal-opportunity access with full-width buttons or central placement for primary controls.

FIGURE 7.2. We’re a flexible lot when it comes to left- vs. right-handed phone use. On

iPhone, Evernote’s full-width buttons (left) and Instagram’s centered “Photo” button (right) cater to left and right hands equally.

292

DESIGNING FOR TOUCH

The screen-bottom rule of thumb, however, applies no matter which hand you use, and it gives you hints about how to organize the visual hierarchy of tap targets. Frequently used buttons should occupy the bottom of the screen for easy tapping, while other controls should be nudged out of harm’s way. It’s often an especially good idea to push controls that post or change data outside of the thumb zone. It’s a convention in iOS, for example, to place “Edit” buttons at the top right, within easy view but just enough of a stretch so that accidental changes are less likely. It’s the same reason that Twitter puts its “New Tweet” button up there, and why Foursquare puts its “Check In” button there, too. It’s not only simple comfort and convenience that drive screen-bottom conventions, but also the awkward fact FIGURE 7.3. Like many iOS apps, that fingers obscure the screen. Pushthe app Things places its “Edit” ing controls below the content keeps button outside the thumb zone. hovering hands out of the way. “Content on top, controls on bottom:” this time-tested, common-sense principle of industrial design keeps the display in clear view no matter what your hands are up to. It’s a layout that’s familiar to most physical devices—iPods, calculators, cell phones, bathroom scales, you name it. Keep the meat out of the way of the display.

BY JOSH CLARK

| 293

FIGURE 7.4. “Content on top, controls on bottom a basic staple of industrial design.

The Robot Always Wins The top/bottom rule is simple enough, but it’s complicated by your application environment. Android, for example, beats app designers to the bottom of the screen with the system buttons hugging the bottom of every Android gadget. (Prior to the Android 3.0 “Honeycomb” release, these were always physical buttons; starting with Android 4.0 “Ice Cream Sandwich,” they became virtual buttons.) For all the reasons already discussed, Android does the right thing in pushing those buttons to screen bottom, but it puts them into finger-baffling competition with app controls. As an app designer, if you follow your instinct to put controls below the content, you wind up stacking them on top of Android’s system buttons. This invites mistaps. Avoid stacking controls at screen bo"om. Piling controls is always a bad idea in touch interfaces, but it’s messiest at screen bottom. In this high-traffic area where a hovering thumb blocks the view, button collisions are inevitable.

294

DESIGNING FOR TOUCH

Unfortunately, the fix for Android apps is to place controls at the top of the screen to avoid crowding the system buttons. It’s not ideal: this puts navigation outside the thumb zone. And when you tap a button, the hand covers the entire screen. But these compromises are better than the bottomstacking alternative, which invites fatfinger errors. For Android phones, app navigation and controls should float to the top. This is the reverse of the convention for iPhone, whose physical “Home” button doesn’t create the same kind of competition as Android’s system buttons. Platform-specific compromises are common in touch design. While the “content on top, controls at bottom” rule FIGURE 7.5. This Android always applies, it depends who gets there home-screen layout, with first. For Android, the operating system application buttons stacked gets there first, and apps have to give way. directly above the system In iOS, apps have the freedom to claim the buttons, results in frequent tap errors. screen bottom… unless it’s a Web app.

Websites Are Apps Inside An App Here’s an inconvenient fact: websites and Web apps operate within the confines of another app, the browser. Many mobile browsers have buttons and controls, creating their own UI friction for Web navigation. In Mobile Safari for iPhone, for example, browser controls live in a toolbar at screen bottom. In that case, pinning your website navigation to screen bottom would stack it on top of the browser toolbar—again, a bad thing.

BY JOSH CLARK

| 295

Don’t pin Web navigation to screen bottom. This guideline is conveniently reinforced by the fact that position:fixed isn’t evenly supported across mobile browsers, making it challenging to consistently glue a toolbar to screen bottom in the first place. Unlike Android, the solution is not to move Web navigation to the top. Because browser chrome is already eating real estate in some mobile browsers, the last thing you should do is crowd out content even more by choking the top of the page with navigation options. That unfortunate mistake is common in responsive designs, which too often follow the lead of desktop layouts by leaving navigation at the top. In that ill-advised pattern, the first screen of every page is an identical list of navigation links, and you have to scroll before seeing any significant content. “Too many mobile Web experiences…start the conversation off with a list of navigation options instead of content,” writes Luke Wroblewski1 in his book Mobile First.2 “Time is often precious on mobile and downloads can cost money, so get people to what they came for as soon as you can.” Web experiences should lead with content and confine primary navigation to page bo"om. That’s page bottom, not screen bottom. This is the “footer anchor” navigation style that Brad Frost introduced in his “Responsive Design Patterns” chapter. Although there are many alternatives, this is the navigation layout I routinely recommend as best practice. You can see it at work in the Ad Age mobile website3 where all navigation is tucked behind a “Menu” button in a toolbar at the top of the screen. Tap the button and the entire screen fills instantly with navigation options. The menu’s appearance is instant and feels like an overlay has appeared, but in reality, it’s actually an anchor link to navigation at the bottom of the page. 1. smashed.by/lukew 2. smashed.by/mobile-first 3. smashed.by/madage

296

DESIGNING FOR TOUCH

FIGURE 7.6. Ad Age employs a simple anchor link to reveal navigation. The “Menu” but-

ton at the top of the page (left) hops you down to navigation at page bottom (right).

Tap the “Back” button to return to the top of the page. Luke favors this pattern, too:

“

This design uses a minimum amount of navigation elements (just a single link at the top), gives people an opportunity to pivot and explore when they get to the end of content, doesn’t duplicate the content of another menu, and (best of all) only requires a simple anchor link to work. That’s right: no fancy JavaScript, overlays or separate navigation pages to maintain—just an anchor that links to the bottom of the page. That’s like HTML.

“Content on top, controls on bottom” is simple enough, but as you’ve seen, it has varying outcomes for app designers when the operating BY JOSH CLARK

| 297

system or the browser claims this premium real estate first. In the end, it works like this for phones: Ŧ On iPhone, put app controls at screen bottom. Ŧ On Android, put app controls at screen top. Ŧ For the Web, favor navigation at page bottom (not screen bottom).

The specifics of every software platform shifts layout guidelines, and so does hardware form factor. When designing touch interfaces, you must pay careful attention to where hands come to rest. That changes according to the shape, size and weight of the gadget. So what happens when the touchscreen gets larger? With the iPad and other tablets, the rules change yet again.

Take Two Tablets and Call Me in the Morning Tablets create entirely new headaches (and opportunities) for touchscreen designers, e.g. the iPad relies upon entirely different ergonomics than the iPhone. Remember how the phone has just three basic grips? No such luck with tablets, which we grab, tilt, lean, cradle and clench in a whole variety of embraces, many of which depend upon stance. The rule of thumb still applies to iPad, except that the thumb zone changes. The special headache here is that the thumb zone isn’t consistent even for individual devices; it changes depending on posture. Standing up, you use two hands to tap away on a large tablet like iPad. You likely hold it halfway up the sides for leverage (hold it too close to the bottom, and the thing goes floppy). Or perhaps you have one arm wrapped around it like a clipboard while you tap with the other hand. Sitting at a table, you’re likely to prop a tablet with one hand at the lower third and tap with the other. Reclining in an armchair, you tend to rest it in your lap and tap with the other hand. Lying down or reclining, you rest the thing on your 298

DESIGNING FOR TOUCH

belly or nestled in a blanket, propping it with one hand, tapping with the other. In all of these grips, fingers fall in different places on the device. On top of that, each stance results in the device being held at varying distances. We tend to hold the iPad closest while standing, for example, and furthest while lying down or reclining. See what’s happening here? When it comes to tablets, we’re all hands. We roam all over the things—all over, that is, except the top and bottom edges. As varied as tablet grips can be, two things are true for all of them. First, we tend to hold tablets at the sides; though the specific location wanders up and down, thumbs tend to settle at the middle to top third of the screen. Second, the larger the screen, the harder it is to take in the whole thing at a glance as you can on a phone. On larger tablets, as with print design, our visual attention naturally focuses FIGURE 7.7. Because the tablet grip is typically on the top of the tablet, and at the side edges, the tablet’s thumb zone is the design’s information hiervery different from the phone’s. archy should reflect that. These factors mean eyes and thumbs naturally occupy the top half of tablets, with thumbs straddling the edges. Spreading navigation and primary controls across the bottom—the standard pattern for phones—turns out to be ergonomically hostile on tablets. Sometimes the bottom isn’t even visible at all. In the laziest and perhaps most common of positions—lying down or reclining—the bottom bezel tends to disappear into blankets, sweaters and soft bellies. BY JOSH CLARK

| 299

FIGURE 7.8. iPad apps Instapaper (left) and Twitter (right) both show good alternative

placement of controls in the tablet thumb zone. Instapaper puts its article tools in the top left and right corners, while Twitter puts its main navigation along the left edge.

Tablet navigation and other frequent controls should hug the sides or top corners for easy thumb access. Avoid forcing people to lift and haul their entire arms over to the top or bottom edges for frequent touch targets. Some arm lifting is of course inevitable. Tablets are thumb and index-finger devices, with the index finger driving interaction inside the tablet’s canvas. You have to move your arm for that, no way around it, but focusing navigation around the thumb as the anchor at least means that you can spare your arm the most frequent taps. The top corners are within thumb striking distance while also remaining in the tablet’s primary visual area. 300

DESIGNING FOR TOUCH

FIGURE 7.9. The Daily’s scrubber bar

reveals page thumbnails, only to have your finger block them from view. As with most touchscreen interfaces, the center top of the screen is a poor place for controls on tablets.

Avoid placing tablet controls at the top center. Touching the top middle requires you to cover the screen and all its content. Here again we see the prime directive of industrial design: controls should never go immediately above the content they aim to affect. The Daily for iPad, for example, offers a sliding scrubber bar at top center to scan through the issue’s pages, but when you do that, the hand covers the thumbnails (see previous page). You have to resort to weird hand contortions to even see the thumbnails while working the slider.

The Bottom Line for Tablets The Daily’s misstep uncovers an exception to the top-corner guideline for tablet controls. In some cases, controls should go to the bottom edge, even though it’s less thumb-friendly than the sides and top corners. When controls display or affect content, place them below or to the side of that content, never above. The Sydney Morning Herald’s iPad app, for example, has a novel way to quickly scan all of the articles in the day’s issue by dragging your finger along an index of page indicators at screen bottom. Because that control reveals a tall list of headlines, it’s appropriate to place those controls at screen bottom; placing them at the top would mean that your hand would cover the list when you touched the controls.

FIGURE 7.10. The Sydney Morning

Herald’s iPad app lists the page’s headlines when you tap the pageindicator dots at screen bottom. The controls’ position leaves the headlines unobscured.

BY JOSH CLARK

| 301

So is it top or bottom? Here’s the difference: Ŧ The iPad’s top corners are ideal for navigation buttons and onetap tools for actions like sharing, favoriting or deleting. Ŧ The bottom of iPad apps is best for controls that browse or preview content in the canvas above. This is why page thumbnails for browsing books or magazines are best placed at bottom. Similarly, if you designed a map-building app that included a palette of landmarks to drag into your map, that palette should go at the bottom so you can interact with your map without constantly covering it with your whole arm.

The Hybrids As I write this very chapter, a whole new category of touch devices has just flooded the consumer market in coordination with the release of Windows 8: touchscreen laptops and tablet–keyboard combos. Some of these so-called hybrids allow you to pluck off the screen and use it unburdened of its keyboard, like a regular tablet, but it’s the combination of touch and keyboard that create a new ergonomic environment… and fresh demands on designers. The new wrinkle: hybrids require us to move our hands back and forth between the keyboard and the touchscreen just behind it. Before this new onslaught of hybrids arrived, many (including a dismissive Steve Jobs) criticized the concept as untenable: people wouldn’t want to shuttle their hands back and forth to point at the screen. The effort would be too much, too inefficient, and the result would be the fatigue of “gorilla arms.” It’s a criticism leveled at Minority Report-style interfaces of science fiction, too: who wants to work with your arms constantly in the air? Early returns suggest those initial worries were unfounded. People do embrace touch with these hybrids, but they do it by barely lifting their arms. In usability studies by John Whalen of Brilliant 302

DESIGNING FOR TOUCH

Experience4 and by Intel,5 newcomers shifted naturally to interacting directly with the touchscreen, ignoring any mouse or trackpad. Despite the availability (and greater precision) of these time-tested pointers, people said the touchscreen felt more intimate and direct. The hand became their preferred pointer for buttons, scrolling, you name it. Even expert users accustomed to tabbing between fields switched to independently selecting form fields by touch. There seems to be something irresistible about the touchscreen, even when more precise or efficient options are available. But what about those gorilla arms? John Whalenâ&#x20AC;&#x2122;s research found that people avoid raising their arms with hybrids by instead resting them alongside the keyboard, keeping a loose grip at the bottom corners of the screen. (Among other things, this grip helps to steady a sometimes floppy laptop screen when you tap at it.) Here again, the rule of thumb calls the shots. Placing primary controls and navigation in easy reach of bottom-corner thumbs means you avoid gorilla arms. The result is a vertically flipped version of the thumb zone we saw for standalone tablets.

FIGURE 7.11. The hot zone for thumbs on hybrid devices is nearly opposite the hot

zone for index fingers.

4. smashed.by/brilliantexperience 5. smashed.by/intel-study

BY JOSH CLARK

| 303

Not everyone adopts the bottom grip, though. Others (especially newcomers) go freeform, jabbing their index finger at the screen. This approach unhinges the hands from the screen edges, giving freedom to roam the interface, though the center of the screen tends to be an easier touch than the corners. Trouble is, this finger hot zone is exactly the reverse of the thumb zone. The upshot: optimizing for thumbs means a subpar experience for the index finger—and vice versa. One layout has to win, though, and as with every other touch device, the winner should always be the thumbs. As it turns out, hybrid users begin to prefer thumb use over time, with expert users going nearly all thumbs, reaching them in and out of the screen from the edges to drive interaction. Once again, thumbs are the primary utility pointer. Cluster primary controls and gestures for hybrid screens around the bo"om corners and sides. That’s one reason Windows 8 uses edge gestures to summon system and app controls. A swipe from the right edge conjures the system charms, and a swipe from the bottom edge brings up a shelf of app tools. Both are easy thumb gestures.

A Touching Problem for Responsive Design What all of this adds up to: input type and grip should drive the placement of controls, not screen size. For Web designers in particular, this is a big headache. For most of its short history, Web design practice has focused on the visual, on screen size. It’s not yet in our industry’s DNA to consider physicality and environment in our layouts. That’s why many are still surprised at the idea that they can’t just use their legacy desktop layout on iPad, even though the screen size is the same. The layout looks good, sure, but that rarely means it’s also finger-friendly.

304

DESIGNING FOR TOUCH

The rise of the hybrids means touch is no longer the sole province of phones and tablets. It’s arrived on desktops and laptops, too. Most desktop website layouts, however, are not optimized for touch. They challenge our clumsy fingers and thumbs with small touch targets for links and menus, or they lean on hover interactions that can’t be triggered by touch at all. Few websites place primary navigation in easy reach of the thumb zone for either tablets or hybrids; they favor cursor-friendly screen-top navigation instead. Ideally, we would all tweak our CSS to accommodate a range of input types in the same way responsive design has encouraged us to accommodate a range of screen sizes. Responsive Web designers have so far used screen size as a proxy to assume support for touch. “If it’s a small screen, it’s touch. If it’s a big screen, it’s mouse-driven.” That distinction was already in trouble with large tablets like the iPad, and hybrids break that approach even more. Unfortunately, we don’t yet have media queries to specifically target touch devices, but that may change soon. Recent draft proposals for CSS4 include a pointer media query6 to target gadgets with “fine” or “coarse” pointing tools. A mouse, trackpad, stylus or any other precision accessory would be a fine pointer, while fingers would be coarse. This would allow you to create specific rules to pamper fat fingers: /* Make input fields taller for touch */ @media (pointer:coarse) { input[type="text"] { min-height:44px; } }

6. smashed.by/mediaqueries

BY JOSH CLARK

| 305

This will get us part of the way, though it’s not clear whether a browser with a keyboard–mouse and a touchscreen should identify itself as coarse or fine. Even better would be targeting the combination specifically. As we’ve already seen, the layout for a touch– keyboard hybrid should be different from that of a touch-only tablet, because the ergonomics are different. That makes it important to identify not only the availability of touch but whether it’s combined with other input types. It would be helpful if media queries could target additional input types. While we’re at it, it would be great to have HTTP headers that announce to the back-end server what type of device it’s dealing with: “Hi, I’m a touchscreen!” “Howdy, I’m a touch–keyboard hybrid.” “Greetings, I have no screen at all…” Until we get these “Hello, my name is” name tags in CSS or HTTP, we have to make do. There’s only one sensible way to do that: Assume that every screen your website serves is a touchscreen. If a device can be used for touch, its interface should be finger-friendly. This isn’t a problem that’s specific to touch, either. A new desktop design language is needed, one that replaces cursor-only interactions with conventions flexible enough to handle any of several potential input styles. For the moment, that means covering touch-only, keyboard and mouse or these new touch– keyboard hybrids. It won’t stop there; even more input methods are on their way. Windows 8 is one of the first ambitious—and imperfect—efforts to try to address this thorny issue. It’s the first attempt at an operating system whose interface can handle any input (from handwriting to speech to touch) and any output (screens of any size or noscreen spoken experiences). That’s a hard problem, and Microsoft is 306

DESIGNING FOR TOUCH

wrestling with it early, but it’s a problem the rest of us will have to address in the very near future. How do we build this new touch-and-every-other-input desktop experience? This is The Mobile Book, not The Reimagined Desktop Book, and spelling out entirely new desktop strategies stretches the mission a bit. But here are a few quick wins that will make desktop designs work for both cursor and touch. Ŧ Pin controls to the left and right edges for easy thumb access on

both tablets and hybrids. Ŧ Favor the left for primary controls. Most index-finger users use their right hand for poking the canvas, leaving the left hand in place for thumb navigation. Ŧ Treat hover as an enhancement, not a requirement. Ŧ Make sure all touch targets are large enough to accommodate fat fingers.

Too Big to Fail Everything discussed here so far has to do with how you hold the touchscreen. The way you handle a device dictates placement of controls, but it’s your finger size that decides how big those controls should be. Touch designers need to make tap targets too big to fail, chunky targets that we can tap without crossing our eyes in concentration. Just how big is big enough when it comes to tap targets? The answer starts with a more fundamental question: what’s the size of a fingertip? All the platforms offer guidance here, but as usual Apple is the most opinionated in design matters, offering what I consider to be the best tap-size guideline. The iOS Human Interface Guidelines7 suggest that all touch targets should be at least 44 points, or

7. smashed.by/apple-hig

BY JOSH CLARK

| 307

about 1/4 inch or 7 mm. Points are Apple’s own measure for iOS apps (not to be confused with the familiar points and picas from print measurements), but 44 remains the magic number for the Web, too. Way back before third-party apps were allowed on the iPhone, Apple published Web app guidelines calling for 44-pixel tap targets, and the advice remains solid. Make tap targets a minimum of 44 pixels. For those of us accustomed to designing for precision pointers like mouse and cursor, controls this big feel enormous and toylike—almost ridiculous. Embrace it. Humongous buttons and gigantic tap targets are not only easy to hit but they’re also easy to see for the occasionally distracted mobile user. While 44 should be your working minimum, it’s okay to go bigger—sometimes much bigger—to provide effortless tap targets, particularly for larger screens. Like Apple, Microsoft’s design guidelines for Windows apps8 recommend a 7 mm minimum for touch targets but also suggests moving up to 9 mm (or 50 pixels) for times when you “can’t afford accidental taps.” You might do this if touching the wrong target would require “more than two gestures, five seconds or a major context change to correct.” On the other hand, sometimes you’re forced to go smaller. In a perfect world, all tap targets would be a minimum 44 × 44, but alas, compromise is sometimes necessary. Even the iPhone’s standard controls fudge Apple’s 44 rule from time to time. In the iOS keyboard, for example, keys are 44 points tall but only 30 points wide—similarly, in landscape view, the buttons are 44 points wide but 38 points high. Apple doesn’t have much choice here; it’s crucial to include the full QWERTY keyboard in this view, but all the keys just won’t fit as 44 × 44 buttons. Something’s gotta give.

8. smashed.by/msdnapps

308

DESIGNING FOR TOUCH

When limited space puts the squeeze on tap targets, here’s the rule I’ve found works best: as long as a tap target is at least 44 points/ pixels in one dimension, you can squeeze the other as small as 30 points/pixels if you really must. That means: the practical minimum size for any tap target is 44 × 30.

Don’t Crowd Me Your faithful author spent many years of his misspent youth with a svelte Casio calculator watch strapped to his wrist before finally retiring it in 1985. The problem wasn’t just its tiny controls or its dampening effect on my prom-king prospects. The buttons were too close together. You’d aim for a five, but come up with a two or an eight, who knows—it was more Wheel of Fortune than calculator. Button size, in other words, isn’t the only determining factor of tap accuracy; you have to consider spacing, too.

FIGURE 7.12. The trouble with my beloved Casio Databank Gold wasn’t just its

small buttons; it’s that they were too close together. Photo by Jon Rawlinson: smashed.by/flickr-photos.

BY JOSH CLARK

| 309

When designing for small screens, you’ll inevitably be tempted to deal with that challenge by crowding the interface. “I’ll just nudge these a little closer. I’ll just add one more button to this toolbar.” To quote a popular phrase of the calculator-watch heyday: “Just say no.” !e closer you squeeze bu"ons together, the larger those bu"ons should be. Consider a pair of VoIP calling apps for iPhone, Skype and Call Global. Both jam their keypad buttons close together, but they compensate by making them much larger than the 44 × 44 minimum. Despite their close proximity, the buttons remain easy to hit.

FIGURE 7.13. Call Global (left) and Skype (right) crowd their buttons but make them

bigger to compensate. Call Global, however, invites mistaps with the skinny buttons stacked above the navigation tab bar.

310

DESIGNING FOR TOUCH

Where the two apps differ, though, is at screen bottom. Both apps stack buttons right on top of the navigation tab bar which, as discussed above, is never ideal. But because those buttons are so big in Skype, the problem is mitigated. In Call Global, though, the buttons hugging the top of the tab bar are too skinny, and their proximity means that mistaps are likely. Even though they’re technically larger than the 44 × 30 minimum, the lack of spacing and the errorprone stacking at screen bottom invite trouble. The layout calls for bigger buttons or more generous spacing. While it might seem counterintuitive, the success of even smallscreen touch interfaces relies on big elements, chunky buttons and airy spacing. The screens may be tiny, but our fingers (and often, our attention spans) are not. Design with fat fingers in mind. Making your design comfortable for hands and fingers is important but still only half the battle, merely the mechanical part of your design. The subtle work happens when you turn to the interaction.

A New Kind of Illusion Every digital interface is illusion, a thin layer of magic stretched over the churn of ones and zeroes below. For the first time, though, touchscreens present the opportunity to create the illusion that there is no illusion. When you remove the cursor and the mouse, these prosthetics that have driven devices for 25 years, all that seems to remain is you and the device, or better… you and the content. This shift brings with it a set of psychological, cognitive, ergonomic and even emotional considerations that are entirely new and specific to touch design. All of them require fresh perspective for designers and a new set of design patterns. Touch will help us sweep away decades of buttons, menus, folders and tabs—all the administrative debris we’ve accumulated over three decades of desktop computing—to work directly with content. Greg Nudelman explores

BY JOSH CLARK

| 311

this concept in his extra chapter with the focus on immersive apps that shoulder aside interface chrome to make way for content. This isn’t change for change’s sake. New metaphors and interactions are necessary, in part, for simple comfort and ergonomics, the topics that have dominated this chapter. Touchscreen buttons demand both physical and cognitive effort to hit, and that effort grows with the size of the screen. That tiny button in the top left of so many iPad apps is not easy enough to hit, for example, yet I’m asked to hit it all the time—to go back, to browse an app’s hierarchy, and so on. I have this big expansive screen, but I’m constantly required to focus my activity on this tiny little patch of pixels in the top left corner. Fitts’ law9 comes into play here. Fitts’ law describes how long it takes to hit a target with a tool or pointer, or move an object to a target. The whole thing boils down to a common-sense conclusion: the smaller the target and the further away it is, the harder it is to hit. (This is why golf is such a miserable sport.) Fitts’ law originally described hitting targets in the physical world, and its underlying mathematical models were found to fit screen interfaces, too. Turns out it applies to touchscreens, too; the bigger the touchscreen, the more closely the model applies. On phones, the problem is not so pronounced. You can sweep the entire screen with your thumb. Buttons throughout iOS—whether on iPad or iPhone—are the exact same size, yet on iPad, they’re harder to use because of the screen size. iPad buttons ask for concentration and effort, because you have to heave the giant meat pointer called your arm up and over. Don’t make people concentrate on hitting little buttons. Let us swipe at the whole screen, not just a little tiny plot of it. Big screens demand big gestures.

9. Wikipedia, “Fitt’s Law,” smashed.by/fittslaw.

312

DESIGNING FOR TOUCH

Do You Even Need Buttons? Pinball HD is a simple example of this. It’s a game for iPad, and the entire screen is a button. Or really two buttons. Tap anywhere on the left to trigger the left flipper, tap anywhere on the right to trigger the right. And it makes for much better gameplay than having to grip the iPad at a specific place on the device. Think: where are opportunities to eliminate buttons? It’s not that you should never use buttons. Particularly as we stagger toward a common gesture vocabulary, we’ll need visible controls or hints to help people, especially to express and trigger abstract actions (“Send to Twitter”). But we can have supplementary gestural alternatives, too. Offer gesture shortcuts to supplement common bu"on presses. That’s what Apple did in iPad’s Mail app in iOS 5, for example, adding a gesture that lets you swipe left to right anywhere on the screen to pull out the message drawer. You can still hit the “Back” button to do the same thing, but now you can also paw at the whole FIGURE 7.14. Pinball HD sure plays a screen, no matter where your mean pinball—no buttons required. hands might rest on the device. More fingers naturally multiply the possibilities. A two-finger swipe becomes a “fast forward” to the next section or chapter of a content app or website, not just the next page. A five-finger touch might toggle between my “Sent mail” and my “Inbox” in an email app. Reeder, an iPad RSS feed reader, offers a useful and increasingly common example. If you’re browsing a specific feed, there’s a “Back” button to return to your list of feeds, but you can also pinch the BY JOSH CLARK

| 313

screen to return to the menu directly. It’s a fast, quick action that uses the entire screen as a control instead of a small portion of it. It’s also a clever metaphor that plays on the familiar pinch/spread gesture to zoom in or out of maps or photos with the illusion of direct physicality. Here, though, you zoom in and out of the information hierarchy, pinching a detail view to close it and reveal the list of detail items in the parent category. The UI bestows physicality to the underlying information architecture. This “semantic zoom” is an emerging convention that’s set to become much more popular thanks to its widespread use throughout Windows 8.

FIGURE 7.15. Reeder supplements the “Back” button with a pinch gesture to close a

detail view and return to a list of feeds, letting you use the entire screen as a control instead of a single button.

314

DESIGNING FOR TOUCH

As handy (ahem) as these multi-touch gestures might be, there are obstacles, too. Accessibility is a big one: not everyone has full mobility of hands and fingers—or even all their fingers for that matter. Discoverability is another: how are we supposed to know when abstract actions like a three-finger swipe are even available? (I’ll turn to strategies for revealing and teaching gestures in a moment.) For both reasons, it’s best to treat more abstract multi-touch gestures as alternatives, supplements to more traditional interactions. Gestures are the keyboard shortcuts of touch. They’re expert actions that let you move rapidly through the interface without mashing buttons. Zipping through time-consuming actions is the motivation behind most custom gestures in nearly all non-game apps. What makes them especially time-saving is the fact that you can just slap at the whole screen: coarse gestures instead of fine-tuned tapping. I’ve been talking about this in the context of ergonomics and concentration, but that’s not the whole story. There’s also a more fundamental conceptual issue at stake here.

Buttons Are A Hack Hacks aren’t all bad. In fact, the best hacks are touched by genius, and that’s true of buttons, too. Buttons are an inspired hack, a clever workaround for a very real problem in both the real and virtual worlds. We invented electrical switches and buttons over a century ago to control objects at a distance. From the start, these switches were mere messengers, interactive middle men carrying our intent to its true destination. They necessarily introduce a layer of separation between us and the thing we want to affect. A light switch over here to turn on a light over there is not obvious. It has to be discovered, learned, explained. When I walk into a new hotel room, it takes me a minute or two of trying switches to figure out how to turn on that BY JOSH CLARK

| 315

one light. But that minor inconvenience is far better than going into a dark room with a ladder and climbing up to screw in the light bulb. It’s an inspired hack. It’s a similar story for buttons in our virtual interfaces. We created buttons and tabs and sliders as the skeuomorphic middle men to work with digital information and actions that were beyond reach or easy representation. The button-laden graphical user interface (GUI) of the desktop is a metaphor born from the need to manipulate information with keyboard and mouse. Bu"ons are o%en necessary, and you should embrace them when you need them, but recognize that they remain workarounds. As we get new interaction models like touch, speech, natural gestures and facial recognition, it’s worth asking: do I still need that hack? Is there an opportunity to manipulate content more directly? Touch helps us reduce complication by getting rid of visual abstractions to work with content directly. Our brains evolved to navigate physical space, to work directly with objects. Don’t get trapped by the abstract metaphors of this temporary and arbitrary alternate universe of the desktop computer interface. Design for humans; design for direct interaction. Stephanie Rieger pointed out earlier that we’ve had the touchscreen for 40 years but have only begun to work out the fine points of using it for fun and effortless experiences. How does this work in practice? How do we start designing for this? What does an interface without buttons look like?

Information As Physical Object A crucial step of designing a forward-looking touch interface is to identify how to reimagine your information and data as a physical object, something you can slide and stretch and manipulate directly. “How would this information behave if I could poke at it under glass?” If you’re choosing a date range, maybe you can set aside the 316

DESIGNING FOR TOUCH

familiar desktop solution of using two date pickers. Instead, you might represent the date range itself as a physical object—and just stretch it. No buttons required. Twitter’s original iPad app eliminated the “Back” button entirely. The app was designed to encourage wide-ranging exploration, to drill a long way through tweets, profiles, URLs, hash tags. Each one of those contexts, each slice of your content journey, was represented as a panel that you could physically slide. You could literally paw through your history. No need for a “Back” button, you could just swipe each panel aside to flip through your history.

FIGURE 7.16. The original Twitter app represented your browsing history physically,

with each stop shown as a panel. Here, the panel for the previous stop, a list of tweets, peeks out from behind the current moment, a detail view of one tweet. To go back, just swipe the panel out of the way.

BY JOSH CLARK

| 317

Every popular touch-based operating system operates on a card metaphor, flipping and swiping through views, each card representing a frozen moment in your history. Thatâ&#x20AC;&#x2122;s the Webâ&#x20AC;&#x2122;s longstanding metaphor, too, for that matter. Mixing buttons into that interaction confuses the metaphor. When was the last time you flipped cards or pages in the real world by hitting a button?

FIGURE 7.17. Adobe Proto borrows the paper sketch notation of wireframing to make

digital wireframes. Here, drawing a big wavy line prompts Proto to insert a Lorem Ipsum header.

318

DESIGNING FOR TOUCH

Let the real world be your guide. One route to a gestural interface is to adopt gestures we already know. Adobe Proto is a wireframing app for iPad and Android tablets, and the app goes even further by adapting gestures from common written notation. You draw interface elements using squiggles familiar to information architects drawing paper sketches. Lay in some text with a wavy line, a video with a triangle or an image with a big X. This may not be any more efficient than creating a wireframe using Visio or OmniGraffle on a desktop, but it’s certainly the most efficient way to do it on a touchscreen. These fluid sketch-like gestures create layouts much faster than you could tapping through panels of controls. The Proto example (see previous page) borrows directly from the way we work a familiar interface—paper—but you don’t have to be so literal. You can also apply real-world physics to more abstract digital objects. Our brains naturally understand and reFIGURE 7.18. Clear’s all-gesture interface relies on simple notions spond to these virtual elements when of physicality. To insert a list item, they behave so predictably. Clear, for example, is a to-do list app just spread two items apart to make room. for iPhone that doubles as an elegant proof of concept for representing simple data objects with physical characteristics. Lists and their items can be moved like building blocks. To insert a new item into a list, just spread your fingers between two other items to make room. Cross off items by, well, crossing them off… with a swipe. Pinch a list to close BY JOSH CLARK

| 319

it. These are simple gestures for simple physical actions, all of them mapping naturally to how you might rearrange list items if they were arranged physically on the desk in front of you. Making the digital behave physically requires coming at design problems from a new perspective. TouchUp, for example, is an iPad app that applies filters or visual effects on photos. The simplest example is just painting a color on the photo, using your finger as a brush. But what if you want to change the brush size? The traditional solution for the desktop is provide either a slider or a brush palette to choose a new size. Thing is, you already have a brush—your finger—and it doesn’t change size. Offering a setting to change the impression the finger leaves on the screen would introduce uncertainty. If it doesn’t match your touch, then you have no idea what will happen until you touch the screen. The game shifts from direct interaction to abstract guesswork. TouchUp handles it differently. Instead of changing the brush size, you change the canvas size. Pinch/spread to zoom in/out. And then draw your next stroke. The finger always keeps its same physical size on the screen. When you deal with touch, you have to rethink familiar solutions. With every solution, ask yourself: does the old way still make sense, or does direct interaction demand a new approach?

Whither The Web? For the moment, the best touch interfaces are native apps. This new class of touchscreen devices creates interaction expectations that browsers aren’t yet very good at. For now at least, browsers don’t play well with gestures, and there are a couple of reasons for this. First, JavaScript gives front-end developers only the most primitive, building-block touch events: touchstart, touchend and touchmove. That makes it easy enough to detect a touch or maybe a swipe, but anything trickier starts to get complicated. Have fun cod320

DESIGNING FOR TOUCH

ing a rotate or three-finger swipe. It would be ideal if we could get events for common gestures on any DOM element: pinch, long tap, swipe, rotate and more. For now, we have to build them ourselves from scratch. And that’s for the browsers that even support touch events at all. On touch devices running older operating systems like Windows Phone 7 or Symbian, the browsers don’t support touch events at all. The second and more vexing problem is that the browsers claim useful gestures for themselves. Like any app, the browser has its own controls and conventions, including gestures. Pinch/spread already has meaning in the browser, as does the long tap and other gestures. To use these gestures within a Web app, you’d have to override the browser’s standard behavior, a usability no-no. Both of these factors mean browsers hem in Web designers, practically limiting touch interactions to tap and swipe. The good news is that swipes are easy enough to implement, and we’re starting to see more and more websites embrace the swipe for next/previous navigation. A few website examples: Ŧ Visit Flickr on iPad, and you can swipe through the photo gal-

lery. Ŧ The New York Times has a front-page carousel that you can swipe through (on desktop, you use buttons), and you can swipe to the next and previous articles from article pages. Ŧ Google Images lets you swipe through search results. You can definitely do more than that in the browser, but it’s harder work. A fun example is Browser Ninja,10 a clone of the game app Fruit Ninja, in which pieces of fruit (or, in this case, browser icons) fly up in the air, and you swipe the screen to slice them. So you can

10. smashed.by/github-ninja

BY JOSH CLARK

| 321

build multi-touch gestures from the primitive touch events JavaScript gives you, though itâ&#x20AC;&#x2122;s not easy work. Despite this progress, Web gestures remain a big challenge. For now, native apps are the real interaction sandbox, and those proprietary environments are where most innovation will occur until standards catch up, which they no doubt eventually will.

Nice Gestures: The Standards You Can Rely On Native apps may be more technically capable than browser-based apps, but they still lack a rich gesture vocabulary. It will take time for a more sophisticated range of gestures to become common knowledge. In the meantime, only a handful of foundational gestures are common across platforms, and these are your gestural building blocks.

FIGURE 7.19. The iOS Maps app uses the double-tap technique to substitute for

what would be a hover display in a cursor-driven interface. Here, tapping on a pin displays a brief description. Tapping again on that description takes you to a detail view describing the location. 322

DESIGNING FOR TOUCH

Tap. This is, of course, the click of the touch universe, the allpurpose action to interact with any screen element. Importantly, it’s also a hover replacement. Many designers from the desktop environment regret the loss of hover from their interactive toolkit, and of course there’s no such thing as hover on the touchscreen. The best you can do is use a tap to “peek” into an object and a second tap to actually open or activate it. As it turns out, this is also how most touchscreen Web browsers handle mouseover events and CSS :hover states, triggering mouseover/hover on the first touch and a proper click on a second touch. Swipe. Like tap, swipe is so familiar that its uses seem obvious and limited: scroll and next/previous. Over time, however, some subtlety has crept into the gesture’s use. It’s used to reveal hidden panels, for example, like Android’s swipe-from-top to reveal statusbar notifications, or the edge gestures mentioned earlier in Windows 8 to reveal the charms panel or app controls. Even better, a swipe is also ideal for a style of defensive design I call gesture jiujitsu. This technique employs a swipe instead of a tap as proof of user intent, a little bit of extra effort for actions you don’t want to trigger by accident. Swipe to unlock the phone, swipe to answer a call, swipe to delete. Swipes are faster, less annoying and at least as effective as “are you sure?” confirmation dialogs. Long tap. This is the right click of touchscreens, typically summoning either a contextual menu of available actions or a preview of detail content. This convention is not evenly used among all operating systems. It’s more commonly used in Android apps than in iOS apps, for example. Its uneven use in iOS means the long tap is typically discovered there only by expert or curious users, so it’s best to use it as a shortcut alternative to going to a detail screen. For the Web, most browsers already use the long tap for contextual menus for links and images. This means Web apps must override default browser behavior to put the long tap to use—again, almost always a bad idea for usability reasons. BY JOSH CLARK

| 323

FIGURE 7.20. A long tap typically conjures a contextual menu with actions for the se-

lected element. In Twitterrific for iPad, a long tap on a tweet offers options for copying the tweet or visiting its Web page.

Pinch and spread. Perhaps the most directly physical gestures, pinch and spread typically shrink and enlarge images, maps and Web pages. As noted earlier, though, pinch and spread are increasingly used for â&#x20AC;&#x153;semantic zoom,â&#x20AC;? zooming in and out of the information hierarchy. In that case, pinch is used to close the current view and navigate up to the level above, while spread is used to explode an element into a view that displays its member elements. Double tap. Like pinch and spread, double tap is most frequently used to zoom in and out. It has few conventional uses beyond that, making the gesture ripe for experimentation. Boston Globe,11 for example, lets subscribers double-tap a headline to save the article for later reading. Even more ripe for experimentation: multi-finger gestures. Two- and three-finger swipes, a full-hand pinch, a five-finger 11 smashed.by/bostonglobe

324

DESIGNING FOR TOUCH

touch… the sheer variety of multi-touch combinations means a slew of options for speeding through an interface. Just touching the screen with several fingers at once opens up lots of possibilities: ten fingers for ten modes. Here’s the trouble: with the exception of pinch and spread, there’s no mainstream meaning attached to any multi-finger gestures. In part, that’s because most people are only just beginning to encounter the first truly multi-touch devices: tablets. While phones have been multi-touch for years, we haven’t used them that way. One-handed use and small screens have encouraged us to tap away with a single finger or thumb. The size and weight of larger tablets, however, require you to use two hands or rest the tablet on your lap, so you always have at least one full hand free. That means tablets always have five fingers available for gestures. Now we just have to figure out what to do with them. Standards have been slow to emerge here, and part of the problem is that as designers we’ve done a poor job of communicating the availability of these more abstract gestures.

Finding What You Can’t See Gestures are unlabeled, invisible. To discover them, we rely on visual clues or, even more usefully, past experience. If an interface element looks or behaves like a physical object, people will try to interact with it like one. Likewise, the less a gesture resembles a physical action, the more difficult it is to find. This explains the effectiveness of skeuomorphic designs, interfaces that clone physical objects as digital interfaces. If an interface looks like a book, it instantly suggests that we should use it like one, swiping through pages to advance through the content. Many designers poo-poo aping real objects for aesthetic reasons, however, and it’s true that this approach can quickly veer into kitsch. But the intuitive teaching value of this approach can’t be denied. BY JOSH CLARK

| 325

Don’t go for “looks like” if you can’t pull off “acts like.” Skeuomorphic design runs into trouble as a teaching device only when the designer doesn’t embrace the metaphor. For the first 18 months of the iPad, for example, the Calendar app’s leather-bound datebook didn’t behave like a datebook: you couldn’t turn its pages. The same is true of Contacts for iPad, only worse: swiping the screen to try to turn the address book’s pages actually starts deleting content. This dangerous case of misdirection shows the damage when visual design doesn’t match interaction design. Be aware not only of the interactive opportunities your interface metaphor proposes but the opportunities it promises. Don’t make me mash buttons to browse if your design promises that I can flip pages.

FIGURE 7.21. Oops. Swipe across the page in Contacts for iPad, and

you start deleting content.

Mouse-driven experience informs expectations, too. In Maps for iOS, newcomers easily discover that tapping twice zooms in. That’s 326

DESIGNING FOR TOUCH

not something they know from the physical world but from the desktop, from double-clicking in Google Maps. But you run into trouble with gestures that have no context or history. Nobody ever figures out that you can do a two-fingered single tap in iOS Maps to zoom out. There’s no experience from either physical or software maps that would suggest even to try it. Abstract gestures require visual hints. Gestures are useful only if people can find them. Otherwise, they’re just easter eggs, hidden treats for the lucky or determined—and most users are neither. If the interface doesn’t obviously suggest a gesture, then you have to teach it explicitly to your users. Everything we know is taught, learned, observed. From our earliest age, we rely on cues in the environment to help us until we obtain mastery. As our industry turns the corner into a new style of touch-driven computing, designers need to embed similar cues to help people learn the advanced moves. All too often, designers turn to manuals, FAQs and elaborate cheatsheet overlays to perform this education about the niceties of three-finger swipes and five-finger pinches. While these are useful as references, they’re terrible learning tools. When we’re presented with too much detail too soon, the result is often overwhelming, giving us the impression that the app or website is more complicated than it actually is. A better approach is to teach gradually and contextually by adding a kind of teaching layer to your application or website. There’s a great way to learn how to do this.

Play More Video Games Video game designers are great at teaching unfamiliar interfaces. In many games, you don’t even know the goal of the game, let alone your capabilities or the obstacles you might encounter. How do you learn this stuff as a player? Not from reading a manual or from watching a screencast. You learn from playing the game. The game itBY JOSH CLARK

| 327

self teaches you how to play, drawing you in and showing you more advanced techniques as you demonstrate you’ve mastered more basic ones. Among other techniques, games lean on three tools to get this done: coaching, leveling up and power-ups. COACHING

Telling how to do something is not nearly as effective as showing, and that’s especially true of physical interfaces (which touchscreen UIs certainly are). You don’t learn to play an instrument or serve a tennis ball by reading a book. Instead, someone shows you, and then you imitate and practice. We learn by doing, and we learn best in the moment. That’s what coaching is about, and that’s what the best self-teaching interfaces do.

FIGURE 7.22. An overlay in Dead Space provides dead-simple coaching to tell

you how to move your character in the first screen of the game. Once you start moving, the overlay goes away.

328

DESIGNING FOR TOUCH

In the iPad version of the game Dead Space, the very first screen teaches you how to move, applying an overlay as a demonstration of what to do, inviting you to try it yourself. After you demonstrate that you’ve learned the lesson, the overlay goes away; one of the most important parts of coaching is knowing when a skill has been learned and when to move on to new skills. Coaching doesn’t have to be so blatant and direct; it can also be hinted with subtle animation. When the first USA TODAY app for iPhone was released, it featured a dial at the top of the screen to navigate main FIGURE 7.23. Adding an animasections. Trouble is, too many people tion to USA TODAY’s dial-style didn’t realize the dial moved and navigation control dramatically instead thought the app had only improved user awareness of how a handful of content sections. The the thing worked. designers added an animation: every time you visited the app’s main screen, the dial zipped in from the right. “Hey, that moves, maybe I can move it, too.” It worked. With the app demonstrating the motion of the control, confusion melted away and users started using the dial as intended. Even better, the USA Today app stopped running this animation after you showed you’d learned the lesson by moving the dial on your own. The best teaching interfaces mark your learning progress and adapt their coaching accordingly. That’s where leveling up comes in. LEVELING UP

Don’t teach everything all at once. We learn best by getting it in doses, building on the basics and then revealing more as we get better. Games often do this literally, dividing progress into explicit BY JOSH CLARK

| 329

levels that typically focus on teaching a new skill in the game. Likewise, when people first encounter a feature in your app is when you should offer coaching about gestures or other functionality. Most application and website experiences aren’t as linear as games, but the learning curves work the same. Teach the basic interactions first, and, once someone demonstrates they’ve learned them, move on to more complex and abstract gestures. (It’s not that people can’t use those advanced gestures whenever they like, it’s just that you don’t specifically teach them right away.) We’re most motivated to learn a new skill at the moment we discover we need it—like, for example, when you encounter a gigantic and hugely scary guy with an enormous sword. Infinity Blade is an iOS game with a wildly sophisticated combat system, but they make it easy to learn by breaking down the elements, teaching one step at a time. The game pauses at incredibly convenient moments to teach

FIGURE 7.24. Infinity Blade pauses at incredibly convenient times to offer

instruction and demonstration.

330

DESIGNING FOR TOUCH

you a relevant skill. Just when the you’re about to get your head knocked off, the game stops the action in freeze frame and offers to teach you to block or dodge. Again, the emphasis here is on both demonstration and practice. The game shows you the gesture or control to use and then waits for you; when you use the new gesture, the action continues… and your first interaction is a success. Now you’re ready to try it out on your own. Interaction demonstrations should encourage imitation and guarantee success the first time out. When the interaction is important enough, it’s okay to stop the action and force people to try the gesture in order to continue. This is the same approach that Apple used when it introduced OS X Lion, a software update that changed the way scrolling works. They turned virtual gravity upside down, so that you now moved the mouse or trackpad in the reverse direction to what you were accustomed to. To teach this change, Apple showed a little dialog right when you installed the software, explaining what was different and inviting you to try scrolling to test it out. In fact, you had to scroll because that’s the only way to get to the “Continue” button. Scroll, click the button and BOOM: you just beat level one of Apple’s operating system. Think about your app as levels. Just like the best games, you want to motivate and enable people to move from novice to expert to master. How do you teach the basics and, once that’s done, the advanced maneuvers? All too often we treat our apps and websites as just one level. We do a quick introduction and then release our users into the cold wilderness of our software. Embracing the concept of leveling up means you follow and teach people throughout their journey to mastery. And every so often, you should give a reward for their progress.

BY JOSH CLARK

| 331

POWER-UPS

In games, as you get better, you earn power-ups, little advantages that turbo-boost your game through some extra speed or special ability. If gestures are the shortcuts of touch, then power-ups are the shortcuts of video games—usable by anyone but especially efficient in the hands of an expert. They not only unlock new abilities but they’re rewards, too, markers of your progress as you move through the game’s levels. Teaching an advanced or abstract gesture is like delivering a power-up, and it gives users a similar thrill of satisfaction that game players get from learning a new game skill. Twitter’s official iPhone app offers a great example of where designers should have used a power-up to teach an abstract gesture. When TwitFIGURE 7.25. This simple mockter overhauled its app in late 2011, up shows how Twitter could it moved access to direct messages improve discovery of its “Direct (DMs) off of the main navigation, Messages” gesture shortcut. burying them a level deeper in the After tapping the “DM” button app’s “Me” tab. For heavy DM users, ten times, the app should show that meant two taps every time they an instructional message with a wanted to check direct messages. To gesture animation. ease this burden, Twitter thoughtfully provided a gesture for easier access: swipe up from the “Me” tab to go directly to your direct messages. Trouble is, they never told anyone about it, so most people don’t know the option is there. 332

DESIGNING FOR TOUCH

It’s useful to let people learn the slow way before you teach shortcuts. In the case of Twitter, learning to tap the “Me” tab and then the “Direct Messages” button reinforces the app’s mental model, teaching people where DMs live inside the app. But after doing that five or ten times, you’ve demonstrated that you’ve learned that route. What Twitter should do at that point is deliver a power-up. After the tenth time tapping the “Direct Messages” button, the app should reveal the gesture shortcut with an animation to demonstrate, and then make you do the gesture in order to continue. The fact that Twitter and other apps need to teach this kind of gesture in the first place only points up the fact that we don’t yet have many common standards for gestural interfaces. That’s even more true for multi-touch gestures. That makes it both an exciting and overwhelming time for designers and consumers alike. We’re all exploring this new realm of touch interaction together.

This Is A Time To Be Generous As we’ve seen over and over again in the last few years, the growing range of devices and platforms continues to make our work both more exciting and more challenging. That means our job is getting harder, but it’s also our job, period. The ideal of the Web, after all, is a platform that can be accessed from any device, no matter what its input or output method. Like any period of invention and experimentation, sharing is more important than ever. We are, all of us together, crafting the future of a new kind of interaction with information and services: direct manipulation by touch. Opportunities for this kind of invention don’t come along very often. Much of this is new, and everyone’s flying by the seat of their pants. We need conventions as soon as possible… but no sooner. It’s dangerous to lock in on ideas before they’re fully baked.

BY JOSH CLARK

| 333

That means it’s more important than ever to try new things, to throw everything we’ve got at the drawing board. As designers, we need to talk to each other, examine each other’s work, share ideas for gestural conventions and commit to establishing them. We have our own coaching and leveling up to do here. Throughout, it’s important to remember that insightful work is part technical know-how (44-pixel touch targets!), but it’s empathy, imagination and an expansive worldview that generate breakthroughs. Grab your touchscreens, think big and go make something amazing.

ABOUT THE AUTHOR

Josh Clark is specialized in mobile strategy and UX. He is the founder of Global Moxie, a company which helps organizations build tapworthy mobile apps and effective websites. The most important lesson he has learned in his career: “Technical knowledge doesn’t stand up for long. Values, principles, and ideas are what count. Figure out what matters to you, why you’re in this business, and hold on tight to that. The rest will follow.”

334

DESIGNING FOR TOUCH

INDEX accessibility, 315 accordions, 145f. AMOLED, 70f. Android, 27, 294 app caching, 191 bandwidth, 150 behavior, 154 big design up front (BDUF), 263, 282f. browser caching, 193 browser detection, 49, 214 browser icons, 321 browser reflow, 207 browser-centric workflow, 113 buttons, 95, 100, 326 caching, 191 canvas-as-page, 229 card emulation mode, 76 card metaphor, 318 column drop, 133 complex navigation, 142f. concatenation, 193ff. conditional loading, 146 connected fridges, 60 connected things with screens, 59 connected TVs, 59 content choreography, 114 content planning, 243, 254, 279 content reference wireframe, 258ff. data tables, 169ff. data URI, 205ff. databases, 229 default browser, 36f. design funnel, 239f. design hypothesis, 269 device lab, 45ff. device market, 20, 29 device vendors, 12, 18 differentiation, 26ff. discoverability, 315 discovery service, 69

display: none, 180ff. downloadable browser, 36f. dual-screen behavior, 59 e-paper, 70ff. fast buttons technique, 209 fat-finger errors, 295 finger-friendly, 306 Fittsâ&#x20AC;&#x2122; law, 312 flexbox, 120 flexible AMOLED displays, 71 flexible grids, 93f. flexible images, 97 fluid typography, 125 fly-out navigation, 136 forms, 167 fragmentation, 10, 26f. front-end style guide, 127, 130 full browsers, 34 group sketching, 246 hardware layer, 30 heads-up displays (HUD), 73f. high-fidelity prototyping, 278 HTML prototyping, 268ff. HTTP Archive (HAR) file, 185f. hybrids, 302ff. hypothesis, 236 icon fonts, 153 icons, 152f. image aspect ratios, 97 image optimization, 100, 195f. information architecture, 254, 268 installed base, 21, 25f. interactive wireframes, 265, 285 Internet of Things, 59, 67 intrinsic ratio, 98f., 157 IoT technologies, 68 iPad, 156, 299ff. iPhone, 44, 55ff. JavaScript libraries, 50 jQuery plugins, 98, 125f.

latency, 178f. lazy loading, 202 localStorage, 200ff. maintenance, 220, 255 media objects, 157 media queries, 102, 134, 180 media queries, vertical, 108, 137 media query breakpoints, 105 minification, 193ff. mobile network, 51, 178 mobile operators, 14 mobile optimization techniques, 188 mobile value chain, 12 molten leading, 127, 160 Mooreâ&#x20AC;&#x2122;s Law, 230, 58 multi-layout system, 236 multi-toggle, 143 multi-touch gestures, 315, 322, 333 narratives, 248 navigation accessibility, 145 near field communication (NFC), 56 network of content, 93, 111 network speed, 105 network-area storage (NAS), 64 ninja blocks, 81ff. object-oriented CSS (OOCSS), 271 open device labs, 46f. Opera, 39 operating system sales, 23ff. operating system vendors, 12 operators, 14ff. opt-out responsive design, 172 P2P mode, 76f. page table, 255ff. paper prototypes, 250ff. pattern libraries, 272 point-of-purchase (POP), 68 progressive disclosure, 165 prototyping behavior, 264, 278 prototyping style, 272, 279

proxy browsers, 34ff., 47 Pughâ&#x20AC;&#x2122;s Funnel, 239 read/write mode, 76 resolution independence, 100 responsive breakpoint, 134 responsive image hierarchy, 122f. responsive Web design, 92 RESS, 211ff. RFID, 74ff. sales, 21ff. semantic zoom, 314, 324 sensors, 81 server-side breakpoints, 216 simulating bandwidth, 187 sketchboards, 250 sketching techniques, 245f. sketching workshops, 250 skeuomorphic designs, 325 Smart TVs, 59, 83 software layer, 30 Sonos system, 64f. sprites, 206f. stick-figure comics, 247 style prototype, 274 style tiles, 113, 131, 272 sub-navigation panel, 234 tabbed navigation, 234 tablets, 53 tap targets, 291ff., 308 thumb, 290 toggle, 140 toolbar icons, 266 vary header, 219 W3C Device APIs, 78ff. waterfalls, 185f. WebKit, 37f. webOS, 32 website weight, 176f. Windows Phone, 28f. wireframing, 260ff.