Embedded Computing Design Winter 2018

WINTER 2018 VOLUME 16 | 4 EMBEDDED-COMPUTING.COM

IOT INSIDER The IoT in China, where everyone (and thing) has a job PG 5 MUSINGS OF A MAKERPRO World Maker Faire New York 2018 PG 8

Voice Assistant Battles BEGINS ON PAGE 22

Development Kit Selector

www.embedded-computing.com/designs/iot_dev_kits

Is Embedded Tech up to the Autonomous Driving Challenge?

BEGINS ON PAGE 10

AD LIST PAGE ADVERTISER 12

Acces I/O Products – PCI Express Mini Card, mPCIe embedded I/O solutions

32

American Portwell Technology – Empowering the connected world

1

Digi-Key Electronics – Development Kit Selector

ASSOCIATE TECHNOLOGY EDITOR Laura Dolan laura.dolan@opensysmedia.com

embedded world – Exhibition & Conference … it’s a smarter world

ASSISTANT MANAGING EDITOR Lisa Daigle lisa.daigle@opensysmedia.com

7

EMBEDDED COMPUTING BRAND DIRECTOR Rich Nass rich.nass@opensysmedia.com EDITOR-IN-CHIEF Brandon Lewis brandon.lewis@opensysmedia.com TECHNOLOGY EDITOR Curt Schwaderer curt.schwaderer@opensysmedia.com

AUTOMOTIVE CONTRIBUTOR Majeed Ahmed CONTRIBUTING EDITOR Jeremy S. Cook

19

Lauterbach Inc. – Trace 32 – Debugger for RH850 from the automotive specialists

29

Lauterbach Inc. – Trace 32 – Trace-based code coverage

CREATIVE DIRECTOR Stephanie Sweet stephanie.sweet@opensysmedia.com

17

Sealevel Systems, Inc. – Push the edge

SENIOR WEB DEVELOPER Aaron Ganschow aaron.ganschow@opensysmedia.com

13

Sensors Expo/Embedded Technologies Expo & Conference 2019 – www.embeddedtechconf.com

3

WinSystems, Inc. – Empowering IIoT

SOCIAL

    

Facebook.com/Embedded.Computing.Design

@Embedded_comp

LinkedIn.com/in/EmbeddedComputing

DIRECTOR OF E-CAST LEAD GENERATION AND AUDIENCE ENGAGEMENT Joy Gilmore joy.gilmore@opensysmedia.com ONLINE EVENTS SPECIALIST Sam Vukobratovich sam.vukobratovich@opensysmedia.com

WEB DEVELOPER Paul Nelson paul.nelson@opensysmedia.com CONTRIBUTING DESIGNER Joann Toth joann.toth@opensysmedia.com EMAIL MARKETING SPECIALIST Drew Kaufman drew.kaufman@opensysmedia.com

SALES/MARKETING

SALES MANAGER Tom Varcie tom.varcie@opensysmedia.com (586) 415-6500

MARKETING MANAGER Eric Henry eric.henry@opensysmedia.com (541) 760-5361

STRATEGIC ACCOUNT MANAGER Rebecca Barker rebecca.barker@opensysmedia.com (281) 724-8021

STRATEGIC ACCOUNT MANAGER Bill Barron bill.barron@opensysmedia.com (516) 376-9838

STRATEGIC ACCOUNT MANAGER Kathleen Wackowski kathleen.wackowski@opensysmedia.com (978) 888-7367

SOUTHERN CAL REGIONAL SALES MANAGER Len Pettek len.pettek@opensysmedia.com (805) 231-9582

SOUTHWEST REGIONAL SALES MANAGER Barbara Quinlan barbara.quinlan@opensysmedia.com (480) 236-8818

INSIDE SALES Amy Russell amy.russell@opensysmedia.com ASIA-PACIFIC SALES ACCOUNT MANAGER Patty Wu patty.wu@opensysmedia.com

BUSINESS DEVELOPMENT EUROPE Rory Dear rory.dear@opensysmedia.com +44 (0)7921337498

Pinterest.com/Embedded_Design/

Instagram.com/Embedded Computing

 youtube.com/user/VideoOpenSystems

WWW.OPENSYSMEDIA.COM

EXECUTIVE VICE PRESIDENT John McHale john.mchale@opensysmedia.com

EXECUTIVE VICE PRESIDENT Rich Nass rich.nass@opensysmedia.com

EMBEDDED COMPUTING DESIGN ADVISORY BOARD Ian Ferguson, ARM Jack Ganssle, Ganssle Group Bill Gatliff, Independent Consultant Andrew Girson, Barr Group David Kleidermacher, BlackBerry Jean LaBrosse, Silicon Labs Scot Morrison, Mentor Graphics Rob Oshana, NXP Kamran Shah, Silicon Labs

2

PRESIDENT Patrick Hopper patrick.hopper@opensysmedia.com

CHIEF FINANCIAL OFFICER Rosemary Kristoff rosemary.kristoff@opensysmedia.com GROUP EDITORIAL DIRECTOR John McHale john.mchale@opensysmedia.com VITA EDITORIAL DIRECTOR Jerry Gipper jerry.gipper@opensysmedia.com TECHNOLOGY EDITOR Mariana Iriarte mariana.iriarte@opensysmedia.com SENIOR EDITOR Sally Cole sally.cole@opensysmedia.com

CREATIVE PROJECTS Chris Rassiccia chris.rassiccia@opensysmedia.com

PROJECT MANAGER Kristine Jennings kristine.jennings@opensysmedia.com

FINANCIAL ASSISTANT Emily Verhoeks emily.verhoeks@opensysmedia.com SUBSCRIPTION MANAGER subscriptions@opensysmedia.com CORPORATE OFFICE 1505 N. Hayden Rd. #105 • Scottsdale, AZ 85257 • Tel: (480) 967-5581 REPRINTS

WRIGHT’S MEDIA REPRINT COORDINATOR Wyndell Hamilton whamilton@wrightsmedia.com (281) 419-5725

Embedded Computing Design | Winter 2018

www.embedded-computing.com

EMPOWERING IIOT

Embed Success in Every Application WinSystems’ embedded single board computers are designed to support a broad range of industry applications in challenging operational environments, from energy and transportation management, to industrial IoT and automation. Our industrial embedded computer solutions enable the collection, processing and transmission of real-time data requirements at the heart of your overall system.

From standard components to full custom solutions, WinSystems delivers world-class engineering, quality and unrivaled technical support. Our full line of embedded computers, I/O cards, and accessories help you design smarter projects offering faster time to market, improved reliability, durability and longer product life cycles.

EBC-C413 EBX-compatible SBC with Latest Generation Intel® Atom™ E3800 Series Processor EPX-C414 Quad-Core Freescale i.MX 6Q Cortex A9 Industrial ARM® SBC

ENERGY

AUTOMATION

TRANSPORTATION

INDUSTRIAL IOT

TEST & MEASURE

MEDICAL

SCADA

Single Board Computers | COM Express Solutions | Power Supplies | I/O Modules | Panel PCs

817-274-7553 | www.winsystems.com ASK ABOUT OUR PRODUCT EVALUATION! 715 Stadium Drive, Arlington, Texas 76011

PX1-C415 PC/104 Form Factor SBC with PCIe/104™ OneBank™ expansion and latest generation Intel® Atom™ E3900 Series processor

QUALITY CERTIFIED

ISO 9001:2015

CONTENTS

Winter 2018 | Volume 16 | Number 4

FEATURES

opsy.st/ECDLinkedIn

10 Solving the critical challenges of autonomous driving

COVER

18

Embedded technology: Is it up to the challenge of autonomous driving?

(Stories start on pg. 10.)

Voice assistants – getting it right is harder than you think!

(Stories start on pg. 22.)

These topics plus the IoT in China and beyond, embedded security, and the community of makers – all in the Winter 2018 issue of Embedded Computing Design.

By Willard Tu, Xilinx

14 Are Self-driving cars: power systems up to the task?

By Tony Armstrong, Analog Devices

18 Solving RF complexity in the connected car

By Berry Leonard, Connie MacKenzie, David Watson, and David Schnaufer, Qorvo

21 Next-generation automotive architecture – domain computing

WEB EXTRAS

By Stefan Drouzos, Socionext

22 Getting a voice user experience right is harder than you think

@embedded_comp

26

By Jeff LeBlanc, ICS and Boston UX

26 Voice-assistant battles

By Todd Mozer, Sensory 28 Low-power solutions for always-on, always-aware voice command systems

 Video: Accelerating Time to Market with Industry-Specific Solutions (at the Advantech IoT Co-Creation Summit) With Brandon Lewis, ECD Editor-in-chief https://bit.ly/2qwrpxe

 Voltage Dividers in Power Supplies

By Frederik Dostal, Analog Devices

By Paul Beckmann, Ph.D., DSP Concepts

https://bit.ly/2ySFycL

 Fail-Safe Data Storage for IoT Applications

By Nilesh Badodekar, Cypress Semiconductor https://bit.ly/2PiOu5c

EVENTS  embedded world

10

COLUMNS 5

IOT INSIDER

The IoT in China, where everyone (and thing) has a job

By Brandon Lewis, Editor-in-Chief

6

TRACKING TRENDS

The multiple dimensions of embedded & IoT security

By Curt Schwaderer, Technology Editor

4

8

February 26-28, 2019 Nuremberg, Germany www.embedded-world.de/en

 Embedded Technologies Expo & Conference/Sensors Expo & Conference 2019 June 25-27, 2019 San Jose, CA www.sensorsexpo.com

MUSINGS OF A MAKERPRO

For the professional maker: World Maker Faire New York 2018

Published by:

By Jeremy S. Cook, Contributing Editor

10 AUTOMOTIVE ANALYSIS

V2X on the road with chips, software stacks, and design kits

By Majeed Ahmad, Automotive Contributor

Embedded Computing Design | Winter 2018

2018 OpenSystems Media® © 2018 Embedded Computing Design All registered brands and trademarks within Embedded Computing Design magazine are the property of their respective owners.

ISSN: Print 1542-6408 Online: 1542-6459 enviroink.indd 1

10/1/08 10:44:38 AM

www.embedded-computing.com

IOT INSIDER

brandon.lewis@opensysmedia.com

The IoT in China, where everyone (and thing) has a job

By Brandon Lewis, Editor-in-Chief I recently traveled to Suzhou, China, to attend Advantech’s IoT Co-Creation Summit. The trip – my first to the People’s Republic – exposed me to several cultural nuances. The most obvious of these was employment.

In Suzhou and Shanghai, it appeared that everyone has a job. The official Chinese unemployment rate stands at 3.95 percent as of October 2017, so that’s of course not true. But consider that Suzhou, a moderately-sized farming community on the southern floodplains of the Yangtze River until fairly recently, by 2015 was home to more than 6.5 million people and dozens upon dozens (upon dozens) of skyscrapers. For some perspective, size-wise, Suzhou is the equivalent of the fifth- or sixth-mostpopulated metropolitan area in the U.S.

Trump plans to invest $115 billion in smart manufacturing and another $40 billion in vocational training, which he estimates will generate an extra $500 billion in GDP and close the manufacturing gap with China in two years.

Thanks to tax incentives, Chinese companies like Fuyao Glass and CRRC Corp. are now investing billions in manufacturing facilities in the U.S. Western-led globalization helped establish China as a dominant manufacturing center, and now a wave of Western-led nationalism may ­dismantle it.

Industry 4.0: Volume production to mass customization Another factor working against China’s traditional manufacturing prominence is the emergence of Industry 4.0 and the IoT. These technologies correlate beautifully with decentralized manufacturing, as intelligent factories distributed in the most sensible geographic regions can still be connected and make collective decisions.

These more specialized, interconnected factories can focus on mass customization and on-demand manufacturing to better satisfy market needs. Of course, such facilities require enabling technologies that can be deployed, updated, and modified quickly and at scale.

In contrast to the U.S. though, Suzhou workers sweep public areas with brooms made of thatch, while others in waterproof boots fish litter out of streams and canals. At the Shanghai Pudong Airport, every single parking kiosk is staffed with an attendant.

In response, Advantech has developed an enablement strategy called “IoT Co-Creation” that addresses the requirements of distributed manufacturing. The technical approach of Co-Creation is based on technology components like Advantech’s WISE-PaaS middleware and embedded systems and modules, which the company then integrates into SolutionReady Packages (SRPs) tailored for specific industries or applications.

MAGA and the price of rice in China In an economy based on communism it’s essential that everyone, or almost everyone, has a job. So although Wes­ terners may not be accustomed to this mindset, the Chinese are just as apprehensive of U.S. economic policies as we are of initiatives like Made in China 2025.

The intent of an SRP is not to provide a complete system, but rather to offer building blocks upon which organizations can innovate. Advantech is also developing industry-specific cloud infrastructure. Users “cocreate” by applying domain expertise, programming skills, and additional hardware components to build products more quickly because the infrastructure design has been abstracted away. SRPs are already in production for factory, retail, healthcare, logistics, smart city, energy, and other environments.

As the world’s leading manufacturer, China's industrial manufacturing sector accounted for roughly 40 percent of it’s GDP in 2015. It should therefore come as no surprise that the Trump administration’s “America First” strategy presents a direct threat to China’s goal of becoming the world’s largest economy. In addition to trade tariffs,

Back to employment The concept of decentralized manufacturing is still young, but has economic and technical merit. Where does that leave China’s workforce?

Perhaps they emigrate to these distributed manufacturing centers, populate foreign lands, enrich our cultures with their influence, and soften the divides between East and West. Or, perhaps, all of this leads to more polarizing nationalism.

www.embedded-computing.com

Food for thought. See you in 2019.

Embedded Computing Design | Winter 2018

5

TRACKING TRENDS

curt.schwaderer@opensysmedia.com

The Multiple Dimensions of Embedded & IoT Security

By Curt Schwaderer, Technology Editor It all started in 2005 when Dr. Markus Kammerstetter of Trustworks KG was working on his master’s degree in security at the Technical University of Vienna and also as a security consultant for enterprise systems and products. This is where he began gaining experience and developing tools to identify security vulnerabilities working with software binaries since his customers were the users, rather than the product companies where source code is available.

In 2007 Markus started to invest in hardware security analysis tools like microscopes, probe stations, and wet chemical etching equipment. In 2012 the venture grew to the point where Trustworks, his company, was born. Today, Trustworks not only provides security testing, but also constantly performs independent security research.

Over time, Trustworks has built up an impressive set of software, firmware, and hardware tools, techniques, and capabilities for broad-based security and vulnerability testing. An example of its prowess includes an initiative to develop a high-performance FPGA [field-programmable gate array]-based Wi-Fi WPA2 implementation in order to test for weak keys, or a software tool that can extract netlist information from highresolution microchip images.

Figure 1 shows a microscope that has been augmented with homemade circuit boards and open source software to perform chip analysis. The user actually used a game controller to provide fine-grain movement and switched to slow motion to get the image in focus. All of these hardware tools and analysis capabilities provide a robust environment for analyzing product security from a system, board, chip, and software perspective.

Views on IoT and embedded security The eye-opening revelation for me talking through the equipment and product/board/chip decomposition capabilities of Trustworks is that security isn’t just focused on whether someone can guess a password, inject viruses, or gain access through software. These systems depend on a lot of on-chip security features; Trustworks dissects these features for a comprehensive chip-to-software-to-product security analysis.

FIGURE 1 A microscope that has been augmented with homemade circuit boards and open source software to perform chip analysis.

(Photo credit: Curt Schwaderer.)

6

Embedded Computing Design | Winter 2018

While meeting with the Trustworks team, Dr. Kammerstetter also cautioned that special attention be paid to established products and capabilities from larger companies that are attempting to migrate existing products to “smart” products. There can be significant security challenges migrating established designs, chips, and software that are not even a concern when developing from scratch. This situation often results in shortcuts where although the new “smart” elements are secure, vulnerabilities remain in the legacy components.

According to Markus, this that means embedded and IoT companies are beginning to make the same mistakes enterprise companies made years ago.

Citing an automotive-security analysis performed by researchers on a variety of auto manufacturer makes and models over a ten-year period, researchers analyzed the cars’ cryptographic keys, performed side-channel attacks, and extracted the keys used. This led to the discovery that one manufacturer used only a few individual keys for all of its models. Anyone who discovered those keys could essentially unlock every make and model of a given car.

This is just one example of a failure wherein the cryptographic algorithms themselves were sound, but the keys and their implementation were the weakness.

Summary Security is serious business and the stakes are becoming even higher with smart, connected embedded products joining the fold. Key takeaways for every developer of an IoT system: Start with security in mind; always perform product, board, circuit, and software security audits; and make sure that the application of security features doesn’t become an inherent vulnerability. www.embedded-computing.com

19foPr free admission 2erw e-code You

ucher

orld.de / vo

-w embedded

Nürnberg, Germany

February 26 – 28, 2019

TODAY, TOMORROW, AND BEYOND Your one-stop resource for the entire spectrum of embedded systems: discover more than 1,000 companies and get inspired by the latest trends and product developments, by renowned speakers and exciting shows.

Keep up to date:

embedded-world.de Media partners

Exhibition organizer NürnbergMesse GmbH T +49 9 11 86 06-49 12 Fachmedium der Automatisierungstechnik

F +49 9 11 86 06-49 13 visitorservice@nuernbergmesse.de Conference organizer WEKA FACHMEDIEN GmbH T +49 89 2 55 56-13 49 F +49 89 2 55 56-03 49 info@embedded-world.eu

MUSINGS OF A MAKERPRO

www.youtube.com/c/jeremyscook

For the professional maker: World Maker Faire New York 2018

By Jeremy Cook, Engineering Consultant In September 2018 I had the pleasure of attending the World Maker Faire in New York (MFNY), where a wide variety of people, robots, and equipment come together to show off what they can do and what they have made. While it was an enjoyable experience personally, from a MakerPro perspective the question is: What were some of the useful and interesting things that I saw there, and was the trip ultimately worth it?

Makers get-together As an engineering consultant and tech journalist, the biggest reason that I went to this show – the people. I knew there would be many there I’d interacted with in various forms on the internet; having everyone together in one place is really an amazing networking opportunity. If you’ve never had the chance to meet people in person whom you’ve worked with virtually, it’s a really neat experience, as people tend to be extremely familiar even if you’ve never been in the same room. Seeing someone like this “live” is both enjoyable and a great way to help cement valuable business contacts.

FIGURE 1 Dropping cars with the Hand of Man, an “interactive sculpture.” Perhaps it won’t directly go into your next project, but maybe a source of inspiration?

Photo credit: Jeremy Cook.

For those who don’t necessarily interact virtually with people that will be at this type of show, it’s easy to meet and talk to participants at their various booths, and there were three stages set up where you could hear people speak in a more formal setting. Afterwards, these people, some of whom are quite well-known on YouTube or otherwise, generally make themselves available for conversation and perhaps to answer any questions that you have.

Tools on display The Faire was also a good opportunity to see what advancements were available in 3D printers, a technique with which I’m just getting started.

The Prusa booth – a premier 3D-printer manufacturer – was extremely crowded, and for good reason, as they announced a brand-new resinbased printer. While the build volume isn’t very big compared to some

8

Embedded Computing Design | Winter 2018

other printers on the market, this type of technology produces rather spectacular prints, quality-wise. Another interesting innovation was the Palette 2 by Mosaic, a standalone device that strategically slices 3D printing filament in order to produce color prints.

These are of course interesting tools for prototyping, but what about when you need hundreds or even thousands of a product you designed? For this purpose, I was pleased to talk to a representative from Jaycon Systems, which bills itself as a company that does “quick turn prototype to production.” While I’ve yet to use a company like this, getting a chance to discuss the type of capabilities they offer is certainly interesting, and certainly something to consider once you’re ready to step into mass manufacturing.

A wide array of electronics-related organizations showed at the Faire, from large players like Digi-Key, a major sponsor of the event, and smaller companies like Turta, MellBell Electronics, and Electronic Cats. While all of these companies have neat products, talking to their representatives can also be interesting simply to meet entrepreneurs growing their product.

Should you go?

The question then becomes: Should you go to an event like this? This was the third MFNY that I’ve attended, and each time I’ve been able to meet an interesting array of people and see quite a few unique devices. Whether it’s paid off in real dollars so far, like many marketing/networking opportunities, it’s really hard to tell. However, when combined with the inspiration that I’ll certainly get from seeing other people’s work (Figure 1) and interacting with other creative individuals, I’d say it was worth the trip! www.embedded-computing.com

AUTOMOTIVE ANALYSIS

V2X on the road with chips, software stacks, and design kits

By Majeed Ahmed, Automotive Contributor The vehicle-to-everything (V2X) designs are getting real with chips, software stacks, and evaluation kits while this manifestation of connected c­ar technology eyes real-world applications such as truck platooning, green-light traffic optimization, and emergency braking.

It’s worth mentioning that the term V2X encompasses multiple flavors of the connected car maxim: vehicle-to-vehicle (V2V) communications, vehicle-to-­infrastructure (V2I) communications, and vehicle-topedestrian (V2P) communications. This holistic technology approach allows vehicles to see through almost everything, across a distance of more than a mile.

The new solutions are based on the Dedicated Short Range Communications (DSRC) technology, which is specifically designed for V2X communications as laid out in the 802.11p specification. The DSRC protocol provides one dedicated channel for real-time data communications to minimize latency and another channel to ensure the security of the safety messages.

First, Thalwil, Switzerland-based u-blox has unveiled a V2X chip that communicates on two channels. The UBX‑P3 communication chip either transports safety and service messages to bolster the security of connected cars or allows it to simultaneously communicate on the same channel using two antennas, enabling full ­coverage with no blind spots. (Figure 1.)

Next, NXP is pairing its V2X single-chip modem with Hitachi Solutions’ software stack to address extensive and costly electronic predevelopment work required for implementing DSRC-based V2X services. The design solution for V2X on-board units (OBUs) will be available in Japan by 2019 and in Europe and the U.S. soon after that. www.embedded-computing.com

The NXP SAF5400, a single-chip standalone modem, integrates advanced transceiver technology with digital baseband, MAC, and firmware.

The V2X modem chip employs software-defined radio (SDR) technology to ensure support for different regional standards with a single hardware solution. NXP claims its V2X modem chip can verify more than 2,000 messages per second.

FIGURE 1 The UBX-P3 chip claims to facilitate V2X features such as blind-spot warning, electronic brake assist, and intersection movement assist.

Image: u-blox.

An evaluation kit complements NXP’s SAF5400 single-chip modem for the development of emerging V2X applications like platooning. The RoadLINK kit (Figure 2) comes with a dedicated compensator unit and a reference design for remote second antenna support. It also facilitates testing for RF communications, spectral mask, and output power.

FIGURE 2 NXP RoadLINK technology enables reliable exchange of messages across an extended range at high speed. Image: NXP.

The V2X-based designs are out on the road a little earlier than expected, which shows how the early doubts about the viability of this connectedcar technology have been off the mark. Especially premature: The prognosis that DSRC technology would become irrelevant due to 5G’s march into the automotive arena.

Embedded Computing Design | Winter 2018

9

AUTONOMOUS VEHICLE TECHNOLOGY

Solving the critical challenges of autonomous driving

By Willard Tu, Xilinx

The technology of “autonomous driving” is building upon already widely used driver-assistance functions: Adaptive cruise control or collision avoidance, building up through temporary supervised autopilot, all the way up to fully self-driving vehicles capable of completing journeys from beginning to end with no human intervention at all.

T

he five levels of autonomy defined by the American Society of Auto­ motive Engineers (SAE) help us visualize the increasing extent of machine involvement in driving vehicles today and tomorrow. (Figure 1.)

Moving up through these various levels, increasingly accurate situational awareness is clearly required. Tasks like object recognition, spatial awareness, and positioning are core human skills. Overcoming the challenge of replicating these in a machine is only now beginning thanks to the march of Moore’s Law in chipprocessing capabilities versus cost and power consumption.

There is a strong case for self-driving vehicles on the grounds of improving road safety. Since over 80 percent of road-traffic accidents result from human error, removing human decision-making

10

could potentially lower the accident rate. The challenge here lies in elimination of human error without introducing unacceptable levels of machine errors. Tolerance of machine errors, in this most sacred and safety-critical of human activities, will be extremely low.

Autonomous-driving systems at the higher SAE levels must be developed within a framework that ensures safety of the design and all its elements. The automotivesafety standard ISO 26262 provides a framework to which any autonomous-systems development must adhere, by defining a set of Automotive Safety Integrity Levels (ASIL) and associated allowable failure rates. Moreover, the autonomous-driving systems that actually make it into the vehicles of the future will also be subject to a range of harsh environments, from tropical heat to arctic cold, with engine heat, thermal cycling, high vibration, shock, humidity, dust, and more, all of which will challenge operability and reliability. Therefore, in addition to being extremely powerful, fast, and energy-efficient, the compute platforms underpinning the autonomous-driving revolution must also satisfy qualification and certification to and beyond AEC-Q100.

The eyes, ears, and other senses of the system In the same way that a human driver relies on visual information, sounds, forces on the body, and even the sense of smell to control the vehicle in various contexts and anticipate events, autonomous systems rely on multiple sensing modalities to provide the raw data for decision-making. The machine has an advantage here over the human in that many more senses can be simulated and monitored to augment cameras working

Embedded Computing Design | Winter 2018

www.embedded-computing.com

Xilinx

www.xilinx.com

TWITTER @xilinxinc

LINKEDIN

www.linkedin.com/company/xilinx/

FACEBOOK

www.facebook.com/XilinxInc

AUTONOMOUS VEHICLE TECHNOLOGY

FIGURE 1

SAE levels of automotive autonomy, as defined by the American SAE.

FIGURE 2

Architecture of centralized processing module for autonomous and full self-driving vehicles.

in the visual spectrum and inertial sensors analogous to the human sense of balance, movement, and positioning.

These additional “senses” include radar, lidar, ultrasonic, infrared, GPS, and wireless V2X data that can provide awareness of hazards that are out of sight.

At the nexus of all of these sensing modal­ities – the eyes, ears, and more, of the system – is a centralized processing mod­ule (CPM) that must make real-time decisions based on all the data streaming continuously from the various channels.

The capabilities of this module will actually define the limits of the system’s capabilities, in terms of the autonomous-driving modes that can be performed safely.

Processing system architecture Figure 2 illustrates the functions needed for the CPM to convert the raw sensor data into safe and appropriate autonomousdriving decisions.

Among the functions shown, the Data Aggre­gation, Pre-Processing and Distri­ bution (DAPD) block interfaces with the different sensor modalities to perform basic processing, routing, and switching of information between processing units and accelerators within the processing unit. www.embedded-computing.com

High-performance serial processing performs data extraction, sensor fusion, and highlevel decision-making based upon its inputs. In some applications, neural networks will be implemented within the high-performance serial processing.

Safety processing performs real-time processing and vehicle control based upon the detected environment provided by preprocessing in the DAPD device and the results garnered from the neural network acceleration and high-performance serialprocessing elements.

Creating the CPM presents the designer with several interfacing, scalability, compliance and performance challenges. Of course, tight SWaP-C [size, weight, power, and cost] constraints always come with the automotive territory. Moreover, applicable standards, best practices, and machine-learning algorithms can be expected to evolve quickly as the higher-level autonomous-driving systems move through serious prototyping and early deployment.

To overcome these challenges, we can conceive an integrated and programmable solution that handles not only the interfacing, preprocessing, and routing ­capabilities of the DAPD, but also integrates certain safety-processing features and potentially Embedded Computing Design | Winter 2018

11

AUTONOMOUS VEHICLE TECHNOLOGY

neural-network machine-learning functionality within the same silicon or as a standalone hardware accelerator device. This opportunity is driving the emergence of a new class of automotive-grade programmable multiprocessor system-on-chip (MPSoC) ICs, which integrate multiple application processor cores and real-time processor cores with highly parallelized programmable logic and high-bandwidth industry-­standard interfaces, all on the same silicon.

With two lockstep cores, a real-time processing unit (RPU) can handle safety-critical functionality up to the level of ASIL-C. To provide the necessary functional safety, such an RPU also needs the ability to reduce, detect, and mitigate single random failures, including both hardware and single-event induced failures.

Safety processors are required to interact directly with vehicle controls such as steering, acceleration, and braking. Clearly, avoiding processing errors in this subsystem is critical. In addition to lockstep capability of the cores, further important

mitigation provisions include support for error correction codes (ECC) on caches and memories to ensure the integrity of both the application program and the data required to implement the autonomous-vehicle control. Built-in ­ self-test (BIST) during power-on is also important to ensure that the underlying hardware has no faults prior to operation and is able to functionally isolate memory and peripherals within the device if found to be defective.

Interfaces with sensor modalities throughout the vehicle will be handled by typical high-speed interfaces including MIPI, JESD204B, LVDS, and Gigabit Ethernet (GigE) for high-bandwidth interfaces such as cameras, radar, and lidar. MPSoC ICs typically contain flexible I/Os that can be configured to connect directly with MIPI, LVDS, and Gigabit serial links, leaving the higher levels of the protocols to be implemented in the programmable logic fabric, often using IP cores.

Interfaces such as CAN, SPI, I2C, and UARTs – already established within the automotive application space – are provided as ready-to-use functions within the processing system of devices.

As this technology progresses, the highest level of fully self-driving vehicles is within reach. Indeed, some high-profile autonomous car projects have already accomplished over one million miles of fully autonomous driving, with only a handful of accidents recorded.

Willard Tu is a senior director at Xilinx, where he leads global business development, product planning, and marketing strategies for the company’s automotive business. Tu has spent over two decades at the axis of the semiconductor, automotive, and computing industries. He was previously at Arm, where he evangelized CPU IP and developed ecosystems to support Arm’s growth in automotive. At NEC Electronics (now Renesas), Tu led the North American automotive sales and marketing teams, growing sales to over $150 million. Tu holds a BS degree in electrical engineering from the University of Michigan and an MBA from the University of Phoenix.

12

Embedded Computing Design | Winter 2018

www.embedded-computing.com

SAVE THE DATE!

embedded technologies expo & conference

2019

Pre-Conference: June 25, 2019 Conference & Expo: June 26-27, 2019

www.embeddedtechconf.com

Who Attends:

Embedded Technologies Conference is the ONLY embedded event focused on what is most important to designers and implementers – education and training.

Systems Level Design Engineers

Engineering Managers

Embedded/IoT engineers, developers, and implementers will experience experience unparalleled education and training covering embedded systems, IoT connectivity, edge computing, AI, machine learning, and more!

Learn from Experts

The 3-day conference will span 100+ hours, including hands-on workshops and peer-to-peer training

Implementers Executives responsible for driving strategies

Connect with the industry Join your peers and colleagues for new connections, ideas and partnerships

Explore Technologies

Together with Sensors Expo,

see hundreds of industry leading exhibitors

Total Forecasted Attendance Across All Events

9,000 – 10,000

REGISTRATION WILL BE OPEN IN DECEMBER

at www.embeddedtechconf.com/register.

Co-located with:

Contact our sales team for more information on exhibiting today: SEAN RAMAN SRAMAN@QUESTEX.COM | 617.219.8369

Partnered with:

AUTONOMOUS VEHICLE TECHNOLOGY

Self-driving cars: Are power systems up to the task?

By Tony Armstrong, Analog Devices

T

Are we ready for self-driving cars? This is a question I have been asking myself a lot recently, and maybe you have too. Of course, in my case, there is a little bit of self-interest since my teenage daughter has just started learning to drive. After her first lesson, I asked her how it went and her response surprised me a little: Driving the car itself did not seem to concern her so much; it was the people driving their cars around her. She complained how they were always close to the rear bumper, they never used their turn-signal indicators, and that they would unexpectedly cut in front of her so that they could make their exit. All fair points and ones I could sympathize with from my own experiences on the roads of Northern California.

hat got me to thinking about auto­n­omous vehicles and the fact that they will not necessarily have a human driver behind the wheel (well, they might have one, but not actually using the control mechanisms as usual). Instead, a bunch of computer code will be running on the equivalent of a mini-mainframe computer connected to an array of different sensors both inside and outside the vehicle. They will be connected to the cloud and can essentially simulate

14

the external environment around the vehicle in real time so that they can anticipate what actions must be taken based on the current traffic conditions around them. This will be done regardless of the range and scope of climatic, environmental, and traffic conditions.

crossing the street outside of a crosswalk.

Although a bicycle was present at the accident scene, it’s not believed that the bike was being ridden at the time of the incident. The victim was rushed to a nearby hospital and was pronounced dead shortly after arriving.

Unfortunately, we have seen the recent death of a cyclist who was hit by an autonomous test vehicle in Arizona. According to the local police, the bicyclist was

In the autonomous, self-driving SUV, there was a human occupant behind the wheel at the time of the incident, but this individual was not in actual control

Embedded Computing Design | Winter 2018

www.embedded-computing.com

Analog Devices www.analog.com

TWITTER

@ADI_news

LINKEDIN

www.linkedin.com/company/analog-devices/

FACEBOOK

www.facebook.com/AnalogDevicesInc

YOU TUBE

www.youtube.com/channel/ UCksc_G6DamESNilfxx2sx4w

AUTONOMOUS VEHICLE TECHNOLOGY

of the vehicle. According to the local authorities, there were no other passengers in the vehicle at the time of the incident. It should be noted that Arizona is one of the few places in the U.S. where it’s legal to test autonomous vehicles without keeping a human in the driver’s seat to take over when necessary. Nevertheless, this is not the sort of incident that inspires the public with confidence in the self-driving capabilities of autonomous vehicles.

ALL OF THESE ADVANCEMENTS WILL NECESSITATE AUTOMAKERS HAVING MANY AUTONOMOUS FEATURES IN THEIR VEHICLES, WHICH COULD POTENTIALLY ALLOW FOR AUTONOMOUS SELF-DRIVING CARS TO BE ON THE ROADS BY THE MID-2030S. The timeline to self-driving cars There can be no doubt that self-driving cars are coming, even if there are a few setbacks along the way. So, a couple good questions might be: When will we get there and how long will it take?

According to the auto industry, there are two standard terms for this transition: an evolutionary one where existing cars get there little by little (analogous to Tesla’s autopilot feature) and a revolutionary one where we have totally self-driving cars (like the ones Google is working on). It is unclear to me whether either path will succeed by itself; it will more likely end up being a symbiotic amalgamation of the two. www.embedded-computing.com

So, what are the next steps over the next couple of years? Here is a listing of advancements that will be made: ›› More advanced driver-assistance features that will be synchronized to navigation and GPS systems ›› Companies like Google will gather and accumulate data about every situation a self-driving vehicle might encounter ›› 3D mapping data of major cities will need to be intensified by the mapping companies ›› The car makers and high-tech auto systems providers will need to closely collaborate with each other to ensure that light detection, lidar, radar sensors, GPS, and cameras all work cohesively together ›› Vehicles incorporating the features listed above must be tested in all terrains and climates Looking further ahead, say to 2020, vehicles equipped with the semiautonomous features outlined above should be able to navigate through intersections, traffic lights, and stop-and-go traffic conditions. Nevertheless, even these highly autonomous cars will still require an actual human being to be up front in case of emergency situations.

Looking further ahead, say to 2024, these semiautonomous vehicles will also function normally in more stringent conditions, such as severe weather and night time. By this timeframe, lift-service providers may start using these types of cars without any driver. Of course, automakers will have to make sure their vehicles understand human signals from pedestrians, like waving them on at a crossing or intersection. All of these advancements will necessitate automakers having many autonomous features in their vehicles, which could potentially allow for autonomous self-driving cars to be on the roads by the mid-2030s.

Of course, all the advancements needed to make this timeline a reality will be a boon to the semiconductor industry, since it will supply the majority of the silicon content for the many systems needed to make it all happen. This silicon content will be made up of both digital and analog integrated circuits (ICs).

Analog ICs Fully autonomous cars will clearly have many different electronic systems with a mix of both digital and analog ICs. These will include advanced driver-assistance systems (ADAS), automated driving computers, autonomous parking assist, blind spot monitoring, intelligent cruise control, night vision, lidar, and more; the list goes on. All of these systems require a variety of different voltage rails and current levels for their correct operation; however, they can be required to be powered directly from the automobile’s battery and/or alternator and, in some instances, from a post-regulated rail from one of these rails. This is usually the case for the core voltages of VLSI digital ICs such as FPGAs and GPUs that can need operating voltages sub-1 V at currents from several up to tens of amps.

System designers must also ensure that the ADAS comply with the various noiseimmunity standards within the vehicle. In an automotive environment, switching regulators are replacing linear regulators in areas where low heat dissipation and efficiency are valued. Moreover, the switching regulator is typically the first active component on the input power bus line and therefore has a significant impact on the EMI performance of the complete converter circuit.

There are two types of EMI emissions: conducted and radiated. Conducted emissions ride on the wires and traces that connect up to a product. Since the noise is localized Embedded Computing Design | Winter 2018

15

AUTONOMOUS VEHICLE TECHNOLOGY

to a specific terminal or connector in the design, compliance with conducted emissions requirements can often be assured relatively early in the development process with a good layout or filter design as already stated.

with a 2 MHz switching frequency over its entire load range. Spread-spectrum frequency modulation is also available to lower EMI levels further.

Similarly, for applications needing a wider input range than that afforded by the LT8650S, Analog Devices also developed the LT8645S, a high-input-voltage-capable monolithic synchronous buck converter that also has low EMI emissions. As can be seen in Figure 3, it is a single-channel design delivering an 8 A output at 5 V.

However, radiated emissions are another story altogether. Everything on the board that carries current radiates an electromagnetic field. Every trace on the board is an antenna and every copper plane is a resonator. Anything other than a pure sine wave or dc voltage generates noise all over the signal spectrum. Even with careful design, a power-supply designer never really knows how bad the radiated emissions are going to be until the system gets tested, and radiated emissions testing cannot be formally performed until the design is essentially complete.

Filters are often used to reduce EMI by attenuating the strength at a certain frequency or over a range of frequencies. A portion of this energy that travels through space (radiated) is attenuated by adding metallic and magnetic shields.

The part that rides on PCB traces (conducted) is tamed by adding ferrite beads and other filters. EMI cannot be eliminated but can be attenuated to a level that is acceptable by other communication and digital components. Moreover, several regulatory bodies enforce standards to ensure compliance.

FIGURE 1

Simplified LT8650S schematic delivering 5 V at 4 A and 3.3 V at 4 A outputs, at 2 MHz.

FIGURE 2 LT8650S radiated EMI performance graph.

High-voltage converters, low EMI/EMC emissions It was because of the application constraints outlined herein that Analog Devices’ Power by Linear Group developed the LT8650S, a high-input-voltagecapable, dual-output, monolithic synchronous buck converter that also has low EMI/EMC emissions. As illustrated in Figure 1, it is a dual-channel design consisting of two high-voltage 4 A channels, delivering voltages as low as 0.8 V, enabling it to drive the lowest voltage microprocessor cores currently available.

Figure 2 shows emissions output characteristics. This improved EMI/EMC performance is not sensitive to board layout, simplifying design and reducing risk even when using two-layer PC boards.

The LT8650S can easily pass the automotive CISPR 25, Class 5 peak EMI limits

16

FIGURE 3

Embedded Computing Design | Winter 2018

Simplified LT8645S schematic delivering 5 V at 8 A output at 2 MHz. www.embedded-computing.com

Figure 4 shows emissions output characteristics. This improved EMI/EMC performance is not sensitive to board layout, simplifying design and reducing risk even when using two-layer PC boards. The LT8645S can easily pass the automotive CISPR 25, Class 5 peak EMI limits over its entire load range. Spread-spectrum frequency modulation is also available to lower EMI levels further.

What’s to come?

The proliferation of automotive systems that will be necessary for the autonomous self-driving cars (and trucks) of the future continues to gain momentum even here in the present. Of course, voltage and current levels will change; nevertheless, the requirements for low EMI/EMC emissions will not go away, and neither will the hostile environment in which they need to operate. Fortunately, there are a growing number

FIGURE 4 LT8645S radiated EMI performance graph.

SEA-18026 - IoT Design.pdf

www.embedded-computing.com

1

3/20/18

9:50 AM

of solutions to assist the system designer in the present and the future, even if the mid-2030s seems a long way off.

As for my daughter’s driving, the systems in today’s vehicles already make it easier for her to deal with the drivers around her. In the not-too-distant future, she’ll be able to sit back in her seat, relax, and enjoy the car taking her for a ride!

Tony Armstrong, marketing director for Analog Devices’ Power

by Linear Group, joined the company in May 2000. He is responsible for all aspects of the power conversion and management products, from conception through obsolescence.

Prior to joining Linear Technology (now a part of Analog Devices), Tony held various positions in marketing, sales, and operations at Siliconix Inc., Semtech Corp., Fairchild Semiconductors, and Intel Corp. (Europe). He attained a B.S. in applied mathematics from the University of Manchester, England. He can be reached at anthony.armstrong@analog.com.

Embedded Computing Design | Winter 2018

17

AUTONOMOUS VEHICLE TECHNOLOGY

Solving RF complexity in the connected car

By Berry Leonard, Connie MacKenzie, David Watson, and David Schnaufer, Qorvo

As the automotive environment becomes more complex and challenging, it will become increasingly important to use radio-frequency (RF) technologies that have already been proven to work reliably in the world of mobile devices, which is similarly complex to the world of the connected auto.

T

he complexity of wireless communications in vehicles is increasing at an extraordinary pace, and the impending arrival of 5G will pave the way for even greater reliance on RF technologies.

Today, vehicles may rely on wireless communications for a dozen or more functions, ranging from safety features and navigation to infotainment and keyless entry (Figure 1). Over the next few years, those functions will increasingly expand to include autonomous driving, as well as communications with infrastructure and the Internet of Things (IoT) – especially as the connected world transitions from 4G to 5G. The growing RF complexity already creates significant challenges for automotive manufacturers; these challenges will only increase as cars rely more heavily on wireless communications. To help make

18

the leap to 5G, the industry established the 5G Automotive Association (5GAA) in 2016 to support and guide 4G and 5G standardization, develop testing, promote solutions, and accelerate commercial availability of automotive products. In the meantime, the challenges facing automotive manufacturers include evolving their designs to support much faster cellular data rates by migrating from the CAT4 modem standard (150 Mbps download speed) to CAT16 (1 Gbps) and beyond.

Communications complexity drives search for proven solutions The RF complexity and rapid pace of change are challenging automotive manufacturers’ ability to quickly develop, test, implement, and deliver products to consumers.

Designs must integrate highly complex wireless systems while ensuring the guaranteed quality of service, reliability, and robustness that the automotive industry expects.

Furthermore, these standards must be achieved in the production design because it is usually unacceptable or even impossible to redesign products after they have been released to consumers.

To meet these challenges, automotive manufacturers need proven wireless components and modules that they can integrate into vehicles. To obtain them, they are increasingly turning to suppliers with extensive experience in the wireless networking and mobile communications industries, which have already been delivering products to smartphone manufacturers and mobile network operators for many years. This background also provides valuable expertise and guidance that can help automotive

Embedded Computing Design | Winter 2018

www.embedded-computing.com

Qorvo

www.qorvo.com

TWITTER

LINKEDIN

@QorvoInc

www.linkedin.com/company/qorvo/

FACEBOOK

www.facebook.com/Qorvo

YOU TUBE

www.youtube.com/channel/ UCArqF8UVKDuSKUVLR9XlwWQ

AUTONOMOUS VEHICLE TECHNOLOGY

manufacturers understand and overcome potential obstacles related to standards compliance, regional requirements, and antenna technology. Such expertise becomes particularly important as vehicles integrate a growing range of wireless protocols and frequency bands.

This can help to solve complex 4G and 5G challenges, such as multiple input-­ multiple output (MIMO) and carrier aggregation (CA) – two widely used techniques for delivering higher data rates.

Critical factors for automotive communication design As wireless communications become responsible for more of the car’s functions, several design factors are critical to achieving success in automotive designs.

The first is ruggedness and reliability.

Wireless components must withstand rigorous everyday use in extreme temperatures and humidity, as well as surviving physical abuse. To ensure that products continue operating reliably in these conditions, RF components designed for automotive use must be qualified to specific automotive industry standards.

IATF 16949 is the global automotive industry standard for quality management systems; the industry generally expects parts to be manufactured, assembled and tested in IATF 16949-qualified facilities. In addition, each component must survive a battery of industry-standard tests during the prerelease qualification phase, including AEC-Q100, which defines the standard tests for active components, such as switches and power amplifiers (PAs); and AEC-Q200, which covers similar tests for passive devices, such as RF filters used in Wi-Fi and cellular communications.

Dual Band Wi-Fi 2.4G/5G GPS/GNSS/BeiDou

V2X (C-V2X & DSRC)

Entertainment Center

Maintenance Cellular Radio System

Cameras

Remote Keyless Entry

eCall/OnStar

FIGURE 1

Tire Pressure Monitoring

Automotive system technologies.

Another is coexistence and interference. With so many radios inside today’s autos, engineers must always be aware of the potential for RF interference issues and take a prudent approach to avoiding them. A variety of coexistence filters are available from RF suppliers to address these interference problems and comply with different government regulations.

Designers also must factor in regional wireless standards. Smartphone makers and automotive manufacturers alike must comply with regional cellular standards and RF spectrum allocations. To keep costs down, smartphone suppliers and automotive manufacturers both try to create as few different models (stock-keeping units, or SKUs) as possible. Typically, this means four SKUs covering North America, China, EU, and the rest of the world.

TRACE 32 ®

Debugger for RH850 from the automotive specialists

DEBUGGING

NEXUS TRACING

RH850, ICU-M and GTM Debugging

Code Coverage (ISO 26262)

AUTOSAR / Multicore Debugging

Multicore Tracing

Runtime Measurement (Performance Counter)

The next factor is longevity: Smartphones are generally only used for three to five years. However, cars often remain on the road for a much longer time, for 10 years or even more. Therefore, RF components must be highly reliable and available as replacement parts for an extended period of time. www.embedded-computing.com

Sirius XM Radio

4G/5G Cellular Connectivity

eec_rh850.indd 1

Onchip, Parallel and Aurora Tracing

www.lauterbach.com/1701.html

Embedded Computing Design | Winter 2018

19

07.11.2018 12:21:20

AUTONOMOUS VEHICLE TECHNOLOGY

Also to consider: DSDA (Dual Sim Dual Active) radios. DSDA technology uses two separate transceivers and antenna pathways to support two different cellular networks simultaneously (Figure 2).

This configuration enables auto manufacturers to provide specific contracted services while offering vehicle owners the ability to add their preferred carrier.

Although DSDA is not currently widely used, as markets mature more use cases will emerge, making this technology more widespread.

Solving RF complexity From a development perspective, it is challenging to deal with the current increases in wireless connectivity while anticipating what 5G will soon bring. Even when using 4G and 5G wireless technology that has been verified in mobile devices, extensive testing is imperative to guarantee reliability and performance in automotive use, due to the additional RF complexity and operating requirements.

When designing complex subsystems such as telematics control units (TCUs) and antenna clusters, it is important to keep several principles top-of-mind.

First, integration reduces RF complexity. Integration is always a good way to reduce complexity. The more that designs use highly integrated RF front ends (RFFEs), the better chance of reducing overall design complexity. These devices combine components such as power amplifiers (PAs), switches, low noise amplifiers (LNAs), and filters into

a single prequalified module. This accelerates design and overall time to market by reducing PCB complexity and the need for external tuning. These integrated RFFEs further reduce complexity by including support for CA, advanced filtering, and diversity RF chains. CA, which is being widely deployed by operators worldwide, aggregates spectrum from multiple bands to deliver higher data rates and improve network performance. The CA landscape is extremely complex, with greater than 300 different band combinations used by different network operators. Today’s mobile device RFFEs help to solve CA problems by addressing many of these combinations, and they can help to solve the same challenges in vehicle communication systems.

Designers can also use proven RF modules and components: Many wireless device manufacturers have large portfolios of proven technology already used in wireless mobile devices. These mobile devices provide reliable and fast access to global wireless networks. Using these proven components can help to guarantee high reliability and performance in automotive wireless systems.

Another consideration is the use of coexistence and interference filters. Coexistence filters help reduce interference issues, which can cause receiver sensitivity problems and regulatory non-compliance. When using highly integrated RF modules, multiple radio transceivers operate in close proximity to each other. It is possible for the transmit power of one RF chain to exceed the power level of the signal reaching a nearby receiver, which can cause receiver-sensitivity issues. Many integrated RFFEs, therefore, include embedded filters to reduce the possibility of interference, particularly between adjacent bands. For example, coexistence filters are used in mobile devices and automotive Wi-Fi systems to enable Wi-Fi to operate concurrently with cellular Band 41.

Going forward By 2025, every new vehicle on the road will be wirelessly connected in some way.

Automotive manufacturers will rely on wireless communications for an extremely wide range of functions, including vehicle safety and control. Those functions will require support for many cellular technologies – including 2G, 3G, 4G, and 5G, as well as Wi-Fi, Bluetooth, and near-field communications (NFC). As the automotive RF environment becomes more complex and challenging, it will become increasingly important to use technology that has already been proven to communicate reliably in the ­similarly complex world of mobile devices.

Berry Leonard is Product Line Director – Automotive; Connie MacKenzie is Senior Product Marketing Manager; David Watson is Senior Marketing Manager; and David Schnaufer is Technical Marketing Manager, all at Qorvo.

Two Active Radios Operating Simultaneously

FIGURE 2

5G

LTE

How DSDA technology works.

1 Gbps DL

SIM1 (Carrier 1) OEM Contract

20

Embedded Computing Design | Winter 2018

5G

LTE

1 Gbps DL

SIM2 (Carrier 2) User Contract

www.embedded-computing.com

IN-VEHICLE INFOTAINMENT ARCHITECTURES

Next-generation automotive architecture – domain computing

By Stefan Drouzos, Socionext

Automotive displays are now a vital in-vehicle requirement, as evidenced by their broad acceptance in the consumer market. Users want to be able to use their vehicle displays as they would any other screen, both in the car and in conjunction with their mobile devices.

I

nnovative operating concepts are a crucial feature for car manufacturers to differentiate themselves in a market where displays play a central role, the number of displays is growing, and their resolutions and sizes are increasing at a rapid rate.

What kinds of display applications are currently used in cars? Instrument clusters, head-up displays, central information display, mirror replacement, passenger display, rear-seat entertainment, various controls on the doors, or air-conditioning controls are just some of the different display applications currently used in cars. The requirements that these displays must meet are just as varied as the different types of applications that use them. The instrument cluster or the headup display, in particular, is subject to strict safety requirements in accordance with the ASIL [Automotive Safety Integrity Level] standard.

Other displays require certain decoding mechanisms in order to be allowed to display protected content, for example, in accordance with the HDCP standard [High-bandwidth Digital Content Protection (HDCP) is a form of digital copy protection]. In some cases, displays must provide a minimum level of protection against tampering.

Socionext Europe

www.eu.socionext.com

www.embedded-computing.com

TWITTER

@Socionext_EU

So much data in so many different forms One approach that can be used to meet these requirements in cars is domain computing architecture. This architecture allows several displays to be controlled within one area; i.e., a domain from a central unit and an HMI computer. Remote displays are connected to the central unit via a high-speed video link that is responsible both for the transmission of the video data to the display and for communications from the display back to the central graphics unit. Use of APIX3 technology [designed to support multiple, automotive ultra-high-def resolution in car displays] here is ideal, as it has all these properties and enables speeds of up to 12 Gbps on the upstream and 187 Mbps on the downstream.

Complete safety in the display itself is equally important. Three essential components – the communication and video link receiver (APIX3) with integrated decoding, the graphic processing and monitoring of the video content, and the output unit to the display – are all brought together in an intelligent and coordinated integration of all functions. ASIL-critical information from the graphics-generating control unit, such as the head unit, must always be transmitted clear of defects by the display connection. If an error occurs, the system must automatically detect it and react to any malfunctions or incorrect data.

An integrated domain architecture in these instances offers scalability for driver assistance and infotainment systems and helps improve safety. Highly integrated devices such as display controllers and graphics-computing systems on chip offer high flexibility and efficiency both during the development phase and in the field, reducing overall system cost.

The next logical step: the fusion of computing domains to serve cross-domain applications.

Stefan Drouzas is responsible for the product management of automotive SoCs and display controllers for IoT and graphics solutions at Socionext Europe.

After graduating in 2001, he held various positions in the semiconductor industry as a developer and expert in technical marketing with a focus on video and display applications.

FACEBOOK

www.facebook.com/SocionextEurope

YOU TUBE

www.youtube.com/channel/UCV8NLtpAJI4_I_fQJq10HsA

Embedded Computing Design | Winter 2018

21

VOICE ACTIVATION

Getting a voice user experience right is harder than you think

By Jeff LeBlanc, ICS and Boston UX

Voice interaction is one of the most disruptive technologies of the 21st century. Every day, more devices are hitting the market with a voice user interface (VUI) component. While many of the technical challenges to voice-enabling a device have already been addressed, for designers, making the experience of using the device a pleasing one for the end user is still an open question. This article addresses some of the challenges and best practices around designing a VUI that is effective, natural, and engaging for the user, including designing for confidence thresholds, accommodating barge in, using N-best lists, and how to talk with (instead of at) the user in a real conversation.

“J.A.R.V.I.S., are you up?”

“For you sir, always.”

– “Iron Man” (2008) While voice user interfaces (VUI) have been on the periphery of the public mindset since 1968, when HAL and Dave Bowman had their disagreements, it wasn’t until Tony Stark started bantering with J.A.R.V.I.S. in 2008 that the notion of a helpful voice-controlled “smart home” started to come into focus.

The hugely successful Amazon Echo device, released in 2014, combined the latest in voice-recognition technology with powerful cloud-based computing to provide an in-home experience that nearly rivaled that depicted in the movies. Turning on the lights or your sound system was never so easy.

Since then, Google, Apple, and other

22

technology companies have jumped into the fray and are tripping over each other to provide the finest interactive voice experiences for your home, workplace, and car.

Voice-rec background This technology has been a long time coming: Bell Labs and IBM worked on speech systems as far back as the 1950s. But, it wasn’t until the late 1990s that Dragon’s NaturallySpeaking software gained enough traction to bring speech recognition to consumers’ collective consciousness. While it was revolutionary at the time, NaturallySpeaking required a fair amount of “training” by the end user to achieve the 90 percent accuracy level that makes speech recognition viable as a form of humancomputer interaction. So the technology was not nearly as natural as it could be.

In the years since then, developers, designers, and technologists have toiled away, trying to “solve voice.” Yet, we’ve only gained an additional 5 percent in recognition accuracy.

Why is designing more human-like voice interfaces so difficult?

When designing VUIs, there are two key aspects that must be addressed. The first is ensuring the interface can capably recognize sound as human speech. Known as Automated Speech Recognition (ASR), this is the core of speech-to-text software engines. ASR can be performed on modern consumer hardware with reasonable processing speed. But ASR is more typically done in the cloud. Devices like the Amazon Echo only do enough local processing to find their “wake word,” while the rest of the

Embedded Computing Design | Winter 2018

www.embedded-computing.com

Boston UX

www.bostonux.com

TWITTER

@BostonUX

LINKEDIN

www.linkedin.com/company/bostonux/

ICS

www.ics.com

TWITTER @icsonqt

LINKEDIN

www.linkedin.com/company/integratedcomputer-solutions-inc./

VOICE ACTIVATION

This interaction is built on the concept of confidence level: How sure is your smart home that it really understood you? If the smart home is pretty sure it understands your request – say greater than 75 percent accuracy – it can just execute it. If it’s only somewhat sure, the device can ask for clarification. By leveraging confidence level and engaging in dialogue, you can clarify your request without having to restart the whole command interaction from the wake word.

FIGURE 1

Speech-recognition systems must move beyond following commands to actually engaging the user in a dialog.

work is done by remote computing resources. So yes, Alexa is listening to everything you say. But she only cares when you say her name.

The second, and more difficult, aspect of the voice experience is ensuring that the device knows what to do with the speech once it’s recognized. Natural Language Understanding (NLU) – which combines a variety of disciplines including linguistics, cognitive science, and artificial intelligence – has challenged computer scientists for years.

Although some experts view ASR as the “hard part” of developing VUIs, I disagree.

We’ve been holding steady at about 95 percent accuracy for many years, comparable to human-to-human communication. Yup, even human-to-human communication isn’t 100 percent accurate. Think about how many times you say “Huh?” or “What?” when speaking with another person. Yet those conversations are easily understood.

Our challenge as UX designers is figuring out how to create an exceptional interactive voice experience, getting as close as possible to mimicking the person-to-person interface experience.

Say what?

This phenomenon is known as a Natural User Interface or NUI. Getting simple commands to work correctly is straightforward; it’s mostly a matter of pulling the correct keywords out of the utterance. For instance, getting your smart home to respond correctly to “turn on the dining room lights” isn’t too complex. It just involves creating an interface that can recognize the desired action (“turn on”) and what to perform that action on (“dining room lights”).

But there are still challenges: Since we have slightly less than 100 percent speechrecognition accuracy, the device might not understand your exact utterance. Perhaps the voice assistant heard you say “turn on the dine room lights.” While a human can easily make the leap from dine room to dining room, that’s not the case in the binary world of computers. “Dine” does not equal “dining,” so your voice assistant doesn’t understand what you’re asking. You end up frustrated, eating in the dark. Fortunately, we can design around this. The solution lies in moving beyond simply taking utterances and commands to actually engaging our user in a dialog. (Figure 1.)

In our example, the smart home understands your intent – you want to turn on the dining room lights – but it didn’t get quite enough information to carry out the task.

So we program the VUI to do something typical in human-to-human interaction: ask for clarification. Our smart home could respond “Sorry, I didn’t quite catch that. What do you want to turn on?” www.embedded-computing.com

On the N-best list This next design technique builds atop this conversational approach to try to predict what you might say based on expected responses from prior conversations. It’s not unreasonable for your smart home to hear “dine” instead of “dining.” Or even other similar-sounding words like “diving.”

By collecting these near misses in something called an N-best list, your smart home can capture likely possibilities.

Now your home’s VUI can either ask you for confirmation of a word on the list or simply go ahead and execute that command. Having your home respond with “I think you asked me to turn on the dining room lights. Is that right?” shows that your home is smart enough to (most likely) figure out what you said but is courteous enough to double check just in case it didn’t quite understand the request 100 percent.

Flowcharts and maps Flowcharts allow VUI designers to map out the possible branches found in even simple interactions. Continuing the conversation about the dining room lights, to ensure a smooth, natural dialog the VUI designer has to think about what your likely response would be. You might answer the request for clarification about turning on the lights with a simple “yes.”

In that case, the smart home should turn on the lights.

But if you listen to recordings of humanto-human conversations, they’re often not as precise. What if you responded with “yup” instead of “yes”? Or “that’s

Embedded Computing Design | Winter 2018

23

VOICE ACTIVATION

right” or “make it so” or any number of affirmations? What if you responded in the negative? “No. Nope. Uh-uh.” Would your smart house know what to do?

This scenario is precisely why checking lists instead of simple keyword matching is critical. It’s the best way to achieve the most natural interaction.

Barging in Another aspect of human-to-human communication that bears mentioning is that of interruption. Sometimes we’re impolite and don’t wait for the other person in the conversation to finish speaking before we start talking. Other times, interrupting is the only way to move a conversation forward in a timely manner. In both cases, the ability to interrupt makes a conversation more natural.

claims department.” You eagerly tap the “3” key and don’t bother to listen to the rest of the list.

This ability to barge in and interrupt the conversation is something VUI designers need to incorporate in order to create a human-like voice interaction. (If your waiter was reading off the list of salad dressings and you said “Stop, I want that one, the vinaigrette” and he kept on listing dressings, things would get a bit awkward.) The Amazon Echo does a great job of supporting barge-in, letting a user say “Alexa, cancel” at any time.

The takeaway Designing a compelling, human-sounding voice assistant is certainly possible.

Google’s new Duplex phone bot, for instance, comes complete with conversational tics common to most humans, including “ahs” and “ums” peppered throughout the dialog. Some people have even expressed concerns about just how human it sounds as the line between AI and human speech becomes increasingly blurred.

Still, this is the future. So how do we deliver? By paying attention to basics like those I’ve outlined, designers can create the natural, effortless voice-powered interactions today’s consumers expect.

Here’s an example. You got into a fender bender and called your insurance company to file a claim. Listening to a long list of options on the company’s automated phone system, you interrupt as soon as you hear “press 3 to reach the

Jeff LeBlanc is Director of User Experience for both Boston UX and ICS. He heads the creative team. With an engineering degree from Worcester Polytechnic Institute (where he’s also an adjunct professor), he’s an expert at bridging the gap between design and development. What makes his day? Applying human factors principles to UX design. And 3D-printing a wearable Iron Man suit.

OpenSystems Media works with industry leaders to develop and publish content that educates our readers.

Qt or HTML5? A Million Dollar Question

By Burkhard Stubert, Chief Engineer, EmbeddedUse With a five times smaller footprint, four to eight times lower RAM requirements, and a more efficient rendering flow than HTML, using Qt for user interfaces provides faster start-up times and maintains the cherished 60 fps and 100 ms response time, where HTML would struggle.

Learn how the author says he could save one of the world’s largest home appliance manufacturers millions of euros by choosing Qt over HTML. The secret?

Qt scales down to lower-end hardware a lot better, without sacrificing user experience. http://www.embedded-computing.com/white-paper-library/qt-or-html5

24

Embedded Computing Design | Winter 2018

Check out our white papers at www.embedded-computing.com/ white-paper-library www.embedded-computing.com

BY ENGINEERS, FOR ENGINEERS In the rapidly changing technology universe, embedded designers might be looking for an elusive component to eliminate noise, or they might want low-cost debugging tools to reduce the hours spent locating that last software bug. Embedded design is all about defining and controlling these details sufficiently to produce the desired result within budget and on schedule.

Embedded Computing Design (ECD) is the go-to, trusted property for information regarding embedded design and development.

embedded-computing.com

VOICE ACTIVATION

Voice-assistant battles

By Todd Mozer, Sensory

I have spoken on a lot of “voice”-oriented shows over the years, and it’s disappointing that there hasn’t been more discussion about the competition in the industry and the factors driving the huge investments we see today. Because companies like Amazon and Google participate in and sponsor these shows, there is a tendency to avoid the more controversial aspects of the industry. Ahead: I share some of my thoughts on what is driving the competition, why the voice-assistant space is so strategically important to companies, and some of the challenges resulting from the voice-assistant battles.

I

n September of 2017 it was widely reported that Amazon had over 5,000 employees working on Alexa with more than 1,000 more to be hired. To use a nice round and conservative number, let’s assume an average Alexa employee’s fully weighted cost to Amazon is $200K. With about 6,000 employees on the Alexa team today, that would mean a $1.2 billion investment. Of course, some of this is recouped by the Echo’s and Dot’s sales profits, but when you consider that Dots sell for $30-$50 and Echos at $80-$100, it’s hard to imagine a high enough profit to justify the investment through hardware sales. For example, if Amazon can sell 30 ­million Alexa devices and make an average of $30 per unit profit, that only covers 75 percent of the cost of the (conservative) $1.2 billion investment.

Other evidence supporting the huge investments being made in voice assistants

26

is the battle in advertising. Probably the most talked-about thing at 2018’s CES show was the enormous position Google took in advertising the Google Assistant. In fact, if you watch any of the most expensive advertising slots on TV (Super Bowl, NBA finals, World Cup, etc.) you will see a preponderance of advertisements with known actors and athletes saying “Hey Google,” “Alexa,” or, “Hey Siri.” (Being in the wakeword business, I particularly like the Kevin Durant “Yo Google” ad.)

And it’s not just the US giants that are investing big into assistants: Docomo, Baidu, Tencent, Alibaba, Naver, and other large international players are developing their own or working with third-party assistants.

What’s my motivation?

Exactly what is driving this huge investment companies are making? Actually, a multitude of factors, including the ability to own the cross-platform user experience and collect user data, entertainment and other service-package sales, and the ability to sell and recommend products to consumers.

Here’s the basic motivation that I see in creating voice assistants: Build a cross-­platform user experience that easily enables consumers to interact, control, and request things through their assistant. This situation will ease adoption and bring more power to consumers, who will then use the products more and in doing so create more data for the cloud providers. This “data” will include all sorts of preferences, requests, searches, and purchases; it will also allow the assistants to learn more and more about the users.

The more the assistant knows about any given user, the better the assistant can help

Embedded Computing Design | Winter 2018

www.embedded-computing.com

Sensory

www.sensory.com

TWITTER

@Trulyhandsfree

LINKEDIN

www.linkedin.com/company/sensory-inc-/

FACEBOOK

www.facebook.com/SensoryInc

YOU TUBE

www.youtube.com/user/sensoryFAE

VOICE ACTIVATION

For example, let’s take the simple case of finding food when I’m hungry: I might say “I’m hungry.” Then the assistant’s response would be much more helpful the more it knows about me. Does it know I’m a vegetarian? Does it know where I’m located, or whether I am walking or driving? Maybe it knows I’m home and what’s in my refrigerator, and can suggest a recipe … does it know my food/taste preferences? How about cost preferences? Does it have the history of what I have eaten recently, and knows how much variety I’d like? Maybe it should tell me something like “Your wife is at Whole Foods, would you like me to text her a request or call her for you?”

FIGURE 1

The more a voice assistant gets used, the better it should get at servicing the user’s needs and requests.

the user in providing services such as entertainment and assisting with purchases (e.g., offering special deals on things the consumer might want). Let’s look at each of these in a little more detail: 1. Owning the cross-platform user experience and collecting user data to make a better voice assistant For thousands of years, consumers interacted with products by touch. Squeezing, pressing, turning, and switching were all the standard means of controlling. The dawn of electronics really didn’t change this interface, and mechanical-touch systems became augmented with electrical-touch mechanisms. Devices got smarter and more capable, but accessing these capabilities got more confusing with more complicated interfaces and a more difficult user experience. As new sensory technologies began to be deployed (such as gesture, voice, pressure sensors, etc.) companies like Apple emerged as consumer-electronics leaders because of their ability to package consumer electronics in a more user-friendly manner.

With the arrival of Siri on the iPhone and Alexa in the home, voice-first user experiences are driving the ease of use and naturalness of interacting with consumer products. Today we find companies like Google and Amazon investing heavily in their hardware businesses and using their assistants as a means to improve and control the user experience.

Owning the user experience on a single device is not good enough: The goal of each of these voice assistants is to be your personal assistant across devices – on your phone, in your home, in your car, wherever you may go. This is why we see Alexa and Google and Siri all battling for, to take an example, a position in automotive. Your assistant wants to be the place you turn for consistent help. In doing so it can learn more about your behaviors, including where you go, what you buy, what you are interested in, who you talk to, and what your history is. This isn’t just scary Big Brother stuff – it’s quite practical. If you have multiple assistants for different things, they may each think of you and know you differently, thereby having a less complete picture. It’s really best for the consumer to have one assistant that knows you best. www.embedded-computing.com

It’s easy to see how these voice assistants could really be quite helpful the more they know about you. But with multiple assistants in different products and locations, their knowledge would be incomplete. In this example it might know that I’m home, but not know what’s in my fridge. Or it might know what’s in the fridge and know that I’m home but not know my wife is currently shopping at Whole Foods, etc.

The more I use my assistant across more devices in more situations and over more time, the more data it could gather and the better it should get at servicing my needs and assisting me. It’s easy to see that once it knows me well and is helping me with this knowledge it will get very sticky; it would be very difficult to get me to switch to a new assistant that doesn’t know me as well. (Figure 1.)

2. Entertainment and other service package sales.

Alexa came onto the scene in 2014 with one very special domain – music.

Amazon chose to do one thing really well, and that one thing was to make a speaker that could accept voice commands for playing songs, albums, bands, the radio. Not long after that, Alexa added new domains and moved into new platforms like Fire TV and the Fire stick controller. It’s no coincidence that an Amazon Music service and Amazon TV services both exist and that

Embedded Computing Design | Winter 2018

27

VOICE ACTIVATION

you can wrap even more services into an Amazon Prime membership. When assistants don’t support Spotify well, there are a lot of complaints. And it’s no surprise that it’s been reported that Spotify is developing its own assistant and speaker. Comcast even has its own voice-control remotes.

works, and they have Alexa listening everywhere, willing to take our orders. Because assistants use a voice interface, there will be a much more serial approach to making recommendations and selling me things. For example, if I do a text search on a device for nearby vegan restaurants, I see a map with a whole lot of choices and long list of options. Typically, these options could include sidebars of advertising or “sponsored” restaurants first in the listing, but I’m supplied a long list. If I do a voice search on a smart speaker with no display, it will be awkward to give me more than a few results, plus I’ll bet the results we hear will become the “sponsored” restaurants and products.

There’s a very close tie between the voice assistants and the services that they bring. Apple is restrictive in what Siri will allow you to listen for. Companies want to keep you within their ecosystem, where they make more money. (Maybe it’s this locked-in ecosystem that has given Apple license to adopt a more relaxed schedule in improving Siri?).

Amazon and Google are really not that different, although they may have different means of leading us to the services they want us to use, they still can influence our choices for media. Spotify has over 70 million subscribers (20 million paying), over $5 billion in revenues; it recently went public with about a $30 billion market cap. Meanwhile, Apple Music just overtook Spotify in terms of paying subscribers. Music streaming has turned the music industry into a growth business again. The market for video services is even bigger: Amazon is now one of the top content producers of video.

The assistant you use will have a lot of influence on the services you choose and how accessible they are. This is one reason why voice-assistant providers might be willing to lose money in getting the assistants out to the market, so they can make more money on services. The battle of voice assistants is ultimately a battle of who controls your media and your purchases.

It would be really obnoxious if Alexa or Siri or Cortana or Google Assistant suddenly suggested I buy something that I wasn’t interested in, but what if it knew what I needed?

For example, it could track vitamin usage and ask if I want more before they run out, or it could know how frequently I wear out my shoes, and would subsequently recommend a sale for my brand and my size when I needed them. The more my assistant knows me the better it can “advertise” and sell me in a way that’s not obnoxious but actually really helpful. And of course the assistant’s company makes extra money in the process.

3. Selling and recommending products to consumers The biggest business in the world is selling products. It’s helped make Amazon, Google, and Apple the giants they are today. Google makes its money on advertising, which is an indirect form of selling products. What if your assistant knew what you needed whenever you needed it? It would uproot the entire advertising industry. Amazon has the ability to pull this off. They have the world’s largest online store, they know our purchase histories, they have an awesome rating system that really

28

It’s not easy to be a retailer today, when more and more people are turning to Amazon for shopping. And why not shop online? Ordering is convenient, with features such as ratings. Delivery is fast and cheap, and returns are easy and free – if you are Prime member. In April 2018, Bezos reported there are more than 100 million Prime members in the world, and the majority of U.S. households are Prime members. Walmart and Google have partnered in an ecommerce play to compete with Amazon, but Walmart is just dancing with the devil. Google will use the partnership to gather data and invest more in their internal ecommerce and shopping experiences. Walmart isn’t giving up, though, and is aggressively pursuing ecommerce and artificial intelligence (AI) initiatives through acquisitions and its Store #8 that acts as an incubator for AI companies and internal initiatives. Question: Why does Facebook have a Building 8 and Walmart have a Store 8 for Skunk Works projects?

It’s not just the retailers that are under pressure, however. If you make consumer electronics it’s getting more challenging too. Google controls the Android ecosystem and is pumping a lot of money into centralizing and hiring around their hardwaredevelopment efforts. Google is competing against mobile phones from Samsung, Huawei, LG, Oppo, Vivo, and other users of their Android OS. Amazon is happy to sell other people’s hardware online (OK, not Google, but others), but they take a nice commission on those sales, and if it’s a hit product they find ways to make more money through Amazon’s in-house brands and warehousing, and potentially even making the product themselves.

The Alexa fund has financed companies that created Alexa-based hardware products that Amazon ended up competing against with in-house developments; when Amazon sells Alexa products it doesn’t need to make a big profit (as described earlier). And Apple … well, Apple has a history of extracting money from anyone that wants to play in its ecosystem. This is business: There’s a very good reason that Google, Amazon, Apple, and other giants are giants. They know how to make money on everything they do. They are tough to compete with. The “free” stuff consumers get (and we do get a lot!) isn’t really free. We are trading our data and personal information for it.

Who’s got it the worst?

So retailers have it tough (and assistants will make it even tougher), service providers have it tough (and assistants with service offerings make it even tougher), and consumer-electronic companies have it tough. But the toughest situation is for the speaker companies. The market for speakers is exploding, driven by the demand for “smart” speakers. Markets and Markets researchers report that the current smartspeaker market stands at over $2.6 billion and is growing at over 34 percent per year. Seems like that would be a sweet market to be in, but a lot of that growth is eating away at the traditional speaker market. So a speaker company gets faced with a few alternatives.

Embedded Computing Design | Winter 2018

www.embedded-computing.com

It could partner with voice assistants within the ecosystem of its biggest competitors (Google, Apple, Amazon, etc.).

This would give all the data collected to their competitors, though, and put them at the mercy of their competitors’ systems. Alternatively, they could develop and support an in-house solution, which could cost way too much to maintain, or they could use a third-party solution, which is likely to cost a lot more and underperform compared to the big guys that are pumping billions of dollars each year into enhancing their AI offerings.

Many are choosing the first option, only to find that their sales are poor because of better-quality, lower-priced offering from Google and Amazon. A company like Sonos, who is a leader in high quality Wi-Fi speakers, has chosen this first option, but with a twist, where they are trying to support Google and Amazon and Apple. The recent IPO filing from Sonos highlights the challenges well: ”Our current agreement with Amazon allows Amazon to disable the Alexa integration in our Sonos One and Sonos Beam products with limited notice. As such, it is possible that Amazon, which sells products that compete with ours, may on limited notice disable the integration, which would cause our Sonos One or Sonos Beam products to lose their voice-enabled functionality.

Amazon could also begin charging us for this integration which would harm our operating results.”

assistants using open-source speech recognizers like Kaldi. This might save the cost of deploying third-party solutions, but it requires substantial in-house efforts, and is ultimately fraught with the same challenges as the third-party approach, which in general are that it’s really hard to compete against companies approaching a trillion-dollar market capitalization when these companies see AI and voice assistants as strategically important and are investing that way.

Retailers, consumer OEMs, and service providers all have a big challenge. I run a small company called Sensory: We develop AI technologies; companies like Google, Amazon, Samsung, Microsoft, Apple, Alibaba, Tencent, Baidu, etc. are our customers and our biggest competitors. My strategy? Move fast, innovate, and move on. I can’t compete head-to-head with these companies, but when I come out with solutions that they need before they have it in- house, I get a one-to-three-year window to sell to them before they switch to an inhouse replacement. That’s not bad for a small company like Sensory.

Bigger companies like Sonos or Comcast could deploy the same general strategy to set up fast-moving innovation pieces that allow them to stay ahead of the game. This appears to be the exact strategy that Walmart is taking on with Store 8 so as to not be left behind! Without a doubt, it’s very tough competing in a world of giants that have no boundaries in their pursuits and no limit to their ambitions!

Todd Mozer is the CEO of Sensory. He holds over a dozen patents in speech technology and has been involved in previous startups that reached IPO or were acquired by public companies. Todd holds an MBA from Stanford University.

His technical experience lies in the areas of machine learning, semiconductors, speech recognition, computer vision, and embedded software.

TRACE 32 ®

Trace-based Code Coverage

They further highlighted that their lack of service integrations could be a challenge should Google, Amazon or others offer discounting (which is already happening): “Many of these partners may subsidize these prices and seek to monetize their customers through the sale of additional services rather than the speakers themselves,” the company said. “Our business model, by contrast, is dependent on the sale of our speakers. Should we be forced to lower the price of our products in order to compete on a price basis, our operating results could be harmed.”

Looking at Sonos’s financials, you can see their margins already starting to erode.

Some companies have attempted the second approach by bringing out in-house www.embedded-computing.com

eec_trace_tools.indd 1

Statement Coverage Decision Coverage MC/DC Coverage

NO INSTRUMENTATION

www.lauterbach.com/1659.html

Embedded Computing Design | Winter 2018

29

07.11.2018 12:20:46

VOICE ACTIVATION

Low-power solutions for always-on, always-aware voice command systems

By Paul Beckmann, Ph.D., DSP Concepts

The algorithms that allow always-on, always-listening voice command products to function are necessarily complex. They must be alert for the wakeword 24/7/365; recognize the wakeword reliably; isolate as best as possible the user’s voice from the surrounding noise; and produce a signal that is clean enough for the voice-recognition engine to use. There are many different algorithms at work, all of which must be tuned to suit the product’s design and application. The design and power-consumption demands of portable products may affect the function of these algorithms.

F

ollowing are the basic components of a voice-command algorithm package, presented in order from the microphone end to the final signal output: ›› Sound detector: Typically, the signal from a single microphone is monitored using a comparator.

When the signal level exceeds a certain threshold – such as when a user speaks the wakeword – the comparator sends a command to power up the rest of the system.

This function is critical to portable products because it allows more components to be shut down to save power. This function must also occur quickly so that the system is able to receive the wakeword. ›› Noise reduction and filtering: To improve sound detection, it helps to filter out sounds that are obviously not human voices and thus safe for the voice-recognition system to ignore. Through choice of microphones, physical design of the product, or audio processing, the product can filter out sounds beyond the human vocal range (which spans roughly 100 Hz to 6 kHz). Audio processing can also remove repetitive sounds, but these functions may require the processor to be powered on and may impact battery life.

30

›› Wakeword detection: Once the system detects sound and powers up, it must record the incoming audio and compare it to a stored digital file of the wakeword (such as “Alexa” for the Amazon Echo). If the waveform of the incoming audio is sufficiently close to the stored file, the device becomes receptive to voice commands. ›› Direction of arrival detection: In order for a microphone array to focus on a user’s voice, it must first determine where the user is relative to the product. The processor determines the direction of arrival by comparing the phase information of the signals from the microphones. It must also include precedence logic that rejects reflections of the user’s voice from nearby objects and adjust its operating threshold to compensate for ambient noise so that the environment does not create false directional cues. ›› Beamforming: The reason for a microphone array is that the signals from the multiple microphones can be processed so that the array becomes directional; sounds coming from the determined direction of arrival are accepted while sounds coming from different directions are rejected. In devices such as smart speakers, remote controls, and home automation wall panels, the desired direction of focus for the beamformer has to be determined, and the response of the array adjusted to focus in the direction of the user. ›› Acoustic echo cancelling (AEC): Acoustic echo canceling rejects sounds (such as music or announcements) coming from the device itself so that the array can pick up the user’s voice more clearly. Because the original signal and response of the device’s internal speaker are known, the signal that comes back through the microphone can be rejected. The echoes of this sound from nearby objects – which is time-delayed and altered in frequency content – must also be rejected. ›› Local command-set recognition: Because portable products may not be able to rely on an Internet connection as today’s smart speakers do, they will likely need to recognize a certain number of basic function commands on their own without the help of external servers. These commands are typically limited to basic functions such as play, pause, skip tracks, repeat, and answer call. Recognition of these commands works in the same way as wakeword detection does. However, even though the command set is limited, the need for local command set recognition increases the load on the processor compared with a home smart speaker that needs to recognize only its wakeword and offloads other voice-recognition tasks to an external server.

Embedded Computing Design | Winter 2018

www.embedded-computing.com

DSP Concepts

www.dspconcepts.com

TWITTER

@DSPConcepts

LINKEDIN

www.linkedin.com/company/ dsp-concepts-llc/about/

FACEBOOK

www.facebook.com/DSPConcepts

YOU TUBE

www.youtube.com/channel/ UCyRih8jKngmR2s6os8sio3w

VOICE ACTIVATION

Algorithm tuning The function of each of the mentioned algorithms is complex and must be adjusted to suit the application – especially in portable products where the environment and use patterns are likely to be different from those of home products. Here are the algorithm functions that must be tuned for optimum voice-recognition accuracy. ›› Detection/wake threshold: The threshold levels for sound detection and wakeword detection must be set high enough to minimize false triggering of the device, but low enough that the user can address the device at a normal speaking level. In portable products, especially, it may be desirable for these levels to be adjusted dynamically so that the performance adjusts to compensate for varying levels of ambient sound. The function of the dynamic compensation will itself have to be tuned. ›› Noise reduction/canceling: Depending on the application, different types of noise may be encountered and the device can be tuned to reject them.

For example, the spectrum of any car’s road and engine noise at different speeds is known by the manufacturer, so the voice-recognition system can be tuned to reject these sounds. The noise-reduction/canceling algorithms can also function dynamically to adapt to changing environments, but this function must also be tuned. ›› Beamformer beamwidth: The tighter the beamwidth of the beamformer, the better it will reject environmental sounds and reflections of the user’s voice from other objects. However, setting beamwidth too tight will cause the unit to reject the user’s voice if the user moves slightly. In products such as remote controls and home-automation panels, beamwidth will have to be set wider to accommodate movement of the user while the user is speaking. ›› Wake/sleep strategies: As suggested previously, one of the goals when minimizing power consumption is to put the device to sleep as often as possible and keep it asleep for as long as possible. However, if the device is put to sleep too quickly after use, it may miss commands that follow the wakeword and require the user to speak the wakeword again. If the device is kept awake longer than necessary, it will consume more power than it needs to.

DSP Concepts’ Voice UI algorithms are specifically designed to make it easy to tune all of the mentioned functions, and to create custom processing configurations to suit any portable or battery-powered voice-command product. Signal-processing chains can be configured through a simple graphical interface, using any combination

FIGURE 1 Screen image showing graphical configuration of an audio processing chain using DSP Concepts Voice UI.

of more than 400 available processing modules (Figure 1). These modules can be tuned through familiar on-screen knobs and buttons in the same intuitive way that a rackmount audio processor is typically adjusted.

Because of power-consumption demands and form-factor limitations, the capabilities of the audio processors used in most portable products are typically less than those of the processors used in home products. Thus, product-design teams must be cautious not to exceed the available processing power when designing signal chains for voice-command products, while also making the most of what processing is available. Because DSP Concepts Voice UI algorithms are already optimized for the processors they run on, they do not need to be rewritten to suit a particular processor, and debugging is not necessary. Signal chains can be tested in real time and also tested for different processors, which makes choosing the right processor for the application easier and faster.

The tunability and versatility of DSP Concepts Voice UI results in demonstrable performance advantages, which can be seen in videos on the DSP Concepts website.

Conclusion The challenges of creating always-on voice command products that can run for many hours to many months on battery power – while achieving functionality similar to that of today’s popular smart speakers – are considerable.

The proper choice of components, combined with careful tuning to suit the application, can result in portable voice command products that deliver a satisfying and reliable experience for consumers. It will be interesting to see what new functions, conveniences, and capabilities these technologies will bring to tomorrow’s portable and batterypowered tech products.

Paul Beckmann, Ph.D., is Founder and CTO of DSP Concepts.

www.embedded-computing.com

Embedded Computing Design | Winter 2018

31

Empowering the Connected World Boosting IoT Designs from Edge to Cloud ▪ Scalable module, IoT gateway, network appliance ▪ Energy-efficient embedded solution ▪ Industrial operating temperature ▪ Extended product longevity ▪ High reliability

Enable Your Design Today www.portwell.com info@portwell.com 1-877-278-8899

ISO 9001, ISO 13485, ISO 14001, TL 9000