Chapter 1 • Introduction

Bluetooth Low Energy (BLE) is a complex technology, but the basics are quite accessible. In this book, you’ll learn how to:

• discover BLE devices in the neighborhood by listening to their advertisements • create your own BLE devices advertising data • connect to BLE devices such as heart rate monitors and proximity reporters • create secure connections to BLE devices with encryption and authentication • understand BLE service and profile specifications and implement them • reverse engineer a BLE device with a proprietary implementation and control it with your own software • reverse engineer a BLE device with a proprietary implementation and control it with your own software • make your BLE devices use as little power as possible

This chapter introduces Bluetooth Low Energy and its advantages. It also lists the software and hardware platforms used in this book, as well as the reasons to choose them.

1.1 What is Bluetooth Low Energy?

Bluetooth is a wireless communication standard in the 2.4 GHz Industrial, Scientific, and Medical (ISM) frequency band. These days, if you hear about Bluetooth support in a product, it is almost always Bluetooth Low Energy (BLE). It’s a radical departure from the original Bluetooth standard, which is now called Classic Bluetooth.

Bluetooth Low Energy and Classic Bluetooth are actually different protocols. Classic Bluetooth is essentially a wireless version of the traditional serial connection. If you want to print a document, transfer a file or stream audio, you want this to happen as fast as possible. Therefore, the focus of development in Classic Bluetooth was on attaining faster and faster speeds with every new version.

However, Classic Bluetooth wasn’t a good fit for devices with low power consumption, for instance those powered by batteries. That’s why Nokia adapted the Bluetooth standard to enable it to work in low-power scenarios. In 2006, they released their resulting technology onto the market, dubbed Wibree.

The Bluetooth Special Interest Group (SIG), the organization that maintains the Bluetooth specifications, showed interest in this new development. After consulting with Nokia, they decided to adopt Wibree as part of Bluetooth 4.0, with the new name, Bluetooth Low Energy. Classic Bluetooth remained available for high-throughput applications.

Note: In practice, many chipsets support both Classic Bluetooth and Bluetooth Low Energy, especially in laptops and smartphones.

1.2 Layered architecture

The Bluetooth Core Specifi cation (https://www.bluetooth.com/specifi cations/specs/ core-specifi cation/) is more than 3200 pages long. And this is only the core specifi cation; there are many supplemental documents for BLE. However, BLE has a layered architecture. Many end-user applications only use the upper layers, so you don’t need to know the details of the architecture’s lower layer. This book doesn’t refer to the Bluetooth Core Specifi cation, but mainly to the supplemental specifi cations, which concern the upper layers.

Figure 1.1 The layered architecture of Bluetooth Low Energy

The BLE architecture consists of three main blocks:tController, Host, andnApplication.

Controller

This has the lower-level layers: the Physical Layer (PHY), Link Layer (LL) and Direct Test Mode (DTM). These are the layers where the Bluetooth radio does its work. The controller communicates with the outside world using the antenna, in a frequency band around 2.4 GHz. It communicates with the host using a standardized interface between the two blocks – the Host Controller Interface (HCI ). 1

Host

This is the block with which the end user or application developer comes in contact. The Logical Link Control and Adaptation Protocol (L2CAP) defi nes channels and signaling commands. On top of it, the Security Manager Protocol (SMP) handles secure connections (with authentication and encryption), and the Attribute Protocol (ATT) defi nes how to expose and access data as attributes. The Generic Attribute Profi le (GATT) 2 builds on the Attribute Protocol to defi ne how to discover services and their characteristics and how to read and write their values. The upper layer of the Host block is the Generic Access Profi le (GAP), which defi nes how devices can discover other devices and connect, pair, and bond to them. The host communicates with the controller using its part of the host controller interface, and applications communicate with the host depending on the APIs exposed by the operating system.

1 As the HCI is an interface between Host and Controller, it’s shown in both layers 2 Yes, GATT doesn’t look like an acronym for Generic Attribute Profi le, but the Bluetooth specifi cation uses this because GAP is already taken by Generic Access Profi le. You’ll get used to it.

Application

This layer builds on top of the Generic Attribute Profile to implement application-specific characteristics, services, and profiles. A characteristic defines a specific type of data, such as an Alert Level. A service defines a set of characteristics and their behaviors, such as the Link Loss Service. A profile is a specification that describes how two or more devices with one or more services communicate with each other. An example is the Proximity profile, which has two roles: Proximity Monitor and Proximity Reporter. The application layer is where the example programs in this book reside, and they make use of the layers in the Host block.

The three blocks don’t have to run on the same processor. In fact, there are three common configurations – one single-chip and two dual-chip:

Single-chip (SoC)

Controller, host, and application code run on the same chip. The host and controller communicate through function calls and queues in the chip’s RAM. Most simple devices such as BLE sensors use this configuration; it keeps the cost down. Some smartphones also use this configuration if they have a SoC with Bluetooth built in.

Dual-chip over HCI

A dual-chip solution with application and host on one chip, and the controller on another chip, communicates over HCI. Because HCI is a standardized interface, it lets you combine different platforms. For instance, on a Raspberry Pi, the Wi-Fi and BLE chip implements a BLE controller. If you connect a BLE dongle to an older Raspberry Pi, this dongle also implements a BLE controller. 3 BlueZ, the Raspberry Pi Linux kernel’s Bluetooth stack, implements a BLE host. So, BlueZ communicates with the BLE controller in the built-in BLE chip or on the BLE dongle (see Figure 1.2). In the former case, the HCI uses SDIO, and in the latter, UART over USB. 4 Many smartphones and tablets also use the dual-chip over HCI configuration, with a powerful processor running the host and a Bluetooth chip running the controller.

Dual-chip with connectivity device

Another dual-chip solution is one with the application running on one chip and the host and controller on another chip. The latter is then called the connectivity device because it adds BLE connectivity to the other device. This approach is useful if you have an existing hardware device that you want to extend with BLE connectivity. Because there’s no standardized interface in this case, the communication between the application processor and the connectivity device needs to make use of a proprietary protocol implemented by the connectivity device.

A three-chip solution with controller, host, and application each running on its own chip is also possible. However, because of the associated cost, this is typically only done for development systems.

3 In fact, you can easily create your own BLE dongle with an nRF52840 Dongle and the controller part of Zephyr’s BLE stack. Just build the sample samples/bluetooth/hci_usb from the Zephyr samples and flash it to your dongle. The next chapter explains how to do this 4 BlueZ even supports multiple controllers simultaneously.

Figure 1.2 A dual-chip implementation of the BLE architecture

1.3 How to communicate with BLE devices

Bluetooth Low Energy has two ways to communicate between devices: with and without a connection.

1.3.1 Without a connection

Without a connection means that the device just broadcasts information in an advertisement. Every BLE device in the neighborhood is able to receive this information.

Some examples of BLE devices broadcasting data are:

Proximity beacons

These devices, often following Apple’s iBeacon standard, broadcast their ID. Receivers calculate their approximate distance to the beacons based on the advertisement’s Received Signal Strength Indicator (RSSI).

Sensors

Many temperature and humidity sensors broadcast their sensor values. Most devices do this in an unencrypted fashion, but some of them encrypt the data to prevent it being read by every device in the neighborhood.

Mobile phones

After the COVID-19 pandemic started in 2020, Google and Apple collaborated on the Exposure Notifi cations standard for contact tracing. As part of this technology, Android phones and iPhones broadcast unique (but anonymous) numbers. Other phones can pick up these numbers and use them later to warn users that they have been in contact with someone who i s known to have had COVID-19.

1.3.2 With a connection

The other way to communicate between BLE devices is with a connection. One device (the client) scans for BLE advertisements to fi nd the device it wants to connect to. Then, optionally, it may do an active scan to ask the device (the server) which services are off ered.

After the client connects to the server, the client can use the server’s services. Each BLE service is a container of specifi c data from the server. You can read this data, or (with some services) write a value to the server.

Some examples of BLE devices using a connection are:

Fitness trackers

Your smartphone can connect to a fitness tracker and read your heart rate, the tracker’s battery level, and other measurements.

Sensors

Some environmental sensors let you read their sensor values over a BLE connection.

Proximity reporters

These devices sound an alert when their connection to another device is lost.

1.4 Advantages of BLE

1.4.1 Low power consumption

As its name implies, Bluetooth Low Energy is optimized for low-power applications. Its whole architecture is designed to reduce power consumption. For instance, setting up a connection, reading or writing data, and disconnecting happens in a couple of milliseconds. The radio is often the most energy-consuming part of a device. Therefore, the idea is to turn on the Bluetooth radio, create a connection, read or write data, disconnect, and turn off the radio again until the next time the device has to communicate.

This way, a well-designed BLE temperature sensor is able to work on a coin cell for ten years or more. You can use the same approach with other wireless technologies, such as Wi-Fi, but they require more power and take more time to set up a connection.

1.4.2 Ubiquitous

BLE radio chips are ubiquitous. You can find them in smartphones, tablets, and laptops. This means that all those devices can talk to your BLE sensors or lightbulbs. Most manufacturers create mobile apps to control their BLE devices.

You can also find BLE radios in many single-board computers, such as the Raspberry Pi, and in popular microcontroller platforms such as the ESP32. 5 This makes it quite easy for you to create your own gateways for BLE devices. And, platforms such as the Nordic Semiconductor nRF5 series of microcontrollers with BLE radio even make it possible to create your own battery-powered BLE devices.

1.4.3 Low cost

There’s no cost to access the official BLE specifications. Moreover, BLE chips are cheap, and the available development boards (based on an nRF5 or an ESP32) and Raspberry Pis are quite affordable. This means you can just start with BLE programming at minimal cost.

5 The first Raspberry Pi with built-in BLE was the Raspberry Pi Model 3B (released in 2016).

1.5 Disadvantages of BLE

1.5.1 Short range

BLE has a short range (for most devices, less than 10 meters) compared to other wireless networks, such as Zigbee, Z-Wave, and Thread. It’s not a coincidence that these competitors all have a mesh architecture, in which devices can forward their neighbors’ messages in order to improve range. Low-power wide area networks (LPWANs), such as LoRaWAN, Sigfox, and NB-IoT, have even longer ranges.

In 2017, the Bluetooth SIG added Bluetooth Mesh, a mesh protocol. This builds upon BLE’s physical and link layers with a whole new stack above them. However, Bluetooth Mesh isn’t as well-established as the core BLE protocol, at least not for home use.

1.5.2 Limited speed

The BLE radio has a limited transmission speed. For Bluetooth 4.2 and earlier, this is 1 Mbps, while for Bluetooth 5 and later, this can be up to 2 Mbps. This makes BLE unsuitable for high-bandwidth applications.

1.5.3 You need a gateway

Wi-Fi devices have their own IP addresses, so you can communicate with them directly from other IP-based devices, and they are integrated into your LAN (local area network). Bluetooth doesn’t have this: to integrate your BLE devices with other network devices, you need a gateway. This device has to translate Bluetooth packets to IP-based protocols such as MQTT (Message Queuing Telemetry Transport). That’s why many BLE device manufacturers have smartphone apps that function as device gateways. 6

1.6 Platforms used in this book

This book focuses on Bluetooth Low Energy programming on three platforms:

Programming language Library Software platform Hardware platform

Python Bleak Windows, Linux, macOS Raspberry Pi or PC

C++ NimBLE-Arduino Arduino framework ESP32

C Zephyr7

nRF52 Table 1.1 BLE platforms used in this book

These choices were made in order to demonstrate a wide range of applications compatible with many software and hardware platforms.

6 You can run IPv6 over BLE with 6LoWPAN, but this isn’t yet well-established. A more developed solution in the same 2.4 GHz frequency band is Thread, which uses 6LoWPAN over the IEEE 802.15.4 wireless protocol. 7 There’s no library for Zephyr in this table because the Zephyr operating system includes all functionality we need for BLE development.

1.6.1 Python/Bleak (Raspberry Pi, PC)

Python is an easy-to-use programming language that works on all major operating systems. There are a lot of Python Bluetooth Low Energy libraries, but many of them support only a single operating system. Bleak (https://bleak.readthedocs.io), which stands for Bluetooth Low Energy platform Agnostic Klient, is a welcome exception. It supports:

• Windows 10, version 16299 (Fall Creators Update) or higher • Linux distributions with BlueZ 5.43 or higher (also on a Raspberry Pi) • OS X 10.11 (El Capitan) or macOS 10.12+

Figure 1.3 With Python and Bleak, a Raspberry Pi is the perfect small computer to communicate with your BLE devices.

Bleak is a GATT client: it’s able to connect to BLE devices that act as GATT servers. It supports reading, writing, and getting notifications from GATT servers, and it’s also able to discover BLE devices and read advertising data broadcast by them.

While Bleak doesn’t implement a GATT server, in practice this isn’t a big limitation. GATT servers are typically implemented on constrained devices, so, for this purpose, the ESP32 and nRF52 hardware platforms are a better match. 8

1.6.2 C++/NimBLE-Arduino (ESP32)

If you’re looking at microcontrollers, the Arduino framework has become quite popular, not only on the original Arduino boards, which don’t have BLE functionality, but also on ESP32 development boards, which do.

8 If you really want to implement a GATT server on a Raspberry Pi or your PC, have a look at Bless (https://github.com/kevincar/bless). This is an acronym for Bluetooth Low Energy Server Supplement, and the library follows a similar API style to Bleak.

Figure 1.4 Espressif’s ESP32-PICO-KIT v4 is one of the many ESP32 development boards that you can use with NimBLE-Arduino for BLE development.

Programming for the Arduino framework is done in a variant of C++, but the framework and many Arduino libraries hide much of C++’s complexity. Even if you only know some C (which is much less complex than C++), you’ll be able to use the Arduino framework.

One of the more popular BLE libraries for Arduino on the ESP32 is NimBLE-Arduino (https:// github.com/h2zero/NimBLE-Arduino). It’s a fork of NimBLE (https://mynewt.apache.org/ latest/network/), which is part of the Apache Mynewt real-time operating system. With NimBLE-Arduino, you can easily create your own GATT server or client.

Note: If you prefer to use Espressif’s ESP-IDF instead of the Arduino framework, you can use esp-nimble-cpp (https://github.com/h2zero/esp-nimble-cpp). In fact, NimBLE-Arduino and esp-nimble-cpp use the same API. This means that many NimBLE-Arduino examples in this book can be reused with ESP-IDF with only a few minor changes.

1.6.3 C/Zephyr (nRF52)

For even more constrained devices, typically battery-powered, you need a specialized realtime operating system (RTOS). This book uses the Zephyr Project (https://zephyrproject. org) on nRF52840-based devices from Nordic Semiconductor. Zephyr has a completely open-source Bluetooth Low Energy stack.

Figure 1.5 Nordic Semiconductor’s nRF52840 Dongle is an affordable development platform for BLE applications with Zephyr.

Zephyr’s BLE stack is highly configurable. You can build Zephyr firmware for three configuration types:

Combined build

Builds the BLE controller, BLE host, and your application for a one-chip configuration.

Host build

Builds the BLE host and your application, along with an HCI driver to let your device communicate with an external BLE controller on another chip. 9

Controller build

Builds the BLE controller with an HCI driver to let your device communicate with an external BLE host on another chip.

With some basic knowledge of C, you can create your own BLE devices with Zephyr, such as BLE beacons, sensor boards, and proximity reporters. Zephyr has extensive documentation of its Bluetooth API, as well as a lot of ready-to-use examples that you can build upon. For the examples in this book, you’ll use the combined build type.

1.7 How to use this book

This is a book about programming. I expect you to have some programming experience. Knowledge of C, C++, and Python would be perfect. However, if you only know one or two of these programming languages, that won’t be a problem. The necessary C++ knowledge is minimal, so a basic knowledge of Python and C will suffice.

You need some (affordable) hardware for the examples in this book:

• The Python examples are hardware-agnostic due to the use of Bleak. So, whether you want to use a Raspberry Pi or another Linux computer, a Windows computer, or a Mac, it won’t make much difference.

• For the NimBLE-Arduino examples, any ESP32 development board will suffice. 10

• For the Zephyr examples, I recommend a Nordic Semiconductor nRF52840 Dongle, but you can use any BLE microcontroller supported by Zephyr. Some other manufacturers have variants of the dongle, such as the April USB Dongle 52840 (with an external antenna) or the nRF52840 MDK USB Dongle from makerdiary (with a case). Consult the appendix at the end of this book if you want to use one of these, because they come with a different bootloader.

9 For instance, on Nordic Semiconductor devices, the BLE controller can be a SoftDevice controller. 10 Note that the ESP8266, the ESP32’s predecessor, isn’t suitable for this book; it doesn’t have BLE.

Figure 1.6 From left to right: April Brother's April USB Dongle 52840, Nordic Semiconductor's nRF52840 Dongle, and makerdiary's nRF52840 MDK USB Dongle..

If you’re really serious about BLE development with Zephyr, I also recommend Nordic Semiconductor’s nRF52840 Development Kit. This is the bigger (and slightly less affordable) brother of the nRF52840 Dongle, offering more capabilities to debug your code easily. If you have one of these, you can do your initial development on it, and move your code to the nRF52840 Dongle when you’ve solved the most complex issues.

Figure 1.7 Nordic Semiconductor’s nRF52840 Development Kit has some powerful debugging capabilities.

So, at the minimum, you need your computer, one ESP32 development board and one BLE microcontroller supported by Zephyr.

Note: I recommend buying a couple of Nordic Semiconductor nRF52840 Dongles. As you’ll see in the next chapter, with the right firmware, you can turn these into adapters to explore other BLE devices and connect to them. You can also turn them into BLE sniffers that pick up BLE packets in the air and sniff traffic between other devices. They’re quite affordable too, with a price of around €10.

At the end of this book, you should be able to create your own BLE software based on the examples here.

Here’s a short overview of what I’ll cover in this book:

Chapter 1: Introduction

An introduction to Bluetooth Low Energy, its layered architecture, and its advantages and disadvantages

Chapter 2: Preparing your development environment

A preparation of your development environment for Python/Bleak, Arduino/NimBLE-Arduino, and Zephyr, as well as some useful tools for exploring BLE devices, and a BLE packet sniffer in Wireshark

Chapter 3: Broadcasting data with advertisements

A deep dive into advertising packets, which are fundamental for device discovery, but also a powerful way to broadcast data without the need to connect to a BLE device

Chapter 4: Connections and services

The use of BLE connections to read and write characteristics and receive notifications and indications from peripherals

Chapter 5: Securing BLE connections

Security measures to encrypt and authenticate BLE connections and use resolvable private addresses to thwart user tracking

Chapter 6: Profiles and roles

A step-by-step guide to implementing a BLE profile, as well as how to understand profile and service specifications

Chapter 7: Reverse engineering BLE devices

Reverse engineering a BLE LED badge by investigating its services and characteristics, decompiling the accompanying mobile app, and sniffing BLE traffic between the device and its mobile app

Chapter 8: Lowering power consumption

An investigation into and optimization of a BLE implementation’s power consumption

Chapter 9: Conclusion

A wrap-up of this book, with some resources for learning more about Bluetooth Low Energy and some ideas for further exploration

Appendix

Some specialized tips that could come in handy in various situations

Note: All of the code examples from this book are published at https://github.com/ koenvervloesem/bluetooth-low-energy-applications. Read the instructions in this GitHub repository for more information about how to download them. The repository also lists errors that may have been found in this book since its publication, as well as information about changes that impact the examples herein.

1.8 Summary and further exploration

In this introductory chapter, I explained the basics of Bluetooth Low Energy and its architecture. I also explained the two different ways to communicate with BLE devices: with and without a connection. After this, you learned about some of the advantages and disadvantages of BLE.

At the end of this chapter, I introduced the platforms used in this book, both hardware and software. Of course, these platforms aren’t the only ones you can use for BLE programming. For instance, I don’t cover BLE programming on mobile devices (Android or iOS). However, with the BLE knowledge learned from this book, you’ll be able to understand the BLE APIs offered by other platforms.

In the next chapter, you’ll learn how to set these platforms up, so that your development environment is ready for the rest of the book.