Non-Intrusive Load Monitoring based on Particle Filtering on a Udoo Board Peter Goldsborough
July 2014
Abstract Non-intrusive Load Monitoring (NILM) is the process of disaggregating a household's power demand from a single power source. During my internship at the Institute for Networked and Embedded systems at the University of Klagenfurt I implemented a system of appliance monitoring on a Udoo single-board computer, using algorithms based on particle fltering developed by Dominik Egarter. In this report I describe the various steps I undertook to complete my project.
The Udoo Board
Introduction The Udoo Board is a single-board computer that was initially funded via a crowdfunding campaign on Kickstarter, an online crowdfunding portal, where it received more than $500.000. It aims to combine the powerful networking and computing capabilities of other popular single-board computers such as the Raspberry Pi with the I/O functionality of the Arduino Due prototyping platform. It can operate either with a Linux distribution such as Ubuntu or Debian, or with an Android operating system commonly used on mobile systems. The micro-processor powering the Linux/Android part of the Udoo board is the Freescale i.MX6, whereas the Atmel ARM Sam6 processor runs on the Arduino part (the same as on an Arduino Due).
Confguration for the Udoo Board Below you can fnd a list of packages and confgurations needed to complete my project with the Udoo Board. Some are optional, some required: Package / Confguration
Importance
Source
Udoobunto operating system
Required
http://www.udoo.org/downloads/
pip – Python package-manager
Required
http://pip.readthedocs.org/en/latest/inst alling.html
pySerial
Required
pip
numpy
Required
pip
pysqlite
Required
pip
virtualenv
Optional
pip
sqlite3
Required
apt-get
sqlalchemy
Optional
pip
Power sampling with the YOMO shield
The YOMO smart meter is an Arduino-compatible shield built by Christoph Klemenjak, capable of measuring voltage, current and various power levels (active, apparent, reactive), which I used throughout my internship to gather energy data. It was built to ft and be used with an Arduino Uno. However, the Udoo's Atmel ARM SAM3 (the micro-processor used by the Arduino Due) is fortunately broken out in a way that makes Arduino (Uno) shields ft well onto the Udoo board. It would therefore be possible to use the YOMO shield with the same source code and confguration on the Udoo, as on an Arduino Uno. Or so we thought. As it turns out, the Arduino Due's microprocessor is quite different from those of all other Arduino boards and various problems caused by these differences needed to be solved. Notable differences between the Arduino Uno and the Udoo/Arduino Due: Arduino Udoo
Arduino Due / Udoo
Voltage
5V
3.3V
Clock speed
16 Mhz
84 Mhz
SPI pins
Digital pins 10:13
External
The voltage problem The Arduino Uno, Mega, Duemilanove, Nano and all other Arduino variants have an operating voltage of 5 Volts, except for the Arduino Due, which operates at 3.3 Volts. What this means is that any shield that operates at 5V and also outputs signals back (at 5V) is very likely to damage and destroy the Due's processor. The YOMO shield communicates with the Arduino via the Serial Peripheral Interface (SPI) protocol, which means that it also sends 5V signals back. Therefore, the shield could not be plugged into the Udoo board directly. Luckily, I had bought a Bi-directional Logic Level Converter from Sparkfun several months earlier, which just happened to be the one component in the world that we needed in this situation. What is a logic level converter? Essentially, it is a small device that converts signals from one higher reference voltage to a lower reference voltage. Actually, the term “conversion” is misleading here, as there is no actual conversion going on, everything is done digitally. When a HIGH signal comes in on one of the pins on the side with the high reference voltage, say 5V, the Logic Level Converter opens the “switch” on the low voltage side (e.g. 3.3V), which sets the opposite side also HIGH by connecting it to the 3.3V voltage source. When this signal goes LOW again, also the opposite side goes LOW. This can go in either direction (bi-directional). Using this Bi-directional Logic Level Converter, I could use the SPI communication between the YOMO shield, operating at 5V, and the Udoo's SAM3 processor, operating at 3.3V, without any worries.
References: https://www.sparkfun.com/products/12009
The clock speed problem The Arduino Uno and its Atmel Atmega328P processor run at 16 Mhz clock speed (by default), whereas the Arduino Due and its SAM3 processor run at 84 Mhz, 5.25 times more. This is not really a big problem in itself. What matters, however, is the SPI clock speed. From page 5 of the ADE7753's datasheet (the DSP chip), I concluded that the clock speed for the SPI connection must be between 1 and 4 Mhz. This made sense to me, because Christoph Klemenjak's original code divides the clock frequency for the SPI connection by 6, giving 16Mhz/6 = 2.6667 Mhz. To make the SPI connection work for the 84Mhz SAM3 processor, I divide the clock by 32, giving 2.625 Mhz. What helps in this situation is that the Arduino Due has the capability of dividing the the clock speed by any value between 1 and 255, which gives us quite a lot of freedom to choose any value between 1 and 4 Mhz for the SPI clock speed.
I only found out about the difference in clock speed between the Arduino Uno and Due after stumbling upon the Due's datasheet. I wish I had known about it earlier, as I had spent a lot of time looking for a certain SPI_CLOCK_DIV32 defnition somewhere in the Arduino SPI library. As it turns out, this defnition is only available in Arduino Uno environments, meaning my search had been relatively useless. References: http://www.analog.com/static/imported-fles/data_sheets/ADE7753.pdf http://www.atmel.com/Images/doc11057.pdf http://www.atmel.com/Images/doc8161.pdf
The SPI pins problem For the Arduino Uno, the SPI pins are shared with the regular I/O pins 10 through 13. The Arduino Due, however, has external male pins that provide for the SPI connection pins (MOSI,MISO,SLCK,SS), which are not easy to fnd on the Udoo board. They are located behind the USB ports. The Arduino Due processor optionally has the capability of having multiple slaves connected to it, however, this was not necessary for my project. References: http://www.gammon.com.au/images/Arudiuno_SPI_pins.png http://udoo.org/download/fles/pinout/Udoo_pinout_diagram.pdf
Transfer of samples from the SAM3 processor to the i.MX6 processor
Once the samples are fetched from the YOMO smart meter shield attached to the Atmel SAM3 processor, they have to be transferred to the i.MX6, running Ubuntu, for processing, storage and further use. Where the Udoo board shines particularly in this case is its ability to make the two on-board processors communicate with each other. Via a simple serial link the data can be written from the SAM3 processor to any serial port of the i.MX6 where it can be retrieved and used in a number of ways. Given that all Unix-based operating systems represent any data as a collection of fles, also a serial port is a fle that can be read from or written to. Therefore, one of the ways to read the data involves simply opening the respective serial port via a bash script and then reading out the values manually. However, this proved too tedious, so I resorted to a very neat python library, pySerial, with which the data extraction could happen within a language framework that opens a lot more possibilities than the bash scripting language.
Once I had decided that I would use Python to connect to the SAM3, I had to think of how this data transmission could be implemented. The simplest method that occurred to me was specifying predefned intervals at which data from the SAM3 would be transmitted and then received by the Python library. However, timing these processes could prove diffcult and if, for any reason, the Python script would not be executed when it should be, samples could go lost. I then thought of a more reliable and practical requestand-response model, where a data transmission is initiated by the Python script, after which the data is sent from the SAM3 to the i.MX6 processor. The simple protocol I implemented also uses the concept of a handshake, meaning that only after the readiness of either processors is validated by the other processor, a data transmission starts. This is done via 2 meta keys, that either signify an OK/GO signal or a STOP signal. They are different for either processor: OK / GO
STOP
i.MX6
$
@
SAM3
&
#
A typical data transmission follows this basic schema: (1) i.MX6 signalised that it wants to fetch sampling data. It sends out its GO symbol: $ (2) SAM3 receives the i.MX6's GO signal and signalises that the connection is established and that it will now send its data. It responds with its OK symbol: & (3) Data is transmitted from the SAM3 to the i.MX6. (4) The transmission can end in two ways: (4.1.1) The SAM3 has fnished sending all the data is has acquired so far and signalises that it has nothing more to send. It sends out its STOP signal: # (4.1.2) The i.MX6 signalises that the reception of data was successful and that the connection can end. It sends its STOP signal: @ (4.2) The i.MX6 has received enough data and signalises that it does not want any more data to be transmitted. It sends out its STOP signal: @ (5) End of transmission After the connection has ended, the Python script stores the data into an sqlite3 database and into csv format for further processing. References: http://imusic1.ucsd.edu/~dmedine/udoo_gpio_comport/S2.xhtml http://pyserial.sourceforge.net
Data processing Once the data had been successfully fetched and transmitted, we needed to apply Non-intrusive Load Monitoring algorithms to disaggregate the total power load values received. The algorithm chosen for this project had been developed by my supervisor, Dominik Egarter, and involved load disaggregation with state estimation through particle fltering. I was given an existing code base and computer algorithms implementing such particle fltering in the Python programming language, which I studied, slightly refactored and fnally extended so that they could be integrated into the existing data pipeline. This meant that data from the YOMO smart meter board would be sent via the aforementioned serial transmission protocol to the i.MX6, where it is processed by particle fltering algorithms and then made available for further use. After particle fltering, the state estimates for the individual devices are stored in plain-text and CSV format, as well as sent to an SQLite database. References: PALDi
Conclusion To conclude, I must note that during our fnal tests that would, for the frst time, send real voltage data from the YOMO board to the i.MX6 processor instead of simulations, the YOMO board failed and we could not recover it. Nevertheless, I can say that I am very happy with the system I built and the fact that the voltage data ultimately had to be simulated instead of fetched from a smart meter was unfortunate, but took away little from the quality of the rest of the project. I learnt a lot during this internship at the Institute for Networked and Embedded systems and am equally grateful that I was given this wonderful opportunity to spend time with such fascinating subjects and areas of technology. I want to thank my supervisor Dominik Egarter for his dedication, his helpfulness, his willingness to share his expertise and also for continuously motivating me.