Visualising sensors and coffee machines with ESP8266, MQTT, InfluxDB and Grafana

A few months ago, here at the synyx office we started out with a simple idea: hook up a couple of sensors to an ESP8266 module (or twenty) and have it write its data somewhere for visualisation purposes. Then we got creative.
Our current setup consists out of a number of ESP8266 (currently NodeMCU) boards, featuring humidity (DHT-22, BME280) as well as CO2 (MH-Z14) sensors which publish their data over MQTT. In addition, we also hooked the Jura coffee machines in the office up to a couple of ESP8266 modules via their serial interface to read out statistics.
All of this is managed centrally from a command & control server (C&C) which communicates over the MQTT broker (Mosquitto), with the nodes actively announcing themselves when they come online and receiving their configuration from the C&C. ESP8266 firmware updates are provided as Over The Air (OTA) updates via HTTP.
A custom MQTT-to-Influx service (based on libmosquitto and POCO) is used to write published sensor and coffee machine MQTT events into the InfluxDB instance using its HTTP line protocol. A Grafana instance then uses these time series to show current temperatures, humidity and CO2 levels, as well as coffee use on a single dashboard.
The resulting dashboard looks like this:

https://snapshot.raintank.io/dashboard/snapshot/Ept58LQH5U8sRSW7LP9hl17Cajy0T7i4
It shows the data from a total of four ESP8266-based nodes:

• One with a single DHT-22 temperature/humidity sensor.
• One with a DHT-22 and MH-Z14 CO2 sensor.
• Two connected to a single coffee machine each.

Although the coffee machines in question can produce more than just coffee and espresso, we deliberately limited ourselves to these two counters so that we would not have to set up a separate dashboard (or two) for just the coffee machines to display the counter for the dozens of products they offer 🙂
Infrastructure
Visualised the infrastructure we created looks like this:

Central to everything is the Mosquitto MQTT broker. It facilitates communication between the ESP8266 nodes, C&C and InfluxDB.
The ESP8266 nodes
For the firmware of the ESP8266 (ESP-12E-based NodeMCU) boards we use C++ and the Sming [1] framework. The latter is compatible with Arduino libraries, but allows one to use a callback-based system instead of the loop-based system of Arduino, in addition to allowing one to use the full C++ language instead of being limited to the Arduino C dialect.
The firmware is identical across all nodes, using a modular (class-based) system to allow each module (DHT, CO2, Jura, Jura Terminal) to be enabled, disabled and configured from the C&C using MQTT messages. The current firmware image is just a tad over 264 kB in size, easily fitting within the 1 MB slot allocated by the rboot boot manager.
As we use ESP-12E ESP8266 modules, we have 4 MB of Flash, split up into two slots for firmware (one active, one as backup or OTA update target) and 1 MB of Flash for storage (using the Spiffs filesystem) for each slot. At this point we do however not store any data locally on the nodes.
With the use of the rboot boot manager, we also gain easy access to HTTP-based OTA updates. An MQTT message from C&C triggers the process, telling Sming’s rboot HTTP update class to fetch the firmware image from the HTTP server, write it to the other firmware image slot and boot from it.
Sensors
We currently use the DHT-22 sensors for humidity and temperature using the Sming-provided DHT library, but will likely switch to the much more accurate BME280 (Bosch-manufactured) sensors. These also provide air pressure and take up less space than a single DHT-22 sensor.
The MH-Z14 CO2 sensor has an analogue output, a PWM output and a UART (bi-directional). Of these we use the UART (connected to UART0 on the ESP-12E) interface primarily for its ease of use. After querying the sensor, we get a response from which we can easily calculate the current CO2 level (in parts per million).
Coffee machine
At the office we have multiple Jura coffee machines. Being higher-end machines they come with a DE-9f serial connector on the back which provides a TTL (5V) level serial interface. Using a logic level shifter (bi-directional), we shift the voltage to the 3.3V UART of the ESP-12E.
We put a NodeMCU and logic level shifter into a small enclosure with DE-9f connector, and connected it via a regular serial cable (1:1) to the coffee machine. Using available sources [2] we knew the pin-out of the DE-9f connector on the machine, as well as the protocol it speaks. The resulting hardware is small enough to be tucked away behind the machine.
A small complication we found is that none of the coffee machines we have here (Xs9, Xs90, XJ9, X3) seem to enable their UARTs when in standby mode (all pins are 0V), which makes it impossible for us to tell the machine to wake up out of standby. This is however not a problem for further functionality.
The +5V from the coffee machine is used to power the ESP-12E when it comes out of standby (power button pressed), after which the node requests its configuration from the C&C server and starts querying the coffee machine for the contents of its EEPROM storage, to read out counters for coffee use.
For this we use the ‘RT:0000’ command, which instructs the system to return the first row of values in the EEPROM (offset 0). This command is encoded using the Jura protocol ‘standard’ before it’s sent. This involves post-fixing CR and LF and taking each bit of the bytes we wish to send, putting them at position 2 or 5 of an 0xFF masked byte, essentially padding out the payload bytes.
The response has to be decoded in the reverse fashion: taking bits 2 and 5 out of each byte we receive and assembling one byte out of four received. This response also ends with a CR and LF.
Decoded, the response looks like this:
rt:008D0001054E0000000F00390000009300290000162500000000000000000000
We get a lower-case confirmation of the command we sent, followed by the value. In this string each two bytes (four hex numbers) form a counter for a product, meaning in theory after 0xFFFF (65,535 decimal) cups of a product we would see an overflow. If we look at the earlier Grafana dashboard, we can however see that the Xs9 is still at fewer than 9,000 cups of coffee (the most popular product), despite having been in use for years. A 16-bit counter is therefore likely sufficient.
So far we confirmed these counters across our coffee machines:

1. espresso
2. 2x espresso
3. coffee
4. 2x coffee

These counters are all in sequential order in the EEPROM. Other counters follow these four, but since not each machine has the same products, the offset of their counter might be slightly different or be absent.
After reading out the relevant counters using appropriate offsets, they are published via MQTT on their own topics, to be written by the MQTT-to-Influx service into the InfluxDB.
LEDs
We’re looking at connecting various types of LED lighting to centrally control them (colour, intensity, etc.), including WS2812b RGB LEDs and kin.
Grafana
Since we use Grafana elsewhere in the company already for data visualisation, we used it for this project as well. When trying to configure things in a slightly more complex fashion (summing series, tables with just labels and associated current value, etc.), we did however quickly run into limitations.
Next steps
The coming time we will mostly take the existing system further: adding more sensors (light level, noise), as well as LEDs. We wish to add the ability to carry one’s favourite coffees between machines using NFC tags or similar. Maybe even add the coffee-carrying robots many of us seem to keep mentioning.
For the sensor nodes we are working on creating (acrylic) enclosures which should be both practical and attractive enough to be put around the office.
The C&C server and associated UI can be made more functional and extensive, with further configuration options added to the ESP8266 firmware’s modules.
The use of Grafana is at this point slightly controversial due to the limitations we found. We may look around for a more scriptable alternative if these limitations prove to be insurmountable.
Beyond all of this the main goal of the project remains to improve comfort and fun levels for everyone 🙂
 
References
[1] https://github.com/SmingHub/Sming
[2] http://protocoljura.wiki-site.com/index.php/Hauptseite