Visualize sensor data

Goal

In this post I am going to show you typical Corlysis usage - real-time visualization sensor data. Lets imagine that you would like to see e.g. temperature in your house from anywhere through Internet. Without Corlysis app you have 2 options. Buy public IP address and configure your own web server or buy virtual machine and install/configure Grafana and InfluxDB. Both options are expensive and proper secure configuration is not easy. With Corlysis you can concentrate to your task and we will care about your time-series data storage and visualization. So lets see how easy it is.

Connecting sensor to Raspberry Pi

Selecting proper sensor depends on your task/goal. In this example I am going to use very common BME280 sensor [1] [2]

Sensor is connected to Raspberry Pi (I selected it because it has network/internet connection, but you can use also another device).

There are plenty posts that describing how to connect BME280 sensor to Raspberry Pi - I recommend to you this post

Reading data from sensor

I did not use Python script from previous post but I used BME280 Sensor Driver.

Tool i2cdetect was used to get sensor address:

sudo apt-get install i2c-tools python-pip
$ i2cdetect -y 1
     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:          -- -- -- -- -- -- -- -- -- -- -- -- --
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
70: -- -- -- -- -- -- 76 --

After HW connecting and I2C configuration RPi.bme280 package was installed:

sudo pip3 install RPi.bme280

Now we are ready for reading data:

#!python

import smbus2
import bme280

port = 1
address = 0x76
bus = smbus2.SMBus(port)

bme280.load_calibration_params(bus, address)
data = bme280.sample(bus, address)

print(data)

Output:

compensated_reading(id=251d4633-bbb1-46d9-b252-5fa50a2d1dc1, timestamp=2017-05-31 20:28:41.007829, temp=24.499 °C, pressure=994.76 hPa, humidity=54.57 % rH)

Sending data to Corlysis

There are several ways how to send your data to Corlysis - e.g. you can use influxdb-python client. In this post I am going to show you how to do it with the HTTP Python library Requests. Data from sensor are reading every 5s and every 10s (2 samples) data are sent to InfluxDB. Local Raspberry Pi time is used as timestamp and it is important to specify “precision” as “ms” (default is nanosecond).

import time
import smbus2
import bme280
import requests
import argparse

# constants
SENSOR_PORT = 1
SENSOR_ADDRESS = 0x76

URL = 'https://corlysis.com:8086/write'
READING_DATA_PERIOD_MS = 5000.0
SENDING_PERIOD = 2
MAX_LINES_HISTORY = 1000


def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("db", help="database name")
    parser.add_argument("token", help="secret token")
    args = parser.parse_args()

    corlysis_params = {"db": args.db, "u": "token", "p": args.token, "precision": "ms"}

    # initialization
    bus = smbus2.SMBus(SENSOR_PORT)
    bme280.load_calibration_params(bus, SENSOR_ADDRESS)

    payload = ""
    counter = 1
    problem_counter = 0

    # infinite loop
    while True:
        unix_time_ms = int(time.time()*1000)

        # read sensor data and convert it to line protocol
        data = bme280.sample(bus, SENSOR_ADDRESS)
        line = "sensors_data temperature={},pressure={},humidity={} {}\n".format(data.temperature,
                                                                                 data.pressure,
                                                                                 data.humidity,
                                                                                 unix_time_ms)

        payload += line

        if counter % SENDING_PERIOD == 0:
            try:
                # try to send data to cloud
                r = requests.post(URL, params=corlysis_params, data=payload)
                if r.status_code != 204:
                    raise Exception("data not written")
                payload = ""
            except:
                problem_counter += 1
                print('cannot write to InfluxDB')
                if problem_counter == MAX_LINES_HISTORY:
                    problem_counter = 0
                    payload = ""

        counter += 1

        # wait for selected time
        time_diff_ms = int(time.time()*1000) - unix_time_ms
        print(time_diff_ms)
        if time_diff_ms < READING_DATA_PERIOD_MS:
            time.sleep((READING_DATA_PERIOD_MS - time_diff_ms)/1000.0)


if __name__ == "__main__":
    main()

This script is also located at github

Visualize data in Grafana

One picture is usually better than “millions” words and video is usually better than “millions” pictures. So I prepared for you video that shows you how to create Grafana dashboard and visualize your sensor data: