readme

weewx-sdr
Copyright 2016-2019 Matthew Wall
Distributed under terms of the GPLv3

This is a driver for weewx that captures data from software-defined radio.
It works with open source rtl sdr software that in turn works with
inexpensive, broad spectrum radio receivers such as the Realtek RTL2838UHIDIR.
These devices cost about 20$US and are capable of receiving radio signals from
weather stations, energy monitors, doorbells, and many other devices that use
unlicensed spectrum such as 433MHz, 838MHz, and 900MHz frequencies.


===============================================================================
Hardware

Tested with the Realtek RTL2838UHIDIR.  Should work with any software-defined
radio that is compatible with the rtl-sdr software.  Uses the modules in
rtl_433 to recognize packets.

Output from many different sensors is supported.  To see the list of supported
sensors, run the driver directly with the list-supported action.

If a sensor is supported by rtl_433 but not by weewx-sdr, it is a fairly simple
matter of writing a parser for that sensor within weewx-sdr.  Things are a bit
more complicated if a sensor is not supported by rtl_433.


===============================================================================
Installation

0) install pre-requisites

a) install weewx
    http://weewx.com/docs.html
b) install rtl-sdr
    http://sdr.osmocom.org/trac/wiki/rtl-sdr
c) install rtl_433
    https://github.com/merbanan/rtl_433

1) download the driver

wget -O weewx-sdr.zip https://github.com/matthewwall/weewx-sdr/archive/master.zip

2) install the driver

sudo wee_extension --install weewx-sdr.zip

3) configure the driver

sudo wee_config --reconfigure --driver=user.sdr --no-prompt

4) run the driver directly to identify the packets you want to capture

cd /home/weewx
sudo PYTHONPATH=bin python bin/user/sdr.py --cmd="rtl_433 -M utc -F json"

5) modify the [SDR] section of weewx.conf using a text editor

- create a [[sensor_map]] for the data you want to capture
- possibly modify the 'cmd' parameter

6) start weewx

sudo /etc/init.d/weewx start


===============================================================================
How to run the driver directly

Run the driver directly for testing and diagnostics.  For example, if weewx
was installed using setup.py:

cd /home/weewx
sudo PYTHONPATH=bin python bin/user/sdr.py --help

If weewx was installed from deb or rpm:

sudo PYTHONPATH=/usr/share/weewx python /usr/share/weewx/user/sdr.py --help


===============================================================================
Configuration

Use the [SDR] section of the weewx configuration file (nominally weewx.conf) to
adjust the driver configuration.

The default configuration uses this command:

rtl_433 -M utc -F json

Specify different options using the cmd parameter.  For example:

[SDR]
    driver = user.sdr
    cmd = rtl_433 -M utc -F json -R 17 -R 44 -R 50

The rtl_433 executable emits data for many different types of sensors, some of
which have similar output.  Use the sensor_map to distinguish between sensors
and map the output from rtl_433 to the database fields in weewx.

Here are some examples:

# collect data from Acurite 5n1 sensor 0BFA and t/h sensor 24A4
[SDR]
    driver = user.sdr
    [[sensor_map]]
        windDir = wind_dir.0BFA.Acurite5n1Packet
        windSpeed = wind_speed.0BFA.Acurite5n1Packet
        outTemp = temperature.0BFA.Acurite5n1Packet
        outHumidity = humidity.0BFA.Acurite5n1Packet
        rain_total = rain_total.0BFA.Acurite5n1Packet
        inTemp = temperature.24A4.AcuriteTowerPacket
        inHumidity = humidity.24A4.AcuriteTowerPacket

# collect data from Acurite 986 fridge/freezer sensor set 1R and 2F
[SDR]
    driver = user.sdr
    [[sensor_map]]
        extraTemp1 = temperature.1R.Acurite986Packet
        extraTemp2 = temperature.2F.Acurite986Packet

# collect data from Acurite 06002RM t/h sensor 3067
[SDR]
    driver = user.sdr
    [[sensor_map]]
        inTemp = temperature.3067.AcuriteTowerPacket
        inHumidity = humidity.3067.AcuriteTowerPacket

# collect data from two Hideki TS04 sensors with channel=1 and channel=2
[SDR]
    driver = user.sdr
    [[sensor_map]]
        outBatteryStatus = battery.1:9.HidekiTS04Packet
        outHumidity = humidity.1:9.HidekiTS04Packet
        outTemp = temperature.1:9.HidekiTS04Packet
        inBatteryStatus = battery.2:9.HidekiTS04Packet
        inHumidity = humidity.2:9.HidekiTS04Packet
        inTemp = temperature.2:9.HidekiTS04Packet

# collect data from Fine Offset sensor cluster with serial number 0026
[SDR]
    driver = user.sdr
    [[sensor_map]]
        windGust = wind_gust.0026.FOWH1080Packet
        outBatteryStatus = battery.0026.FOWH1080Packet
        rain_total = rain_total.0026.FOWH1080Packet
        windSpeed = wind_speed.0026.FOWH1080Packet
        windDir = wind_dir.0026.FOWH1080Packet
        outHumidity = humidity.0026.FOWH1080Packet
        outTemp = temperature.0026.FOWH1080Packet

To figure out the sensor identifiers, run the driver directly, possibly with
the --debug option.  Another option is to run weewx with the logging options
for [SDR] enabled to display the sensors found by rtl_433, the sensor
identifiers used by weewx, and the sensors actually recognized by weewx.

[SDR]
    driver = user.sdr
    log_unknown_sensors = True
    log_unmapped_sensors = True

By default the logging options are False.


===============================================================================
How to diagnose problems

First try running the rtl_433 application to be sure that it works properly:

sudo rtl_433

Be sure that you are capturing data from the sensors you care about.  To do
this, you might want to experiment with some of the options to rtl_433.  For
example, this will eliminate extraneous output (-q), and use UTC for timestamps
(-M utc):

sudo rtl_433 -M utc -F json

If you know exactly which sensors you want to monitor, try the -R option to
reduce the clutter.  For example,

sudo rtl_433 -M utc -F json -R 9 -R 31

Once that is working, run the driver directly to be sure that it is collecting
data from the rtl_433 application.

cd /home/weewx
sudo PYTHONPATH=bin python bin/user/sdr.py

Make note of the sensor identifiers.  Each identifier is a three-part string
consisting of the observation, sensor id, and rf packet type, separated by
periods.  You need these in the sensor map to tell weewx the association
between sensors and database fields.

In the weewx configuration file, enter the rtl_433 options (if necessary) and
the sensor map.  Then run weewx directly in one shell while you monitor the
weewx log in a separate shell:

in shell 1:
cd /home/weewx
sudo ./bin/weewxd weewx.conf

in shell 2:
tail -f /var/log/syslog

At this point, verify that the mapping you made in the weewx configuration file
is working as you intend.  You should see data from your sensors in the weewx
LOOP and REC output.  If not, check the log file.  Use the logging options in
the [SDR] section of the weewx configuration file to help figure out which
sensors are captured, recognized, and parsed.

Once you are satisfied with the output when running weewx directly, run weewx
as a daemon and configure rc script or systemd to run weewx at system startup.

sudo /etc/init.d/weewx start


===============================================================================
Environment

The driver invokes the rtl_433 executable, so the path to that executable and
any shared library linkage must be defined in the environment in which weewx
runs.

For example, with rtl_433 and rtl-sdr installed like this:

/opt/rtl-433/
/opt/rtl-sdr/

one would set the path like this:

export PATH=/opt/rtl-433/bin:${PATH}
export LD_LIBRARY_PATH=/opt/rtl-sdr/lib

Typically this would be done in the rc script that starts weewx.  If rtl_433
and rtl-sdr are install to /usr/local or /usr, then there should be no need
to set the PATH or LD_LIBRARY_PATH before invoking weewx.

If you cannot control the environment in which weewx runs, then you can specify
the LD_LIBRARY_PATH and PATH in the weewx-sdr driver itself.  For example:

[SDR]
    driver = user.sdr
    cmd = rlt_433 -M utc -F json
    path = /opt/rtl-433/bin
    ld_library_path = /opt/libusb-1.0.20/lib:/opt/rtl-sdr/lib
    [[sensor_map]]
        ...


===============================================================================
libusb

I have had problems running rtl-sdr on systems with libusb 1.0.11.  The rtl_433
command craps out with a segmentation fault, and the rtl_test command sometimes
leaves the dongle in a weird state that can be cleared only by unplugging then
replugging the dongle.

Using a more recent version of libusb (e.g., 1.0.20) seems to clear things up.


===============================================================================
Support for new sensors

To add support for new sensors, capture the output from rtl_433.  To capture
output, run the driver directly and hide known packets:

PYTHONPATH=bin python bin/user/sdr.py --cmd "rtl_433 -M utc -F json" --hide parsed,out,empty

This should emit a line for each unparsed type.  For example:

unparsed: ['{"time" : "2017-01-16 15:44:51", "temperature" : 54.140, "humidity" : 34, "id" : 221, "model" : "LaCrosse TX141TH-Bv2 sensor", "battery" : "OK", "test" : "Yes"}\n']

If there is no json ouput, remove the '-F json' option to see the (deprecated)
non-json output:

unparsed: ['2016-11-04 16:12:39 :\tFine Offset Electronics, WH2 Temperature/Humidity sensor\n', '\tID:\t 38\n', '\tTemperature:\t 54.4 C\n', '\tHumidity:\t 55 %\n']

If you are not comfortable writing your own parser, post the output to the
issues section of the weewx-sdr repository and some helpful person might write
the parser for you.