Payload Decoders

LoRa devices send their readings as bytes instead of a full JSON object in order to save bandwidth. For Datacake to be able to process and store the sent measurements, you need a piece of JavaScript code called a Payload Decoder.

A Payload Decoder takes a buffer (list of bytes) as well as the port of the message and converts them to a normalized object Datacake can understand.

If you want to send data to your devices, you can define Downlinks (or Payload Encoders). You'll find all the Informations about this here:

A simple Payload Decoder

Imagine a LoRa sensor sending a single measurement, which is a float indicating a temperature.

To encode this measurement, one could use the following code:

float temperature = 23.5;
byte payload[1];
payload[0] = round(temperature * 100);

In this example, we are first convert the float to an integer without losing precision by multiplying it with 100. To decode it back to a float, we simply divide it by 100 again. The following code is an example, how a Payload Decoder can look on the Datacake platform.

function Decoder(payload, port) {
if(port === 1) {
return [
{
field: "TEMPERATURE",
value: payload[0] / 100
}
];
}
}

As you can see, a Payload Decoder must contain a function Decoder which takes the payload and port as arguments and returns a list of measurements in the {field: str, value: any, timestamp?: number} format.

Recording multiple measurements

A Payload Decoder can return a list of an arbitrary number of measurements.

function Decoder(payload, port) {
return [
{
field: "FIELD1",
value: true
},
{
field: "FIELD2",
value: false
}
];
}

Recording historic data

Some LoRa devices send an aggregated list of measurements for e.g. the last hour. Each measurement can take an optional timestamp parameter which is a UNIX timestamp (seconds since Jan 01 1970 (UTC)).

function Decoder(payload, port) {
return [
{
field: "FIELD1",
value: true,
timestamp: 1589815300
},
{
field: "FIELD1",
value: false,
timestamp: 1589811700
}
];
}

Converting TTN Payload Decoders

You can use JS-Style Payload Decoders that work in your TheThingsNetwork Applications on Datacake. However, they must be adjusted so that the output of these decoders meets the expectations of Datacake.

Mostly a TTN Decoder Looks like the following:

/**
* Ursalink Sensor Payload Decoder
*
* definition [channel-id] [channel-type] [channel-data]
*
* 01: battery -> 0x01 0x75 [1byte] Unit: %
* 03: temperature -> 0x03 0x67 [2bytes] Unit: °C
* ------------------------------------------ EM500-PT100
*/
function Decoder(bytes, port) {
var decoded = {};
​
for (var i = 0; i < bytes.length;) {
var channel_id = bytes[i++];
var channel_type = bytes[i++];
// BATTERY
if (channel_id === 0x01 && channel_type === 0x75) {
decoded.battery = bytes[i];
i += 1;
}
// TEMPERATURE
else if (channel_id === 0x03 && channel_type === 0x67) {
decoded.temperature = readInt16LE(bytes.slice(i, i + 2)) / 10;
i += 2;
} else {
break;
}
}
return decoded;
}
​
/* ******************************************
* bytes to number
********************************************/
function readUInt16LE(bytes) {
var value = (bytes[1] << 8) + bytes[0];
return value & 0xffff;
}
function readInt16LE(bytes) {
var ref = readUInt16LE(bytes);
return ref > 0x7fff ? ref - 0x10000 : ref;
}

This Decoder returns the payload structured as follows:

payload = {
"temperature": 21.3,
"battery": 55
};
​
return payload;

Datacake expects this to be:

payload = [
{ "field": "TEMPERATURE", "value": 21.3 },
{ "field": "BATTERY", "value": 55 },
];
​
return payload;

So instead of having a dictionary with a key:value-pair for every field or measurement value (like temperature, battery), Datacake expects an Array to be returned with a Dictionary for each value you would like to forward into Datacake.

Each Dictionary inside that Array consists of at least 2 elements:

  1. The Field-Identifier ("field": "TEMPERATURE")

  2. The actual value of that field ("value": 21.3)

Also as Field-Identifiers in Datacake are always in capital letters you have to turn temperature into TEMPERATURE in order for the payload decoder to match the returned values to the database fields of your device.

Now let's convert this to Datacake.

/**
* Ursalink Sensor Payload Decoder
*
* definition [channel-id] [channel-type] [channel-data]
*
* 01: battery -> 0x01 0x75 [1byte] Unit: %
* 03: temperature -> 0x03 0x67 [2bytes] Unit: °C
* ------------------------------------------ EM500-PT100
*/
function Decoder(bytes, port) {
// decoded needs to be an array so change this:
// var decoded = {};
// to:
var decoded = [];
​
for (var i = 0; i < bytes.length;) {
var channel_id = bytes[i++];
var channel_type = bytes[i++];
// BATTERY
if (channel_id === 0x01 && channel_type === 0x75) {
decoded.push({
"field": "BATTERY",
"value": bytes[i]
});
i += 1;
}
// TEMPERATURE
else if (channel_id === 0x03 && channel_type === 0x67) {
decoded.push({
"field": "TEMPERATURE",
"value": readInt16LE(bytes.slice(i, i + 2)) / 10;
});
i += 2;
} else {
break;
}
}
return decoded;
}
​
/* ******************************************
* bytes to number
********************************************/
function readUInt16LE(bytes) {
var value = (bytes[1] << 8) + bytes[0];
return value & 0xffff;
}
function readInt16LE(bytes) {
var ref = readUInt16LE(bytes);
return ref > 0x7fff ? ref - 0x10000 : ref;
}

Accessing Metadata

You can access the global variable normalizedPayload which contains the following metadata:

{
"deveui": string, // The device's DevEUI
"port": number, // The message's port
"counter": number,
"frequency": number, // GHz
"data_rate": string, // Data rate according to https://blog.dbrgn.ch/2017/6/23/lorawan-data-rates/
"coding_rate": string,
"gateways": [
"id": string,
"datetime": date,
"channel": number,
"rssi": number,
"snr": number
]
}

This data is extracted from the LoRaWAN Network Server as being forwarded to Datacake over the Webhook. This information is normalized, meaning no matter whether you use TTN, TTI, Loriot, ChirpStack, or Wanesy, the information is always equally available and always formatted in the same way.

Raw Metadata

If you rather want to access the raw payload received from the LNS, you can do so via the global rawPayload variable.

Please note that when you access the rawPayload, the information is not normalized and the content of rawPayload looks different between different LoRaWAN Network Server!

Example Usage

In the following snippet you find an example of how to read the Metadata from your Payload and how to bring that into your Datacake Payload Decoder section.

function Decoder(payload, port) {
// Output normalized Payload
console.log(JSON.stringify(normalizedPayload,0,4));
// Output raw payload coming from webhook of your LNS
console.log(JSON.stringify(rawPayload,0,4));
// Extract RSSI and Data-Rate from normalized Payload:
// As being "normalized" this works between all LNS:
var LORA_RSSI = normalizedPayload.gateways[0].rssi;
var LORA_DATARATE = normalizedPayload.data_rate;
// Build up an array that can be used to forward it onto Datacake:
var decoded = [
{
field: "LORA_RSSI",
value: LORA_RSSI
},
{
field: "LORA_DATARATE",
value: LORA_DATARATE
}
];
return decoded;
}

Create corresponding fields in Database

In order to store those Metadata-Fields you can create new Fields on your Device and have historical Data for each field that you forward and store. Here is how this looks for the above example:

Accessing measurements

The global measurements object allows you to access the device's current measurements. It is structured like this:

{
"BATTERY": {
"field_name": "BATTERY",
"timestamp": "1600378287",
"value": 3.301
},
"SOIL_CONDUCTIVITY": {
"field_name": "SOIL_CONDUCTIVITY",
"timestamp": "1600378287",
"value": 121
}
}

Tips and tricks

Execution time

A Payload Decoder must not take longer than 100ms (milliseconds) to return, otherwise its execution will be aborted and you will find a note on the Debug tab of the device.

Access to base64-encoded payload

The base64-encoded payload is available via the global b64payload-variable.

Converting the buffer to a string

You should avoid sending strings over LoRa, since text uses a lot of bytes. You can still decode the buffer to string using:

String.fromCharCode.apply(null, payload);