Incoming Webhooks
How to connect Devices to Datacake using Webhook (from Third-Party IoT Platform such as Pycom PyBytes, Actinius I/O, Blues Wireless Notehub / Notecard - or any other source that can send data via Web)
With our new Webhook integration, anything that can relay messages via HTTPS call can be connected to Datacake. This is also called Webhook.
The smart thing about this is that we don't require a structure, but instead provide tools that you can use to accept any format and "decode" it appropriately.
We do this by providing Payload Decoders - a concept well known on LoRaWAN ecosystems.
To add a new Device that can receive data through our Webhook Integration please press on "Add Device" in the fleet overview of your workspace.
This will bring up a modal where you have to select the "API" Device type.
Now chose "New Product" and provide a new for this product, something like "My Temperature Product" or "My Sensor Template".
Each device on Datacake belongs to a product, which defines:
- The dashboard
- The fields in the database
- Payload Decoder
- Routing of the data to the devices
If you add another device and add it to the product we are creating right now, this new device will inherit the dashboard, decoder and database fields from the product.
- The data is stored individually.
- Dashboard, Decoder and the number of fields in the database will be shared between all devices belonging to the same product.
- If you change settings on the product (dashboard, add/remove or change fields) then these changes immediately affect all devices that belong to the product.
In the next step you are asked for the devices that are being created and added to the product you defined in the step before.
Right now we only want to create a single device but this dialog would allow us to create multiple devices and add them all to the same product.
Serial numbers are used to identify devices when ingesting data via Webhook. Datacake auto-generates a Serial Number for you which you at any time later can change to a custom one.
We leave this set to "Auto-Generate" for now.
Enter a descriptive name for this device here.
Before we can continue we need to select a plan this device (or multiple devices) will belong to. We do support many different plans and your first two devices are free.
So in case you are new to Datacake, select "Free". This will let you continue directly.
For more Information on Pricing please visit the following page on our website:
Now that we have created our first device, we open it up by clicking on the List-Entry and use the Navigation Tab-Bar to access its Configuration-View:
Next step is creating a few fields in the database section of this device. So we now scroll down a bit until we reach the "Fields" section:
Here we press on "Add Device".
Using this Modal / View we create the following fields:
- Temperature
- Humidity
- Battery Voltage
So that in the end the Database Section would look like the following:
Please note that the Database Fields are Product-wide Settings!
Even if it seems that we have only attached the fields of the database to the respective device, the fields are stored in the database in the definition of the product.
This means: If you now add another device to the existing product, it will automatically receive the same database fields (but can store its own data).
Now that we have the Database Section set up we can move on and get to the part where the data is being forwarded.
Each product on Datacake has its own custom API endpoint.
This URL can be found in the Configuration under the Payload Decoder Settings section.
And it looks a bit like this:
https://api.datacake.co/integrations/api/2de1c5ade79bcef8/You can add custom paths to your Webhook URL if you want to divide the data of the payload into separate API calls, for example:
https://api.datacake.co/integrations/api/2de1c5ade79bcef8/data/https://api.datacake.co/integrations/api/2de1c5ade79bcef8/status/https://api.datacake.co/integrations/api/2de1c5ade79bcef8/diagnostics/
If your Content Type is
application/x-www-form-urlencoded - You can send the payload as a You can send JSON and parse that into an object.
You can also send data directly as Query Parameters (in combination with custom Paths and User Payload Data):
https://api.datacake.co/integrations/api/2de1c5ade79bcef8/?device=123&check=truehttps://api.datacake.co/integrations/api/2de1c5ade79bcef8/diagnostics/?foo=bar
With the Payload Decoder you have the possibility with the help of a small piece of JavaScript code to customize the data of the webhook so that you can forward it to Datacake's database.
In summary, a payload decoder has the following features:
- Code snippet that is executed when the webhook URL is called.
- Access to measured values of all devices of the product.
- Routing into the respective device or a group of devices.
The Payload Decoder is connected to the Webhook URL and is executed whenever a message / HTTP call is received via this URL.
Payload Decoder are product-wide settings!
Since each product on Datacake has its own webhook URL, so is the payload decoder tied to that product.
To forward data from the payload decoder to the respective device, a corresponding ID (serial number) must be passed. We will show you how to do this in the following steps.
Let's get together a very quick example. Imagine your Webhook contains device data encapsulated as a JSON String. So this could look like this:
curl --header "Content-Type: application/json" -X POST
--data '{"device":"MySerial01","data":{"temp":23.01,"bat":3.21}}'
https://api.datacake.co/integrations/api/2de1c5ade79bcef8/
To extract this data the decoder on Datacake would look like this:
function Decoder(request) {
// Parse JSON into Object
var payload = JSON.parse(request.body);
// Extract Serial for Routing into Device
var device = payload.device;
// Load Data from JSON
var data = payload.data;
var temperature = data.temp;
var humidity = data.hum;
var battery = data.bat;
// Forward Data into Device using Serial
return [
{
device: device,
field: "TEMPERATURE",
value: temperature
},
{
device: device,
field: "HUMIDITY",
value: humidity
},
{
device: device,
field: "BATTERY_VOLTAGE",
value: battery
},
];
}
You can test the Decoder using the built in tools that you also find in the Payload Decoder section in the Device Configuration View.
Simply paste in the JSON as a text into the Body-Field (marked red) and press on "Try Decoder". You can see the Output of the Decoder in the Debug-Fields (marked blue).
If the payload decoder was able to successfully forward data into a device (given correct serial number) you can see the mapping under the "Recognized Measurements" section:
You can use CURL or any other URL Calling tool to simulate the API Webhook and test the Decoder under real conditions.
curl --header "Content-Type: application/json" -X POST
--data '{"device":"MySerial002","data":{"temp":23.01,"bat":3.21,"hum":54.22}}'
https://api.datacake.co/integrations/api/25f1de28-2630-bea1-be33-c5adc79bef68/
After Data has come in you can see this on the Database Fields in the Fields-Section.
You can see all incoming Webhooks in the Logs-Section.
Simply click on "Show Logs" which will open a Logging-View showing the last recent Webhook calls.
The following is important. Please read carefully!
Decoders are bound to the URL and the Product the URL belongs to. They are not per Device. So you need to return a routing information so that Datacake can route your data into the actual device.
We do this by providing an additional Key/Value pair when returning data from the payload decoder. You might have already seen that all Payload Decoder Snippets above return a
device-Key/Value in their return type, so like:return [
{
device: deviceID, // Serial Number or Device ID
field: "TEMPERATURE",
value: temperature
}
];
When creating a new API Device Datacake does autogenerate a Serial Number for you. However you can override this Serial Number and paste in Serial Number or any other Identification Info that is also present in the payload of your Webhook.
So imagine your Webhook does contain the following ID:
{"id":"MySerial001","temperature":22.01}
The important part here is the:
"id":"MySerial001" - We can now override the Serial Number of the Device on Datacake and set it to MySerial001.So in the Configuration-View on the Device in Datacake you find an API Configuration Section that shows the current Serial Number and a Button "Change".
If you click on that you can override the Serial. In the case of this example here we override this to
MySerial001.
So once more. Imagine you have the following Webhook Payload:
{"id":"MySerial001","temperature":22.01}
The Decoder with correct device routing information would look like this.
function Decoder(request) {
var payload = JSON.parse(request.body);
var deviceID = payload.id;
var temperature = payload.temperature;
return [
{
device: deviceID, // Serial Number or Device ID
field: "TEMPERATURE",
value: temperature
}
];
}
What if your Webhook does not contain device identification like a serial number or any other? You can of course hard-code the Device ID in your Decoder, so this would look like this:
return [
{
device: "MyHardcodedSerial",
field: "TEMPERATURE",
value: temperature
}
];
You may have noticed that the concept of Device-ID Identification is not present on the native integrations for LoRaWAN or Particle.io.
This is correct, because here the routing is automated in the backend. This works because the webhook structure is fixed and Datacake always knows where to find the ID (namely DevEUI or Particle Device-ID -Which is also the reason why the serial number for LoRaWAN or Particle is the serial of the respective integration).
Let's assume you are sending JSON as payload:
curl --header "Content-Type: application/json" -X POST
--data '{"device":"MySerial01","data":{"temp":23.01,"bat":3.21}}'
https://api.datacake.co/integrations/api/123456/
If you want to decode JSON you can turn
// Parse JSON into Object
var payload = JSON.parse(request.body);
// Extract Serial for Routing into Device
var device = payload.device;
// Load Data from JSON
var data = payload.data;
var temperature = data.temp;
var battery = data.bat;
// Forward Data into Device using Serial
return [
{
device: device,
field: "TEMPERATURE",
value: temperature
},
{
device: device,
field: "BATTERY",
value: battery
},
];
if your Content Type is:
application/x-www-form-urlencoded - you can send a body like this:deviceid=MyDeviceID123&temperature=23.01&battery=3.43
The request-Object in your Payload Decoder will hold already parsed Objects of this payload which you can access in the following way:
function Decoder(request) {
var payload = request["POST"];
var deviceID = payload.deviceid[0];
var battery = payload.battery[0];
var temperature = payload.temperature[0];
return [
{
device: deviceID, // Serial Number or Device ID
field: "TEMPERATURE",
value: temperature
},
{
device: deviceID, // Serial Number or Device ID
field: "BATTERY",
value: battery
}
];
}
If you are Sending Data as Query Parameters like this:
https://api.datacake.co/integrations/api/2de1c5ade79bcef8/?device=123&check=true
The following snippet shows you how to access those in the Decoder:
function Decoder(request) {
var payload = request["GET"];
var deviceID = payload.device[0];
var temperature = payload.temperature[0];
return [
{
device: deviceID, // Serial Number or Device ID
field: "TEMPERATURE",
value: temperature
}
];
}
If you want to combine URL Query Parameters with Form Encoded Data or any other Payload you can do this using the following snippet:
function Decoder(request) {
// query is always GET - even on POST
var queryParameters = request["GET"];
// payload is on POST only
var payloadData = request["POST"];
...
}
You can query info about the current Method and use this as a routing info for your payload decoder:
function Decoder(request) {
if (request.method === "POST") {
// Do Stuff when "POST"
} else if (request.method === "GET") {
// Do Stuff when "GET"
}
...
}
You can use Custom URL Endings on your API Webhook URL to separate Actions inside the Payload decoder. So let's assume you are using the following URL Endings:
https://api.datacake.co/integrations/api/2de1c5ade79bcef8/data/https://api.datacake.co/integrations/api/2de1c5ade79bcef8/status/https://api.datacake.co/integrations/api/2de1c5ade79bcef8/diagnostics/
Here is how you use these in a Payload Decoder:
function Decoder(request) {
var data = JSON.parse(request.body);
var urlElements = request.path.split('/');
var lastURLSegment = urlElements[urlElements.length-2];
if (lastURLSegment === "data") {
} else if (lastURLSegment === "status") {
} else if (lastURLSegment === "diagnostics") {
}
}
If your Webhook Payload contains data for multiple devices you can return them all in a single call.
Imagine your Webhook is sending JSON Data that contains payload for multiple devices, structured something like this.
[
{"id":"MySerial001","temperature":22.01},
{"id":"MySerial002","temperature":22.01},
{"id":"MySerial003","temperature":22.01}
]
In this case the Decoder would look something like this:
function Decoder(request) {
var devices = JSON.parse(request.body);
for (var i = 0; i < devices.length; i++) {
var device = devices[i];
datacakeFields.push({
device: device.id, // Serial Number or Device ID
field: "TEMPERATURE",
value: device.temperature
});
}
return datacakeFields;
}
Please make sure that the JSON-Data contains information that can be used to identify the devices in the payload decoder (eg. Serial Number).
Lets assume you have a temperature sensing device that collects four individual measurments (over a time of 15 Minutes) and then sends them as a combined message (each hour).
The payload would look like this:
{"id":"MySerial001","temperatures":[21.23, 22.34, 45.43, 56.40]}
Our Decoders can return timestamps so you can backfill the Database Fields. This would look something like the following snippet:
function Decoder(request) {
var devices = JSON.parse(request.body);
var deviceID = payload.id;
var temperatures = payload.temperatures;
// current Unix Timestamp
var timestamp = Math.floor(Date.now() / 1000);
for (var i = 0; i < temperatures.length; i++) {
var temperature = temperatures[i];
datacakeFields.push({
device: deviceID, // Serial Number or Device ID
field: "TEMPERATURE",
value: temperature
timestamp: timestamp
});
// decrease Timestamp 15 Minutes (go back in time)
timestamp = timestamp - (15 * 60);
}
return datacakeFields;
}
Let's assume you are calling the following Webhook:
curl -X POST -d 'post=parameter'
https://api.datacake.co/integrations/api/123456/optionalsubpath?foo=bar
In your Payload Decoder you have the access to the following parameters of your HTTP Webhook everytime there is a Webhook call happening:
{
"method": "POST",
"GET": {
"foo": ["bar"]
},
"POST": {
"post": ["parameter"]
},
"body": "post=parameter",
"headers": {
"Content-Length": "14",
"Content-Type": "application/x-www-form-urlencoded"
},
"path": "/integrations/api/123456/optionalsubpath"
}
If your device has GPS coordinates and you want to store them to a geolocation field type, you have to return the latitude and longitude in the form of a tuple. See the following example for an explanation.
function Decoder(request) {
var payload = JSON.parse(request.body);
var deviceID = payload.serial;
var lat = payload.latitude;
var lon = payload.longitude;
var position = "(" + lat + "," + lon + ")";
return [
{
device: deviceID,
field: "LOCATION",
value: location
}
];
}
Last modified 1yr ago