Golioth

How to integrate Golioth.io IoT Platform with Datacake

About Golioth

The Golioth.io IoT platform is a middleware solution for managing and connecting IoT devices. As a connectivity option, it provides IoT devices with a CoAP interface and makes connecting them to cloud systems a breeze.

Incoming messages can be cached on the Golioth platform or forwarded to applications via output streams as HTTP messages with JSON payload.

Read more about Golioth here: https://golioth.io/

In this tutorial, we will show you how to connect the Golioth IoT platform to Datacake.

Create Account

If you don't have an account on the Datacake platform yet, you should create one now. To do so, register for free on the platform by opening the following link.

https://app.datacake.de/signup

If you already have an account, please sign up.

New Workspace

Only for registered and trained Datacake users:

If you have already registered and are familiar with the Datacake platform, we still recommend that you create a new workspace for the integration of your first device to better organize the structure in your account.

Create a New Device

Whether you are newly registered or logged in (and optionally created a new workspace) we now need to create a new device.

To do this, please click on the "Add Device" button, which you can find in the device listing of our platform.

Add Device Button on Datacake

This will open a window which asks as a first step for the type of device you want to create.

Here please select the type "API".

Select API Device Type on Datacake Add Device Modal

As a further step, the window now asks for the "Datacake Product". Here please select "New Product from template".

You will see a list of available start templates for the creation of your device or product.

Select Golioth.io API Template

In this list, you will find the Datacake template for our Golioth integration. Please select it and confirm your selection by clicking on the bottom "Next" button.

Do not forget to press next button

Now the window will ask you for details about the device you want to create, especially the serial number and a name.

Device Detail Modal asking for serial number

Serial Number

Datacake usually automatically creates a serial number for your device, which can be changed at any time.

However, if you have already created a device on the Golioth platform, it now makes sense for to copy the serial number of the device from the Golioth platform and use it as the serial number for the new device to be created on Datacake.

To do this, please open your Golioth Console and the Detail View of the particular device you want to integrate into Datacake.

Golioth.io Management Console

In the detail view, you will find an ID. Copy this ID and paste it into the serial number mask on Datacake.

Datacake Device Add Modal asking for Serial Number and Name

Finally, enter a name for the device and confirm your entries by clicking on the "Next" button.

Device Plan

Now you need to decide on a device plan. If you have just registered or created a new workspace, you have the possibility to create up to 2 devices for free.

Chose Datacake IoT Device Plan

For more information, please visit our pricing page. If you plan to use many devices, we will also be happy to create a custom quote for you.

Now select the "Free Plan" and confirm your selection by clicking on the "Add 1 Device" button. The device is now automatically added to your workspace.

Open Device

If the creation of the device was successful, you will now find the new device in the device listing.

Datacake Device List

To continue with the configuration we have to open the device. To do this, please click on the device in the listing. The detail view will open, which starts with the view of a dashboard.

IoT Dashboard for Golioth.io ESP32 Demo Device

This dashboard comes with the template and gives you an easy start with the platform.

Of course, there is no data on the dashboard yet, as we still need to realize a connection with the platform. In the next chapter, we will show you what steps are required in order to do that.

Add Output Stream

Now we need to set up a forwarding of the data from the Golioth platform to Datacake. This is realized on the Golioth side via so-called Output Streams.

The settings for this can be found in the section of the same name, which you can find in the sidebar in the Golioth Console.

Click on the Output Stream element in the sidebar to open the overview of all streams that have already been created.

Add a new Output Stream on Golioth

Here you can create a new output stream that will take care of forwarding the data from your Golioth devices.

To do this, click on "Create an Output Stream".

Create Output Stream

Now you will be asked for the type of Output Stream. Please select the type "Webhooks".

Chose Webhook Output Stream Type on Golioth.io

You will now be asked for a handful of parameters and inputs.

Enter Output Stream Details

Please fill in these fields as described in the following points.

Name

Choose an individual name here. This is for better organization when working with many output streams. For example, you can choose "Datacake Webhook" or "Datacake Integration" as the name.

Event Type Filter

Golioth triggers redirects via webhook for a wide variety of events, which can originate from the devices but also from the platform. If you are interested in forwarding only a specific event type to Datacake, you could use this filter to select it.

It doesn't matter to Datacake though, so we'll leave the filter field blank in this step.

Webhook URI

Now we need to specify the address that will receive the data from the output streams. This is the so-called webhook.

We get this address from Datacake. To do this, switch to the configuration view of your device on Datacake by opening the "Configuration" tab on the device view.

Access Device Configuration on Datacake

Scroll down a bit in the configuration view until you get to the "HTTP Payload Decoder" section.

Below the editor for the payload decoder, you will find the webhook URL, which you can copy directly to your clipboard by clicking on the "Copy" button on the right.

Datacake Webhook Endpoint URL

Now switch back to Golioth and paste this URL into the "Webhook URI" field.

Enter URL on Golioth

Extra Headers

Since we don't need to assign extra headers, you can confirm the entry by clicking the "Save" button.

After the successful creation, Golioth should show you the detail-view of your output stream in the browser. It looks something like this:

Output Stream Overview

Run a short Test

If you want to test the forwarding, you can now open the tab "Custom Test" in the same view. You can specify a custom demo payload and send it to Datacake as a test.

Run an Output Stream Testing

The JSON set by default in the editor can remain unchanged here.

Now trigger the test via a kick on the "Send Test Event" button.

See Logs on Datacake

Now switch back to the Datacake platform and the configuration view.

In the HTTP Payload Decoder section, you will find a button "Show logs" on the right side next to the header.

How to find Logs on Datacake API Devices

Clicking this button will open a window listing all the messages coming in via the webhook URL.

The Golioth.io Output Stream Test Message on Datacake IoT Platform

If the URL was entered correctly on the Golioth.io Console, you should now also find the test event here.

Troubleshooting

I don't see a test message

Please double-check that you have entered the correct Webhook URL.

Working with Payload

For the rest of the steps, let's now assume that your device on Golioth sends the following data via the output streams.

{
   "data":{
      "humidity":33.532226,
      "press":97.51296,
      "temp":17.62
   },
   "device_id":"620c567d8fd41a59112fe028",
   "project_id":"datacake-integration",
   "timestamp":{
      "nanos":819743612,
      "seconds":1645362730
   }
}

Let's take a closer look at the payload.

The payload of the device is encapsulated in the data object. There are measured values like temperature, etc.

We now want to forward this measurement data to the template device on Datacake.

Payload Decoder

To do this, Datacake uses the principle of payload decoders. This is a small piece of JavaScript code that is executed on incoming messages, and which can be used to extract data and forward it to appropriate devices.

This forwarding takes place based on the serial number of the Datacake device, and if you recall, we replaced this serial number with the ID we found specified on the Golioth platform for the device.

We now also find this ID in the payload that the output stream forwarded to us.

In order to be able to forward the data to the corresponding Datacake device, we need the following payload decoder:

function Decoder(request) {

    // Get Event Type from Request Headers
    var eventType = request.headers["Ce-Type"];

    // Route based on Event Type (Device User Data Stream)
    if (eventType === "DEVICE_STREAM_TYPE") {

        // Convert Request Payload to JSON
        var payload = JSON.parse(request.body);

        // Extract Golioth Device-ID to use as Datacake Serial
        var serial = payload.device_id;

        // Read User Data from Event Type
        var data = payload.data.environment;

        // Return User Data to Datacake
        return [
            {
                device: serial
                field: "TEMPERATURE",
                value: data.temp
            },
            {
                device: serial
                field: "PRESSURE",
                value: data.press
            },
            {
                device: serial
                field: "HUMIDITY",
                value: data.humidity
            },
        ];
    }
}

At the end of the decoder, you can see that the last step is to return an array containing the device's payload data with a serial number and an indication of a field.

This field is a unique identifier for the respective database field on the device that carries the serial number.

Template Decoder

Since we created the device on Datacake using a template, this device already includes a payload decoder prepared for the most common event types of the Golioth platform. So you don't need to do any further steps. The part above about payload decoders is just for understanding.

Database Fields

Payload decoders store the data in corresponding fields of the database of your device. You can freely define the number and types of these fields on Datacake.

A listing of all fields for the device can be found in the "Fields" section on the same configuration view of your Datacake device.

Simply scroll down until you reach the "Fields" section.

Database Fields on Datacake IoT Platform

Here you will already find some fields that were automatically created by the Template you selected.

Since we created the device via a template in this example, the device already has some fields. You can now add any fields if your device has more data than we show here in this example.

However, since we will stay with the current example in the rest of this documentation, we will not make any modifications to the fields or add any new ones now. That is up to you.

If needed, you are welcome to add more fields, delete existing ones or modify them if necessary. More details about fields can be found here.

Testing the Decoder

Since we created the device on Datacake using the Golioth template in this example, it already contains a payload decoder that supports and decodes the standard Golioth Console event types and Fields

However, we don't know what payload your device sends to Datacake via Golioth, so you still might need to configure the Datacake decoder to the payload of your device and add additional database fields accordingly.

At the end of the HTTP Payload Decoder section, you will find tools for testing or simulating the payload decoder.

Decoder Section on Datacake IoT

Here you can enter the JSON payload of the output stream into a text field and simulate the decoding by pressing a button. These tools are perfect for debugging the decoders.

Add Header

In order for us to simulate a Golioth Output Stream message, we need to specify a specific header type. To do this, please use the Headers field and specify the following header.

• Header: Ce-Type

• Header-Value: DEVICE_STREAM_TYPE

Add Payload

Now use the body text field and insert the JSON of an output stream here. You can also use the payload given above to check the example.

Run the Test

In the HTTP Payload Decoder area, you will now find a button for running the simulation.

Test the Decoder

Clicking this button will start the simulation. You will find the result at the end of the section in the respective output areas.

Output Section on Datacake IoT Platform

Output-Area

The left element of the output fields shows you what the payload decoder returned to the Datacake API.

Here the format should always correspond to the Datacake API and be structured as follows:

[
    {
        "device": "SERIAL_OF_DEVICE_ON_DATACAKE",
        "field": "FIELD_IDENTIFIER",
        "value": 12.34,
        // "timestamp": 293840295 // OPTIONAL UNIX Timestamp
    },
    ...
]

Recognized Measurements

The right element of the output shows you which measurement fields were detected by the API in the payload returned by the decoder.

If measured values are not listed here, although they were returned by the decoder (and can also be seen on the left output), this usually means that no field has yet been created in the database for this measured value (and identifier).