MQTT

Introduction

Quickstart

To use the new MQTT integration, you just need to create an API device. If you already have API devices in your workspace, then the MQTT integration is available there as well.

Add new API Device

In the "Add Device" modal, select the device type "API" and then click on "New Product".

Define Product

By chosing to add the device under a "New Product" you are required to give a name for this product. You can name it something like "My First MQTT Product".

To complete the step, click the "Next" button.

What is a Product?

Each device on Datacake belongs to a product. The behavior and appearance of the device is defined in this product. Thus, the product defines the following things:

  • The design of the dashboard

  • The fields in the database

  • The payload decoder (for API, MQTT)

  • The individual downlinks

You can add multiple devices to one product. These devices will all have the same dashboard, payload decoders, and database fields. If you make changes to the product, they affect all the devices that belong to the product.

Create Device

In the second step you have to define the device which should be added to the product. Here you can create several devices in one action, but we will limit ourselves to the creation of a single device.

Enter a name here (such as "My First MQTT Device") and complete the creation of the device by clicking on the "Next" button.

Serial Number

When creating the device, you have the option of assigning an individual serial number. This can then be used to allocate existing publishes to a corresponding device.

For example, the individual serial number could be embedded in the topic. The payload decoder can then extract it and forward it along with the data to the device on Datacake whose serial number matches the one from the publish.

Create Database Fields

After creating the device, it is listed in the table of the fleet view. Now open the device by clicking on the entry in the list.

You will then see a device view with an empty dashboard. Now, the first thing we want to do is navigate to the Device configuration. To do this, use the tab bar and click on "Configuration".

Then scroll down on the Configuration View so that you get to the "Fields" section. This looks something like this.

To create a first database field, please click on the "Add Field" button as marked in the screenshot above.

This will open another modal asking for some required input for your first field.

We now want to create a field that will store a temperature. For this we choose the following properties:

  • Type: Float

  • Name: Temperature

When you type in the name for that field, the identifier autofills to the same name but in uppercase letters. This identifier is unique and cannot be changed later on. You can override this field, but it's required for accesing the field from within payload decoder or API.

Once done, please click on "Add Field". You should see the newly created field now added to the list of fields.

Now we are ready to set up our MQTT Integration

Set up Broker

The first thing we want to do now is to connect to a broker through which we will receive the data from the device. To do this, scroll up a little in the same configuration view until you reach the "MQTT Configuration" view.

There you will find a dropdown that has not yet been activated and a button with the title "Add new MQTT Server". Now please click on this button to create a new external broker.

A modal will pop up that is asking you for various information that it needs to connect to your third-party MQTT Broker.

Now enter the server details here as well as authentication and, if necessary, certificate. Datacake supports SSL/TLS as well as unencrypted and non-authenticated brokers, but it is recommended to use SSL in any case.

Testing Connection

Next to the button for creating the broker, in the same modal there is a button for testing the entered server details.

Here you can quickly check whether Datacake can connect to your broker before creating it.

Add Subscriptions

If you have successfully created the broker, then we can now proceed with the next steps and create an initial payload decoder for an MQTT topic.

What is a Payload Decoder?

On the Datacake platform and in the product of your Devices, you can store a separate Payload Decoder for one or multiple Topics. This payload decoder is a subscription to one or multiple topics defined by you.

If data arrives with the topic via the external broker, the respective payload decoder is executed for this topic.

Payload decoders are simple Javascript code functions that unpack the incoming data, process it and forward it to the database of the respective device.

The payload decoder can also be used to extract a serial number from the topic or the data and to use it for correct forwarding to Datacake.

Create your first Subscription

To create a subscription on the broker and a corresponding uplink decoder, click on the "Add Uplink Decoder" button in the "MQTT Configuration".

This will open up a new modal where you can define:

  • The Topic to subscribe to

  • The Payload Decoder Javascript Function Code

Define Topic

After clicking "Add Uplink Decoder", a setup modal will appear where we now specify a subscription to a topic as well as specify a first simple payload decoder.

For the next example, let's imagine that an MQTT device sends us information about a temperature as JSON-Payload via a topic called device/dev0001/data.

To do this, we enter the topic we want to subscribe to in the first input field.

Here you can work with wildcards or subscribe to several topics at the same time.

If the payload decoder is called due to incoming data, the respective topic via which the data was received is available in the payload decoder via the topic variable.

Write Payload Decoder

So now we continue with writing our first Payload Decoder function. This code snippet will take care of the following tasks:

  • Convert incoming payload data to JSON

  • Extract Temperature information

  • Forward this into the actual device on Datacake

For this example we assume that the device we want to connect is sending us the following data:

  • Topic: device/dev0001/data

  • Payload: {"t":32.56, "b": 34, "state":"ONLINE", "di1":true}

The complete Payload Decoder will look like the following:

function Decoder(topic, payload) {
    
    // Transform incoming payload to JSON
    payload = JSON.parse(payload);
    
    // Remember, data is: {"t":32.56, "b": 34, "state":"ONLINE", "di1":true}
    
    // Extract Temperature from payload, do calculation
    var temperature = payload.t;
    var battery = payload.b;
    var state = payload.state;
    var digital_input_1 = payload.di1;
    
    // Forward Data to Datacake Device API using Serial, Field-Identifier
    return [
        {
            device: "bef29be9-c311-4324-ade3-d6e1db8c4b88", // Serial Number or Device ID
            field: "TEMPERATURE",
            value: temperature
        },
        {
            device: "bef29be9-c311-4324-ade3-d6e1db8c4b88", // Serial Number or Device ID
            field: "BATTERY",
            value: battery
        }
        {
            device: "bef29be9-c311-4324-ade3-d6e1db8c4b88", // Serial Number or Device ID
            field: "STATE",
            value: state
        }
        {
            device: "bef29be9-c311-4324-ade3-d6e1db8c4b88", // Serial Number or Device ID
            field: "DIGITAL_INPUT_1",
            value: digital_input_1
        }
    ];
    
}

The steps are easy to understand.

First of all we are parsing the payload as JSON. This allows us to simply access the payload information as an object.

Second we are apply a short conversion to transform the temperature value of 2342 into 23.42

As a third and last step we are forwarding the decoded temperature information to the actual Datacake device. This is done by returning an array of dictionaries, where each dictionary holds the information for the actual field. This is structure like the following:

// Forward Data to Datacake Device API using Serial, Field-Identifier
return [
    // One dictionary for each field, value
    {
        device: << Device UUID or Serial Number >>
        field: << Field Identifier of Field in Database >>
        value: << the actual field value >>
        // Optional: 
        timestamp: << a timestamp for historical data >>
    }
];

Device Routing

Every Device on Datacake belongs to a Product and Payload Decoders are executed on the Product Layer. This means that next to the Field Identifier and Value (you want to store) you have to provide a routing information that identifies the device you want to store the data on.

There are two ways of providing this routing information:

  • Hardcoded

  • Dynamically by taking a serial number from Topic-Structure (wildcards)

In the above example, we have been using the hard-coded way and provided the UUID of the device we want to store the data on.

// Forward Data to Datacake Device API using Serial, Field-Identifier
return [
    {
        device: "bef29be9-c311-4324-ade3-d6e1db8c4b88", // Serial Number or Device ID
        field: "TEMPERATURE",
        value: temperature
    }
];

Example of Wildcards

In the following example, we show you how you can use device ids embedded in an MQTT topic.

  • Create a new Subscription on your Datacake Product.

  • Set this Subscription to device/+/data

  • Paste the following payload decoder:

function Decoder(topic, payload) {

    // Topic is "device/dev0001/data"
    // now we extract "dev0001" as serial
    var serial = topic.split("/")[1]; // "dev0001"
    
    // Transform incoming payload to JSON
    payload = JSON.parse(payload);
    
    // Remember, data is: {"t":32.56, "b": 34, "state":"ONLINE", "di1":true}
    
    // Extract Temperature from payload, do calculation
    var temperature = payload.t;
    var battery = payload.b;
    var state = payload.state;
    var digital_input_1 = payload.di1;
    
    // Forward Data to Datacake Device API using Serial, Field-Identifier
    return [
        {
            device: serial,
            field: "TEMPERATURE",
            value: temperature
        },
        {
            device: serial,
            field: "BATTERY",
            value: battery
        }
        {
            device: serial,
            field: "STATE",
            value: state
        }
        {
            device: serial,
            field: "DIGITAL_INPUT_1",
            value: digital_input_1
        }
    ];
    
}

Override Device Serial

Now you are returning the payload data using a custom serial. Remember also that the payload decoder is being executed on the product level so this means that if you want to add multiple devices to a single product, you need to make sure that the routing takes place on the serial number.

In order to override a Datacake serial number and provide your custom one (which is being used on the device and on the topic, just like above) you need to go into the configuration of your device and set a custom serial.

Test Decoder Function

Below the editor for payload decoder function you find a button ”Show” that lets you unfold the payload decoder testing area.

If you click on that button, the following inputs appear.

In here you can paste in a Topic and Payload. If you then click on the ”Try decoder function” button the actual payload decoder function will be executed in a simulated environment.

You can see the output of your function in the areas below the Input fields.

This simulation does not result in recording the actual returned fields and their values into the device database.

Add Uplink Decoder

If you are done setting up your first subscription and writing a corresponding payload decoder, you can finialize the steps by clicking on the ”Add uplink decoder” button at the bottom of this modal. This will create the uplink decoder and close the modal.

See Data Log

After closing the uplink decoder modal you are back on the configuration page and should be able to see the newly created decoder in the overview list.

If your device (or service) already sends data, you should see an indication of incoming data on the ”LAST DATA” column.

In the screenshot above you can see that last data has arrived ”a few seconds ago”. If you want to see the logs for this uplink decoder, simply click on the button next to the ”a few seconds ago”.

This will open up a modal that shows all incoming topics / publishes as well as the information about what fields it had returend and if it was able to route these onto a Datacake Device (and which device).

To see the measurements and device routing simply press the ”Show” Button on the Measurements Column.

Create a Downlink

So now that we have set up our first Uplink Subscription we can continue and define our first Downlink.

What is a Downlink?

Downlinks are (MQTT-)Messages that are being send from Datacake to the actual device. On the Datacake platform Downlinks do exists for all Device-Types and can be used:

  • As a Button on the Dashboard

  • As a Trigger Action inside our Rule-Engine

  • To show a Dialog with User Input

Add a new Downlink

To add your first Downlink use the Tab-Bar on a Device-View to navigate into the ”Downlinks” section. You will find a view similar to the following:

In the Downlinks-Section you see an overview of all available Downlinks. If there is none, like in this example, you can go ahead and create your first Downlink by clicking on the ”Add Downlink” Button. This will bring up a new modal, similar to the following:

Downlink Type

In order to create an MQTT Downlink you need to select the ”MQTT” Downlink-Type.

Name

Here you can define a short name for your Downlink. This is being used everywhere the Downlinks are being listed.

Description

Using the description, you can provide additional information that allows users to better understand what the actual downlink does.

Fields used

If you want to query the user for input before the downlink is being send, you can define these fields here. This can be used if your downlink does send configuration to a device and you want to query the user for a new configuration.

Even if you don’t include any fields here, you still have access to the last stored value of any field in your device database.

Trigger on Measurements

By activating this option, everytime there is new data written / recorded on one of the fields defined in ”Fields used”, the Downlink will be triggered automatically. This is very helpful for providing auto-Downlinks on Messages coming from your devices.

Only works in combination with Database Fields defined in the ”Fields used” option.

Downlink Payload Encoder

Next, let us continue with writing the actual Encoder function. This is similar to an Uplink Decoder (as it uses Javascript) but instead of decoding JSON coming from a publish to Datacake, this code can be used to define the structure of both topic and payload, that is being send to your Device.

function Encoder(device, measurements) {
    
    // Get Device Information
    var device = device.id;
    var serial = device.serial_number;
    var name = device.name;
    
    // Create JSON Payload for Publish
    var payload = {
        "message": "Hello World!",
        "info": "Sent from Datacake IoT Platform",
        "device": device,
        "name": name,
        "serial": serial,
        "temperature": 23.45,
        "status": true
    }
    
    // Return Topic and Payload
    return {
        topic: 'device/dev01234/status', // define topic
        payload: JSON.stringify(payload) // encode to string
    };
}

Working with Serials

Sending and routing Downlinks to a specific device often requires including a serial number in the topic structure of the MQTT message.

Datacake allows you to access the custom serial number for the device that is sending the downlink by accessing the device.serial_number variable inside the Downlink Encoder. You can use this to create a dynamic topic, just like the following:

function Encoder(device, measurements) {
    
    // Get Device Information
    var serial = device.serial_number;
    
    // Create JSON Payload for Publish
    var payload = { .. }
    
    // Return Topic and Payload
    
    return {
        topic: 'device/' + serial + '/status', 
        payload: JSON.stringify(payload)
    };
}

Testing the Downlink

Underneath the code editor you find a ”Try Encoder” button to test the downlink. This simulates the downlink but does not send it. You can see the output of this downlinks in the areas below the button.

Calling the Downlink

You can call a Downlink without placing a button on a Dashboard or using this in the Rule Engine. Once you have added a new Downlink to your Device (Product, as Downlinks are also defined on Product-Layer) this downlink will be added to the Table on the ”Downlink”-View.

To call a Downlink, simply press on the ”Send Downlink” button.

Configure and Send Downlink

If you have defined Fields in the ”Fields used” option, the button will say ”Configure and Send Downlink”. By pressing this button, a modal will open up, querying you for the values you want to set.

Create Dashboad

We now have everything set up on the configuration part and can continue with creating our first Dashboard.

Use the Tab-Bar on your Device to navigate into the ”Dashboard” view.

Start Editing

In order to be able to edit a Dashboard (make changes, add widgets) you need to activate the ”Edit Mode”. This can be done by clicking on the small switch on right end of the Tab-Bar (see above screenshot).

Add Downlink Button

Now that we have enabled the Edit Mode we can add our first Widget, a Downlink-Button that will call a selected Downlink.

To add a new Widget press on ”Add Widget” next to the ”Edit Mode” switch. This will bring up a widget selector. Now select the Downlink-Widget from the List.

This will bring you directly into a dialog where you need to configure the Downlink Widget.

Choose the Downlink by selecting this from a Drop-Down and add some additional settings, details like Title, Color, etc. You can see a preview of how the Widget will look like in the editor.

Once done, click on ”Save”-Button on top right.

You can now see the new added Downlink-Button on the Dashboard. If you are still in the Edit-Mode you can also redefine the Size and Position of the Downlink-Widget.

Add Value Widget

Just like the above example on how to add a Downlink Widget, you can use the same steps to create a "Value Widget" that can be used to display a numerical (or String) value.

Simply select "Value" from the widget list and configure the widget using the "Edit Value Widget" modal that will appear after you have selected the widget.

Once completed with editing you can close the modal by pressing on "Save". This will add the Widget to the Dashboard.

That's it. You have successfully created a Dashboard that has a Downlink-Button and Value Widget for your MQTT IoT Device.

Uplink Decoders

Working with Wildcards

Working with Serial Numbers

Downlinks

Create Downlink Encoder

Working with Serial Numbers

Working with Measurement Data

Working with Triggers

External MQTT Broker

Add your Broker

Adding your third-party MQTT Broker can be done in the Device-Configuration-View of your Datacake Device.

Protocol / Version

Hostname / Port

SSL / TLS Mode

Authentication

Testing Connection

Configure Broker

If you need to make changes to your Broker, you can do this by navigating into the ”Integrations” section on side-bar.

Use the Tab-Bar to navigate into the ”External MQTT” section. This will show you a list of all Brokers that are added to your workspace. If you want to make changes to one of your Brokers, simply click on the ”...”-Button at the end of the list entry.

This will bring up a context-menu. Select ”Edit Server” here.

This will open up a Modal where you can do changes to the Broker.

Remove Broker

If you want to remove a Broker, use the same context menu as mentioned above but select ”Delete Server” instead.

You will see a notification which you need to confirm.

If one or more devices are still using this Broker, you will see the affected devices in this notification as well. These devices will loose connectivity.

Show Broker Logs

Last updated

Was this helpful?