Understanding Internal vs. External MQTT Brokers on Datacake

When to use internal vs. external MQTT brokers on Datacake. What are the pro and cons?

Introduction

Datacake offers two primary MQTT broker options for users to manage and ingest data: the internal MQTT broker (mqtt.datacake.co) and external MQTT brokers through a third-party MQTT bridge. This document details the features, limitations, and usage of both options to help users select the appropriate broker based on their needs.

Internal MQTT Broker

Overview

The internal MQTT broker is tightly integrated with the Datacake backend and is primarily used for real-time communication between the backend and dashboards. It allows direct data ingestion into Datacake's database and supports user authentication.

Features

• Real-time Communication: Enables real-time updates between Datacake's backend and dashboards.

• Direct Database Integration: Ingests data directly into Datacake's database fields.

• Fixed Topic Structure: Utilizes a fixed topic structure for consistency and security.

• User Authentication: Requires user authentication for publishing and subscribing.

Topic Structure

Subscribe

To receive messages via MQTT when readings arrive in the Datacake Cloud, subscribe using the following structure:

dtck/<product_slug>/<device_id>/<field_name>

Messages are published whenever there is a change to the corresponding database field.

Publish

To upload data into the Datacake Cloud and a specific device, publish to:

dtck-pub/<product_slug>/<device_id>/<field_name>

Note: Always use the prefix dtck-pub/ to import measured values.

Topic Elements

• Product-Slug (<product_slug>): Specifies the product path. This is useful for managing multiple devices of the same product type. The specification can be found in the metadata section of the device view.

• Device-ID (<device_id>): Identifies a specific device. This ID is a unique UUID and can be found in the device metadata or URL.

• Field-Name (<field_name>): The field name of the measured value as it appears in the database. Use uppercase for consistency.

Examples

Subscription

Subscribe to all measurements of a specific device:

dtck/my-product/6b98a3bb-9ae1-418f-9375-f23091a849cd/+

Subscribe to all temperature measurements of all devices under a product:

dtck/my-product/+/TEMPERATURE

Publish

Publish a temperature measurement to a specific device:

publish("dtck-pub/my-product/6b98a3bb-9ae1-418f-9375-f23091a849cd/TEMPERATURE", 22.003)

Use Case

The internal broker is ideal for users needing fast, secure, and real-time data ingestion directly into the Datacake database, with the limitation of a fixed topic structure and single measurement messages.

More Information

For detailed setup instructions, refer to the Datacake Internal MQTT Broker Documentation.

External MQTT Brokers

Overview

The external MQTT brokers provide more flexibility for users needing custom topics and the ability to send multiple measurements in a single message.

Features

• Custom Topics: Users can define their own topics.

• Multiple Measurements: Supports sending multiple measurements in a single message, including JSON format.

• Payload Decoders: Utilizes payload decoders for processing subscriptions from external brokers.

• Third-Party Integration: Integrates with various third-party MQTT brokers.

Use Case

The external broker is suitable for users requiring flexible topic structures, complex payloads, or integration with third-party systems.

More Information

For detailed setup instructions, refer to the Datacake External MQTT Broker Documentation.

Summary

Feature	Internal MQTT Broker	External MQTT Brokers
Real-time Communication	Yes	Depends on third-party broker
Direct Database Integration	Yes	No
Fixed Topic Structure	Yes	No
Custom Topics	No	Yes
Multiple Measurements	No	Yes
Payload Decoders	No	Yes
User Authentication Required	Yes	Depends on third-party broker
Performance	Fastest	May vary
Use Case	Direct database ingestion, real-time status updates	Flexible topic structure, complex payloads, third-party integration

This guide should assist users in understanding the differences between the internal and external MQTT brokers provided by Datacake, helping to choose the right broker based on specific needs.