MQTT

Preliminary Considerations

The following terms and definitions apply:

Broker

Middleware that handles data communication between clients. In the MQTT context, both the Edge server and any MQTT device it connects to are considered clients.

The Edge server does not come bundled with an MQTT broker; therefore, the latter must be set up and configured separately. Three approaches exist:

1. Self-hosted, on-site
   Broker software runs on hardware ('the server') that is installed locally and connects to the Edge server via LAN only. Site owner/operator is responsible for the server uptime and maintenance. Most brokers have a web interface for client management.

   These are some of the popular and well-documented open-source brokers available as of Q1 2024:

   • HiveMQ
   • RabbitMQ (requires a plugin for MQTT support)
   • Mosquitto
   • EMQX
   • VerneMQ
   • ... and many others.

   EnergyMachines™ endorses RabbitMQ and Mosquitto and can provide a setup and configuration service on-site as part of the Edge server commissioning process; contact our support for details.

2. Self-hosted, remote
   Essentially identical to the previous approach, this setup requires an off-site server accessible via a URL. The responsibility for renting and configuring a remote host lies with the customer, although EnergyMachines™ can still provide a setup and configuration service for the broker of choice. In any case, EnergyMachines™ relinquishes and all responsibility for broker server uptime and maintenance, which lies with the customer and/or, by extension, the remote hosting provider.

   Documentation for open-source brokers often includes deployment guidelines for popular hosting platforms such as Microsoft Azure or AWS and might therefore be a good place to start for those who are still considering their options.

3. Cloud-based (SaaS)
   Third-party broker deployment and hosting is often available as software-as-a-service (SaaS) with access to the administration dashboard. Such a service can be provided by the developers/distributors of open-source solutions (e.g., HiveMQ Cloud, EMQX Cloud), by other parties deploying open-source brokers on their hardware (CloudMQTT), or as a commercial-only product.

   Note that some of those service providers also offer MQTT broker management: a service where the customer provides a hosting platform but delegates broker configuration, deployment, and maintenance to a third party.

While there are no functional restrictions with regard to the approach, EnergyMachines™ recommends method #1 above to avoid potential security complications and/or issues due to excessive data traffic.

Topic

MQTT is a publish-subscribe protocol:

• A client is said to publish data if it sends messagesMessage
  Any complete packet of data that has a timestamp and might carry additional service flags, e.g., for Quality of Service purposes. to the broker. In the context of the Cloud, devices that provide data, such as temperature sensors, meters, gauges, etc., are considered publishing clients.
• A client is said to subscribe to data if it connects to the broker to read messages published there. In the context of the Cloud, the Edge server is considered the subscribing client.

In case of writable tags, the roles are reversed: the Edge server becomes a publishing client whereas the device containing a writable tag acts as a subscribing client; see details below.

In order for subscribing clients to be able to selectively receive only the required data from the broker, all messages are assigned a topic.

The topic of a message is a UTF-8 string that represents an abstract “path” to the message in the brokered message hierarchy:

This simplified example is a four-level topic, with the uppermost level, Cph, representing the city of Copenhagen, and the lowermost levels, humidity, representing a humidity sensor.

Cph/Vesterbro/Office/humidity

Similarly, the subscribing client provides the broker with a topic, which in its case would be the path to the messages it wants to listen to. Thus, when the broker receives a new message from a publishing client, all clients subscribed to its specific topic are each forwarded a copy of that message.

Note that topic levels do not have to represent geographical locations, device positions, roles, or any other particular features; topic hierarchy can be defined in any manner. In any case, the following principles apply:

• A publishing client can push messages to multiple topics.
• A subscribing client can receive messages from multiple topics.
• A topic can carry multiple subscribers but only one publisher. Handling of topic conflictsSituations where different publishing clients attempt to write to the same topic varies from broker to broker.
• Forward-slash (/) separates the topic string into segments, each representing a topic level; brokers create new levels or assign topics to existing levels automatically.

Avoid leading and trailing slashes in topic strings.

Device Configuration

MQTT implementation in the Cloud uses JSON-formatted messages; connected MQTT devices can use the following topics and formats to publish their readings:

• some/data/endpoint:

  {
    "read": V
  }
  

• some/data/endpoint/read:

  {
    "v": V
  }
  

  where V is the reading the device needs to communicate to the Edge server and thus to the Cloud.

In order to support writability, the device must subscribe to some/data/endpoint/write, where the Cloud posts values manually written by the user in the HMI, e.g., when using Tag Lookup. Written values are also received in the JSON format:

  {
    "v": V
  }

A single data endpoint corresponds to a single topic. It is recommended that the topics for all data endpoints in a device only differ in the lowermost level (the rightmost segment). In case one device carries multiple data endpoints, assign each to a subtopic. Thus, if a device has multiple independent sensors and/or an independent writeable parameter, each of those is a subtopic within the topic of that device.

The following subsections are written with an assumption that the site already has a functional broker in any form, and the device has been configured as described above.

Device Parameters

For device parameters not covered herein, see Device-Based Tags.

Address

Topic the device publishes to, exclusive of the lowermost level (the rightmost segment); in case the device has multiple data endpoints, exclude the subtopic; see also address (tag).

Broker

For a detailed explanation of what a broker is, see Broker.

• URL
  URL of the broker, formatted as follows:

  `${protocol}://${url}:${port}/${basepath}`
  

  The Cloud’s MQTT implementation supports websocket, MQTT-over-TCP/IP, and SSL connections; TLS is not supported.

• User name, Password
  Credentials for authenticating with the broker.
• Retained
  If checked, allows the broker to store the message and post it to new subscribers when pushing a written value to a tag,. For a detailed explanation, see What are Retained Messages in MQTT?
• Quality of service
  Specifies how many times to attempt to deliver the message to subscribers when pushing a written value to a tag:
  • At most once, or QoS 0 — send the message once and do not verify its receipt. Written value pushed to a tag might be lost due to connectivity issues or for other reasons.
  • At least once, or QoS 1 — keep the message until the recipient confirms receipt; resend if timed out.
  • Exactly once, or QoS 2 — keep the message until the recipient confirms receipt, then send a release packet related to the same message; the recipient will keep the initial message until it receives a release packet, after which the stored messages will be discarded, and the original sender will be sent a completion packet. The critical different between QoS 1 and QoS 2 is that the latter is intended to prevent packet duplication. For a detailed explanation, see What is MQTT Quality of Service (QoS) 0, 1 & 2?

Retention and QoS parameters do not affect the reading of tag values.

Encryption

Each message published to, or received from, an MQTT-brokered topic consists of two parts:

• timestamp (Unix time)
• payload (actual data, e.g., in JSON format)
• service flags (e.g., for retention purposes, see Broker above).

The broker pushes the payload to subscribing clients as is, i.e., without processing. For security reasons, the payload can be encrypted:

• Enable encryption
  Check to encrypt two-way communication between the Edge server and the target device; the latter must support at least one of the two encryption algorithms listed below.
• Algorithm
  Specifies the algorithm to use for payload encryption:
  • Advanced Encryption Standard (AES) or
  • Data Encryption Standard (DES)
• Key, Initialization vector
  Encryption parameters; should be identical with those used by the target device. Both can be either copied and pasted from the target device’s configuration or generated in this interface to be copied and pasted into such configuration.

The above concerns payload encryption only and does not apply end-to-end encryption; the latter requires using an encrypted protocol (e.g., SSL) that is explicitly supported by the broker.

Health Check

• Enable health check
  Check to cause the Edge server to continuously probe the connection to the broker.
• Period
  Specifies how frequently the broker connection is probed.
• Timeout (ms)
  Specifies the maximum broker response waiting time before raising a high-priority alarm: The device communication error

Miscellaneous

• Confirm receiving data
  Check to send acknowledgment to the broker on successful message receipt; required for QoS 1 / QoS 2 to function properly if set on the target device.
• Enabled
  Check to activate the connection; broker connection will not be attempted unless checked.

Tag Parameters

For tag parameters not covered herein, see Tag Editor.

Address

Data endpoint-specific subtopic. The device must be configured to publish either a "read"-keyed value to this subtopic or a "v"-keyed value to subtopic/read, where subtopic is this subtopic; see also Device Configuration.

Unlike other device-based tags, MQTT does not require a device configured in the Cloud to actually correspond to a single unit of physical hardware. Multiple physical units can be represented as a single device in the Cloud; conversely, a single unit can send/receive data from multiple “devices” configured in the Cloud.

When configuring an MQTT device and tag in the Cloud, the following topic segmentation applies:

`${device_topic}/${tag_topic}`
// OR
`${topic}/${subtopic}`
// These represent the segmentation of the same string in different terms; the latter is what used throughout the article to denote the different segments (topic levels).

Both the device_topic (or simply topic) and the tag_topic (or subtopic) can in fact be segmented into multiple topics. This effectively allows differently configured devices and tags to subscribe to the same data endpoint. For example:

  Device_address: "Cph/Amager/Office"
  Tag_address: "humidity"

and

  Device_address: "Cph/Amager"
  Tag_address: "Office/humidity"

will both read/write the same data endpoint.

For simplicity, the general recommendation is to reserve only the last segment of the topic string for tags, i.e., to use single-level subtopics. However, a different configuration might be needed if, for example, different data endpoints of the same device require different brokerage parameters such as QoS, in which case multiple devices must be created to represent the same unit of hardware.

Writable

See also Tag Editor » Writable.

Check to allow overwriting the tag’s values. Unlike with other Device-Based Tags, writing to an MQTT tag does not result in overwriting values at specific addresses in the controller’s memory. Instead, the Edge server will publish a message formatted as:

{
  "v": V
}

to

`${topic}/${subtopic}/write`

The logic that handles such writeable values has to be configured separately on the target device, see its manual for specific instructions. Note as well that in order for the Cloud to display such manually (over)written values, the device must publish them to the same subtopic as described in Device Configuration.