How MQTT Fits Into Industrial Architectures

In a recent episode of the Augmented Ops podcast, we explored the landscape of industrial IoT architectures and the role that MQTT plays with Dominik Obermaier, Co-Founder and CTO of HiveMQ. Titled "Building Industrial Architectures with MQTT," the discussion with Obermaier examines the role of MQTT, how it supports new architectures like Unified Namespace (UNS), and why many manufacturers are rearchitecting their digital infrastructure to take advantage of these new technologies and approaches.

From founding HiveMQ fresh out of university, to serving on the MQTT technical committee for over a decade and helping develop the standard, Obermaier provides a nuanced perspective on this technology and its application in manufacturing. In particular, he calls on manufacturers to adopt an MQTT to build industrial infrastructures at enterprise scale.

Obermaier detailed the history of MQTT, explaining how it was first developed in 1999 by IBM for an oil and gas use case before being open sourced in 2010, which allowed anyone to implement the specification without having to fork over royalty fees. This change provided the initial momentum for MQTT to proliferate, first seeing use in home automation setups, before becoming a preferred communication protocol for everything from connected cars, to industrial and manufacturing use cases. But what exactly sets MQTT apart from the myriad of other contemporary protocols like HTTP or OPC UA?

First and foremost, MQTT follows the publish/subscribe (often shortened to Pub/Sub) communication pattern. This means that any node in an MQTT infrastructure can publish data to a specific address (known as a topic) in a central broker, and can also subscribe to data being published to the broker from other nodes. The result is that when a message is published, it is automatically received by all subscribers — as opposed to protocols that involve continuously polling a client at regular intervals to request updated data. As Obermaier explains, “This means you are decoupling producers of data from consumers of data.” MQTT clients are also typically designed to ‘report-by-exception’, which means that they only publish a message to the broker when data values actually change.

This ties into another hallmark of MQTT — its lightweight design, which makes it particularly suitable for the constrained networking environments often found in industrial settings. Unlike more verbose protocols that require substantial packet overhead, MQTT uses a binary format for communication between clients and brokers, with a fixed header of just 2 bytes, along with a variable header of at most 12 bytes. In conjunction with report-by-exception, this results in a significant reduction of network traffic, making MQTT ideal for large scale applications that require high throughput.

The key to MQTT’s proliferation is its simplicity. The specification was deliberately designed to “make it extremely easy on the clients to implement,” says Obermaier. Thanks to it being made openly available by the MQTT standards body, OASIS, anyone can implement the specification from scratch using the programming language of their choice. On top of this, there exist a number of open source MQTT libraries in various languages that make it even simpler for developers to use MQTT in their projects.

While the simplicity and flexibility of the MQTT specification provide a number of advantages over other protocols, it also presents a number of challenges. For instance, Obermaier explains that “the protocol does not care” about the structure of the data being sent — which can be anything from text, to images, and any other data that can be turned into a binary, up to a whopping 256 megabytes. While flexibility when it comes to payload and many details of the implementation makes the spec highly versatile, it also makes it more complicated to ensure interoperability with various systems that may each be implementing MQTT in different ways.

Simply specifying MQTT as the desired data protocol is not enough to guarantee seamless interoperability between two systems, since it does not specify how the data is encoded or what the payload actually contains. And while this makes the spec straightforward for producers of data to implement, this places a greater burden on the systems that consume it to actually make use of that data. Obermaier points to this as the reason why frameworks like Sparkplug B have begun gaining in popularity, since they standardize numerous aspects of MQTT to make interoperability much easier.

Sparkplug B is an open-source framework that enables seamless integration of data from various sources in a bi-directional and interoperable way by defining consistent patterns for MQTT topic structure, payload, state management, and more. Obermaier explains that by setting a standard for “how payload should look like, how MQTT topic structure should look like, and so on,” frameworks like Sparkplug have made it easier to achieve interoperability without having to worry about the details of how the MQTT spec is implemented by disparate systems.

One kind of architecture that MQTT lends itself particularly well to implementing is the Unified Namespace (UNS) concept, which has been gaining traction across the industry in recent years. Obermaier highlights how traditional technologies “work with point-to-point protocols, or sometimes bus protocols that are not really designed for end-to-end data movement across an organization,” which results in “a lot of data silos.” With a UNS, however, there is a “central hub accessible to OT folks and IT folks and to applications,” he says. This eliminates the need for point-to-point connections between nodes, allowing anyone to access and query the data in the namespace.

But why use MQTT? The minimum technical requirements for a UNS are that it is edge-driven (meaning nodes at the edge report to the central hub), is report-by-exception, is lightweight, and is built on an open architecture. MQTT fits the bill for all of those, thanks to its use of a publish/subscribe communication pattern, its ability to report-by-exception, its low bandwidth usage, and the fact that the specification is open and can be implemented free of charge by anyone. Although there is a strong synergy between MQTT and UNS, Obermaier stresses that “Unified Namespace, to make it very clear, it's a concept, it's not a technology.” And while “Sparkplug and MQTT are technologies, UNS is a concept” that can be implemented with a number of different messaging technologies.