Data exchange is one of the huge challenges in Internet of Things (IoT) with billions of heterogeneous devices already connected and many more to come in the future. Improving data transfer efficiency, scalability, and survivability in the fragile network environment and constrained resources in IoT systems is always a fundamental issues. In this paper, we present a novel message routing algorithm that optimizes IoT data transfers in a resource constrained and fragile network environment in publish-subscribe model. The proposed algorithm can adapt the dynamical network topology of continuously changing IoT devices with the rerouting method. We also present a rerouting algorithm in Message Queuing Telemetry Transport (MQTT) to take over the topic-based session flows with a controller when a broker crashed down. Data can still be communicated by another broker with rerouting mechanism. Higher availability in IoT can be achieved with our proposed model. Through demonstrated efficiency of our algorithms about message routing and dynamically adapting the continually changing device and network topology, IoT systems can gain scalability and survivability. We have evaluated our algorithms with open source Eclipse Mosquitto. With the extensive experiments and simulations performed in Mosquitto, the results show that our algorithms perform optimally. The proposed algorithms can be widely used in IoT systems with publish-subscribe model. Furthermore, the algorithms can also be adopted in other protocols such as Constrained Application Protocol (CoAP).
The Internet of Things (IoT) has emerged as the next evolution of the technology which covers a wide variety of devices and system platforms, such as embedded systems, networked sensors, actuators, and smart home devices. Inevitably, the resource starved nature in most IoT devices makes it error-prone to efficiently manage and maintain a reliable IoT system, not to mention the security issues associated with some scarcely attended IoT devices [
Trends comparison of MQTT, CoAP, and AMQP.
The history of MQTT dates back to 1999, which was invented by IBM. MQTT uses the so-called “publish/subscription” model to transmit data among the IoT things. MQTT is more mature and stable than CoAP and AMQP. Due to the low-cost and power-constrained nature of most IoT devices, the communication efficiency is extremely important. For example, if we can improve the power efficiency, the IoT device can be operated for even longer time period than low power efficiency without having to replace batteries. This is one of the fundamental motivations for our focus on IoT data communication protocol efficiency in this paper. An IoT system consists of a large number of internet-connected resource-constrained and dynamic nature devices [ Development of a topic routing algorithm to communicate data between IoT devices Development, implementation, and evaluation of a Broker Network to improve the survivability and robustness of connected-device data exchange in IoT
We evaluate our algorithms with MQTT protocol implementation Mosquitto [
Devices in IoT system often have limited processing capabilities, low bandwidth, and small memory capacities and are powered by batteries with limited working hours. Therefore, improving the usage efficiency of the network and reducing power usage in IoT scenarios are extremely important. As a promising data communication protocol in IoT, MQTT is a machine-to-machine (M2M), lightweight publish-subscribe messaging transport protocol. In publish-subscribe model, subscriber registers its interest topics to a broker and publisher sends data to a broker [
If this information is configured manually, while not the automatically, it is a huge challenge to manage thousands of nodes. When the network structure is changed, the configurations need to be done. Therefore, automatic discovering and configuring the IoT network are a huge challenge. And agriculture processing system scenario has the following features: data from farm in fields to table on Internet, field data spanning up to one year, wide geographical areas of crops, and fragile network conditions.
As noted earlier, MQTT used in IoT system emphasizes data exchange, not mainly for data integration. The question arises, why in the agriculture process system we need data exchange? There are several reasons to make it necessary for data exchange in agriculture processing system and data exchange about storm, hurricanes, cyclones bad weather. It is known to all that storm may influence several hundreds of miles. It is very important to spread related information about agriculture. In fact, weather conditions have a heavy influence on the crop management. Data exchanges among different farms are very important. For example, in Pennsylvania, there may have been dozens of farms located in a region where pests and diseases spread very quickly. Similarly, farms in remote and regional areas in Australia are spread over on thousands of acres. If the information around one regain can exchange on time, precautions can be taken to prevent or reduce loss from any natural disaster which may otherwise affect drastically the agriculture processing system.
The IoT things are spread around hundreds of miles in field or plain. According to the mechanism of MQTT protocol, data exchanges among different nodes and brokers should be configured according to the IP addresses and topic names. Only after manual configuration or by configuration files, the data can be exchanged along specific path to another node, which gets damaged crushes down, we will not know until proactive testing is performed. Neighboring nodes might have information about the damaged nodes but the important question here is how to dynamically figure out which nodes have the corresponding data and how to arrive at that node with the shortest path automatically. In our proposed algorithms, we have focused on this problem. On the contrary, if the reconfiguration is done manually or through CLI scripts, data exchange with MQTT in IoT is not reliable and is not easy to recover. Maintenance and management of data exchange in this model is a hard task which can be costly. Therefore, a reliable, automatic data discovery and data exchange model with a shortest path in a fragile, dynamic network environment is very important. With our algorithms, we can find the specific nodes for specific data with one topic and get the shortest path to that nodes dynamically and automatically. We need a smarter recovery program, which can save money (automatically) to avoid remote login and can perform recovery efficiently. Generally speaking, through the broadcast method and the way of active subscription, we can achieve data exchange automation. The existing practices often involve manual configuration where messages are read or processed manually. In our proposed model, we aim for automatic data exchange.
Data collection is one of the key functions in IoT system. However, battery powered devices in IoT often work in low bandwidth and fragile network environments. One of the major challenges is to provide efficient, stable, and scalable services in IoT. For example, to extend a device’s effective working life-span as much as possible, efficiency should be considered to save power. At the same time, to ensure the availability and scalability of data communication services, broker network is a natural choice in IoT system [
When servers in the cloud system gather data from things as messages in IoT system, we will route the messages flow through broker network in IoT to cloud. For example, client 1 subscribes to broker 1 with topic “top” and client 2 publishes to broker 2 with topic “top.” Broker 2 should route the message published by client 2 to broker 1, so that client 1 can get the corresponding data. This is the so-called topic-based message routing problem in IoT. The key point here is the efficiency when broker nodes synchronize data among brokers, which include the following: Forwarding all messages to the other bridges Routing of messages to the brokers
For the CPU, bandwidth, and memory constrained devices, the routing decision algorithm should be simple enough to save resource. Normally, we can divide the nodes into two types: one is routing node and the other is terminal subscriber node or publisher node. Our data routing decision algorithm is depicted in Algorithm
When a broker receives data with topics from one client, the broker matches topics in its topic tree. If the node is just the terminal subscriber, then broker forwards it to that node, and then go to 5. If the node is not the terminal consumer of the topic, go to 4. If the node is not the terminal consumer of the topic, the data will be forwarded to the other broker nodes, which is the next hop of the routing. Continuing with 3, with next hop, data eventually will be sent to cloud or terminal nodes. The messages are sent with the shortest path to the final destination. The topic may be sent to many different receivers who subscribe the topics. All the receivers will receive the same messages. The broker will decide whether to store the topic or not.
In summary, there are three scenarios: One to one directly One to many directly One to remote nodes through routers
In one to remote nodes through routers scenario, the cloud servers gather data from devices as messages transmit through the local IoT network. We address the issue of collecting specific data and how to transfer it efficiently in our algorithm through finding the shortest path based on topics for message delivery to cloud or server. That is to say, a routing protocol is considered. Our message routing decision algorithm is depicted in Figure
Message routing decision algorithm IoT network.
In Figure Figuring out the shortest path between router ⑥ and router ①, it is a single-source shortest path topic-based problem in graph theory. We can address it, for example, with Dijkstra’s algorithm. Here, the shortest path is router serial ⑥ In reverse sequence, router ① subscribes topics from router ②, router ② subscribes topics from router ④, and router ④ subscribe topics from router ⑥. When sensors publish topics to router ⑥, the messages will be pushed back to hop by hop until router ①.
When the topology structure of router network changes, we can figure out the new shortest path from the source to the end as described in Step (1) dynamically. The “routing tables” are critical to the system because IoT network may not be using the traditional TCP/IP protocol stack which makes real-time data transmission with the shortest path without considering the network protocol even more challenging. That is to say, regardless of TCP/IP or Zigbee, the IoT system can still find the data source and the shortest path transparently based on topics. In the next section, we will describe this algorithm in detail.
In order to facilitate problem description, first we provide some relevant definitions. Then algorithm is depicted in detail with an example. We present efficiency evaluation of our algorithm in Section
Routing table structure: in order to make nodes find each other and in the shortest path, some information should be stored in every nodes. Table
Structure of the routing table.
Node | Path and topics | Weight |
---|---|---|
Subscribing node | Subscribing node queue | Number of nodes in queue |
Subscribing node: node 1 subscribes to node 2; node 1 is the so-called subscribing node.
Subscribing node queue: A subscribes to B, B subscribes to C, C subscribes to D, and “ABC” is the so-called subscribing node queue.
Weight (T): the number of node in a string.
Host_Topic (T): the ultimate data consumer and corresponding topic. For example, “Host_Topic (ABCD/TOP)” = “D/TOP”.
Searched_Set: a node set, in which every node has finished building the routing table.
NotSearched_Set: a node set, in which every node has not finished building the routing table.
Adjacent (X): a set of adjacency of node X.
Match (top): determining whether the node can provide the corresponding topic data.
The algorithm essentially completes the following tasks. First, the algorithm determines the node where the topic and corresponding data reside. Second, the algorithm dynamically finds the shortest path. Then the data can be transported to Internet. That is to say, the algorithm solves that which node has the topic and data and how the data can be transported to the ultimate user. We solve this problem with our new routing algorithm named Topic-Based Routing Algorithm (TBRG). The rerouting algorithm is given as follows. In the algorithm, “
Next, we elaborate our algorithm through a concrete example illustrated in Figure ① Initialization of node Ⓐ in Table ② A subscribing to nodes Ⓑ, Ⓓ, and Ⓔ, that is, Ⓐ ③ Next, B and E subscribing to node Ⓒ and node Ⓕ, respectively, that is, node Ⓑ ④ Next, node Ⓒ and node Ⓕ subscribing to nodes Ⓘ and Ⓖ, respectively, that is, node Ⓒ ⑤ Node Ⓖ subscribing to Node I, that is, node Ⓖ
Example of topic routing protocol.
Routing table of node A.
Node | Path and topics | Weight |
---|---|---|
0 | 0/top | 0 |
Routing table of node B.
Node | Path and topics | Weight |
---|---|---|
A | A/top | 1 |
Routing table of node D.
Node | Path and topics | Weight |
---|---|---|
A | A/top | 1 |
Routing table of node E.
Node | Path and topics | Weight |
---|---|---|
A | A/top | 1 |
Routing table of node C.
Node | Path and topics | Weight |
---|---|---|
B | BA/top | 2 |
Routing table of node F.
Node | Path and topics | Weight |
---|---|---|
E | EA/top | 2 |
Routing table of node of G.
Node | Path and topics | Weight |
---|---|---|
F | FEA/top | 3 |
Routing table of node I.
Node | Path and topics | Weight |
---|---|---|
C | CBA/top | 3 |
Possible routing table of node
Node | Path and topics | Weight |
---|---|---|
G | GFEA/top | 4 |
Figure
Routing tables in message router.
Each broker may serve for thousands of clients’ connections. In case one broker crashes down, clients should have the ability to connect to another broker and work continually and smoothly, which is called the rerouting problem. To address this problem, we propose a rerouting algorithm to take over the topic-based session flows with a controller when a broker crashes down. Although it is almost impossible to communicate when broker crashes down, data can still be communicated by another broker with rerouting mechanism. Therefore, high availability can be gained with this model in IoT, as shown in Figure
Fault migrating.
As in Section
Topic_Session: when node
Normally, we can use process expression to depict communication behavior. Actually, from the perspective of mathematics, this is a binary partial order relation. That is to say,
Topic_Session.
If broker A crashes down, we want to find another broker B to take over the topic session flows in broker A, in which brokers A and B at least have the Strong Behavior Equivalence relation A
Wireless network broadcast.
In normal wireless network, if one node loses its connection, the node may update its status and broadcast its identification information to all neighbor nodes, just as shown in Figure
The IoT network node is different from the normal wireless network node as follows. Node In MQTT, one broker is specified when nodes publish or subscribe to a broker. That is to say, the subscriber or publisher cannot automatically reconfigure the broker. For example, MQTT protocol with QoS 0 and the node even are not aware when the broker crashes down.
The problem here is that nodes and brokers have an equal status. So nodes almost know nothing about each other. Because of mutual independence to each other, one node cannot change another node’s behavior.
To sum up, when node What conditions should be met when a broker substitutes one or a few brokers to take over that broker? Who will we find a crashed down broker? How can we take measures on the nodes or brokers to take over that broker? Where should we store the relation
According to process behavior equivalence, if brokers A and B have a weak behavior equivalence relation
The process expression of broker A is
The notation
Essentially, Strong Behavior Equivalence
As noted in the previous section, publishers are not aware if a broker crashes down. Even the publisher knows the absence of broker, the topic-based session cannot go on without outside help, not to mention finding one or a few brokers to take over the shutdown broker. Therefore, the extra controller is necessary to ensure the survivability of IoT system, just as Software Defined Networks (SDN) do [ Reviving of dead broker through heart beating Figuring out the substitute node Transferring the topic-based session flow from crash down broker to alternative broker
IoT controller.
But the controller will never be used as data communication. Its main usage is to manage the nodes in IoT network.
Through heart beating, for example, subscribing “hello world” to managed node on a regular timer with Quality of Service (QoS), it is easy to get the information in broker node in order to determine a substitute node. To figure out substitute node, the controller should collect the following information in regular time. For example, as shown in Table
Topic session flow.
No. | Things | Broker | Sub/Pub | Topic | On/off |
---|---|---|---|---|---|
1 | 0 | 1 | |||
2 | 1 | 0 | |||
3 | 1 | 1 | |||
4 | 1 | 1 | |||
5 | 0 | 1 | |||
6 | 1 | 0 | |||
7 | 0 | 11 |
In Table
Through the content in Table
This comes to the conclusion
The Algorithm
We implement a working prototype of the above algorithm using C and MQTT on Ubuntu system with Mosquitto and its client libmosquitto.lib as shown in Figure
In our evaluation system, Figure
Evaluation system.
Intuitively, in resource constrained device, the efficiency will decrease dramatically with data scale increasing. The trend of data transmission with data scale is one of the experimental contents of in this section. Especially to the specific protocol, the data transmission laws are more worthy of attention. In this section, we will evaluate the average transmission time with increased data scale. In Figure
Transmission time with increased data scale.
In broker network, a huge number of topics will be transmitted through the nodes. However, according to the above experiments, we know that the efficiency will decrease dramatically with increasing topics transmitted. Therefore, reducing the number of transmission nodes can reduce propagation delay time, and improve the transmission efficiency. This is just our primary goal in this paper. As the evaluation system shows with our algorithms proposed in this paper we can reduce the number of transmission nodes, which can improve the data transmission efficiency. At the same time, according to the experiment results, whole system efficiency will be improved as evident from the experiment results in the next section.
As Figure #!/bin/sh while true; do echo hello. sleep 1 done | mosquitto_pub −l −
Data transmission without the shortest path.
The other nodes can get the data through Mosquitto_sub with a counter. The counter is used to control the data scale. In the program, except subscription and counter, we calculate the time of data transmission. Through a series of subscription behaviors, the contents with corresponding topics can be passed to the final data consumer indirectly node by node and step by step. Each node automatically becomes a data generator pump. Just as described above, we have built an IoT simulation network with libmosquitto library and virtual machines as shown Figure
Just as described above, our algorithm can determine nodes where the topic and corresponding data reside, and it dynamically adopts the shortest paths between nodes in data consumer and edge routers. The shortest paths mean fewer nodes needed for data transmission. Fewer transmission nodes can save data transmission time. We use the network topology shown in Figure
Data transmission with the shortest path.
To get the influence of the data transmission path with the data scale, we obtain the evaluation results with different workload to test the algorithm. The primary performance metrics are data transmission completion time. We get the data from 500 to 10000 items with 500 as a step. We run a Mosquitto on each node in transmission network as a broker. At the same time, we develop a program based on libmosquitto as a client to get the data and calculate the transmission completion time on each node. Figure
Comparison of data transmission completion time to the data consumer.
Comparison of the overall performance on two path.
We compare the data transmission with and without the shortest path in Figure
Comparison of data transmission completion time to the edge router nodes.
In order to further verify this view, we compute the average transmission completion time of all the nodes in the path shown in Figure
We have presented novel topics-based data routing protocols and algorithms in MQTT. The routing algorithms optimize IoT data transfers with constrained resource and fragile network environments in publish-subscribe model. Our proposed algorithms can adapt the dynamical network topology and are capable of adopting continuously changing devices with rerouting method in IoT system. Through experimental results, we have demonstrated efficiency of our algorithms for message routing, and dynamically adapting the continually changing device and network topology. We have evaluated our algorithms with a prototype with a Mosquitto based on MQTT protocol. With the experiments performed and simulations on Mosquitto, the results show that our algorithms work well. The proposed algorithms can be widely used in IoT system with publish-subscribe model. Moreover, the algorithms have the capability to be tested in other protocols such as CoAP [
From security view, identity and authorization of nodes are critical problems avoiding data to be published or subscribed by hacker or crack down the broker in IoT. In this paper, we focus on the data exchange efficiency in IoT, mainly under the MQTT protocol and not involving the security problem, which we will deeply research in future work.
Any data can be obtained from the dropbox url (
The authors declare that they have no conflicts of interest.
The authors thank Professor Zia Tanveer for his kind review and help. This work was supported in part by National Science Foundation of China under grant nos. 61672104, 61170209, 61702570, and 61602537 and U1509214, Program for New Century Excellent Talents in University under grant no. NCET-13-0676, and Shenzhen Institute of Artificial Intelligence and Robotics for Society.