In-Network Data Processing in Software-Defined IoT with a Programmable Data Plane

Making an SDN data plane flexible enough to satisfy the various requirements of heterogeneous IoTapplications is very desirable in terms of software-defined IoT (SD-IoT) networking. Network devices with a programmable data plane provide an ability to dynamically add new packetand data-processing procedures to IoT applications. /e previously proposed solutions for the addition of the programmability feature to the SDN data plane provide extensibility for the packet-forwarding operations of new protocols, but IoTapplications need a more flexible programmability for in-network data-processing operations (e.g., the sensingdata aggregation from thousands of sensor nodes). Moreover, some IoTmodels such as OMGDDS, oneM2M, and Eclipse SCADA use the publish-subscribe model that is difficult to represent using the operations of the existing message-centric data-plane models. We introduce a new in-network data-processing scheme for the SD-IoT data plane that defines an event-driven dataprocessing model that can express a variety of in-network data-processing cases in the SD-IoT environment. Also, the proposed model comprises a language for the programming of the data-processing procedures, while a flexible data-plane structure that can install and execute the programs at runtime is additionally presented. We demonstrate the flexibility of the proposed scheme by using sample programs in a number of example SD-IoT cases.


Introduction
In recent years, the software-defined Internet of ings (IoT), or SD-IoT, has become one of the main topics of IoTrelated researches.It deals with several issues for the deployment and management of numerous IoT-infrastructure nodes by adopting the idea of software-defined networking (SDN) for which a centralized controller is employed, whereby flexible control and management capabilities are realized regarding the heterogeneous IoT-infrastructure nodes such as sensors, sensor gateways, network devices, and cloud servers.In the SD-IoT, the IoT applications in the cloud servers collect data from the sensor platforms that are then connected through the SD-IoT network.
e SD-IoT network basically provides a data-transferal service that is based on a message-centric model.In this model, the unit of information exchange is the message itself.
e role of the network infrastructure is the ensuring of the delivery of the messages to their intended recipients, irrespective of the message contents.e focus of the current SDN research is this model, which can be applied for the data transferal of the SD-IoT network.
A number of the nodes of the SD-IoT network, however, are required for the implementation of the data processing in addition to the data-transferal functionalities of this network; here, sensor gateways for sensor platforms (Figure 1) or vehicular ad hoc network (VANET) nodes are typical examples.e overall capability of these nodes means they can process the collected data for the transmission of useful information to their servers, instead of sending whole raw sensing data.Most of the IoT applications use the datacentric publish-subscribe (DCPS) model for the nodal information exchange with the cloud servers.In this model, the publisher node supplies data whenever data are available to the remote subscribers that are interested in the data.e publishers and the subscribers interact with each other using the standard interface.e subscriptions should be processed on the control/management plane, and upon their generation, the data are sent via the data plane.
To provide the IoT application that uses the DCPS model information-exchange method with the current messagecentric SDN, the data-processing module must be located at the SDN controller; this is because the current SDN data plane is not su ciently exible to implement the dataprocessing module for the DCPS model.At present, these cases cannot be processed in the SD-IoT network as a result, so a separate middleware-application server emulates them.In [1], a Data Distribution Service (DDS) middleware that provides a DCPS abstraction to the applications is located over the SDN controller and is used by the IoT applications.While it presents a simple solution, its compatibility with the SDN concept is not precise because the data processing is addressed over the SDN control plane.According to the SDN concept, the data processing should occur on the data plane of each network node, and the control plane inserts the processing rules into the nodes.
For an incorporation of the two communication models that is consistent with the SDN concept, the SDN data plane should be upgraded to handle in-network data processing.
e focus of the current message-centric SDN data plane is the matching of the incoming packets with the ow table entries and the forwarding of these packets; therefore, the table-based processing architecture presents a natural corresponding solution.e requisite complex operations of the in-network data processing, however, cannot be satis ed by this architecture.
e Event-driven Instruction-Based Packet Processing (E-IPP) scheme is presented in this paper as the SD-IoT data plane for the enhancement of the corresponding programmability.
e event-driven processing model of the proposed scheme supports various prede ned or userde ned events, so its extensibility is regarding programs that can be run on various events as well as the packet events.
e scheme includes the event model, the E-IPP language, and the E-IPP Virtual Machine (E-IPP VM) structure.e language is used to program the application-speci c procedure for the IoT data.e E-IPP VM is the data plane for the program that has been written in the E-IPP language, which is versatile enough to incorporate the DCPS model as well as the message-centric model; furthermore, it supports dynamic program loading at run-time.

SDN in the IoT.
A number of SDN-adoption studies that are regarding IoT networking have been completed.Subsequently, a segment of researchers favor the SDN-based control of every IoT feature as well as the network resources, calling it SD-IoT.Jararweh et al. suggested an SD-IoT framework [2] for which they introduced the concept of a software-de ned system (SDSys).e SDSys hides all of the complexities of the management and control functionalities of the system resources from the end users.ey also proposed a software-de ned IoT control framework for which multiple SDSys types such as software-de ned networks, software-de ned storage systems, and software-de ned security are integrated.Liu et al. suggested an SD-IoT architecture for an urban-sensing case [3] that consists of a data acquisition service for sensors, a data transmission service for network devices, and a data processing service for cloud servers.ey also emphasized the role of data aggregation and compression, just as many studies have previously claimed in WSN [4][5][6][7].is means that an intermediate node with data processing capability is needed in the SD-IoT network.

DCPS Model for the IoT.
For the DCPS model, the concept of a "global data space," where the known-structure data values are exchanged, is built upon.Here, the applications that want to contribute information to this data space declare their intent to become a publisher.Similarly, the applications that want to access portions of this data space declare their intent to become subscribers.Each time a publisher wants to post new data into this global data space, it propagates the information to all of the interested subscribers.To handle the DCPS communications, each node is made aware of the contents of the incoming packets, meaningful information is generated from them, and the global data are nally updated. is model is used by some of the IoT network protocols that are present between the IoT devices and their client applications.
e DDS [8] that is standardized by the Object Management Group introduces one of the IoT network protocols with the DCPS model.
is protocol provides platform/language-independent mechanisms for the construction of distributed publish-subscribe systems with various quality of service (QoS) guarantees and reliability-control capabilities.
e Message-Queuing Telemetry Transport (MQTT) protocol [9] is another IoT protocol of the DCPS model, and a broker is used to o oad the burden of the handling of a large number of upstream server requests for IoT devices.Some IoT applications require horizontal integrations wherein inter-protocol conversions occur.Recently, several attempts were made to address this issue including the oneM2M project that aims to provide a common IoT service platform by consolidating the currently isolated protocols [10].oneM2M utilizes the representational state transfer (REST) system for the representation and management of  Mobile Information Systems IoT devices, and common application protocols including the hypertext transfer protocol (HTTP), the constrained application protocol (CoAP), and the MQTT are used for the interworking with other systems besides oneM2M.e Eclipse IoT project provides another integrated IoT service platform [11].e project is composed of many subprojects such as supervisory control and data acquisition (SCADA), Krikkit, and Ponte.Krikkit is a data-acquisition architecture that uses the DCPS model, while SCADA and Ponte provide a common communication mechanism and uniform open application programming interfaces (APIs) for programmers, thereby enabling conversions between various IoT protocols, such as the HTTP, CoAP, and MQTT.

Berkeley Packet Filter (BPF). In 1993, Steven McCanne
and Van Jacobson introduced a novel way of ltering packets in the kernel, and it is called Berkeley Packet Filter (BPF) [12].
e BPF has been widely used for network applications such as libpcap and tcpdump.e BPF de nes a virtual lter machine and the corresponding language.
e language contains several instructions regarding the fetching of data from packets, the performing of arithmetic operations, and data comparison.A lter is de ned using these instructions.e virtual machine executes the lter to decide whether an incoming packet is acceptable or not (Figure 2).
A number of the BPF extensions [13][14][15] have been proposed to improve the BPF speed and expressivity, but they do not extend the BPF functionality.Even though the BPF is excellent for packet matching, functional extensions are needed to make it suitable for versatile packet processing such as packet modi cation or generation.Jouet et al. [16] proposed the application of the BPF to the SDN data plane, but they used only the BPF to match the incoming packets.
e Instruction-based Packet Processing (IPP) scheme [17] has been proposed for the use of the BPF in the provision of the data-plane programmability in SDN. e IPP language supports various data-processing features as well as those of its packet processing, and its data plane supports the run-time installation of new programs.

E-IPP Scheme
e aim of the E-IPP scheme is the introduction of a programmable data plane for SD-IoT devices.It adopts many of the features of the IPP scheme, and the IPP has been extended so that it can be used as an event-driven processing model, thereby enabling in-network data processing as well as packet processing in various IoT situations. is scheme consists of an event model, a language to program the procedures for application-speci c processing, and a virtual machine to install and run the program.

An E-IPP Event
Model.An E-IPP event comprises an identi er, a class, a subclass, and option data.e identi er uniquely identi es the E-IPP event, the class categorizes the E-IPP event, and the subclass de nes a unique event type within a speci c class.e <class, subclass> pair identi es a unique event type, and it indicates a speci c event handler to deal with the event.e option data vary depending on the event type.
e two event categories are as follows: system-de ned and user-de ned classes.
e system-de ned class represents prede ned event types such as the init, packet-in, and timer subclasses.
e user-de ned class includes application-speci c event types that can be dynamically de ned by IoT applications.To de ne a new event type, an IoT application assigns an unused event subclass that identi es the event type.en, it registers the event subclass to the E-IPP VMs. e IoT applications may register their own event handlers, which includes their applicationspeci c procedures, for both the system-or user-de ned event types.
e E-IPP VM consists of several event-handling policies.Each event type is matched to one of the event-handling policies.A policy represents the number of event handlers that can be registered to an event type and the manner in which they are executed in the presence of more than one handler.
e three policies are as follows: singleton, sequential, and parallel.e singleton policy is the representation of an event that carries an event-speci c handler within itself.e init and timer event types represent the singleton policy.e sequential policy represents multiple event handlers that can be registered to an event type with their priorities, and the handlers will be executed according to their priority-based order.While the VM is executing the handlers, a proceeding handler may block the other handler executions of the lower priority.e packet-in event type is an example of the sequential-policy types.e parallel policy represents the independent execution of all of the registered

Mobile Information Systems
handlers.e user-defined event types represent the parallel policy.
e events are generated from a variety of sources.e system-defined events are generated from the loader, the input packet buffer, and the internal timer, all of which are described later in this paper.e user-defined events are generated when an event generation code is executed, and the event-generation code may be placed in another event handler to concatenate the event handling.

E-IPP Language.
e E-IPP code is written using the E-IPP language, which is a low-level language for the E-IPP VM.It consists of a series of directives, instruction statements, and comments.e directives direct the loader regarding the installation of the E-IPP code into the VM. e instruction statements are the symbolic machine codes of the E-IPP VM and are translated into executable codes by the loader.e E-IPP language is derived from the BPF, especially the Linux Socket Filter (LSF) [18].Accordingly, many of the BPF-language features have been inherited, such as the instruction-statement syntax, instruction set, scratch memory (addressable registers of a limited size), and BPF extensions (platform-dependent variables provided by the Linux kernel).
e specialization of the original BPF, however, is regarding the packet matching, and it is not suitable for data processing and event handling.e E-IPP language extends the LSF to include the event-handling features.e directives and the predefined procedures that access the VM event-handling features are provided; the utilization of these allows for the defining of the new event types and event handlers that are registered to the VM.Further, new instructions are added to provide the modification and forwarding features of the packet to the LSF.
Table 1 show the directives, extended instructions, and library procedures that were used in the proposed examples.A user-defined event type can be defined by a defevent directive statement.e defevent directive statement consists of the event-type ID and the size of the option data, if any are present.e external libraries, which are imported by the import directive statement, provide a simple way to program complicated procedures with respect to a number of specific cases.e library procedures are called using the call instruction; its operand indicates a procedure using the symbol that is exported from the library.When a procedure is called, the scratch memory (a predefined array with the symbol "M" in the LSF) is used for the passing of the arguments.e reg_hnd procedure is used for the registeration of the event handler of a specific event type, and the event is generated by the gen_ev procedure.e generated event facilitates the scheduling of the registered event handlers of its type.

E-IPP VM
e structure of the E-IPP VM is derived from the IPP, but the behavior of each component changes on an event-driven basis.e E-IPP VM is composed of an event scheduler, an execution engine, a loader, packet buffers, and a runtime storage (Figure 3).

Event Scheduler.
e event scheduler is a component of the VM that schedules the event handlers for the received events to the execution engine.e scheduler is composed of an event queue and an event-type table.When an event is generated, it is queued at the event queue.e event-type table stores the information for the event types and their event handlers.A table entry consists of an event-type field, an event-handling policy, and an ordered event-handler list.
When an event exits at the event queue, the event scheduler dequeues the event.e scheduler looks up the event-type table using the event type, and if a matching entry exists, it schedules the event handlers at the matching entry to the execution engine according to its policy.For the sequential policy, the scheduler links the event handlers of the list at the matching entry in the order of their priorities.en, the scheduler places the ordered list into the wrapper handler that executes the handlers sequentially, and it may also terminate the execution prematurely.e wrapper is scheduled at the execution engine.In the case of the parallel policy, the scheduler schedules all of the event handlers in the list to the execution engine.In the case of the singleton policy, an event handler is not present in the event-type table entry; instead, the event itself carries its own handler identi er in its option data.e scheduler schedules the handler in the event to the execution engine.

e E-IPP Loader.
e E-IPP loader installs the E-IPP code into the VM on behalf of the E-IPP application.When E-IPP code is received by the E-IPP loader, the loader compiles the code into the form that can be executed by the execution engine.e executable form of the E-IPP code is called the E-IPP instance.e E-IPP instance includes the runtime-data section for the nonvolatile data that are used to remember the state of the E-IPP instance.
en, the E-IPP loader stores the E-IPP instance at the runtime storage and generates an init event.e init event includes an initialization function that serves as the corresponding event handler.e initialization function usually registers new event types and event handlers to the scheduler.

Execution Engine.
e execution engine runs the event handlers that are scheduled by the scheduler.e execution engine comprises an event-handler queue and a task queue.When an event handler is queued at its event-handler queue, the execution engine dequeues the event handler and assigns a task to it; then, the task is queued at its task queue.e execution engine runs the tasks of the task queue according to its scheduling policy.e default policy is rst come, rst served (FCFS).

Input and Output Packet Bu ers and the Runtime
Storage.
e input and output bu ers consist of their own packet queues and schedulers.When an incoming packet arrives at the E-IPP VM, the input-bu er scheduler receives it.It queues the packet into its packet queue and generates a packet-in event.
e output bu er stores the outgoing packets, and the packets in the output bu er are transmitted outside of the VM by the output-bu er scheduler.
e runtime storage maintains the E-IPP instances and libraries for the external procedures.e libraries contain the prede ned complicated procedures, and the procedures are called while the E-IPP instances are being executed.

An Example of SD-IoT Applications and Implemented E-IPP Codes
To explain the way in which IoT applications can utilize the E-IPP scheme, an example is now presented.For this example, it was assumed that three temperature sensors have been connected to a sensor gateway that is controlled by the SD-IoT controller.e sensors periodically send temperature data to the sensor gateway via the IEEE 802.15.4 protocol.Each packet that is sent from the sensors contains a destPANId and a srcPANId in the header and an action eld and sensed data in the payload.e sensor gateway embeds the E-IPP VM as its data plane, and the E-IPP codes in the sensor gateway process the raw temperature data.e E-IPP codes send application-speci c data to the IoT application that subscribes the application-speci c data to the SD-IoT controller.Figure 4 shows the SD-IoT architecture that was used for the example.
e aim of the IoT application A is the attainment of the temperature data that are sent from the sensors, so it subscribed to the SD-IoT controller to receive the temperature-update events.
e IoT application B is interested in the alert event that is regarding the exceeding of the temperature from one of the sensors beyond the set threshold of the application.When the IoT applications subscribe to the SD-IoT controller regarding their interests, the SD-IoT controller obtains the E-IPP codes for these interests and installs them in the E-IPP VM.

Mobile Information Systems
Figures 5 and 6 show the simpli ed E-IPP codes that were installed for each of the subscriptions.In Figure 5, the rst directive statement imports an external procedure named send_ip_msg.As shown in Figure 7, the arguments <dest, event data size, event data> are passed onto the procedure, and the procedure creates a new User Datagram Protocol (UDP)/Internet Protocol (IP) packet with the given event data, and the packet is then sent to the dest.

6
Mobile Information Systems e other directive statements in Figure 5 de ne a new event type (update_temperature:0x000A), an array for the event option data, three variables (sensor_id, temperature, and destA), and the initialization code label (init).
e initialization code sets the IP address of the application A server in the variable destA.Further, it registers the new event handlers, PParser and SendTemp, for the PACKET_IN and the update_temperature event types, respectively.
e PParser is the parser for the sensor packets. is parser uses the destPANId eld to check whether the destination of the packet is the VM itself.e srcPANId is stored in the sensor_id.Next, the parser inspects the action eld.If this eld is update temperature, the following data are counted as the temperature data and are stored in the temperature variable.en, it generates an update_temperature event.e sensor_id and the temperature are included in the event option data.
e SendTemp publishes the updated temperature to the application A server.It uses the imported procedure send_ ip_msg to send a packet to the destA for which it shifts the option data from the update_temperature event to the sending-packet payload, thereby resulting in the sending of the sensor_id and the temperature.
e second E-IPP code (Figure 6) includes the procedures for the publishing of a re alert to the application B.
is code shares the update_temperature event that was generated by the rst E-IPP code to reduce the packetparsing load, and it checks the temperature data in the packet.If the temperature exceeds the threshold, it sends a re alert to the application B server.

Comparison with Other Programmable
Data Planes is section presents the comparison of the proposed method with the other existing programmable data planes.e comparison subjects are the BPF, two methods that adopt the BPF (Jouet et al. and IPP, which are introduced in the Related Work section), and other existing programmable data planes that are introduced in the following subsection.

Existing Programmable Data Planes.
A number of proposals have been presented to introduce programmability into the SDN data plane, such as P4 [19], open deeply programmable network node architecture (DPN) [20], ClickOS [21], and OpenState [22].e P4 proposal suggests a propriety packet-processing language for SDN switch operations.e P4 language is targeted for the abstraction of parse-match-action pipeline operations in dedicated hardware.DPN and ClickOS are based on the Click modular router [23], and they use the Click module to program their data-plane operations.As the Click modular router does not support the run-time programmability, DPN and ClickOS also do not allow the addition of new actions in run-time.OpenState suggests a stateful data-plane model and it introduces an extended nite state machine (XFSM) for the data-plane programmability.
e SDN controller de nes the states and the events for ows, whereby the action for a state and an event are installed inside the network device.When an event occurs, the network device handles the event using the current ow state and the precon gured actions.e data plane can be programmed to make a forwarding decision in consideration of the network situation without the controller intervention.
In spite of their programmable features, their languages are targeted to the table-based packet-processing models (e.g., forwarding model in Figure 8).ey comprise ingress and egress tables, and the entire network programs are translated into packet-processing rules and are placed at one of these tables.e aim is the enabling of programmers so they can describe match-action tables that are dynamically populated by clearly articulated rules.
In the IoT environment, the table-based model is not exible enough to cover all of the requirements of the IoT applications; for example, in the case where a sensormanagement application needs to register a timer event on a sensor gateway node for a dead-sensor-indication procedure.Furthermore, in the presence of multiple sensorstatus-monitoring clients, the gateway node generates an internal user-de ned event for the dead-sensor indication to activate multiple noti cation procedures for each of the subscriber clients.It is obvious that these noti cation procedures are not packet-processing procedures, and the existing table-based packet-processing model is not capable in this case.e requisite processing model is su ciently exible to process these procedures as required.Mobile Information Systems all of the other BPF-based methods comprise data-processing advantages, because they abstract the processor operation and consist of a number of the requisite instructions of data-processing operations.Non-volatile storage is used for network status management or data aggregation.All of the methods use the nonvolatile storage, because it is essential for basic operations such as the packet counter in network equipment, but the BPF-storage size for each filter is very small.is size was improved for IPP and E-IPP so that programmers can freely create a variable or an array within their code.

Comparison of the Programmable Data Planes.
Timer-event processing is an important feature in many IoT-application cases, such as the periodical reporting of aggregated data or the finding of dead sensors.P4 does not comprise this feature, but DPN, ClickOS, and OpenState include the timer-event handling for packet retransmission and other purposes.BPF and the other existing BPF-inherited schemes do not comprise timer-related instructions, but the proposed scheme includes the timer-event-handling feature.
In situations where multiple IoT applications install their own code into the VM, user-defined events are necessary for the interworking between the various codes.
is feature allows the handlers that are registered in the application code to receive the events that are generated from another application code, thereby allowing them to function as the subscriber code and the publisher code, respectively.is is the feature only the proposed scheme.
e run-time installation is the essential feature of dynamic provisioning, which is one of the advantages of the SD-IoT that is inherited from SDN.When an IoTapplication seeks a new service, the SD-IoT network needs to install the application code into the data plane at runtime so that it can be immediately reflected in the SD-IoT network. is feature is considered for P4, IPP, and E-IPP.

Conclusion
is paper contains the proposal of an SD-IoT data-plane scheme that is specialized for in-network data processing.It is based on a generic event-driven model, and it is suitable for the handling of the various requirements of in-network data processing as well as those of normal packet forwarding.It also includes a language for the defining of the userdefined procedures of the description of context-specific operations and their dynamic-installation interfaces.
e presented examples show that the SD-IoT application can program the requisite data-aggregation and DCPS-agent functions of the SD-IoT gateway using the proposed scheme.
ese examples mean that the SD-IoT application can add in-network data processing to run-time services without the deployment of any additional network entities, and this is simply achieved by the implementation of several lines of code onto the data plane.
e comparison with other programmable data planes shows that the features of the proposed scheme are suitable for various IoT applications.

Disclosure
is is an extended and revised version of a preliminary conference report that was presented at the 8th International Conference on Information and Communication Technology Convergence (ICTC) in 2017 [24].

Figure 1 :
Figure 1: A SD-IoT architecture with a sensor gateway.

Figure 2 :
Figure 2: e concept of packet ltering in BPF.

Figure 5 :
Figure 5: E-IPP code for example application A: ltering packets and reporting to the server.

Figure 6 :
Figure 6: E-IPP code for example application B: reporting to the server when certain conditions are met.

Figure 7 :
Figure 7: E-IPP code for packet sending procedure.

Table 1 :
Directives, new instructions, and predefined procedures.

Table 2
shows whether the programmable features of each of the data planes are those that are required by the various IoT applications.All of the scheme features allow for the arbitrary packet matching that is required by the IoT applications for the programming of new protocols.Among them, however, BPF and the scheme of Jouet et al. cannot modify and forward packets, because the corresponding researchers only considered packet-ltering cases.DPN, ClickOS, and OpenState show weaknesses in terms of the data-processing features.ese weaknesses of DPN and ClickOS are inherited from the simple click-module structure they use, and the focus of OpenState is the extension of the ow table; moreover, data-processing cases have not been considered.Alternatively, for P4, the programming exibility for data processing has been considered, even though it adopts the table-based model.BPF and

Table 2 :
Comparison with other programmable data planes.