Pattern-Identified Online Task Scheduling in Multitier Edge Computing for Industrial IoT Services

In smart manufacturing, production machinery and auxiliary devices, referred to as industrial Internet of things (IIoT), are connected to a unified networking infrastructure for management and command deliveries in a precise production process. However, providing autonomous, reliable, and real-time offloaded services for such a production is an open challenge since these IIoT devices are assumed lightweight embedded platforms with limited computing performance. In this paper, we propose a pattern-identified online task scheduling (PIOTS) mechanism for the networking infrastructure, where multitier edge computing is provided, in order to handle the offloaded tasks in real time. First, historical IIoT task patterns in every timeslot are used to train a self-organizing map (SOM), which represents the features of the task patterns within defined dimensions. Consequently, offline task scheduling among edge computing-enabled entities is performed on the set of all SOM neurons using the Hungarian method to determine the expected optimal task assignments. In real-time context, whenever a task arrives at the infrastructure, the expected optimal assignment for the task is scheduled to the appropriate edge computing-enabled entity. Numerical simulation results show that the proposed PIOTS mechanism overcomes existing solutions in terms of computation performance and service capability.


Introduction
Recently, smartization in manufacturing has been considered as one of the major trends realizing the fourth industrial revolution [1].In the smart factory, production machinery and auxiliary devices, which are referred to as industrial Internet of things (IIoT), maintain permanent connections to a unified networking infrastructure, where Internet service is available.In this environment, all of the connected IIoTdevices acquire mutual cooperation with each other and they request the working commands from central management entities in the network.For instance, precise production processes require trigger feedback from control entities to adjust reactions of machines and robots if an unexpected issue occurs.Another example is when smoke is detected in a warehouse, a real-time shutdown command should be immediately dispatched to electrical working chains and a real-time activation command should force the fire extinguishing system to be activated [2].However, since these IIoT devices are assumed to be lightweight embedded platforms with limited computing resources, it is inappropriate to execute real-time services using its own power of the devices.
Fortunately, thanks to the advances of the emerging fifth-generation (5G) technologies, the 5G mobile edge computing (MEC) can provide autonomous, reliable, and real-time offloaded services for these IIoT devices and applications.Defined by the European telecommunications standards institute (ETSI), the MEC provides cloudcomputing capabilities and an IT service environment at the edge of the network [3].To be more specific, multitier MEC (mMEC) is a virtualized hierarchical MEC framework where edge computing-enabled entities (ECEs) are layered according to its computing performance.e operation of mMEC framework is supervised by a central orchestrator [4].e orchestrator schedules the tasks offloaded from IIoT devices among ECEs in order to maximize computation performance and decrease energy consumption while keeping the IIoT applications' requirements.
Aiming at this objective, we propose a patternidentified online task scheduling (PIOTS) mechanism for the mMEC framework in smart factory for handling the offloaded IIoT tasks in real time.First, historical IIoT task patterns in every timeslot are used to train a self-organizing map (SOM), which represents the features of the task patterns within defined dimension [5].As a result, a set of typical SOMs for IIoT task patterns are identified.At the beginning of each timeslot, offline task scheduling among ECEs is performed on the neuron set of the typical SOM using the Hungarian method to determine the expected optimal task assignments.
ereafter, whenever a task arrives at the mMEC framework, the task is matched into the current SOM to find the best matching SOM neuron for it.Based on that, the expected optimal assignment for this matched SOM neuron is scheduled for the task, assigning to appropriate ECE in online manner.
e main contributions of this paper are three-fold: (i) First, expected optimal assignments are scheduled for offloaded IIoT tasks that arrived at the mMEC framework in real time.In other words, a task assignment delay is reduced significantly.(ii) Second, the duration of task assignment calculation is decreased because this operation works on the SOM neuron set within defined dimension.e neuron set is a representative of the historical incoming tasks, which is not greater than the size of real task patterns in each timeslot.As a result, the computational complexity and calculation latency are reduced as well.(iii) ird, the expected optimal task assignment calculation is performed independently on the task scheduling timeline.erefore, the task assignment can be operated continuously in real time without waiting for the task assignment calculation to be completed.
e remainder of this paper is organized as follows.Section 2 surveys the state-of-the-art-related work.Section 3 describes the proposed pattern-identified online task scheduling mechanism in detail.Section 4 provides the system setup, evaluation methodology, and evaluation metric definitions.Based on that, the performance of the PIOTS mechanism in comparison to other techniques is analyzed in Section 5. Finally, the paper is concluded in Section 6.

Related Work
Optimal task assignments in multitier edge computing have been classified into three main categories: latency awareness, energy awareness, and quality-of-service (QoS) awareness including their variants [6,7].e latency-aware approaches focus on minimizing execution latency of the task offloaded to the edge servers.e execution latency involves three portions: transmission duration from the IIoT devices to the edge servers, queuing and processing duration at the edge servers, and return duration for successful reception of the result in the IIoT devices [8,9].e mMEC framework can be designed and supported by several emerging technologies such as network functions virtualization (NFV) and software defined networking (SDN).In [10], Dao et al. proposed an adaptive balancing scheme (ARB) to distribute tasks among edge servers in the remote radio heads in order to improve the serviceability of the network, especially in terms of task execution latency.e ARB scheme combines the Hungarian method and the backpressure algorithm for this purpose.In [11], Mao et al. aim at reducing the task execution latency by using a low-complexity Lyapunov optimization-based dynamic computation offloading (LODCO) algorithm.e LODCO algorithm handles CPU utilization in the edge servers according to the task arrival.On the other hand, Liu et al. [12] considered application buffer queuing state, available computing powers, and channel quality to conduct the optimal offloading decision.e tasks will be assigned to the edge servers if the total execution latency made by the edge server is shorter than the duration spent to execute the task locally by the IIoT devices.Otherwise, the IIoT devices will execute their tasks themselves.
e energy-aware approaches aim at minimizing energy consumption for task execution in edge computing.Typically, the energy consumption is considered on task delivery and task computation.In [13], an energy reduction method was proposed based on the Bak-Tang-Wiesenfeld sandpile.When an edge server exceeds a certain capacity threshold, it collapses and initiates an avalanche of migrating tasks in order to balance the workload among edge servers.e achievement results in a significant reduction in task assignment errors and redundancy, which dominates the overhead of energy consumption.On the other hand, You et al. [14] considered the transmission energy for multiple IIoT devices offloaded tasks to the edgecomputing framework.
e optimal solution was developed and resolved so as to minimize the weighted sum mobile energy consumption under the constraint on computation latency.In [15], the problem of interdependent task scheduling was addressed within an edge system of deep memory hierarchies to obtain energy efficiency.e intermediate data of the tasks are prioritized and assigned at appropriate levels of cache memory to optimize latency and energy in data access.In order to make the offloading decision, Li et al. formulated the problem in a 0-1 nonlinear integer programming with a consideration of channel interference threshold and the time deadline [16].Based on that assumption, a reverse auction based offloading policy has been proposed to obtain energy efficiency improvement for task execution.
e QoS-aware approaches consider multiple criteria of IIoT service requirements such as execution latency, service availability, transmission throughput, and security as 2 Mobile Information Systems trade-o problems, which have to be optimized.In [17], a distributed optimization algorithm that cooperated among edge servers, called o oad forwarding, is proposed by using the distributed alternating direction method of multipliers (ADMM) via variable splitting to maximize QoS and energy e ciency.For focusing on maximizing the transmission throughput for task o oading, Vu et al. [18] utilize the Hungarian method to optimize the downlink sum rate in fog computing-enabled networks.On the other hand, Zeng et al. [19] proposed a security-aware and budget-aware work ow scheduling strategy (SABA), which considers the task distribution among providers to achieve cost-e ective and secure services in the context of convergent network. is strategy is applicable to the edge computing environment where edge servers act the role of service providers for o oaded tasks from the IIoT devices.Within the same purpose, Xu et al. proposed the green o oading (GO) scheme [20] that uses the reverse auction theory to develop the o oading decision while satisfying the user quality-ofservice (QoS) requirements, bandwidth, and the maximum transmit power.
Although the existing approaches have signi cantly contributed to improve the performance of the edge computing, most of the existing approaches face a drawback in online task handling, where the tasks that arrive at the edge framework have to be queued before the scheduling decision is made.is queuing behavior generates signi cant latency, especially in cases of smart manufacturing, where real-time response requirement is a crucial criterion for precise production processes.In the next section, our proposed PIOTS mechanism, which overcomes these issues, is described in detail.

Pattern-Identified Online Task Scheduling
3.1.Basic Assumptions.In this paper, we consider a networking infrastructure for smart manufacturing where mMEC framework is covered.Machinery and auxiliary devices participate in a smart production process, generate their IIoT tasks, and o oad them to the mMEC framework for centralized handling.e IIoT tasks are processed and responded to the IIoT devices on demands.Typically, the production process operates continuously in a long duration (e.g., weeks, months, and even years).erefore, we assume that the IIoT task patterns maintain their permanent trends even though instant tasks are generated randomly by the IIoT devices.In this circumstance, real-time response is considered as a crucial requirement for precise productions.Figure 1 illustrates the system model where the IIoT devices are supported by the mMEC framework in smart manufacturing environment.
e IIoT devices are connected to the network via wireless access points (APs) and switches (SWs).
e mMEC framework consists of APs/SWs, routers, and local servers/storages, which have various computing capacities.In a mMEC framework, all these ECEs are managed and scheduled their computations by a central device named orchestrator.

Self-Organizing Map.
Self-organizing map is an arti cial neural network (ANN) that utilizes unsupervised learning on the high-dimensional data to produce a de ned lowdimensional representation called map (Figure 2).Each SOM is formed by a de ned number of neurons, which re ect the map dimension.
e operation of the SOMs includes two modes: training and mapping.
e training mode develops a map based on the input set of tasks.Meanwhile, the mapping mode is used for classifying new arrived tasks.
e SOM algorithm to train the map is iterated by two steps after initialization as follows.
Initialization: Assume that there are n tasks in the input set X, wherein each task i(1 , where w j → [w j1 , w j2 , . . ., w jl ].To start the SOM algorithm in order to train the map, each neuron initiates its own weight vector with random values.Afterwards, the two steps below are contiguously repeated for each task in the X.In  detail, for task i, the SOM algorithm performs the following steps: Step 1 (matching): Euclidean distance between task i and each neuron in the SOM is calculated in order to nd the best matching unit.
e best matching unit for the task i is the neuron w * i → , which has the smallest Euclidean distance to the task i.In other words, w * i → is given by Step 2 (neuron update): all neurons in the neighborhood of the neuron w * i → update their weight vectors to be closer to task i by where w j → (t) is the weight vector of neuron j at this iteration t. θ(t) is the neighborhood function, which has diminished every iteration.θ(t) determines the distance from the neuron w * i → to de ne the neighboring neurons.μ(t) is the learning factor, which calculates the amount of e ect from the input task i to the neuron j based on distance between them [21].e mathematical expressions of θ(t) and μ(t) are described as follows: where R is the maximum radius from the center of the map (i.e., 0.5 max MapWidth, MapHeight ) and σ(t) is the neighborhood radius that is given by 3.3.Pattern-Identi ed Online Task Scheduling.In this section, we describe the PIOTS mechanism in detail.e rationale behind the PIOTS includes three steps: (i) identifying the set of typical SOMs for IIoT task patterns, (ii) calculating expected optimal task assignment on the typical SOM prior to each timeslot, and (iii) online assigning new arrived tasks to appropriate ECEs.Steps (i) and (ii) are in the o ine mode and step (iii) is in the online mode.

O ine Mode. (1)
Typical SOM identi cation: in terms of task execution o oading, a IIoT task i is characterized by a four-dimensional feature vector as given by where u i , c i , r i , andτ i are relative task size, relative average processing complexity, relative response size, and relative execution deadline compared to their minimum and maximum values of all trained tasks, respectively.As mentioned in Section 3.1, the production process operates continuously during a given period.Within each timeslot in this period, given that there is an average number N x tasks in the set χ(t) of IIoT tasks o oaded to the mMEC framework, the number k of neurons in the SOM is selected as where • is the nearest integer function and λ is a scale level, which supports reduction of the SOM size.Based on these settings, the k-neuron two-dimensional SOM is trained using all o oaded IIoT tasks in the given period.For each distinctive period, we obtain a typical trained SOM to represent the o oaded IIoT tasks.A distinctive period is de ned as a cycle of the o oaded IIoT task arrival; after this duration, the task arrival is repeated again in terms of volume and characteristics.
(2) Expected optimal task assignment calculation: Figure 3 describes the PIOTS operation timeline in a random IIoT task arrival context.Prior to timeslot t, the PIOTS mechanism performs the H(t) function at t − Δt in order to determine the expected optimal task assignment during timeslot t. e H(t) function uses the Hungarian method [22] to address the expected optimal task assignment problem between the neuron set of the typical SOM in timeslot t and the current ECE set.
e result of H(t) function will be applied for online task assignment in the whole duration from timeslot t to timeslot t + 1 even though the H(t + 1) function will start at (t + 1) − Δt.
e expected optimal task assignment problem is dened as "Given the current performance status (i.e.

, task bu er and CPU frequency) of ECEs and the transmission status (i.e., access data rate between IIoT devices and the mMEC framework and forwarding data rate among ECEs), minimize the total latency of task execution when assigning the neuron set of the typical SOM to the ECEs."
e expected optimal task assignment problem can be formulated as follows.Let W and M denote the set of k neurons in the SOM and the set of m ECEs in the mMEC framework, respectively.Since the SOM neurons have trained by using the historical tasks, the weight vector of the neuron re ects the average values of the task features, accordingly.at is, w j1 , w j2 , w j3 , and w j4 in the weight vector w j → of neuron j re ect the average values of the relative task size, relative average processing complexity, where C s , ra js , and rb js are the CPU frequency of ECEs in Hz, the access data rate from the IIoT device that owned task j to the network in bps, and the internal forwarding rate to the ECEs in bps, respectively.Equation ( 8) consists of (a) uploading latency, (b) queuing latency, (c) task processing latency, and (d) response latency.e access data rate ra js is given by where BW js and SINR j are the bandwidth allocated for the IIoT device and the channel quality between the IIoT device and the network, respectively.Based on that, the expected optimal task assignment problem (F ) can be described as follows: where the indicator z js is given by Constraint (13) ensures that a neuron could only be assigned to one ECE and the maximum matchings are established for all m neurons of the SOM.
It is observed that task assignment problem F regards a bipartite graph of two sets W and M. We apply the Hungarian method (referred as H(•)) on the graph to achieve the optimal solution.Function H(•) is performed as follows (see Algorithm 1): (i) Augment the latency matrix of all possible task assignments from neuron set W to ECE set M into a square matrix (named V) with dimension max (|W|, |M|) by supplementing additional entries of constant number (e.g., 0).(ii) In each row of V, subtract the smallest row entry from all of the row entries.Similarly, in each column of V, subtract the smallest column entry from all of the column entries.(iii) Find the minimum number υ of rows and columns by which all zero entries are covered.(iv) If υ is equal to the size of V, the optimal solution is found.Pick up a set of 0 entries satisfying in which no more than two 0 entries are in the same row or column.Otherwise, if υ is less than the size of V, determine the smallest entry c that is uncovered in the previous step.Afterwards, subtract c from all uncovered rows and add c to all covered columns.Return to the previous step to find the minimum number υ again.
For each timeslot, the function H(•) is calculated prior Δt before the beginning of the timeslot in order to find the expected optimal task assignment solution.

Online Mode.
Online task assignment: in the online mode, when an IIoT task arrives at the mMEC framework, the task is matched to the SOM so as to seek the best matching neuron, using (1) with the computational complexity of O(1).According to the optimal assignment for the found neuron that was determined by function H(•) in the offline mode, this assignment is performed on the arrived IIoT task immediately.

Evaluation Preparations
4.1.System Settings.In order to evaluate the performance of the PIOTS scheme, we considered a network model, where Mobile Information Systems the mMEC framework consists of 20 ECEs equipped with various CPU frequencies of {1.5, 2.5, 5.0, 10.0, 25.0} GHz. e total number of IIoT devices associated with the network is 300.e task processing complexity is determined as {10, 50, 100, 500, 1000} cycles/bit derived from the practical analysis presented in [23].Execution deadline is randomly set within (0.5, 1.5) s.Lastly, timeslot duration is set to 0.1 s. ese simulation parameters are summarized in Table 1.
In terms of task and response size settings, we derived these parameters from a part of the CAIDA anonymized Internet traces data set [24]. e CAIDA data set contains anonymized passive traffic traces from the Equinix-Chicago Internet data collection monitor located at an Equinix datacenter in Chicago, IL, on high-speed Internet backbone links.Table 2 shows statistical indexes of the task samples used for SOM training and system evaluation.e tasks are selected for three types of traffic as follows: (i) Environmental sensor data: the information of environmental conditions is reported to central applications in the mMEC framework during each fixed period.e task size is configured within constant and small dimension, and these tasks do not require response from the applications.(ii) Video surveillance data: live streaming data from monitoring cameras are delivered to surveillance applications in the mMEC framework for video analysis and storage.ese tasks are packetized in fixed and large size.e response data might be issued from the applications if there are any alarms that should be announced.(iii) Production control data: these data are generated by the machinery during precise production processes.e tasks are regularly offloaded to the mMEC framework within a determined size.Tasks require responses from the applications to exactly handle production work.

Evaluation
Methodology.Within the aforementioned system settings, our proposed PIOTS scheme has been compared to the offline Hungarian task assignment (OHTA) algorithm and the online greedy task assignment (OGTA) algorithm.Initially 30,000 samples of the selected tasks derived from the CAIDA data set are used to train the SOM in the PIOTS scheme.en, 90,000 other samples are used for evaluation (Table 2).e operations of these schemes are described as follows: (i) e PIOTS scheme is performed as shown in Section 3.3.(ii) e OHTA algorithm gathers all arrived tasks at the input buffer of the mMEC framework during one timeslot.At the end of each timeslot, the Hungarian method is utilized to decide the optimal task assignment for all tasks in this timeslot [22].(iii) e OGTA algorithm determines an ECE, which provides the lowest latency for task execution following ( 8), for the arrived task [25].
e simulation results are logged in terms of task execution latency and execution error rate.e execution error rate evaluates the ratio of the over-deadline task executions and the total offloaded tasks.

Numerical Result Analysis
First, the proposed PIOTS scheme is compared to the optimal solution of the objective function F . e optimal solution for the integer programming problem F used in this paper is a combination of the Hungarian method and backpressure algorithm [10].Figure 4 illustrates the outcomes of F depending on various IIoT task arrivals through 300 timeslots.Numerical results reveal that the PIOTS scheme provides an approximate performance compared to the optimal solution.e difference in average task processing latency is 0.159 ms.It is worth noting that the PIOTS scheme performs the expected task assignment based on the typical task set derived from the SOM map; then, it does real task assignment online.Moreover, the SOM map is trained by using the collected IIoT tasks arrived at the network in the past.erefore, within a sufficient number of neurons in the SOM map that well represents for the typical characteristics of the incoming IIoT tasks, the mMEC framework can classify the incoming tasks immediately and then it handles the tasks by approximately optimal assignment.at is, each incoming task is immediately assigned to appropriate ECE right after the task arrives at the network.Meanwhile, the optimal solution is calculated based on the set of gathered IIoT tasks in each timeslot.
Table 3 shows statistical indexes of time consumption in ms for task assignment decision of the orchestrator when applying the PIOTS, OHTA, and OGTA schemes, respectively.e average decision-making duration that the PIOTS scheme consumes is 0.038 ms, which is smaller than the OHTA scheme's 100.1027 ms (approximate 2635 times of reduction) and greater than the OGTA scheme's 0.0355 ms (approximate 15 times of increase).e reason is because Mobile Information Systems the PIOTS scheme calculates (1) for all neurons in the SOM map, while the OHTA scheme must wait until the end of each timeslot to calculate the optimal solutions.e OGTA scheme achieves the smallest duration since it performs (8) for only 20 ECEs and selects the smallest latency value.Although there is a di erence between the PIOTS and OGTA schemes, the average values are considered to insigni cantly a ect the task processing latency.e standard deviation and standard error mean show that all schemes are performed in a stable condition.Figure 5 demonstrates a comparison of the average task processing latency when applying three competitors (PIOTS, OHTA, and OGTA).e PIOTS scheme achieves an e ective performance since the typical task set is used to predetermine the expected task assignment for incoming tasks.Meanwhile, the OHTA scheme performs task assignment based on the gathered incoming tasks during each timeslot.Although the OHTA scheme provides better adaptation to varying task arrivals, it must wait until the end of each timeslot to collect the tasks and then determine the optimal assignment.On the other hand, the OGTA scheme greedily assigns tasks to the ECEs of lowest latency.In statistic perspective, for average take processing latency, the PIOTS scheme overcomes the OHTA scheme and the OGTA scheme by 41.47% and 4.47%, respectively.
In order to evaluate the service capability of the network, we utilize the execution error rate, which is de ned by the percentage of deadline-violated IIoT devices in the total associated devices in the network.Figure 6 shows simulation results corresponding to three thresholds of execution deadline including 0.5 s, 1.0 s, and 1.5 s as aforementioned in the system settings (Table 1).During 300 simulated timeslots, the execution error rates for 0.5-second deadline are approximate among PIOTS, OHTA, and OGTA schemes (0.970%, 0.976%, and 0.977%, resp.).Meanwhile, in terms of 1.5-second deadline, the PIOTS scheme decreases the execution error rate to 0.826% (approximate 5.7% and 1.2% decreases compared to the OGTA and OHTA schemes, resp.). is analysis demonstrates that the PIOTS scheme provides better balanced task distribution among ECEs in order to satisfy task execution deadlines.
Figure 7 plots average bu ering latency for IIoT tasks arrived at the ECEs during 100, 200, and 300 timeslots.Since the arrived task volume has been con gured to over capacitate the ECE performances leading to a saturated condition in the network, bu ering latency increases by   Mobile Information Systems timeslot.It is observed that the OGTA scheme makes signi cant bu ering latency in every timeslots.On the other hand, the PIOTS scheme achieves the lowest bu ering latency by 44.33% and 4.32% in timeslot 300, compared to the OGTA scheme and OHTA scheme, respectively.Figure 8 depicts task distribution among ECEs, which helps to clarify the reason for the bu ering latency achievements of the schemes.e y-axis represents the assigned task volume in gigacycles (a.k.a.computational CPU cycles) among ECEs, which is given by a production function of task size and task processing complexity for all assigned tasks in each ECE.In the OGTA scheme, the arrived tasks are assigned more equally among ECEs (represented by the width of the box plotted) in comparison with other schemes.Since the PIOTS scheme and OHTA scheme are able to adapt task distribution according to the diversity of ECE computation performances, they provide better task assignments resulting in lower bu ering latency.

Concluding Remarks
In this paper, a pattern-identi ed online task scheduling mechanism has been proposed to deliberate on real-time task assignment in the smart manufacturing system.e proposed PIOTS scheme utilizes SOM technology for task identi cation and then assigns the task to appropriate ECE by using the Hungarian method.Simulation results demonstrate that the PIOTS scheme overcomes the existing algorithms in terms of task processing latency and service capability for satisfying IIoT applications.In future research, individual requirements of IIoT applications will be considered and veri ed via several popular datasets within the purpose of achieving the optimal performance for task handling in the entire network.Moreover, a consideration of applying game-theoretic approach should be studied to develop a distributed computational mMEC framework.

Figure 1 :
Figure 1: System model where the IIoT devices are supported by the mMEC framework in smart manufacturing environment.

( 1 )
Generate a square latency matrix V from W and M.(2) In each row, subtract all entries by the smallest row entry.(3)In each column, subtract all entries by the smallest column entry.(4) Find the minimum number υ of rows and columns that cover all zero entries.(5) If υ �� |V|, the optimal solution is found.(6) If υ < |V|, Find the smallest entry c in the matrix, Subtract all uncovered rows by c, Add all covered columns by c, Return to Line (4).ALGORITHM 1: Pseudocode of function H(•).

Figure 4 :
Figure 4: PIOTS scheme versus optimal solution for the objective function F.

Table 2 :
Statistical indexes of task samples for SOM training and system evaluation.

Table 3 :
Time consumption in ms for task assignment decision of the orchestrator.