Fuzzy-Based Mobile Edge Orchestrators in Heterogeneous IoT Environments: An Online Workload Balancing Approach

Online workload balancing guarantees that the incoming workloads are processed to the appropriate servers in real time without any knowledge of future resource requests. Currently, by matching the characteristics of incoming Internet of Things (IoT) applications to the current state of computing and networking resources, a mobile edge orchestrator (MEO) provides high-quality service while temporally and spatially changing the incoming workload. Moreover, a fuzzy-based MEO is used to handle the multicriteria decision-making process by considering multiple parameters within the same framework in order to make an o ﬄ oading decision for an incoming task of an IoT application. In a fuzzy-based MEO, the fuzzy-based o ﬄ oading strategy leads to unbalanced loads among edge servers. Therefore, the fuzzy-based MEO needs to scale its capacity when it comes to a large number of devices in order to avoid task failures and to reduce service times. In this paper, we investigate and propose an online workload balancing algorithm, which we call the fuzzy-based (FuB) algorithm, for a fuzzy-based MEO. By considering user con ﬁ guration requirements, server geographic locations, and available resource capacities for achieving an online algorithm, our proposal allocates the proximate server for each incoming task in real time at the MEO. A simulation was conducted in augmented reality, healthcare, compute-intensive, and infotainment applications. Compared to two benchmark schemes that use the fuzzy logic approach for an MEO in IoT environments, the simulation results (using EdgeCloudSim) show that our proposal outperforms the existing algorithms in terms of service time, the number of failed tasks, and processing times when the system is overloaded.


Introduction
Nowadays, fifth-generation (5G) cellular technology has tremendously changed our daily lives by supporting various new applications, such as video streaming analysis, augmented reality (AR), the Internet of Things (IoT), and autonomous driving [1,2]. In particular, the 5G-enabled IoT is a key major driver of the growth in IoT applications [3]. Nonetheless, in the era of the IoT, there is a large number of sensors, actuators, and mobile devices that generate a heavy load for networks [2,4]. These applications require a very short response time and large amounts of computational resources. Moreover, 5G-enabled IoT applications require real-time responses and are dramatically increasing at the same time [5]. As a result, processing massive amounts of data traffic, and the growing demands for high data rates and computational capabilities, are key features of the future Internet [6].
In conventional cloud computing, mobile devices upload all data to centralized servers to utilize the powerful computing capability of the cloud over a wide area network (WAN) [7,8]. In heterogeneous IoT environments, cloud computing cannot ensure the requirements of new delay-sensitive applications. Moreover, cloud computing is deployed at some distance from IoT devices; therefore, it cannot support contextaware computing for IoT applications [9].
To address the challenges in the cloud for heterogeneous IoT environments, the edge computing paradigm brings servers closer to the IoT devices and is deployed at the edge of the network [10]. The mobile edge computing (MEC) provides latency-sensitive applications and can reduce traffic bottlenecks in the core and backhaul networks while assisting in the offloading of heavy computational tasks from mobile devices to the edge [3,11]. However, the heterogeneous IoT environment opens up potential innovations for efficient resource management that meets the quality requirements of different IoT application types, such as how the edge system responds to the dynamic flow of requests [12].
One such approach is known as the mobile edge orchestrator (MEO), as discussed by the European Telecommunications Standards Institute 1. It is designed to integrate end devices, edge servers, and the cloud to form a hierarchical IoT architecture [6]. We depict this system in Figure 1. The MEO can manage the available computing/storage/network resources and the life cycle of the application [13]. The MEO chooses the target server (i.e., local edge server/neighboring edge server/cloud server) for each requested task.
On the other hand, one of the main characteristics of a heterogeneous IoT environment is rapidly changing uncertain systems. Therefore, it is difficult to provide accurate mathematical models. Recently, machine learning techniques enable edge devices to maximize a predefined objective such as QoS and cache hit ratio (CHR) after they monitor their environment by learning and predicting various network features, such as channel dynamics, traffic patterns, and content requests [14]. However, the wireless edge networks have complex dynamics, such as channel variations, user mobility, user preference, and content popularity. The time complexity of solving optimization problems increases exponentially [14,15]. Especially, when the frequency of user mobility is high, it leads to increased latency, higher energy consumption, and inefficient resource utilization [16].
Fuzzy logic has recently become a good alternative to controlling real-world imprecision. Because the fuzzy logicbased approach has lower computational complexity than other decision-making algorithms, it is significant for solving online and real-time problems. The decisions based on fuzzy logic are made considering edge and cloud computing characteristics [17]. Considering the parameter link delay and the signal-to-noise ratio, the fuzzy-based offloading ratio is calculated [18]. By using network information and matching it with the requirements received from applications, a fuzzy logic-based orchestrator can execute processes ;so, the target mobile edge host can process applications [12]. In a fuzzybased MEO, the input variables chosen have a significant effect on system task execution. Additionally, the role of a fuzzy-based MEO is to choose a target server, which can be a local edge server, a neighboring edge server, or a cloud server, based on the profile of an incoming application task and mobile edge computing characteristics. In [19], Nguyen et al. propose the collaborative task offloading scheme between mobile device-edge-cloud servers by adding the WLAN delay as a fuzzy input parameter. Meanwhile, Hossain et al. [20] use the neighboring MEC VM utilization as the fuzzy input parameter. However, the existing system leads to unbalanced loads among edge servers when the system is overloaded. Moreover, it also did not study how to find the best neighboring edge server where to offload the task is beneficial and scale its capacity when it comes to a large number of devices in order to avoid task failures and to reduce service time.
In this paper, we investigate the MEO system and the fuzzy-based MEO using different online workload balancing algorithms. We propose a new algorithm, achieved by considering user configuration requirements, server geographic locations, and available resource capacities. Our proposal allocates the proximate server when the user requests arrive at the MEO in real time. The key contributions of this paper are as follows.
(1) We investigate and compare fuzzy-based MEOs using different online workload balancing algorithms. Simulation results suggest that the fuzzybased MEO should use a worst fit online workload balancing algorithm (2) We propose a new approach to not only find the proximate server but also to balance the resource capacities of edge servers. By keeping track of server geographic locations and available resource capacities, our proposal can find the proximate server when the user requests arrive at the MEO in real time (3) Simulation results demonstrate the following. (1) Our proposal has better performance results, compared with the baseline online workload balancing algorithms, in terms of failed task ratio, average service times, and processing times for augmented reality and healthcare applications. (2) Compared to the existing algorithms, when the system load is not high, our proposal has results approximately the same as the fuzzy approach [12] and higher than competitor approach [17]. When the system is overloaded, our proposal outperforms both of these algorithms in terms of the failed task ratio (4) Our proposal allows the huge number of requested tasks from IoT applications that are processed when the system is overloaded. By using our proposal, the number of failed tasks is reduced. Moreover, our proposal can process more AR and healthcare application tasks than competitors The remainder of this paper is organized as follows. Section 2 reviews the related works. Section 3 describes the online workload balancing algorithm. Section 4 provides a performance evaluation of the simulation results, followed by conclusions and proposed future research in Section 5.

Related Works
The mobile edge orchestrator, which is the central function in the MEC system, maintains all information on MEC, such as mobile edge geographic locations, available resource capacities, and the profiles of the applications creating the demand [21]. Additionally, the MEO is basically used to manage and control this information to make the offloading decisions for incoming tasks [22]. The MEO uses the requirements received from the application task for the decision on where to offload the process and on which target server. The basic decision method is made by matching the resource 2 Wireless Communications and Mobile Computing information with the application requirements so the MEO can select the target ME host [23]. Two major actions in the MEO are to monitor uncertain systems and determine the target virtual machines (VMs) for offloading the requested tasks. Many approaches have been proposed for solving the workload orchestration problem. The decisions made can be based on various conditions, such as a combination of network latency, data transfer size, and remote server capabilities [24]. Based on the WLAN, the metropolitan area network (MAN), WAN communications, and the requirements of the application task, the decision can be made based on the edge/fog computing infrastructure [25]. However, these approaches do not take network congestion into account.
Baktir et al. [26] proposed a server centric method by using the orchestration of software-defined networking on the edge server. In an edge computing system, Hegyi et al. [23] optimized placing the components of IoT applications, and Karagiannis and Papageorgiou [27] proposed placement based on the partitioned shortest path method to solve the placement optimization problem. In fog computing infrastructures, Santoro et al. [28] proposed efficient and optimized use of the infrastructure while satisfying the application requirements, naming the approach Foggy. On the other hand, Bittencourt et al. [25] proposed an edge-ward placement algorithm where modules of the same application are grouped to be placed in the same device. This algorithm was designed according to three different policies: (1) delay-sensitive versus delay-tolerant, (2) resource management, and (3) allocation. The results in this paper conclude that scheduling strategies can be designed to cope with different incoming application classes to take advantage of both fog computing and cloud computing characteristics [25].
To apply a fuzzy logic-based approach to an MEO system, Sonmez et al. [12] proposed an automated management system to solve online problems. The script input parameters under consideration were WAN bandwidth, the length of the incoming tasks, VM utilization on the edge server, the delay sensitivity of the related tasks, MAN delay, local edge VM utilization, and remote edge VM utilization. The offloading decisions made consisted of local, neighboring edge, and cloud servers. In our work, we consider online balancing algorithm to improve the better service delay performance for IoT application when the system is overloaded.
To improve the system performance, online workload balancing algorithms [29,30] operate in coordination with the MEO to ensure the workloads are distributed to the proximate server. A classic problem in the context of online computation is known as bin packing. Li et al. [31] concluded that certain bin-packing strategies were considered for renting  Wireless Communications and Mobile Computing servers in the cloud. Some approaches have been studied for connecting the bin-packing problem to the centralized scheduling problem [29,30,[32][33][34][35][36][37].
Sambit et al. [38] proposed a taxonomy for load balancing algorithms in the cloud. In online mode, some approaches were proposed, such as opportunistic load balancing (OLB) [39,40], minimum execution time (MET) [41], minimum compilation time (MCT) [42], and simulated annealing (SA). In OLB [39,40], the requested tasks are arbitrarily allocated to virtual machines based on the execution time of each task. On the other hand, based on the lowest execution time, MET [41] allocates each task to a VM. Considering both ready-to-execute time and the expected execution time of the tasks, MCT [42,43] allocates tasks based on the shortest completion time. Based on a probability distribution, Moschakis et al. [44] proposed to model the workloads of IoT applications in order to maximize the performance of scheduling such applications. In [45], to formalize the load balancing, the authors used standard deviation which can measure the dispersion of a set of values. The lower standard deviation indicates that the values are closer to each other. In this paper, the authors use nondominated sorting genetic algorithm III (NSGA-III) to search for a set of edge server (ES) placement with low latency and a balanced workload.
Based on these approaches, we summarize five basic online workload balancing algorithms: first fit, best fit, random fit, worst fit, and next fit. The operations, advantages, and disadvantages of each algorithm are discussed in Table 1. On the other hand, the fuzzy logic-based approach is significant for solving online, and real-time problems because it based on fuzzy logic are made considering edge and cloud computing characteristics [46]. Additionally, the role of a fuzzy-based MEO is to find a target server based on the profile of an incoming application task and mobile edge computing characteristics [19]. By combining the fuzzy logic-based approach and our proposed workload balancing algorithm, our proposal gets better performance results when supporting fuzzy-based MEO in IoT environments, as discussed in the next section. We will summarize our contributions between this paper and the existing Fuzzy-based MEO approaches in Table 2.

Online Workload Balancing Algorithm for a Mobile Edge Orchestrator
In the MEO, fuzzy logic is suitable for addressing technical challenges. One of important challenges is known as changing uncertain systems, i.e., CPU utilization on a virtual machine (VM) and the bandwidth fluctuation. We will explain why fuzzy logic is an excellent alternative for handling real-world imprecision in MEO.
(1) Based on well-understood principles and the use of imprecise information provided in the form of human language, it can handle uncertainty in predictable environments [47] (2) Fuzzy logic-based approaches have lower computational complexity than other decision-making algorithms [48] Moreover, under rapid changes in the various workloads, the existing decision-making algorithms based on multiconstraint optimization with a mathematical model need to know details about resource utilization by the server and details on the network's condition. In contrast, fuzzy logic allows for the consideration of multiple parameters in the same framework.

Fuzzy-Based Mobile Edge Orchestrator Preliminaries.
The main objective of the fuzzy-based mobile edge orchestrator is to handle the multicriteria decision-making process to obtain crisp output by considering multiple parameters in Table 1: Five baseline online workload balancing algorithms.

Name Operation Advantages Disadvantages
First fit [49] Allocates VMs in the order of requested tasks from applications sent from a mobile user to the MEO.
The fastest algorithm.
The remaining unused memory areas are wasted if it is smaller than the required resource capacity of the requested tasks. Consequently, a larger resource capacity for the application is not accomplished.
Best fit [41] Allocates the VM that has the lowest resource capacity.
Memory utilization is better than first fit.
(i) Tends to full up a tiny resource capacity (ii) Shows several variances in load across the machine [38] Worst fit [42,43] Assigns a VM that has the highest resource capacity and continues in a similar manner.
Reduces the waste of resource capacities in the overall system network.
When the system is overloaded, a larger resource capacity for the application cannot be accommodated because the highest resource capacity is already occupied.
Next fit [50] Similar to first fit, but a pointer parameter in the next fit method points to the next VM to be allocated.
The same competitive ratio for each VM.
When the system is overloaded, the performance is poor [34].
Distributes the requested tasks evenly based on resource capacities.
(i) Cannot provide for different computational complexities as a requirement of the applications (ii) Poor make span when multiple objectives are considered simultaneously in a cloud environment [38] 4 Wireless Communications and Mobile Computing the same framework. First, we explain the global fuzzy logic system. The fuzzy logic system (FLS) consists of four main components: the fuzzifier, the rules, the inference engine, and the centroid defuzzifier, as shown in Figure 2. The fuzzification step is used to transfer crisp input to the fuzzy input set by using membership functions. The defuzzification step is used to again transfer from the fuzzy output set to crisp output, a.k.a., the offloading decision. We summarize the operation of the FLS as follows [12,17,18,51,52].
(1) Fuzzification is used to transform crisp input sets to fuzzy sets. Note that a crisp set is converted to a linguistic variable (LV) for each indicator. The LV is decomposed into linguistic terms (LTs). We use a membership function to quantify an LT (2) Fuzzy input sets are introduced to the inference engine used to evaluate and combine the fuzzy rules in order to make an inference (3) The resulting fuzzy output, called a crisp output value, is processed in the defuzzification step by using a centroid defuzzifier method For example, we consider a fuzzy-based mobile edge orchestrator as being used to make offloading decisions for incoming tasks. Two offloading decisions are designed based on the crisp output value. To do that, the system will determine the input variables that have a significant effect on performance results. These variables will be sent to the fuzzy-based mobile edge orchestrator, and they play a decisive role in the task-offloading decision process. According to the profile of the application creating the demand, we take the MAN delay, local edge VM utilization, and remote edge VM utilization to provide formal input variables. We describe the role of these crisp input variables as follows.
(1) MAN delay (denoted by a): If the MAN resources are congested, the local edge server is chosen [12] (2) Local edge VM utilization (denoted by b): Many tasks fail because of a limited resource capacity on the local edge server; therefore, local edge VM utilization has to be considered  Crisp input variables: Based on the characteristics of the requested task, we define three of the abovementioned input variables for the fuzzy-based mobile edge orchestrator, as shown below: Linguistic variables: The linguistic variables of fa, bg are described as low, medium, and high, abbreviated as L, M, and H, respectively. In addition, L (light), M (normal), and H (heavy) are used for linguistic variables fcg.
Membership function: In our model, we use the most commonly used triangular form representing the membership function. The values used in membership functions for fa, b, cg were tested by Sonmez et al. [12]. The shape of the membership functions for all crisp input variables is depicted in Figure 3.
We associate a grade with each linguistic term, and the crisp value is transformed into a fuzzy value in the fuzzification step by using the following membership functions: Fuzzy rules: A simple IF-AND-THEN rule, a.k.a., hypotheses followed by a conclusion, is used to conduct the fuzzy rule set [53]. To determine the fuzzy rules, we vary and find empirically fuzzy rule sets. Among the different fuzzy rule sets, we chose the best rule combination in the computational experiments [12]. The number of fuzzy rules is n = 3 4 = 81 based on four membership functions with three linguistic terms. Table 3 shows example fuzzy rules found empirically for the placement problem.
In the aggregation step, we use minimum (min) and maximum (max) functions based on the rule set chosen in the fuzzy rule step. We calculate a fuzzy value for selecting the mobile device, the local edge, and the candidate edge server as follows: where n is 81, and the min functions are based on the fuzzy rule step. For example, in Table 3, the min functions are given as where m, n, l are the crisp input parameters for the fuzzy inference system. Defuzzification: In the defuzzification step, according to Mendel [53], the centroid defuzzifier method, which is used to calculate the inference, achieves the center of gravity (COG) of the area under the curve, as shown in Figure 4. It is calculated as where i is a local edge and a remote edge. After applying the centroid defuzzifier, a crisp output value, ω, is in the range ½ Similar to the fuzzy-based mobile edge orchestrator [12], our system also has a two-stage FLS to find the target server: a local edge server, a neighboring edge server, or a cloud server. To find the best neighboring edge server to improve the number of requested tasks that are processed successfully, we add a function named the online workload balancing algorithm, as shown in Figure 5. The first stage was discussed above according to three input variables: MAN delay, local edge VM utilization, and remote edge VM utilization. Consequently, two offloading decisions are made for a local edge server and a remote edge server. The second-stage operation is on four input variables: WAN bandwidth, the length of the incoming task, VM utilization on the edge server, and delay sensitivity of the incoming task [12]. These input variables are defined as follows: (1) WAN bandwidth: If the WAN communication delay is higher than the application QoS requirement, the offloading decision cannot be to a cloud server [12] (2) Task length: The execution time is calculated based on the task length. A complicated task should be offloaded to a cloud server to reduce execution time (3) Average VM utilization: This provides the remaining computational capacity. Congestion is defined as an average edge VM utilization that is greater than a set threshold level [12] (4) Delay sensitivity task: We determine the tolerance of the task if it takes a long time due to network congestion or server utilization levels The centroid for the defuzzification process.

Wireless Communications and Mobile Computing
The decisions in the second stage are about the neighboring edge server and the cloud server. The mobile user will send the profile of the requested task, including its character-istics (such as the length of the task, the delay sensitivity, and the required computation resources). By using network information (i.e., VM utilization on the edge server, WAN

Result:
The best neighboring edge server, O i . Initialization:; L j : List of neighboring edge servers to edge server j; i: Mobile device uploading an application task to connecting edge server j; k: Edge server close to edge server j; ðx j , y j Þ: The geographic location of edge server j; ðx k , y k Þ: The geographic location of edge server k; for k ∈L j do Find k satisfy constraint (13c); Find VM resource capacity on edge server k to satisfy constraint (13c); ifThere is available resource capacitythen O i = cloud; break; end end ifThere is no available resource capacitythen O i = cloud; else L j =L j \ fkg; end Algorithm 1: Online workload balancing algorithm. 8 Wireless Communications and Mobile Computing bandwidth, and MAN delay) and matching it with the requirements received from the application, the orchestrator determines the target mobile edge host to process the application.

Online Workload Balancing Algorithm.
Under IoT systems, it is necessary to design an edge computing system to handle a dynamic flow of requests that can be efficiently processed. Therefore, the workload orchestration problem must be studied based on both computational and networking resources [12]. Moreover, the MEO needs to scale its capacity with respect to a large number of devices in order to avoid task failure and to reduce service time. To cope with this issue, we propose the online workload balancing algorithm used in a fuzzy-based MEO architecture as shown in Figure 6 in order to improve system performance. The goal of our algorithm is to ensure the workloads distributed to the appropriate servers when the mobile user requests arrive at MEO in real-time scenario.

Transmission Delay at the Edge Servers.
According to the standard practice in computing offloading modeling [5,[54][55][56], the time it takes to transmit the requested task of an application from local edge server j connecting to mobile user i to neighboring edge server k depends on the size of the input data that the task is associated with, b ij , which is given as where R MAN is the rate of transmission in the MAN. The MAN propagation delay is calculated as where s MAN is the wave propagation speed.

Processing Time.
In accordance with the usual representation of processing time in the task allocation problem [5,54], the time needed for a task from mobile user i at edge server j based on the number of CPU cycles required is given as where the k th edge node allocates the i th device at f e k giga instructions per second (GIPS). β i,j is the number of CPU cycles required to compute one-bit data for this task [57].

The Best Neighboring Edge Server Decision Mechanism.
In our model, the MEO will determine whether the tasks will be executed on the edge server or the cloud server based on the abovementioned offloading decision. However, to minimize latency for each task, and to balance the edge servers, the best neighboring edge server is chosen. First, the total delay of a task from the i th device served is We aim at minimizing the delay of each task and attaining a balance between the resource capacities of edge servers that are managed by the MEO. The optimization problem can be formulated as follows: where L j is a list of edge servers neighboring edge server j that are not used at the current time. The solution of the optimized problem in (15) can easily be found as follows. First, based on constraint (17), the MEO will look up the neighboring edge server to edge server j that is close to j based on geographic location. Second, the MEO will check the available resource capacity of each VM in edge server k. If constraint (18) can be satisfied, the task requested by the application will be processed by edge server k. If not, the MEO will choose the next-best neighboring edge server to edge server j based on geographic location. If there is no available resource capacity to process the task requested by the application, the task will be uploaded to the cloud server.
The algorithm takes the user's specified constraints into consideration, including server geographic location and the available resource capacity. The details of the online workload balancing algorithm are shown in Algorithm 1.

Performance Evaluation
To evaluate our proposal, we used a realistic model known as the multitier edge computing infrastructure, named Edge-CloudSim [58]. The simulation parameters are presented in Table 4. According to [12,58], the CPU speed of VMs per edge and cloud servers is 10 GIPS and 100GIPS, and the edge and cloud servers operate 8 VMs and 4 VMs when the system can handle over 500 mobile devices, respectively. Moreover, the characteristics of the potential applications vary according to the requirements. In this paper, we consider different incoming tasks from four applications: augmented reality, healthcare, compute-intensive, and infotainment applications. They have different profiles in terms of task arrival distribution, delay tolerance, and task size, as shown in Table 5. The interarrival times and task sizes are exponentially distributed random variables [12].
According to [12,58], to simulate more realistically, we use four different application types in our study. First, an augmented reality application on Google Glass is presented in [59]. Second, the infotainment application is discussed in [47]. Third, a health application that uses a foot-mounted inertial sensor to analyze the walking pattern of the users is studied in [60]. Finally, the compute intensive application is discussed in [12]. All applications are set up in EdgeCloud-Sim simulator [58].
In our simulations, we assume that the mobile devices offload tasks that belong to a predefined set of application categories. The user wearing the smart glass offloads captured pictures to the remote servers, which provide face recognition service. For some time, it collects data and sends it to the edge services. The user with the foot-mounted inertial sensor offloads sensor data to the remote servers which provide a fall risk detection service. The AR application requires high CPU computation, a small amount of data to download, and a bigger amount of data to upload [58]. Meanwhile, the infotainment application requires less CPU power compared to the augmented reality application. To process these tasks, infotainment and compute-intensive applications send their tasks to the remote servers, which provide related services.
In the performance evaluation, we discuss three main results from different perspectives. We demonstrate the following. (1) The online balancing algorithms have different effects according to each method. We suggest that random fit, worst fit, and next fit balancing algorithms have better performance in terms of average failed task ratio, as shown in Figure 7. (2) We compare our proposed fuzzy-based FuB algorithm with five basic methods in terms of average failed task ratio, service time, and processing time, as shown in Figure 8. (3) We compare our proposal with the existing methods, as shown in Figure 9. Simulation results show that our proposal outperforms in terms of average failed task ratio, service time, processing time, and VM utilization when the system is overloaded.

A Comparison between Nonbalancing and Balancing
Algorithms Used for a Fuzzy-Based MEO. To emphasize why the fuzzy-based MEO uses a balancing algorithm, we compared it with nonbalancing algorithms in the same environments. We considered an important metric in system performance: the failed task ratio. This metric is calculated by computing the ratio between the number of failed tasks and the total number of tasks. Figure 7 shows the failed task ratio for First fit [49], Best fit [41], Worst fit [42,43], Next fit [50], and Random fit [39,40] algorithms. When the system load is under 600 mobile users, the results of best fit and first fit balancing algorithms are worse than nonbalancing algorithms, whereas random fit, worst fit and next fit have the same results. All balancing algorithms were better than nonbalancing algorithms when the number of mobile users was higher than 600. We classified balancing algorithms into two groups according to performance results: group A (best fit and first fit) and group B (random fit, worst fit and next fit). Moreover, group B showed performance results that were better than group A, as shown in Figure 7. Therefore, we suggest that group B has to be considered for the fuzzy-based MEO.

A Comparison of Fuzzy-Based MEO Using Different
Workload Balancing Algorithms and Our Proposal. In this section, we compare our proposed FuB algorithm and the two abovementioned groups in terms of average failed tasks, service times, and processing times based on all application types and for different application types, as shown in Figures 8, 10, and 11.
First, based on all application types, Figure 8 shows the average failed tasks, service times, and processing times. Our proposed FuB algorithm has a higher number of successfully processed tasks than groups A and B, as shown in   Because group A has fewer successfully processed tasks than group B and the FuB algorithm, as seen in Figure 10(a), the service time for these successfully processed tasks is lower. Moreover, the worst fit algorithm showed the best performance, compared with group B in terms of the failed task ratio. The worst fit and FuB algorithms had better perfor-mance than both groups A and B. Therefore, they take longer while having lower failed task ratios than group A. Figure 10(d) clearly shows the service time for each application type. According to Table 5, the task lengths generated are small, medium, big, and very big, for healthcare, AR, infotainment, and compute-intensive applications, respectively.
Similarly, Figure 8(c) and Figure 11 show the processing times for the application tasks. In all cases, group A showed longer processing times because the requested tasks were allocated to the first server with free resources or the smallest free-resource capacity. It makes for a long queuing time at the edge server [54]. Worst fit and FuB algorithms have approximately the same performance results when the system load is low. However, when the system is overloaded, if  big and very big tasks for infotainment and computeintensive applications are processed on the edge servers, healthcare and AR applications can be offloaded to the cloud server, Therefore, the processing times for healthcare and AR applications were lower with the worst fit algorithm.
The following concluding remarks can be made based on the experimental results in Figures 8, 10, and 11: (1) The FuB algorithm shows the lowest failed task ratio compared with the five basic balancing algorithms used for a fuzzy-based MEO. Among the five basic balancing algorithms, worst fit shows the best performance results in terms of the failed task ratio (2) Best fit and first fit take the longest service time and processing time because the requested task will be allocated to the first available server with resource capacity. Random fit randomly chooses the server with available resource capacity that meets the requirement of the requested process, whereas next fit starts searching for available resource capacity from where it left off, not from the beginning. Two methods have the highest failed task ratios (Figure 8(a)); therefore, the processing and service times for these successfully requested tasks are lower (3) If the system has a low number of mobile users (e.g., fewer than 1600 devices used), our proposal and worst fit have approximately the same results. However, when the system is overloaded, according to Algorithm 1, using the Fub algorithm, the healthcare  13 Wireless Communications and Mobile Computing and AR applications can be offloaded to the cloud server to improve service time and processing time (4) To compare our proposal to other existing algorithms, the existing fuzzy-based MEO will use the worst fit balancing algorithm. This is discussed in the next section

A Comparison of Our Proposal and Existing Benchmark
Schemes. In this paper, we compare two benchmark schemes that use the fuzzy logic approach for an MEO in IoT environments, as follows: (1) Fuzzy approach [12] (named fuzzy): The fuzzy-based approach considers both computational and communication resources and makes a decision. The MEO finds a target server: either a local edge server, the nearest neighboring edge server or a cloud server (2) Competitor approach [17] (named comp): This approach utilizes fuzzy logic to decide whether to execute tasks on an edge server or a cloud server Figure 9 shows the average failed tasks, service times, and processing times based on all application types. When the system is overloaded, our proposal has the lowest failed task ratio, as shown in Figure 9(a). The FuB balancing algorithm provides the proximate server in real time by considering user configuration requirements, server geographic locations, and available resource capacities. Fuzzy uses the worst fit balancing algorithm to distribute tasks to the proximate server that has the largest available free resource capacity, so that the resource capacity left will be big enough to be useful.

Wireless Communications and Mobile Computing
On the other hand, comp prefers offloading the tasks to the edge server. Therefore, the failed task ratio in comp was the highest, compared to FuB and fuzzy algorithms, because of the impact of MAN congestion on task failures. Consequently, the service time and processing time for tasks in comp were lowest when the system is overloaded, as shown in Figures 9(b) and 9(c). Figure 12 shows the processing times for each application type. The task lengths generated are small, medium, big, and very big, for healthcare, AR, infotainment, and computeintensive applications, respectively (see Table 5). Although comp prefers offloading tasks to the edge server, the healthcare, AR, infotainment, and compute-intensive applications will be processed on the edge server. Hence, when the number of mobile users is less than 1400, the processing time in comp was the best. When the number of mobile users is too high, the number of failed tasks in comp is high. Consequently, to execute fewer requested tasks than FuB and fuzzy, comp had the lowest performance. Now, we compare our proposal (FuB) and fuzzy. For healthcare and AR applications when the system load is not high (e.g., under 1400 mobile users), FuB and fuzzy have approximately the same results. Nevertheless, when the system is overloaded, by using Algorithm 1, FuB can support the proximate server in real time by considering user configuration requirements, server geographic locations, and available resource capacities. Accordingly, FuB provides better results, as shown in Figures 12(a) and 12(b). On the other hand, the big and very big task sizes of infotainment and compute-intensive applications are mostly executed on the cloud server, which provides better results because a VM running on a cloud server is very powerful. Inevitably, the processing times for FuB and FuB have approximately the same results and are lower than comp.

Conclusions
In this paper, we proposed the online workload balancing algorithm support for a mobile edge orchestrator. Our proposal can support the proximate server when user requests arrive at the MEO in real time by considering user configuration requirements, server geographic location, and available resource capacity for the online algorithm. According to simulation outcomes, our proposal provides better results when the system is overloaded, compared with the existing algorithms. For healthcare and AR applications, our proposal provides not only the lowest failed task ratio but also the shortest processing time. In future work, we will apply a genetic algorithm to task scheduling in order to improve quality of service.

Data Availability
The data used to support this work are available from the corresponding author upon request.

Conflicts of Interest
The authors declare that they have no conflicts of interest.