Proactive Network Fault Management for Reliable Subscribed Network Slicing in Software-Defined Mobile Data IoT Services

Proactive network solutions (PNS) become the precise management and orchestration (MANO) in the applied artificial intelligence (AI) era. The PNS proposed to invent future mobile edge communications by predicting the fault networks for reliable slicing configurations. Furthermore, federated learning (FL) systems have been appealed to apply for critical mobile data privacy of the Internet of Things (IoT) services. Therefore, FL-based IoT communications need a precise PNS to pretend the network failures to maximize the model inference and improve end-to-end (E2E) quality of services (QoS). This paper proposed an adopted software-defined network slicing (NS) for IoT communications based on network failure prediction and resource allocations by utilizing a deep-Q-network approach (DQN). The proposed proactive reliable subscribed network slicing was based on software-defined DQN-based proactive dynamic resource allocations (SDQN-PDRA) for adaptive communication configurations. The experiment showed that the proposed approach enhanced the significant outcomes of stability, reliability, convergence time, and other communication QoS.


Introduction
In next-generation (NG) communication technology, the mitigated mobile edge computing (MEC) from the remote cloud, called mobile cloud computing (MCC), is intended to empower fronthaul computing resources and enhance NG infrastructure as a service (IaaS) for overcoming heterogeneity novelty applications, including Internet of ings (IoT), heterogeneous IoT (HIoT), Internet of healthcare things (IoHT), and Internet of Vehicles (IoV), and especially, for the time critical-communications [1][2][3]. MEC plays an essential role in enabling local services for 5G perspectives, which aim to provide agile response services for user devices with ultra-dense new radio (NR) services for massive user terminals. According to the enlargement of edge network infrastructure, intelligent resource management and orchestration (MANO) towards autonomous network con gurations have become the critical research areas [4]. Additionally, network systems self-organizing networks (SON) must be enhanced because autonomous networks can be empowered by adopting arti cial intelligence (AI) algorithms. Deep learning (DL) models were introduced to handle and improve SON perspectives, especially in distributed network areas.
DL approaches have contributed e ectively to the largescale complexity of network datasets in terms of classication, recommendation, and prediction problems [5]. DL can be applied for e cient MANO heterogeneous network resources for these reasons. Especially in IIoT applications, various IoT devices will generate a large-scale network dataset. For example, in the federated learning (FL) based IoV paradigms, each vehicle has its data cloud to store and compute privacy constraint information and share its training model and model parameters to the distributed cloud server for aggregation. e training model will be shared between MEC servers in the vehicle edge networks (VEN) [6][7][8][9]. Because of the high-speed movement of the vehicles, the exchange of information between MEC servers to the classification phase for distinguishing the distinct network conditions (e.g., MEC loading, traffic loading, path delay, etc.), which are essential for SDR rulemaking. We deploy DQN for autonomous resource management and to minimize network load fluctuations. SDN controller invests the DQN model for optimal Q-value selection, by implementing DQN in SDN architecture to explore the most appropriate action for allocating the resource to maintain the optimal network loading metrics.
We provide the E2E evaluation metrics of our proposed SDQN-PDRA with various approaches in three communication aspects, including the model convergence reliability of FL in IIoT, network stability, and QoS analytics. e intelligent network computation and configuration were based on proactive network solutions (PNS). e SDN controller was considered to handle the loading predictions and adjustment proactively.
Continuing of the manuscript is organized as follows. Section 2 presents the related work and IIoT communication system and issues. In addition, our solution is described in Section 3. e experiment and numerical evaluation results with detailed interpretation are given in Section 4. Finally, Section 5 presents a conclusion and future work.

Related Work
Each WSN is attached to a personal edge server with a sufficient local cloud system to perform data storing, local training model, and other significant computations (see Figure 1). Whenever the local edge has insufficient resources to operate heavy traffic, there will be a high network delay with the congestion caused by the network failure [27,28]. e FL-based system uses model transferring instead of raw data sharing [29]. FL-based network architecture reduces the amount of traffic over the network since the raw data captured from the sensor network is stored in the local cloud and performs local training [30]. e FL-based communication can be described in three layers. First, the data gathering and local training layer required to satisfy KPI obligations in terms of data integrity, clearance of the dataset, and the sensor can be sensing overdetection information that will not be utilized for model evaluation due to the PNS required reliable model inference. Moreover, some of the computation and decisions must be made inside the personal edge service to support an innetwork processing conducted based on band communication over Ethernet or wireless communications.
However, the synchronization processes between local and global entities will be frequently made, while the vast local models require aggregation transfer to edge servers. At the same time, the heavy computation is not suitable for computing inside a local server, especially the missing data for model decisions. Moreover, due to the fast speed and high IoT networking, the joining radio networks consist of high failure ratios that obligate handling for uRLLC. Additionally, the optimal remote radio head (RRH) recommendations will diminish the failure ratio of joining the radio access network (RAN). e aggregation server shares its model with the server in charge of continuous computing in the handover process whenever the alternative is mandatory. In the process of sharing, heavy tra c will be generated, and new route installation will be obligated.
Moreover, the rapidly converged network will become a key challenge issue when each router has a large-scale network database that takes long periods of computing processes. erefore, PNS (advanced computation) and route installation in time critical IoT reduce delays for massive routing decisions. Additionally, the rapid routing will utilize SDN architecture since the routing computation and installation will be conducted in the control plane (CP), while the data plane (DP) takes a function of data forwarding based on the installed route [31][32][33][34]. us, the computation can be wholly separated and performed in advance in the CP.

System Model.
e system model is based on the ENI architecture by converging the three primary contributors, including DQN, caching, and SDN controller. e network conditions in terms of delay, congestion window, and resource limitation are considered network loading parameters. e local server has been attached to the IoT system in federated IoT to store the sensed data from various intelligent sensor devices. e local training is conducted by splitting the local dataset between each client into minibatches of size db , which are included in the set c db 1 , db 2 , . . . , db c }. e local trained and updated models are sent to the global servers for aggregation which can be modeled as follows: While w update is the model parameter update from local IoT clients w c update ∈ w 1 update , w 2 update , . . . , w c update ., local data minibatches of total client c ∈ db 1 , db 2 , .., db c , and MSE is the Mean Squared Error representing the loss function for deep neural network (DNN). e local client transmits the updated model w c update over the wireless networks to the aggregation server. e global server collects the up-to-date model w c update from various aggregation servers for model accumulation. e global server will send the average global models to the local client. e global model w t G can be modeled as where w t i is the updated model at each time step t. w t+1 G is the global update summation at time t + 1; the increasing Rounds Server (Global model) Scienti c Programming number of the round-trip time (RTT) communications from local to server will boost the global training accuracy. However, the number of RTT model communications will reduce the E2E transmission QoS. Generally, RTT of FL communication, communicating over the 5G communication technology, consists of poor service identity towards intelligent handling for service level agreement (SLA) that requires additional resource allocations with the high-level priority.

Communication
Overhead. e overall overhead, which reduces the network QoS, can be expressed based on the M/M/c/K queuing system. In each network node and aggregation server, the serving overheads at a particular queue interface can be determined and modeled as M/M/1/K queuing system with the limitation of K capacity of the server. ρ is the serving ratio between the arriving task λ and serving resource μ. For μ > 0, user traffic P 0 and n user's traffic P n in the system can be measured as follows: e mean waiting time Q length in the single individual edge server can be measured as en, with the mean number of user traffic N in the single individual edge, the system can be modeled as e communication overhead will be considered in the computation and communication delays. Furthermore, we denote the communication rate between serving nodes (i to k interface) in wired-based networks with bandwidth as W and transmission power as P, the noise power between the i to k interface as z 2 i,k , and communication channel gain as H i, k . us, the transmission rate from the aggregation server to another global server can be expressed as k ∈ 2, . . . , K + 1 { } and i ∈ 2, . . . , I + 1 { }, and i ≠ k; then, the transmission rate between i and k nodes can be expressed as

Our Solution
SDN-based DQN will allocate resource in advance for feasible flow handling for every predicted loading metric DN-based DQN appraoch performs the allocation processes and optimal parth selections (see Figure 2). e network loading metric will be under the defined threshold in the optimal state. e forwarding process of the information from the local to global servers is according to the predicated and resource adjustment schemes. SDN controller establishes the routing policy according to the comparisons of real-time observation metrics; the path with minimum metric is considered for feasible routing path. However, in the case that loading metric at each possible routing path has loading metrics more than the defined threshold (nonoptimal state), the SDN controller will be adjusted resource by attempting to query the optimal action a t i,k until the observed condition reaches an optimal state. e SDN controller will select the optimal gateway to ensure the transferring of updated model parameters of local devices and the aggregated model download from the aggregation server (see Algorithm 1). e ENI infrastructure will not be utilized in steady situations, and the SDR will be coordinated to handle the model sharing based on the current route. is method will be helpful in the stability of communication, while DP communications share similar loading metrics. However, the caching metric will not be utilized for route configuration in the frequently fluctuating communication statuses since the feasible optimal aggregation server is obligated to be defined. Furthermore, the IIoT applications are unsuitable for utilizing the restriction scheme, which restricts sending resources during the heavy loading network, increasing the waiting time at the source devices.

Joined Environment for Resource Adjustment.
To evaluate autonomous resource allocations, the environment with varied network stability between 0 and 255 represents the loading metric randomly. A variety of entities can accomplish the IoT environment in terms of data plane conditions called state space s t and the SDN controller considers taking action a t at each time-space t based on the optimal policy π * , which provides the maximum Q-value.
e detailed description is explained in the following.

State Space.
At each time-space t, the SDN controller maintains the below information, which can be affected by the network conditions.
(i) W i,k is the bandwidth of the wired between ith to k interfaces (ii) I i,k is the number of the assigned tasks. (iii) N is the number of the user requests at the time t.
(iv) Q is the average number of queue lengths at time t.
(v) TR 1,k is the transmission rate of the wired between serving entity at the time t Loading metrics are at interfaces between R ∈ 1, 2, 3 . . . , R { }and M ∈ 1, 2, 3, . . . , M { } network devices at the time t ∈ 1, 2, 3, .., τ { } . So, the above information can be the entities of the system state space S, that is, , due to the communication bandwidth at i and k interface in the wired link, will be sharing the same metric for the system state information s t ∈ S and can be expressed as

Action
Space. e agent takes a signi cant role in deciding the optimal action for I i,k ow requests according to the network states. In this action space, the agent will consider optimal action to meet the de ned state information. We denote A as our global action space at each timespace at the time τ ∈ 1, 2, 3, .., t { } , and a τ i,k is the action tacking at the i and k interfaces between R ∈ 1, 2, 3 . . . , R { } and M ∈ 1, 2, 3, . . . , M { } network devices. While A denotes the global action space, a t ∈ A performs at time slot t, a t ∈ a 1 i,k , a 2 i,k , a 3 i,k , . . . , a t i,k ; then a t i,k is varied corresponding to the state space information.

Reward
Calculation. In our system, the the agent randomly selected action at each time t, and the reward will be o ered based on the network state information. e feedback from the environment trains the agent to determine optimal action, and the optimal action a t performs the optimal state. After a t was taken, the reward r t was immediately provided.
s optimal state (t) 0<i≤t,∀r i 1 Based on equations (9)-(11), ∀r ∈ R t ; the rewards metric r represents the good and bad states at each timespace. Further, the reward accumulation presents the optimal network condition for entire communication periods.

Optimal Policy.
e reward R t metric is corresponding to the system environment status based on the optimal action a t selection.
e agent explores the optimal action for reducing the network loading metrics. Based on the experiences, the agent can select the optimal action a t at the similar network states based on the experiences.
e agent chooses action according to the policy π, and the optimal policy π * can return the maximum sequence corresponding reward R t . To maximize the Q-value, the action required the optimal policy π * , and our Q-value can be obtained by utilizing Bellman optimal equation e optimal Q-value Q * (s, a) at state s and action a with the policy π will correspond to the discount parameter c, and the weight in 0 ≤ c ≤ 1 will re ect the agent to put on the future rewards. Moreover, for any main network θ and target  Scienti c Programming network parameter θ ′ , the optimal policy π * (a|s) that performs maximum Q-value in paired state s and action a is denoted as From (12), the agent explores the action policy in the obvious state to find the maximum Q-value. en optimal Q-value is relative to the summation of rewards and discounted parameter multiply of maximum Q-value of next state s ' and action a ' , as depicted in (14). However, when c is close to 1, the system will suffer from the computation delay since the agent has more chances to discover the optimal Q-value. Consequently, our target network y i will be written as en, the loss between target and actual network parameters of k minibatches can be expressed as X ∈ x 1 , x 2 , . . . x n denotes n of general features as and Y ∈ y 1 , y 2 , y 3 . . . . y m denotes m target features in the global dataset required for client and server (1) Initialize the synchronous c, n epoch , w update parameters, α, w g for aggregation server (2) Ensure the optimal sharing path between client and server, and the routing path (see Algorithms 2) (3) [Aggregation Server] (4) for each epoch in range (n epoch ) (5) Select lowDNN() for c clients (6) Aggregating the model w g for the next epoch by using the FedAvg algorithm [35]. (7) end for (8) [Client Server] (9) for each db c in (Data) do (10) Input α, w g , and w update parameters (11) Define class lowDNN(self, α, w g ): (12) for each client in c do (13) w c update � w g − αΔMSE(w c update ) (14) end for (15) end for ALGORITHM 1: Exchange model between client and server.
(1) Initial the main, target parameters, and replay buffer, θ, θ ′ , and D, respectively (2) Define N number of episodes (3) for each step in the episodes, then (4) State s t observation (5) DQNagent selects action based on the optimal policy π * (a|s) (6) Action a t selection and explore the next state s ′ and obtain the reward R t (7) At each time slot t SDN controller executes the action a t ∈ a 1 i,k , a 2 i,k , a 3 i,k , . . . , a t i,k , (8) if the size of s t , a t , R t s t ′ the size of replay buffer D (9) cache s t , a t , R t , s t ′ into the replay buffer D (10) else (11) Replace queue tail element with the current s t , a t , R t , s t ′ as the FIFO process.  Scientific Programming e SDQN-PDRA is based on the resource adjustment scheme to reduce the loading metrics at each loading period (see Algorithm 2). e SDN controller adjusts the observed loading metrics based on the DQN method. We suppose each serving server can retrieve the global resource from the root or global MEC server in this scenario. e network loading metric will be under the defined threshold in the optimal state. e forwarding process of the information from the local to global servers is according to the predicated and resource adjustment schemes. SDN controller establishes the routing policy according to the comparisons of real-time observation metrics; the path with minimum metric is considered for feasible routing path. However, in the case that loading metric at each possible routing path has a loading metric more than the defined threshold (nonoptimal state), the SDN controller will be adjusted resource by attempting to query the optimal action a t i,k until the network condition reaches an optimal state.

Numerical Evaluation
is section provides a precise description of the used system evaluation parameters for experiment installation in terms of experimental parameters, hyperparameters, and experiment components used to conduct E2E simulation. Furthermore, the numerical result evaluation of the prediction model, model convergence accuracy between client and server in various network conditions, and efficiency of resource adjustment based on SDQN towards the evaluation of communication QoS are performed.

Simulation Environments.
During simulations, the captured delay was utilized to represent the real-world network loading metrics. In addition, the opened source dataset EMNIST [35] was loaded from the federated EMNIST to evaluate the converged network reliability. e EMNIST dataset was sliced to meet the number of clients for testing using the Google platform. Each client has each slice of the dataset (individual dataset) and training model, and the aggregate server utilizes the FedAvg function offered by TensorFlow Federated [35]. Moreover, the E2E evaluations were based on the simulated metrics captured from the NS3 [36] simulations (see Table 1) and the hyperparameters and the experiment components, respectively (see Tables 2 and  3).

Results and Discussion.
e state observation during 1000 episodes in a real-world VEN environment was done and we applied our DQN approach for adjusting the state space metric to meet the determined optimal states threshold, which was set at ∀s t ∈ 0, 1 { } (see Figure 3).. e VEN states in natural communication consist of the average bad network state and good network state counts at 397.724 and 32.276, respectively (see Figure 3(a)). In some cases, the natural VEN environment has 0 optimal and 430 bad states (100% of observed states are in bad conditions). With the c � 0.85, and learning rate lr � 0.02, our proposed resource adjustment reduced the communication overhead in VEN and reached the average optimal state count at 413.1537688 and the bad state counts of 16.84623116 (see Figure 3(b)). With this effective result, in some episodes, the proposed scheme reached 100% (430 optimal states) optimal state handling with 0% (0 bad states) of bad network state counts. Based on the notable metrics, our scheme performed the optimal state up to 96.08227%.
Four conditional simulations of the federated model experiments are conducted to emphasize FL model reliability in actual situation network routing (see Figure 4). e graph presents the remarkable outperformance of the convergence accuracy based on optimal network selected path (ONSP) over the three other possibility routing paths; for example, simple congestion of network chosen path (SCNSP), congestion of network selected path (CNSP), and heavy congestion of network selected path (HCNSP) were simulated to reflect the model reliability in network   environments. e mean training loss metrics in 99 communication rounds of ONSP, SCNSP, CNSP, and HCNSP is at 0.484335, 0.743309, 1.129355, and 1.354101, respectively (see Figure 4(a)). And the minimum loss of the ONSP, SCNSP, CNSP, and HCNSP is at 0.052342, 0.309779, 0.694963, and 0.94137, respectively. Based on the mean loss comparisons, the ONSP has a lessened loss metric compared to SCNSP, CNSP, and HCNSP at 0.64502%, 0.258974%, and 0.869765%, respectively. e model aggregation will rely on the network situations. e congestion environment will lead to a loss in mode sharing between aggregation servers, and it can cause low accuracy in terms of global model reliability.
e E2E model reliability corresponds to the global model accuracy comparison between the ONSP, SCNSP, CNSP, and HCNSP (see Figure 4(b)). e ONSP approach reached the maximum accuracy metric at 0.998873%, while the SCNSP, CNSP, and HCNSP have the accuracy of 0.941435%, 0.925075%, and 0.825702, respectively.
e ONSP enhanced the other possibility routing paths based on the numerical comparison at 0.058974138%, 0.14566234%, and 0.270408005%, respectively. Due to the ONSP delivering the optimal scheduling approach, the network loading metrics have lessened based on the proactive network con gurations. Furthermore, the proposed ONSP approach will also enhance the possibility of saving the computation power in the CP.
In terms of E2E communication QoS metric evaluation, we compared our proposed integrated software-de ned DQN in proactive resource allocations (SDQN-PDRA) with the other approaches, including software-de ned RNN in dynamic routing (SDRDR), software-de ned dynamic routing (SDDR), and software-de ned experience routing (SDER). Our proposed SDQN-PDRA approach illustrated remarkable outperformed results over the other approaches, such as SDRDR, SDDR, and SDER, in terms of the packet drop counts, packet drop ratio, packet delivery ratio, and communication delay, respectively (see Figure 5). e natural network environment consists of limited network loading awareness for improving the routing experience.
us, the local and external data sharing can be elaborated as static and dynamic routing protocols. Likewise, the static and dynamic routing protocols have a weakness when considering selecting the optimal path with e ciency costless. Our proposed SDQN-PDRA provided high accuracy at loading shape network prediction for the lowest cost routing and can reduce the loading state to meet the de ned optimal conditions metric threshold. Our proposed SDQN-PDRA obtained the lowest packet drop counts over the SDRDR, SDDR, and SDER in mean metrics at 69, 120.2666667, 339.0666667, and 737, respectively (see Figure 5(a)).
Our proposed SDQN-PDRA obtained the lowest packet drop ratio over the SDRDR, SDDR, and SDER: 0.01925501%, 0.027260413%, 0.097242413%, and 0.176594407%, respectively (see Figure 5(b)). erefore, the proposed SDQN-PDRA lessened the E2E communication loss between client and server in network environments based on the given graphs. For the E2E communication reliability for the communication drop ratio, see Figure 5(c). Our proposed SDQN-PDRA achieved the highest E2E communication reliability. e average E2E communication reliability of SDQN-PDRA, SDRDR, SDDR, and SDER was achieved at 99.98074499%, 99.97273959%, 9.90275759%, and 99.82340559%, respectively. Based on the presented reliability metrics, our proposed SDQN-PDRA is 0.008005403%, 0.077987403%, and 0.157339397% higher than the communication reliability metric of SDRDR, SDDR, and SDER, respectively. e selected routing path containing high loading metric will su er computation overhead, which can postpone the serving request and increase the waiting time of arrival tra c. Moreover, the network bu er can be limited whenever the serving rate is under the requested tasks. To cope with these issues, real-time loading resource reduction plays an essential role in improving the communication experience. e E2E communication delay between the proposed SDQN-PDRA approach and SDRDR, SDDR, and SDER approaches has been presented (see Figure 5(d)). e proposed SDQN-PDRA effectively reduced the network loading metric and selected the optimal path for route installation, and the communication delay was completely reduced for E2E data sharing. Based on the graphs, the proposed SDQN-PDRA reached the minimum average delay at 8.294905267 milliseconds, while the SDRDR, SDDR, and SDER consume a higher delay at 19.62816013 milliseconds, 71.227377 milliseconds, and 163.8931339 milliseconds, respectively. Furthermore, our proposed SDQN-PDRA quickly responded 11.33325487 milliseconds, 62.93247173 milliseconds, and 155.5982286 milliseconds faster than SDRDR, SDDR, and SDER, respectively. for reliable FL-based IIoT. It is worth noting that IoV communication perspectives are obligated to be handled as a real-time service in distributed edge routing. Moreover, the high-speed mobility sensor will face many challenges in data processing, ultra-high mobility communication, and frequently alternative edge cloud computing. e FL-based IoT will be a large-scale distributed cloud requiring intelligent routing that effectively performs URLLC in routing and network convergence processes. e network assurance plays an essential contribution to reliable FL in IoT systems, while the reliability network systems influence the reliability of FL convergence models in terms of accuracy and decision making. Our proposed SDQN-PDRA approach prevents routing failure and adjusts the network condition to meet the optimal states. e proposed SDQN-PDRA provided remarkable contributions to IoT systems regarding IoT stability conditions and E2E communication QoS, including reliability, latency, and communication throughput. For future work, we will explore the computation cost influence on the communication overhead in routing and expand the routing environment to reflect real-world IoT communications in the 5G system. Furthermore, the SDN-based multidimensional deep-Q-network approaches will be invested in improving the autonomous routing policy.

Conclusion
Data Availability e data and finding are included within the article.

Conflicts of Interest
e authors declare that there are no conflicts of interest regarding the publication of this paper.