Optimal Control through Leadership of the Cucker and Smale Flocking Model with Time Delays

The Cucker and Smale model is a well-known flocking model that describes the emergence of flocks based on alignment. The first part focuses on investigating this model, including the effect of time delay and the presence of a leader. Furthermore, the control function is inserted into the dynamics of a leader to drive a group of agents to target. In the second part of this work, leadership-based optimal control is investigated. Moreover, the existence of the first-order optimality conditions for a delayed optimal control problem is discussed. Furthermore, the Runge–Kutta discretization method and the nonlinear conjugate gradient method are employed to solve the discrete optimality system. Finally, the capacity of the proposed control approach to drive a group of agents to reach the desired places or track the trajectory is demonstrated by numerical experiment results.


Introduction
Individual behaviors influence the dynamics of social systems. Studying the behavior of interacting individuals within a group of animals or a community of people is a new mathematical research field. e emergence of a collective level is one interesting issue raised by this type of study, with many natural systems showing collective phenomena, for instance, a flock of birds or a group of fish having coordinated movements, cells, chemical compounds, organisms or crystals, vehicular traffic, crowd dynamics, market economies, opinion formations, distributions of wealth, networks, and artificial intelligence (see, e.g., [1][2][3][4][5][6]). ese collective behaviors are described as pattern configurations. In particular, collective action results from a superimposition of interactions between every possible pair of all agents. In general, the strength of such interaction forces depends on the amount of collaborative distance between agents. For example, birds orient with their nearest one in the group, and people more often agree with others who already conform to their beliefs. Nowadays, the application of these collective features is enormous and has a wide range. It appears in several engineering applications such as those using collective properties to perform complex tasks or in swarm robotics [7]. In addition, crowd models are applied in civil engineering to plan evacuation strategies in buildings (see, e.g., [8,9]).
In recent years, researchers have been interested in studying collective phenomena, starting with using a few rules of interaction among agents to predict and explain unexpected phenomena or describe the emergence of pattern structure. Despite many works studying the emergence of consensus (see, e.g., [4,10,11]), investigating the enforcement and stabilization of pattern formation including the presence of noise or communication with time delays is especially interesting. Communication delays result from traffic congestion or finite speeds of transmission and spread. For example, the flocking model has a processing delay when analyzing information about the location and velocity of neighboring agents. is fact has motivated an increased interest in studying the consensus problem of a flocking system, including the effect of time delays. Pioneering work on this issue was done in [12][13][14][15][16]. Additionally, when the system's behavior does not realize the desired result, control is essential to force the system to attain given objectives. is problem is related to the control of a selforganized system. Most results focus on the controllability of systems in which the topology of a group communication network is fully connected. In this case, all members of the group are regulated by the same distributed control law. ese model settings, however, are limited when dealing with large networks. In particular, efficient control should apply only to a few group members, and this control strategy is known as "sparsity" in the mathematical literature. From the perspective of controllability, the hierarchical leadership concept provides the aggregation states of the system and some forms of group patterns for longtime behavior. e literature on control problems in these systems has recently been documented in [17][18][19][20][21][22][23].
e main contribution is divided into two parts. e first part discusses the Cucker and Smale (CS) flocking model, where the position and velocity of agents include the effect of time delay. Furthermore, primarily based on this model, a leader is added into the system to implement control functions.
is control action can be applied to other agents through the mechanism of interaction force. e objective of the control strategy in this construction is to drive the evolution of flocking to track the desired trajectory.
is control through leadership approach is constructed in the framework of optimal control problem mainly discussed in the second part of this work. In the second part, firstly, the formulation of optimal control problems with time delay is provided. e objective function includes three tracking terms and the cost of control function. e first and second terms account for the tracking error of the position of the leader and the desired target position at the final time and follow the desired path, respectively. e last tracking term is a measure of the distance between the leader and the other agents of the flock. Further, the theoretical and numerical results of optimization problems constrained with delayed CS systems are investigated. e resulting optimal control is obtained by solving an optimality system composed of the CS model's delayed dynamical system, associated adjoint equations, and an optimality condition. Additionally, the aspect of numerical solution and implementation is discussed. An efficient conjugate gradient optimization procedure evaluates optimal control [24,25]. e gradient of the reduced cost functional is computed using the adjoint framework by solving the forward and backward delayed CS flocking equations that appear in the optimality system. ese equations are discretized using an accurate highorder Runge-Kutta scheme (see more details in [26]). e subsequent sections of our work are organized as follows. In Section 2, the description of the flocking model is provided. is model is based on the Cucker and Smale model, which takes time delays and the presence of a leader into account. Additionally, the existence of solutions and stability for these systems are discussed. Furthermore, numerical simulations are demonstrated to examine the characteristic properties of our constructed model. Section 3 presents the formulation of optimal control of the Cucker and Smale system with time delays. e theoretical issues regarding the existence of first-order necessary optimality systems of delayed optimal control problems are discussed. Section 4 is comprised of two parts. In the first part, the control problem with time delays is discretized by using the Runge-Kutta scheme. en, using the first-discretize-then-optimize strategy, a discrete gradient formula for the optimization problem is obtained. e second part of Section 4 contains numerical experiments demonstrating the validity and effectiveness of the proposed control strategy. e conclusion is provided in the last section to complete this work.

Cucker and Smale Flocking Model with Time Delays and Leadership
In this section, we investigate the flocking model, including the effect of time delays and the presence of a leader. In this work, we focus on the Cucker and Smale (CS) model. is model is one of the well-known flocking models that involve the alignment of the agent; that is, each agent changes its speed in response to an averaging weight of its relative speed to the other agents, which can be seen in [5]. Moreover, we investigate the effect of time delays, which have influenced to converge to state transition and pattern structure. In our case, we consider constant delay function as proposed in [27]. In this context, delays are defined as a lag in the processing of information. To illustrate this phenomenon, consider a situation involving two agents in which the second agent attempts to follow the first but receives information from the first agent after some time. As a result, it requires time to elaborate on its response. We define τ as a positive constant time delay. Additionally, a leader is incorporated into the CS system to facilitate developing a control strategy for guiding the flock dynamical system toward a target (described in the following section). e CS flocking system with one leader and N followers including time delay is presented as follows: for b � 1, . . . , N, with given initial conditions where (x i (t), v i (t)) ∈ R d × R d , d ≥ 1, are position and velocity of the i th agent at time t. e initial conditions x 0 i (t) and v 0 i (t) are given continuous functions. e connectivity function a ij measures the interaction strength between agents depending on the distance between i th and j th agents. e connectivity function is assumed to be positive, continuous, bounded, and nonincreasing function.
In this work, we consider the connectivity function with the following form: where K > 0, σ ≥ 0, and the notation ‖·‖ denotes Euclidean norm in R n . e force F 0 b governs the action of the leader with time delay. e leader force can take many forms, such as using short-range repulsion and long-range attraction described as Morse potential function in [18] or considering hierarchical leadership structure in [16]. In this work, the leader's action takes the following form: where the parameter c > 0 corresponds to the strength of the interaction force taken by the leader.

Stability of the Cucker and Smale Model with Time Delays.
e solution of the CS flocking system is influenced by the parameters K and σ. In particular, when σ < (1/2), the CS model without time delays exhibits unconditional flocking for all initial configurations, in which the velocities v i approach asymptotically to the common limit velocity v * . On the other hand, the flocking is conditional with σ ≥ (1/2), i.e., the asymptotic behavior of the system is dependent on the value of K and the initial configuration, as detailed in references [5,14,27]. To analyze the solution and stability of system (1), we consider the dispersion and disagreement as defined in [16,19].

Definition 1 (dispersion and disagreement). Given a solution
we define dispersion and disagreement as en, we say that the solution tends to consensus if Next, the numerical simulation results of the CS system with time delays, including the presence of a leader, are demonstrated. Figure 1 depicts the numerical results of system (1) in one dimension, while Figures 2 and 3 illustrate results in two dimensions. For all experiments, we consider one leader and N � 9 followers. e corresponding parameters in connectivity function are chosen as K � 2 and σ � (1/4). e leader interaction strength parameter is given as c � 10. Consider 1D case, and Figure 1(a) shows the trajectories of agents with no time delay, whereas the motion of flocking, including the effect of time delay (τ � 5), is presented in Figures 1 It is observed that due to time delay, agents require more time to adjust their alignments and follow the group.
We also consider one leader and N � 9 followers in the experiments for 2D cases. e parameters for the 2D experiments are the same as those for the 1D studies. Agents begin with an initial configuration that places them all in random positions with random velocities, with a leader outside the group. After t � 2, each agent begins to adapt and organize its movement to follow the leader. Figure 3 shows that for the longtime behavior t ⟶ ∞, the value V(t) tends to be constant at zero and the dispersion X(t) is becoming constant. It explains that the consensus is reached after some time. e pattern configuration is formed, and each agent's velocity is synchronized and moves together at the same speed.

Optimal Control of Cucker and Smale Model with Time Delays
is section discusses the optimal control problem constrained with the delayed dynamical system of the CS model, including a leader. We note that this control problem is the leadership-based control strategy which is discussed in [18] for the refined flocking model. In addition, the theoretical results corresponding to optimal control problems with delays are investigated. In particular, the existence and characterization properties of solutions and optimality conditions are provided. In the following, we consider the optimal control problem governed by the delayed CS system and the presence of a leader presented as together with the delayed differential constraints given by for b � 1, . . . , N, along with their corresponding initial conditions. e state of the system is represented by the notation e control function is denoted by u ∈ L 2 ((0, T); R d ). e parameters η, μ, and ] in the cost functional are positive constants. e first term in objective functional aims to minimize the terminal position of the leader and the given target. e second term is related to track the desired path by the leader agent. Also, the distance between the leader agent and others is measured in the third term. e cost of control is minimized in the last term.

Complexity
To investigate the solution of our optimal control problems (8), we consider the optimal control problem that state and control variables have an effect of time delays, that is, state variable x(t) and control variable u(t) include time delays τ x ≥ 0 and τ u ≥ 0, respectively. e theoretical results of the delayed optimal control problems are investigated in [28,29]. e general framework of an optimal control problem with delay in state and control is presented as follows: We assume that e function ϕ: R nx ⟶ R represents the objective function and is assumed to be contin- represents the dynamics of the model and is assumed to be continuously differentiable. e initial conditions x 0 : [−τ x , 0] ⟶ R nx and u 0 : [−τ u , 0] ⟶ R nc are continuous. We note that a solution of the dynamical system in (11) is uniquely determined for a given control u and the initial conditions denoted by x � x(u); moreover, u ↦ x(u) is a differentiable map. For all admissible u, we aim to find a control u * such that J x u * , u * ≤ J(x(u), u).
Furthermore, to obtain the necessary optimality conditions for delayed optimal control (11), we introduce two additional variables y ∈ R nx and w ∈ R nc for delayed state and control variables, respectively, where the functions y and w are defined as follows: e first-order optimality system can be obtained by considering the Hamiltonian function that is given by   H(t, x, y, u, w, p) ≔ p ⊤ f(t, x, y, u, w), (14) where p ∈ R nx .

Theorem 1.
Let (x * , u * ) be locally optimal solution to optimal control problem (11). Assume that there exists an open set Ω ⊂ R nx × R nc such that the neighborhood B ε (x * , u * ) with a small radius ϵ > 0 of (x * , u * ) is a subset of Ω for every t ∈ [0, T]. We assume that ∇ x f and ∇ u f are Lipschitz continuous in Ω. Moreover ∇ x J is Lipschitz continuous in the neighborhood B ε (x * (T)). en, there exists an adjoint function p * ∈ W 1,∞ ([0, T], R nx ) that satisfies the first-order optimality conditions at (x * , p * , u * ), for a.e. t ∈ [0, T] such that , x(t), y, u(t), w(t)), where ∇ x H(t), ∇ y H(t), ∇ u H(t), and ∇ w H(t) refer to the evaluation of the partial derivatives H with respect to x, y, u, and w, respectively. e function χ is defined as To investigate the solution of our problem (10), we formulate our optimal control in form (11) by introducing the following equation: en, the problem of optimal control (10) can be transformed into the following problem: together with corresponding initial conditions, and the compact form can be rewritten as follows:  F(t, x, u). (20) Now, the state variable is and F(x, u) represents the dynamics of the transformed delayed CS systems. Further, we discuss the first-order optimality conditions for optimal control problem (20). For this purpose, the delayed system is transformed to nondelayed system by using variables and we denote y(t) � (s 0 (t), s 1 (t), . . . , s N (t), z 0 (t), z 1 (t), . . . , z N (t)) ⊤ . e Hamiltonian function for system (20) is presented as where

Assumption 1.
ere exist Lagrange multipliers p ∈ W 1,∞ ([0, T], R nx ) corresponding to the optimization constraint equation of problem (20). Moreover, the adjoint p satisfies the following equation: erefore, a solution of (20) can be characterized by the first-order optimality system as presented in the following: F(t, x, u), In addition, the corresponding gradient can be expressed as In particular, the explicit formulations of adjoint equations (20) can be expressed as follows: Complexity 7 with transversality condition e numerical solution of problems (20) representing as the considered optimality systems (25) will be shown and discussed in the next section.

First-Discretize-en-Optimize Strategy and Discrete
Optimality Conditions. e discretization of the reduced gradient is a critical step in the numerical method for solving optimal control problems. To obtain an accurate discretization for the reduced gradient, we consider the firstdiscretize-then-optimize strategy. To carry out this strategy, one can follow the procedures outlined below. e cost functional and corresponding differential constraints representing the optimal control problem are discretized in the first step using the Runge-Kutta method, discussed in detail in [26]. Second, the discrete Lagrangian function corresponding to the discrete optimal control problem must be constructed. e final step is to obtain the first-order optimality system for the discretized problem.
For the discretization of optimal control problem (20), the Runge-Kutta scheme is employed on a uniform mesh in the time intervals (0, T) such that the step-sized h is defined as with the total number of discrete time intervals, n. It is important to match the uniform step size h > 0 with positive delay τ; that is, we choose any integer fraction of h to refine the discretization grid. For this task, for any step size h > 0, there exist positive integers n, m τ ∈ N satisfying By this setting, we denote the value of x(t) at the discrete time t k by erefore, the discretization of the s-stage Runge-Kutta scheme setting for optimal control problem (20) becomes subject to for k � 0, . . . , n − 1 and 1 ≤ i, j ≤ s. Note that the control u k ∈ R nc×s denotes the s stages of the RK discrete control function at the stage k, which can be written in the following form: u k � u k1 , u k2 , . . . , u ks .
We remark that the order of a Runge-Kutta discretization for optimal control problem depends on coefficients a ij and b i (see more details in [26]). e discrete optimality system that corresponds to (32) is expressed as the following equations: 8 Complexity Further, the gradient is in the following form: for 1 ≤ i, j ≤ s, and 0 ≤ k ≤ n − 1.
To solve optimization problem (32), the nonlinear conjugate gradient (NCG) strategy with the Hager and Zhang scheme [24,25] is implemented. First, we use the following algorithm to compute the gradient specified in First, we use the following algorithm to compute the gradient specified in (35).
Following that, we apply the gradient derived from Algorithm 1 to the NCG scheme defined in Algorithm 2.

Numerical Results.
is section presents the numerical results of our flocking models. ese results demonstrate the control performance of our leader-based control strategy. e test is divided into two parts as follows: (i) Test I: reach the desired target point.
(ii) Test II: follow the desired path.
In our experiments, we investigate problem (20) in two dimensions by considering a system composed of nine agents and one leader. As a result, the state variables' total nx � 20, while the control variables' total nc � 1. In numerical test I, we focus on controlling the leader agent to reach the goal position at the final time. e purpose of numerical test II is to force the flocks to follow the desired trajectories. We provide two examples of the desired trajectory in these tests: the linear and circular paths. We use the same initial conditions in all three examples, where agents' positions are distributed, and their initial velocities are random. Additionally, the initial position of the leader is set to x0 � (0, 0).
In numerical test I, the goal position is given as x des � (5, 5) and the end time is set to T � 10. e interaction force parameters are K � 2, σ � 0.25 and the strength of leader interaction force is c � 10. e objective of this test is to force the system to reach the desired target at the final time; as a consequence, the corresponding parameters in objective functional are μ � 0.1, ] � 1, and η � 0. As illustrated in Figure 4, the leader is capable of reaching the desired location and leading the flock there.
In numerical test II, the corresponding parameters and initial data are chosen similarly to those in the numerical test I, except that η � 10 is used to keep the leader agent tracking the desired trajectories. We illustrate two scenarios using two distinct tracking trajectories. e results of the first example are depicted in Figure 5. e plot of snapshots for ten agents traveling along the specified linear path is shown in Figure 5. Figures 5(a)-5(e) illustrate the solutions to optimal control problems at various time points. As illustrated in Figure 5(f ), the leader attempts to move in the direction of a given trajectory while the group of agents follows the leader. In the last example, the desired path is given as a circle. As shown in Figure 6, the leader agent tracks the path, and the other agents follow the leader. It can be observed from three examples that agents keep some distance from the leader because time delay creates some Input u; (1) e initial conditions are provided and then solve the forward the discrete CS model; (2) Compute the terminal condition and solve the discrete adjoint equation in (34); (3) Evaluate the gradient ∇ u J(u) using (35); ALGORITHM 1: Evaluation of the gradient at u. Input: u 0 index k � 0, maximum k max , tolerance tol > 0. While (k < k max and ‖∇J(u k )‖ > tol) do (1) Obtain x k from step 1 in Algorithm 1 with u k ; (2) Get P k from step 1 in Algorithm 1 with (x k , u k ); (3) Evaluate the gradient ∇ u J(u k ) from step 1 in Algorithm 1; (4) Compute α k by using backtracking line search scheme with Armijo's condition [30]; (5) Compute β k by using Hager and Zhang search direction formula [24];    absence of leader position. It would take some time for the group to respond to this emotion and attempt to follow the leader; however, the group eventually achieves it.

Conclusion
e Cucker and Smale flocking model, including the effect of time delays and the presence of a leader, was studied. A control-based leadership technique was investigated for this model. An optimal control problem was formulated, and the corresponding discretization of an optimal problem was solved using an accurate RK method such that accurate gradients of the reduced objectives can be guaranteed. e nonlinear conjugate gradient scheme was implemented to compute the gradients. e results of the numerical experiment show that the proposed control approach is practical. From a modeling point of view, this flocking system could be enhanced by including the effects of attraction and repulsion and alignment with the vision cone and delay. In addition, the delay in control function would be considered a part of the optimal control problem.

Data Availability
No data were used to support this study.

Conflicts of Interest
e authors declare that they have no conflicts of interest.