Discrete Dynamics in Nature and Society
Volume 2016 (2016), Article ID 7217364, 11 pages
http://dx.doi.org/10.1155/2016/7217364

Review Article

A Review of Fuzzy Logic and Neural Network Based Intelligent Control Design for Discrete-Time Systems

Yiming Jiang,¹ Chenguang Yang,1,2 and Hongbin Ma³

¹Key Lab of Autonomous Systems and Networked Control (MOE), School of Automation Science and Engineering, South China University of Technology, Guangzhou 510640, China
²Zienkiewicz Centre for Computational Engineering, Swansea University, Swansea SA1 8EN, UK
³State Key Lab of Intelligent Control and Decision of Complex Systems, Beijing Institute of Technology, Beijing 100081, China

Received 5 November 2015; Accepted 29 December 2015

Academic Editor: Juan R. Torregrosa

Copyright © 2016 Yiming Jiang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

Over the last few decades, the intelligent control methods such as fuzzy logic control (FLC) and neural network (NN) control have been successfully used in various applications. The rapid development of digital computer based control systems requires control signals to be calculated in a digital or discrete-time form. In this background, the intelligent control methods developed for discrete-time systems have drawn great attentions. This survey aims to present a summary of the state of the art of the design of FLC and NN-based intelligent control for discrete-time systems. For discrete-time FLC systems, numerous remarkable design approaches are introduced and a series of efficient methods to deal with the robustness, stability, and time delay of FLC discrete-time systems are recommended. Techniques for NN-based intelligent control for discrete-time systems, such as adaptive methods and adaptive dynamic programming approaches, are also reviewed. Overall, this paper is devoted to make a brief summary for recent progresses in FLC and NN-based intelligent control design for discrete-time systems as well as to present our thoughts and considerations of recent trends and potential research directions in this area.

1. Introduction

It is well known that control design is critical to the performance of the closed-loop system response, while an accurate system model is usually necessary for a high quality control design. But there are inevitable uncertainties during modeling of any practical systems. These modeling uncertainties may result in poor performance and may even lead to instability of the closed-loop systems. To improve control performance, many control strategies have been developed to consider these uncertainties in the control design stage. As one of the major control approaches, adaptive control has been developed for more than half a century with intense research activities involving rigorous problem formulation, stability analysis, robustness design, performance analysis, and applications [1–9].

Early progress of adaptive control focused on identification and closed-loop system analysis of linear systems. Self-tuning regulator and model reference adaptive control are two typical adaptive controllers based on different design philosophies, and they have attracted renown scholars such as Åström and Wittenmark [10], Ljung [11], and Goodwin et al. [12] due to the challenges in closed-loop stability analysis. In fact, even for the discrete-time linear systems, the closed-loop system will become highly nonlinear due to the injection of adaptive controller such as Aström-Wittenmark self-tuning regulator, whose closed-loop stability was eventually resolved by Guo based on his cooperation with Chen et al. [6, 13, 14].

Besides linear systems, discrete-time adaptive control has also been developed for various different classes of nonlinear systems, such as nonlinear systems with linear growth rate [15], nonlinear systems with convex/concave parameterization [16, 17], nonlinear systems with polynomial-form nonlinear growth rate [18–21], nonlinear systems with multiple/finite model structure uncertainty [22–25], nonlinear systems with nonlinear parameterization [26, 27], nonlinear systems with semiparametric uncertainties [28], and so on. Many challenges have been seen in these existing studies to cope with the interaction of nonlinearity and adaptation. Now it is clear that discrete-time adaptive control can be extended to nonlinear systems with linear growth rate by using the modification technique such as projection [15]; however, we also know that there exist intrinsic difficulties or even impossibility for discrete-time adaptive control when the nonlinear growth rate is high enough [29, 30], which is quantitatively characterized by fundamental limitations of feedback mechanism in terms of certain introduced uncertainty measure.

In addition to the progress from linear systems to the nonlinear systems, rigorous stability analysis of the closed-loop adaptive system has also been well established [31]. Compared with robust control, the advantage of adaptive control lies in its ability in estimating and compensating for parametric uncertainties in a large range. However, towards the increasingly complexity of systems with complicated nonlinear functional uncertainties, it is necessary to develop more powerful control design tools. To this end, researchers noticed some limitations of traditional adaptive control in mainly coping with parametric uncertainties. And fuzzy logic (FL) and NN-based intelligent control have been introduced in the early 1990s [6].

The concept of fuzzy set was initially proposed by Zadeh [32], and since then, fuzzy technique gained a rapid development. FL offers human reasoning capabilities to capture nonlinearities and uncertainties, which cannot be described by precise mathematical model. In [33], Wang and Mendel found that linear combinations of a series of fuzzy basis functions are able to approximate smooth nonlinear functions and proved such feature of universal approximator by using the Stone-Weierstrass theorem. Based on this work, an adaptive fuzzy control method was further proposed by combining the universal approximation of the FL controller and learning ability of the adaptive method [34]. And later, a stable adaptive fuzzy controller was successfully designed for a class of nonlinear systems. In this approach, the adaptive law was constructed to update the parameters of the FL controller during the adaptation procedure while it was not necessary to have the accurate mathematical model of the system. From then on, fuzzy control has attracted ever increasing research interest, since it is able to provide an effective solution to control complex and uncertain plants by employing the knowledge of specific experts for the controller design.

NN is inspired by biological neuronal systems witch consist of a number of simple processing neurons interconnected to each other. McCulloch and Pitts introduced an idea to study the computational abilities of networks composed of simple models of neurons in the 1940s [35]. With highly parallel structure, NN is of great computational power and learning ability to emulate various systems dynamics. It is well established that NN is capable of universally approximating unknown smooth functions with errors that can be made arbitrarily small [36–38]. In addition to system modeling and control, NN has also been successfully applied in many other fields such as learning, pattern recognition, and signal processing. In NN control methodology, NN has been extensively studied as function approximators to compensate for the system uncertain nonlinearities in control design. In the last two decades, it has been verified that NN control is very useful for controlling highly uncertain nonlinear systems and NN control has demonstrated superiority over traditional control methods. Particularly, the marriage of adaptive control theories and NN techniques gives birth to adaptive NN control, which guarantees stability, robustness, and convergence of the closed-loop NN control systems without beforehand offline NN training [6].

Nowadays, most of the control algorithms are realized by digital computers, and thus the desired control signals are calculated in a digital manner. Digital control systems have advantages such as easy to build, less sensitive to environment variation, flexible to change, and less expensive. A model based digital controller is designed in discrete-time form and operated in continuous-time with analogy signals. Since the data are generally processed at discrete-time instants, it is necessary for us to build the system model in discrete-time for ease of control design [6].

Yet in the past decades, many significant progresses of FLC and NN control for nonlinear systems were in continuous-time form and there is considerable lag in the development for nonlinear discrete-time systems [39, 40]. Although for linear systems there are numerous results of NN and fuzzy controls in continuous-time systems, the NN and fuzzy control of nonlinear discrete-time systems have been considerately less studied than their counterparts in continuous-time. As a matter of fact, many techniques developed for continuous-time systems cannot be directly applied in discrete-time, especially when the systems are nonlinear. Discrete-time systems can be expressed by various equations, which in great contrast to the differential equations of continuous-time systems involve states at different time steps. Due to the different nature of difference equation and differential equation, some concepts in discrete-time have very different meanings from those in continuous-time and the stability analysis techniques become much more intractable. For example, the linearity property of the derivative of a continuous-time Lyapunov function is no longer existed in a discrete-time Lyapunov function [41]. Therefore, in order to design discrete-time fuzzy or NN controllers and implement them into digital control systems, they deserve in-depth investigation for fuzzy control and NN control of nonlinear discrete-time systems.

The remainder of this paper is organized as follows. In Section 2, a brief review of fuzzy control for discrete-time systems is presented. Some research problems of fuzzy control and discrete-time systems, such as adaptive control, robustness issue, stability analysis, and time-delay systems, are concerned and introduced for they are both theoretically challenging and practically meaningful. In Section 3, NN control methods for discrete-time systems are briefly reviewed. Research of adaptive NN control for discrete-time systems is introduced and NN-based adaptive dynamic programming methods for discrete-time systems are also discussed. Finally, in Section 4 a brief conclusion is given.

2. Fuzzy Logic Control for Discrete-Time Systems

Nowadays, techniques for FLC have developed rapidly, especially in modeling complex nonlinear systems. Since Wang first proved that the linear combinations of a series of fuzzy basis functions are universal approximators of any nonlinear systems, the universal approximation property of the fuzzy logic systems has been extensively studied [42–45]. The combination of adaptive control and fuzzy logic system allows adaptive laws to update parameters of the FL controller during the adaptation procedure while adaptive fuzzy control provides an efficient method to model complex nonlinear systems. In practice, the difficulty of design a fuzzy controller lies in the fact that there are usually various requirement imposed on the systems to ensure stability and performance, while most of complex control systems today are using digital computers to calculate the control signals in a digital form. Hence, the continuous-time methods could not be directly applied to most practical systems. The modeling/control for discrete-time systems is crucial to take the controllers into real plants. Thus, it is significant to study the discrete-time fuzzy systems as well as their molding and control. To this end, researchers have paid efforts to develop the discrete-time FLC. A typical procedure of the multiple-input multiple-output (MIMO) fuuzy logic system (FLS) approximating an unknown function is comprised of three primary components as follows [46].

(1) Fuzzification. Take as the input of the fuzzy system, and is the estimation of the output. Both the input and the output are fuzzified into fuzzy linguistic terms with fuzzy membership function.

(2) Fuzzy Rules. The collection of the fuzzy MIMO IF-THEN rules are designed to comprise the knowledge base for constructing the FLS. Using prior expert knowledge, the fuzzy rules can be obtained as

where and are the premise variables consisting of and , respectively; and are the linguistic variables of the fuzzy sets; .

(3) Fuzzy Inference Engine and Defuzzification. Using the strategy of the sum-product inference engine and the center-average defuzzifier, the output of the FLS can be described as

where is the point where fuzzy membership function achieves its maximum value. Define

and . Then, the output of MIMO FLS can be rewritten as

where the subscript represents the th column vector of the matrix. Therefore, the estimation of is

Then can be written as

2.1. Adaptive FLC of Discrete-Time Systems

FLC has been extensively investigated in both academic and industrial communities. However, problems with high dimensionality also existed in higher-dimensional fuzzy systems due to the mathematical complications and intuitive limitations. Therefore, it required more systematic methods for advanced FLC design and synthesis, such as the adaptive FLC. Originally adaptive control was proposed for aircraft autopilots to deal with parameter variations during changing flight conditions. In the 1960s, the advances in stability theory and the progress of control theory improved the understanding of adaptive control. By the early 1980s, several adaptive approaches have been proven to provide stable operation and asymptotic tracking. Since then, the adaptive control problems have been rigorously formulated.

Generally, adaptive FLC design for nonlinear discrete-time systems is much more difficult than those in continuous-time. The stability analysis techniques become much more intractable for difference equations than those for differential equations. Many nice Lyapunov adaptive control design methodologies developed in continuous-time are not applicable to discrete-time systems. Thus, it is significant to consider the adaptive FLC in discrete-time form.

In recent years, the adaptive FLC for discrete-time nonlinear systems has been extensively studied. Jagannathan et al. [45, 47] developed an adaptive fuzzy logic controller for a general class of discrete-time nonlinear systems using basis vectors and provided rigorous proofs to stability of the discrete-time adaptive FLC. In this work, a multiloop structure controller was constructed with a fuzzy approximate linearization loop, an outer tracking loop, and a robustifying loop. By selecting FLC’s structure and the adaptation laws and giving a number of assumptions on a class of discrete-time nonlinear systems, closed-loop signals can achieve uniform ultimate boundedness. This method is significant in sense that no persistence of excitation is used and no certainty equivalence assumption is required. The controller has also been extended to a class of unknown feedback linearizable nonlinear dynamical systems under persistence of excitation in [48], in which rigorous stability proofs of discrete-time adaptive FLC for feedback linearizable unknown nonlinear systems were presented. Indirect fuzzy control of uncertain discrete-time nonlinear system was shown by Qi and Brdys [49]. This work presented an indirect adaptive FLC for the uncertainties in nonlinear plants and employed a Takagi-Sugeno (T-S) model to deal with the unknown dynamics in input-output form. A feedback linearization control law was designed by using structure, states, and parameters of the model. Gradient descent algorithm and recursive least square estimation method were used to online update the modeling parameters. The extension of sliding mode control (SMC), adaptive fuzzy SMC for a class of uncertain discrete-time nonlinear systems, was investigated by Lin et al. [50]. This work presented an adaptive interval type-2 fuzzy sliding mode controller for a class of unknown nonlinear discrete-time systems with training data disturbed by external disturbances. It employed an adaptive interval type-2 fuzzy control scheme and SMC approach to control the plant tracking a reference trajectory and prevented big chattering of the control effort. While adaptive fuzzy control has also been applied in discrete-time chaotic systems. In [51], Feng and Chen presented a novel adaptive control algorithm for discrete-time chaotic systems. The basic idea is representing the chaotic system as a T-S fuzzy model and designing a local linear adaptive controller in each local region. Meanwhile a global adaptive controller on the entire domain was reconstructed and the stability of the closed-loop adaptive control system was proved.

2.2. Robustness Issue in Discrete-Time Fuzzy Control

In practice, nonlinearities and uncertainties exist in almost all the industrial plants, while uncertainties in the modeling and control of nonlinear systems are still one of the most challenging problems in the control field. The presence of nonlinearities and uncertainties brings difficulties to controller design. In order to conquer these problems, a number of schemes have been developed, and among these, robust control is one of the most effective methods. Robustness is regarded as one of the most important requirements for a control system. In the past two decades, considerable attention has been paid to the robustness in fuzzy model-based control of discrete-time systems and many modification techniques were proposed to enhance the robustness of FLC for discrete-time systems [52–57].

The work of Lee et al. [52] illustrated an approach of robust fuzzy control for nonlinear discrete-time systems. In this work, a systematic control structure was presented for nonlinear discrete-time T-S fuzzy systems with parametric uncertainties using the control design approach. This work also gave a novel solution to the robust stabilization problem of nonlinear systems by using basis-dependent Lyapunov function.

disturbance attenuation is one of the most important requirements for a fuzzy control system. Cao and Frank [53] have applied the control to address the robust stability for a class of uncertain discrete-time fuzzy systems via linear matrix inequality approach. They studied both the robust stabilization and the disturbance attenuation, while they presented several sufficient conditions to ensure robust stability of the fuzzy models based on coupled linear matrix inequalities. They also analyzed the robust performance of the fuzzy model-based discrete-time nonlinear systems by addressing the robust disturbance attenuation problems. The robust stability of the models is achieved independently of the uncertainties.

How to develop a robust controller for discrete-time systems using the basis-dependent Lyapunov function was also shown by Zhou et al. [54]. In their work, a novel linear matrix inequalities characterization with norm bound was presented for discrete-time fuzzy systems. While additional matrix variables were used to decouple the Lyapunov function and system matrices as well as facilitate the design approach of the controller, this approach provides some sufficient results in the shape of strict linear matrix inequalities.

Studied by Choi and Park, the state feedback control for discrete-time systems was investigated by constructing Lyapunov functions with fuzzy weights. The fuzzy weighting Lyapunov functions were designed with both current-time and one-step-past information while designing controller parameterized with linear matrix inequalities (PLMIs). By selecting the structures of variables appropriately in the PLMIs, a special case LMI formulation was obtained. The control with output feedback for discrete-time fuzzy systems has also been rigorously studied by Xu and Lam [55], while Wu et al. [58] investigated the reliable control for discrete-time systems with delays and stochastic actuator faults by representing the stochastic behavior with a discrete-time homogeneous Markov chain.

Tseng and Chen [56] used a fuzzy observer to estimate the premise variables which depend on the state variables, while the fuzzy observer was also used to address the nonlinear -gain control problem. This work extended the control from linear discrete-time systems to nonlinear discrete-time systems. By using the T-S fuzzy model to represent the nonlinear discrete-time system, an observer-based fuzzy controller, which minimized the upper bound of -gain and attenuated the peak of perturbation was designed. Xu et al. [57] gave stability analysis for discrete-time singular fuzzy systems in the presence of time-varying uncertainties while they ensured the system to be regular, causal, and stable. It has been shown that, for these systems, robust stability condition can be obtained by giving a sufficient condition in terms of a set of linear matrix inequalities.

2.3. Stability Issue in Discrete-Time Fuzzy Control

Stability is one of the most important properties for a system while FLC are used to address the stability problems for discrete-time systems as well. The quadratic stability for uncertain discrete-time fuzzy dynamic systems was shown by Feng and Ma [59]. This work gave some sufficient conditions of the quadratic stabilization for an uncertain fuzzy dynamic system. Stability was achieved when the suitable Riccati equation or a set of Riccati equations were solved. Stabilizing feedback control laws were also obtained by the developing algorithms. The nonquadratic stabilization conditions for nonlinear discrete-time fuzzy systems were also analyzed by Kruszewski et al. [60]. They considered discrete-time uncertain nonlinear models in a T-S form and studied the stability through a nonquadratic Lyapunov function. The stabilization conditions were developed by considering the Lyapunov function with a -sample variation and extended to uncertain T-S models. Based on a switching fuzzy model and piecewise Lyapunov function, Wang et al. [61] proposed two stabilization criteria for discrete-time T-S fuzzy systems.

Zhou et al. [54] used a basis-dependent Lyapunov-Krasovskii function to give stabilization analysis for discrete-time fuzzy systems. A robust control design approach was also developed by using the Lyapunov-Krasovskii function and facilitated by introducing additional instrumental matrix variables. Robust control problem for systems with time delay was concerned as well. Feng [62] used a piecewise smooth Lyapunov function to analyze the stability of discrete-time T-S fuzzy dynamic systems. In this study, stability of the system guaranteed by constructing a piecewise Lyapunov function. Meanwhile the Lyapunov function was obtained using the linear matrix inequalities method. This work showed that the piecewise quadratic Lyapunov functions based stability is less conservative than the common quadratic Lyapunov function based stability.

The stabilization for discrete-time T-S fuzzy systems with state time-varying delay was shown by Gao et al. [63]. In their work, a fuzzy Lyapunov function was constructed to improve the delay-dependent stability condition. By avoiding the utilization of the bounding inequalities for the cross products between two vectors, reduction of the conservatism of stability condition was achieved. A delay-dependent stabilization algorithm was also developed for both state feedback and observer-based output feedback cases using a parallel distributed compensation scheme.

2.4. Fuzzy Control for Discrete-Time Systems with Time Delays

In practice, time delays are the intrinsic nature of various physical systems, such as communication, hydraulic, chemical processes, and electronics. The existence of time delays could generally lower the system performance and even cause instability and oscillation. Hence, remarkable attention has been devoted to the analysis and synthesis of time delay nonlinear systems as well as the discrete-time fuzzy systems [55, 64–69].

The stability and stabilization problems of systems time-varying delay were investigated by Wu et al. [64]. In their work, the existing stability analysis approaches were divided into two types: delay-dependent (including the delay size) and delay-independent (irrelevant to the delay size). The stability of discrete-time T-S fuzzy systems with time-varying state delay was also investigated. They developed a delay partitioning method and employed a fuzzy Lyapunov-Krasovskii function to analyze the stability of delay-dependent time-varying state delay systems. Through delay partitioning, the less conservative stability condition is obtained and solved by LMI optimization techniques.

The work of Su et al. [65] also gave a solution to the dynamic output feedback control problem for fuzzy systems with time-varying delays. They proposed a comparison model by approximating the time-varying delay state, while they gave a sufficient condition to ensure the asymptotical stability by adopting the scaled small-gain theorem as well as the delay partitioning method. This method allows designing desired dynamic output feedback controller by using optimization techniques. A filter design technique for discrete-time systems with time-varying delay has also been presented by Su et al. [66]. Both full-order and reduced-order filters were designed while approximators were employed to derive an input-output based sufficient condition. Tseng [67] studied the time delay problem for a nonlinear discrete-time system based on model reference fuzzy tracking control. In this work, the T-S fuzzy model was applied to approximate a time delay discrete-time system while the fuzzy controller was designed to reduce the tracking error based on this fuzzy model. This method has advantage in that no feedback linearization or adaptive approaches are used in the controller design.

The robust control problem for uncertain discrete-time delay systems has been ivestigated by Xu and Lam [55]. They studied the state-space T-S fuzzy model with time delays under the assumption that the parameter uncertainties were norm-bounded. The output feedback controller was designed with full-order fuzzy dynamic to guarantee the closed-loop system is robust asymptotic stable as well as the admissible uncertainties are norm bound constraint. While solvability of this problem was addressed using a sufficient condition given by the linear matrix inequalities. The induced filter design of T-S fuzzy discrete-time stochastic systems with time-varying delays has been investigated by Su et al. [68]. Wu et al. [69] studied the model approximation for discrete-time state delay fuzzy T-S systems.

3. NN Control for Discrete-Time Systems

NN is of powerful computing ability and learning ability to emulate various systems dynamics and is capable of approximating an unknown function with arbitrary accuracy. NN is very successful in system modeling and control by its capacity of universal approximate highly nonlinear and uncertain, nonlinear and complex dynamic of systems. NN’s approximation ability has been shown by the Stone-Weierstrass theorem, which states that a universal approximator can approximate, to an arbitrary degree of accuracy, any real continuous function on a compact set. Besides the universal approximation abilities, NN also shows its excellence in parallel distributed processing abilities, learning, adaptation abilities, natural fault tolerance, and feasibility for hardware implementation. These advantages make NN particularly attractive and prospective for nonlinear control and modeling. NN has been successfully applied to robot manipulators control [70], distillation column control [71], chemical processes identification [72], flight control [73–75], and so forth. To extend the NN technique from continuous-time to discrete-time, many efforts have been made to study the NN controlled discrete-time systems [76–78].

3.1. Adaptive NN Control for Discrete-Time Systems

In the early stage, backpropagation (BP) algorithm [79] greatly boosted the development of NN control. It is noted that, in the early NN control design, the control performances were demonstrated through simulation or by particular experimental examples, and consequently there were shortage of analytical analysis. In addition, an offline identification procedure was essential for achieving a stable NN control system. Thereafter, the emergence of Lyapunov-based NN design makes it possible to use the available adaptive control theories to rigorously guarantee stability, robustness, and convergence of the closed-loop NN control systems. We call the control design combining adaptive control theories and NN techniques as adaptive NN control. It updates NN weights online and guarantees the stability of the closed-loop system. Adaptive NN control design has been elegantly developed for nonlinear systems with parametric uncertainties, while many efforts have been dedicated in adaptive NN control for nonlinear discrete-time systems [6].

For high-order affine nonlinear system in normal form, adaptive NN controls using LPNN and MNN have been developed in [80, 81] using a filtered tracking error. In [82], the controller is designed incorporating reinforcement learning technique to improve control performance. In this work, a critic NN has been introduced to approximate the strategic utility function which is considered as the long-term system performance measure. For discrete-time systems in strict-feedback form, adaptive NN control has been developed via backstepping design after system transformation [83]. In [84], adaptive NN control has been investigated for discrete-time system in affine NARMAX form.

In the above mentioned results, the adaptive NN control designs were carried out through either feedback linearization or backstepping. But these approaches are not applicable to nonaffine systems, especially feedback linearization based methods, which greatly depend on the affine appearance of control variables. As a matter of fact, adaptive NN control methods for nonaffine systems have been less studied in comparison with large amount of research work on affine nonlinear systems, because of the difficulty of control design caused by the nonaffine form of control input. To overcome the difficulty, linearization based NN controls have been put forward. In [17], the nonaffine discrete-time system has been divided into two parts: one is linear and another is nonlinear, and consequently a linear adaptive controller and a nonlinear adaptive NN controller have been designed with a switching rule. In [85], it directly utilized NN as emulator of the “inverse" of the nonlinear discrete-time systems. Furthermore, the study of discrete-time systems for adaptive NN control using implicit function to assert the existence of an ideal inverse control was investigated in [86]. Thereafter, the implicit function based adaptive NN control has been widely studied in discrete-time form [87, 88]. Block triangular discrete-time systems with normal form subsystems have been studied in [80, 81]. For the block triangular systems with strict-feedback subsystems, state feedback and output feedback adaptive NN control have been developed in [89, 90] by extending the systems transformation based backstepping technique proposed for SISO case in [83]. In [91], adaptive NN control has been used for sampled-data nonlinear MIMO systems in general affine form based on linearization. The control scheme is an integration of an NN approach and a variable structure method.

An effort has been made in [92] to explore the adaptive NN control of a class of nonaffine systems in discrete-time. This work aimed to solve the nonaffine appearance and noncausal problems of the following pure-feedback discrete-time system:

where are system states, are unknown nonlinear functions, and and are system input and output, respectively, while denotes the external disturbance.

Using a states prediction technique, the pure-feedback discrete-time systems in (7) are shown to be transformable to an -step-ahead predictor as below:

And by future output predictions, the above system (8) can be further transformed into an input-output model for the output feedback control:

This shows that the system presentation (9) in NARMAX form is a transformation of the pure-feedback system (8). After transformation, both state feedback and output feedback controls only need to employ a single NN in the controller design, rather than a number of NNs in previous researches.

Based on the SISO pure-feedback system (8) and its transformation NARMAX model in (9), the control design has been further investigated by using high-order neural network to approximate unknown functions [93]. In the above mentioned control design, availability is assumed for knowledge of control directions, which are defined as “the signs of control variable gains in affine systems, or the signs of partial derivatives over control variables in nonaffine systems” [93].

The paper overcame the problems of output-feedback control when the prior knowledge of the control directions is unknown. The main idea is to introduce a discrete Nussbaum gain to counter the lack of knowledge on control gain in adaptive NN control.

A rigorous definition of discrete Nussbaum gain in [93] is given as follows.

Remark 1. Consider a discrete nonlinear function defined on a sequence with . is a discrete Nussbaum gain if and only if it satisfies the following two properties:
(i) if increases without bound, then

(ii) if , then with some positive constants and , where is defined as

with . In addition, neither the upper bounds nor the lower bounds of the control gains are required to be known.

Unknown control directions problem for MIMO discrete-time nonlinear systems was solved by using adaptive output feedback NN control in [94]. The studies in [92, 93] all studied the discrete NN control for SISO systems, and [94] extended the study to a class of MIMO discrete-time systems with each subsystem in the nonaffine pure-feedback as follows:

where are vectors of states variable of subsystem; is vector of all state variables; are unknown nonlinear functions; and are system inputs and outputs, respectively; while denote the external disturbance.

For nonlinear MIMO discrete-time higher order systems, a major drawback is that there are too many online-tuned adaptive parameters and large online computation burden. In [95], an adaptive neural output feedback adaptive controller was designed for MIMO nonlinear discrete-time systems with fewer adaptive parameters. The output feedback adaptive control for a class of nonlinear discrete-time systems with unknown control directions was investigated in [96].

3.2. NN-Based Dynamic Programming Algorithm for Discrete-Time Systems

In the past several decades, optimal control for nonlinear systems has been a key focus in the control fields. While adaptive dynamics programming (ADP) is a very useful approach to solve the optimal control problems. ADP is a reinforcement learning method to give solution to the dynamic programming utilizing function approximating the value function based on adaptive approach [97]. P. Werbos [98] classified the discrete-time ADP into a number of schemes: action-dependent HDP, dual HDP (DHP), heuristic dynamic programming (HDP), and action-dependent dual HDP, while NN has been widely used to solve the adaptive dynamic programming problems of discrete-time systems [97, 99–104].

He and Jagannathan [99] showed the reinforcement learning with NN-based controller for nonlinear discrete-time systems. This work developed an adaptive-critic-base discrete-time NN controller to deliver a desired tracking performance for nonlinear systems in the presence of actuator constraints. The critic NN and action NN were designed to approximate the strategic utility function while uniformly ultimate boundedness of the closed-loop tracking error was proved by Lyapunov approach. The reinforcement learning NN control for SISO discrete-time pure-feedback systems was studied in [100]. The controller design is based on the transformed predictor and two NNs of control architecture. One critic NN to approximate the strategic utility function is shown as follows.

By introducing a utility function based on the tracking error :

where , is the positive design parameter. And the strategic utility function is defined as

where is the horizon. Then, the critic NN is used to approximate the strategic utility function :

where is the estimation of optimal NN weights. The uniformly ultimate boundedness of closed-loop signals is established via Lyapunov stability analysis.

Al-Tamimi et al. [97] proved the convergence of a value function solution for nonlinear dynamical systems using HDP algorithm, and NNs were employed to approximate the value and the control action at each iteration. Liu et al. [101] used the HDP algorithm to develop a NN-based optimal controller for unknown discrete-time nonlinear systems. The main idea of this work is to introduce an iterative ADP algorithm and a globalized HDP technique into the controller design. And by using the cost function and control law, the convergence of the optimal control is guaranteed. In [102], an iterative ADP algorithm for discrete-time systems was developed as well. An optimal controller was designed to address the infinite-horizon discrete-time nonlinear systems with finite approximation errors. It was shown that when convergence conditions are satisfied, iterative performance index function can converge to the performance index functions’ lower bound, while the performance index functions were approximated by NNs to compute optimal control policy.

The ADP control in the unknown discrete-time Markov jump systems was investigated by Zhong et al. [103]. And Wang et al. [105] also studied the ADP for discrete-time systems to obtain the -optimal control by using neural networks. In [106], iterative ADP algorithm was used to study the near-optimal control with control constraints in discrete-time systems. An iterative ADP for discrete-time systems was studied in [104]. In this work, the optimal controller was designed with control constraints while NN was used to identify the unknown dynamical systems with stability proof.

4. Conclusion

In this short survey, despite the impossibility in identifying or listing all the related contributions, best efforts have been made to summarize the major achievements in the area of discrete-time adaptive control with modern techniques based on FL and NNs, which are also conventionally termed as important part of “soft-computing” approaches or “intelligent control” in the control communities. In particular, the marriage of “adaptive control” and “intelligent control” has been reviewed for the purpose of clarifying main contributions and outlining some possible trends for the development of this promising area.

Generally speaking, adaptive control for discrete-time linear systems was extensively explored in the past decades, despite the fact that the complete closed-loop stability analysis for the so-called self-tuning regulator was ever regarded as a long-term challenging problem. Later it became clear that discrete-time adaptive control can be extended to nonlinear systems with linear growth rate. However, general nonlinear systems with parametric and/or nonparametric uncertainties are usually very difficult to cope with, which motivated various solutions for different certain classes of nonlinear systems.

Among the various solutions, intelligent control emerged as one important way to resolve the challenges caused by the nonlinearity. The key for usability of intelligent control based on FL or NNs lies in their universal approximator property, at the cost of tuning rules of FL or updating parameters of NNs. FL and NNs are motivated by different background; hence their design philosophies look different; however, essentially speaking, both of them can be expressed as weighted sum of some kernel functions, where the weights can be tuned to approximate arbitrary smooth or continuous nonlinear function. The ideas borrowed from adaptive estimation and adaptive control provide such a way to adaptively tune the weight parameters in FL or NNs; thus the new area of adaptive FLC or NN control emerged with extensive studies.

In a summary, a brief review on FLC for discrete-time systems is provided by highlighting the adaptive FLC, robustness issue, and stability issue, while NN control for discrete-time systems is also reviewed with focuses on adaptive NN control and NN-based dynamic programming. We believe this topic would promote increasing investigations in both theories and applications. And some emerging techniques, such as deep learning and big data, could also be merged with the intelligent control for discrete-time systems and give birth to brand new design approaches of intelligent control in the future.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgment

This work is partially supported by the National Natural Science Foundation of China (NSFC) under Grants 61473120 and 61473038, Guangdong Provincial Natural Science Foundation 2014A030313266 and International Science and Technology Collaboration Grant 2015A050502017, and Fundamental Research Funds for the Central Universities under Grant 2015ZM065. The authors would like to thank Dr. Bin Xu of Northwestern Polytechnical University for his constructive comments during the preparation of this paper.

References

G. Goodwin and K. Sin, Adaptive Filtering, Prediction and Control, Prentice-Hall, Englewood Cliffs, NJ, USA, 1984.

K. Aström and B. Wittenmark, Adaptive Control, Addison-Wesley, 1989.

M. Krstić, I. Kanellakopoulos, and P. V. Kokotović, Nonlinear and Adaptive Control Design, John Wiley & Sons, New York, NY, USA, 1995.

G. Tao and P. Kokotovic, Aaptive Control of Systems with Actuator and Sensor Non-Linearities, John Wiley & Sons, Hoboken, NJ, USA, 1996.

S. S. Ge, C. C. Hang, T. H. Lee, and T. Zhang, Stable Adaptive Neural Network Control, Kluwer Academic Publishers, Norwell, Mass, USA, 2001.

C. Yang, Adaptive control and neural network control of nonlinear discrete-time systems [Ph.D. thesis], National University of Singapore, 2009.

C. Yang, H. Ma, and M. Fu, “Adaptive predictive control of periodic non-linear auto-regressive moving average systems using nearest-neighbour compensation,” IET Control Theory & Applications, vol. 7, no. 7, pp. 936–951, 2013.

C. Yang, L. Zhai, S. S. Ge, T. Chai, and T. H. Lee, “Adaptive model reference control of a class of MIMO discrete-time systems with compensation of nonparametric uncertainty,” in Proceedings of the American Control Conference, pp. 4111–4116, IEEE, Seattle, Wash, USA, June 2008.

S.-L. Dai, C. Yang, S. S. Ge, and T. H. Lee, “Robust adaptive output feedback control of a class of discrete-time nonlinear systems with nonlinear uncertainties and unknown control directions,” International Journal of Robust and Nonlinear Control, vol. 23, no. 13, pp. 1472–1495, 2013.

K. J. Åström and B. Wittenmark, “On self tuning regulators,” Automatica, vol. 9, no. 2, pp. 185–199, 1973.

L. Ljung, “Analysis of recursive stochastic algorithms,” IEEE Transactions on Automatic Control, vol. 22, no. 4, pp. 551–575, 1977.

G. C. Goodwin, P. J. Ramadge, and P. E. Caines, “Discrete time multivariable adaptive control,” IEEE Transactions on Automatic Control, vol. 25, no. 3, pp. 449–456, 1980.

L. Guo and H. F. Chen, “The Åström-Wittenmark self-tuning regulator revisited and ELS-based adaptive trackers,” IEEE Transactions on Automatic Control, vol. 36, no. 7, pp. 802–812, 1991.

L. Guo, Time-Varing Stochastic Systems, Jilin Science and Technology Press, Changchun, China, 1993 (Chinese).

H. F. Chen and L. Guo, Identification and Stochastic Adaptive Control, Birkhäuser, Boston, Mass, USA, 1991.

F. P. Skantze, A. Kojic, A.-P. Loh, and A. M. Annaswamy, “Adaptive estimation of discrete-time systems with nonlinear parameterization,” Automatica, vol. 36, no. 12, pp. 1879–1887, 2000.

L. Chen and K. S. Narendra, “Nonlinear adaptive control using neural networks and multiple models,” Automatica, vol. 37, no. 8, pp. 1245–1255, 2001.

L. Guo and C. Wei, “LS-based discrete-time adaptive nonlinear control: feasibility and limitations,” Science in China Series E: Technological Sciences, vol. 39, no. 3, pp. 255–269, 1996.

L. L. Xie and L. Guo, “Adaptive control of discrete-time nonlinear systems with structural uncertainties,” in Lectures on Systems, Control, and Information, vol. 17 of AMS/IP Studies in Advanced Mathematics, American Mathematical Society, International Press, Providence, RI, USA, 2000.

J. D. Boskovic, “Stable adaptive control of a class of first-order nonlinearly parameterized plants,” IEEE Transactions on Automatic Control, vol. 40, no. 2, pp. 347–350, 1995.

A. L. Fradkov, I. V. Miroshnik, and V. O. Nikiforov, Nonlinear and Adaptive Control of Complex Systems: Mathematics and Its Applications, Kluwer Academic Publishers, Dordrecht, The Netherlands, 2004.

D. Angeli and E. Mosca, “Adaptive switching supervisory control of nonlinear systems with no prior knowledge of noise bounds,” Automatica, vol. 40, no. 3, pp. 449–457, 2004.

H. B. Ma, “Finite-model adaptive control using an LS-like algorithm,” International Journal of Adaptive Control and Signal Processing, vol. 21, no. 5, pp. 391–414, 2007.

H. B. Ma, “Finite-model adaptive control using WLS-like algorithm,” Automatica, vol. 43, no. 4, pp. 677–684, 2007.

H. B. Ma, “Several algorithms for finite-model adaptive control: partial answers to finite-model adaptive control problem,” Mathematics of Control, Signals, and Systems, vol. 20, no. 3, pp. 271–303, 2008.

S. S. Ge, C. C. Hang, and T. Zhang, “A direct adaptive controller for dynamic systems with a class of nonlinear parameterizations,” Automatica, vol. 35, no. 4, pp. 741–747, 1999.

C. Y. Li and L. Guo, “On feedback capability in a class of nonlinearly parameterized uncertain systems,” IEEE Transactions on Automatic Control, vol. 56, no. 12, pp. 2946–2951, 2011.

H. Ma, K.-Y. Lum, and S. S. Ge, “Adaptive control for a discrete-time first-order nonlinear system with both parametric and non-parametric uncertainties,” in Proceedings of the 46th IEEE Conference on Decision and Control (CDC '07), pp. 4839–4844, IEEE, New Orleans, La, USA, December 2007.

L. Guo, “Exploring the capability and limits of the feedback mechanism,” in Proceedings of the International Congress of Mathematicians (ICM '02), Beijing, China, August 2002.

H.-B. Ma, “An ‘impossibility’ theorem on a class of high-order discrete-time nonlinear control systems,” Systems and Control Letters, vol. 57, no. 6, pp. 497–504, 2008.

I. Kanellakopoulos, P. V. Kokotovic, and A. S. Morse, “Systematic design of adaptive controllers for feedback linearizable systems,” IEEE Transactions on Automatic Control, vol. 36, no. 11, pp. 1241–1253, 1991.

L. A. Zadeh, “Fuzzy sets,” Information and Control, vol. 8, no. 3, pp. 338–353, 1965.

L.-X. Wang and J. M. Mendel, “Fuzzy basis functions, universal approximation, and orthogonal least-squares learning,” IEEE Transactions on Neural Networks, vol. 3, no. 5, pp. 807–814, 1992.

L.-X. Wang, “Stable adaptive fuzzy control of nonlinear systems,” IEEE Transactions on Fuzzy Systems, vol. 1, no. 2, pp. 146–155, 1993.

W. S. McCulloch and W. Pitts, “A logical calculus of the ideas immanent in nervous activity,” The Bulletin of Mathematical Biophysics, vol. 5, pp. 115–133, 1943.

K. Hornik, M. Stinchcombe, and H. White, “Multilayer feedforward networks are universal approximators,” Neural Networks, vol. 2, no. 5, pp. 359–366, 1989.

T. Khanna, Foundations of Neural Networks, Addison-Wesley, Reading, Mass, USA, 1990.

R. M. Sanner and J.-J. E. Slotine, “Gaussian networks for direct adaptive control,” IEEE Transactions on Neural Networks, vol. 3, no. 6, pp. 837–863, 1992.

Y. J. Liu, Y. J. Fang, and M. A. Bao-Ping, “Sliding-data-window-driven Bayesian-Gaussian neural network and its application to modeling of nonlinear system,” Control Theory & Applications, vol. 26, no. 12, pp. 1435–1438, 2009.

D. Wang and J. Huang, “Adaptive neural network control for a class of uncertain nonlinear systems in pure-feedback form,” Automatica, vol. 38, no. 8, pp. 1365–1372, 2002.

Y. Song and J. W. Grizzle, “Adaptive output-feedback control of a class of discrete-time nonlinear systems,” in Proceedings of the American Control Conference, pp. 1359–1363, June 1993.

B.-S. Chen, C.-S. Tseng, and H.-J. Uang, “Robustness design of nonlinear dynamic systems via fuzzy linear control,” IEEE Transactions on Fuzzy Systems, vol. 7, no. 5, pp. 571–585, 1999.

T. Chai and S. Tong, “Fuzzy direct adaptive control for a class of nonlinear systems,” Fuzzy Sets and Systems, vol. 103, no. 3, pp. 379–387, 1999.

K. Tanaka, T. Ikeda, and H. O. Wang, “Robust stabilization of a class of uncertain nonlinear systems via fuzzy control: quadratic stabilizability, control theory, and linear matrix inequalities,” IEEE Transactions on Fuzzy Systems, vol. 4, no. 1, pp. 1–13, 1996.

S. Jagannathan, M. W. Vandegrift, and F. L. Lewis, “Adaptive fuzzy logic control of discrete-time dynamical systems,” Automatica, vol. 36, no. 2, pp. 229–241, 2000.

Y. Jiang, Z. Liu, C. Chen, and Y. Zhang, “Adaptive robust fuzzy control for dual arm robot with unknown input deadzone nonlinearity,” Nonlinear Dynamics, vol. 81, no. 3, pp. 1301–1314, 2015.

M. W. Vandegrift, F. L. Lewis, S. Jagannathan, and K. Liu, “Adaptive fuzzy logic control of discrete-time dynamical systems,” in Proceedings of the IEEE International Symposium on Intelligent Control, pp. 395–401, IEEE, Monterey, Calif, USA, August 1995.

S. Jagannathan, “Adaptive fuzzy logic control of feedback linearizable discrete-time dynamical systems under persistence of excitation,” Automatica, vol. 34, no. 11, pp. 1295–1310, 1998.

R. Qi and M. A. Brdys, “Stable indirect adaptive control based on discrete-time T-S fuzzy model,” Fuzzy Sets and Systems, vol. 159, no. 8, pp. 900–925, 2008.

T.-C. Lin, S.-W. Chang, and C.-H. Hsu, “Robust adaptive fuzzy sliding mode control for a class of uncertain discrete-time nonlinear systems,” International Journal of Innovative Computing, Information and Control, vol. 8, no. 1, pp. 347–359, 2012.

G. Feng and G. Chen, “Adaptive control of discrete-time chaotic systems: a fuzzy control approach,” Chaos, Solitons and Fractals, vol. 23, no. 2, pp. 459–467, 2005.

H. J. Lee, J. B. Park, and G. Chen, “Robust fuzzy control of nonlinear systems with parametric uncertainties,” IEEE Transactions on Fuzzy Systems, vol. 9, no. 2, pp. 369–379, 2001.

Y.-Y. Cao and P. M. Frank, “Robust disturbance attenuation for a class of uncertain discrete-time fuzzy systems,” IEEE Transactions on Fuzzy Systems, vol. 8, no. 4, pp. 406–415, 2000.

S. Zhou, G. Feng, J. Lam, and S. Xu, “Robust control for discrete-time fuzzy systems via basis-dependent Lyapunov functions,” Information Sciences, vol. 174, no. 3-4, pp. 197–217, 2005.

S. Xu and J. Lam, “Robust control for uncertain discrete-time-delay fuzzy systems via output feedback controllers,” IEEE Transactions on Fuzzy Systems, vol. 13, no. 1, pp. 82–93, 2005.

C.-S. Tseng and B.-S. Chen, “Robust fuzzy observer-based fuzzy control design for nonlinear discrete-time systems with persistent bounded disturbances,” IEEE Transactions on Fuzzy Systems, vol. 17, no. 3, pp. 711–723, 2009.

S. Xu, B. Song, J. Lu, and J. Lam, “Robust stability of uncertain discrete-time singular fuzzy systems,” Fuzzy Sets and Systems, vol. 158, no. 20, pp. 2306–2316, 2007.

Z.-G. Wu, P. Shi, H. Su, and J. Chu, “Reliable control for discrete-time fuzzy systems with infinite-distributed delay,” IEEE Transactions on Fuzzy Systems, vol. 20, no. 1, pp. 22–31, 2012.

G. Feng and J. Ma, “Quadratic stabilization of uncertain discrete-time fuzzy dynamic systems,” IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, vol. 48, no. 11, pp. 1337–1344, 2001.

A. Kruszewski, R. Wang, and T. M. Guerra, “Nonquadratic stabilization conditions for a class of uncertain nonlinear discrete time TS fuzzy models: a new approach,” IEEE Transactions on Automatic Control, vol. 53, no. 2, pp. 606–611, 2008.

W.-J. Wang, Y.-J. Chen, and C.-H. Sun, “Relaxed stabilization criteria for discrete-time T-S fuzzy control systems based on a switching fuzzy model and piecewise Lyapunov function,” IEEE Transactions on Systems, Man, & Cybernetics Part B: Cybernetics, vol. 37, no. 3, pp. 551–559, 2007.

G. Feng, “Stability analysis of discrete-time fuzzy dynamic systems based on piecewise Lyapunov functions,” IEEE Transactions on Fuzzy Systems, vol. 12, no. 1, pp. 22–28, 2004.

H. Gao, X. Liu, and J. Lam, “Stability analysis and stabilization for discrete-time fuzzy systems with time-varying delay,” IEEE Transactions on Systems, Man, and Cybernetics Part B: Cybernetics, vol. 39, no. 2, pp. 306–317, 2009.

L. Wu, X. Su, P. Shi, and J. Qiu, “A new approach to stability analysis and stabilization of discrete-time T-S fuzzy time-varying delay systems,” IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 41, no. 1, pp. 273–286, 2011.

X. Su, P. Shi, L. Wu, and Y.-D. Song, “A novel control design on discrete-time takagi-sugeno fuzzy systems with time-varying delays,” IEEE Transactions on Fuzzy Systems, vol. 21, no. 4, pp. 655–671, 2013.

X. Su, P. Shi, L. Wu, and Y.-D. Song, “A novel approach to filter design for T-S fuzzy discrete-time systems with time-varying delay,” IEEE Transactions on Fuzzy Systems, vol. 20, no. 6, pp. 1114–1129, 2012.

C.-S. Tseng, “Model reference output feedback fuzzy tracking control design for nonlinear discrete-time systems with time-delay,” IEEE Transactions on Fuzzy Systems, vol. 14, no. 1, pp. 58–70, 2006.

X. Su, P. Shi, L. Wu, and S. K. Nguang, “Induced filtering of fuzzy stochastic systems with time-varying delays,” IEEE Transactions on Cybernetics, vol. 43, no. 4, pp. 1257–1264, 2013.

L. Wu, X. Su, P. Shi, and J. Qiu, “Model approximation for discrete-time state-delay systems in the TS fuzzy framework,” IEEE Transactions on Fuzzy Systems, vol. 19, no. 2, pp. 366–378, 2011.

F. L. Lewis, S. Jagannathan, and A. Yesildirek, Neural Network Control of Robot Manipulators and Nonlinear Systems, Taylor & Francis, London, UK, 1999.

A. M. Shaw and F. J. Doyle III, “Multivariable nonlinear control applications for a high purity distillation column using a recurrent dynamic neuron model,” Journal of Process Control, vol. 7, no. 4, pp. 255–268, 1997.

K. Najim, Process Modeling and Control in Chemical Engineering, Marcel Dekker, New York, NY, USA, 1989.

B. Xu, D. Wang, F. Sun, and Z. Shi, “Direct neural discrete control of hypersonic flight vehicle,” Nonlinear Dynamics, vol. 70, no. 1, pp. 269–278, 2012.

B. Xu and Y. Zhang, “Neural discrete back-stepping control of hypersonic flight vehicle with equivalent prediction model,” Neurocomputing, vol. 154, pp. 337–346, 2015.

B. Xu, F. Sun, H. Liu, and J. Ren, “Adaptive Kriging controller design for hypersonic flight vehicle via back-stepping,” IET Control Theory & Applications, vol. 6, no. 4, pp. 487–497, 2012.

B. Xu, “Robust adaptive neural control of flexible hypersonic flight vehicle with dead-zone input nonlinearity,” Nonlinear Dynamics, vol. 80, no. 3, pp. 1509–1520, 2015.

B. Xu, X. Huang, D. Wang, and F. Sun, “Dynamic surface control of constrained hypersonic flight models with parameter estimation and actuator compensation,” Asian Journal of Control, vol. 16, no. 1, pp. 162–174, 2014.

B. Xu and Z. Shi, “An overview on flight dynamics and control approaches for hypersonic vehicles,” Science China Information Sciences, vol. 58, no. 7, pp. 1–19, 2015.

D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning internal representations by error propagation,” in Parallel Distributed Processing, vol. 1, pp. 318–362, MIT Press, 1986.

S. Jagannathan and F. L. Lewis, “Discrete-time neural net controller for a class of nonlinear dynamical systems,” IEEE Transactions on Automatic Control, vol. 41, no. 11, pp. 1693–1699, 1996.

S. Jagannathan and F. L. Lewis, “Multilayer discrete-time neural-net controller with guaranteed performance,” IEEE Transactions on Neural Network, vol. 7, no. 1, pp. 107–130, 1996.

P. He and S. Jagannathan, “Neuro-controller for reducing cyclic variation in lean combustion spark ignition engines,” Automatica, vol. 41, no. 7, pp. 1133–1142, 2005.

S. S. Ge, G. Y. Li, and T. H. Lee, “Adaptive NN control for a class of strict-feedback discrete-time nonlinear systems,” Automatica, vol. 39, no. 5, pp. 807–819, 2003.

S. S. Ge, T. H. Lee, G. Y. Li, and J. Zhang, “Adaptive NN control for a class of discrete-time non-linear systems,” International Journal of Control, vol. 76, no. 4, pp. 334–354, 2003.

C. J. Goh, “Model reference control of non-linear systems via implicit function emulation,” International Journal of Control, vol. 60, no. 1, pp. 91–115, 1994.

C. J. Goh and T. H. Lee, “Direct adaptive control of nonlinear systems via implicit function emulation,” Control Theory and Advanced Technology, vol. 10, no. 3, pp. 539–552, 1994.

A. U. Levin and K. S. Narendra, “Control of nonlinear dynamical systems using neural networks—part II: observability, identification, and control,” IEEE Transactions on Neural Networks, vol. 7, no. 1, pp. 30–42, 1996.

S. S. Ge, J. Zhang, and T. H. Lee, “Adaptive MNN control for a class of non-affine NARMAX systems with disturbances,” Systems & Control Letters, vol. 53, no. 1, pp. 1–12, 2004.

S. S. Ge, J. Zhang, and T. H. Lee, “Adaptive neural network control for a class of MIMO nonlinear systems with disturbances in discrete-time,” IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 34, no. 4, pp. 1630–1645, 2004.

J. Zhang, S. S. Ge, and T. H. Lee, “Output feedback control of a class of discrete MIMO nonlinear systems with triangular form inputs,” IEEE Transactions on Neural Networks, vol. 16, no. 6, pp. 1491–1503, 2005.

F. C. Sun, Z. Sun, and P.-Y. Woo, “Stable neural-network-based adaptive control for sampled-data nonlinear systems,” IEEE Transactions on Neural Networks, vol. 9, no. 5, pp. 956–968, 1998.

C. Yang, S. S. Ge, C. Xiang, T. Chai, and T. H. Lee, “Output feedback NN control for two classes of discrete-time systems with unknown control directions in a unified approach,” IEEE Transactions on Neural Networks, vol. 19, no. 11, pp. 1873–1886, 2008.

S. S. Ge, C. Yang, and T. H. Lee, “Adaptive predictive control using neural network for a class of pure-feedback systems in discrete time,” IEEE Transactions on Neural Networks, vol. 19, no. 9, pp. 1599–1614, 2008.

Y. Li, C. Yang, S. S. Ge, and T. H. Lee, “Adaptive output feedback NN control of a class of discrete-time MIMO nonlinear systems with unknown control directions,” IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 41, no. 2, pp. 507–517, 2011.

Y.-J. Liu, C. L. P. Chen, G.-X. Wen, and S. Tong, “Adaptive neural output feedback tracking control for a class of uncertain discrete-time nonlinear systems,” IEEE Transactions on Neural Networks, vol. 22, no. 7, pp. 1162–1167, 2011.

C. Yang, S. S. Ge, and T. H. Lee, “Output feedback adaptive control of a class of nonlinear discrete-time systems with unknown control directions,” Automatica, vol. 45, no. 1, pp. 270–276, 2009.

A. Al-Tamimi, F. L. Lewis, and M. Abu-Khalaf, “Discrete-time nonlinear HJB solution using approximate dynamic programming: convergence proof,” IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 38, no. 4, pp. 943–949, 2008.

P. Werbos, “Approximate dynamic programming for real-time control and neural modeling,” in Handbook of Intelligent Control Neural Fuzzy & Adaptive Approaches, Van Nostrand Reinhold, 1992.

P. He and S. Jagannathan, “Reinforcement learning neural-network-based controller for nonlinear discrete-time systems with input constraints,” IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 37, no. 2, pp. 425–436, 2007.

B. Xu, C. Yang, and Z. Shi, “Reinforcement learning output feedback NN control using deterministic learning technique,” IEEE Transactions on Neural Networks and Learning Systems, vol. 25, no. 3, pp. 635–641, 2014.

D. Liu, D. Wang, D. Zhao, Q. Wei, and N. Jin, “Neural-network-based optimal control for a class of unknown discrete-time nonlinear systems using globalized dual heuristic programming,” IEEE Transactions on Automation Science and Engineering, vol. 9, no. 3, pp. 628–634, 2012.

D. Liu and Q. Wei, “Finite-approximation-error-based optimal control approach for discrete-time nonlinear systems,” IEEE Transactions on Cybernetics, vol. 43, no. 2, pp. 779–789, 2013.

X. Zhong, H. He, H. Zhang, and Z. Wang, “Optimal control for unknown discrete-time nonlinear markov jump systems using adaptive dynamic programming,” IEEE Transactions on Neural Networks and Learning Systems, vol. 25, no. 12, pp. 2141–2155, 2014.

D. Liu, D. Wang, and X. Yang, “An iterative adaptive dynamic programming algorithm for optimal control of unknown discrete-time nonlinear systems with constrained inputs,” Information Sciences, vol. 220, pp. 331–342, 2013.

F.-Y. Wang, N. Jin, D. Liu, and Q. Wei, “Adaptive dynamic programming for finite-horizon optimal control of discrete-time nonlinear systems with ε-error bound,” IEEE Transactions on Neural Networks, vol. 22, no. 1, pp. 24–36, 2011.

H. Zhang, Y. Luo, and D. Liu, “Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints,” IEEE Transactions on Neural Networks, vol. 20, no. 9, pp. 1490–1503, 2009.