A brief review of neural networks based learning and control and their applications for robots

As an imitation of the biological nervous systems, neural networks (NN), which are characterized with powerful learning ability, have been employed in a wide range of applications, such as control of complex nonlinear systems, optimization, system identification and patterns recognition etc. This article aims to bring a brief review of the state-of-art NN for the complex nonlinear systems. Recent progresses of NNs in both theoretical developments and practical applications are investigated and surveyed. Specifically, NN based robot learning and control applications were further reviewed, including NN based robot manipulator control, NN based human robot interaction and NN based behavior recognition and generation.


Introduction
In recent years, the research of neural network (NN) has attracted great attention.It is well known that, mammals' brain, which consists of billions of inter-connected neurons, has the ability to deal with complex and computationally demanding tasks, such as face recognition, body motion planning and muscles activities control.Fig. 1 shows the cellular structure of a mammalian neuron.Inspiring by the neuron structure, artificial NN (ANN) was developed to emulate the learning ability of the biological neurons system [1][2][3].The concept of artificial NNs was initially investigated by McCulloch and Pitts in the 1940s [3], where the network consisted of simple neurons models with highly parallel structure.This structure was of simple processing neurons interconnected to each other.The basically mathematical model of NN consists of three layers, i.e., input layer, hidden layer and output layer, which are of simple parallel computational structure but with appealing learning ability and computational power to predict nonlinear dynamic patterns.
In past decades, the NN technique has been studied by a wide range of applications to various fields, such as control engineering, aerospace, medicine, automotive, psychology, economics, energy science and many other fields [4][5][6][7].It has been reported that NN can approximate any unknown continues nonlinear function by overlapping the outputs of each neurons.The approximation errors could be made arbitrarily small by choosing sufficient neurons [8][9][10].Thanks to the parallel structure and good generalization performance, the NN has become an appealing tool to deal with tough problems in complex nonlinear system [11][12][13][14][15][16].In addition to system modeling and control, NN has also been successfully applied in many other fields such as learning [17][18][19][20], pattern recognition [21], and signal processing [22].In the control methodology, NN has been extensively used as a for functions approximation to compensate for the system uncertain nonlinearities in control design [23][24][25][26][27][28][29][30][31][32][33][34].In the last two decades, NN control has been proved to be very useful for controlling highly uncertain nonlinear systems and has demonstrated superiority over traditional control methods.
Recently, more and more studies have focused on the study of robotics as the increasing importance of the robots in both industrial applications and daily life [35][36][37][38][39][40].Many advance robots such as YuMi made by ABB, Baxter made by Rethink, Rolins' Justin developed by German Aerospace Agency (DLR) have also been widely allocated.The robots manipulator system is characterized with high-nonlinearity, strong coupling and time-varying dynamics, thus controlling a robot with not only positioning accuracy, but also enough flexibility to complete a complex task, became an interesting yet challenge work.To achieve a high performance robot control, dynamics of the robot should be known in advance.However, in practice, the robot dynamic model is often rarely known and also ardors to derive a closed-form robotic dynamic model due to complex modelling processes, let along various uncertainties such as parametric uncertainties or modeling errors existed in the robot dynamics.Therefore, the robot systems provide ideal platforms to test the advanced control techniques.Thanks to the universal approximation and learning ability, the NN has been widely applied in robot control with various applications, e.g., robot control with unknown dynamics, robot control in unstructured environment, robot impedance learning control, robot cognitive control.The combination of NN control and robotics has resulted in a promising 1 2 research field, i.e., NN based robot control, and provides practical solutions to the control of robots.In this paper, we make a relatively comprehensive review of research progress on control of the complex nonlinear systems and robots by means of neural network.The rest of the paper is organized as follows.
After the introduction, in Section 2, we present preliminaries of several popular neural network structures, such as RBF NN, CMAC NN, etc. Section 3 introduces a number of theoretical developments of NN in the fields of adaptive control, optimization and evolutionary computing.In addition, Section 4 revisits the robot applications of the neural network control with the applications that include but are not limited to robot manipulators control, human robot interaction and robot cognitive control.Section 5 gives a brief discussion about the neural network control and its future research.

Preliminaries of Neural Networks
In this section, we will introduce several types of NN structure, which have been popularly employed in the control engineering.[17,18] The Radial Basis Function Neural Network (RBFNN) is a neural network architecture that can solve the function approximation problem.As shown in Fig. 2, RBFNN consists of three layers namely input layer, hidden layer and output layer.In the input layer, the NN inputs are applied.In hidden layer, the data is transformed from input space to Fig. 2 Structure of the RBF NN hidden space, which is always with a higher dimension.The RBFNN can be used to approximate any continuous vector function

Radial Basis Function Neural Network (RBF NN)
where F ˆ (Z ) is the estimation of F (Z ) , Z is an NN inputs vector.
T is the regressor, and l denotes the number of NN nodes.Generally, the regressor could be chosen as Gaussian radical basis function as below.
where u i (i  1,, l) are distinct points in state space, and  i is the width of Gaussian membership function.It has been well recognized that, using the powerful approximate ability of the RBFNN, we can approximate any continues nonlinear function over a compact set as where W * is the optimal weight vector, and  is the approximate error. (3)

Cerebellar model articulation controller (CMAC) NN [42]
There has been a predominant tendency to study the learning and control techniques of robotic systems by exploring the principles of biological systems.This is because the biological creatures, mechanisms and underlying principles are likely to bring novel ideas to improve control performance of the robot in a complex environment.It is well recognized that, the cerebellum, which is a part of the animal's brain, enables animals to precisely coordinate the motion of body parts like legs, wings, arms and fingers and it exhibits excellent adaptation ability to the change of the environment.In 1972, Albus proposed a learning mechanism that imitates the structure and function of the cerebellum, called cerebellar model articulation controller (CMAC) based on the cerebellum neurophysiological model [44].It can be classified as a memory network with overlapping receptive-fields, revealing motor learning properties of biological creatures such as learning interference, forgetting and generalization [45].In comparison to the back-propagation neural network, the CMAC neural network (NN) was adopted widely in modeling and control of robots system for its rapid learning speed, simple structure, insensitivity of data sequence and easy implementation [42,43].3 shows the basic structure of the CMAC neural network.The CMAC could be used to approximate the unknown continuous function, where Z  R m denotes the m dimensional inputs space.As shown in Fig. 3, two components are involved in the CMAC neural network to determine the value of the approximated nonlinear function G(Z ) , where and R() denotes the mapping from the input vector to the association space, i.e., =R(Z).The outputs are computed through P(), by using a projection of the association vector  onto a weights vector, such that It should be noted that R(Z) can be represented by a multi-dimensional receptive filed function such that each point in input Z are assigned with an activation value.The receptive-field basis functions of the association vector is chosen to be Gaussian function as below where l is number of blocks of the associate space, h ik denotes the kth block associated with the input z i , u ik denotes the receptive field's center, and  ik is the variance of Gaussian function.Then, the multidimensional receptive-field function can be described where following property shows the approximation ability provided by the CMAC neural network.
Lemma 1: For a continuous nonlinear function F(z), there exists an ideal weight value W * , such that F(z) could be uniformly approximated by a CMAC with the multiplication of the optimal weights W * and the associate vector S(Z) as where  is the NN construction errors and satisfied||  ||  N , and  N positive value.

Adaptive neural control
is a small bounded During the past two decades, various neural networks have been incorporated into adaptive control for nonlinear systems with unknown dynamics, where the closed-loop stability can be rigorously proved.In [46], a multiplayer discrete-time neural network controller was constructed for a class of multi-input multioutput (MIMO) dynamical systems, where NN weights were trained using an improved online tuning algorithm.An adaptive NN output feedback control was proposed to control two classes of discrete-time systems in the presence of unknown control directions [4].A robust adaptive neural controller was developed for a class of strict feedback systems in [47], where a Nussbaum gain technique was employed to deal with unknown virtual control coefficients.A dynamic recurrent NN was employed for construction of an adaptive observer with online turned weights parameters in [48] and to deal with the time-delay of a class of nonlinear dynamical systems in [49].The time-delay of strict-feedback nonlinear systems was also addressed by using NN control with proper designed Lyapunov-Krasovskii functions in [50].For a class of unknown nonlinear affine timedelay systems, an adaptive control scheme was proposed by constructing two high-order NNs for identifying system uncertainties [51].This idea has been further extended to affine nonlinear systems with input time delay in [52].
It should be noticed that, the universal approximation ability of NNs are only proven for continues nonlinear function.However, piecewise continuous functions such as frictions, backlash and deadzone are widely existed in industrial plants.Such piecewise continuous functions could be approximated by using a novel NN structure, which consists of a standard activation function and a jump approximation basis function [53].In [51], a CMAC NN was employed for the close-loop control of nonlinear dynamical systems with rigorous stability analysis, and in [55] a robust adaptive neural network control scheme was developed for cooperative tracking control of higher-order nonlinear systems.
The adaptive NN control scheme was also proposed for pure-feedback systems.In [56], a high-order sliding mode observer was proposed to estimate the unknown system states while two NNs were constructed to deal with approximate errors and the unknown nonlinearities, respectively.In comparison to the conventional control design for pure-feedback systems, the state-feedback control was achieved without using the backstepping technique.In [57], a neural control framework was proposed for nonlinear servo mechanism to guarantee both the steady-state and transient tracking performance.In this work, a prescribed performance function was employed in an output error transformation, such that regulation of the transformed output can guarantee the tracking performance of the original control.In [58], an adaptive neural control was also designed for a class of nonlinear systems in the presence of time-delays and input deadzone, and high-order neural networks were employed to deal the unknown uncertainties.In this work, a salient feature lies in that only the norm of the NNs' weights (a scalar) needs to be online updated, such that the computational efficiency in the online implementation could be significantly improved.In [59], the authors developed a neural network based feedforward controller to compensate for the nonlinearities and uncertainties of a dynamically substructured system consisting of both numerical and physical substructures, where an adaptive law with a new leakage term of NN weights error information is developed to achieve improved convergence.An experiment on a quasi-motorcycle testing rig validated the efficacy of this control strategy.In [60], a neural dynamic control was incorporated into the strict-feedback control of a class of unknown nonlinear system by using the dynamic surface control technique.For a class of uncertain nonlinear systems with unknown hysteresis, NN was used for compensation of the nonlinearities [61].In [62], to deal with unknown nonsymmetrical input saturations of unknown nonaffine systems, NNs were used in the state/output feedback control based on the mean value theorem and the implicit function.To avoid using the backstepping synthesis, a dynamic surface control scheme was designed by combining the NN with a nonlinear disturbance observer [63].

NN-based adaptive dynamic programming
In addition to adaptive control, neural networks have also be adopted to solve the optimization problem for nonlinear systems.In convention optimal control, the dynamic programming method was widely used.It aims to minimize a predefined cost function, such that a sequence of optimal control inputs could be derived.However, the cost function is usually difficult to online calculate due to the computation complexity in obtaining the solution of the Hamilton-Jacobi-Bellman (HJB) equation.Therefore, an adaptive/approximate dynamic programming (ADP) technique was developed in [64], Fig. 4 An overview of the HDP structure where a NN was trained to estimate the cost function and then to derive solutions for the ADP.Generally, the ADP has several different synonyms, including approximate dynamic programming, heuristic dynamic programming (HDP), critic network and reinforcement learning (RL) [65][66][67].Fig. 4 shows the basic framework of the HDP with a critic-actor structure, which was widely used in the ADP.In [68], a discrete-time HJB equation was solved using an NN-based HDP algorithm to derive the optimal control of nonlinear discrete-time systems.In [69], three neural networks were constructed for an iterative ADP, such that optimal feedback control of a discrete-time affine nonlinear system could be realized.In [70], a globalized dual heuristic programming was presented to address the optimal control of discrete-time systems.In each iteration, three neural networks were used to learn the cost function and the unknown nonlinear systems.In [71], a reference network combining with an action network and a critic network was introduced in the ADP architecture to derive an internal goal representation, such that the learning and optimization process could be facilitated.The reference network has also been introduced in the online action-dependent heuristic dynamic programming by employing a dual critic network framework.A policy iteration algorithm was introduced for infinite horizon optimal control of nonlinear systems using ADP in [72].In [73], a reinforcement learning method was introduced for the stabilizing control of uncertain nonlinear systems in the presence of input constraints.By using this RL-based controller, a constrained optimal control problem could be solved with construction of only one critic neural network.In [74], an ADP technique for online control and learning of a generalized multiple-input-multipleoutput (MIMO) system was investigated.In [75], an adaptive NN based ADP control scheme was presented for a class of nonlinear systems with unknown dynamics.The optimal control law was calculated by using a dual neural network scheme with a critic NN and an identifier NN.Particularly, parameters estimation error was used to online identify the learning weights to achieve the finite-time convergence.Optimal tracking control for a class of nonlinear systems was investigated in [76], where a new 'identifier-critic' based ADP framework was proposed.

Evolutionary computing
Despite of the capacity of addressing the approximation and the optimization problem of the NN, there has been also a great interest in using the evolutionary approaches to train of the neural networks.With the evolvement of NN architectures, learning rules, connection weights and the input feature, evolutionary artificial neural networks (EANN) could provide superior performance in comparison to conventional training approaches.A literature review of the EANN was given in [77], where the evolution strategies such as feedforward artificial NN and genetic algorithms (GA) have been introduced for the EANNs.In [78], several EANN frameworks were introduced by embedding the evolution algorithms (EA) to evolve the NN structure.In [79], an EPNet evolution system was proposed for evolving the feedforward NN based on a Fogel's evolutionary programming (EP) method, which could improve the NN's connection weights and architectures at the same time as well as decrease noise in the fitness evaluation.Good generalization ability of the evolved NN has been constructed and verified in the experiments.In [80], a GA based technique has been employed to train the NNs in direct neural control systems such that the NN architectures could be optimised.A deficiency of the EANN is that the optimization process would often result in a low training speed.To overcome this problem and facilitate adaptation processes, a hybrid multi-objective evolutionary method was developed in [81], where the singular-value-decomposition (SVD) technique was employed to choose the necessary neurons number in the training of a feedforward NN.The evolutionary approach was applied to identify a grey-box model with a multi-objective optimization between the clearly known practical systems and approximated nonlinear systems [82].Applications of evolutionary algorithms for robotic navigation have been introduced and investigated in [83].A survey of using machine learning technique to improve the evolutionary computation was reported in [84].
The evolution algorithms have been employed in many aspects for evolvements of NNs.They can be used as global approaches to train the NN connection weights, or useful tools to obtain near-optimal NN architectures, as well as adapting learning rules of NNs to their environment.In a word, the evolution algorithms provide NNs with the ability of learning to learn and also to build the relationship between evolution and learning.The EANN could perform favorable ability to adapt to changes of the dynamic environment.

NN based robotic manipulator control
Generally speaking, the control methods for robot manipulators can be roughly divided into two groups, model free control and model based control.For the model free control approaches like proportional-integral-derivative (PID) control, satisfactory control performance may not be able to guarantee.In contrast, the model based control approaches exhibit better control behavior but heavily depend on the validity of the robot model.In practice, however, a perfect robotic dynamic model is always not available due to the complex mechanisms and uncertainties.Additionally, the payload may be varied according to different tasks, which makes the accurate dynamics model hardly to be obtained in advance.To solve such problems, the NN approximation-based control methods have been used extensively in applications of robot manipulator control.A basic structure of the adaptive neural network control for robot manipulator is shown in Fig. 5. Consider a dynamic model of a robot manipulator given as below [85], where M (q) , C(q , q) , G(q) are the inertial matrix, Coriolis matrix and gravity vector respectively.Then NN control design could be given as follows： where e 1 is the tracking error, e 2 is the velocity tracking error and W ˆ S (Z ) is the NN controller with W ˆ being the weights matrix, and S (Z ) being the NN regressor vector, K 1 and K 2 are control gains specified by the designer.
From the equation ( 8), we can see that the robot controller consists of a PD like controller and a NN controller.In traditional model based controllers, the dynamic model of the robot should be adopted as a feedforward to address the effect caused by the robot motion.In practice, however, M (q) , C(q, q) and G(q) may not be known.
Therefore, the NN are used to approximate the unknown dynamics f  M (q)q   C(q , q)q   G(q) and to improve the performance of the system via the online estimation.To adapt the NN weights, adaptive laws are designed as below.
where  is a specified positive parameter and  is a positive parameter.The last terms of right-hand side of equation ( 9) are sigma modifications, which are used to enhance the convergence and robustness of the parameters adaptation.
In [85], a NN based share control method was developed to control a teleoperated robot with environmental uncertainties.In this work, the RBFNN was constructed to compensate for the unknown dynamics of the teleoperated robot.Especially, a shared Fig. 5 A basic structure of the adaptive NN robot control control strategy was developed into the controller to achieve the automatic obstacle avoidance combining with the information of visual camera and the robot body, such that the obstacle could be successfully avoided and the operator could focus more on the operated task rather than the environment to guarantee the stability and manipulation.In addition, error transformations were integrated into the adaptive NN control to guarantee the transient control performance.It has shown that, by using the NN technique, the control performance in both kinematic level and dynamic level of the teleoperated robot was enhanced.In [86], an extreme learning machine (ELM) based control strategy was proposed for uncertain robot manipulators to identify both the elasticity and geometry of an object.This ELM was applied to deal with the unknown nonlinearity of the robot manipulator to enhance the control performance.Particularly, by utilizing ELM, the proposed controller could guarantee that the robot dynamics follow a reference model, such that the desired set point and the feedforward force could be updated to estimate the geometry and stiffness of the object.
As a result, the reference model could be exactly matched with a limited number of iterations.In [87], the NN controller was also employed to control a wheel inverted pendulum, which has been decomposed to two subsystems, a fully actuated second order planar moving subsystem and a passive first order pendulum subsystem.Then the RBFNN was employed to compensate for the uncertain dynamics of the two subsystems by using its powerful learning ability, such that the enhanced control performance could be realized by using the NN learning.In [88], a global adaptive neural control was proposed for a class of robot manipulators with finite-time convergence learning performance.This control scheme employed a smooth switching mechanism combining with a nominal neural network controller and a robust controller to ensure globally uniformly ultimately bounded stability.The optimal weights were obtained by the finite-time estimation algorithm such that after the learning process, the learning weights could be reused next time for repeated tasks.The global NN control mechanism has been further extended to the control of dual arm robot manipulator in [89], where knowledge of both robot manipulator and the grasping object is unavailable in advance.By integrating prescribed functions into the design of controller, the transient performance of the dual arm robot control was regularly guaranteed.The NN was also employed to deal with synchronization problem of multiple robot manipulators in [90], where the reference trajectories are only available for part of the team members.By using the NN approximation controller, the robot has shown better control performance with enhanced transient performance and enhanced robustness.A RBFNN was constructed to compensate for the nonlinear terms of a five-bar manipulator based on an error transformation function [91] and a NN approximation technique was employed in the manipulator tracking control to deal with the unknown dynamics, kinematics and actuator properties [92].

NN based robot control with input nonlinearities
Another challenge of the robot manipulator is that the input nonlinearities such as friction, dead-zone and actuator saturation may inevitably exist in the robot system.These input nonlinearities may lead to larger tracking errors and degeneration of the control performance.Therefore, a number of works have been proposed to handle the nonlinearities by utilizing the neural network design.A neural adaptive controller was designed to deal with the effect of input saturation of the robot manipulator in [93] as below.
where z 1 is the robot position tracking error, z 2 is the velocity tracking error and  1 is an auxiliary controller.W ˆD , W ˆC and W ˆG are the NN weights, S D (Z D ) , S C (Z C ) and S G (Z G ) are the NN regressor vectors, and K p , K r are control gains specified by the designer. is an auxiliary system designed to reduce the effect of the saturation.
where  is the torque error caused by saturation, and K  is small positive value.To adapt the NN weights, adaptive laws are designed as below.
are specified positive parameters,  Dk ,  Ck and  Gk are positive In [94], an adaptive neural network controller was constructed to approximate the input deadzone and the uncertain dynamics of the robotic manipulator, while the output constraint was also considered in the feedback control.In [95], the NN was applied for the estimation of the unknown model parameters of a marine surface vessel and in [96] the full-state constraint of an n-link robotic manipulator was achieved by using the NN control.The NN controller was also constructed for flexible robotic manipulators to deal with the vibration suppression based on lumped spring-mass model [97] while in [98], two RBFNNs were constructed for flexible robot manipulators to compensate for the unknown dynamics and the deadzone effect, respectively.
The NN has also been used in many important industrial fields, such as autonomous underwater vehicles (AUV), hypersonic flight vehicle (HFV), etc.In [99], the NN has been constructed to deal with the attitude of AUVs in the presence of input deadzone and uncertain model parameters.In [100], the adaptive neural control was employed to deal with underwater vehicle control in discrete-time domain encountered with the unknown input nonlinearities, external disturbance and model uncertainties.Then the reinforcement learning was applied to address these uncertainties by using a critic NN and an action NN.The hypersonic flight vehicle control was investigated in [101] where the aerodynamic uncertainties and unknown disturbances were addressed by a disturbance observer based NN.In [102], a neural learning control was embedded in the HFV controller to achieve the global stability via a switching mechanism and a robust controller.

NN based human robot interaction control
Recently, there is a predominant tendency to employ the robots in the humansurrounded environment, such as household services or industrial applications, where humans and robots may interact with each other directly.Therefore, interaction control has become a promising research field and has been widely studied.In [103], a learning method was developed such that the dynamics of a robot arm could follow a target impedance model with only knowledge of the robotic structure (See Fig. 6).The NN was further employed in robot control in interaction with an environment [104], where impedance control was achieved with the completely unknown robotic dynamics.In [105], a learning method was developed such that the robot is able to adjust its impedance parameters when it interacts with unknown environments.In order to learn optimal impedance parameters in the robot manipulator control, an adaptive dynamic programming (ADP) method was employed when the robot interacts with unknown time-varying environments, where NNs were used for both critic and actor networks [106].The ADP was also employed for coordination of multi-robots [107], in which Fig. 6 Block of the learning impedence control possible disagreement between different manipulators was handled and dynamics of both robots and the manipulated object were not required to be known.In this work, the controller consists of two parts, a critic network which is used to approximate the cost function, and an actual NN which is designed to control the robot.The critic NN is designed as follows [107], where Z = ,   [x  T , z T , x T ] T with x o being the position of the object, and z being tracking error, W ˆC is the NN weight and S C is the regressor vector.
The critic NN is used to approximate a cost function c(t)   T Q  u T Ru , where u denotes the control input, and Q and R are positive definite matrix.Since the control objective is to minimize the control effort, the adaptation law is designed as where  is the learning rate, E  On the other hand, the actual NN control is designed to control the robot as where W ˆ T S(Z )could learn the dynamics of the robot, with W ˆ being the NN weight and S a being the regressor vector.e and e v are the position and velocity tracking errors, respectively, and K 2 is the control gain.
Since the control objective is to guarantee the estimation of both robot dynamics and the cost function  ˆ (t) , the adaptive law is selected as below, where  a and k r W a,i   a (W a,i S a  k r )S a are positive constants. ( On the other hand, as a fundamental element of the next generation robots, the human robot collaboration (HRC) has been widely studied by roboticists and NN has also been employed in HRC with its powerful learning ability.In [108], the NNs were employed to estimate the human partner's motion intention in human-robot collaboration, such that the robot was able to actively follow its human partner.To adjust the robot's role to lead or to follow according to the human's intention, game theory was employed for fundamental analysis of human-robot interaction and an adaptation law was developed in [109].Policy iteration combining with NN was adopted to provide a rigorous solution to the problem of the system equilibrium in human-robot interaction [110].

NN based robot cognitive control
According to the predictive processing theory [111], the human brain is always actively anticipating the incoming sensorimotor information.This process exists because the living beings exhibit latencies due neural processing delays and a limited bandwidth in their sensorimotor processing.To compensate such a delay, in human brain, neural feedback signals (including lateral and top-down connections) modulate the neural activities via inhibitory or excitatory connections by influencing the neuronal c a population coding of the bottom-up sensory-driven signals in the perception-action system.Similarly, in robotic systems, it is claimed that such a delay and a limited bandwidth also can be compensated by the predictive functions learnt by recurrent neural models.Such a learning process can be done via only visual processing [112] or in the loop of perception and action [113].
Based on the hierarchical sensorimotor integration theory, which advocates that action and perception are intertwined by sharing the same representational basis [114], the representation on different levels of sensory perception does not explicitly represent actions; instead, there is an encoding of the possible future percept which is learnt from prior sensorimotor knowledge.In the Bayesian, once this perception and action links have been established after learning, these perception-action associations in this architecture allow the following operations: First, these associations allow to predict the perceptual outcome of given actions by means of the forward models (e.g.Bayesian Model).It can be written as where E estimates the upcoming perception evidence given an executed action A and other prior information you have already known in I .The term P(E | A, I ) suggests a pre-learnt model representing the possibility of a motor action A will be executed given a (possible) resulting sensory evidence E is perceived (backward computation).Second, these associations allow to select an appropriate movement given an intended perceptual representation.From the backward computations introduced in the following equation, a predictive sensorimotor integration occurs: where A indicates a particular action selected given the (intended) sensory information E and a goal G.Here we assume that one's action is only determined by the current sensory input and the goal.
In terms of its hierarchical organization, it also allows this operation: with bidirectional information pathways, a low level perception representation can be expressed on a higher level, with a more complex receptive field, and vice versa e low  e high .This can be realised by the bi-directional deep architectures such as [124].
Conceptually, these operations can be achieved by extracting statistical regularity shown in Fig. 7.

 
Fig. 7 The process of cognitive control Since both perception and action processes can be seen as temporal sequences, from the mathematical perspective, the recurrent networks are Turing-Complete and have a learning capacity to learn time sequences with arbitrary length [115], if properly trained.Furthermore, such recurrent connections can be placed in a hierarchical way in which the prediction functions on different layers attempt to predict the non-linear time-series in different time-scales [116].From this point, the recurrent neural network with parametric bias units (RNNPB) [117] and multiple time-scale recurrent neural networks (MTRNN) [118] were applied to predict sequences by understanding them in various temporal levels.The difference of the temporal levels controls the properties of the different levels of the presentation in the deep recurrent network.For instance, in the MTRNN network [124], the learning of each neuron follows the updating rule of classical firing rate models, in which the activity of a neuron is determined by the average firing rate of all the connected neurons.Additionally, the neuronal activity is also decaying over time following an updating rule of leaky integrator model.Assuming the i-th MTRNN neuron has the number of N connections, the current membrane potential status of a neuron can be defined as both by the previous activation as well as the current synaptic inputs: where w i, j represents the synaptic weight from the j-th neuron to the i-th neuron, x j ,t is the activity of j-th neuron at t-th time-step and τ is the time-scale parameter which determines the decay rate of this neuron: a larger τ means their activities change slowly over time compared with those with a smaller time scale parameter τ .
In [119], the concepts of predictive coding was discussed in detail, where the learning, generation and recognition of actions can be conducted by means of the principle of prediction error minimization.By using the predictive coding, the RNNPB and MTRNN are capable for both generating own actions and recognizing the same actions performed by others.Recently, the study on neurorobotics experiments has shown that the dynamic predictive coding scheme can be used to address fluctuations in temporal patterns when training a recurrent neural network (RNN) model [120].This predictive coding scheme enables organisms to predict perceptual outcomes based on current intentions of actions to the external environment, and to forecast perceptual sequences corresponding to given intention states [120].
Based on this architecture, two-layer RNN models were utilized to extract visual information [121], understand one's intention [122] or emotion status [123] in social robotics, three-layer RNN models were used to integrate and understand multi-modal information for a humanoid iCub robot [124,125].Moreover, the predictive coding framework has been extended to variational Bayes predictive coding MTRNN, which can arbitrate between deterministic model and probabilistic model by setting a meta parameter [126].Such extension could provide significant improvement in dealing with noisy fluctuated sensory inputs which robots are expected to experience in more real world setting.In [127], a MTRNN was employed to control a humanoid robot and experimental results have shown that by using only partial training data, the control model can achieve generalization by learning in a lower feature perception level.
The hierarchical structure of RNN exhibits a great learning capacity to store multimodal information which is beneficial for the robotic systems to understand and predict in a complex environment.As the future models and applications, the state-of-the-art deep learning techniques or the motor actions of robotic systems can be further integrated into this predictive architecture.

Conclusion
In summary, great achievements for control design of nonlinear system by means of neural networks have been gained in the last two decades.Despite the impossibility in identifying or listing all the related contributions in this short review, effort have been made to summarize the resent progress in the area of NN control and its particular applications in the robot learning control, the robot interaction control and the robot recognition control.In this paper, we have shown that significant progresses of NN have been made in control of the nonlinear systems, in solving the optimization problem, in approximating the system dynamics, in dealing with the input nonlinearities, in human robot interaction and in the pattern recognition.All these development accompany not only with the development of techniques in control and advanced manufactures, but also theatrical progresses in constructing and developing the neural networks.Although huge efforts have been made to embed the NN in practical control systems, there are still a large gap between the theory and practice.To improve the feasibility and usability, evolutionary computing theory has been proposed to train the NNs.It can automatically find a near-optimal NN architecture and allow an NN to adapt its learning rule to its environment.However, the complex and long training process of the evolutionary algorithms deters their practical applications.More efforts need to be made to evolve the NN architecture and NN learning technique in the control design.On the other hand, in human brain, the neural activities are modulated via inhibitory or excitatory connections by influencing the neuronal population coding of the bottom-up sensorydriven signals in the perception-action system.In this sense, how to integrate the sensormotor information into the network to make NNs more feasible to adapt to the environment and resembling the capacity of the human brain, deserve further investigations.Moreover, the data-driven control technology, which utilizes the historical data to control the plants, is promising nowadays.The RNN could use their feedback connections to store recent inputs information in form of activations.For this reason, the RNN based data-driven control technology deserves deeply investigated.
In conclude, a brief review on neural networks for the complex nonlinear systems is provided with adaptive neural control, NN-based dynamic programming, evolution computing and their practical applications in the robotic fields.We believe this area may promote increasing investigations in both theories and applications.And emerging topics, such as deep learning [128][129][130][131], big data [132][133][134] and so on, may be incorporated into the neural network control for complex systems especially robot systems, for example, deep neural networks could be used to process massive amounts of unsupervised data in complex scenarios, neural networks can be helpful in reducing the data dimensionality and the optimization of NN training may be employed to enhance the learning and adaptation performance of robots.

Fig. 3
Fig. 3 Structure of a CMAC Neural Network