^{1}

^{1}

^{2}

^{3}

^{1}

^{2}

^{3}

This paper addresses the stability problem of uncalibrated image-based visual servoing robotic systems. Both the visual feedback delay and the uncalibrated visual parameters can be the sources of instability for visual servoing robotic systems. To eliminate the negative effects caused by kinematic uncertainties and delays, we propose an adaptive controller including the delay-affected Jacobian matrix and design an adaptive law accordingly. Besides, the delay-dependent stability conditions are provided to show the relationship between the system stability and the delayed time in order to obtain less conservative results. A Lyapunov-Krasovskii functional is constructed, and a rigorously mathematic proof is given. Finally, the simulation results are presented to show the effectiveness of the proposed control scheme.

For human beings, vision is an important sensory channel. Through visual sensors, robots also can monitor the circumstance and perform the tasks. Nowadays, the advanced visual processing techniques and high-speed image processors make vision-based robot systems capable of handling dynamical tasks, and the vision-based control has been applied to many industrial robot systems. It has become the mainstream of robot control.

Vision-based control can be traced back to 1980s [

Visual servoing control scheme structure.

Position-based visual servoing

Image-based visual servoing

In the existing literature, there are two challenges in the field of visual servoing control: (a) the difficulties of calibration and (b) the image feedback signals of inferior quality.

The calibration of visual servoing systems includes the camera calibration, kinematic calibration, and dynamic calibration. For the sake of identifying unknown or uncertain system parameters, periodical and high-accurate calibration work usually is required, which is tedious and demanding. Without such calibration work, the system models cannot be accurately characterized and the closed-loop visual servoing systems could be unstable. To avoid such calibration work, the uncalibrated control approaches are proposed [

As another cause of system instability, the visual signals of inferior quality are also nonnegligible. Generally speaking, noise and delays in the visual signals are the main inducements. In this paper, we consider delays as the main reason for inferior image signals. As we know, the visual signal flows are expected to be synchronized with other system signals. However, asynchronization could happen due to many reasons including the limitation of image processing [

The two challenges make a visual servoing robotic system become typical complex industrial systems. This is because the mainstream noncalibration techniques usually require accurate image signals to compensate for the parametric errors or to update the unknown parameters. Under the delayed image feedback loop, there is no accurate synchronized visual feedback available. In this context, the control of such systems is of high nonlinearity and complexity. Consequently, it is worthwhile and challenging to be investigated. This paper therefore will concentrate on the influence of visual transmission delays upon the uncalibrated visual servoing robotic systems.

In the literature of this area, [

The paper is organized as follows. Section

Let

Let

Consider functional differential equation

Let

In this section, we present the mathematical modeling of delayed visual servoing robotic systems with the eye-in-hand configuration. In the modeling process, both kinematics and dynamics are considered. To illustrate the kinematics of the system, Figure

Robotic system with the eye-in-hand configuration.

Let

The relationship between

Combining with (

The nonlinear mapping

The dynamics of robots can be given with Euler-Lagrange equation as follows [

On the left side of (

From Figure

To facilitate analysis, we present Figure

The structure of a delayed visual servoing robotic system.

In this section, we will investigate the uncalibrated dynamic-based visual servoing robotic system with visual feedback delays and kinematic uncertainties. In our study, the formulation of the uncalibrated VS robotic system is partly based upon the

From (

Additionally, from (

Therefore (

In the uncalibrated dynamic-based visual servoing system, the estimate of Jacobian matrix is usually used as the replacement of unknown exact Jacobian matrix. It can be easily seen from (

For a vector

Due to the limitation of pages, see proof in Appendix

By Property

Using the

Based on all above analyses, we now propose the controller for delay-affected uncalibrated VS robotic systems as follows:

Additionally, recalling (

Consequently,

Hence,

From (

Consider the uncalibrated delayed visual servoing system described by (

Combining (

Substituting controller (

As aforementioned, the fact that

Let us consider the following nonnegative Lyapunov-Krasovskii functional candidate,

The time derivative of

Multiplying

Rewriting the (

After taking differential of

Substituting (

Likewise, with Lemma

Besides, from (

Having obtained the results in (

We will analyze the term

With Lemma

Substituting (

Then, we consider the term

Combining (

It can be clearly seen that the delay-dependent stability condition is presented in Theorem

In order to fully control 6-DOFs or more degree robots, we need more noncollinear feature points. For instance, three noncollinear feature points should be considered for a 6-DOF manipulator. The scheme proposed in this paper can be effortlessly generalized to the case of multiple feature points by the similar method described in [

To show the effectiveness of the control scheme described in (

The actual visual parameters are set as follows:

For the setting of the camera’s position and pose, the

The gravitational acceleration is set as ^{2}.

Parameters of manipulator in the simulation.

Link | ||||||
---|---|---|---|---|---|---|

1 | 1 | 0 | 0 | 1 | 0.5 | |

2 | 1 | 180 | 0 | 1 | 0.5 | |

3 | 0 | 0 | 1 | 1 | 0.5 |

Notes:

From Property

The feature point’s coordinates w.r.t. the base frame are (150, 20)^{T}^{T}^{T}

Besides, we set

Based on all above settings, two simulations are conducted. In the first simulation, the proposed control scheme is used to track the desired position under two different constant delays:

Position tracking errors.

Scheme 1

Scheme 2

Position trajectory of the feature point on the image plane.

Scheme 1

Scheme 2

Velocity of the feature point on the image plane.

Scheme 1

Scheme 2

Position trajectory of the feature point on the image plane.

Scheme 1

Scheme 2

Estimated parameters from

To demonstrate the superiority of the proposed control scheme, we make a comparison between the two control schemes: the scheme 1 and the scheme 2. The scheme 1 is the method proposed in this paper, and the scheme 2 originating from [

It should be noted that the Jacobian matrix in the scheme 2 does not consider the delay effects. Then, we conduct the second simulation. In this simulation, we use

From Figures

In this paper, we have proposed a control method for uncalibrated dynamic-based visual servoing robotic systems to cope with the delay problem existing in the visual feedback loops. To handle the unknown camera intrinsic and extrinsic parameters, we introduced the

Let

When

Besides, because

When

List of notations and symbols.

Notations | |

Perspective projection matrix | |

Homogeneous transformation matrix from the end effector to the base | |

Rotation matrix included in | |

Translation vector included in | |

Homogeneous transformation matrix from the camera to the end effector | |

The Cartesian coordinates of the feature point w.r.t. the robot base frame | |

The matrix consists of the left | |

The 1st and 2nd rows of | |

The 3rd row of | |

Jacobian matrix | |

The inertia matrix of manipulator dynamics | |

The Coriolis and centrifugal forces | |

The gravitational force | |

The control input | |

The depth-independent Jacobian matrix | |

The vector derived from | |

The delay-affected depth-independent Jacobian matrix | |

A novel composite Jacobian matrix | |

The control gain matrices | |

The regressor matrices | |

The unknown parameter vector | |

Symbols | |

The standard Euclidean norm | |

The transposition of matrix | |

The matrix consists of the minimum elements of | |

The matrix consists of the maximum elements of | |

The element in the | |

The visual signal delayed unknown constant duration of |

The data used to support the findings of this study are available from the corresponding author upon request.

The authors declare that they have no conflicts of interest.

This work was financed by Science and Technology Program of Tianjin, China under Grant 15ZXZNGX00290.