We investigate a coefficient-based least squares regression problem with indefinite kernels from non-identical unbounded sampling processes. Here non-identical unbounded sampling means the samples are drawn independently but not identically from unbounded sampling processes. The kernel is not necessarily symmetric or positive semi-definite. This leads to additional difficulty in the error analysis. By introducing a suitable reproducing kernel Hilbert space (RKHS) and a suitable intermediate integral operator, elaborate analysis is presented by means of a novel technique for the sample error. This leads to satisfactory results.

We study coefficient-based least squares regression with indefinite kernels from non-identical unbounded sampling processes. In our setting, functions are defined on a compact subset

Let

By the definition of the dual space

Classical learning algorithm is conducted by a scheme in a reproducing kernel Hilbert space (RKHS) [

And the regularized penalty term is imposed on the coefficients of function

We define the following coefficient-based regularizer:

Then we have

If

Let

We immediately have the following proposition [

Consider

We use the RKHS

In order to estimate

We will conduct the error analysis in several steps. The major contribution we make is on the sample error estimate; the main difficulty is the non-identical unbounded sampling of the samples; we overcome this difficulty by introducing a suitable intermediate operator.

In order to give the error analysis, we assume that the kernel

We say that the Mercer kernel

Since sample

There is a large literature on error analysis for learning algorithm (

Now we can state our general results on learning rates for algorithm (

Assume moment hypothesis condition (

If we take

When the samples are drawn i.i.d from measure

Assume moment hypothesis condition (

Here we get the same learning rate as the one in [

In this section, we will state the error analysis in several steps.

In this subsection, we address a bound for the regularization error

Assume

This subsection is devoted to the analysis of the term

Assume

For any

Assume

From (

This in connection with Proposition

In this subsection we will conduct the estimation of the term

It is easy to see that both

Now we state our estimation for the sample error. The estimates are more involved since the sample is drawn by non-identical unbounded sampling processes. We overcome the difficulty by introducing a stepping integral operator

Let

We will estimate

This together with (

The term

If we define

Applying the conclusion as shown in [

This yields

Now we are in a position to give the proofs of Theorems

Theorem

For

Combining all the bounds together and noting that

When the samples are drawn i.i.d. from measure

The conclusion follows by discussing the relationship between

The author would like to thank Professor Hongwei Sun for useful discussions which have helped to improve the presentation of the paper. The work described in this paper is supported partially by National Natural Science Foundation of China (Grant no. 11001247) and Doctor Grants of Guangdong University of Business Studies (Grant no. 11BS11001).