Curve fitting is a very challenging problem that arises in a wide variety of scientific and engineering applications. Given a set of data points, possibly noisy, the goal is to build a compact representation of the curve that corresponds to the best estimate of the unknown underlying relationship between two variables. Despite the large number of methods available to tackle this problem, it remains challenging and elusive. In this paper, a new method to tackle such problem using strictly a linear combination of radial basis functions (RBFs) is proposed. To be more specific, we divide the parameter search space into linear and nonlinear parameter subspaces. We use a hierarchical genetic algorithm (HGA) to minimize a model selection criterion, which allows us to automatically and simultaneously determine the nonlinear parameters and then, by the least-squares method through Singular Value Decomposition method, to compute the linear parameters. The method is fully automatic and does not require subjective parameters, for example, smooth factor or centre locations, to perform the solution. In order to validate the efficacy of our approach, we perform an experimental study with several tests on benchmarks smooth functions. A comparative analysis with two successful methods based on RBF networks has been included.
In the literature, there are many methods to tackle the curve fitting problem, which remains challenging and elusive. In this study, the goal is to build a compact representation of the curve that corresponds to the best estimate of the unknown relationship between two variables from a set of data points. Curve fitting is a fundamental tool in scientific and engineering applications such as system identification, data analysis and visualization, geometric modeling, CAD/CAM systems, medical imaging, and reverse engineering [
The curve fitting problem has been mainly addressed by using typical methods based on linear models [
In [
In [
It is found that RBFs have rarely been applied to curve fitting by using a linear combination to change the number of radial basis functions and simultaneously to optimize their parameters. This paper elucidates the feasibility of using RBFs in a linear combination to fit a set of data points disturbed by noise; that is, we propose a new method to tackle the curve fitting problem using a strictly linear combination of RBFs. The proposed approach divides the parameter search space into linear and nonlinear parameter subspaces. We use a hierarchical genetic algorithm to minimize a model selection criterion, which allows us to automatically and simultaneously determine the nonlinear parameters: spread, centre locations, and number of RBFs. The coefficients of the linear model (linear parameters) are computed solving a set of linear equations using the least-squares method through Singular Value Decomposition (SVD) method. In order to validate the efficacy of the proposed approach, we perform an experimental study with several tests on smooth functions benchmarks. The comparative analysis with two successful methods based on RBF networks has been included.
RBFs are a particular class of functions whose value depends only on the distance from the centre. Specifically, their response decreases (
Types of RBFs commonly used.
Name | Function |
---|---|
Linear |
|
Cubic spline |
|
Thin-plate spline |
|
Multiquadric |
|
Gaussian |
|
A linear model for a function
HGA is a revised version of a genetic algorithm, which is an optimization scheme based on the biological evolution. Unlike the traditional genetic algorithm where the genotype structure is fixed, the chromosome in the HGA does not have these restrictions. The HGA has a bicoded chromosome scheduled in a hierarchical mode, which consists of two kinds of genes: control and parametric genes [
Located at the first level, the control genes are encoded as binary digits, while parametric genes at the next levels are encoded as real numbers generally, and their activation depends upon the value of associated control genes. Specifically, the control genes activate or inactivate the parametric genes; this is particularly important to determine the phenotype with different lengths within the same chromosomal representation. Hence, HGA will search over a larger search space and converge to the right solution with an increasingly high degree of accuracy.
The goal of curve fitting is to find the best estimate of the unknown functional relationship
In this study, we assume that
If the type of basis function
In this study, we use a typical radial function with local response, the Gaussian function. The popular Gaussian RBF is radially symmetric with a maximum value at central point
When the type of basis functions has been specified, our method applies the HGA to determine the number
To simultaneously determine the number of the basis functions, the centre locations, and the spread values, we propose a two-level bicoded description, where each bit in the binary chromosome carries the implication of presence (“1”) or absence (“0”) of a specific basis function in the solution. The real chromosome represents the magnitude of the spread parameters of the basis functions. We represent the chromosome of an individual as
Hierarchical structure of a chromosome.
To determine the model structure, as well as the basis parameters, we propose a fitness function based on the Akaikes Information Criterion (AIC). Originally, for parametric problems, this criterion was developed as a measure of the fitness of the model and the complexity of the model. The AIC is given by
Reproduction is the process whereby the best fit individual in the population receives a corresponding large number of copies in the next generation. We use the roulette wheel method as a reproduction process. In this method, each individual in the population has a roulette wheel slot sized in proportion to its fitness value. This selection strategy favors the most fit individuals, but it gives a chance to the less fit individuals to reproduce. Otherwise, in order to prevent premature convergence, it tries to keep the selection pressure relatively constant over the entire evolution process by using the sigma scaling method [
The crossover process selects genes from the parent chromosomes and creates new offspring. In the uniform crossover scheme, two parents are chosen to recombine into a new individual. Each bit of the new individual is selected from one of the parents depending on a fixed probability. We use the uniform crossover operator for both the control and the parametric genes with the same probability.
In the mutation process, there are also two types of mutations. The bit mutation method is applied over the control genes. With this method, each bit is inverted or not depending on the mutation probability. For the parametric genes mutation each numeric value
In order to evaluate the effectiveness of the proposed method (HGA), we carried out numerical simulations on five test functions, which are mathematically defined by the equations in Table
Mathematical definition of test functions.
Test functions | |
---|---|
|
|
|
|
|
|
|
|
|
The problem addressed in this paper is the automatic curve fitting given a set of noisy data points. Thus, the experimental setup was designed to study the effects of varying the number of samples and noise levels on the curve fitting task. The idea is that each test function is evaluated at
For each function, a collection of 101 noisy data sets is generated from each of the different configurations. In this study we use three different SNR values (2, 3, and 4) and three different sized samples for
For the HGA, we use a maximal number of basis function (
Setup for the HGA.
Parameter | Value |
---|---|
Crossover probability | 0.90 |
Mutation probability | 0.15 |
Number of elite individuals | 11 |
Generations | 3000 |
Population size | 300 |
To compare and evaluate qualitatively the performance of the methods, the best estimates obtained using each method for the different configurations were obtained. These best estimates correspond to the minimal MSE value obtained using each method from the collection of 101 noisy data sets generated from each specific configuration. For practical reasons, we present only the results of 1 of the 9 configurations from the experimental study. These results are shown in Figure
Data and estimates correspond to the best function estimated from the experimental setup with 50 samples and SNR = 3. The columns correspond from left to right to the GRNN, HGA, and RBNN methods, respectively (
For each simulated data set, the numerical measure that was used to quantitatively evaluate the quality of the estimated function
In Figure
Additionally, we compute the mean and standard error of mean for MSE values from all methods. These results are summarized in Tables
Test | Method | SNR = 2 | SNR = 3 | SNR = 4 |
---|---|---|---|---|
1 | RBNN |
|
|
|
GRNN |
|
|
|
|
HGA |
|
|
|
|
|
||||
2 | RBNN |
|
|
|
GRNN |
|
|
|
|
HGA |
|
|
|
|
|
||||
3 | RBNN |
|
|
|
GRNN |
|
|
|
|
HGA |
|
|
|
|
|
||||
4 | RBNN |
|
|
|
GRNN |
|
|
|
|
HGA |
|
|
|
|
|
||||
5 | RBNN |
|
|
|
GRNN |
|
|
|
|
HGA |
|
|
|
Test | Method | SNR = 2 | SNR = 3 | SNR = 4 |
---|---|---|---|---|
1 | RBNN |
|
|
|
GRNN |
|
|
|
|
HGA |
|
|
|
|
|
||||
2 | RBNN |
|
|
|
GRNN |
|
|
|
|
HGA |
|
|
|
|
|
||||
3 | RBNN |
|
|
|
GRNN |
|
|
|
|
HGA |
|
|
|
|
|
||||
4 | RBNN |
|
|
|
GRNN |
|
|
|
|
HGA |
|
|
|
|
|
||||
5 | RBNN |
|
|
|
GRNN |
|
|
|
|
HGA |
|
|
|
Test | Method | SNR = 2 | SNR = 3 | SNR = 4 |
---|---|---|---|---|
1 | RBNN |
|
|
|
GRNN |
|
|
|
|
HGA |
|
|
|
|
|
||||
2 | RBNN |
|
|
|
GRNN |
|
|
|
|
HGA |
|
|
|
|
|
||||
3 | RBNN |
|
|
|
GRNN |
|
|
|
|
HGA |
|
|
|
|
|
||||
4 | RBNN |
|
|
|
GRNN |
|
|
|
|
HGA |
|
|
|
|
|
||||
5 | RBNN |
|
|
|
GRNN |
|
|
|
|
HGA |
|
|
|
Basis functions corresponding to the best function estimated from the experimental setup with 50 samples and SNR = 2.
Test | Basis |
|
|
|
---|---|---|---|---|
1 |
|
2.624918 | 0.689655 | 0.175274 |
|
1.306060 | 0.827586 | 0.057716 | |
|
−3.316849 | 0.896552 | 0.278344 | |
|
2.739275 | 1.000000 | 0.126029 | |
|
||||
2 |
|
−18.659943 | 0.103448 | 2.150076 |
|
1.013608 | 0.448276 | 0.057623 | |
|
2.113588 | 0.517241 | 0.043472 | |
|
18.093719 | 0.620690 | 2.118272 | |
|
||||
3 |
|
−1.048773 | 0.206897 | 0.533407 |
|
1.155269 | 0.482759 | 0.027828 | |
|
1.225578 | 0.517241 | 0.049997 | |
|
1.236929 | 0.827586 | 0.423159 | |
|
||||
4 |
|
−9.654642 | 0.344828 | 0.409970 |
|
−1.730152 | 0.448276 | 0.083107 | |
|
3.042676 | 0.482759 | 0.075049 | |
|
11.017281 | 0.551724 | 0.656271 | |
|
−7.058442 | 0.965517 | 0.455500 | |
|
||||
5 |
|
−0.079952 | 0.137931 | 0.358084 |
|
−0.374830 | 0.448276 | 0.108851 | |
|
−0.580771 | 0.482759 | 0.035342 | |
|
0.715895 | 0.517241 | 0.039981 | |
|
0.225844 | 0.586207 | 0.193723 |
Basis functions corresponding to the best function estimated from the experimental setup with 50 samples and SNR = 3.
Test | Basis |
|
|
|
---|---|---|---|---|
1 |
|
2.581205 | 0.724138 | 0.162365 |
|
1.355051 | 0.827586 | 0.064088 | |
|
−9.424124 | 0.862069 | 0.168524 | |
|
6.909147 | 0.896552 | 0.168821 | |
|
||||
2 |
|
−17.234314 | 0.137931 | 1.933229 |
|
−1.310536 | 0.448276 | 0.086132 | |
|
2.834948 | 0.482759 | 0.072595 | |
|
16.774952 | 0.586207 | 1.913092 | |
|
||||
3 |
|
−1.116542 | 0.206897 | 0.391998 |
|
1.133581 | 0.482759 | 0.037636 | |
|
1.355708 | 0.517241 | 0.032508 | |
|
1.067411 | 0.793103 | 0.463377 | |
|
||||
4 |
|
−3.792799 | 0.344828 | 0.292119 |
|
2.455837 | 0.517241 | 0.070303 | |
|
−1.031267 | 0.551724 | 0.044479 | |
|
5.217404 | 0.655172 | 0.611903 | |
|
−4.459106 | 1.000000 | 0.570569 | |
|
||||
5 |
|
−0.209363 | 0.448276 | 0.097197 |
|
−0.561304 | 0.482759 | 0.042001 | |
|
0.580368 | 0.517241 | 0.035747 | |
|
0.254328 | 0.551724 | 0.094056 | |
|
−0.512819 | 0.724138 | 0.508370 | |
|
0.607906 | 0.758621 | 0.402507 |
Basis functions corresponding to the best function estimated from the experimental setup with 50 samples and SNR = 4.
Test | Basis |
|
|
|
---|---|---|---|---|
1 |
|
2.019124 | 0.689655 | 0.158368 |
|
1.461036 | 0.827586 | 0.067736 | |
|
−2.826506 | 0.862069 | 0.201430 | |
|
1.857911 | 1.000000 | 0.126618 | |
|
||||
2 |
|
−8.652872 | 0.275862 | 1.484126 |
|
1.139507 | 0.482759 | 0.072935 | |
|
0.928305 | 0.517241 | 0.052756 | |
|
8.930352 | 0.896552 | 1.567421 | |
|
||||
3 |
|
−1.052161 | 0.137931 | 0.321876 |
|
1.214086 | 0.482759 | 0.041275 | |
|
1.065797 | 0.517241 | 0.033858 | |
|
0.987050 | 0.896552 | 0.404330 | |
|
||||
4 |
|
−4.415701 | 0.310345 | 0.292465 |
|
5.736092 | 0.448276 | 0.605733 | |
|
1.014713 | 0.482759 | 0.056770 | |
|
0.868994 | 0.517241 | 0.047533 | |
|
−3.297666 | 0.931035 | 0.889548 | |
|
||||
5 |
|
−0.160610 | 0.344828 | 0.281685 |
|
−0.272302 | 0.448276 | 0.081592 | |
|
−0.413769 | 0.482759 | 0.029738 | |
|
0.464189 | 0.517241 | 0.023852 | |
|
0.333063 | 0.551724 | 0.080594 | |
|
0.101939 | 0.620690 | 0.396612 |
To clarify the effects of varying the number of samples and noise levels on the curve fitting task by using the HGA method, we include Figures
Effect of varying the number of samples on the curve fitting task by using the HGA method. Data and estimates correspond to the best function estimated from the experimental setup with SNR = 3 (
Effect of varying the signal-to-noise ratio on the curve fitting task by using the HGA method. Data and estimates correspond to the best function estimated from the experimental setup for data sets with 50 samples (
Finally, it could be assumed that the success of our approach is due to the simultaneous global search performed over all the parameters of the linear model. This shows the potential of HGA to handle the loss of information and high noise level. The average execution time for the methods based on neural networks was 0.29 seconds for experiments with 200 samples, compared to 132 seconds when using HGA. However, this drawback can be solved if we consider a parallel implementation of the model HGA. Our algorithm was implemented in C++ language. All methods were executed on a PC using an Intel Core i7 processor running at 2.2 GHz with 8 GB of RAM.
In this paper, we have proposed an efficient hierarchical genetic algorithm to tackle the automatic curve fitting problem. The method introduces a novel hierarchical gene structure for the chromosomal representation that permits the optimization of all free degrees of a linear model based on RBFs. To be more specific, our algorithm finds the best linear model using the fewest basis functions, optimal radii, and centre locations, and coefficients of the model simultaneously. The method does not require subjective parameters such as smooth factors or centre locations to perform the solution. On the other hand, the main drawback is the computational time required; however, it can be solved using parallel computing in order to reduce costs. This study focused on smooth functions, but it can be extended to approximate unsmooth functions. The algorithm was tested on several benchmark functions and compared to two successful methods based on RBF neural networks. Simulation results show that the proposed method presents qualitatively and quantitatively better results than the methods based on RBF neural networks because the method can handle the loss of information and high noise level more effectively. One of the attractive features of our method is that it can be easily extended to surface fitting.
Given the performance characteristics of the proposed approach, our future work will be to apply this method to fit real experimental data. We are interested in extending our approach to surface fitting, to experiment with variable length chromosomes and different basis functions.
The authors declare that there is no conflict of interests regarding the publication of this paper.
The authors acknowledge the support of CONACYT Mexico, DAIP of the University of Guanajuato, the Information Technology Laboratory of CINVESTAV, Tamaulipas, and ITESI, Mexico. Also, they gratefully acknowledge the help of Dr. Carlos Montoro Rubin to improve the English language of their paper.